ANNOTATION CASE STUDY
Annotation case study
From class definitions to object boundaries, a road-image pilot shows how to align rules, record disagreements and retain the reasons behind each revision.
All case studies
01 / PILOT BRIEF
A vision team wants to clarify classes, occlusion and truncation before expanding production. Images provide the unit for linking rules, issues and recheck decisions.
02 / EDGE CASES
Challenge tags identify points to review in 60 sample images. Small objects, occlusion and truncation can overlap in the same image; their counts must not be added as a total image count.
SMALL OBJECTS
Distant signs and road users have limited visible detail. Define recognition criteria and exclusions before reviewing classes and box boundaries.
Example recordANN-008 · A traffic-sign boundary was disputed, revised under the candidate rules and passed on recheck.
OCCLUSION
An occluder can hide part of an object. Specify whether boxes follow visible or estimated full extent, supported by attributes and example notes.
Example recordANN-037 · The class of a partly occluded road user remains ambiguous and is kept unresolved.
TRUNCATION
For objects extending beyond the image, box clipping and truncation attributes need a consistent rule. Truncation and occlusion remain separate concepts.
Example recordANN-052 · Correct the truncation attribute of a vehicle at the image edge and retain the revision and recheck record.
24 / 18 / 12 are overlapping tag counts, not exclusive scene groups, bounding-box counts or issue counts. Some sample images may have none of these tags.
03 / WORKING METHOD
Start with a class dictionary and edge-case notes. Keep rule clarification, annotation changes and review decisions distinct. Open items remain in the candidate rules until agreement allows a version to be frozen.
01 / THE APPROACH
Define class boundaries and exclusions from visible evidence. List ambiguous examples separately instead of forcing a class.
01 / THE APPROACH
Agree on box extent and attributes separately. Calibrate with examples rather than combining different challenges into one vague rule.
01 / THE APPROACH
Track initial review, rework, recheck and current status by image. Multiple issues do not cause an image to be counted twice.
04 / PILOT REVIEW
Review 60 representative images. An image with any unresolved check is excluded from passed images. The remaining 300 images follow the next review schedule.
| Sample range | Reviewed | Initial pass | Recheck pass / rework | Current pass |
|---|---|---|---|---|
| ANN-001–020 | 20 | 18 | 2 / 2 | 20 |
| ANN-021–040 | 20 | 17 | 2 / 3 | 19 |
| ANN-041–060 | 20 | 19 | 1 / 1 | 20 |
| Total | 60 | 54 | 5 / 6 | 59 |
The three sample-ID ranges are disjoint and total 60 images. Rates describe this review only, not box accuracy, batch acceptance or model performance. They are not extrapolated to the other 300 images.
Explore delivery & quality05 / HANDOVER STRUCTURE
Connect image indexes with labels, class rules and review decisions so every revision retains its reasoning.
Image IDs, file references and batch membership, with image and bounding-box counts kept separate.
Agreed 2D boxes, classes and occlusion / truncation attributes. These labels are not included in this download.
Definitions for four classes, boundary examples, exclusions, candidate rules and changes.
Image-level sampling denominators, rework and recheck decisions, and unresolved items.
LET’S DEFINE YOUR ANNOTATION TASK
Describe your image type, draft classes and current challenges to frame a pilot and its review approach.