Computer Vision
Occlusion is where video datasets quietly break
Most annotator disagreement in video happens when an object is partly hidden. Deciding that once, in writing, is what keeps a track coherent.
3 min read
03 — Data type
Video Annotated Data
Structured visual datasets for computer vision and video intelligence.
What it is
The hard part is not the drawing.
Video annotation is where most datasets quietly fail. A box drawn correctly on one frame and inconsistently on the next teaches a tracker the wrong thing.
The work is less about drawing and more about holding a definition steady — what counts as the object, where its boundary sits when it is partly hidden, and when a track should end rather than drift.
Those decisions are written down before annotation begins and checked across sequences, not just within single frames.
Three things that decide whether it trains well
- 01
Consistency beats precision
A slightly loose box applied the same way every time is more useful than a tight box whose definition drifts. Models learn the rule you actually followed, not the one you intended.
- 02
Occlusion is the hard case
Most disagreement in video annotation happens when an object is partly hidden. Deciding that once, in writing, is what keeps a track coherent across a sequence.
- 03
Identity has to survive the gap
When an object leaves frame and returns, whether it keeps its identity is a modelling decision. Made implicitly, it produces tracks that fragment for no reason the model can learn.
Where the data comes from
Annotate what you hold. Collect where coverage is thin.
Annotating footage you hold
The common case: existing video, annotated to a specification written with your team, with edge cases escalated rather than resolved silently by whoever is on the task.
Collection where coverage is thin
Where a class or condition is under-represented, footage can be captured specifically to fill that gap rather than waiting for it to appear in production data.
Label schema definition
If the schema does not exist yet, defining it is part of the work — a label set that cannot be applied consistently by two people is not finished.
Annotation operations
Scoped to the project, not sold as a bundle.
- 01Bounding boxes
- 02Object tracking across frames
- 03Frame and clip classification
- 04Activity and event labelling
- 05Temporal segmentation
- 06Consistency review across sequences
What arrives at the end
The output is agreed before work starts.
- Annotations in the format your training pipeline reads, agreed before work starts
- Track identity maintained across frames and across occlusion, per the agreed rule
- Frame and sequence indices preserved so annotations align to the source video
- The annotation guideline delivered with the data, including the occlusion and identity rules
What this data trains
- Computer Vision
- Object Detection
- Object Tracking
- Activity Recognition
- Human-Object Interaction
- Video Intelligence
- Autonomous Systems
Quality workflow
Six gates between raw input and delivery.
The process does not change per data type, so a corpus assembled across categories holds together.
- 01
Collection
Capture or sourcing against a defined specification, so the input is already in scope.
- 02
Guidelines
A written definition of correct, with worked examples and explicit edge-case handling.
- 03
Annotation
Trained annotators working to that guideline, escalating ambiguity rather than guessing.
- 04
QA
Structural and consistency checks across the batch, not just within single items.
- 05
Human Validation
Review by a second person against the acceptance definition.
- 06
Final Dataset
Structured delivery with the guideline and review notes attached.
FAQ
Video Annotated Data questions.
Bounding boxes, object tracking across frames, classification, activity labelling and temporal segmentation, scoped to the project requirement.
Annotation follows a written guideline that defines object boundaries and occlusion handling, and review looks across sequences rather than at individual frames in isolation.
Yes. Annotation can be applied to footage you already hold, as well as to data collected for the project.
The output schema is agreed with your team at the start so the delivered data loads directly into your pipeline.
Yes. Where a schema does not exist, defining one that two annotators can apply the same way is part of the engagement rather than a prerequisite for it.
Related reading
All articlesData Annotation
Annotation guidelines are the real deliverable
Two annotators disagreeing about the same example is not a training problem. It is a definitions problem, and it is far cheaper to solve before collection than after.
3 min read
Dataset Quality
Inter-annotator agreement, and what it actually tells you
A single agreement score is a weak summary of a rich signal. What matters is where the disagreement sits and whether it is structured.
3 min read
Start a project
What Does Your AI Need to Learn?
Tell us about your data requirements, and let’s explore the right approach.