Data Annotation for
Computer Vision:
YOLO, COCO, IoU & Beyond

What Is Data Annotation for Computer Vision?

Computer vision models learn to see by studying examples — thousands or millions of images where a human has already identified what is in each one. Data annotation is the process of attaching those labels: drawing a box around every car, colouring every pixel that belongs to a road, or marking each joint of a human skeleton. Without annotation, a model has no ground truth to learn from.

The relationship between annotation quality and model performance is direct and unforgiving. A model cannot exceed the accuracy ceiling set by its training data. Loose bounding boxes (IoU 0.6 where 0.9 is achievable), mislabelled classes, or systematically missed instances will embed those errors permanently into model weights. Researchers at Google Brain found that a 1% improvement in annotation quality correlates with approximately 2–3% improvement in mAP on standard benchmarks.

This guide covers every layer of the annotation stack: what to annotate and how, the formats your team needs to know (YOLO and COCO), the metrics that define quality (IoU, mAP, IAA), strategies to build better datasets (augmentation, stratified sampling, edge case mining), and the economics of outsourcing annotation to India.

Where Annotation Sits in the ML Pipeline

Raw data collection → Annotation → Dataset splits → Model training → Evaluation → Deployment. Annotation sits at the foundation. Everything downstream — training time, model architecture choices, hyperparameter tuning — is constrained by the quality of annotated data fed in. The most common reason computer vision projects fail is not insufficient model capacity but insufficient annotation quality or quantity.

For teams outsourcing annotation: India-based providers like Data Terminal handle all major annotation types — bounding boxes, polygons, semantic segmentation, keypoints, LiDAR 3D — with four-layer QA and COCO/YOLO/Pascal VOC export. Contact: +91-9014387222 | contact@dataterminal.co.

Annotation Types: The Complete Taxonomy

Choosing the wrong annotation type is one of the most expensive mistakes a CV team can make — you cannot cheaply convert polygon annotations to segmentation masks, and re-annotating 50,000 images is weeks of lost time. Here is every annotation type, when to use it, and what it costs.

Annotation Type Decision Framework

YOLO Format: The Industry Standard

YOLO format is the most widely used annotation format for object detection. It is plain text: one .txt file per image, containing one line per object. Every coordinate is normalised to the range [0, 1] relative to the image dimensions.

<span class="comment"># Format: class_id x_center y_center width height</span>
<span class="comment"># All values normalised 0–1 relative to image dimensions</span>
<span class="comment"># File: image_001.txt (one per image)</span>

<span class="number">0</span> <span class="number">0.512</span> <span class="number">0.438</span> <span class="number">0.234</span> <span class="number">0.187</span>   <span class="comment"># class 0 (car), center at 51.2%, 43.8% of W/H, 23.4% wide, 18.7% tall</span>
<span class="number">1</span> <span class="number">0.721</span> <span class="number">0.612</span> <span class="number">0.098</span> <span class="number">0.124</span>   <span class="comment"># class 1 (pedestrian)</span>
<span class="number">2</span> <span class="number">0.143</span> <span class="number">0.289</span> <span class="number">0.412</span> <span class="number">0.356</span>   <span class="comment"># class 2 (truck)</span>

<span class="key">path</span>: <span class="string">/datasets/my_project</span>       <span class="comment"># root dataset directory</span>
<span class="key">train</span>: <span class="string">images/train</span>
<span class="key">val</span>: <span class="string">images/val</span>
<span class="key">test</span>: <span class="string">images/test</span>            <span class="comment"># optional</span>

<span class="key">nc</span>: <span class="number">3</span>                          <span class="comment"># number of classes</span>
<span class="key">names</span>: <span class="bracket">[</span><span class="string">'car'</span>, <span class="string">'pedestrian'</span>, <span class="string">'truck'</span><span class="bracket">]</span>

YOLO Coordinate System

A critical point: YOLO uses centre coordinates, not top-left corner. To convert from absolute pixel coordinates to YOLO format for an image of width W and height H:

<span class="comment"># Given: top-left corner (x1, y1), bottom-right corner (x2, y2) in pixels</span>
<span class="comment"># Image dimensions: W (width), H (height)</span>

x_center = (x1 + x2) / 2 / W
y_center = (y1 + y2) / 2 / H
width    = (x2 - x1) / W
height   = (y2 - y1) / H

<span class="comment"># Example: box from (245, 180) to (695, 500) on a 1920×1080 image</span>
x_center = (245 + 695) / 2 / 1920 = <span class="number">0.2448</span>
y_center = (180 + 500) / 2 / 1080 = <span class="number">0.3148</span>
width    = (695 - 245) / 1920      = <span class="number">0.2344</span>
height   = (500 - 180) / 1080      = <span class="number">0.2963</span>

YOLO Format Pros & Cons

COCO Format: Rich Metadata for Complex Tasks

COCO (Common Objects in Context) format is a JSON structure that encodes images, annotations, and category definitions in a single file. It is the standard for instance segmentation, keypoint detection, and panoptic segmentation tasks. Frameworks like Detectron2, MMDetection, and Hugging Face's transformers library all use COCO natively.

Unlike YOLO format, COCO bounding box coordinates are absolute pixel values in [x, y, width, height] format where (x, y) is the top-left corner — NOT normalised, NOT centre coordinates.

<span class="bracket">{</span>
  <span class="key">"info"</span>: <span class="bracket">{</span> <span class="key">"year"</span>: <span class="number">2026</span>, <span class="key">"description"</span>: <span class="string">"My CV Dataset"</span>, <span class="key">"contributor"</span>: <span class="string">"Data Terminal"</span> <span class="bracket">}</span>,

  <span class="key">"images"</span>: <span class="bracket">[</span>
    <span class="bracket">{</span> <span class="key">"id"</span>: <span class="number">1</span>, <span class="key">"file_name"</span>: <span class="string">"image_001.jpg"</span>, <span class="key">"width"</span>: <span class="number">1920</span>, <span class="key">"height"</span>: <span class="number">1080</span> <span class="bracket">}</span>,
    <span class="bracket">{</span> <span class="key">"id"</span>: <span class="number">2</span>, <span class="key">"file_name"</span>: <span class="string">"image_002.jpg"</span>, <span class="key">"width"</span>: <span class="number">1920</span>, <span class="key">"height"</span>: <span class="number">1080</span> <span class="bracket">}</span>
  <span class="bracket">]</span>,

  <span class="key">"categories"</span>: <span class="bracket">[</span>
    <span class="bracket">{</span> <span class="key">"id"</span>: <span class="number">1</span>, <span class="key">"name"</span>: <span class="string">"car"</span>,        <span class="key">"supercategory"</span>: <span class="string">"vehicle"</span> <span class="bracket">}</span>,
    <span class="bracket">{</span> <span class="key">"id"</span>: <span class="number">2</span>, <span class="key">"name"</span>: <span class="string">"pedestrian"</span>, <span class="key">"supercategory"</span>: <span class="string">"person"</span> <span class="bracket">}</span>
  <span class="bracket">]</span>,

  <span class="key">"annotations"</span>: <span class="bracket">[</span>
    <span class="bracket">{</span>
      <span class="key">"id"</span>: <span class="number">1</span>,
      <span class="key">"image_id"</span>: <span class="number">1</span>,
      <span class="key">"category_id"</span>: <span class="number">1</span>,
      <span class="key">"bbox"</span>: <span class="bracket">[</span><span class="number">245</span>, <span class="number">180</span>, <span class="number">450</span>, <span class="number">320</span><span class="bracket">]</span>,  <span class="comment">// [x, y, width, height] in pixels — NOT normalised</span>
      <span class="key">"area"</span>: <span class="number">144000</span>,
      <span class="key">"iscrowd"</span>: <span class="number">0</span>,
      <span class="key">"segmentation"</span>: <span class="bracket">[[</span><span class="number">245</span>,<span class="number">180</span>, <span class="number">695</span>,<span class="number">180</span>, <span class="number">695</span>,<span class="number">500</span>, <span class="number">245</span>,<span class="number">500</span><span class="bracket">]]</span>  <span class="comment">// polygon coords</span>
    <span class="bracket">}</span>
  <span class="bracket">]</span>
<span class="bracket">}</span>

COCO Evaluation Metrics

COCO benchmark uses a set of metrics far more demanding than simple AP@0.5. The primary metric is mAP@[0.5:0.95] — the mean of mAP computed at IoU thresholds from 0.5 to 0.95 in steps of 0.05. This penalises models with imprecise localisation heavily. A model that achieves AP50=85% but AP75=40% has great classification but poor box precision.

IoU: Measuring Annotation Accuracy

Intersection over Union (IoU) is the universal metric for measuring how well two bounding boxes overlap — whether comparing a prediction to a ground truth, or comparing two annotators' boxes to each other. It is simple, intuitive, and scale-invariant.

When two boxes perfectly overlap, the intersection equals the union and IoU = 1.0. When they do not overlap at all, the intersection is zero and IoU = 0. Every meaningful quality threshold in computer vision is expressed as an IoU threshold.

IoU for Annotation Quality Control (IAA)

Inter-Annotator Agreement (IAA) using IoU works as follows: take a calibration set of 50–100 images. Have two annotators independently label the same images. For each ground-truth object, match it to the closest box from each annotator and compute IoU between the two annotators' matched boxes. Average across all matched pairs.

<span class="comment"># Simple IoU computation between two boxes</span>
def iou(box_a, box_b):
    <span class="comment"># boxes: [x1, y1, x2, y2] in absolute pixels</span>
    xi1 = max(box_a[0], box_b[0])
    yi1 = max(box_a[1], box_b[1])
    xi2 = min(box_a[2], box_b[2])
    yi2 = min(box_a[3], box_b[3])

    intersection = max(0, xi2 - xi1) * max(0, yi2 - yi1)
    area_a = (box_a[2]-box_a[0]) * (box_a[3]-box_a[1])
    area_b = (box_b[2]-box_b[0]) * (box_b[3]-box_b[1])
    union = area_a + area_b - intersection

    return intersection / union if union > 0 else 0

<span class="comment"># IAA threshold check</span>
iaa_score = iou(annotator_a_box, annotator_b_box)
if iaa_score < <span class="number">0.85</span>:
    flag_for_guideline_review()

Common Annotation Errors by IoU Impact

Dataset Quality: Errors, Bias & Best Practices

A dataset is more than a collection of annotated images. Its composition — class distribution, scene diversity, split ratios, representation of edge cases — determines whether a model trained on it will generalise to production conditions. These structural issues are harder to fix than annotation errors.

Class Imbalance

When one class has 10× more examples than another, a model learns to be overconfident on the majority class and recall-deficient on the minority. The threshold for concern is roughly a 10:1 ratio for balanced detection tasks. Fixes: oversample rare classes, undersample majority classes, use focal loss during training, annotate more rare-class examples.

Train / Val / Test Splits

Scene Diversity Requirements

QA Layers for Production Annotation

Data Augmentation Strategies

Augmentation transforms existing annotated images to create additional training examples without additional annotation cost. A well-configured augmentation pipeline can multiply the effective dataset size 5–15x and dramatically improve model robustness to real-world variation.

The standard library is Albumentations (Python) — GPU-accelerated, bounding-box and keypoint aware, with 70+ transforms. Roboflow's web interface offers visual augmentation configuration without code.

Albumentations Quick-Start

import albumentations as A
from albumentations.pytorch import ToTensorV2

transform = A.Compose([
    A.HorizontalFlip(p=0.5),
    A.RandomBrightnessContrast(p=0.3),
    A.HueSaturationValue(hue_shift_limit=10, p=0.3),
    A.GaussNoise(var_limit=(10, 50), p=0.2),
    A.MotionBlur(blur_limit=7, p=0.2),
    A.RandomShadow(p=0.2),            # for outdoor/automotive datasets
    A.Rotate(limit=15, p=0.4),
    ToTensorV2()
], bbox_params=A.BboxParams(
    format='yolo',                    # or 'coco', 'pascal_voc', 'albumentations'
    label_fields=['class_labels']
))

Annotation Tools Compared

The annotation tool you choose affects annotator productivity, QA workflow, export format support, and team collaboration. There is no universal best tool — the right choice depends on team size, annotation type, budget, and hosting preference.

Tool Selection Matrix

Building Your Annotation Pipeline

An annotation pipeline is a repeatable system that takes raw images as input and produces export-ready annotated datasets as output — consistently, at scale, without quality degradation over time. Most annotation failures are pipeline failures, not annotator failures.

Outsourcing to India: Cost & Quality Guide

India has become the dominant global hub for computer vision data annotation. English literacy, a large technically educated workforce, competitive pricing, and flexible timezone coverage (IST = GMT+5:30, overlapping with both EU afternoon and US morning) make Indian annotation providers uniquely positioned for global AI teams.

2026 India Annotation Pricing

What to Ask an Indian Annotation Vendor

Before committing to any annotation vendor, ask these questions. A serious provider answers all of them without hesitation:

Getting Started: Your First 1,000 Images

The hardest part of starting a computer vision project is not the model — it is getting from zero annotated images to a working baseline quickly, without making decisions you will regret at 50,000 images. Here is the fastest path.