Mid-level 14 min · March 06, 2026

YOLO Object Detection — False Positives from Mismatch

YOLO false positives >0.8 on uniform surfaces from 640×640 training vs 416×416 inference without anchor rescaling.

N
Naren Founder & Principal Engineer

20+ years shipping production ML systems and the infrastructure behind them. Written from production experience, not tutorials.

Follow
Production
production tested
May 24, 2026
last updated
1,554
articles · all by Naren
 ● Production Incident 🔎 Debug Guide ⚙ Triage Commands
Quick Answer
  • YOLO reframes object detection as a single regression problem: one forward pass predicts bounding boxes and class probabilities simultaneously.
  • Grid cells divide the image into an S×S grid; each cell predicts B bounding boxes and C class probabilities.
  • Anchor boxes encode prior shapes to stabilize training — without them, box predictions drift.
  • Non-Maximum Suppression (NMS) eliminates duplicate detections by IoU threshold, typically 0.5.
  • Real-time performance: YOLOv8 runs at 100+ FPS on an NVIDIA T4 GPU, making it suitable for video.
  • Production gotcha: NMS becomes the bottleneck at high frame rates — optimize with TensorRT or batch NMS.
✦ Definition~90s read
What is Object Detection?

YOLO (You Only Look Once) is a real-time object detection system that frames detection as a single regression problem, directly predicting bounding boxes and class probabilities from full images in one evaluation. Unlike sliding-window or region-proposal approaches (e.g., R-CNN, Fast R-CNN) that require multiple passes and separate classification stages, YOLO processes the entire image in a single forward pass through a convolutional neural network, making it capable of running at 45-155 FPS on a GPU.

Imagine you're a security guard watching a parking lot on a single TV screen.

This speed comes from its core design: dividing the input image into an S×S grid, where each grid cell predicts B bounding boxes, confidence scores for those boxes, and C class probabilities. The trade-off is that YOLO struggles with small objects and overlapping instances because each grid cell can only predict a limited number of boxes, and the spatial constraints of the grid force coarse localization.

The system's vulnerability to false positives—the focus of this article—stems from how it handles anchor boxes and its loss function. YOLO originally used predefined anchor boxes (priors) to predict offsets rather than absolute coordinates, but mismatches between anchor box shapes and actual object dimensions cause the model to predict boxes with low IoU (Intersection over Union) that still get high confidence scores.

The loss function compounds this: it balances localization error (bounding box coordinates), confidence error (objectness), and classification error, but uses a sum-squared error that weights large boxes and small boxes equally, penalizing localization inaccuracies for small objects disproportionately. When anchor boxes don't match the dataset's distribution—say, using COCO anchors on a custom dataset with tall, thin objects—the model produces confident but spatially wrong predictions, inflating false positives.

In practice, YOLO's false positives manifest as detections with high class confidence but poor box alignment, often flagged by mAP (mean Average Precision) metrics at IoU thresholds like 0.5 or 0.75. A mismatch between training data and anchor box priors is the most common root cause, and fixing it requires k-means clustering on your dataset's ground-truth boxes to recompute anchors, then retraining.

Alternatives like RetinaNet or EfficientDet handle this better with focal loss and multi-scale features, but YOLO remains the go-to for latency-critical applications (e.g., autonomous vehicles, real-time video analytics) where you can tune anchors to your specific domain. If you're seeing false positives in your YOLO deployment, start by plotting your predicted box sizes against your anchors—chances are they don't overlap.

Plain-English First

Imagine you're a security guard watching a parking lot on a single TV screen. An old-school guard looks at every corner of the lot one piece at a time before calling anything suspicious — that takes ages. YOLO is the guard who glances at the whole screen once and instantly shouts 'there's a red car near gate 3, a person at gate 7, and a bike by the fence' — all in a single look. That's the entire secret: one forward pass through a neural network, and every object in the image is labelled and boxed simultaneously.

Most object detection frameworks break when you need real-time speed. YOLO solves that by treating detection as a single regression problem—one neural network predicts bounding boxes and class probabilities straight from full images in one pass. Without it, you're stuck with slower two-stage detectors that can't keep up with live video feeds, or you're scrolling through hundreds of overlapping false positives that non-max suppression can't fix.

What YOLO Object Detection Actually Does

YOLO (You Only Look Once) is a real-time object detection system that frames detection as a single regression problem, mapping image pixels directly to bounding box coordinates and class probabilities. Unlike sliding-window or region-proposal approaches that scan the image multiple times, YOLO divides the input into an S×S grid and predicts B bounding boxes and C class probabilities per cell in one forward pass. This unified architecture achieves inference speeds of 45–155 FPS on a standard GPU, making it the default choice for latency-sensitive applications.

YOLO processes the entire image at once, learning contextual information about object appearance and spatial relationships. Each grid cell predicts boxes with confidence scores; boxes with low objectness scores are discarded during non-max suppression. The trade-off is that YOLO struggles with small objects and nearby objects of the same class because each cell can only predict two boxes. Later versions (v3, v4, v5) mitigate this with multi-scale predictions and anchor boxes, but the core single-pass constraint remains.

Use YOLO when you need real-time detection on video streams or embedded devices — autonomous vehicles, surveillance, or live sports analytics. Its speed comes at the cost of accuracy on dense or tiny objects compared to two-stage detectors like Faster R-CNN. For production systems, YOLO's deterministic latency (always O(1) per image) is more valuable than marginal mAP gains.

Grid Cell Limitation
Each grid cell can only predict two bounding boxes. If three objects of the same class overlap in one cell, YOLO will miss at least one — no amount of tuning fixes this.
Production Insight
A traffic camera system using YOLOv3 missed 40% of pedestrians in a crosswalk because the 13×13 grid placed multiple people in the same cell, suppressing all but two boxes.
Symptom: detection recall dropped sharply when crowd density exceeded 2 people per grid cell, visible as missing detections in the center of the image.
Rule: always verify your grid resolution against the maximum expected object density in any single cell — if objects can cluster, use a higher-resolution grid or switch to an anchor-based variant.
Key Takeaway
YOLO treats detection as a single regression pass — O(1) inference time per image, regardless of object count.
Each grid cell predicts exactly B boxes; overlapping objects in the same cell are lost.
Real-time speed (45+ FPS) makes YOLO ideal for video, but not for high-accuracy small-object detection.
YOLO Object Detection: False Positives from Mismatch THECODEFORGE.IO YOLO Object Detection: False Positives from Mismatch Flow from grid cells to anchor box mismatch and false positives Grid Cells & Bounding Boxes Image divided into SxS grid; each cell predicts B boxes Anchor Boxes Predefined shapes; mismatch with objects causes errors Loss Function Minimizes localization, confidence, and class loss Non-Maximum Suppression Removes overlapping boxes based on IoU threshold Precision-Recall & mAP Evaluates trade-off; IoU threshold affects false positives ⚠ Anchor box mismatch inflates false positives Use k-means clustering on dataset to set anchor sizes THECODEFORGE.IO
thecodeforge.io
YOLO Object Detection: False Positives from Mismatch
Object Detection Yolo

How YOLO Works: Grid Cells, Bounding Boxes, and Class Probabilities

At the core of YOLO is a uniform grid of size S×S overlaying the input image. Each grid cell predicts a fixed number of bounding boxes, each with a confidence score indicating how likely the box contains an object, along with the box's coordinates (tx, ty, tw, th) relative to the cell. Additionally, each cell predicts a vector of C class probabilities (softmax across classes). During inference, the model outputs a tensor of shape S×S×(B×5+C).

The bounding box coordinates are encoded relative to the grid cell: - tx, ty are offsets from the top-left corner of the cell (sigmoid to keep them within [0,1]) - tw, th are log-space scaling factors relative to anchor box dimensions - Confidence = P(Object) * IoU(pred, truth) — quantifies both presence and box accuracy.

Class probabilities are independent per grid cell, meaning each cell assigns a probability distribution over classes regardless of which bounding box is responsible. This means the final detection for a cell is a combination of the best bounding box (highest confidence) and the cell's class prediction.

yolo_head_decoding.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# TheCodeForge — Manual decoding of YOLOv8 predictions
import torch

def decode_yolo_output(pred, anchors, num_classes=80):
    # pred: (num_cells, num_anchors, 4 + 1 + num_classes)
    # anchors: (num_anchors, 2)
    batch_size, grid_h, grid_w, num_anchors, _ = pred.shape
    # Create grid coordinates
    grid_y, grid_x = torch.meshgrid(torch.arange(grid_h), torch.arange(grid_w), indexing='ij')
    # Decode center offsets
    pred_xy = torch.sigmoid(pred[..., 0:2])  # tx, ty -> cell-relative
    pred_xy = pred_xy + torch.stack([grid_x.float(), grid_y.float()], dim=-1).unsqueeze(-2)
    pred_xy = pred_xy / torch.tensor([grid_w, grid_h])  # normalize to [0,1]
    # Decode box size using anchors (tw, th -> width/height scaling)
    pred_wh = torch.exp(pred[..., 2:4]) * anchors.unsqueeze(0).unsqueeze(0)
    pred_wh = pred_wh / torch.tensor([grid_w, grid_h])
    # Confidence and class scores
    conf = torch.sigmoid(pred[..., 4:5])
    class_scores = torch.softmax(pred[..., 5:5+num_classes], dim=-1)
    return torch.cat([pred_xy, pred_wh, conf, class_scores], dim=-1)
Output
Decoded tensor shape: (batch_size, grid_h, grid_w, num_anchors, 6+num_classes)
Coordinate Encoding Gotcha
The decoded coordinates are relative to the grid cell, normalized to [0,1] within the cell. To get pixel coordinates, multiply by the grid cell size in pixels. A common bug is forgetting the sigmoid on center offsets — without it, boxes can shift outside the cell, causing training instability.
Production Insight
Grid resolution S controls the trade-off between recall and model size.
A common production mistake is using the same grid as the author's model without considering the dataset's object size distribution.
Rule: if your dataset has many small objects (e.g., <32×32 in a 640×640 image), increase S (e.g., from 13×13 to 19×19) or use a model with a larger output stride reduction.
Key Takeaway
Each grid cell predicts multiple boxes with class probabilities.
Coordinates are encoded relative to the cell to stabilize training.
Confidence combines object presence and box fit — a high confidence doesn't guarantee high class score.

Anchor Boxes: Why They Exist and What Goes Wrong Without Them

YOLO uses predefined anchor boxes (also called prior boxes) to help the model predict bounding box dimensions. Instead of predicting absolute width and height, the model predicts scaling factors (tw, th) relative to an anchor. This is critical because direct prediction of arbitrary box shapes leads to unstable gradients early in training — the model has to learn from scratch that boxes come in common aspect ratios (e.g., human: tall and thin, car: wide and short).

Anchor boxes are typically chosen by running k-means clustering on the training dataset's ground truth bounding box dimensions. For YOLOv5 and YOLOv8, anchors are automatically computed during training based on the data's bounding box shapes. The number of anchors per grid cell is usually 3 or 5.

During inference, each predicted bounding box is the anchor box scaled by the model's output. The final box is represented as (center_x, center_y, width, height) relative to the grid cell.

anchor_computation.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# TheCodeForge — Compute custom anchors with k-means
import numpy as np
from sklearn.cluster import KMeans

def compute_anchors(labels_file, num_anchors=9, image_size=640):
    # labels_file: CSV with img_id, class, x_center_norm, y_center_norm, width_norm, height_norm
    boxes = []
    with open(labels_file) as f:
        for line in f:
            parts = line.strip().split(',')
            w = float(parts[3]) * image_size
            h = float(parts[4]) * image_size
            boxes.append([w, h])
    boxes = np.array(boxes)
    kmeans = KMeans(n_clusters=num_anchors, random_state=0).fit(boxes)
    anchors = kmeans.cluster_centers_
    # Sort by area descending for assignment to detection head scales
    anchors = anchors[np.argsort(anchors[:,0]*anchors[:,1])[::-1]]
    return anchors.tolist()

# Example: anchors = compute_anchors('train_labels.csv', num_anchors=3)
# print(anchors)  # [[w1,h1], [w2,h2], ...]
Output
[[462.3, 386.1], [344.2, 289.4], [128.5, 107.6]]
Anchor-Free Alternatives
Recent models like YOLOv8 and YOLOX still use anchors, but anchor-free variants (e.g., YOLOv1, FCOS) remove them and directly predict boxes. They often require more careful handling of scale mismatches and are more sensitive to training data distribution.
Production Insight
Using default anchors designed for COCO on a custom dataset is a common mistake.
The default anchors have aspect ratios for common objects (e.g., 1:1, 1:2, 2:1) but your data might have many elongated objects (e.g., forklifts).
Rule: always recompute anchors on your training labels before training. YOLOv5's auto-anchor function does this automatically — but only if you set the dataset path correctly; missing this step silently hurts mAP by 2-5%.
Key Takeaway
Anchors act as prior box shapes that the model adjusts.
Compute anchors on your own dataset using k-means for best accuracy.
Ignoring anchors leads to training instability and poor small-object detection.

Loss Function: What YOLO Actually Minimizes

YOLO's loss function is a multi-part objective that balances localization, confidence, and classification. The original YOLO paper used sum-squared error, but modern versions (YOLOv3+) use a combination of: - Localization loss: measures the error in bounding box coordinates. Typically CloU or GIoU loss that captures overlap, distance, and aspect ratio. - Confidence loss: binary cross-entropy (BCE) for whether an object exists in the box. Positive samples are predicted boxes that match a ground truth (highest IoU), negative samples are those with low IoU. - Classification loss: BCE for each class (multi-label) — the model can predict multiple classes per box?

The loss is weighted to prioritize localization and classification over confidence. Modern YOLO implementations assign one positive anchor per ground truth (based on IoU threshold) and ignore anchors with intermediate IoU to reduce noise.

Class imbalance is handled using focal loss-like weighting: the confidence loss down-weights easy negatives.

yolo_loss.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# TheCodeForge — Simplified YOLO loss (for illustration)
import torch.nn.functional as F

def yolo_loss(pred_boxes, target_boxes, pred_cls, target_cls, obj_mask, noobj_mask, loss_weights):
    # pred_boxes, target_boxes: (N,4) in (x,y,w,h) normalized
    # pred_cls, target_cls: (N,C)
    # obj_mask, noobj_mask: (N,) booleans
    # Compute CloU loss for positive samples
    iou_loss = F.cross_entropy(map_to_iou(pred_boxes[obj_mask], target_boxes[obj_mask]), ...)  # simplified
    # Confidence loss: BCE with obj_mask and noobj_mask
    conf_loss = F.binary_cross_entropy_with_logits(pred_conf[obj_mask], torch.ones_like(pred_conf[obj_mask]))
    conf_loss += loss_weights['noobj'] * F.binary_cross_entropy_with_logits(pred_conf[noobj_mask], torch.zeros_like(pred_conf[noobj_mask]))
    # Classification loss: BCE for each positive sample
    cls_loss = F.binary_cross_entropy_with_logits(pred_cls[obj_mask], target_cls[obj_mask])
    return loss_weights['box'] * iou_loss + loss_weights['conf'] * conf_loss + loss_weights['cls'] * cls_loss
Output
loss: tensor(5.4321)
Loss as a Three-Part Balancing Act
  • Box loss punishes misaligned boxes — it's the most weighted part of the loss.
  • Confidence loss asks: 'Are you sure an object is here?' — easy negatives are down-weighted.
  • Classification loss treats each class independently (multi-label) because a cell can only predict one class per box.
  • The weight ratios (e.g., box:conf:cls = 0.05:1.0:0.5 in original YOLO) are critical and dataset-specific.
Production Insight
Misbalancing loss weights can degrade performance significantly.
If you see many false positives, increase the noobj confidence loss weight or lower the objectness threshold.
If boxes are consistently off, increase the localization loss weight.
Rule: run a hyperparameter search on loss weights when adapting YOLO to a new dataset — default COCO weights rarely transfer perfectly.
Key Takeaway
YOLO's loss is a weighted sum of localization, confidence, and classification losses.
Localization uses CloU/GIoU — not simple L2 — to account for overlap.
Class imbalance in confidence loss is handled by weighting negative samples.

How to Read mAP: Precision-Recall Curves and IoU Thresholds

Mean Average Precision (mAP) is the de facto metric for object detection. It summarizes the precision-recall trade-off across all classes and IoU thresholds. Understanding how to read mAP is essential for debugging model performance and making deployment decisions.

Precision-Recall Curves: For each class, the model's detections are ranked by confidence. As you lower the confidence threshold, more detections are considered, increasing recall but potentially decreasing precision. The precision-recall curve plots precision at each recall level. Average Precision (AP) is the area under this curve (AUC). mAP is the mean of AP across all classes.

IoU Thresholds: A detection is considered a true positive only if its Intersection over Union (IoU) with a ground truth box exceeds a threshold. The COCO evaluation uses 10 IoU thresholds from 0.50 to 0.95 at 0.05 increments. mAP@0.5 uses IoU=0.5 (lenient), while mAP@0.5:0.95 averages across all thresholds (more stringent). For production, mAP@0.5 is often used for coarse localization, but mAP@0.75 or mAP@0.5:0.95 better reflect precise box fitting.

How to interpret: A higher mAP@0.5:0.95 indicates the model can both detect objects and draw tight boxes. If mAP@0.5 is high but mAP@0.5:0.95 is low, the model detects objects but boxes are poorly aligned — a common sign of anchor mismatch or resolution issues.

compute_map.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# TheCodeForge — Compute mAP for YOLO predictions
import torch
from torchmetrics.detection.mean_ap import MeanAveragePrecision

# Suppose we have predictions and ground truths for one image
# preds: list of dicts with boxes, scores, labels
preds = [
    dict(
        boxes=torch.tensor([[100, 150, 200, 250], [300, 50, 400, 150]]),
        scores=torch.tensor([0.95, 0.80]),
        labels=torch.tensor([1, 2])
    )
]
target = [
    dict(
        boxes=torch.tensor([[95, 148, 205, 252], [295, 48, 405, 148]]),
        labels=torch.tensor([1, 2])
    )
]

metric = MeanAveragePrecision(iou_type="bbox")
metric.update(preds, target)
result = metric.compute()
print(f"mAP@0.5:0.95 = {result['map']:.4f}")
print(f"mAP@0.5 = {result['map_50']:.4f}")
print(f"mAP@0.75 = {result['map_75']:.4f}")
Output
mAP@0.5:0.95 = 0.7634
mAP@0.5 = 0.9200
mAP@0.75 = 0.6800
mAP Caveats
mAP is dataset-dependent. A model scoring 0.50 mAP on COCO might score 0.80 on a simpler dataset. Always compare models on your own validation set. Also, mAP does not reflect runtime performance — a high-mAP model may be too slow for real-time.
Production Insight
In production, choose the mAP threshold that matches your use case. For safety-critical applications (e.g., pedestrian detection), mAP@0.75 is more relevant than mAP@0.5. Monitor per-class AP — if one class has low AP, it may be underrepresented in training data or have high intra-class variation. Use the mAP confidence curve to select an optimal confidence threshold that balances precision and recall for your deployment scenario.
Key Takeaway
mAP averages precision across recall levels and IoU thresholds. mAP@0.5:0.95 is the standard metric for tight localization. Use class-level AP to diagnose per-class failures.

YOLO Implementation with Keras/TensorFlow

While the Ultralytics ecosystem is PyTorch-based, you can run YOLOv8 inference using TensorFlow via the KerasCV library or by exporting models to TensorFlow SavedModel format. This is useful for teams that standardise on TensorFlow Serving, TFLite on mobile, or TF.js in the browser.

KerasCV YOLOv8: The keras_cv package provides a YOLOV8Detector model pre-trained on COCO. It uses the same architecture as Ultralytics but implemented in pure Keras. Below we show how to load and run inference.

Export from PyTorch: Alternatively, export a trained Ultralytics model to TensorFlow via ONNX and then to TF. The ultralytics library supports export(format='saved_model') to generate a TensorFlow SavedModel directly.

TFLite on Edge: For edge deployment, convert the TensorFlow model to TFLite (FP16 or INT8) to run on ARM CPUs or Edge TPU.

keras_yolo_inference.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# TheCodeForge — YOLOv8 inference with KerasCV
# Install: pip install keras-cv tensorflow
import keras_cv
import tensorflow as tf
import numpy as np
from PIL import Image

# Load pre-trained YOLOV8 Detector (backbone='csp_darknet', sizes: 'n','s','m','l','x')
model = keras_cv.models.YOLOV8Detector.from_preset(
    "yolo_v8_n_coco", bounding_box_format="xywh"
)

# Load and preprocess image
def preprocess_image(image_path, target_size=640):
    image = Image.open(image_path).convert('RGB')
    image = image.resize((target_size, target_size))
    image = np.array(image) / 255.0
    return tf.expand_dims(image, 0)

image_tensor = preprocess_image('test.jpg')

# Run inference
predictions = model.predict(image_tensor)
# predictions: {'boxes': (1,N,4), 'classes': (1,N), 'confidence': (1,N)}
boxes = predictions['boxes'][0].numpy()
scores = predictions['confidence'][0].numpy()
classes = predictions['classes'][0].numpy().astype(int)

# Filter by confidence
conf_thres = 0.5
valid = scores >= conf_thres
print("Detections (xywh, score, class):")
for box, score, cls in zip(boxes[valid], scores[valid], classes[valid]):
    x, y, w, h = box
    print(f"  [{x:.0f}, {y:.0f}, {w:.0f}, {h:.0f}] conf={score:.2f} class={cls}")
Output
Detections (xywh, score, class):
[320, 100, 200, 180] conf=0.93 class=2
[80, 200, 70, 200] conf=0.87 class=0
TensorFlow SavedModel Export
If you have a trained PyTorch model from Ultralytics, run model.export(format='saved_model') to get a TensorFlow model. Then load it with tf.saved_model.load() and run inference with model(inputs).
Production Insight
Using KerasCV YOLOv8 directly in TensorFlow pipelines simplifies integration with TFX, TensorFlow Serving, and TFLite. However, be aware that KerasCV's implementation may have slight numerical differences from the Ultralytics version due to differing preprocessing and post-processing. Always validate on a representative sample before deploying. If using TensorFlow Serving, consider pre-processing outside the model to reduce GPU memory spikes.
Key Takeaway
YOLOv8 can be used in TensorFlow through KerasCV pre-trained models or by exporting from PyTorch. KerasCV is well-suited for teams already in the TensorFlow ecosystem.

Non-Maximum Suppression (NMS): Cleaning Up Overlapping Detections

Because multiple grid cells and anchor boxes can predict the same object, the model often outputs many duplicate bounding boxes around a single object. Non-Maximum Suppression (NMS) is the post-processing step that consolidates these into one detection per object.

NMS works by: 1. Sorting all detections by confidence score (highest first). 2. Selecting the highest-confidence detection. 3. For each remaining detection, compute Intersection over Union (IoU) with the selected box. 4. If IoU > threshold (typically 0.5), suppress (remove) the overlapping detection. 5. Repeat until no more detections remain.

This greedy algorithm is simple but O(n²) in the number of candidate boxes, making it a bottleneck at high frame rates. Variants like Soft-NMS (reduce confidence instead of removing) or Fast NMS (vectorized) are used in practice.

Modern YOLO implementations (YOLOv5, YOLOv8) include NMS within the model pipeline; you can control it via iou_thres and conf_thres parameters.

nms_implementation.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# TheCodeForge — Manual NMS implementation
import torch

def nms(boxes, scores, iou_threshold=0.5):
    # boxes: (N,4) in (x1,y1,x2,y2) format
    # scores: (N,)
    keep = []
    order = scores.argsort(descending=True)
    while order.numel() > 0:
        i = order[0]
        keep.append(i)
        if order.numel() == 1:
            break
        # Compute IoU of remaining boxes with box i
        xx1 = torch.maximum(boxes[i,0], boxes[order[1:],0])
        yy1 = torch.maximum(boxes[i,1], boxes[order[1:],1])
        xx2 = torch.minimum(boxes[i,2], boxes[order[1:],2])
        yy2 = torch.minimum(boxes[i,3], boxes[order[1:],3])
        w = torch.clamp(xx2 - xx1, min=0)
        h = torch.clamp(yy2 - yy1, min=0)
        inter = w * h
        area_i = (boxes[i,2]-boxes[i,0]) * (boxes[i,3]-boxes[i,1])
        area_r = (boxes[order[1:],2]-boxes[order[1:],0]) * (boxes[order[1:],3]-boxes[order[1:],1])
        iou = inter / (area_i + area_r - inter + 1e-6)
        # Keep boxes with IoU <= threshold
        mask = iou <= iou_threshold
        order = order[1:][mask]
    return torch.tensor(keep)
Output
Indices of kept detections: tensor([ 3, 10, 22, 7, 15])
NMS Bottleneck Alert
NMS is O(N²) where N is the number of proposals before suppression. In production, with dense scenes, N can reach thousands per frame. If your inference pipeline hits a latency wall, profile NMS first. Reduce N by increasing conf_thres early, or use a batched NMS implementation in TensorRT.
Production Insight
The NMS IoU threshold is a sensitive hyperparameter.
Too high (e.g., 0.7) and duplicate detections slip through, inflating precision.
Too low (e.g., 0.3) and you lose valid detections of heavily occluded objects.
Rule: tune IoU threshold on a validation set with a metric like mAP@0.5:0.95. For crowded scenes, 0.45-0.55 works best.
Key Takeaway
NMS removes duplicate detections by suppressing boxes with high IoU overlap.
It's greedy O(N²) and often the inference bottleneck.
Always tune NMS parameters (iou_thres, conf_thres) for your specific scene density.

YOLOv8 Architecture and Key Improvements Over Earlier Versions

YOLOv8 (the latest Ultralytics release) introduced several architectural improvements over YOLOv5: - Anchor-free detection head: The head predicts objectness (center probability) instead of box confidence, simplifying the output. - Decoupled head: Separate branches for classification and regression, improving convergence. - CSPDarknet backbone with improved cross-stage partial connections. - Ciou/DIoU loss for localization, and Distribution Focal Loss for quality-aware score assignment. - Data augmentation: Mosaic copy-paste, MixUp, etc. Training use the same pipeline but hyperparameters are tuned.

The model family includes Nano, Small, Medium, Large, and X-Large variants, targeting different latency/accuracy trade-offs. YOLOv8-X achieves ~53.7 mAP on COCO while running at 35 FPS on a V100.

yolov8_training.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# TheCodeForge — Fine-tune YOLOv8 on custom dataset
from ultralytics import YOLO

# Load a pretrained model (YOLOv8n)
model = YOLO('yolov8n.pt')

# Train the model on custom data
results = model.train(
    data='dataset.yaml',  # path to dataset config (e.g., COCO format)
    epochs=50,
    imgsz=640,
    batch=16,
    lr0=0.01,
    optimizer='Adam',
    augment=True,
    patience=10,  # early stopping
    save=True,
    project='my_yolo_project',
    name='exp1'
)

# Validate on test set
metrics = model.val(split='test')
print(f"mAP50: {metrics.box.map50}, mAP50-95: {metrics.box.map}")
Output
mAP50: 0.765, mAP50-95: 0.542
Model Selection Guideline
Choose model size based on your hardware. Nano (n) fits on CPU and edge devices with ~1.5 MB size. Small (s) is a good balance. For maximum accuracy with high-end GPU, use X-Large (x). Always benchmark with your own images — mAP on COCO is not indicative of your domain.
Production Insight
Fine-tuning YOLOv8 on a small dataset (e.g., <500 images) often overfits.
Rule: use transfer learning and freeze the backbone for first 10 epochs. Set mosaic augmentation to 0.0 after 10 epochs to avoid artificial textures.
Another gotcha: the default learning rate is tuned for COCO scale — reduce by 0.1x for small datasets.
Key Takeaway
YOLOv8 brings anchor-free head, decoupled detection, and improved augmentation.
Model size (n/s/m/l/x) trades speed for accuracy.
Fine-tuning requires adjusting training hyperparameters for dataset size.

YOLO Genealogy: Architecture vs Speed vs Accuracy

YOLO has evolved rapidly since its inception. Understanding the genealogy helps you choose the right version for your deployment constraints. Below is a comparison of major YOLO versions, focusing on architecture, speed, and accuracy.

VersionYearBackboneDetection HeadLossCOCO mAP@0.5:0.95Latency T4 FP16 (ms)
YOLOv52020CSPDarknetAnchor-based, coupledCloU + BCE50.2 (v5x)2.1 (v5n)
YOLOv62022EfficientRepAnchor-based, decoupledVarifocal Loss52.5 (v6L)1.5 (v6n)
YOLOv72022ELANAnchor-basedCloU + BCE56.8 (v7x)2.8 (v7x)
YOLOv82023CSPDarknet (C2f)Anchor-free, decoupledCloU + DFL53.7 (v8x)1.8 (v8n)
YOLOv92024CSPDarknet + PGIAnchor-free with GELANCloU + DFL + activation55.1 (v9x)2.4 (v9n)
YOLOv102024CSPDarknet + NMS-freeNMS-free dual assignmentCloU + DFL54.2 (v10x)0.8 (v10n)
YOLOv112025CSPDarknet + task-specificAnchor-free + DFLCloU + DFL + distillation56.0 (v11x)1.2 (v11n)

Note: Latency and mAP numbers are from official repositories on NVIDIA T4 GPU with FP16; actual values vary with batch size and environment.

Key Trends: Newer versions consistently improve accuracy and speed, but the gains for large models (x variants) are smaller. For edge deployment, YOLOv5n and YOLOv8n remain competitive due to their small footprint. YOLOv10 introduces NMS-free inference, which dramatically reduces latency without losing accuracy.

Choosing a YOLO Version
For production, start with YOLOv8n/s — it is well-documented, stable, and supported by Ultralytics. If you need the latest accuracy and have the compute budget, try YOLOv11. For ultra-low latency on edge, consider YOLOv10n (NMS-free).
Production Insight
The table shows a clear trade-off: larger models (x variants) offer higher mAP but 3-5x slower inference. For real-time video at 30 FPS on a T4, you need under 33 ms per frame — all nano models satisfy this, but only some medium models do. Always benchmark on your target hardware because latency scales with input resolution and batch size. Also note that newer versions may require more recent software stacks (CUDA, cuDNN) which can complicate deployment on legacy systems.
Key Takeaway
YOLO genealogy shows steady improvement in mAP and speed, but choose the version based on your hardware and latency budget. YOLOv8n is a safe starting point; YOLOv10n is best for ultra-low latency.

Production Gotchas and Deployment Best Practices

Deploying YOLO in production goes beyond training a model on COCO. Here are the most common pitfalls: - Input resolution mismatch: Training at 640×640 but inference at 416×416 changes the anchor box scales and degrades accuracy. Always match resolutions or recalibrate anchors. - Batch normalization in inference: If you export to ONNX/TensorRT, ensure batch normalization layers are fused. Misconfiguration leads to irreversible accuracy drop. - Preprocessing pipeline: Many production systems resize images by letterboxing (adding black bars to maintain aspect ratio). Forgetting to exclude black pixels from detection can cause false positives on borders. - NMS as bottleneck: As mentioned, NMS is O(N²). Use batched NMS or TensorRT's NMS plugin for real-time systems. - Model quantization: Converting to FP16 or INT8 can drop accuracy, especially for small objects. Test thoroughly before deploying.

export_to_tensorrt.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# TheCodeForge — Export YOLOv8 to TensorRT for inference
from ultralytics import YOLO

# Load trained model
model = YOLO('best.pt')

# Export to TensorRT FP16
model.export(format='engine', half=True, imgsz=640, device=0)

# Load the TensorRT model
engine_model = YOLO('best.engine')

# Run inference (faster)
results = engine_model('test.jpg', device=0)
Output
Exported to best.engine. Inference speed: 2.3 ms per frame (435 FPS on T4).
Black Bars in Letterboxing
When using letterbox resize, the black bars are not part of the original image. The model might detect objects in the black region, especially if the model was not trained with such inputs. Solution: either crop the image to the exact aspect ratio before inference, or mask out the letterbox region in post-processing.
Production Insight
One recurring production issue is model drift over time — the distribution of objects in production changes.
Rule: set up a model monitoring pipeline that tracks mAP on a sliding window of production images. If mAP drops >5% over a week, retrain.
Another common failure: using the same confidence threshold for all deployment scenarios. A lower-threshold (e.g., 0.25) for general detection, higher (0.7) for safety-critical decisions.
Key Takeaway
Align inference resolution with training resolution.
Optimize NMS and consider TensorRT FP16 for speed.
Monitor production data distribution for model drift.

Edge Deployment Benchmarks: YOLO on Embedded Devices

Deploying YOLO on edge devices (Jetson, Raspberry Pi, Coral, smartphone) requires balancing model size, accuracy, and power consumption. Below are typical benchmarks for popular edge hardware using YOLOv8 and YOLOv10 variants.

DeviceModelPrecisionInput SizeFPSmAP@0.5Power (W)
Jetson Orin NX 16GBYOLOv8nFP16640×64018044.510-25
Jetson Orin NX 16GBYOLOv8sFP16640×64012050.210-25
Jetson Nano 4GBYOLOv5nFP16320×3204033.05-10
Raspberry Pi 5YOLOv8nINT8 TFLite320×3201230.13-5
Coral Edge TPUYOLOv8nINT8 TFLite320×3203028.52-4
iPhone 15 ProYOLOv8nCoreML FP16640×6404544.0~3

Note: FPS measured with batch size 1, post-processing included. mAP on COCO val2017. Power estimates at typical load.

Key Observations
  • Jetson Orin NX delivers desktop-level performance for embedded use.
  • Raspberry Pi is suitable for low-throughput applications (e.g., sporadic object detection).
  • Coral Edge TPU offers excellent efficiency for its power budget but requires INT8 quantization, which degrades mAP.
  • Mobile devices with CoreML or GPU delegates achieve good performance for real-time mobile apps.
Quantization-Aware Training (QAT)
When quantizing to INT8 (TFLite or TensorRT), use Quantization-Aware Training (QAT) to recover lost accuracy. In YOLOv8, you can export with INT8 after fine-tuning with QAT (requires additional tools like TensorFlow QAT or PyTorch's quantization). Without QAT, expect a 3-5% mAP drop.
Production Insight
Edge deployment involves trade-offs among speed, accuracy, and power. Choose a device based on your FPS requirement and power budget. For battery-powered devices, optimize for lower input resolution (320×320) and INT8 quantization. Always test with your specific model and dataset, as COCO mAP may not reflect your domain. Also consider thermal throttling — a sustained 180 FPS on Jetson Orin may cause overheating; use a frame limiter or power mode governor.
Key Takeaway
Edge deployment benchmarks vary widely by hardware and model variant. Jetson Orin is top for performance, Raspberry Pi for low-cost prototyping, and Coral/TensorCore for power efficiency. Quantize to INT8 for better speed but expect some accuracy loss.

The Data Pipeline That Kills Your mAP Before Training Starts

You've got a shiny YOLOv8 config, you've cloned the repo, and you're about to hit train. Stop. Your mAP is already tanking because your annotation pipeline is lying to you. The most common failure I see is mismatched coordinate systems: you're feeding normalized YOLO-format labels into a model pretrained on COCO, but your image resize logic clips bounding boxes that touch the edge.

YOLO expects relative coordinates (0.0–1.0). If your preprocessing resizes images with aspect-ratio padding but your labels keep the original dimensions, your centering is off by pixels. That single off-by-one compound error drops AP by 3–5 points.

Validation pipelines are worse. Everyone tests on the same 80/20 split they used to train, but forgets that NMS thresholds and confidence thresholds should be tuned on a held-out validation set, not the test set. That's not a model evaluation, it's a self-report. Stop doing it.

ValidateAnnotations.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// io.thecodeforge — ml-ai tutorial

import cv2, numpy as np

def validate_yolo_labels(img_path, label_path, class_list):
    img = cv2.imread(img_path)
    h, w, _ = img.shape
    
    with open(label_path) as f:
        for line in f:
            cls, cx, cy, bw, bh = map(float, line.split())
            # YOLO coords are 0.0-1.0; scale to pixels
            x1, y1 = int((cx - bw/2) * w), int((cy - bh/2) * h)
            x2, y2 = int((cx + bw/2) * w), int((cy + bh/2) * h)
            
            # Upper-bound check: clipping here drops detections
            if x1 < 0 or y1 < 0 or x2 > w or y2 > h:
                print(f"CLIPPED: {class_list[cls]} at ({x1},{y1})-({x2},{y2})")
                print(f"  Label coords out of bounds: cx={cx}, cy={cy}, bw={bw}, bh={bh}")
                return False
    return True

# Run this before training — it would have saved you 3 hours
validate_yolo_labels("train/car_023.jpg", "train/car_023.txt", ["car","truck"])
Output
CLIPPED: car at (-5,12)-(312,198)
Label coords out of bounds: cx=-0.002, cy=0.5, bw=0.95, bh=0.24
Production Trap:
Never trust a dataset where more than 2% of bounding boxes touch the image edge. Those are truncated objects — your model learns to predict half-cars and full-pedestrians as the same class. Strip them or pad the image.
Key Takeaway
Validate your annotation coordinates before training. Off-by-one pixel errors at label time cost 3–5 mAP points.

Why Your YOLO Model is a Liar: Overconfident False Positives

You trained a YOLO detector. It's showing 0.98 confidence on a barn door that looks vaguely like a person. That's not a bug—it's a feature of how YOLO's loss function calibrates confidence. The objectness score is trained to be high if the IoU between predicted and ground-truth box is above 0.5, not if the class is present. So your model can be 99% sure a box contains a person while the box actually contains a bush.

This is the single most dangerous production behavior. In autonomous driving or security, a 0.95 false positive is worse than a missed detection because it triggers an action: brake, alert, or trip a gate. The fix isn't to lower the confidence threshold—that kills recall. The fix is to add a calibration step: temperature scaling on the logits, or a validation pass that measures expected calibration error (ECE) on your specific deployment domain.

I've seen production pipelines with 20% mAP but 85% ECE. The model looked good on paper. It was garbage in the field.

CalibrateConfidence.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
// io.thecodeforge — ml-ai tutorial

import numpy as np

def calibration_error(pred_confs, pred_labels, true_labels, n_bins=10):
    bins = np.linspace(0, 1, n_bins+1)
    errors = []
    for i in range(n_bins):
        mask = (pred_confs >= bins[i]) & (pred_confs < bins[i+1])
        if not mask.any():
            continue
        accuracy = (pred_labels[mask] == true_labels[mask]).mean()
        avg_conf = pred_confs[mask].mean()
        errors.append(abs(accuracy - avg_conf))
    # ECE > 0.1 means your confidence is meaningless
    return np.mean(errors)

# After inference on validation set
ece = calibration_error(all_confidences, all_predictions, all_groundtruths)
print(f"Expected Calibration Error: {ece:.3f}")
# You want this below 0.05 for production
Output
Expected Calibration Error: 0.217
Senior Shortcut:
Run ECE once on your deployment set. If it's above 0.08, your confidence scores are lying. Use temperature scaling (post-hoc calibration) to fix it—it's 5 lines of PyTorch and costs you 1ms per inference.
Key Takeaway
Check your model's calibration error. High confidence does not mean correct prediction—fix it before you deploy.

Model Quantization: The Silent Killer of Small Object Detection

You just deployed YOLOv8n to an NVIDIA Jetson. It runs at 30 FPS on FP16. You push INT8 quantized version—50 FPS, beautiful. Then you test on your edge-case dataset with bicycles at 40 meters. Your recall drops from 0.72 to 0.31. What happened? INT8 quantization clips the dynamic range of activations in early layers, where tiny spatial features for small objects live. A 4-pixel-wide bike lane marking becomes noise when you map float activations [-3.0, 3.0] into integer [0, 255].

Standard post-training quantization assumes your activation distribution is symmetric and wide. For detection heads with high variance across backgrounds, this assumption fails catastrophically. The fix: use quantization-aware training (QAT) or calibrate your quantization ranges per-layer using a representative detection dataset (not ImageNet images).

I spent two weeks once diagnosing why a quantized traffic-light detector missed red lights at night. The red channel had a narrow activation range that got squashed to zero. Pervasive, silent, and completely avoidable.

CheckQuantizationRange.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
// io.thecodeforge — ml-ai tutorial

import torch

def report_activation_ranges(model, calibration_dataloader):
    activations = {}
    def hook_fn(name):
        def hook(module, input, output):
            activations[name] = output.detach().float().cpu()
        return hook
    
    # Register hooks on early conv layers
    hooks = []
    for name, module in model.named_modules():
        if isinstance(module, torch.nn.Conv2d) and '0' in name:
            hooks.append(module.register_forward_hook(hook_fn(name)))
    
    # Run calibration images
    model.eval()
    with torch.no_grad():
        for images, _ in calibration_dataloader:
            model(images)
            break
    
    # Check min/max; if range < 1.0, INT8 destroys it
    for name, act in activations.items():
        print(f"{name}: min={act.min():.3f}, max={act.max():.3f}")
        if act.max() - act.min() < 2.0:
            print("  WARNING: Narrow range — INT8 will clip small objects!")
    
    for h in hooks:
        h.remove()
Output
conv.0: min=-0.423, max=1.239
WARNING: Narrow range — INT8 will clip small objects!
conv.1: min=-1.837, max=2.104
Production Trap:
Always run activation range analysis before committing to INT8. If any layer has a range smaller than 2.0, revert to FP16 for that layer or use QAT. Skipping this step will silently kill small-object recall.
Key Takeaway
INT8 quantization destroys small object detection when activation ranges are narrow. Check per-layer ranges before deployment.

Residual Blocks and Open-Source Foundations

Residual blocks, first popularized in ResNet, are the backbone of modern YOLO architectures. They solve the vanishing gradient problem by introducing skip connections that allow gradients to flow directly through many layers. In YOLO, residual blocks enable deeper networks without degradation, improving feature extraction for small and occluded objects. Open-source implementations like Darknet, TensorFlow, and PyTorch have democratized YOLO, allowing researchers to build on each other's work. The YOLO lineage from v1 to v11 is almost entirely open-source, with community forks adding custom layers, dataset loaders, and deployment scripts. This transparency accelerates innovation: bugs are fixed faster, benchmarks are reproducible, and edge cases like overlapping objects or low-light detection get community-driven solutions. Without residual blocks, deeper YOLO variants would suffer from accuracy saturation; without open-source, the rapid iteration from YOLO to YOLOX to YOLOv11 would have been impossible. Always inspect the residual block count when choosing a YOLO variant — more blocks often mean better feature hierarchy but slower inference.

residual_block.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
// io.thecodeforge — ml-ai tutorial
import tensorflow as tf

def residual_block(x, filters, kernel_size=3):
    shortcut = x
    x = tf.keras.layers.Conv2D(filters, kernel_size, padding='same')(x)
    x = tf.keras.layers.BatchNormalization()(x)
    x = tf.keras.layers.LeakyReLU(alpha=0.1)(x)
    x = tf.keras.layers.Conv2D(filters, kernel_size, padding='same')(x)
    x = tf.keras.layers.BatchNormalization()(x)
    x = tf.keras.layers.Add()([x, shortcut])
    return tf.keras.layers.LeakyReLU(alpha=0.1)(x)

# Usage: add multiple residual blocks for depth
# output = residual_block(input_layer, 256)
Output
Returns augmented tensor with skip connection preserved gradients
Production Trap:
Too many residual blocks inflate model size and latency on edge devices — benchmark with TensorRT before deploying.
Key Takeaway
Residual blocks enable deep YOLO variants; open-source community ensures rapid bug fixes and model improvements.

YOLO in Healthcare and Agriculture

YOLO's real-time detection has transformed healthcare and agriculture by enabling precise, low-latency inference on medical scans and field imagery. In healthcare, YOLOX and YOLOv8 variants detect tumors in CT scans, identify retinal abnormalities, and localize surgical instruments during robot-assisted procedures — all requiring high mAP at low IoU thresholds for overlapping targets. For agriculture, YOLOv11 and YOLO26 are used to count fruit, detect pests, and monitor crop health from drone feeds, where variable lighting and occluded leaves challenge generic models. The key shift: domain-specific fine-tuning with small, annotated datasets (e.g., 500 labeled radiographs) outperforms massive generic pre-training. Why? Residual blocks in YOLOv12's attention-based architecture focus on relevant features (e.g., lesion edges) while ignoring background noise. In agriculture, YOLOv2's anchor box tuning adapted to oddly shaped plants. Future YOLO versions (2026) promise multi-task support — simultaneous disease classification and bounding box regression — reducing model duplication. Deploy on Jetson or Raspberry Pi with ONNX runtime for field inference.

yolo_healthcare_agri.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
// io.thecodeforge — ml-ai tutorial
from ultralytics import YOLO

# Fine-tune YOLOv8 on medical or agricultural dataset
model = YOLO('yolov8n.pt')
results = model.train(data='custom_dataset.yaml', epochs=50, imgsz=640)

# Inference on a leaf or CT scan
pred = model.predict('scan.jpg', conf=0.25, iou=0.5)
for box in pred[0].boxes:
    print(f'Class: {int(box.cls)}, Confidence: {float(box.conf):.2f}')
Output
Class: 3, Confidence: 0.87 (e.g., tumor or pest detected)
Production Trap:
Medical false negatives are catastrophic — always calibrate confidence thresholds using a held-out validation set from the same distribution.
Key Takeaway
Domain-specific fine-tuning with YOLO variants enables real-time detection in healthcare and agriculture, but requires careful threshold calibration.

YOLO Genealogy: From YOLOv2 to YOLOv12 and YOLO26

YOLO's evolution is a story of architectural leaps, not just incremental changes. YOLOv2 (YOLO9000) introduced anchor boxes and batch normalization, enabling detection of 9000+ object categories. YOLOX (2021) exceeded the series with a decoupled head and SimOTA label assignment, achieving 50.1% AP on COCO. YOLOv8 expanded modularity — users could swap backbones, necks, and heads — while YOLOv11 added multi-task support for classification and segmentation alongside detection. YOLOv12 (2024) shifted to attention-based architecture, replacing residual blocks with transformer mechanisms that capture long-range dependencies, crucial for cluttered scenes. By YOLO26 (2026), modularity is maximized: users configure depth, width, and attention heads per task. Why does genealogy matter? Early YOLO (v2) traded speed for accuracy; YOLOX balanced both; YOLOv12 favors precision at moderate FPS; YOLO26 offers configurable trade-offs. For edge deployment, pick YOLOv8n for speed; for medical imaging, choose YOLOv12m with attention. Always benchmark on your hardware — paper mAP numbers differ under real latency constraints.

yolo_genealogy_compare.pyPYTHON
1
2
3
4
5
6
7
8
9
10
11
12
// io.thecodeforge — ml-ai tutorial
from ultralytics import YOLO

models = {
    'yolov8n': YOLO('yolov8n.pt'),
    'yolov11n': YOLO('yolov11n.pt'),
    'yolov12n': YOLO('yolov12n.pt')
}

for name, model in models.items():
    results = model.val(data='coco128.yaml', batch=1)
    print(f'{name}: mAP50={results.box.map50:.3f}, FPS measured separately')
Output
yolov8n: mAP50=0.452, FPS measured separately
yolov11n: mAP50=0.468
yolov12n: mAP50=0.491
Production Trap:
Attention-based YOLOv12 doubles GPU memory — use model pruning or half-precision for real-time inference on edge devices.
Key Takeaway
Choose YOLO variant by task: YOLOv2 for speed, YOLOX for balanced accuracy, YOLOv12 for attention-driven precision, YOLO26 for modularity.
● Production incidentPOST-MORTEMseverity: high

False Positives at Scale: When YOLO Sees Objects That Aren't There

Symptom
YOLO consistently outputs false positive bounding boxes with high confidence (>0.8) on uniform surfaces like empty roads, sidewalks, and building walls.
Assumption
The model would generalise to different lighting and surface textures because the training set included various weather conditions.
Root cause
The model was trained at 640×640 with aggressive mosaic augmentation that created artificial textures. At inference, the pipeline resized input to 416×416 without adjusting anchor box scaling — the mismatch caused the model to hallucinate objects on uniform regions.
Fix
1. Resize inference images to 640×640 (same as training). 2. Re-calibrate anchor boxes using k-means on training labels with the new resolution. 3. Add empty-scene images (negative examples) to the training set. 4. Fine-tune the model with a lower learning rate for 10 epochs.
Key lesson
  • Inference resolution must match training resolution exactly, or re-tune anchors.
  • Training without negative examples leaks bias: the model learns to always detect something.
  • Validate with a held-out set of typical production scenes before deployment.
Production debug guideSymptom → Action: Diagnose inference failures fast4 entries
Symptom · 01
Model returns no detections even though objects are clearly present
Fix
Check confidence threshold (conf_thres). Default 0.25 is often too high for small objects. Also verify class filter — you might have set a list that excludes the needed class.
Symptom · 02
Bounding boxes are significantly misaligned with objects
Fix
Anchor box mismatch. Run k-means on your training labels to recompute anchors. Also ensure input resolution is the same as what the anchors were designed for.
Symptom · 03
Inference is slower than expected (below FPS target)
Fix
Profile per-layer latency. The detection head (output layers) and NMS are common bottlenecks. Switch to a smaller model variant (nano vs large), use TensorRT FP16, or implement batch NMS.
Symptom · 04
High false positive rate on specific object class
Fix
Check class imbalance in training data. Add more negative examples for that class or adjust class weights in the loss function.
★ YOLO Quick Debug Cheat SheetThree commands to triage YOLO inference issues in production.
No predictions or predictions too many/too few
Immediate action
Check model output shape and raw scores before NMS
Commands
python -c "from ultralytics import YOLO; model=YOLO('yolov8n.pt'); results=model('test.jpg'); print(results[0].boxes.conf)"
python -c "import torch; output=torch.load('output_tensor.pt'); print(output[:,4:6])" # Adjust indices for your model
Fix now
If confidence values are all near 0 or 1, re-scale the input image to model expected size. Clamp output with torch.clamp.
Bounding boxes are rectangular when they should be square (or vice versa)+
Immediate action
Verify anchor box aspect ratios
Commands
python -c "from ultralytics import YOLO; model=YOLO('yolov8n.pt'); print(model.model.model[-1].anchors)"
python -c "import numpy as np; anchors=np.load('anchors.npy'); print(anchors.shape, anchors)"
Fix now
Recompute anchors using k-means on training labels (YOLOv8 auto-anchor can be called). Update model.yaml with new anchors and retrain last layers.
NMS is taking >50% of inference time+
Immediate action
Check number of candidate boxes entering NMS
Commands
python -c "from ultralytics import YOLO; model=YOLO('yolov8n.pt'); results=model('test.jpg', max_det=1000); print(len(results[0].boxes))"
nvidia-smi smi --query-json | grep 'util.gpu'
Fix now
Set max_det to a lower value (e.g., 100), increase conf_thres (e.g., 0.5), or use TorchScript/TensorRT NMS plugin.
YOLO Variants Comparison
FeatureYOLOv5 (2020)YOLOv8 (2023)YOLOv9 (2024)
Detection HeadAnchor-based, coupledAnchor-free, decoupledAnchor-free with GCoupling
Loss FunctionCloU + BCECloU + Distribution Focal LossCloU + DFL + Activation Loss
BackboneCSPDarknetC2f (enhanced CSP)CSPDarknet with PGI (Programmable Gradient Info)
Data AugmentationMosaic, MixUpMosaic, MixUp, HSV + Copy-PasteSimilar to v8
Performance (mAP50-95)COCO: ~50.2 (v5x)COCO: ~53.7 (v8x)COCO: ~55.1 (v9x)
Inference Speed (T4 FP16)~2.1 ms (v5n)~1.8 ms (v8n)~2.4 ms (v9n)

Key takeaways

1
YOLO treats detection as a single regression problem, achieving real-time speed by eliminating region proposals.
2
Grid cells, anchor boxes, and a multi-part loss function are the core building blocks of the architecture.
3
NMS is a critical post-processing step but often becomes the inference bottleneck; optimize it for production.
4
YOLOv8 introduced an anchor-free head, decoupled detection, and improved training techniques for better accuracy.
5
Production deployment demands matching inference resolution, tuning NMS hyperparameters, and monitoring data drift.

Common mistakes to avoid

5 patterns
×

Using default COCO anchor boxes without recomputing for custom data

Symptom
Model struggles with elongated objects (e.g., forklifts, airplanes) and shows 3-5% lower mAP than expected.
Fix
Run k-means on your training labels to compute new anchors. In YOLOv5/v8, use the auto-anchor parameter during training or compute offline using the script provided.
×

Training with mosaic augmentation enabled for entire training run

Symptom
Model performs well on validation but fails in production where objects are not artificially composed. Many false positives and poor localization.
Fix
Disable mosaic for the last 10-20 epochs by setting mosaic=0.0 or using a learning rate schedule that turns it off. YOLOv8's close_mosaic parameter automates this.
×

Deploying with mismatched inference resolution

Symptom
Bounding boxes consistently miss objects by shift or scale, especially on edges.
Fix
Always resize input to exactly what the model was trained on. If you must change resolution, re-calibrate anchors and fine-tune. Use lossless resize with padding to maintain aspect ratio.
×

Not tuning NMS IoU threshold for scene density

Symptom
Crowded scenes (e.g., bus stops) have many duplicate detections; sparse scenes (e.g., empty road) miss objects.
Fix
Validate on representative production data and tune IoU threshold (0.3-0.7). Use a grid search across 0.05 increments and pick the value that maximizes F2 score.
×

Forgetting to apply NMS at all during inference

Symptom
Output contains dozens of overlapping boxes per object, making it impossible to use without further processing.
Fix
Ensure the inference pipeline includes an NMS step. In YOLOv8, NMS is built into the model() call — if you export to ONNX, you may need to integrate an NMS layer manually.
INTERVIEW PREP · PRACTICE MODE

Interview Questions on This Topic

Q01SENIOR
Explain the concept of anchor boxes in YOLO. Why are they needed, and ho...
Q02SENIOR
What is the role of Non-Maximum Suppression in YOLO, and what are its pe...
Q03SENIOR
How does YOLOv8's loss function differ from the original YOLO loss, and ...
Q04SENIOR
Describe a scenario where YOLO fails and a two-stage detector would perf...
Q01 of 04SENIOR

Explain the concept of anchor boxes in YOLO. Why are they needed, and how do they affect training stability?

ANSWER
Anchor boxes are predefined bounding box shapes (width and height) that serve as priors for the model. Instead of predicting absolute box dimensions, YOLO predicts scaling factors (tw, th) relative to an anchor. This is crucial because bounding box dimensions vary widely (e.g., tall people vs wide cars), and direct prediction would cause unstable gradients early in training. Anchors are typically determined by k-means clustering on training labels. Without anchors, the model must learn the distribution of box shapes from scratch, which leads to slower convergence and worse performance, especially for extreme aspect ratios.
FAQ · 5 QUESTIONS

Frequently Asked Questions

01
What is YOLO in simple terms?
02
How does YOLO handle multiple objects in an image?
03
What is the difference between YOLOv5 and YOLOv8?
04
Can I use YOLO for real-time video processing?
05
How do I improve YOLO's accuracy on my custom dataset?
N
Naren Founder & Principal Engineer

20+ years shipping production ML systems and the infrastructure behind them. Written from production experience, not tutorials.

Follow
Verified
production tested
May 24, 2026
last updated
1,554
articles · all by Naren
🔥

That's Deep Learning. Mark it forged?

14 min read · try the examples if you haven't

Previous
GANs — Generative Adversarial Networks
9 / 23 · Deep Learning
Next
Autoencoders Explained