YOLO Object Detection — False Positives from Mismatch

Every time your phone unlocks with your face, a Tesla decides not to brake for a shadow, or a warehouse robot grabs the right box off a conveyor belt, an object detector is running in the background. The demand for detectors that are both accurate and fast enough to run in real time has never been higher — and that tension between accuracy and speed is exactly where YOLO was born.

Before YOLO (You Only Look Once), the dominant paradigm was two-stage detection: a region-proposal network first suggests thousands of bounding-box candidates, then a separate classifier scores each one. Models like R-CNN and Faster R-CNN achieved excellent mean Average Precision (mAP), but their pipeline was fundamentally serial. On a 2015 GPU, Faster R-CNN ran at roughly 7 frames per second — nowhere near the 30+ fps required for real-time video. YOLO reframed detection as a single regression problem, collapsing both stages into one convolutional network pass and hitting 45 fps on the same hardware.

By the end of this article you'll understand exactly how YOLO divides an image into a grid, predicts bounding boxes and class probabilities simultaneously, why anchor boxes exist and what goes wrong without them, how Non-Maximum Suppression cleans up overlapping detections, and what the loss function is actually penalising. You'll also run a complete YOLOv8 inference and fine-tuning pipeline, and walk away knowing the production gotchas that trip up even experienced ML engineers.

What is Object Detection — YOLO?

Object detection is the computer vision task of locating and classifying objects within an image. YOLO (You Only Look Once) treats it as a single regression problem directly from image pixels to bounding box coordinates and class probabilities. Unlike sliding-window or region-proposal approaches, YOLO divides the input image into an S×S grid. Each grid cell is responsible for predicting B bounding boxes and C class probabilities — all in one forward pass of a convolutional neural network.

Why does this matter? Because detection speed becomes frame-rate independent. Two-stage detectors first generate thousands of region proposals (slower) then classify each one. YOLO eliminates the region proposal step entirely, achieving real-time inference on consumer hardware.

# TheCodeForge — Real inference with YOLOv8
# Ensure ultralytics is installed: pip install ultralytics
from ultralytics import YOLO
import cv2

# Load a pretrained YOLOv8n model (nano: smallest, fastest)
model = YOLO('yolov8n.pt')

# Run inference on an image
results = model('parking_lot.jpg')

# Extract detections: (x1, y1, x2, y2, confidence, class_id)
detections = results[0].boxes.data.cpu().numpy()

# Loop over detections and print
for *box, conf, cls in detections:
    x1, y1, x2, y2 = [int(v) for v in box]
    label = results[0].names[int(cls)]
    print(f'{label}: ({x1},{y1},{x2},{y2}) conf={conf:.2f}')

# Visualize and save
annotated = results[0].plot()
cv2.imwrite('output.jpg', annotated)

How YOLO Works: Grid Cells, Bounding Boxes, and Class Probabilities

At the core of YOLO is a uniform grid of size S×S overlaying the input image. Each grid cell predicts a fixed number of bounding boxes, each with a confidence score indicating how likely the box contains an object, along with the box's coordinates (tx, ty, tw, th) relative to the cell. Additionally, each cell predicts a vector of C class probabilities (softmax across classes). During inference, the model outputs a tensor of shape S×S×(B×5+C).

The bounding box coordinates are encoded relative to the grid cell: - tx, ty are offsets from the top-left corner of the cell (sigmoid to keep them within [0,1]) - tw, th are log-space scaling factors relative to anchor box dimensions - Confidence = P(Object) * IoU(pred, truth) — quantifies both presence and box accuracy.

Class probabilities are independent per grid cell, meaning each cell assigns a probability distribution over classes regardless of which bounding box is responsible. This means the final detection for a cell is a combination of the best bounding box (highest confidence) and the cell's class prediction.

# TheCodeForge — Manual decoding of YOLOv8 predictions
import torch

def decode_yolo_output(pred, anchors, num_classes=80):
    # pred: (num_cells, num_anchors, 4 + 1 + num_classes)
    # anchors: (num_anchors, 2)
    batch_size, grid_h, grid_w, num_anchors, _ = pred.shape
    # Create grid coordinates
    grid_y, grid_x = torch.meshgrid(torch.arange(grid_h), torch.arange(grid_w), indexing='ij')
    # Decode center offsets
    pred_xy = torch.sigmoid(pred[..., 0:2])  # tx, ty -> cell-relative
    pred_xy = pred_xy + torch.stack([grid_x.float(), grid_y.float()], dim=-1).unsqueeze(-2)
    pred_xy = pred_xy / torch.tensor([grid_w, grid_h])  # normalize to [0,1]
    # Decode box size using anchors (tw, th -> width/height scaling)
    pred_wh = torch.exp(pred[..., 2:4]) * anchors.unsqueeze(0).unsqueeze(0)
    pred_wh = pred_wh / torch.tensor([grid_w, grid_h])
    # Confidence and class scores
    conf = torch.sigmoid(pred[..., 4:5])
    class_scores = torch.softmax(pred[..., 5:5+num_classes], dim=-1)
    return torch.cat([pred_xy, pred_wh, conf, class_scores], dim=-1)

Anchor Boxes: Why They Exist and What Goes Wrong Without Them

YOLO uses predefined anchor boxes (also called prior boxes) to help the model predict bounding box dimensions. Instead of predicting absolute width and height, the model predicts scaling factors (tw, th) relative to an anchor. This is critical because direct prediction of arbitrary box shapes leads to unstable gradients early in training — the model has to learn from scratch that boxes come in common aspect ratios (e.g., human: tall and thin, car: wide and short).

Anchor boxes are typically chosen by running k-means clustering on the training dataset's ground truth bounding box dimensions. For YOLOv5 and YOLOv8, anchors are automatically computed during training based on the data's bounding box shapes. The number of anchors per grid cell is usually 3 or 5.

During inference, each predicted bounding box is the anchor box scaled by the model's output. The final box is represented as (center_x, center_y, width, height) relative to the grid cell.

# TheCodeForge — Compute custom anchors with k-means
import numpy as np
from sklearn.cluster import KMeans

def compute_anchors(labels_file, num_anchors=9, image_size=640):
    # labels_file: CSV with img_id, class, x_center_norm, y_center_norm, width_norm, height_norm
    boxes = []
    with open(labels_file) as f:
        for line in f:
            parts = line.strip().split(',')
            w = float(parts[3]) * image_size
            h = float(parts[4]) * image_size
            boxes.append([w, h])
    boxes = np.array(boxes)
    kmeans = KMeans(n_clusters=num_anchors, random_state=0).fit(boxes)
    anchors = kmeans.cluster_centers_
    # Sort by area descending for assignment to detection head scales
    anchors = anchors[np.argsort(anchors[:,0]*anchors[:,1])[::-1]]
    return anchors.tolist()

# Example: anchors = compute_anchors('train_labels.csv', num_anchors=3)
# print(anchors)  # [[w1,h1], [w2,h2], ...]

Loss Function: What YOLO Actually Minimizes

YOLO's loss function is a multi-part objective that balances localization, confidence, and classification. The original YOLO paper used sum-squared error, but modern versions (YOLOv3+) use a combination of: - Localization loss: measures the error in bounding box coordinates. Typically CloU or GIoU loss that captures overlap, distance, and aspect ratio. - Confidence loss: binary cross-entropy (BCE) for whether an object exists in the box. Positive samples are predicted boxes that match a ground truth (highest IoU), negative samples are those with low IoU. - Classification loss: BCE for each class (multi-label) — the model can predict multiple classes per box?

The loss is weighted to prioritize localization and classification over confidence. Modern YOLO implementations assign one positive anchor per ground truth (based on IoU threshold) and ignore anchors with intermediate IoU to reduce noise.

Class imbalance is handled using focal loss-like weighting: the confidence loss down-weights easy negatives.

# TheCodeForge — Simplified YOLO loss (for illustration)
import torch.nn.functional as F

def yolo_loss(pred_boxes, target_boxes, pred_cls, target_cls, obj_mask, noobj_mask, loss_weights):
    # pred_boxes, target_boxes: (N,4) in (x,y,w,h) normalized
    # pred_cls, target_cls: (N,C)
    # obj_mask, noobj_mask: (N,) booleans
    # Compute CloU loss for positive samples
    iou_loss = F.cross_entropy(map_to_iou(pred_boxes[obj_mask], target_boxes[obj_mask]), ...)  # simplified
    # Confidence loss: BCE with obj_mask and noobj_mask
    conf_loss = F.binary_cross_entropy_with_logits(pred_conf[obj_mask], torch.ones_like(pred_conf[obj_mask]))
    conf_loss += loss_weights['noobj'] * F.binary_cross_entropy_with_logits(pred_conf[noobj_mask], torch.zeros_like(pred_conf[noobj_mask]))
    # Classification loss: BCE for each positive sample
    cls_loss = F.binary_cross_entropy_with_logits(pred_cls[obj_mask], target_cls[obj_mask])
    return loss_weights['box'] * iou_loss + loss_weights['conf'] * conf_loss + loss_weights['cls'] * cls_loss

How to Read mAP: Precision-Recall Curves and IoU Thresholds

Mean Average Precision (mAP) is the de facto metric for object detection. It summarizes the precision-recall trade-off across all classes and IoU thresholds. Understanding how to read mAP is essential for debugging model performance and making deployment decisions.

Precision-Recall Curves: For each class, the model's detections are ranked by confidence. As you lower the confidence threshold, more detections are considered, increasing recall but potentially decreasing precision. The precision-recall curve plots precision at each recall level. Average Precision (AP) is the area under this curve (AUC). mAP is the mean of AP across all classes.

IoU Thresholds: A detection is considered a true positive only if its Intersection over Union (IoU) with a ground truth box exceeds a threshold. The COCO evaluation uses 10 IoU thresholds from 0.50 to 0.95 at 0.05 increments. mAP@0.5 uses IoU=0.5 (lenient), while mAP@0.5:0.95 averages across all thresholds (more stringent). For production, mAP@0.5 is often used for coarse localization, but mAP@0.75 or mAP@0.5:0.95 better reflect precise box fitting.

How to interpret: A higher mAP@0.5:0.95 indicates the model can both detect objects and draw tight boxes. If mAP@0.5 is high but mAP@0.5:0.95 is low, the model detects objects but boxes are poorly aligned — a common sign of anchor mismatch or resolution issues.

# TheCodeForge — Compute mAP for YOLO predictions
import torch
from torchmetrics.detection.mean_ap import MeanAveragePrecision

# Suppose we have predictions and ground truths for one image
# preds: list of dicts with boxes, scores, labels
preds = [
    dict(
        boxes=torch.tensor([[100, 150, 200, 250], [300, 50, 400, 150]]),
        scores=torch.tensor([0.95, 0.80]),
        labels=torch.tensor([1, 2])
    )
]
target = [
    dict(
        boxes=torch.tensor([[95, 148, 205, 252], [295, 48, 405, 148]]),
        labels=torch.tensor([1, 2])
    )
]

metric = MeanAveragePrecision(iou_type="bbox")
metric.update(preds, target)
result = metric.compute()
print(f"mAP@0.5:0.95 = {result['map']:.4f}")
print(f"mAP@0.5 = {result['map_50']:.4f}")
print(f"mAP@0.75 = {result['map_75']:.4f}")

YOLO Implementation with Keras/TensorFlow

While the Ultralytics ecosystem is PyTorch-based, you can run YOLOv8 inference using TensorFlow via the KerasCV library or by exporting models to TensorFlow SavedModel format. This is useful for teams that standardise on TensorFlow Serving, TFLite on mobile, or TF.js in the browser.

KerasCV YOLOv8: The keras_cv package provides a YOLOV8Detector model pre-trained on COCO. It uses the same architecture as Ultralytics but implemented in pure Keras. Below we show how to load and run inference.

Export from PyTorch: Alternatively, export a trained Ultralytics model to TensorFlow via ONNX and then to TF. The ultralytics library supports export(format='saved_model') to generate a TensorFlow SavedModel directly.

TFLite on Edge: For edge deployment, convert the TensorFlow model to TFLite (FP16 or INT8) to run on ARM CPUs or Edge TPU.

# TheCodeForge — YOLOv8 inference with KerasCV
# Install: pip install keras-cv tensorflow
import keras_cv
import tensorflow as tf
import numpy as np
from PIL import Image

# Load pre-trained YOLOV8 Detector (backbone='csp_darknet', sizes: 'n','s','m','l','x')
model = keras_cv.models.YOLOV8Detector.from_preset(
    "yolo_v8_n_coco", bounding_box_format="xywh"
)

# Load and preprocess image
def preprocess_image(image_path, target_size=640):
    image = Image.open(image_path).convert('RGB')
    image = image.resize((target_size, target_size))
    image = np.array(image) / 255.0
    return tf.expand_dims(image, 0)

image_tensor = preprocess_image('test.jpg')

# Run inference
predictions = model.predict(image_tensor)
# predictions: {'boxes': (1,N,4), 'classes': (1,N), 'confidence': (1,N)}
boxes = predictions['boxes'][0].numpy()
scores = predictions['confidence'][0].numpy()
classes = predictions['classes'][0].numpy().astype(int)

# Filter by confidence
conf_thres = 0.5
valid = scores >= conf_thres
print("Detections (xywh, score, class):")
for box, score, cls in zip(boxes[valid], scores[valid], classes[valid]):
    x, y, w, h = box
    print(f"  [{x:.0f}, {y:.0f}, {w:.0f}, {h:.0f}] conf={score:.2f} class={cls}")

Non-Maximum Suppression (NMS): Cleaning Up Overlapping Detections

Because multiple grid cells and anchor boxes can predict the same object, the model often outputs many duplicate bounding boxes around a single object. Non-Maximum Suppression (NMS) is the post-processing step that consolidates these into one detection per object.

NMS works by: 1. Sorting all detections by confidence score (highest first). 2. Selecting the highest-confidence detection. 3. For each remaining detection, compute Intersection over Union (IoU) with the selected box. 4. If IoU > threshold (typically 0.5), suppress (remove) the overlapping detection. 5. Repeat until no more detections remain.

This greedy algorithm is simple but O(n²) in the number of candidate boxes, making it a bottleneck at high frame rates. Variants like Soft-NMS (reduce confidence instead of removing) or Fast NMS (vectorized) are used in practice.

Modern YOLO implementations (YOLOv5, YOLOv8) include NMS within the model pipeline; you can control it via iou_thres and conf_thres parameters.

# TheCodeForge — Manual NMS implementation
import torch

def nms(boxes, scores, iou_threshold=0.5):
    # boxes: (N,4) in (x1,y1,x2,y2) format
    # scores: (N,)
    keep = []
    order = scores.argsort(descending=True)
    while order.numel() > 0:
        i = order[0]
        keep.append(i)
        if order.numel() == 1:
            break
        # Compute IoU of remaining boxes with box i
        xx1 = torch.maximum(boxes[i,0], boxes[order[1:],0])
        yy1 = torch.maximum(boxes[i,1], boxes[order[1:],1])
        xx2 = torch.minimum(boxes[i,2], boxes[order[1:],2])
        yy2 = torch.minimum(boxes[i,3], boxes[order[1:],3])
        w = torch.clamp(xx2 - xx1, min=0)
        h = torch.clamp(yy2 - yy1, min=0)
        inter = w * h
        area_i = (boxes[i,2]-boxes[i,0]) * (boxes[i,3]-boxes[i,1])
        area_r = (boxes[order[1:],2]-boxes[order[1:],0]) * (boxes[order[1:],3]-boxes[order[1:],1])
        iou = inter / (area_i + area_r - inter + 1e-6)
        # Keep boxes with IoU <= threshold
        mask = iou <= iou_threshold
        order = order[1:][mask]
    return torch.tensor(keep)

YOLOv8 Architecture and Key Improvements Over Earlier Versions

YOLOv8 (the latest Ultralytics release) introduced several architectural improvements over YOLOv5: - Anchor-free detection head: The head predicts objectness (center probability) instead of box confidence, simplifying the output. - Decoupled head: Separate branches for classification and regression, improving convergence. - CSPDarknet backbone with improved cross-stage partial connections. - Ciou/DIoU loss for localization, and Distribution Focal Loss for quality-aware score assignment. - Data augmentation: Mosaic copy-paste, MixUp, etc. Training use the same pipeline but hyperparameters are tuned.

The model family includes Nano, Small, Medium, Large, and X-Large variants, targeting different latency/accuracy trade-offs. YOLOv8-X achieves ~53.7 mAP on COCO while running at 35 FPS on a V100.

# TheCodeForge — Fine-tune YOLOv8 on custom dataset
from ultralytics import YOLO

# Load a pretrained model (YOLOv8n)
model = YOLO('yolov8n.pt')

# Train the model on custom data
results = model.train(
    data='dataset.yaml',  # path to dataset config (e.g., COCO format)
    epochs=50,
    imgsz=640,
    batch=16,
    lr0=0.01,
    optimizer='Adam',
    augment=True,
    patience=10,  # early stopping
    save=True,
    project='my_yolo_project',
    name='exp1'
)

# Validate on test set
metrics = model.val(split='test')
print(f"mAP50: {metrics.box.map50}, mAP50-95: {metrics.box.map}")

YOLO Genealogy: Architecture vs Speed vs Accuracy

YOLO has evolved rapidly since its inception. Understanding the genealogy helps you choose the right version for your deployment constraints. Below is a comparison of major YOLO versions, focusing on architecture, speed, and accuracy.

Note: Latency and mAP numbers are from official repositories on NVIDIA T4 GPU with FP16; actual values vary with batch size and environment.

Key Trends: Newer versions consistently improve accuracy and speed, but the gains for large models (x variants) are smaller. For edge deployment, YOLOv5n and YOLOv8n remain competitive due to their small footprint. YOLOv10 introduces NMS-free inference, which dramatically reduces latency without losing accuracy.

Production Gotchas and Deployment Best Practices

Deploying YOLO in production goes beyond training a model on COCO. Here are the most common pitfalls: - Input resolution mismatch: Training at 640×640 but inference at 416×416 changes the anchor box scales and degrades accuracy. Always match resolutions or recalibrate anchors. - Batch normalization in inference: If you export to ONNX/TensorRT, ensure batch normalization layers are fused. Misconfiguration leads to irreversible accuracy drop. - Preprocessing pipeline: Many production systems resize images by letterboxing (adding black bars to maintain aspect ratio). Forgetting to exclude black pixels from detection can cause false positives on borders. - NMS as bottleneck: As mentioned, NMS is O(N²). Use batched NMS or TensorRT's NMS plugin for real-time systems. - Model quantization: Converting to FP16 or INT8 can drop accuracy, especially for small objects. Test thoroughly before deploying.

# TheCodeForge — Export YOLOv8 to TensorRT for inference
from ultralytics import YOLO

# Load trained model
model = YOLO('best.pt')

# Export to TensorRT FP16
model.export(format='engine', half=True, imgsz=640, device=0)

# Load the TensorRT model
engine_model = YOLO('best.engine')

# Run inference (faster)
results = engine_model('test.jpg', device=0)

Edge Deployment Benchmarks: YOLO on Embedded Devices

Deploying YOLO on edge devices (Jetson, Raspberry Pi, Coral, smartphone) requires balancing model size, accuracy, and power consumption. Below are typical benchmarks for popular edge hardware using YOLOv8 and YOLOv10 variants.

Note: FPS measured with batch size 1, post-processing included. mAP on COCO val2017. Power estimates at typical load.