← Back to Research
Research Note · Computer Vision & Real-Time Systems

#2 You Only Look Once (YOLO) — Real-Time Object Detection

Published: Aug 2026•8 min read•Author: Chetraj Jaishi
YOLOObject DetectionPaper NotesComputer VisionReal-Time Inference
Abstract: A personal research note on the seminal YOLO paper (Redmon et al.) — reframing object detection from complex multi-stage pipelines into a single unified regression problem running at 45–155 FPS.
Research Note from a Famous Research PaperAnnotated Reading
This is a research note that I took from a famous research paper.
Source: "You Only Look Once: Unified, Real-Time Object Detection" (Redmon et al., 2016)
Context · ResNet Carryover
Their ResNet-101 model was so powerful that it improved object detection scores (mAP — mean Average Precision) by 28% compared to the previous industry standard, VGG-16.

✦THE CORE IDEA

Older systems (R-CNN, DPM) basically repurpose classifiers to hunt for objects — run a classifier on a bunch of regions and see what sticks. YOLO does something different: it treats object detection as a single regression problem.
One CNN looks at the full image and directly predicts bounding boxes + class probabilities in one shot. No separate steps, no region-by-region classification.
The whole pipeline is just 3 steps:
1.
Resize the image
2.
Run it through the single CNN
3.
Apply non-max suppression (removes duplicate/overlapping boxes)
That's it. One network, one pass.

✦ADVANTAGES

•
Standard YOLO: 45 FPS
•
Fast YOLO: 155 FPS
•
Global Context: Because it sees the ENTIRE image at once (both training and testing), it understands context way better → far fewer "background errors" (mistaking a patch of background for an object) compared to Fast R-CNN
•
Domain Generalization: Learns general representations of what objects look like, so it transfers well to new domains (e.g. trained on natural photos, still works decently on artwork)

✦LIMITATIONS

•
Localization Precision: Faster, but less precise at pinpointing exact object location vs SOTA systems
•
Small Objects: Specifically struggles with small objects
Diagram Placeholderin this area - grid overlay diagram showing bounding box predictions
Research Schematic
Dog: 0.88Car: 0.64Grid Overlay & Prediction• Input partitioned into S × S cells (7×7)• Highlighted cell detects object center• Box 1: [x, y, w, h] + confidence• Box 2: [x, y, w, h] + confidenceOutput: S×S×(B*5 + C) Tensor
Figure: in this area - grid overlay diagram showing bounding box predictions

✦UNIFIED DETECTION — WHY THIS WAS NEEDED

THE OLD WAY (R-CNN STYLE)
Prior SOTA systems treated detection as a multi-step pipeline — slow, disjointed, hard to optimize end-to-end.
Diagram Placeholderin this area - flowchart of traditional R-CNN pipeline
Research Schematic
1. Input ImageFull resolution2. Region Proposals~2000 boxes(Selective Search)3. CNN Extraction2000 CNN passesSLOW: 40s/image4. SVM & BBoxClassification+ NMS CleanupTraditional Multi-Stage Pipeline: High latency, non-unified, ~40+ s/image
Figure: in this area - flowchart of traditional R-CNN pipeline
The steps looked like:
1.
Region Proposal — ~2000 candidate boxes generated via selective search
2.
Feature Extraction — CNN pulls features for every single box
3.
Classification — SVM scores each box
4.
Refinement — linear model refines box coordinates, NMS removes duplicates
Problem: this could take 40+ seconds per image. Generating thousands of region proposals per image = massive computational bottleneck. No way this runs in real time.
THE SHIFT
YOLO drops this entire pipeline. One CNN, one evaluation, predicts multiple bounding boxes + class probabilities directly from the full image at once.
Quick definitions for context:
•
CNN: neural net built for images, learns visual patterns (edges → textures → parts → objects) using small filters swept across the image
•
R-CNN: detection method that first proposes candidate boxes, then runs a CNN on each one to classify + refine it

✦HOW UNIFIED DETECTION ACTUALLY WORKS

The image is split into an S x S grid. If an object's center lands inside a grid cell, THAT cell is responsible for detecting it.
Diagram Placeholderin this area - grid diagram with bounding box predictions [x, y, w, h, conf] and class probabilities
Research Schematic
Grid Cell (i, j)Center of ObjectBox 1: [x, y, w, h, confidence]Box 2: [x, y, w, h, confidence]Classes: [C1, C2, ... C20 probabilities]Output Tensor Shape: S × S × (B × 5 + C) = 7 × 7 × 30
Figure: in this area - grid diagram with bounding box predictions [x, y, w, h, conf] and class probabilities
Each grid cell predicts:
•
B bounding boxes, each with its own confidence score
•
C conditional class probabilities
All of this gets packed into one tensor: S × S × (B × 5 + C)
For PASCAL VOC specifically: 7 × 7 × 30 tensor
→ S = 7, B = 2, C = 20

✦THE ARCHITECTURE

Inspired by GoogLeNet. 24 conv layers + 2 fully connected layers.
Uses 1×1 reduction layers to shrink the feature space (keep things efficient).
Whole thing trained end-to-end, directly optimizing for detection — not classification first then bolted-on detection.

✦TRAINING PROCESS

STEP 1 — PRETRAINING
•
Conv layers pretrained on ImageNet at 224×224 resolution
•
Uses the ImageNet 1000-class dataset
•
Only the first 20 conv layers are used here, followed by average-pooling + 1 fully connected layer
•
Trained/run using Darknet — an open-source, high-performance framework written in C and CUDA
•
This pretraining alone hits 88% top-5 accuracy on ImageNet 2012 validation set
Diagram Placeholderin this area - flowchart showing layer reduction from 448x448 input down to 7x7x30 output tensor
Research Schematic
Input448 × 4483 channels24 Conv Layers1×1 Reduction+ 3×3 Convolutions(GoogLeNet style)2 FC LayersDense 4096Dropout 0.5Output7 × 7 × 30TensorUnified End-to-End Inference Pipeline (45 FPS)
Figure: in this area - flowchart showing layer reduction from 448x448 input down to 7x7x30 output tensor
STEP 2 — CONVERTING TO A DETECTOR
Once pretrained, the network gets converted for detection:
•
Input resolution doubled: 224×224 → 448×448 (detection needs finer detail than classification)
•
4 new conv layers + 2 new fully connected layers added on top, randomly initialized
•
Final layer predicts two things at once:
•
class probabilities (what the object is)
•
bounding box coordinates (where the object is)
BOUNDING BOX NORMALIZATION
To keep numbers consistent and stable during training:
•
Box width/height are divided by image width/height → stays between 0 and 1
•
Box x/y are expressed as offsets from the grid cell's location → also kept between 0 and 1

✦TL;DR

•
Old detection (R-CNN): multi-step, slow (40+ sec/image), thousands of region proposals, hard to train end-to-end.
•
YOLO: one CNN, one pass over the full image, predicts boxes + classes together as a regression problem → real-time speed (45–155 FPS), better context understanding, generalizes well to new domains, but trades off some precision especially on small objects.

Key Research Takeaways

  • Detection as Unified Regression: Replaces multi-step candidate classification with a single forward pass predicting boxes and classes directly.
  • Real-Time Throughput: Delivers 45 FPS (standard YOLO) to 155 FPS (Fast YOLO), eliminating the 40+ second bottleneck of selective search.
  • Holistic Context Awareness: Sees the full image at once during training and inference, drastically reducing background false positives.
  • S × S Grid Structure: Encodes bounding boxes (x, y, w, h, confidence) and class probabilities in a compact tensor (e.g., 7 × 7 × 30 for VOC).