Transfer Learning & CNN Applications

1 Why transfer learning

• Training from scratch (e.g. ResNet-152 on ImageNet) needs ~1.2M labeled images, 8 GPUs, weeks. Reality: a few hundred images, one laptop/Colab GPU, a deadline.
• Transfer learning: use knowledge learned on a large dataset (ImageNet) to solve a different task with limited data.
• Two approaches: (1) Feature extraction — freeze the pre-trained net, train a new head. (2) Fine-tuning — unfreeze some layers, adapt with a small LR.
• In practice you almost never train from scratch — transfer learning is the default.

2 What CNN layers learn

• Early layers = generic: edges, colors, textures → transfer everywhere.
• Later layers = task-specific: patterns → object parts → specialized for the original task.
• Hierarchy: Layer 1 (edges/colors) → L2 (textures) → L3 (patterns) → L4–5 (object parts).
• This is why you keep early layers and only retrain the end for a new task.

3 Feature extraction

• Use the frozen pre-trained CNN as a fixed feature extractor; it outputs a feature vector (e.g. 512-dim) that you feed to a new trainable classifier.
• Steps: load pre-trained model → freeze all layers (requires_grad=False) → replace final FC head → train only the new head.
• Fast, needs little data, trains only ~0.1% of parameters yet hits high accuracy.

model = resnet50(weights='IMAGENET1K_V1')
for param in model.parameters():
    param.requires_grad = False        # freeze backbone
model.fc = nn.Linear(2048, num_classes) # new head (trainable)
optimizer = Adam(model.fc.parameters(), lr=0.001)

Note: a frozen layer still runs in the forward pass to produce features; its weights just never update in backprop.

4 Fine-tuning

• Fine-tuning: unfreeze some pre-trained layers and train end-to-end with a small learning rate.
• Differential LR: new head LR ≈ 1e-3; unfrozen pre-trained layers get 10–100× smaller LR (≈ 1e-4); deeper-frozen layers LR = 0.
• Critical: a large LR on pre-trained layers destroys useful features in the first epoch (catastrophic forgetting). Small LR adapts gently without erasing ImageNet knowledge.

Three strategies:

• Conservative — unfreeze only last 1–2 blocks. Safe for small data, minimal forgetting.
• Aggressive — unfreeze all layers, very small LR. Best for large data / very different domains.
• Gradual unfreezing — start head-only, unfreeze more each epoch. Safest for beginners.

Practical tips: always train the head first for 1–2 epochs before unfreezing; keep batch norm in eval mode while fine-tuning; monitor val loss (overfits fast on small data); use early stopping.

5 When to use which — the 2×2

• Rule of thumb: start with feature extraction; unfreeze more if needed.
• Early layers (edges, textures) are the most likely to transfer across different image domains.

6 Domain shift

• Domain shift: model trained on domain A, deployed on domain B → performance drops. E.g. studio→phone photos, US→rural clinic X-rays, synthetic→real-world.
• Simplest fix: fine-tune on a small labeled set from the target domain — even 100–500 target samples can significantly close the gap.

7 Object detection

• Detection = WHAT + WHERE: class labels and bounding boxes (x1,y1,x2,y2). Uses: self-driving, security, medical, retail.
• Naive sliding window: classify a window at every position & scale → millions of forward passes → too slow.
• Faster R-CNN (two-stage): Stage 1 Region Proposal Network (RPN) proposes ~2000 regions from shared backbone features; Stage 2 classifies each region + refines the box. Backbone computed once, reused.
• YOLO (single-shot): divide image into an S×S grid; each cell predicts B boxes + confidence and C class probabilities. One forward pass, real-time (30+ FPS); slightly weaker on small objects.
• IoU = area of overlap / area of union between predicted & ground-truth box. IoU > 0.5 ≈ a "correct" detection.

Both use pre-trained CNN backbones — transfer learning is the foundation. YOLO dominates industry (speed beats the last 1% accuracy); R-CNN family for high-stakes accuracy.

8 Segmentation

• Progression: Classification ("cat") → Detection (box) → Segmentation (pixel mask).
• Semantic: classify every pixel (all cats one color). Instance: distinguish individuals (cat 1 vs cat 2).
• Encoder–decoder architectures (FCN, U-Net): encoder downsamples to build semantic features; decoder upsamples back to full resolution (transposed conv or bilinear interp + conv).
• Skip connections (U-Net) pass high-res spatial detail from encoder to decoder. Without them the decoder only has the compressed bottleneck → blurry boundaries.
• Why: medical imaging / autonomous driving need exact boundaries, not rectangles.

★ Likely exam questions