1 What is neural style transfer?
Imagine you have a photo of your dog, and you have a famous swirly painting by Van Gogh. You hand both to a magical artist and say: "Paint my dog β but use Van Gogh's swirls and colours." The artist keeps what's in your photo (the dog, where it sits, its shape) but borrows how the painting looks (the brushstrokes, the colours, the texture). The result is your dog, painted like a Van Gogh. That's exactly what neural style transfer does β automatically, with a computer.
Neural style transfer (NST) is a technique that takes two images and blends them in a very specific way:
- A content image β the photo whose subject and layout you want to keep (your dog, a city skyline, a selfie).
- A style image β the artwork whose look and feel you want to borrow (Van Gogh's Starry Night, a watercolour, a comic-book panel).
The output is a brand-new image that has the content of the first and the style of the second. It was introduced in 2015 by Gatys, Ecker, and Bethge in a paper called "A Neural Algorithm of Artistic Style" β one of the most fun and visual results in deep learning.
The key insight: content and style can be separated
This sounds almost impossible β how do you pull apart "what's in a picture" from "how it looks"? The breakthrough idea is that a pretrained convolutional neural network (recall CNNs from our earlier vision sessions) already learned to represent images in layers, and those layers happen to separate these two things naturally:
We never train a network here. We take an already-trained image network, use it as a measuring instrument for "how much content" and "how much style" an image has, and then we slowly edit a single image until it matches the content of one photo and the style of another. The thing we optimize is the image itself, not the network.
Why this is different from a normal neural network task
In every session so far, "learning" meant adjusting the network's weights to fit data. Here it's flipped: the network's weights are frozen forever, and instead the pixels of the output image are the thing we adjust. The network just tells us how far off we are. That single mental flip is the heart of this whole project β keep it in mind.
Content = a photo of the Brandenburg Gate. Style = Van Gogh's Starry Night. The output keeps the gate's columns, arches, and overall composition perfectly recognisable, but every surface is rendered with thick swirling brushstrokes and deep blues and yellows. You can still tell exactly what building it is β it just looks hand-painted by Van Gogh. Swap in a different style image (say, a Japanese woodblock print) and the same gate comes out looking like a woodblock print instead.
What we build today is the original, optimization-based method: slow (seconds to minutes per image) but needs no training and works on any style. There's also a faster family β fast neural style transfer β where you train a small network once per style so it can repaint images instantly. We focus on the original because it teaches the core ideas most clearly; the fast version is just a speed optimization on top.
2 Content & style representations
Think of a CNN as a stack of filters that look at a picture and notice things β early filters notice tiny stuff (edges, dots, little colours), and deeper filters notice big stuff (eyes, wheels, whole shapes). To capture content ("what's in the picture and where"), we listen to a deep filter layer that knows about big shapes. To capture style ("what textures and colours, ignoring where"), we listen to several layers but only pay attention to which patterns appear together, not where they are.
We use VGG19 β a famous, simple, deep CNN trained on millions of images (ImageNet). It's just a long stack of convolutional layers (recall these from our CNN sessions: small filters sliding over the image, each producing a feature map) interleaved with pooling. Because it was trained to recognise objects, its internal feature maps are rich descriptions of images β and we mine them.
Part A β Content representation
When you push an image through VGG19, each layer outputs a stack of feature maps. Deeper
layers respond to high-level structure (the arrangement of objects and shapes) while throwing
away exact pixel detail. So to describe content, we simply record the raw feature-map values
at one deep layer (a common choice is the layer called conv4_2).
If we matched content at an early layer, the output would be forced to copy the photo almost pixel-for-pixel, leaving no room for style. A deep layer says "there's a gate-shaped thing here, columns there" without dictating the exact brushstrokes β leaving freedom for the style to take over the texture. That freedom is what makes the effect work.
Part B β Style representation: the Gram matrix
Style is trickier. We want "the textures and colours and brushstroke patterns" but explicitly not "where they are." The clever trick from the paper is the Gram matrix.
At a chosen layer, the feature maps are a stack of C channels, each a grid of numbers. Each channel is like a detector for one kind of pattern (one channel might fire on swirls, another on a certain yellow, another on diagonal strokes). The Gram matrix measures, for every pair of channels, how much they tend to fire together across the whole image β that is, the correlations between feature detectors.
Saying "the swirl detector and the yellow detector fire together a lot, everywhere" describes a texture β swirly yellow stuff β without saying where it is. Throwing away position is exactly what turns "the specific picture" into "the general look." That's why correlations between channels capture style and discard content.
Concretely: take a layer's feature maps, flatten each channel into a long vector, and stack them
into a matrix F of shape (C Γ N), where C is the number of
channels and N is the number of pixels per channel. The Gram matrix is the matrix product
G = F Β· Fᵀ, giving a (C Γ C) matrix. Entry
G[i][j] is the dot product of channel i with channel j β
their correlation. Note its size depends only on C, not on the image size β so position is
gone, exactly as we wanted.
Here's the Gram matrix computed in PyTorch. We'll reuse this function in the full program in the next section.
import torch def gram_matrix(feature): # feature has shape (batch=1, C, H, W) b, c, h, w = feature.size() # flatten each channel into one long row -> shape (C, H*W) f = feature.view(c, h * w) # G[i,j] = how much channel i and channel j fire together g = torch.mm(f, f.t()) # (C, C) # normalize so big feature maps don't dominate return g.div(c * h * w)
The normalisation by c * h * w keeps the numbers in a sane range so that
layers with bigger feature maps don't overwhelm smaller ones when we add up the style loss.
Which layers do we use?
| Purpose | VGG19 layers (typical) | Why |
|---|---|---|
| Content | conv4_2 (one deep layer) | Captures high-level layout without locking in pixels. |
| Style | conv1_1, conv2_1, conv3_1, conv4_1, conv5_1 (five layers) | Mixing shallow + deep layers captures style at many scales β fine grain to coarse shapes. |
Using several layers for style is what gives you both the fine brush texture (from shallow layers) and the larger compositional feel (from deep layers). We'll see how to weight them shortly.
Content = raw feature-map values at a deep layer (keeps "what & where"). Style = Gram matrices (channel correlations) at several layers (keeps "look", discards "where"). These two different ways of reading the same network are what let us pull content and style apart.
3 The optimization & full code
Imagine a "wrongness meter." We start with a blurry guess image. The meter tells us two things: "your picture doesn't show the dog enough" (content wrongness) and "your picture doesn't look swirly enough" (style wrongness). We nudge the picture's colours a tiny bit to make both numbers smaller, check again, nudge again β hundreds of times β until both numbers are small. Slowly, the dog appears and it becomes swirly. We're not teaching a brain; we're sculpting one picture.
Now we put it together. We turn "match the content and match the style" into numbers we can minimise. These numbers are loss functions (recall from training: a loss is a single number measuring how wrong we are, which we push down with gradient descent).
The three losses
1. Content loss. How different the output's content features are from the content image's. It's the mean-squared difference of the feature maps at the content layer:
L_content = mean( (F_output β F_content)Β² )
2. Style loss. How different the output's Gram matrices are from the style image's, summed over all the style layers:
L_style = Ξ£_layers weight Β· mean( (G_output β G_style)Β² )
3. Total variation loss. A gentle "smoothness" term that penalises neighbouring pixels being wildly different. It tames the speckled, noisy artefacts that optimization can introduce, giving a cleaner result:
L_tv = Ξ£ |pixel β right_neighbour| + |pixel β below_neighbour|
The total loss is a weighted sum, and the weights are the main dials you tune:
L_total = Ξ±Β·L_content + Ξ²Β·L_style + Ξ³Β·L_tv
Normally PyTorch computes gradients with respect to the weights. Here we set the network's
parameters to not require gradients (frozen) and instead make the output image a tensor that
requires gradients. Then loss.backward() computes "which way should
each pixel move to reduce the loss," and the optimizer nudges the pixels. The image is the
parameter being trained.
Why L-BFGS instead of Adam?
Because we're optimizing a single fixed image (not a dataset), we can use a powerful classical optimizer called L-BFGS, which converges in far fewer steps than Adam for this kind of problem. It's the optimizer used in the original paper and the official PyTorch tutorial. (Adam works too if you prefer; you'll just need more steps.)
This is the whole project in one file. It loads a content image and a style image, builds the loss
using VGG19, and optimizes the output image. Save it as style_transfer.py,
put two images next to it, and run it. (Requires torch,
torchvision, and Pillow.)
# style_transfer.py β optimization-based neural style transfer with VGG19 import torch import torch.nn as nn import torch.optim as optim import torchvision.transforms as T from torchvision.models import vgg19, VGG19_Weights from PIL import Image device = torch.device("cuda" if torch.cuda.is_available() else "cpu") IMG_SIZE = 512 if torch.cuda.is_available() else 256 # smaller on CPU for speed # ---------- 1. load & preprocess images ---------- loader = T.Compose([ T.Resize((IMG_SIZE, IMG_SIZE)), T.ToTensor(), # pixels -> [0,1], shape (C,H,W) ]) def load_image(path): img = Image.open(path).convert("RGB") return loader(img).unsqueeze(0).to(device) # add batch dim -> (1,C,H,W) content_img = load_image("content.jpg") style_img = load_image("style.jpg") # VGG was trained on ImageNet-normalized inputs; normalize the same way. cnn_mean = torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1).to(device) cnn_std = torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1).to(device) def normalize(x): return (x - cnn_mean) / cnn_std # ---------- 2. load frozen VGG19 feature extractor ---------- vgg = vgg19(weights=VGG19_Weights.DEFAULT).features.to(device).eval() for p in vgg.parameters(): p.requires_grad_(False) # freeze the network β we never train it # Map torchvision's numeric layer indices to the names from the paper. LAYER_NAMES = { 0: "conv1_1", 5: "conv2_1", 10: "conv3_1", 19: "conv4_1", 21: "conv4_2", 28: "conv5_1", } CONTENT_LAYERS = {"conv4_2"} STYLE_LAYERS = {"conv1_1", "conv2_1", "conv3_1", "conv4_1", "conv5_1"} def get_features(x): # run x through vgg, collecting the layers we care about feats = {} x = normalize(x) for i, layer in enumerate(vgg): x = layer(x) if i in LAYER_NAMES: feats[LAYER_NAMES[i]] = x return feats def gram_matrix(feature): b, c, h, w = feature.size() f = feature.view(c, h * w) return torch.mm(f, f.t()).div(c * h * w) # ---------- 3. precompute the targets (these never change) ---------- content_feats = get_features(content_img) style_feats = get_features(style_img) style_grams = {l: gram_matrix(style_feats[l]) for l in STYLE_LAYERS} # ---------- 4. the image we optimize: start from the content photo ---------- output = content_img.clone().requires_grad_(True) # THIS is our parameter optimizer = optim.LBFGS([output]) # ---------- 5. loss weights (tune these!) ---------- CONTENT_WEIGHT = 1.0 STYLE_WEIGHT = 1e6 # style needs a big weight; Grams are small numbers TV_WEIGHT = 1e-4 # gentle smoothing STYLE_LAYER_WEIGHTS = {"conv1_1": 1.0, "conv2_1": 0.8, "conv3_1": 0.5, "conv4_1": 0.3, "conv5_1": 0.1} def total_variation(x): dh = torch.abs(x[:, :, 1:, :] - x[:, :, :-1, :]).mean() dw = torch.abs(x[:, :, :, 1:] - x[:, :, :, :-1]).mean() return dh + dw # ---------- 6. optimization loop ---------- STEPS = 300 step = [0] while step[0] <= STEPS: def closure(): # keep pixels in the valid [0,1] range with torch.no_grad(): output.clamp_(0, 1) optimizer.zero_grad() feats = get_features(output) # content loss c_loss = 0.0 for l in CONTENT_LAYERS: c_loss = c_loss + nn.functional.mse_loss(feats[l], content_feats[l]) # style loss s_loss = 0.0 for l in STYLE_LAYERS: g = gram_matrix(feats[l]) layer_loss = nn.functional.mse_loss(g, style_grams[l]) s_loss = s_loss + STYLE_LAYER_WEIGHTS[l] * layer_loss # total variation (smoothness) tv_loss = total_variation(output) loss = (CONTENT_WEIGHT * c_loss + STYLE_WEIGHT * s_loss + TV_WEIGHT * tv_loss) loss.backward() step[0] += 1 if step[0] % 50 == 0: print(f"step {step[0]:4d} " f"content {c_loss.item():.4f} " f"style {s_loss.item():.6f}") return loss optimizer.step(closure) # ---------- 7. save the result ---------- with torch.no_grad(): output.clamp_(0, 1) result = output.squeeze(0).cpu() T.ToPILImage()(result).save("output.jpg") print("saved output.jpg")
Run it with python style_transfer.py. After a few hundred steps you'll
have output.jpg β your content photo repainted in the style image's look.
(1) Forgetting to freeze VGG or forgetting requires_grad_(True)
on the output means nothing changes. (2) Forgetting the ImageNet normalization gives
washed-out, wrong results β VGG expects it. (3) L-BFGS calls the closure multiple times per
step(), which is why we count steps inside the closure, not outside.
(4) Always clamp_ pixels back to [0,1] or colours
drift out of range.
4 Running, tuning & results
Think of two volume knobs: one for "show the photo" and one for "make it arty." Turn up the arty knob and your dog gets very painty β maybe so painty you can barely see the dog. Turn it down and you mostly see the dog with just a hint of paint. Tuning style transfer is mostly just turning these two knobs until it looks the way you like. There's no single "correct" setting β it's taste.
Now that the program runs, getting good-looking results is about adjusting a handful of dials. Here are the ones that matter, in order of impact.
The content / style weight trade-off
This is the single most important knob. It's the ratio STYLE_WEIGHT / CONTENT_WEIGHT
(often written Ξ±/Ξ² in the paper). It decides how much the result leans toward "faithful photo" versus
"abstract painting."
| Ratio (style Γ· content) | Result | When to use |
|---|---|---|
| Low (e.g. 1e3) | Photo dominates β subtle stylisation, very recognisable. | Portraits, when you want to keep the subject clear. |
| Medium (e.g. 1e5β1e6) | Balanced β clear subject, strong painterly texture. A good default. | Most images. Start here. |
| High (e.g. 1e7+) | Style dominates β heavy, abstract, the subject starts dissolving. | Bold artistic effects, when content can blur. |
Because Gram matrices produce small numbers, the style weight usually needs to be much larger than the content weight (often a factor of a million) just to make the two losses comparable in size. Don't be alarmed by the big number β it's normal.
Layer choices
- Content layer. Deeper (e.g.
conv4_2) = more freedom for style, looser fidelity. Shallower = tighter copy of the photo, less room for style. - Style layers. Including shallow layers (
conv1_1,conv2_1) captures fine brush texture; deep layers (conv5_1) capture larger swirling shapes. Using the full set gives the richest result. Down-weighting deep style layers (as in ourSTYLE_LAYER_WEIGHTS) keeps the texture from getting too blobby.
Number of steps & starting image
- Steps. With L-BFGS, 200β500 steps is usually plenty; the result often looks good by ~200. More steps = sharper style match but diminishing returns.
- Starting image. Starting from the content photo (as our code does) converges fast and keeps composition. Starting from random noise gives a more "from scratch painting" feel but needs more steps. Both are valid β try both.
You run with the defaults and the dog looks too abstract β you can barely see it. Fix: lower
STYLE_WEIGHT (say from 1e6 to
1e5) so content matters relatively more. Now it's recognisable but the swirls
are weak β fix: raise the shallow style-layer weights to bring back fine texture. The
output is grainy/speckled β fix: raise TV_WEIGHT a little
(e.g. to 1e-3) to smooth it. Three small adjustments and it looks great.
Common issues & quick fixes
| Symptom | Likely cause | Fix |
|---|---|---|
| Output looks like the plain photo | Style weight too low | Increase STYLE_WEIGHT. |
| Subject unrecognisable / too abstract | Style weight too high | Decrease STYLE_WEIGHT or raise content weight. |
| Grainy, speckled noise | Too little smoothing | Increase TV_WEIGHT. |
| Washed-out / strange colours | Missing ImageNet normalization, or pixels not clamped | Normalize inputs; clamp output to [0,1]. |
| Very slow | Large image on CPU | Use a smaller IMG_SIZE, or run on a GPU. |
| Out-of-memory error | Image resolution too large for the GPU | Reduce IMG_SIZE. |
Pick a style image with strong, clear texture (paintings work better than flat photos). Match the rough colour palette if you want a natural feel. Higher resolution gives crisper brushwork but costs time and memory. And remember: this is art β keep an eye on the saved image and stop when you like it, not when a metric says so.
TV_WEIGHT to fight graininess.
Most "bad" results trace back to weights, missing normalization, or unclamped pixels.
β Putting it all together
You just built a complete computer-vision project. Here's the one-paragraph story tying the four topics together:
Neural style transfer recombines the content of one image with the style of another. We use a frozen, pretrained VGG19 as a measuring tool: its deep feature maps describe content ("what & where"), while Gram matrices of its feature maps across several layers describe style ("look, not where"). We define a content loss, a style loss, and a total-variation smoothness loss, combine them with weights, and then β the crucial flip β we run gradient descent on the output image's pixels (not on the network) with L-BFGS until the picture matches both targets. Tuning is mostly about the style-to-content weight ratio and the choice of layers. In Session 18 we move from images back to sequences with text generation & translation.
Quick self-check
In style transfer, what exactly is being "trained"?
The output image's pixels, not the network. VGG19 is frozen; we make the image a tensor that requires gradients and optimize it to minimise the combined loss.
Why do we use a Gram matrix for style instead of the raw feature maps?
The Gram matrix measures correlations between channels β which patterns appear together β while throwing away where they appear. Discarding position is exactly what turns a specific picture into a general "texture/look," i.e. style.
Why use a deep layer (e.g. conv4_2) for content rather than a shallow one?
A deep layer encodes high-level layout ("a gate here, columns there") without forcing an exact pixel copy, leaving freedom for the style to repaint the textures. A shallow layer would lock the output too tightly to the original photo.
Your stylised image is so abstract you can't tell what it is. What do you change?
Lower the style weight (or raise the content weight) β the styleΓ·content ratio is too high. That brings the recognisable subject back.
Why does the style weight have to be such a huge number (e.g. 1e6)?
Gram-matrix values are small, so the raw style loss is tiny compared to the content loss. A large multiplier just rescales it so the two losses are comparable and both actually influence the result.
What does the total-variation loss do, and when do you increase it?
It penalises big differences between neighbouring pixels, smoothing the image. Increase it when the output looks grainy or speckled with high-frequency noise.
π References & Further Reading
Class material
- SST Deep Learning handout (Session 17) β your course handout for this project week, covering neural style transfer end to end.
Papers, docs & deep dives
- SST Deep Learning handout (online) β the companion online version of the class notes for this session.
- Gatys, Ecker & Bethge β "A Neural Algorithm of Artistic Style" (2015) β the original paper that introduced everything in this session; surprisingly readable.
- PyTorch Neural Style Transfer tutorial β the official, well-documented implementation our code is based on; great for going step by step.
- torchvision VGG19 docs β reference for loading the pretrained network and its layer structure.