StyleGAN
Most generators stir $z$ into every layer at the same time. StyleGAN split it apart: first map $z$ to an intermediate $w$, then inject $w$ at every resolution level through AdaIN. That single change untangled the latent space and made photorealistic faces a solved problem for seven years running.
The Problem
A DCGAN maps $z$ to an image through a stack of transposed convolutions. The problem: $z$ controls everything, pose, lighting, identity, background, entangled together. Move along one axis of $z$, all four change. You cannot ask the model "same person, different pose" because the representation does not factor that way.
Karras et al. (2019, NVIDIA) proposed: stop feeding $z$ directly into conv layers. Feed a constant 4×4×512 tensor as the network input. Learn an 8-layer MLP that maps $z \in Z \to w \in W$. Inject $w$ at every resolution via adaptive instance normalization (AdaIN): normalize each conv feature map, then scale and shift by affine projections of $w$. Add per-layer noise for stochastic detail (skin pores, hair strands).
The result: $W$ has roughly orthogonal axes for "high-level style" (pose, identity) vs "fine style" (lighting, color). You can swap styles between two images by using image A's $w$ for the low-resolution levels and image B's $w$ for the high. This unlocked editing, cross-domain stylization, and the entire "StyleGAN-inversion" line of research.
The Concept
Mapping network. $f: Z \to W$, an 8-layer MLP. $Z = \mathcal{N}(0, I)^{512}$. $W$ is not forced to be Gaussian, it learns a data-adapted shape.
Synthesis network. Starts from a learned constant 4×4×512. Each resolution block: upsample → conv → AdaIN(w_i) → noise → conv → AdaIN(w_i) → noise. Resolutions double: 4, 8, 16, 32, 64, 128, 256, 512, 1024.
AdaIN.
where $y_{\text{scale}}$ and $y_{\text{bias}}$ come from affine projections of $w$. Normalize per feature map, then restyle. "Style" here is the first- and second-order statistics of the feature map.
Per-layer noise. Single-channel Gaussian noise added to each feature map, scaled by a learned per-channel factor. Controls stochastic detail without affecting global structure.
Truncation trick. At inference, sample $z$, compute $w = \text{mapping}(z)$, then $w' = \hat{w} + \psi \cdot (w - \hat{w})$ where $\hat{w}$ is the mean $w$ over many samples. $\psi < 1$ trades diversity for quality. Almost every StyleGAN demo uses $\psi \approx 0.7$.
StyleGAN 1 → 2 → 3
| Version | Year | Innovation |
|---|---|---|
| StyleGAN | 2019 | Mapping network + AdaIN + noise + progressive growing. |
| StyleGAN2 | 2020 | Weight demodulation replaces AdaIN (fixes droplet artifacts); skip/residual architecture; path-length regularization. |
| StyleGAN3 | 2021 | Alias-free convolution + equivariant kernels; eliminates texture sticking to pixel grid. |
| StyleGAN-XL | 2022 | Class-conditional, 1024², ImageNet. |
| R3GAN | 2024 | Rebrands with stronger reg; closes gap to diffusion on FFHQ-1024 with 20x fewer params. |
In 2026 StyleGAN3 remains the default for (a) narrow-domain photorealism at high FPS, (b) few-shot domain adaptation (train on a new dataset with 100 images, freeze mapping), (c) inversion-based editing (find the $w$ that reconstructs a real photo, then edit that $w$). For open-domain text-to-image, it is not the tool, diffusion is.
Build It
code/main.py implements a toy "style-GAN lite" in 1-D: a mapping MLP, a synthesis function that takes a learned constant vector and modulates it with $w$-derived scale/bias, and per-layer noise. It shows that injecting $w$ via affine-modulation matches or beats concatenating $z$ into the generator's input.
Step 1: mapping network
def mapping(z, M):
h = z
for i in range(num_layers):
h = leaky_relu(add(matmul(M[f"W{i}"], h), M[f"b{i}"]))
return hStep 2: adaptive instance normalization
def adain(x, w_scale, w_bias):
mu = mean(x)
sd = std(x)
x_norm = [(xi - mu) / (sd + 1e-8) for xi in x]
return [w_scale * xi + w_bias for xi in x_norm]Per-feature-map scale and bias come from $w$ via linear projection.
Step 3: per-layer noise
def add_noise(x, sigma, rng):
return [xi + sigma * rng.gauss(0, 1) for xi in x]Sigma per-channel is learnable.
Pitfalls
- Droplet artifacts. StyleGAN 1 produced a blobby droplet in the feature maps because AdaIN zeroed out mean. StyleGAN 2's weight demodulation fixes it by scaling the convolution weights instead.
- Texture sticking. StyleGAN 1 and 2 textures followed pixel coordinates, not object coordinates (visible when interpolating). StyleGAN 3's alias-free convolutions fix this with windowed sinc filters.
- Mode coverage. Truncation
ψ < 0.7looks clean but samples from a narrow cone; useψ = 1.0if you need diversity. - Inversion is lossy. Inverting a real photo into $W$ is usually done through optimization or an encoder (e4e, ReStyle, HyperStyle). Results drift over many iterations.
Use It
| Use case | Approach |
|---|---|
| Photoreal human faces (anime, product, narrow) | StyleGAN3 FFHQ / custom fine-tune |
| Face editing from a photo | e4e inversion + StyleSpace / InterFaceGAN directions |
| Face swap / reenactment | StyleGAN + encoder + blending |
| Avatar pipelines | StyleGAN3 w/ ADA for low-data fine-tune |
| Domain adaptation from a few images | Freeze mapping network, fine-tune synthesis |
| Multi-modal or text-conditioned generation | Don't, use diffusion |
For product-grade demos where the answer is "photo of a person's face", StyleGAN beats diffusion on inference cost (single forward pass, <10ms on a 4090) and sharpness for the same quality bar.
Ship It
Save outputs/skill-stylegan-inversion.md. Skill takes a real photo and outputs: inversion method (e4e / ReStyle / HyperStyle), expected latent loss, editing budget (how far in $W$ you can move before artifacts), and a list of known-good editing directions (age, expression, pose).
Exercises
- Easy. Run
code/main.pywithadain_on=Trueandadain_on=False. Compare the spread of outputs for a fixed latent vs perturbed latent. - Medium. Implement mixing regularization: for a training batch, compute $w_a$, $w_b$, and apply $w_a$ for the first half of synthesis and $w_b$ for the second half. Does the decoder learn disentangled styles?
- Hard. Take a pretrained StyleGAN3 FFHQ model (ffhq-1024.pkl). Find the $w$ direction that controls "smile" by training an SVM on labelled samples; report how far you can push before identity drifts.
Key Terms
| Term | What people say | What it actually means |
|---|---|---|
| Mapping network | "The MLP" | $f: Z \to W$, 8 layers, decouples latent geometry from data statistics. |
| W space | "The style space" | Output of the mapping network; roughly disentangled. |
| AdaIN | "Adaptive instance norm" | Normalize feature map, then scale + shift by $w$-projection. |
| Truncation trick | "Psi" | $w = \text{mean} + \psi \cdot (w - \text{mean})$, $\psi<1$ trades diversity for quality. |
| Path-length regularization | "PL reg" | Penalizes large changes in image per unit change in $w$; makes $W$ smoother. |
| Weight demodulation | "The StyleGAN2 fix" | Normalize conv weights instead of activations; kills droplet artifacts. |
| Alias-free | "StyleGAN3's trick" | Windowed sinc filters; eliminates texture sticking to the pixel grid. |
| Inversion | "Find w for a real image" | Optimize or encode $x \to w$ so $G(w) \approx x$. |
Production note: why StyleGAN still ships in 2026
StyleGAN3 on a 4090 generates a 1024² FFHQ face in under 10 ms, num_steps = 1no VAE decode, no cross-attention pass. In production terms this is the floor latency for any image generator. A 50-step SDXL + VAE-decode pipeline at the same resolution is ~3 seconds. That is a 300× gapand for narrow-domain products (avatar services, ID document pipelines, stock face generation) it wins on TCO.
Two operational consequences:
- No scheduler, no batcher. Static batch at the target occupancy is optimal. Continuous batching (essential for LLMs and diffusion) provides zero benefit because every request takes the same FLOPs.
- Truncation
ψis the safety knob.ψ < 0.7samples from a narrow cone of the mapping network's range. This is the only lever the serving layer has over sample variance. Lowerψat peak load, raise it for premium users.
Further Reading
- Karras et al. (2019). A Style-Based Generator Architecture for GANsStyleGAN.
- Karras et al. (2020). Analyzing and Improving the Image Quality of StyleGANStyleGAN2.
- Karras et al. (2021). Alias-Free Generative Adversarial NetworksStyleGAN3.
- Tov et al. (2021). Designing an Encoder for StyleGAN Image Manipulatione4e inversion.
- Sauer et al. (2022). StyleGAN-XL: Scaling StyleGAN to Large Diverse DatasetsStyleGAN-XL.
- Huang et al. (2024). R3GAN: The GAN is dead; long live the GAN!modern minimal GAN recipe.
The full code
Every file for this lesson. Copy it, run it, then change it.
import math
import random
def leaky(x, a=0.2):
return x if x > 0 else a * x
def randn_matrix(rows, cols, rng, scale=0.3):
return [[rng.gauss(0, scale) for _ in range(cols)] for _ in range(rows)]
def matmul(W, x):
return [sum(w * xi for w, xi in zip(row, x)) for row in W]
def add(a, b):
return [x + y for x, y in zip(a, b)]
def mean_std(xs):
m = sum(xs) / len(xs)
v = sum((x - m) ** 2 for x in xs) / len(xs)
return m, math.sqrt(v + 1e-8)
def adain(features, scale, bias):
m, s = mean_std(features)
return [scale * (f - m) / s + bias for f in features]
def mapping(z, layers):
h = z
for W, b in layers:
pre = add(matmul(W, h), b)
h = [leaky(x) for x in pre]
return h
def init_mapping(z_dim, w_dim, depth, rng):
layers = []
dims = [z_dim] + [w_dim] * depth
for i in range(depth):
layers.append((randn_matrix(dims[i + 1], dims[i], rng), [0.0] * dims[i + 1]))
return layers
def stylegan_forward(w, const, synth, noise_sigma, rng, adain_on=True):
"""Very small 'synthesis' network: three resolution blocks on a 4-channel constant."""
h = list(const)
for i in range(3):
W = synth[f"W{i}"]
b = synth[f"b{i}"]
pre = add(matmul(W, h), b)
h = [leaky(x) for x in pre]
if adain_on:
scale = sum(synth[f"scale{i}"][j] * w[j] for j in range(len(w)))
bias = sum(synth[f"bias{i}"][j] * w[j] for j in range(len(w)))
h = adain(h, scale, bias)
if noise_sigma > 0:
h = [x + noise_sigma * rng.gauss(0, 1) for x in h]
return h
def init_synth(hidden, w_dim, rng):
synth = {}
for i in range(3):
synth[f"W{i}"] = randn_matrix(hidden, hidden, rng)
synth[f"b{i}"] = [0.0] * hidden
synth[f"scale{i}"] = [rng.gauss(0, 0.3) for _ in range(w_dim)]
synth[f"bias{i}"] = [rng.gauss(0, 0.3) for _ in range(w_dim)]
return synth
def main():
rng = random.Random(3)
z_dim, w_dim, hidden = 8, 8, 6
mapping_net = init_mapping(z_dim, w_dim, depth=4, rng=rng)
synth = init_synth(hidden, w_dim, rng)
const = [rng.gauss(0, 0.3) for _ in range(hidden)]
print("=== compare: style inputs via AdaIN vs no AdaIN ===")
print("sample 5 random z, look at std of output under each mode")
for mode in [True, False]:
outs = []
for _ in range(5):
z = [rng.gauss(0, 1) for _ in range(z_dim)]
w = mapping(z, mapping_net)
h = stylegan_forward(w, const, synth, 0.0, rng, adain_on=mode)
outs.append(h)
flat = [v for row in outs for v in row]
m, s = mean_std(flat)
label = "with AdaIN" if mode else "no AdaIN "
print(f" {label}: mean {m:+.3f} std {s:.3f}")
print()
print("=== truncation trick: sample many w, take mean, interpolate ===")
ws = []
for _ in range(200):
z = [rng.gauss(0, 1) for _ in range(z_dim)]
ws.append(mapping(z, mapping_net))
w_bar = [sum(w[i] for w in ws) / len(ws) for i in range(w_dim)]
z_test = [rng.gauss(0, 1) for _ in range(z_dim)]
w_test = mapping(z_test, mapping_net)
for psi in [0.0, 0.5, 0.7, 1.0]:
w_psi = [w_bar[i] + psi * (w_test[i] - w_bar[i]) for i in range(w_dim)]
h = stylegan_forward(w_psi, const, synth, 0.0, rng, adain_on=True)
print(f" psi={psi:.1f}: output = {[f'{v:+.2f}' for v in h]}")
print()
print("=== per-layer noise injection (pose fixed, stochastic detail changes) ===")
z_fixed = [rng.gauss(0, 1) for _ in range(z_dim)]
w_fixed = mapping(z_fixed, mapping_net)
for seed in range(3):
rng_local = random.Random(seed)
h = stylegan_forward(w_fixed, const, synth, 0.1, rng_local, adain_on=True)
print(f" seed {seed}: {[f'{v:+.2f}' for v in h]}")
print()
print("notice: with the same w, outputs vary slightly with noise seed.")
print(" that is the stochastic-detail vs global-style split.")
if __name__ == "__main__":
main()