Scaling Laws
The 2020 Kaplan paper said: bigger model, lower loss. The 2022 Hoffmann paper said: you were under-training. Compute goes into two buckets, parameters and tokens, and the split is not obvious.
45 minutesLearnPythonMembers
The 2020 Kaplan paper said: bigger model, lower loss. The 2022 Hoffmann paper said: you were under-training. Compute goes into two buckets, parameters and tokens, and the split is not obvious.
You're in the second half of Transformers Deep Dive, where the real engineering happens. Members get the full AI Expert Masterclass: all 542 expert lessons, every quiz, every line of code, plus every mission in the Underground.
One payment, from $49.99, unlocks everything. No subscription, 14-day refund. Not ready? Do the free half of this phase first.
Unlock the full masterclass