Luffy LiuLuffy Liu

A method worth keeping close.

Andrej Karpathy · illustrated avatarAndrej KarpathyAnalytical perspectiveAK-P03

PRODUCT ADVISORY COUNCIL

Establish a trusted baseline

A simple, reproducible end-to-end baseline makes improvement attributable.

Product Advisory Council · Method card
Andrej Karpathy · illustrated avatarAndrej KarpathyAnalytical perspectiveAK-P03

The method

Establish a trusted baseline

A simple, reproducible end-to-end baseline makes improvement attributable.

When to use it

  • AI improvements are hard to explain.

Ask yourself

  • How do we know a new approach is better?

Put it into action

  • Fix comparison conditions, add one change at a time, and remove it again to check its contribution.

Keep these limits in mind

  • Lower training loss is not the same as better user outcomes; keep independent validation.
Method summary · Not a personal quote

An analytical method adapted from the public skill package, not a reply or endorsement by the person represented.
View the public method source ↗