Andrej Karpathy's Autonomous ML Loop: The Shift from Coding to Experimental Design

On the night of March 7, Andrej Karpathy pushed a 630-line Python script to GitHub and went to sleep. By morning, his agent had run 50 experiments, discovered a better learning rate, and committed the improvements to git—without a single human instruction in between.

While this story highlights autonomous ML research, the real breakthrough is the design pattern beneath it and the 40-line Markdown file that orchestrates the entire process.

1. The Karpathy Loop: Three Core Primitives

Karpathy's AutoResearch automates the tedious cycle of tweaking hyperparameters, running models, reading metrics, and deciding whether to keep changes. What makes this loop generalizable beyond ML training are three strict constraints:

1. •Editable Asset: The agent is only allowed to modify a single file. This confines the search space and keeps every hypothesis reviewable as a simple git diff.
2. •Scalar Metric: A single number (like validation bits per byte) determines improvement. It must be computable without human judgment and entirely unambiguous.
3. •Time-Boxed Cycle: A fixed duration makes every experiment directly comparable, whether the agent changed the model size or the learning rate.

These primitives ensure that Goodhart's Law doesn't derail the agent. The loop is minimal by design, optimized for relentless efficiency.

2. Markdown as the Human-Agent Interface

The unsung hero of the AutoResearch repository isn't the Python training script; it's program.md.

This single Markdown document simultaneously serves as instructions, constraints, and stopping criteria. No other format achieves this balance. YAML encodes structure but lacks reasoning; Python is executable but hard to read as a strategy; JSON has no narrative. Markdown sits at the perfect intersection of human editability and agent parseability.