I packaged up the "autoresearch" project into a new self-contained minimal repo if people would like to play over the weekend. It's basically nanochat LLM training core stripped down to a single-GPU, one file version of ~630 lines of code, then: - the human iterates on the prompt (.md) - the AI agent iterates on the training code (.py) The goal is to engineer your agents to make the fastest research progress indefinitely and without any of your own involvement. In the image, every dot is a complete LLM training run that lasts exactly 5 minutes. The agent works in an autonomous loop on a git feature branch and accumulates git commits to the training script as it finds better settings (of lower validation loss by the end) of the neural network architecture, the optimizer, all the hyperparameters, etc. You can imagine comparing the research progress of different prompts, different agents, etc. https://t.co/YCvOwwjOzF Part code, part sci-fi, and a pinch of psychosis :)
Karpathy Open-Sources autoresearch for Autonomous LLM Training by AI Agents
· Updated
autoresearch, a self-contained open-source repo for autonomous LLM training research, is now on GitHub. Built on nanochat, a minimal LLM training codebase, the setup has three files:
prepare.py (fixed data utilities), train.py (the file the agent edits), and program.md (instructions the human writes). Each run lasts exactly 5 minutes with val_bpb (validation bits per byte) as the metric. The agent loops on a git feature branch, accumulating commits as it finds better architecture, optimizer, and hyperparameter settings.Rather than manually tweaking hyperparameters, an AI agent handles the full experiment loop — roughly 12 experiments per hour, up to 100 overnight. The human's job shifts to writing program.md, engineering the context that drives the fastest research progress.
Use the repo to test how different program.md strategies affect research velocity — the fixed 5-minute budget makes every run directly comparable regardless of what the agent changes.
AK
Andrej Karpathy
@karpathy
3.6kretweets28klikes
View on X
