
Can anyone beat the standard way AI models are trained?
Yes. Independent researchers beat AdamW, the industry default, by perplexity. Same model, same data, same compute.
See the leaderboard
Refinery accelerates open-source AI through a decentralized search for better training algorithms. We begin with the optimizer: the rule that turns gradients into learning.
The open lab for AI training. Researchers compete, results are verified, everything is public.
Every lab tunes the same pretraining recipe. Refinery picks the parts of it nobody has explored, defines a measurement that can’t be gamed, and puts a bounty on it. Independent researchers attack it around the clock; every result is public and independently checkable; the winner becomes the baseline for the next challenger.
How the search worksEach challenge isolates one part of training, fixes everything else, and pays only for confirmed improvement.

Yes. Independent researchers beat AdamW, the industry default, by perplexity. Same model, same data, same compute.
See the leaderboard
What to keep, what to weight, in what order, and what to predict. A few early studies say it matters. Nobody has searched it systematically.
First confirmed result: suite points over the baseline.
See the leaderboardLower perplexity than AdamW, the industry default. Same model, data and compute.
Held-out perplexity of the confirmed Challenge 1 frontier against AdamW-exact, a miner-submitted full-precision AdamW and the strongest AdamW run on record. Both are read live from the published validator record.
AdamW and Muon are milestones, not the endpoint. A wider search, accurate evaluation, and rewards for real improvements could move optimizer development substantially further.
Model capability has advanced dramatically. We believe there is still substantial headroom in the lower-level operations used to produce it. A better update rule can improve what a training run achieves with the compute it already has.
The optimizer is part of what makes that capability attainable. From stochastic gradient descent and momentum to adaptive methods such as Adam and AdamW, improvements to the update rule have helped shape modern training. In language-model experiments, optimizer choice can substantially change the loss reached within a fixed training budget—not just how quickly the same result arrives.
Published experiments: Kempner Institute’s LLM optimizer comparisons ↗
Useful algorithmic advances can come from academic research, open-source experiments, and independent contributors as well as frontier labs. Refinery makes this work an ongoing, incentivized search: miners propose improvements, a common evaluation measures them, and verified progress earns rewards.
The aim is stronger open LLMs: advances that compound across the training process and give open-source AI a stronger foundation. Better algorithms should become building blocks for everyone.
Read Refinery’s research thesisThe remaining gap with closed systems is not only data. It is also the algorithms that decide how much a given budget of compute and data can produce.
Refinery exists to move the trajectory of open-source language models. Capability has advanced quickly, but we do not think the distance that remains is explained by data alone. We see substantial headroom in the algorithms, optimizers, kernels, the model architecture itself, and how inputs are processed. This includes components that the field has treated as settled for years.
Some of that headroom is already public. Looped transformers are a small, open example of an older idea still being pushed. FlashAttention and Muon are larger ones: methods that produced large gains in speed or training efficacy once they reached the open community. We believe there are more improvements of that kind, and that a wider, incentivized search can find them.
We start with the optimizer. From launch, for at least six weeks, miners are rewarded for update rules that beat AdamW and Muon on small models under a fixed budget. That is the first target, not the last.
We think this is a consequential moment. Whether comparable capability exists in the open will shape who can use it. Our aim is to use Bittensor to point a global pool of researchers at discoveries the open research community has not had in years, and to put those discoveries in everyone’s hands.
Refinery’s ambition extends beyond a better optimizer: advance open-source AI, demonstrate what Bittensor’s global talent pool can produce, and bring more researchers into the network.
Better training algorithms should empower the open-source world. By sharing methods, code and evaluation results, we aim to give researchers and builders stronger foundations for the next generation of open AI.
Use the network’s global talent pool to produce work others can check. Shared methods, code and evaluation results give the field a common record to build on.
Make those results an invitation. Give talented researchers a concrete route into mining: contribute an optimizer, test it against a shared benchmark, and earn rewards for confirmed progress while advancing open research.
The aim is a reinforcing cycle: Bittensor enables useful research; that work is published; and more researchers join as miners to push it further.
Explore contributing as a miner ↗Our view is that optimizer ideas are more practical to compare in smaller training runs than changes to the model architecture itself or the kernels that execute it. Keeping the model and data fixed lets us ask a precise question: does this update rule train better within the same budget?
Miners can refine established methods or propose a different update rule. Human insight, automated experiments, and LLM research agents can all contribute to the search.
Compare update rules under the same model, data, and compute budget. A shared evaluation environment makes results comparable, and a separate confirmation run checks that improvements hold up.
Bittensor aligns a global pool of researchers around a shared target. Confirmed improvements earn rewards, while openly shared methods and evaluation records let everyone build on the work.
Better training algorithms are already worth pursuing. Frontier model capabilities expand what researchers can explore through automated research. Refinery uses Bittensor to direct that global talent at specific problems, with common tests, rewards for progress, and discoveries shared openly.
Explore Refinery’s research approachDescent algorithms are our starting point. Pretraining data quality and reinforcement-learning environments are other components where a wider pool of contributors could make useful advances.
When we compare models from OpenAI with open-weight families such as Qwen and Kimi, we see the outcome of many interacting choices across training. Refinery’s ambition is to help close capability and efficiency gaps by improving those components, one measurable target at a time.
Explore the broader direction ↗Emissions are available from day one, earned by verified progress. After six weeks, we will review the results and discuss whether a research pause would help the next stage. No pause is scheduled automatically.
Read schedule