Refinery accelerates open-source AI through a decentralized search for better training algorithms. We begin with the optimizer: the rule that turns gradients into learning.

Accelerating open LLMs through incentivized research.

Live · Bittensor subnet 125

The open lab for AI training. Researchers compete, results are verified, everything is public.

Every lab tunes the same pretraining recipe. Refinery picks the parts of it nobody has explored, defines a measurement that can’t be gamed, and puts a bounty on it. Independent researchers attack it around the clock; every result is public and independently checkable; the winner becomes the baseline for the next challenger.

How the search works
Global
A decentralized pool of researchers
20 h
Equal evaluation budget
2 runs
Confirmation before reward
Shared
Discoveries others can build on
Open challenges

Two open challenges.

Each challenge isolates one part of training, fixes everything else, and pays only for confirmed improvement.

Challenge 1 · Optimizer

Can anyone beat the standard way AI models are trained?

Yes. Independent researchers beat AdamW, the industry default. Same model, same data, same compute.

See the leaderboard
Since September 2026
Challenge 2 · Refiner New

What should a model take from a document?

What to keep, what to weight, in what order, and what to predict. A few early studies say it matters. Nobody has searched it systematically.

See the leaderboard
Since October 2026
The opportunity

Performance starts with algorithms and data.

AdamW and Muon are milestones, not the endpoint. A wider search, accurate evaluation, and rewards for real improvements could move optimizer development substantially further.

Model capability has advanced dramatically. We believe there is still substantial headroom in the lower-level operations used to produce it. A better update rule can improve what a training run achieves with the compute it already has.

The optimizer is part of what makes that capability attainable. From stochastic gradient descent and momentum to adaptive methods such as Adam and AdamW, improvements to the update rule have helped shape modern training. In language-model experiments, optimizer choice can substantially change the loss reached within a fixed training budget—not just how quickly the same result arrives.

Published experiments: Kempner Institute’s LLM optimizer comparisons ↗

Useful algorithmic advances can come from academic research, open-source experiments, and independent contributors as well as frontier labs. Refinery makes this work an ongoing, incentivized search: miners propose improvements, a common evaluation measures them, and verified progress earns rewards.

The aim is stronger open LLMs: advances that compound across the training process and give open-source AI a stronger foundation. Better algorithms should become building blocks for everyone.

Read Refinery’s research thesis
The vision

Change the trajectory of open models.

The remaining gap with closed systems is not only data. It is also the algorithms that decide how much a given budget of compute and data can produce.

Refinery exists to move the trajectory of open-source language models. Capability has advanced quickly, but we do not think the distance that remains is explained by data alone. We see substantial headroom in the algorithms, optimizers, kernels, the model architecture itself, and how inputs are processed. This includes components that the field has treated as settled for years.

Some of that headroom is already public. Looped transformers are a small, open example of an older idea still being pushed. FlashAttention and Muon are larger ones: methods that produced large gains in speed or training efficacy once they reached the open community. We believe there are more improvements of that kind, and that a wider, incentivized search can find them.

We start with the optimizer. From launch, for at least six weeks, miners are rewarded for update rules that beat AdamW and Muon on small models under a fixed budget. That is the first target, not the last.

We think this is a consequential moment. Whether comparable capability exists in the open will shape who can use it. Our aim is to use Bittensor to point a global pool of researchers at discoveries the open research community has not had in years, and to put those discoveries in everyone’s hands.

Why this matters for Bittensor

Open research.

Refinery’s ambition extends beyond a better optimizer: advance open-source AI, demonstrate what Bittensor’s global talent pool can produce, and bring more researchers into the network.

01Open LLM research

Advance research everyone can use.

Better training algorithms should empower the open-source world. By sharing methods, code and evaluation results, we aim to give researchers and builders stronger foundations for the next generation of open AI.

02A public record

Make progress inspectable.

Use the network’s global talent pool to produce work others can check. Shared methods, code and evaluation results give the field a common record to build on.

03Research talent

Bring researchers into Bittensor.

Make those results an invitation. Give talented researchers a concrete route into mining: contribute an optimizer, test it against a shared benchmark, and earn rewards for confirmed progress while advancing open research.

The aim is a reinforcing cycle: Bittensor enables useful research; that work is published; and more researchers join as miners to push it further.

Explore contributing as a miner ↗
Why start with gradient descent?

A focused target. A wide search space.

Our view is that optimizer ideas are more practical to compare in smaller training runs than changes to the model architecture itself or the kernels that execute it. Keeping the model and data fixed lets us ask a precise question: does this update rule train better within the same budget?

Search

Explore beyond the defaults.

Miners can refine established methods or propose a different update rule. Human insight, automated experiments, and LLM research agents can all contribute to the search.

Measure

Standardize the environment.

Compare update rules under the same model, data, and compute budget. A shared evaluation environment makes results comparable, and a separate confirmation run checks that improvements hold up.

Coordinate

Direct a global pool of researchers.

Bittensor aligns a global pool of researchers around a shared target. Confirmed improvements earn rewards, while openly shared methods and evaluation records let everyone build on the work.

Research at network scale

Global talent. Shared progress.

Better training algorithms are already worth pursuing. Frontier model capabilities expand what researchers can explore through automated research. Refinery uses Bittensor to direct that global talent at specific problems, with common tests, rewards for progress, and discoveries shared openly.

Explore Refinery’s research approach
Bittensor subnet 125Live nowRefinery is live. The dashboard shows current evaluations and published round results.Visit the dashboard →
The longer view

Compounded advances across the set of algorithms behind a training run.

Descent algorithms are our starting point. Pretraining data quality and reinforcement-learning environments are other components where a wider pool of contributors could make useful advances.

When we compare models from OpenAI with open-weight families such as Qwen and Kimi, we see the outcome of many interacting choices across training. Refinery’s ambition is to help close capability and efficiency gaps by improving those components, one measurable target at a time.

Explore the broader direction ↗
Research cadence

Test the idea against results.

Emissions are available from day one, earned by verified progress. After six weeks, we will review the results and discuss whether a research pause would help the next stage. No pause is scheduled automatically.

Read schedule