How we rank
AI models.
The arena ranks image and video models by direct comparison of their outputs rather than by benchmarks or vendor claims. People vote on blind pairwise matchups, and a TrueSkill rating model turns those votes into an ordered leaderboard.
The process
Three stages turn a single prompt into a leaderboard position.
One prompt, every model
Each challenge is a single prompt, sent unchanged to every model in the challenge. Outputs differ only by the model that produced them.
Blind comparison
Voters see two outputs side by side, with no model names or labels, and pick the one they prefer or mark a tie. Hiding identity keeps the vote about the output.
Ratings update
Each vote adjusts both models' ratings. Beating a higher-rated model raises a rating more than beating a lower-rated one. After enough matchups, ratings settle into a stable order.
The rating
A single number for each model, plus a measure of how confident that number is.
The leaderboard shows each model one rating, presented as an Elo score. As in chess, a higher number indicates stronger measured performance: a top player rates around 2800, a beginner around 400. The arena uses the same scale, with one addition.
We label the number Elo because it is the familiar term from chess, games, and other model leaderboards such as Chatbot Arena and Artificial Analysis. The underlying rating system is TrueSkill, developed by Microsoft Research. Like Elo, it expresses skill as a single number, but it also tracks how uncertain that estimate is. That uncertainty is the key difference.
Each rating combines two values
Mu (μ) · the estimate
The current best estimate of a model's true skill, based on its results so far.
Sigma (σ) · the uncertainty
The confidence in that estimate. A high sigma means few votes and low confidence; a low sigma means it has settled.
The formula
The rating shown on the leaderboard is not the raw skill estimate. It is a conservative lower bound on that estimate, scaled to a familiar range.
Elo = 1000 + 10 × ( μ − 3σ )
The inner term, μ minus three sigma, is a lower bound on a model's estimated skill. New models start at 1000. Consistent wins move the rating into the 1100 to 1300 range. Because the bound only rises as uncertainty falls, a short run of wins cannot inflate it.
Rating updates
The size of each update depends on how surprising the result was, and on how settled the ratings already are.
Upset win
A lower-rated model beats a higher-rated one. The result is unexpected, so the winner's rating rises sharply and the loser's falls. These matchups move ratings the most.
Expected win
A higher-rated model beats a lower-rated one. The result matches the prediction, so both ratings change only slightly.
Tie
A tie between similarly rated models changes little. A tie against a much higher-rated model raises the lower-rated one, because the draw was better than expected.
New model
A new model has high uncertainty, so its early votes move its rating by a large amount. As it accumulates matchups, uncertainty falls and each vote moves it less.
A model must complete at least 4 matchups before it appears on any leaderboard, so a single result cannot place it.
Ranking scopes
Every vote updates ratings at three levels at once. A model can rank first overall and third within a category, because different prompts reward different strengths.
Overall ranking
Every vote within a mode, such as Text-to-Image or Image Editing, contributes to one rating per model. This is the default leaderboard.
Category ranking
Categories such as Photorealism group related challenges. A model can rank well in one category and poorly in another.
Challenge ranking
A single challenge is one prompt sent to every model, showing how each model handled that specific prompt.
Fairness
Several controls keep each vote about the output rather than the model that produced it.
Blind voting
Model names are hidden during voting and shown only after a choice is made, so brand preference cannot influence the result.
Randomized sides
Left and right placement is randomized each matchup, removing any bias toward a screen position.
Identical prompts
Within a challenge, every model receives the same prompt, so no model is favored by a tailored input.
Exploration-weighted pairing
Matchmaking favors models with fewer votes and higher uncertainty, so newer models are evaluated quickly rather than repeating popular pairings.
Conservative scoring
Displaying mu minus three sigma requires a model to support its rating with consistent results before it rises.
Parameters
The full set of TrueSkill and normalization values the system runs on.
| Parameter | Value | What it means |
|---|---|---|
| μ0 | 25.0 | Initial skill estimate for a new model. After normalization this corresponds to Elo 1000. |
| σ0 | 8.333 | Initial uncertainty. Set high because a new model has no recorded results yet. |
| β | 4.167 | Performance variance, accounting for run-to-run variation in output quality. |
| τ | 0.083 | Dynamics factor. Prevents sigma from reaching zero, so ratings stay responsive to genuine changes over time. |
| Draw probability | 10% | Expected rate of ties, calibrated from observed voting. |
| Conservative multiplier | 3 | Standard deviations subtracted from mu. Three corresponds to about 99.87% confidence that true skill is at least the displayed value. |
| Minimum matchups | 4 | Matchups a model must complete before it appears on leaderboards. |
| Elo normalization | 1000 + 10× | Offset and scale applied to the conservative score (μ−3σ) to produce the displayed Elo number. |
| Algorithm | TrueSkill | Developed by Microsoft Research. Read the paper |
The leaderboard reflects the votes behind it.
Vote on a few blind matchups to help refine the rankings.