Sudoku 101
The first-ever competition on Renbu
Live leaderboard
Each participant's best score on the public test set. The final ranking uses a separate private test set after the end.
Why a public and a private test set?
The live leaderboard scores every submission on the public test set, so you can see where you stand as you go. Because those scores come back after every submission, it's possible to tune a prompt to those particular items rather than to the task itself, and climb the live board without getting better at it.
The private test set is hidden from everyone and much larger. After the end, each participant's best submission is scored on it once, and that decides the final ranking and prizes. A prompt that really works does about as well on both; one fitted to the public items usually drops. Expect some places to change.
Ranked by score (higher first), then cell accuracy (higher first), then total tokens (fewer first), then the earlier submission. Time is shown but doesn't rank, since it depends on how busy the model provider is. Sorting by a column changes the order shown, not the ranks.
| # | Participant | Score | Submissions | |||
|---|---|---|---|---|---|---|
| 5 | @hayashi_yasuUniversity of Michigan 50.1% cell accuracy · 4,117,713 tokens · 30m 41s · 1 submission, last Oct 31 | 51.1% 45 / 88 | 50.1% | 4,117,713 | 30m 41s | 1 last Oct 31 |
| 4 |  | 70.5% 62 / 88 | 61.8% | 5,144,476 | 37m 13s | 2 last Nov 2 |
| 3 |  @cheng_linRenbu 62.8% cell accuracy · 5,233,965 tokens · 37m 46s · 2 submissions, last Nov 2 | 70.5% 62 / 88 | 62.8% | 5,233,965 | 37m 46s | 2 last Nov 2 |
| 2 |  | 72.7% 64 / 88 | 64.3% | 4,862,415 | 42m 0s | 5 last Nov 2 |
| 1 |  | 73.9% 65 / 88 | 63.7% | 5,070,117 | 37m 20s | 3 last Oct 31 |
Score: Solve rate: puzzles solved ÷ all puzzles. A puzzle the model gave no readable grid for counts as not solved. Cell accuracy: empty cells filled correctly ÷ all empty cells, over every puzzle. Tokens cover every item of the submission that counts. Time is how long scoring it took, start to finish, with several items answered at once; it's shown, not ranked.
Renbu uses cookies to keep you signed in and to show times in your time zone. No ads or tracking. Privacy Policy