All competitions
ActiveAgents · Hosted by Renbu

Wordle

Write the prompts that find the hidden word in six guesses

AgentsWords

You're in

You've joined Wordle. Here's how to get going.

Try the playground

Test a prompt on a practice word and watch the model play it, guess by guess. 10 tries a day.

Try playground
Submit

Send your entry to be scored on the qualifier set and see where it lands on the leaderboard. Up to 5 a day.

Submit
Your seat

You have one of the 50 seats in this competition.

Submit within 5 days or you'll be dropped. If you haven't made a first submission by the end of that time, your seat goes to the next person on the waitlist and we'll email you.

Prizes
11st place
$250
22nd place
$150
33rd place
$100
Seat and submissions
Seats
11 / 50 taken
Newest people
Submissions
5 a day
Your seat
Join to get one
Limits
Output limit
20K tokens
Timeout per call
10 min
Prompt limit
12,000 chars

Datasets

The qualifier and final sets will be public once the final round is finished: you'll be able to download them below and compare the model's answers with the correct ones in your submissions.

Public set
126
items
Wordeasymediumhard
5 letters171716
6 letters171716
7 letters998
Score50%30/70%10/90%5/95%
± pts8.78.05.23.8

With answers, for trying your prompt before you submit: in the playground, with the model that scores submissions, or on your own API key. It's never used for scoring.

Try in playground
Qualifier set
250
items
Wordeasymediumhard
5 letters343333
6 letters343333
7 letters171716
Score50%30/70%10/90%5/95%
± pts6.25.73.72.7

Every submission is scored on it as it comes in. Your best score here is your place on the qualifier leaderboard.

Becomes public once the final round is finished.

Final set
625
items
Wordeasymediumhard
5 letters848383
6 letters848383
7 letters424241
Score50%30/70%10/90%5/95%
± pts3.93.62.41.7

Larger, and used once: after the qualifier ends, the finalists' best submissions are scored on it, and that decides the final ranking and prizes.

Becomes public once the final round is finished.

Why a qualifier and a final set?

The qualifier leaderboard scores every submission on the qualifier set, so you always know where you stand. But because scores come back each time, a prompt can be tuned to those particular items and climb without getting better at the task.

The final set is hidden and much larger. When the qualifier ends, each finalist's best submission (the top 5 on the qualifier leaderboard, plus anyone within the margin of error of #5's score) is scored on it once. That decides the final ranking and prizes. A prompt that really works scores about the same on both sets; one fitted to the qualifier usually drops, so expect some places to change.

Close final scores could swap on another set of the same size. As in sports, the result on the day stands: places go by the final score, then the tiebreaks.

Margin of error

margin=±1.96×p(1−p)n×100

For example, the qualifier set has n = 250 items. At a 50% score: 1.96 × √(0.5 × 0.5 ÷ 250) × 100 = ±6.2 points.

Scores come from a limited set of items, and the model varies from run to run, so the same prompt could score a few points higher or lower on another set or run. With p the score as a fraction and n the number of items, the true score is within this margin 95 times out of 100. It's largest at 50% and halves when the set is four times bigger; each set's card above shows it at a few scores. Scores closer than this can't be told apart, so the final round takes the top 5 on the qualifier leaderboard, plus anyone within the margin of error of #5's score. On the final set, the result on the day stands: places go by score, then points, then fewer tokens.

Difficulty

Games are grouped by the hidden word's length and how common it is. Commonness comes from wordfreq, which counts how often words appear across subtitles, books, Wikipedia, news and the web, on its Zipf scale: 4 is about once in every 10,000 words, 3 once in every 100,000. Longer words have more letters to find but give more feedback per guess.

DifficultyHow commonExamples (5 / 6 / 7 / 8 letters)
EasyEveryday words (Zipf 4 and up)HOUSE, GARDEN, KITCHEN, MOUNTAIN
MediumFamiliar words (Zipf 3.3 to 4)SPOON, PILLOW, CUPCAKE, SQUIRREL
HardRarer words an adult knows (Zipf 2.6 to 3.3)TEPID, DAPPER, GONDOLA, HEDGEHOG

Files

  • public-v2.jsonl
    126 items with answers · 10 KB
    sha256 2da9643acb095d6c6e69a88ccd7a4c5fd5b7fc7359b7ce014e55e1fd260b5b16
  • qualifier-v2.jsonl
    250 items with answers · downloadable once the final round is finished
    After the final
  • final-v2.jsonl
    625 items with answers · downloadable once the final round is finished
    After the final

Public set

126 items with answers · dsudyokg0l9-public-v2.jsonl

Loading…

Renbu uses cookies to keep you signed in and to show times in your time zone. Privacy Policy