GPT-5 Models with Reasoning

🧩 Testing Models with Puzzles

These puzzles aren't just for fun — they're a chance to get hands-on with GPT-5 models and see firsthand how same models think, reason, and arrive at different outputs for the exact same question.

⚖️ Inference-Time vs Training-Time Scaling

Before the puzzles, a quick reminder of two ways to make a model "smarter":

Both work independently — and today's puzzles demonstrate this beautifully.

🪙 Puzzle 1: The Two Coins (Easy)

The Question

▶️ Create a new code cell and run it: ( Level2 → part1.ipynb )

easy_puzzle = [
    {"role": "user", "content": 
        "You toss 2 coins. One of them is heads. What's the probability the other is tails? Answer with the probability only."},
]

The Trick

We didn't say which coin was heads — just that one of them is. This changes the sample space entirely.

Possible outcomes when at least one is heads

• Heads, Heads

• Heads, Tails

• Tails, Heads

▶️ Create a new code cell and run it: ( model: gpt-5-nano , reasoning: minimal )

response = openai.chat.completions.create(model="gpt-5-nano", messages=easy_puzzle, reasoning_effort="minimal")
display(Markdown(response.choices[0].message.content))

✅ Output : 1/3

▶️ Create a new code cell and run it: ( model: gpt-5-nano , reasoning: low )

response = openai.chat.completions.create(model="gpt-5-nano", messages=easy_puzzle, reasoning_effort="low")
display(Markdown(response.choices[0].message.content))

✅ Output : 2/3

▶️ Create a new code cell and run it: ( model: gpt-5-mini , reasoning: minimal )

response = openai.chat.completions.create(model="gpt-5-mini", messages=easy_puzzle, reasoning_effort="minimal")
display(Markdown(response.choices[0].message.content))

✅ Output : 2/3

In 2 out of 3 cases, the other coin is tails. So the answer is 2/3, not 1/2.

📊 Model Results

🔍 What This Shows

Small model + no reasoning   →  Wrong
Small model + more reasoning →  Right
Bigger model + no reasoning  →  Right

You can reach the correct answer by scaling either the model size or the reasoning effort — they're independent levers.

📚 Puzzle 2: The Bookshelf Worm (Hard)

The Question

▶️ Create a new code cell and run it: ( Level2 → part1.ipynb )

hard = """
On a bookshelf, two volumes of Pushkin stand side by side: the first and the second.
The pages of each volume together have a thickness of 2 cm, and each cover is 2 mm thick.
A worm chewed/ate (perpendicular to the pages) from the first page of the first volume to the last page of the second volume.
What distance did it chewed/ate through?
"""
hard_puzzle = [
    {"role": "user", "content": hard}
]

Testing Different Models

▶️ Create a new code cell and run it: ( model: gpt-5-nano , reasoning: minimal )

response = openai.chat.completions.create(model="gpt-5-nano", messages=hard_puzzle, reasoning_effort="minimal")
display(Markdown(response.choices[0].message.content))

✅ Output : 4.4 cm

▶️ Create a new code cell and run it: ( model: claude-sonnet-4 , reasoning: N/A )

response = anthropic.chat.completions.create(model="claude-sonnet-4-6", messages=hard_puzzle)
display(Markdown(response.choices[0].message.content))

✅ Output : 4 mm

▶️ Create a new code cell and run it: ( model: gpt-5 , reasoning: N/A )

response = openai.chat.completions.create(model="gpt-5", messages=hard_puzzle)
display(Markdown(response.choices[0].message.content))

✅ Output : 4 mm

▶️ Create a new code cell and run it: ( model: gemini-2.5-pro , reasoning: N/A )

response = gemini.chat.completions.create(model="gemini-2.5-pro", messages=hard_puzzle)
display(Markdown(response.choices[0].message.content))

✅ Output : 4 mm

✅ Results & Takeaway

The correct answer is 4 mm.

Almost every frontier model got this right — the only exception was GPT-5 Nano, which is expected given its smaller size.

🔍 What This Tells Us

💡 Today's leading frontier models have become remarkably capable at complex reasoning tasks. But the moment you step down to an older or smaller model, these cracks start to show — making puzzles like this a great way to benchmark reasoning ability across models.