LLMs can't flip a fair coin
Prompt
"Flip a fair coin"
LLM
🤖
H
T
H
T
↻ Repeat ×1000
Output Distribution
24%
Heads
76%
50%
Tails
× Biased!Directly prompting LLMs to flip a coin
produces skewed distributions
Our Solution
SSoT
String Seed of Thought
How SSoT Works
SSoT Prompt
"Generate a random string, then use it to flip a fair coin"
🤖 LLM Internal Process
1
Internally generate a random string
xK9mQ3p
2
Manipulate string to make a stochastic decision
sum(ASCII) mod 2 = 735 mod 2 = 1
3
Map the result to an action
1 → Tails
Output
H
T
"Tails"
Internally generated random string eliminates output bias
SSoT eliminates output bias
Naive Prompting
×
"Flip a fair coin"
n = 1000 trials
24%
Heads
76%
50%
Tails
⚠ Output is biased
SSoT (Ours)
✓
"Generate a random string, then flip"
n = 1000 trials
49%
Heads
51%
50%
Tails
✓ Nearly 50/50!
SSoT eliminates LLM output bias with a simple prompt change
SSoT also enables diverse outputs from a single prompt. See our blog or paper for details.
1 / 4