From logits to probabilities
The model's raw scores are turned into probabilities with the softmax function: each logit is divided by the temperature T, raised as a power of e, and divided by the total, so P(token) = elogit/T / Σ elogit/T. Only the differences between logits matter. Adding the same amount to every logit changes nothing.
What each setting does
- Temperature stretches or squashes the differences. Below 1, the likely tokens become even more likely and the answer becomes more predictable. Above 1, the odds even out and the output becomes more varied, and eventually less sensible. At 0, the model simply takes the most likely token every time, which is called greedy decoding.
- Top-k keeps only the k most likely tokens and shares the probability between them.
- Top-p, also called nucleus sampling, keeps the smallest group of most likely tokens whose probabilities add up to at least p, then shares the probability between them. Unlike top-k, the number of tokens it keeps changes with how confident the model is.
This tool applies them in the order Hugging Face Transformers uses: temperature first, then top-k, then top-p, and finally it scales the survivors back up to 100 %. Other software can use a different order, and some APIs don't let you set all three. Their defaults also differ, so check the documentation for the model you use.
Entropy
The tool also shows the entropy of the final choice in bits, a measure of how uncertain it is. One certain token is 0 bits, and four equally likely tokens are 2 bits. Lower temperature and tighter top-p both lower it.
For a fuller explanation with examples, read temperature and sampling explained.