LLM temperature, top-p, and the parameters that shape every AI answer

OpenAI's default temperature is 1.0. Anthropic's is 1.0. Google Gemini defaults to 0.9. Perplexity's public playground ships with 0.2 on its API. Change any of those numbers and the same prompt, on the same model, returns a different answer, sometimes very different. That gap between defaults is one reason two teams running "the same" model in production get different behavior, and one reason parameter drift shows up in your own prompts before you notice it.

The parameters here are not the billions of weights the model learned during training. They are the runtime knobs the API exposes to you: temperature, top-p, top-k, frequency and presence penalties, max output tokens, and stop sequences. This piece walks through each one, what value to pick by task, and the defaults each major provider ships as of July 2026.

What LLM temperature actually controls

Every time a model generates the next token, it computes a probability for every candidate token. Temperature scales those probabilities before the sampler picks one. Low temperature sharpens the distribution, the top candidate becomes even more likely and long-tail tokens shrink toward zero. High temperature flattens it, giving less-likely tokens a real shot.

A temperature of 0 is deterministic-ish: the model picks the highest-probability token every step (with the caveat that some providers still allow tie-breaking randomness). A temperature of 1 uses the raw distribution. Values above 1 amplify surprise.

Temperature is not a "quality" knob. It doesn't make the model smarter. It shifts the trade-off between predictability and variety. Same model, different behavior.

Temperature by task: the decision table

The right value depends on what you're asking the model to do. I've grouped the common tasks against a range that most teams end up using, cross-checked against the parameter guides published by OpenAI, Anthropic, and the Prompt Engineering Guide.

Two practical notes. First, providers cap temperature differently: OpenAI accepts 0 to 2, Anthropic 0 to 1, Gemini 0 to 2. Setting 1.5 on Anthropic's API returns an error. Second, if a task is failing at temperature 0.7, dropping to 0.3 is usually the first thing to try before adding more prompt engineering.

Top-p, or nucleus sampling: the alternative dial

Top-p (published as nucleus sampling by Holtzman et al. in 2019) samples from the smallest set of tokens whose cumulative probability reaches the top-p threshold. Top-p of 0.9 means: keep the most likely tokens until their probabilities sum to 90%, throw the rest out, then sample from what's left.

The practical difference: top-p dynamically resizes the candidate pool. If the model is very confident (one token dominates), top-p returns a tiny set and the output is nearly deterministic. If the model is uncertain (probability spread across many tokens), top-p keeps more of them in play. Temperature does no such adaptation; it scales the whole distribution the same way regardless of confidence.

The rule of thumb every provider's docs converge on: adjust one, hold the other at default. If you're tuning temperature, leave top-p at 1.0. If you're tuning top-p, leave temperature at 1.0. Adjusting both at once makes the effect hard to reason about, and reproducibility suffers.

For most product uses, temperature is enough. Top-p becomes useful when you want confident-when-confident behavior: highly deterministic on easy answers, permissively creative when the model is genuinely uncertain.

Top-k: the third randomness knob

Top-k caps the candidate pool at a fixed number of top tokens regardless of their probability mass. Top-k of 40 means: consider only the 40 most likely tokens, sample from those.

Not every provider exposes top-k. Anthropic and Google Gemini do. OpenAI's chat completions API doesn't. That gap alone is a reason to check each provider's parameter table before you port a prompt across models.

Top-k pairs oddly with top-p: they compete for the same job (restricting the candidate set) using different rules. If your API accepts both, set top-p to 1.0 and use top-k, or set top-k to a very high value and use top-p. Most teams pick one.

Frequency and presence penalties: repetition control

Frequency penalty subtracts a value from each token's logit proportional to how many times that token has already appeared in the response. It reduces repetition of exact tokens.

Presence penalty subtracts a fixed value from any token that has appeared at least once, regardless of count. It pushes the model toward new vocabulary earlier.

Typical ranges: 0.0 to 2.0 on OpenAI, with 0.0 as the default (no penalty). A value of 0.5 is a soft nudge; 1.0 is strong; above 1.5 the output starts to fragment.

When to reach for which: frequency penalty helps when the model loops on a phrase ("The product is great. The product is great."); presence penalty helps when responses lean on the same vocabulary across a long answer. Neither replaces a good prompt, but both save token budget when the model gets stuck.

Max tokens and stop sequences: shape the output

Max output tokens sets a hard cap on the response length. It doesn't teach the model to be concise, it truncates. Use it as a safety limit (protect your bill) and pair it with a prompt instruction like "answer in under 200 words" if you want short responses.

Stop sequences are strings that, when generated, cause the model to halt immediately. Common uses: preventing runoff after a JSON block ends, cutting off role-play at "\nUser:", or ending a list at "\n\n". Stop sequences are the cleanest way to enforce structured output, more reliable than prompt-level "and stop here" instructions.

Provider defaults, side by side

Verified against each vendor's public API reference on July 9, 2026. Values can change; check the docs before you build.

The Perplexity default at 0.2 is worth flagging. Perplexity's product is answer generation over retrieved sources, and the low default reflects that: they want factual adherence, not variety. If you use Perplexity's API as a generic chat model without knowing this, you'll wonder why responses feel flat compared to ChatGPT.

A Python example

The point of a code snippet here is to show all the parameters in one call. This is the OpenAI Python SDK; other SDKs mirror the same shape.

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="gpt-4o",
    messages=[
        {"role": "system", "content": "You extract product SKUs from copy. Return JSON."},
        {"role": "user", "content": product_description},
    ],
    temperature=0.1,           # low, we want reproducible extraction
    top_p=1.0,                 # hold at default, tuning temperature only
    max_tokens=500,            # bill safety
    frequency_penalty=0.0,     # no repetition risk on structured output
    presence_penalty=0.0,
    stop=["```"],              # end at the JSON fence
)

Two things to notice. First, tuning one randomness parameter at a time (temperature here, top-p held at 1.0). Second, stop sequences enforce structure at the API layer, so your parser doesn't have to strip trailing prose.

A parameter tuning workflow

Four steps that save time over ad-hoc guessing:

1. Baseline. Run the prompt at provider defaults. Log the response.
2. Move one parameter. Change temperature or top-p by 0.2, hold everything else. Log again.
3. Compare on a fixed set of inputs. Ten to twenty prompts is usually enough to see whether the change helps or hurts.
4. Save the winning config. Version-control the parameter set alongside the prompt.

The mistake most teams make is treating parameters as a search over one input at a time. Parameters interact with prompts. A prompt tuned at temperature 0.3 might behave badly at 0.9, so re-evaluate parameters whenever you rewrite the prompt.

What parameters mean for AI Visibility

If your brand shows up (or doesn't) when a shopper asks ChatGPT or Google AI a question about your category, the temperature that model runs at is set by the platform, not by you. You can't tune it. What you can measure is which prompts, which models, and which sessions the brand appears in, and how that pattern shifts week to week.

That's the loop Mention Network tracks: same shopper questions, same models, run repeatedly, so a real signal separates from the sampling noise that any single check would show. Parameter variability is one reason a screenshot proves nothing. Pattern detection does.

For your own workflows (chatbots, RAG, product-description drafting), the parameters above are the controls that matter. Get them wrong and outputs drift in ways your users notice before you do.

You can run a free brand check at mention.network to see how your brand and products show up across ChatGPT, Gemini, and Google AI.

FAQ

Can I use temperature and top-p at the same time? You can, and every API accepts it, but the effects overlap and reproducibility gets harder. The convention is to adjust one and hold the other at default (1.0).

What is greedy decoding? Greedy decoding picks the single highest-probability token at every step. It's what temperature 0 approximates, though most APIs still allow tiny tie-breaking noise so results aren't bit-perfect deterministic across runs.

Do reasoning models like o1 or Claude thinking use these parameters? Partially. OpenAI's reasoning models accept some parameters but ignore or fix others (temperature is capped or fixed on some reasoning endpoints). Anthropic's extended thinking accepts temperature and top-p normally. Check the endpoint-specific docs.

Which parameter should I tune first? Temperature. It's the most universally exposed, most predictable in effect, and the one you can teach a team to reason about in a few minutes. Reach for penalties, top-p, or top-k only when temperature alone isn't solving the failure mode.

Why do OpenAI and Perplexity ship such different defaults (1.0 vs 0.2)? Product intent. OpenAI's chat completions serve a broad set of use cases; a middle-of-the-road default lets each caller tune. Perplexity's product is answering with cited sources, so factual adherence is the default posture.

How do I know my parameter change actually helped? Fix the prompt and inputs, change one parameter, run 10-20 samples, compare output quality against a rubric you wrote before the change. Ad-hoc "this feels better" is how teams lock in worse configs.

Methodology

Provider defaults verified against OpenAI, Anthropic, Google Gemini, and Perplexity public API references on July 9, 2026. Nucleus sampling behavior references Holtzman et al., "The Curious Case of Neural Text Degeneration" (2019). Recent temperature-impact benchmarks reference Zhang et al., "Exploring the Impact of Temperature on Large Language Models" (arxiv 2506.07295, June 2025). Parameter values in the task decision table reflect ranges commonly cited across the OpenAI cookbook, Anthropic prompt engineering guide, and the Prompt Engineering Guide, plus my own testing on GPT-4o and Claude 3.5 Sonnet.