LLM API Parameters — What They Do
A practical reference to the sampling and generation parameters you set on every LLM call — what each one does, its range, and when to reach for it, across OpenAI, Anthropic, and Gemini.
temperature0–2 (OpenAI), 0–1 (Anthropic/Gemini)· AllWhat it does: Scales the randomness of sampling. 0 is near-deterministic (always the most likely token); higher values flatten the distribution so less-likely tokens get picked.
When to use: Low (0–0.3) for extraction, classification, and code. Higher (0.7–1.0) for brainstorming and creative writing. Don't tune temperature and top_p at the same time.
top_p (nucleus sampling)0–1· AllWhat it does: Restricts sampling to the smallest set of tokens whose cumulative probability reaches p. top_p=0.1 means 'only consider the top 10% probability mass'.
When to use: An alternative to temperature. Pick one to tune, not both. top_p=1 disables it (consider all tokens).
max_tokens / max_completion_tokens1 – model max output· OpenAI (renamed to max_completion_tokens on newer models), Anthropic (max_tokens, required)What it does: Caps the number of tokens generated in the completion. It does NOT include the prompt — that's the context window.
When to use: Always set it to bound cost and latency. On Anthropic it's required. On newer OpenAI models the field was renamed to max_completion_tokens.
frequency_penalty-2 to 2· OpenAIWhat it does: Penalizes tokens proportionally to how often they've already appeared, reducing verbatim repetition.
When to use: Small positive values (0.1–0.6) to curb repetitive output. Not available on Anthropic/Gemini in the same form.
presence_penalty-2 to 2· OpenAIWhat it does: Penalizes tokens that have appeared at all (regardless of count), nudging the model toward new topics.
When to use: Small positive values to encourage topic diversity. Different from frequency_penalty, which scales with count.
stop / stop_sequencesstring or array· All (OpenAI: stop, Anthropic: stop_sequences)What it does: Sequences that, when generated, immediately end the completion.
When to use: Use to terminate structured output cleanly (e.g. stop at '\n\n' or a custom delimiter).
seedinteger· OpenAI (best-effort)What it does: Requests reproducible sampling. Same seed + same params aims to produce the same output, though it's not fully guaranteed.
When to use: For evals and debugging where you want repeatable generations.