Updated October 12, 2026
Every request to a chat model includes settings alongside the prompt. Three key LLM parameters shape the result: temperature controls how random the word choice is, top_p limits how many likely words the model can choose from, and max_tokens sets the maximum length of the answer. While most tutorials describe each setting in one sentence, this article explains their effects using numerical results. On 7 October 2026, approximately 180 requests were sent to two models through the AI/ML API, and their outputs were compared. The total cost of all requests was approximately one cent.
Key LLM Parameters Explained
| Setting | What it controls | Range in the AI/ML API docs | What was observed |
| Temperature | Randomness of word choice | 0 to 2 | 2 different taglines in 10 runs at 0, and 9 at 1.0 |
| Top_p | Share of likely words the model may choose from | 0.01 to 1 | 2 different taglines in 10 runs at 0.1, and 9 at 1.0 |
| Max_tokens | Upper limit on generated tokens | Minimum 1 | Answer cut in the middle of a sentence at 20 tokens, and an empty answer on a reasoning model at 300 |
| Seed | Repeatability of sampling | Documented, minimum 1 | 10 of 10 identical answers at temperature 1.0 |
How LLM Parameters Affect the Next Word?
A language model writes text one token at a time. A token is a piece of text, often a short word or part of a word. At each position, the model assigns a probability to every possible next token and then picks one. Temperature and top_p change how that pick is made. Max tokens does not change the pick. It only stops the process after a fixed number of tokens.
Understanding these LLM parameters helps developers control output diversity, manage response length, and make model behavior more predictable.
How the LLM Parameter Tests Were Run?
Two models were used. The first is openai/gpt-4o-mini, a model without a reasoning step, priced at 0.195 dollars per million input tokens and 0.78 dollars per million output tokens on its AI/ML API page. The second is deepseek/deepseek-v4-flash, a reasoning model priced at $0.39 and $1.56 per million input and output tokens, respectively. Each test sent the same prompt 10 times in parallel and compared the answers. The results report the number of different answers and a diversity score. The score is one minus the average word overlap between every pair of answers, so 0 means identical answers and 1 means no shared words. Ten runs per setting is a small sample, so the numbers show direction, not a precise rate.
1. Temperature
Temperature rescales the probabilities before the pick. A low value makes the most likely token even more likely. A high value gives unlikely tokens a better chance. The AI/ML API documentation lists a range of 0 to 2 for this model. The test asked for a one-sentence tagline for a small coffee shop at four temperatures.
| Temperature | Different answers in 10 runs | Diversity score | Example answer |
| 0 | 2 | 0.26 | Brewing warmth in every cup. |
| 0.7 | 8 | 0.5 | Brewing warmth, one cup at a time. |
| 1 | 9 | 0.75 | Where every sip feels like home. |
| 1.5 | 8 | 0.85 | Sip, Savor, and Share a Moment Perfected. |
Two results need a comment. Temperature 0 did not give identical answers. Two different sentences appeared in 10 runs, so a temperature of 0 lowers randomness but does not remove it. Temperature 1.5 produced the highest diversity score, while the number of different answers stayed at 8 because several runs repeated the same sentence.
The effect depends on the task. The model was also asked to label the sentiment of a mixed review with one word: positive, negative, or neutral. At temperature 0 and at 1.0, all 10 runs answered neutral. At temperature 2.0, seven runs answered neutral, one answered negative, and two answered mixed, which was not one of the three labels requested. A task with a clear answer keeps the same label at moderate temperatures. The labels change only at the top of the range.
High values also reduce the quality of longer text. We requested a 100-word explanation of an API three times at temperature 1.0 and three times at 2.0, with a 200-token limit. All three runs at 1.0 were clean. At 2.0, one run was clean; one turned into unrelated words and symbols after roughly 70 words; and one did the same after roughly 40 words, then hit the token limit. Three runs per setting is very few, but the pattern was clear enough to avoid values near 2 for text that people will read.
Different taglines in 10 runs for each setting, gpt-4o-mini.
2. Top P
Top P, also called nucleus sampling, keeps only the most likely tokens whose probabilities add up to the chosen value and picks among them. At 0.1, the model chooses from a very small set. At 1, it may choose from all tokens. The documentation lists a range of 0.01 to 1 and advises changing temperature or top_p but not both.
| top_p | Different answers in 10 runs | Diversity score |
| 0.1 | 2 | 0.26 |
| 0.5 | 3 | 0.45 |
| 1 | 9 | 0.77 |
All three rows used temperature 1.0. At top_p 0.1, the answers were as repetitive as at temperature 0, even though the temperature was 1.0. This is why the documentation advises changing one of the two LLM parameters at a time. If both are set, a smaller top_p can cancel out most of the effect of a high temperature, making the output hard to explain.
3. Max Tokens
Max tokens sets the upper limit of generated tokens. It is not a target length. The model does not adjust its answer to fit the limit, as the cut sentences below show. When it reaches the limit, generation stops, and the response’s finish_reason field says length instead of stop. The documentation also lists max_completion_tokens next to max_tokens. The gpt-4o-mini model was asked to explain what an API is in about 150 words with three limits.
| max_tokens | finish_reason | Words in the answer | Cost of the request |
| 20 | length | 18, cut before the end of the sentence | $0.00 |
| 60 | length | 56, cut before the end of the text | $0.00 |
| 250 | stop | 146, complete | $0.00 |
The cost comes from the usage information that the AI/ML API returns with each response. Billing follows the tokens generated, so a lower limit means a cheaper request and a shorter answer.
4. Max Tokens on a Reasoning Model
Reasoning models write a hidden chain of reasoning before the visible answer. The response reports these tokens in a separate field, reasoning_tokens, but they count toward the same limit. We ran the same explanation prompt on deepseek/deepseek-v4-flash with three limits.
| max_tokens | finish_reason | Visible answer | Reasoning tokens | Total tokens | Cost |
| 50 | length | Empty | $50.00 | $50.00 | $0.00 |
| 300 | length | Empty | $300.00 | $300.00 | $0.00 |
| 2000 | stop | 129 words | $802.00 | $972.00 | $0.00 |
With limits of 50 and 300, the model used the entire budget on reasoning and returned no answer. Both requests were billed. Only the 2000 limit produced an answer, and 802 of its 972 tokens were reasoning.
A limit that works for a model without reasoning can therefore return nothing on a reasoning model. For a reasoning model, set the limit to several times the expected answer length. This is an important consideration when configuring LLM parameters for production applications.
5. Seed
The documentation lists a seed parameter that makes sampling deterministic when it is set. The tagline prompt was sent 10 times at temperature 1.0 with seed 42 and 10 times with seed 7. Seed 42 returned the same sentence 10 times, and seed 7 returned a different sentence 10 times. The same prompt at temperature 1.0 without a seed had given 9 different answers in 10 runs. A seed helps with tests and comparisons. It gives the same answer for the same request, and a different seed gives a different answer.
6. LLM Parameters That Reasoning Models Ignore
The DeepSeek documentation states that thinking mode does not support temperature, presence_penalty, and frequency_penalty. Setting them causes no error and has no effect. It also states that top_p is limited to the range 0.95 to 1.0 in thinking mode and that lower values are raised to 0.95.
The test is consistent with this guidance. The tagline prompt was sent to deepseek-v4-flash 10 times at temperature 0 and 10 times at 1.5, with a limit of 1500 tokens. Both settings returned 6 different answers in 10 runs. The diversity scores were 0.62 and 0.74, a small difference over 10 runs that does not show a clear effect.
The thinking parameter with the value disabled was also sent in the request body. The responses still contained reasoning tokens, averaging 71 to 74 per request compared with 78 to 82 before, so we couldn’t test the model without reasoning through this request.
The practical point is the same: on a reasoning model, you shouldn’t expect temperature to control the output when the model’s thinking mode ignores that setting. Always check the provider’s documentation before adjusting LLM parameters.
Recommended Starting Values
These values are starting points to test on real prompts and not fixed rules. They follow the results above and the documentation.
| Task | Temperature | Max tokens | Note |
| Classification and extraction | 0 to 0.3 | 10 to 50 | Leave top_p alone |
| Support answers and summaries | 0.3 to 0.7 | 150 to 500 | Check the finish_reason for length |
| Slogans and brainstorming | 0.8 to 1.2 | 60 to 200 | Run several times and choose |
| Reasoning models | Ignored by DeepSeek thinking mode | 2,000 or more | Limit must cover the reasoning tokens |
Python Example
The AI/ML API endpoint accepts the OpenAI Python library when you change the base URL, so the same code works with the library you may already be using. The script below reads the key from an environment variable, asks for the same tagline at two temperatures, and then shows a cut answer with a limit of 20 tokens.
The test script. The key is read from an environment variable.
This is the output of one run.
Output of the script. At temperature 0, the three answers were identical in this run.
Common Mistakes When Configuring LLM Parameters
- Setting max_tokens too low on a reasoning model: The model may return an empty but billed answer because the reasoning process consumes the entire token budget.
- Expecting temperature 0 to give identical answers every time: A seed can improve repeatability, but its effect should be checked on real prompts.
- Setting temperature and top_p together: One setting can hide the effect of the other, making the results harder to interpret.
- Using very high temperature values for text intended for readers: Values near the top of the range may produce unrelated or lower-quality content.
- Assuming every model supports the same settings: Providers may restrict or ignore specific parameters, especially in reasoning modes.
- Treating max_tokens as a target length: The model can stop abruptly when it reaches the limit rather than completing its answer.
Limitations of These LLM Parameter Tests
Each setting was tested with one or two prompts, 10 runs per setting, on two models. Other prompts and other models can behave differently, especially at the extremes of the ranges. We did not test the frequency_penalty and presence_penalty parameters, stop sequences, or DeepSeek V4 Pro.
Therefore, treat the results as practical observations rather than universal rules. A larger test set covering different prompts, models, and task types would be needed to establish more reliable patterns.
How to Try These LLM Parameters?
All the requests above went through the AI/ML API, where one key reaches both models. The AI/ML API model catalog lists more than a thousand models with prices and supported features. The DeepSeek V4 Pro API page lists reasoning among its supported features. It also lists a price of $1.17 per million input tokens and $3.51 per million output tokens. The max_tokens behavior in this article is worth checking on that model before production use.
Before integrating any model into an application, review the provider’s documentation for supported settings, valid ranges, token accounting, and pricing. Then run controlled tests to identify the most appropriate LLM parameters for your use case.
Frequently Asked Questions (FAQs)
Q1. What is temperature in an LLM?
Answer: Temperature controls how random the choice of each next token is. Low values make the output more repetitive. In the test, a one-sentence tagline gave 2 different answers in 10 runs at temperature 0 and 9 at 1.0.
Q2. What is top_p?
Answer: Top P limits the choice to the smallest group of tokens whose probabilities add up to the chosen value. At 0.1 it gave 2 different taglines in 10 runs, and at 1.0 it gave 9, both at temperature 1.0.
Q3. Should temperature and top_p be changed together?
Answer: The documentation advises changing one, not both. In the test, a top_p of 0.1 made the output repetitive even at temperature 1.0.
Q4. What does max_tokens do?
Answer: It sets the maximum number of generated tokens and stops the answer when it reaches the limit. The finish_reason field then says length. It does not make the model write a shorter answer.
Q5. Why is the answer empty?
Answer: On a reasoning model, the reasoning tokens count toward max_tokens. In the test, limits of 50 and 300 returned an empty answer and were still billed. A limit of 2000 returned the full answer.
Q6. Does temperature 0 always give the same answer?
Answer: No. In the test, it gave 2 different taglines in 10 runs. A fixed seed gave 10 identical answers in 10 runs for each of the two seeds that were tried.
Q7. Which models ignore temperature?
Answer: The DeepSeek documentation says thinking mode ignores temperature, presence_penalty, and frequency_penalty. The documentation for each reasoning model in use should be checked before changing these LLM parameters.
Recommended Articles
We hope this guide helps you understand LLM parameters, including temperature, top_p, and max_tokens, and how they influence AI-generated outputs. Explore our recommended articles for more insights on large language models, generative AI, prompt engineering, AI development, and machine learning.
