Key Learnings: Controlling LLM Responses

• 100 tokens ≈ 75 words.

Controlling How LLM’s Respond

This image summarizes the key controls (hyperparameters) used to shape how a large language model (LLM) generates text. At a high level, these settings influence creativity, focus, length, and repetition of responses.

• Temperature – Controls randomness. Higher values make responses more creative and diverse; lower values make them more predictable and focused.
• Top-K – Limits the model to choosing from the top K most likely next words, reducing unlikely or noisy outputs.
• Top-P (nucleus sampling) – Selects from the smallest set of words whose total probability exceeds a threshold, balancing diversity and coherence.
• Max Response (tokens) – Sets the maximum length of the generated output.
• Frequency Penalty – Reduces repetition by penalizing words that have already been used frequently.
• Presence Penalty – Encourages introducing new topics or words rather than repeating existing ones.

Together, these controls act like “communication style knobs,” adjusting how the model speaks—its tone, variety, and verbosity.