Generation Is a Policy: Alignment, Sampling, and Reliable Foundation-Model Output

An AI feature can use the same base model and still behave like a different product after a provider changes post-training or an engineer changes decoding settings.

That is not a cosmetic detail. A model does not produce one inevitable answer. It produces a probability distribution over possible next tokens, then a decoding policy chooses how to turn that distribution into text. Before decoding begins, post-training has already shaped what the model treats as a helpful, safe, or preferred response.

Generation settings and alignment choices are part of an application's behavior contract. They should be chosen for the job, tested against real failure cases, and versioned like any other production dependency.

Pre-Training Creates a Capable Predictor, Not a Finished Assistant

Pre-training gives a language model broad knowledge of patterns by teaching it to predict tokens from a large corpus. That is why it can complete code, imitate prose, and make surprising connections across many subjects.

It does not automatically make the model a useful assistant. A raw next-token predictor can continue a document in a way that is plausible but unhelpful, respond to a dangerous instruction, or give a confident answer where the right response is uncertainty.

Post-training changes the response-level objective. Two common stages are:

1. Supervised fine-tuning (SFT): train on high-quality examples of instructions and desired responses so the model learns the shape of a helpful interaction.
2. Preference fine-tuning: use comparisons between outputs to favor responses people or another evaluator prefer. Approaches include reinforcement learning from human feedback (RLHF), direct preference optimization (DPO), and reinforcement learning from AI feedback (RLAIF).

In one widely discussed training setup, post-training represented only a small fraction of total training compute. Its effect on product behavior was still disproportionate because it directs a capable base model toward particular kinds of answers. A relatively small curated dataset can decide whether a model hedges, refuses, explains, flatters, or persists with an incorrect premise.

That is powerful, but it is not a proof of universal alignment.

A Preference Dataset Is a Product Decision Made Upstream

Preference optimization needs a definition of better. Someone has to write examples, rank outputs, or specify the principles used to generate feedback.

The resulting model can inherit the blind spots of that process. If labelers are drawn from a narrow demographic or are rewarded for answers that feel decisive, the model may learn a style that does not represent the full user population or a safe response to ambiguity. AI-generated feedback can increase scale, but it also makes the choice of principles and evaluator behavior more important.

This is why "aligned" should not be treated as a binary vendor property. Ask aligned to what, measured by whom, and under which language and cultural context? A response that reads as polite and useful in one setting can be misleading, dismissive, or unsafe in another.

For a product team, the practical work is local:

• define the behaviors users need and the failures they cannot absorb,
• test whether the chosen model behaves that way with representative requests,
• segment results by language and user group where it matters,
• give humans authority over outcomes with meaningful legal, financial, health, or safety consequences.

Post-training can improve a model's general helpfulness. It cannot replace the application's responsibility to enforce permissions, business rules, and escalation paths.

Every Response Starts as a Distribution

For the next token in a response, a language model produces a score for every token in its vocabulary. These scores, called logits, are transformed into probabilities. Decoding decides which token to choose, adds it to the context, and repeats the process.

The most likely token is not always the most useful one. Greedy decoding always chooses it. That can be a sensible default for a classification-style task, but it often makes open-ended language repetitive and can lock into an early mistake.

Sampling permits other plausible tokens. It adds variation, but variation is not a free quality improvement. The settings decide which possibilities remain eligible and how strongly the model favors the most likely ones.

Control	What it changes	Useful question
Temperature	Flattens or sharpens the probability distribution	Should this job prefer consistency or diverse phrasing?
Top-k	Limits choices to a fixed number of likely tokens	Is a fixed candidate pool sensible across very different contexts?
Top-p	Limits choices to the smallest pool meeting a probability threshold	Does the candidate set need to adapt to the model's confidence?
Min-p	Excludes candidates too improbable relative to the leading option	Can the task avoid low-plausibility continuations with a simpler threshold?
Stop conditions	Determine when generation ends	What must be complete before the application treats the output as usable?

There is no globally correct configuration. A content ideation tool can accept diverse options. An extraction step that controls a financial workflow should favor stable output and explicit uncertainty. The mistake is reusing a creative-chat setting for a structured action merely because it produced a friendly demo.

Treat Sampling as Task Configuration, Not a Global Personality Slider

A reliable application tends to have several generation policies, each matched to a bounded job.

For example, a JavaScript product might use:

• low-variance decoding for assigning a known ticket category,
• constrained JSON generation for extracting fields from a submitted form,
• a moderate-diversity policy for drafting alternative button copy,
• multiple candidates plus a verifier for a difficult code transformation,
• a short, source-grounded response for a support agent rather than an unbounded answer.

Store those policies with the prompt template, model version, output schema, and evaluation results. A change from one temperature or stopping rule to another can alter user-facing behavior as meaningfully as a backend dependency upgrade.

This also makes incidents debuggable. When a structured extraction begins returning a new failure shape, the team should be able to answer which model, prompt, candidate count, decoding parameters, source documents, and validator version produced it. "The AI was inconsistent" is not an operational diagnosis.

More Attempts Can Improve a Result, but They Spend a Budget

Sampling more than one completion can be useful. A system can generate several candidates, choose the most common answer, score candidates with a verifier, or ask a user to select among genuinely distinct options.

This is often called best-of-$N$. It can improve difficult tasks because one sampled path may avoid an error another path makes. It also multiplies inference work. A workflow that asks for 20 candidates has taken on roughly 20 times the generation cost before ranking, validation, or tool use.

The tradeoff should be explicit:

• Is the additional accuracy worth the latency and spend?
• Can a deterministic validator reject bad candidates cheaply?
• Is the task asynchronous enough to tolerate parallel generation?
• Would a clearer input, a smaller subtask, or a source lookup improve the result more directly?

For browser applications, do not solve this by letting the client fan out unbounded requests. Keep the policy on a server-side boundary where rate limits, authorization, cancellation, and cost controls are enforceable. The UI can expose a deliberate action such as "generate alternatives" rather than quietly turning every keystroke into an expensive search over model outputs.

Syntax Guarantees Are Useful, but They Do Not Prove Meaning

Production workflows often need a model to return a particular format: JSON for a form, a regular expression, a tool-call payload, or SQL that passes a parser. Constrained decoding can limit generation so invalid token sequences are never emitted. That is far more reliable than merely requesting "valid JSON" in a prompt.

It is still not a complete trust boundary. A valid JSON object can contain a nonexistent user ID, an unauthorized action, a harmful query, or a plausible but wrong amount. A response can also stop early because of a token limit or cancellation, leaving an incomplete result despite a strong format instruction.

The application should therefore separate four checks:

1. Parse: is the response syntactically complete?
2. Validate: does it conform to the expected schema and value limits?
3. Authorize: may this user and workflow perform the requested action?
4. Verify: does the proposed value agree with authoritative data or a deterministic rule?

For example, a model can propose a database filter from natural language. Code must still choose a read-only query surface, bind parameters, limit the rows and execution time, and enforce the caller's tenant access. A well-formed string is not an entitlement.

Hallucination Is Not Solved by Lowering the Temperature

Hallucinations are especially damaging in factual work because fluent language can make an unsupported claim sound earned. Sampling randomness can contribute to inconsistent answers, but it does not fully explain the problem. A model can confidently construct a continuation that matches familiar patterns even when the specific fact, source, or object is wrong.

Errors can snowball. Once a response accepts a false premise, subsequent tokens may make the premise sound more coherent rather than correcting it. Longer responses provide more opportunities for that chain to continue.

Lowering temperature can make an output more repeatable. It cannot turn the model into a factual database. Post-training can encourage truthfulness, but it may also optimize for answers that evaluators prefer, including answers that sound complete when a model lacks evidence. These objectives can conflict.

The dependable response is architectural:

• retrieve current, authorized evidence for claims that must be accurate,
• request citations or source references only when the application can inspect them,
• make "I do not have enough evidence" an acceptable outcome,
• keep answers bounded to the question rather than rewarding unnecessary elaboration,
• use deterministic checks and human review before irreversible actions,
• record and evaluate the failures users actually encounter.

The interface must participate. A source panel, an uncertainty state, a confirmation step, and an easy correction path are not decorative details around a language model. They are how users learn what the system knows, challenge it when it is wrong, and keep authority in the right place.

Make Generation Choices Legible and Testable

Foundation models give teams a flexible response generator. That flexibility moves part of the product policy into post-training assumptions and decoding configuration.

Choose those settings per task. Keep structured-output constraints, schema validation, authorization, and verification outside the model. Measure candidate counts against cost. Test hallucination behavior with real source gaps and misleading premises. Give users an interface that represents uncertainty instead of disguising it.

The goal is not to make a model deterministic. It is to make the application's promise reliable even though the component producing language is probabilistic.