Foundation Models Change the Contract: From Deterministic Software to Probabilistic Systems

Architecture Patterns

Foundation models are often introduced as an API upgrade: send text in, receive text out, add a capable feature to an application.

That framing is convenient, but it hides the architectural shift.

An ordinary library is expected to follow a tightly specified contract. Given the same valid input and version, a sorting function returns the same ordered list. Given the same database query and committed data, an API can usually promise a precise result. A foundation model offers something different: a generated continuation shaped by its training, the current input, model settings, and sometimes nondeterministic serving behavior.

The result can be exceptionally useful. It can also be plausible, incomplete, overly confident, or formatted in a way the next component cannot safely consume. Treating that system like a deterministic function is how an impressive prototype becomes an unreliable feature.

The useful mental model is not "AI is magic" or "AI is just autocomplete." It is: a foundation model is a powerful probabilistic component with an unusually broad input interface. That model tells us what application architecture needs to do around it.

A Language Model Predicts Tokens, Not Facts

Before a language model can process text, the text is split into tokens. A token can be a whole word, part of a word, punctuation, whitespace, or another recurring unit in the model's vocabulary. The model receives token IDs rather than sentences in the human sense.

An autoregressive model then repeatedly estimates a probability distribution for the next token. Generation is sequential:

  1. the application sends a prompt,
  2. the model chooses a likely next token,
  3. that token becomes part of the next input,
  4. the process repeats until a stopping rule is reached.

This is the reason output length has a direct latency and cost consequence. A long answer is not one large response computed all at once. It is a chain of dependent generation steps. A response cannot produce token 200 before it has produced the context that leads to token 199.

It also explains why a fluent answer is not necessarily a verified answer. The model is optimized to produce a likely continuation, not to retrieve a citation from an authoritative database or prove that each statement is true. Sometimes those goals align beautifully. Sometimes they do not.

For an application, that distinction changes the contract. Do not wire a generated paragraph directly into a regulated decision, an irreversible mutation, or a customer-visible billing action merely because the output sounds confident. Give the model bounded jobs, verify what can be verified, and leave an explicit path for uncertainty.

Scale Made One Model Useful in Many Contexts

Earlier machine learning systems were commonly built for a narrowly defined task: flag suspicious payments, rank search results, forecast demand, or classify an image. Their data, labels, features, and evaluation target were built around that task.

Foundation models are trained at much larger scale, often with self-supervised objectives that can create learning signals from the input itself. For example, a training process can hide part of a sequence or predict its next part without requiring a human to label every example. This makes it practical to learn broad patterns from enormous collections of text, code, images, audio, and other data.

The important product consequence is transfer. The same general-purpose model can be asked to draft a support reply, explain code, extract fields from a document, translate text, or reason over an image. The model has not become a universal source of truth. It has become a reusable starting point for a surprisingly wide range of tasks.

That is why model-as-a-service changed the economics of application development. A small product team can experiment with capabilities that previously required large training datasets, specialized infrastructure, and a long model-development cycle.

But access to a shared capable model also means the base capability alone is rarely the whole product. A competitor can often call a similar model. Durable value usually comes from the workflow around it: proprietary context, careful interaction design, feedback loops, integrations, safety controls, and a real understanding of the user's job.

Multimodality Broadens the Interface, Not the Guarantee

Text is no longer the only useful model input. Models can now work across text, images, audio, video, and structured representations. A product can let a user upload a photo of a damaged shipment, dictate a note, and ask a question in the same interaction.

That opens new interfaces, but it should not make the system less disciplined.

An image-capable support workflow, for example, may be able to suggest a damage category. It still needs application code to enforce which categories exist, request a clearer photo when confidence is low, and keep a human in the loop for a refund over a risk threshold. A multimodal prompt can improve the available evidence; it does not remove the need for product policy.

It also widens the privacy boundary. A screenshot, voice clip, or document may contain more personal information than the user's typed question. Before adding it to model context, teams need to answer ordinary but essential questions:

  • Is this data necessary for the task?
  • May this provider retain it or use it under the current service terms?
  • Can the user understand that the content leaves the product boundary?
  • How are uploads redacted, access-controlled, deleted, and audited?

The easiest context to send is not always the context the product is entitled to send.

Choose an Adaptation Method by What Is Actually Missing

A foundation model can be adapted in several ways. These approaches are often presented as a maturity ladder, but they solve different problems.

Prompt Engineering Shapes a Single Interaction

A prompt supplies instructions, examples, constraints, and the immediate task. It is usually the fastest way to test whether a model can help at all.

For a JavaScript team, a prompt can turn a generic model into a first pass at summarizing a pull request, generating accessible alt text, or extracting a short explanation from logs. The advantage is iteration speed. The limitation is that a prompt is an input, not a durable guarantee. A minor wording change, a provider model update, or a longer conversation can alter the result.

Prompt engineering is appropriate when the task is well bounded and the application can validate the result. It is a poor substitute for an authoritative data source or a business rule that should live in code.

Retrieval Adds Current, Inspectable Knowledge

Retrieval-augmented generation, commonly called RAG, gives the model selected documents or data at request time. Rather than hoping model training contains the current return policy, the application retrieves the policy revision that applies to the customer's order and places that material in the context.

This helps with information that is private, recent, product-specific, or likely to change. It also gives engineers a better debugging surface: they can inspect which documents were retrieved, whether they were current, and whether the model grounded its answer in them.

RAG does not automatically make an answer correct. Bad chunking, weak metadata, irrelevant retrieval, missing access checks, or a prompt that invites the model to ignore its sources can still produce a bad answer. The retrieval layer needs the same attention to permissions, freshness, relevance, and observability as any other data-serving system.

Fine-Tuning Changes Repeated Behavior

Fine-tuning updates a model using task-specific examples. It can help when an application needs a reliable recurring style, format, or behavior that a prompt alone cannot establish efficiently.

It has real costs. Teams need a representative dataset, a way to judge whether the adapted model improved, infrastructure to reproduce the training run, and a plan to update it as the product changes. Fine-tuning is not usually the first answer to "the model does not know today's policy." Retrieval is often a better fit for changing facts.

A practical decision sequence is:

  1. Start with a prompt to prove the user value.
  2. Add retrieval when the task depends on current or private information.
  3. Consider fine-tuning when repeated behavioral gaps remain after the task, data, and evaluation are clear.

That sequence is not a law. It is a way to avoid buying model complexity before identifying the actual constraint.

Build a Boundary Around Model Output

The model should not be the final authority on what enters the rest of the system. Treat its response as untrusted input, even when it is generated by a vendor your product deliberately chose.

For a feature that extracts an action item from a meeting note, the application can request JSON but must still parse, validate, and constrain the result before creating a task:

type ProposedTask = {
  title: string
  ownerEmail?: string
  dueDate?: string
}

function isProposedTask(value: unknown): value is ProposedTask {
  if (!value || typeof value !== 'object') return false

  let task = value as Record<string, unknown>
  return (
    typeof task.title === 'string' &&
    task.title.trim().length > 0 &&
    task.title.length <= 200 &&
    (task.ownerEmail === undefined || typeof task.ownerEmail === 'string') &&
    (task.dueDate === undefined || typeof task.dueDate === 'string')
  )
}

That check is deliberately mundane. It is also where reliable software begins. The next layer can verify the email belongs to an allowed workspace member, parse the date, ask for confirmation, and record the exact prompt, model version, and retrieved context that led to the proposal.

The same principle applies to tool use. A model may recommend a database query or API call, but code should enforce authorization, permitted operations, rate limits, idempotency, and approval rules. Natural-language reasoning must not bypass the controls that protect every other caller.

The User Interface Carries Part of the Reliability Model

Frontend engineers have an unusually direct role in making probabilistic behavior safe and useful.

The interface can distinguish a draft from a completed action. It can show the sources behind an answer, reveal that a response is still streaming, let a user correct the result, and provide a graceful fallback when a request exceeds a latency budget. It can avoid implying certainty that the system does not have.

Consider the difference between these two experiences:

  • "Your refund has been approved" after a model-generated interpretation of a photo.
  • "We found a possible refund request. Review the details before submitting."

The second design makes the automation useful without silently transferring authority to it. It also produces correction data that can improve the workflow later.

Latency deserves the same product attention. A streaming response can reduce perceived wait time by showing the first useful token early, but it does not make the whole workflow instant. If downstream validation, retrieval, or a human review is necessary, the interface should represent those states honestly instead of pretending every AI interaction is a chat bubble.

The New Contract Is an Engineering Opportunity

Foundation models let teams build with capabilities that were once expensive to create. That is the opportunity. The engineering work is to turn that broad capability into a narrow, reliable promise to a user.

Start from the fact that model output is generated rather than guaranteed. Attach current knowledge through deliberate retrieval. Validate every structured handoff. Keep sensitive context within a defensible data boundary. Design the interface so users can see, correct, and trust what the system is doing.

The result is not less ambitious than an AI demo. It is more useful: a product that knows where the model ends and where the application must take responsibility.