Model gateway, prompt registry, structured output layer, guardrails. Four boxes on every AI architecture diagram, and the first two quarters of most AI platform programs. For an enterprise consuming a managed model, they reduce to a Terraform module, a folder in source control, one request parameter, and a function you were going to write anyway.
First of four groups in a component-by-component teardown. The overview is in The Enterprise AI Platform: What’s Left When You Unpack It.
Access Is Real, and It Belongs to Cloud Engineering
This is the one component in the group with genuine infrastructure behind it, and your cloud team already knows how to build it.
A managed model service provides the whole capability natively: an IAM role per workload, inference profiles carrying cost allocation tags, invocation logging, private network paths, and the contractual coverage regulated data requires.
A proxy in front of it is redundant unless you are genuinely running two cloud providers’ model services. It also places a platform team in the path of every inference call, which is an availability problem you created for yourself.
Three traps default to wrong.
- Invocation logs contain your raw data. For a referral, that is the patient name and member ID sitting in a log bucket. Classification, encryption, and retention are decisions somebody has to make, and nobody makes them by default.
- The document extraction vendor sits upstream. Your model call may be compliant while the OCR step in front of it is not.
- Endpoint compliance is not pipeline compliance. The model call is one hop.
Source Control Is the Prompt Registry
Prompt management products offer versioning, diff, authorship, approval, and rollback. Your source control already provides all five.
Four rules, and no new system:
- The prompt lives in a file, never as a string inside code
- The output schema is versioned in the same folder and changes with it
- A released prompt is never edited — a change means a new version
- The version string is derived from the filename and logged on every call
The audit question is: for this record, what prompt produced this output, who approved it, and when did it change? Three systems you already run answer it. The application log holds record to version. Source control holds version to diff and approver. The invocation log holds the actual request.
A registry earns its place when a non-engineer needs to edit prompts, or when discovery across dozens of them becomes real. Neither applies to one team and one use case.
And runtime prompt swapping is a feature to refuse in a regulated path. An unreviewed prompt change reaching production without passing your build checks is precisely what you are trying to prevent.
Structured Output Comes from the Model
You attach a schema to the request and the model is constrained to fill it. It works identically whichever hosting channel you use — which is exactly why it does not belong to the platform layer.
The mechanism is free. The schema design is the work.
Every field is an object, not a bare value. A field returns its value, a status, and where in the document it came from. Bare values cannot express “I could not read this,” and you cannot add that later without breaking every consumer.
Status is a required enumeration. Extracted, not present, illegible. On a referral, “not present” means call the referring office and “illegible” means request the fax again. Collapse them into an empty value and the routing decision is destroyed.
Constrain types in the schema. Date patterns, code formats, enumerated values. Every constraint expressed is a class of error the model cannot produce.
Nothing is optional. Force an explicit “not present” rather than a silently missing field. An absent field and an absent value are different failures, and only one is recoverable.
Then validate the response against the schema anyway. Constrained generation is very good, not guaranteed.
AI output crossing a system boundary is a versioned contract. Teams skip the discipline because model output feels like text.
Guardrails Are Four Unrelated Things
Sold as one layer. They are not.
| Concern | Where it actually lives |
|---|---|
| Output shape | The schema, above |
| Content safety — PII, topics, grounding | Cloud configuration |
| Domain correctness — code validity, identifier formats | Your own validation function |
| What the model is permitted to cause | An architectural rule |
Guardrail products are built for chat and retrieval — open-ended input, prose output, human reader. They fit a staff-facing policy assistant well.
They are useless or harmful for extraction. The marquee feature is PII filtering, which on a referral would mask the patient name and member ID you asked the model to extract. Mandating that every AI call passes through the guardrail service would break your highest-value use case.
And domain validation is not AI work. Checking that a diagnosis code is active or an invoice’s line items sum to the total belongs on human-entered data too. Build it as shared intake validation used by both paths, and it stops being an AI platform component at all.
The Rules Worth Writing Down
Three, and they cost nothing to state.
- Model output never authorises an action. Validation passing does. Confidence is advisory and only orders the review queue.
- Tools available to a model reading untrusted input are read-only. An incoming fax or invoice is untrusted input.
- Anything leaving the organisation gets human sign-off.
Most teams already do all three by accident. Writing them down is what makes the next team inherit them.
Enforcement, or It Did Not Happen
Publishing principles does not work. Nobody reads them and there is no consequence.
Every convention needs exactly one of: an automatic check, an item on the review checklist, or a gate before release. If it has none, it is a document.
Think of the clerk at the passport office. She does not read your application or judge whether you deserve a passport. She checks that the photo is attached, the form is signed, and the fee is paid. Four seconds, and it catches the mistake that would otherwise surface six weeks later.
| Convention | Enforced by |
|---|---|
| Prompt in a file, version logged | Automatic check |
| Schema attached and strict | Automatic check |
| New version per change, never edit | Review checklist |
| Model output does not authorise action | Review checklist |
| Accuracy report published | Release gate |
Five conventions, and only two need a human. That ratio is what makes a standard survive after you stop watching.
A standard with no mechanism behind it is a wiki page, not a standard.
Final Thoughts
Four components. One Terraform module, one folder in source control, one request parameter, and a page of rules.
No new system, no new team, and nothing here an enterprise does not already know how to do. What is left is discipline — and discipline only holds when something mechanical enforces it.
Put the rules in the build.
Not in a document.
Tech Canvas 