Model is portable — except when it isn't

A team building an EU healthcare app committed to a closed-frontier US-hosted model in week one. The architecture looked beautiful. Demos landed. In week twenty-six, legal explained patient data could not leave the EU. The model ran in no region legally available to the data. Six weeks of architecture work disappeared into a redo of the 1-day model swap described by the inaugural piece.

It would have been a 1-day swap. If they’d known to ask the question on day one.

The claim the inaugural piece made

The inaugural article, in its Model section, said:

Avoid early model-choice anxiety — most architectures are model-portable, so evidence can drive a later swap.

This advice fits the median project. It bites non-median projects hardest because skipped model conversations rarely return until constraint forces the issue. By the time constraint surfaces, the rest of the system often rests on assumptions the now-required model cannot satisfy.

This piece is the explicit list of when “swap later” is the wrong instinct, and when picking the model is the first architectural decision rather than the last one.

In the determinism-ladder lens

Every other lever in the stack assumes a callable model. The inaugural piece treats the choice of which model as a soft constraint: solvable later, swappable freely. This is another flavor of the same trade: pushing a decision into the future, closer to better information and a better model landscape.

For most projects, this remains the right trade. The autonomy-vs-determinism question lives at the layer level, not the model level, and layers compose the same way regardless of provider API.

When the model itself faces constraint — jurisdiction, latency, competition rule, air-gap, domain specialization — deferring the decision silently locks in an assumption. The determinism trade flips: deferred choice adds uncertainty about the entire system rather than reducing it.

Recognizing project class becomes the prerequisite to honest use of the model lever.

The exceptions

Five classes make “swap later” the wrong instinct. In any of them, the model question becomes the first conversation, not the last.

1. Regulated industries with data-residency constraints

The most common bite. Healthcare in the EU (GDPR + national health-data laws), public-sector work in jurisdictions with sovereignty mandates (France, Germany, India, Australia, Singapore), defense, certain financial services, anything touching the GDPR-protected categories.

The constraint: customer data, patient data, citizen data cannot leave the jurisdiction (or sometimes the specific provider’s certified zone). Closed-frontier models from US-hosted providers may not be legally callable on this data, even if a regional inference endpoint exists, because the training pipeline, logging, or fallback behavior of the closed provider isn’t auditable in the way the regulator requires.

The workaround: self-host an open-weight model in the certified region, or use a closed provider with a certified residency offering for the data class. Both decisions cascade into infrastructure, evals, and ops choices far away from “just call the API.”

Recognize early. Talk to legal and compliance before week three. The cost of finding out in week twenty-six is six weeks of rework; the cost of finding out post-launch is regulatory exposure.

2. Latency-critical paths

Real-time voice agents (~150 ms first-token target). High-frequency trading. In-game NPCs. Edge inference on mobile or embedded devices. Anything where tokens-per-second is part of the user-facing experience.

The constraint: raw inference speed becomes part of the product. Closed-frontier models are typically slowest because they are largest. Open models in the 7B-14B range, often with custom inference engines (vllm, tensorrt-llm, llama.cpp), can deliver 10x throughput at 80% of quality on tasks where 80% suffices.

The workaround: usually a smaller model (open or distilled), self-hosted on infrastructure tuned for inference latency. Sometimes speculative decoding to claw back another 2–3×. Sometimes a tiered system where a fast small model handles 90% of queries and only escalates to a frontier model on the long tail.

A threat surface specific to the tiered system. The escalation boundary itself creates data-exfiltration / classification-leak risk: the small in-house model sees the query first; unanswered queries forward to the frontier provider; sensitive content can leave the network precisely on the hard queries most likely to contain unusual or sensitive material. Mitigations: classify the query at the small-model layer before escalation; apply refuse-or-redact rules at the boundary, not only confidence thresholds; log every escalation to an audit trail for data-classification review; for regulated workloads, replace frontier-API escalation with a larger in-region self-hosted model. Treat small-to-frontier escalation as network egress and apply the same allow-list discipline used elsewhere.

Recognize early. Latency requirements are usually known on day one. The mistake is treating them as something the platform team will handle later. The model choice is the latency budget.

3. Competition rules and locked benchmarks

Kaggle competitions. NeurIPS / ACL challenges. Research benchmarks where the rules require a specific base. Internal “show work on this exact model” reviews.

The constraint: the rules name the model. Often the rules also constrain how the model can be used (no fine-tuning, no external retrieval, no system prompt longer than N tokens). Picking a different model breaks the submission.

The workaround: none. Competition participation fixes the model; non-participation avoids the constraint.

Recognize early. This is the easiest constraint to spot because rules spell it out. Projects still mis-scope by assuming a closed frontier prototype can switch later to the rules-locked model. Architectural choices made under the closed frontier (long system prompts, free RAG, multi-turn agent loops) often violate competition rules.

4. Air-gap and security clearance

Defense work, intelligence community work, certain government and pharma work. Some financial-services environments. Some hospitals.

The constraint: no internet egress. The model cannot make outbound calls to a closed provider API. The model must run inside the same network as the data.

The workaround: self-host an open-weight model in the secured environment. This typically means smaller models because the largest open models still need expensive hardware secured environments provision slowly. It also accepts a frontier-generation gap.

Recognize early. This is binary: either the project is air-gapped or it is not. Teams sometimes assume an API-call exception can appear later; security regimes usually answer no. Build for the air-gap from day one inside the air-gap class.

5. The model is the moat

Some niches have specialized open models genuinely outperforming frontier general models at the niche task: medical imaging foundation models, genomics models, code-specialized models like Codestral or DeepSeek-Coder-V2, domain-specific transcription models, music generation, and legal-document specialists.

The constraint: the niche-specialized model exists, performs better than the general frontier, and replacing it with “prompt the closed frontier for the same thing” would lose meaningful quality.

The workaround: the niche model is the model. Architect around it. Sometimes the hybrid pattern works: niche model handles the niche task; general frontier handles surrounding workflow. The niche model stays fixed.

Recognize early. This is not always obvious because the general frontier looks good enough on the surface. Domain experts repeating “the frontier is missing something” provide the signal. Listen to them.

The decision rule

When does model choice flip from “swap later” to “constraint up front”?

Run through this on day one:

1. Jurisdictional data constraint? If yes, decide model and inference region together. Ask legal early.
2. First-token latency budget below ~300 ms? If yes, latency-critical constraints apply. Open + self-host usually fits.
3. External rule requires a specific base model? Competition, contract, or regulation can lock the model. Architect around the rules.
4. Air-gap or no-egress requirement? If yes, self-host open-weight inside the secure perimeter.
5. Niche where specialized open models outperform the general frontier? If yes, the niche model is the foundation; everything else builds on it.

If all five are no, the inaugural piece’s advice holds: don’t agonize, swap later. If any is yes, the model conversation moves to week one.

Threat surface per exception (axiom #17)

Each exception class has a threat surface inherited by the workaround. Model choice comes first; threat-surface engineering comes second.

Exception	Threat surface	Required mitigations
Data residency	Data crossing jurisdictional boundary; auditable pipeline requirement; inference-region != training-region misclassification	Self-hosted in certified region OR closed provider with certified-residency contract; documented training pipeline lineage; egress monitoring; logging in-region
Latency-critical	Smaller models more vulnerable to prompt injection (less robust to adversarial inputs); custom inference engines often less hardened than vendor APIs	Adversarial eval set on the smaller model; rate-limiting at the inference layer; defense-in-depth on the prompt boundary
Competition rules / locked benchmarks	The rules ARE the threat model; disallowed augmentation (RAG, fine-tuning, system prompt overruns) becomes the failure mode	Lint and CI checks for rule compliance; red-team submission against rule violations before submitting
Air-gap / security clearance	Egress paths (intentional or accidental); supply-chain integrity of the open-weight model and any updates; insider threat on the secured environment	Strict no-egress firewall; signed weights with provenance; reproducible inference builds; per-clearance access controls on the inference servers
Niche-specialized (model is the moat)	Specialized model’s training data + provenance often opaquer than frontier models; supply-chain on the specialized model itself	Vendor due diligence on the specialized model; cryptographic pinning of model weights; eval on adversarial domain inputs

The pattern: every exception forcing early model decision also forces early security decision. The two questions belong at the same desk in the same week.

Spirit

The model lever anchors the determinism-ladder diagram because every other lever assumes a model. For most projects, the foundation stays interchangeable. For projects in this article, the foundation becomes the constraint defining everything above it. Acknowledging this distinction up front is itself a unit of determinism: uncertainty about viable models moves from week-twenty-six surprise into week-one design.

The opening mistake cost six weeks not because the team picked the wrong model, but because nobody asked the model-constraint question before the rest of the architecture hardened around a false assumption.

Ask the question. Then either skip this piece because the answer is “no constraint” — or build for the constraint from day one.

---

Next in the Determinism Ladder series: cheaper alternatives to MCP — when gh, kubectl, and plain curl beat the protocol, and the break-even point where the protocol earns its weight.