WHY WE SEPARATE REUSABLE AI FROM CLIENT DATA — AND WHAT NEVER LEAVES THE ENGAGEMENT
Most generative features fail in production for an operational reason, not a model-selection reason. Teams fine-tune a foundation model on client text, ship the weights in the same artifact they intend to reuse on the next engagement, and only later realize that the “accelerator” now contains someone else’s corpus. Or they call a hosted model with no retrieval layer, no approval gate, and no fallback—so when the model is wrong, the product is wrong.
At A.M. Tech Consulting we have been treating this as a delivery and governance problem, not a novelty demo. The pattern we now default to on language features is simple to state and strict to operate: the trainer is shareable; the tune is not. Public or open-weight models, retrieval code, evaluation harnesses, and CI live in reusable modules. Client writing, labels, adapters, and evaluation sets stay in private stores with explicit retention and offboarding. Drafts never publish themselves.
This post is the architecture and operating model. It builds on the AI lifecycle and solution types we have already written about—ethical governance and trustworthy-AI controls, automation across development/QA/DevOps, machine-learning architectures (including retrieval-augmented generation), and decommissioning of models and infrastructure. It also follows the same public/private split we described for frameworks: philosophy and decision scaffolds in the open; applied runbooks and client artifacts inside the engagement.
WHAT KIND OF SOLUTION THIS IS
Not every “AI feature” is the same class of system. Mixing them is how teams over-train, over-share, or skip controls.
In Building Scalable and Resilient LLMs for AI Applications we distinguished architecture (how data, training, inference, and monitoring connect) from frameworks (the toolkits that implement that blueprint), and we listed solution types that still matter in 2026: supervised and unsupervised learning, transformers, and retrieval-augmented generation (RAG) versus loading everything into weights. For product language features we add an explicit ladder. You climb it only when evidence says the current rung failed.
Deterministic software — rules, templates, schemas, and policy in code. No model required. This remains the control plane.
Hosted or open foundation model, zero client weights — generic generation. Fast to stand up. High leakage of generic tone. Unacceptable as the only layer when the output must sound like a specific operator.
RAG over a private corpus — retrieve passages from an access-controlled store, then generate or extract. Weights of the foundation model do not change. This is our default for drafting, summarization, and “write in our register” features.
Parameter-efficient fine-tuning (PEFT), typically LoRA — a small adapter trained on approved examples. The base model stays public; the adapter is a private artifact. We schedule this; we do not do it on every request.
Full fine-tune or custom trained model — reserved for cases where retrieval and PEFT cannot meet quality or latency bars. Highest data-governance and decommissioning cost.
Two more product distinctions, from the same governance work:
Recommendation / scoring / classical ML is not generative drafting. Different evaluation, different failure modes, different data minimization.
Decision-support (employment, credit, health, safety) is a higher risk class than drafting copy a human will edit. Controls scale with class. We said this in Establishing Ethical AI Governance in Responsible & Trustworthy AI Development: classify before you build; do not apply one control set to every model.
The feature described here is human-supervised generation: the system proposes language; a reviewer accepts, edits, or rejects; publishing is a separate, deterministic path. The UI must not block on model success. If retrieval or inference is down, extractive snippets or a non-AI template still produce a draft. That is the same rule we use across AI-enabled products: the product works when the model does not.
ARCHITECTURE: TRAINER VERSUS TUNE
We split the system into four planes. If they collapse into one repository with one visibility flag, isolation has already failed.
1. Shared trainer (code)
Ingest adapters (URL, file, spreadsheet), chunking, retrieval, prompt assembly, platform-length transforms, evaluation stubs, and optional training jobs. No tenant identifiers. No production corpora. This is what another team can reuse.
2. Data plane (private corpus)
Source documents classified at ingest: public web, licensed, confidential, restricted. Stored as versioned records (source, channel, hash, ingested_at) in a store that is private by default—object storage and/or a private model-hub dataset. Client text does not land in git. Training and eval splits are derived from this plane, not from chat logs.
3. Model plane
Base: a pinned open-weight or API foundation model (versioned).
Optional: a private LoRA (or equivalent PEFT) adapter in a private registry.
Inference: first RAG; optionally adapter-on-base; never “train on the live request.”
4. Control plane (product + ops)
AuthZ on who may ingest and who may approve. Human-in-the-loop states (draft → approved → scheduled → published). Feature flags. Structured logs that do not persist raw prompts containing sensitive source text. Kill switch and pause flags. Cadence jobs: ingest, evaluate, retrain, offboard.
Retrieval is not a slogan. Operationally it means: tokenize and score chunks (BM25 and/or embeddings), return top-k with source metadata, cap context windows, and prefer extractive composition when the risk class is “do not invent.” Hallucination risk drops because the generator is constrained to retrieved spans; it does not go to zero. Review remains a publish blocker.
PEFT is a promotion, not a default. We promote when RAG quality plateaus on a held-out set of human-approved outputs—not when a stakeholder is impatient. Adapters are tenant-scoped. Sharing the trainer never includes the adapter.
This maps to the ML component list we already use: data sources and pipelines, quality management, training processes, applications, compute/storage, orchestration. RAG sits in the application + pipeline layers. LoRA sits in training + registry. Decommissioning has to know which layer you are turning off.
OPERATIONS ALONG THE LIFECYCLE
In the governance article we walked the Responsible AI / OECD-style lifecycle: plan and design; collect and process data; build and use model; verify and validate; deploy and use; operate and monitor—and we called out decommission as a first-class phase, expanded in Decommissioning AI Systems. Automation in Development, QA, and DevOps is how those phases stay repeatable: CI on trainer code, gated jobs on data and adapters, no TTY-into-prod “just train it.”
How we run this class of system against that lifecycle:
Plan and design
Risk class = drafting with human approval (medium), not autonomous decisions (high). Non-goals are explicit: no live continual learning; no client corpus in the shared module; no publish-without-approve. Success metrics are operational (time-to-draft, approval rate, rollback count)—not vanity generation scores.
Collect and process
Allowlisted sources only. Skip lists for placeholders and internal-only documents. PII and restricted labels fail ingest or are isolated. Idempotent record IDs so re-ingest does not duplicate. Spreadsheet and CMS exports are treated as data pipelines, not as files in source control.
Build and use
v1 = retrieval + extractive or lightly composed drafts against a pinned base model. Training code may exist in the trainer and still be a no-op until gates pass. We do not require GPU capacity to ship the vertical slice.
Verify and validate
Verification: retrieval returns attributed chunks; drafts stay within platform constraints; skip lists hold. Validation: reviewers score a sample for fidelity to source and for fabricated claims. Automation runs the first; humans own the second. That split is the same Dev/QA point we made for model CI: automated tests catch regressions; they do not certify “sounds like us.”
Deploy and use
Trainer ships through normal CI (build → test → deploy). Corpus and adapters deploy through a separate, permissioned path (private registry, workload identity—not long-lived keys in git). Feature flag: if corpus is empty, fall back to non-AI templates so the rest of the product still runs.
Operate and monitor
Ingest is a job, not a side effect of chat. Typical cadence we use in delivery: refresh corpus when source systems publish; quarterly sample review (status, three held-out topics, drift notes); consider PEFT only after a threshold of approved outputs (we use a working number on the order of a few dozen, not “after the first weekend”). Monitor retrieval miss rate, approval-edit distance, and fallback usage. Alert on empty index and on public visibility of a dataset that must be private.
Decommission
Offboard means: revoke registry access, delete or retain-per-contract the corpus and adapter, rotate tokens, confirm the shared trainer has no residual tenant files, and document what was archived versus destroyed. Hardware and GPU notes from the decommissioning post apply if you trained; RAG-only tenants often have little accelerator footprint—do not skip the data offboard because “we never trained.” The corpus is the sensitive artifact.
We do not train on every user interaction. Continual in-production weight updates destroy auditability, make rollback folklore, and are how one tenant’s tone lands in another tenant’s adapter. Ingest on purpose. Retrain on purpose. Same discipline we apply to migrations: one-off jobs, never exec into the running system.
WHAT WE DELIBERATELY DO NOT PUBLISH
Consistent with Why We Use Frameworks — And What We Share (and Don’t):
Client corpora, captions, labels, or evaluation sets
Adapter weights, tokenizer diffs, or registry URIs tied to a tenant
Prompt contracts and routing tables as copy-paste kits
Identity configuration, vault paths, or CI OIDC bindings for a specific cloud account
Step-by-step internal playbooks that replace discovery
Public writing covers the decision scaffold: isolate trainer from tune; RAG before PEFT; HITL before publish; lifecycle including offboard. If a reader can reconstruct a client’s voice from the post, we failed the assignment.
HOW THIS SHOWS UP ON AN ENGAGEMENT
Discovery
What is generating, what is deterministic, what is the risk class, where the source of truth lives, and whether any existing “custom GPT” already mixed tenant data into a shared workspace.
Alignment
Decision memo: RAG-first; private dataset; no git for source text; approval states; cadence; explicit non-goals (for example: no autonomous publish, no training Jobs on a free tier if that is a constraint, no third-party social schedulers). Done gates: one happy path from ingest → retrieve → draft → approve.
Execution
Thin slice. Stub inference behind a flag. Tests for skip lists, retrieval, and “empty corpus still serves a template.” Security from day one: private repos, no secrets in fixtures, config validated at startup.
Handoff
Architecture notes, ingest/retrain/offboard runbook, cadence calendar, and which artifacts are theirs (corpus, adapter, eval set) versus ours (trainer). Their index remains the source of truth for voice. Our module remains reusable.
WHO THIS IS FOR
Fit when you are adding generative drafting or summarization to a product or operations workflow and you need:
Isolation between reusable engineering and tenant data
A retrieval layer you can audit
Human approval before anything leaves the building
An operating cadence that does not require a research org
A decommission path that includes the dataset, not only the endpoint
Poor fit when you want an always-on model that “just learns from production,” when you need an unsupervised decision engine with no reviewer, or when you are unwilling to keep a private store.
Final Thoughts — REACH OUT!
Generative features are software systems with a data plane. The industry failure mode we keep cleaning up is operational: weights, corpora, and product code treated as one blob. Separate the shareable trainer from the private tune. Prefer retrieval until evaluation says you need PEFT. Keep a human on the publish path. Run ingest, retrain, and offboard as jobs you can name in a runbook.
That is how we have been implementing language features alongside the rest of the lifecycle—governance, automation, architecture, and decommissioning—not as a one-off demo.
If you are scoping a drafting or knowledge-assist feature and need the isolation model applied to your stack, visit amtechconsulting.org or use the contact page. Tell us the risk class, where the source documents live, and what “done” means for the next milestone.
Contact
Email: info@amtechconsulting.org