Private model gateways: keeping your customer data yours

When a small business asks ChatGPT a question that includes a customer's name, address, or medical information, that data leaves the business. It travels to OpenAI. Depending on the API tier and the date of the agreement, it may or may not be used to improve the model. The privacy posture for casual consumer use of these tools is, charitably, ambiguous.

For the businesses we work with, ambiguity is not acceptable. A medical practice cannot have patient data in a model provider's training set. A law firm cannot have client communications somewhere they don't control. Even for businesses without regulated data, basic professional courtesy requires that customer information not become someone else's training data.

The architecture that fixes this is a private model gateway. Instead of your applications calling OpenAI or Anthropic directly, they call a private endpoint that we (or you) operate. The endpoint authenticates, logs, applies your data policies, optionally redacts sensitive fields, and forwards the request to the model under enterprise terms that explicitly forbid training use of your data.

The gateway also gives you observability. Every request, every response, every token spend, every model used — logged in a journal that's yours, on infrastructure you control. When something behaves oddly, you can audit it. When a regulator asks what happened to a particular customer's data on a particular day, you can answer.

Setting up a gateway is not technically hard. It's a couple of weeks of work for a competent engineer, plus the operational policies. It is, however, the kind of work that's only done when someone makes it a priority — which usually means it gets skipped by businesses doing AI on the cheap. We make it part of the foundation of every engagement, because the cost of getting it wrong is much higher than the cost of getting it right.