AI and privacy: what "secure by design" actually means in practice

The question comes up in almost every conversation we have with businesses exploring AI. It takes different forms "will our client data end up training someone else's model?", "what if the system leaks sensitive information?", "how do we stay compliant if AI is touching personal data?" but it is always the same underlying concern: we want the benefits of AI without the privacy and security risks that seem to come with it.

It is a legitimate concern, and the right answer is not "don't worry, it's fine." The right answer is to explain how AI systems can be built so that privacy and security are designed in from the start, not patched on afterwards. This is not just a technical preference, it is increasingly a legal requirement and, more practically, the only approach that holds up when someone actually asks hard questions.

The problem with treating security as an afterthought

Most AI security failures are not the result of sophisticated attacks. They are the result of systems built without considering data flows, access controls and exposure surfaces from the start. A model given access to more data than it needs. Outputs not sanitised before being displayed or stored. A third-party API endpoint transmitting sensitive data in plaintext. Prompts that can be manipulated to extract information the system should not reveal.

The OWASP Top 10 for Large Language Model Applications — the reference list for LLM-specific risks, maintained by the Open Web Application Security Project — identifies sensitive information disclosure as one of the most common and serious vulnerabilities in deployed AI systems. The core issue it describes is straightforward: if a model is trained on or given access to sensitive data, and the system is not designed to control what it outputs and to whom, that data can leak. The fix is not a better model — it is a better-designed system around the model.

What "by design" actually means

Article 25 of the GDPR establishes the principle of data protection by design and by default: the requirement to integrate data protection into the design of systems from the beginning, not as an add-on. It explicitly references pseudonymisation and data minimisation as the technical tools for doing this. Anonymised data falls outside the scope of the GDPR entirely; pseudonymised data remains personal data but carries significantly lower risk and regulatory burden.

When we build AI systems, this principle translates into a set of concrete technical decisions made at design time:

• Encryption in transit and at rest. All data exchanged between system components — between the application layer and the model, between the model and any external databases, between users and the interface is encrypted end-to-end. Data stored by the system is encrypted at rest. This is baseline, non-negotiable infrastructure.
• Data masking and anonymisation before the model sees it. Where AI systems process documents, records or inputs that contain personal or sensitive data, that data is masked or anonymised upstream, before it reaches the model. Names, identifiers, account numbers, addresses: these are replaced with placeholders or tokens. The model receives the structure and content it needs to do its job, without ever handling the raw sensitive data. This is not just good practice, it is directly aligned with what GDPR Art. 25 requires and what OWASP LLM02 warns against when violated.
• Minimum necessary access. The model is given access only to the data it needs to perform its specific function — nothing more. Access controls are defined at the system design stage, not adjusted later when a problem appears.

Where the data lives matters as much as how it is handled

The other half of the privacy question is not how data is processed, but where it goes. Many businesses have a reasonable concern about data leaving their environment — going to a third-party model provider, being stored on external servers, being used for training. The answer to this concern is private deployment.

There are two main approaches. The first is on-premise deployment: the model runs on infrastructure owned and operated by the organisation, entirely within its network perimeter. Data never leaves the building. No third party has access to prompts, outputs or any of the information the system processes. This is the most conservative option, appropriate for organisations with strict data sovereignty requirements, which is exactly what regulations like GDPR (data residency) and MAS TPRM frameworks in Singapore increasingly demand.

The second is a Virtual Private Cloud (VPC): the model runs on cloud infrastructure, but within a dedicated, isolated environment accessible only to the organisation. It is not on the public internet. No other tenant shares the infrastructure. Traffic to and from the VPC is controlled, monitored and logged. This approach offers the operational flexibility of cloud without the exposure of a shared environment.

Both options mean that the organisation retains full data sovereignty: the principle that data remains under the control of the entity that generated it, subject to the laws of the jurisdiction it operates in, not the terms of a third-party provider.

The frameworks that back this up

It reflects what the major international frameworks for AI risk management require. The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1) organises AI risk management across four functions: Govern, Map, Measure, Manage. Moreover, it explicitly addresses privacy risks as part of the risk mapping and measurement process. It frames privacy not as a compliance checkbox but as an ongoing operational responsibility across the full AI system lifecycle.

ISO/IEC 42001:2023, the first international management system standard specifically for AI, includes controls on data governance: data quality, lineage and privacy, as well as security in model development and monitoring in production. Organisations seeking to demonstrate rigorous AI governance increasingly use this standard as the reference framework.

What both frameworks have in common is the insistence that security and privacy cannot be handled reactively. They need to be addressed at the design stage, revisited throughout development, and monitored continuously in production. This is the posture we take on every project we build.

What this means for your business, practically

When a business asks us about AI and privacy, the conversation always comes back to the same three questions: what data will the system touch, where will that data go, and who can access it? The answers determine the architecture. If the data is sensitive, we design the masking and anonymisation layer before the model ever comes into scope. If data sovereignty is a requirement, we design for private deployment. If regulatory compliance is part of the picture — GDPR, MAS, ISO 42001 — we design the audit trail, the access controls and the governance documentation alongside the system itself, not afterwards.

The businesses that have the smoothest AI implementations are not the ones that worry the least about security.

If you have questions about how AI can work securely within your specific environment, book a free 30-minute call. We will tell you honestly what the right approach looks like for your situation.