Encrypted Inference
& Encrypted
Vector Memory

Modules 3 and 4 secured data at ingestion, in storage, and at access control. But there is one step in every AI system that still operates entirely on plaintext, even in well-secured deployments: inference. The moment when the AI model processes inputs, activates layers, and generates outputs.

Even if your vector database encrypts embeddings at rest, the retrieval service decrypts them before feeding them into the LLM prompt. Even if your pipeline uses TLS throughout, the AI provider's inference server processes plaintext tokens. Even if your access controls are airtight, the model weights and layer activations operate on unencrypted data inside the provider's infrastructure.

This is the last plaintext gap. An attacker who compromises the inference server, or an AI provider whose security posture you must accept on trust, can see every query, every retrieved document, and every generated response.

Homomorphic encryption (HE) allows computation directly on encrypted data. The result of the computation, when decrypted, matches what you would have gotten by computing on the plaintext. The server performing the computation never sees the actual values. There are three generations of this technology, each with different capabilities and performance trade-offs.

Every FHE ciphertext contains a small amount of random noise added during encryption. This noise is what makes the scheme secure: without it, an attacker could solve for the plaintext from the ciphertext. But noise grows with each operation. Add two ciphertexts and you get a little more noise. Multiply them and noise roughly squares. Keep operating and eventually the noise is so large the ciphertext cannot be correctly decrypted.

Craig Gentry's 2009 breakthrough introduced bootstrapping: a procedure that takes a noisy ciphertext and produces a fresh ciphertext with much less noise, representing the same plaintext value. Bootstrapping is itself a homomorphic computation, which makes it mathematically remarkable. By periodically refreshing ciphertexts, you can chain unlimited operations, enabling full neural network inference on encrypted data.

VectaX provides noise control parameters through the SDK. Teams can tune how much noise buffer to maintain before bootstrapping triggers. A tighter buffer means more frequent bootstrapping, more compute, but higher accuracy. A looser buffer means fewer bootstrapping operations and lower compute cost but slightly less floating-point precision. For embedding similarity search, the precision requirements are well understood, and VectaX's defaults are calibrated for high accuracy at practical latency.

VectaX encrypted inference is not FHE applied to vectors in isolation. It combines three components: FHE for the encryption scheme, vector encryption for similarity-preserving storage, and RBAC for policy-controlled access. All three are necessary. FHE alone lets you compute on encrypted data but does not preserve the similarity relationships needed for vector search. Vector encryption alone protects stored embeddings but not the inference computation. RBAC alone controls access but does not prevent extraction attacks through repeated queries.

AI agents build up memory as they work. They store conversation history to maintain context across turns. They cache retrieved document embeddings to avoid redundant lookups. They write summaries, intermediate reasoning states, and task progress to memory stores. In most deployments, all of this is stored in plaintext or with only standard database encryption at rest.

Encrypted vector memory changes this. Every memory entry stored by the agent is encrypted at the moment of creation, before it is written to any storage system. The agent can later retrieve and reason over its memory because VectaX's Similarity-Preserving Search keeps the embeddings searchable. But the memory store itself, and any infrastructure between the agent and the store, never holds plaintext.

This matters most for long-running agents that accumulate sensitive information over many interactions. Without encrypted memory, the memory store becomes a consolidated plaintext record of everything sensitive the agent has ever retrieved or been told. With encrypted memory, even if the memory store is breached, the attacker sees encrypted blobs that cannot be reversed without the decryption key.

The context window is the most sensitive surface in a RAG system at inference time. It contains the system prompt (which may include confidential instructions), the retrieved documents (the sensitive content you went to all this trouble to protect), the conversation history (which may contain personal information shared by the user), and the user's current query.

In a standard RAG deployment, all of this travels to the AI provider's inference server in plaintext. The provider can technically read every element of the context window, even if their policy says they do not.

VectaX encrypted context windows keep retrieved document embeddings and conversation history entries in encrypted form until the last possible moment. Decryption happens only at an authorised endpoint. This is particularly valuable when the system prompt contains proprietary instructions that the organisation does not want to expose to the model provider's infrastructure.

Encrypted inference protects the input data. But there is a second concern: protecting the model itself. Model weights represent significant intellectual property. An organisation that fine-tunes a foundation model on proprietary data has built something valuable. Exposing those weights to provider infrastructure, or to users through model extraction attacks, is a real risk.

VectaX addresses model security through two mechanisms. First, encrypted inference means model weights can be deployed to infrastructure that never processes plaintext inputs or outputs, reducing information available for extraction attacks. Second, RBAC on predictions means access to specific model endpoints can be controlled at the same role, group, and department level as data access.

Model extraction attacks work by querying a model repeatedly with carefully chosen inputs to reconstruct its decision boundaries. Encrypted inference makes this harder because the attacker sees encrypted outputs that must be decrypted before they can be used to train a surrogate model. Combined with rate limiting and query pattern monitoring from Module 6, this significantly raises the cost of extraction attacks.

Sometimes the most valuable RAG systems are built on data from multiple organisations. A hospital network wants to build a diagnostic AI using patient records from 12 hospitals. A financial consortium wants to detect fraud patterns across member banks. In both cases, each organisation has data too sensitive to share with the others, but the combined dataset would produce a much better model.

Secure Multiparty Computation (SMPC) solves this. It allows multiple parties to jointly compute a function over their combined inputs without any party revealing their individual data to the others. Each party learns only the final result of the computation, not the inputs from other participants.

SMPC and FHE complement each other. FHE allows a single party to process encrypted data without decrypting it. SMPC allows multiple parties to combine their data without sharing it. Used together, as the Cisco 2024 white paper on securing vector databases notes, FHE can reduce the communication overhead typically associated with SMPC, because homomorphically encrypted data can be processed by a single party without constant back-and-forth between participants.

For teams evaluating whether to build their own FHE implementation or use a solution like VectaX, understanding the open-source library landscape is useful. These libraries provide the cryptographic primitives. They do not provide AI-optimised implementation, noise control tuning, RBAC integration, or production deployment tooling.

The Model Context Protocol (MCP) is a standard that lets AI models like Claude connect to external data sources through a defined interface. It is sometimes described as the "USB-C port for AI": a universal connector that any MCP-compatible tool can plug into any MCP-compatible model without custom integration work.

VectaX provides an MCP server that acts as a secure vector database backend. When Claude Desktop connects to the VectaX MCP server, it gains access to semantic search over your vector store with all VectaX security controls enforced: similarity-preserving FHE encryption, RBAC at role, group, and department level, audit logging of all interactions, and compliance-ready monitoring.

Encrypted inference changes the compliance conversation for AI in regulated industries. The fundamental problem with using third-party AI services for sensitive workloads has always been that you are handing plaintext data to an external infrastructure provider. FHE removes this objection. The provider's infrastructure processes ciphertext, not your data.