Vector Database
Architecture & the
RAG Pipeline

A traditional database looks for exact matches. You search for a customer ID, a product name, or a date range, and it returns rows where the values match. That works for structured business data. It does not work when you want to ask an AI system something like "find me all the documents that are similar in meaning to this paragraph."

Vector databases solve this. Instead of storing data as rows and columns, they store data as long lists of numbers called vectors. Each vector represents the meaning or properties of a piece of content. When you search, the database does not look for an exact match. It calculates which stored vectors are mathematically closest to your query vector. The closest matches are the most semantically similar.

This is called similarity search, and it is the foundation of how RAG systems retrieve context before generating a response.

Common vector databases used in production RAG systems include Pinecone, ChromaDB, pgvector (a PostgreSQL extension), MongoDB Vector Search, and FAISS (Facebook AI Similarity Search). ChromaDB and FAISS are popular during development because they are easy to set up. Neither was built with enterprise access control in mind, which creates security gaps when teams ship them to production without adding controls on top.

An embedding is a numerical representation of content. You feed text, an image, or a document into an embedding model, and it outputs a long array of decimal numbers. That array is the embedding. Documents with similar meaning produce arrays that are mathematically close together. Documents with very different content produce arrays that are far apart.

The numbers themselves do not have human-readable meaning. You cannot look at position 47 in an embedding array and know what it represents. The meaning is in the relationships between all the numbers together.

The similarity between vectors is typically measured using cosine similarity, which measures the angle between two vectors rather than the distance between them. A cosine similarity of 1.0 means identical. A score of 0.0 means completely unrelated. Most similarity searches retrieve the top-K vectors above a similarity threshold.

Vector embeddings typically have hundreds or thousands of dimensions. This causes a mathematical problem that affects both performance and security: the curse of dimensionality.

As dimensions increase, the volume of the vector space grows exponentially. Data points spread out so much that they all appear approximately equidistant from each other. Distance measures that work well in low-dimensional spaces become unreliable in high-dimensional ones. The "nearest neighbor" to your query might only be marginally closer than the 1,000th nearest neighbor.

Dimensionality reduction techniques like Principal Component Analysis (PCA) compress high-dimensional vectors into lower-dimensional representations while preserving as much structure as possible. This improves search efficiency, but also changes the security properties of the embeddings.

The practical security implication is this: because distance measures are less reliable at high dimensions, it is harder to reconstruct the original content from a stolen embedding vector. But it also means that two different pieces of content can have similar embeddings, making it possible to craft a document that retrieves alongside a target document through a poisoned injection.

At the core of every vector database is an index. The index determines how vectors are stored, how similarity search is performed, and how fast results are returned. Most vector databases use approximate nearest neighbor (ANN) algorithms rather than exact search because exact search across millions of high-dimensional vectors is computationally impractical.

The choice of index affects more than performance. Poorly configured indexes can leak information through similarity patterns and expose systems to query-based attacks. Security teams auditing vector databases need to understand what index type is in use and what its specific failure modes are.

Vector databases store more than just embeddings. They hold metadata, document references, and identifiers that connect vectors back to source content. These elements are spread across different storage layers, each with its own performance profile and security requirements.

Most security reviews focus on the vector store as a single entity. The reality is that sensitive data is distributed across at least three distinct layers, each of which can be a security gap if left unaddressed.

RAG stands for Retrieval-Augmented Generation. The idea is straightforward: instead of relying only on what the model learned during training, you retrieve relevant documents at query time and inject them into the prompt as context. The model generates its response based on both its training and the retrieved content.

This is why RAG systems can answer questions about company policies, private documents, real-time data, and anything else that was not in the model's training data. The retrieval step brings in the relevant context. The generation step uses that context to produce the answer.

Every step in this pipeline is a potential attack surface. Source document ingestion can be abused to introduce poisoned content. The embedding model can be a supply chain risk if it comes from an untrusted source. The vector store holds sensitive data in plaintext in most default configurations. The retrieval step is the primary injection surface: retrieved content goes directly into the LLM's context window. And the generation step can leak sensitive retrieved content through the response.

The security mental model for RAG is not "protect the model." It is "protect every step of the pipeline." This is what the remaining modules in this path cover.

A production RAG system is not just documents and a vector database. It has eight distinct layers, each with its own security properties and failure modes. Security teams auditing RAG systems need to cover all eight. Most assessments stop at the vector database and miss the rest.

LLM caches store the outputs of recent queries so that identical or near-identical queries can be answered instantly without running a full retrieval and generation cycle. Tools like Redis and GPTCache are commonly used. In high-traffic systems, this can significantly reduce latency and API costs.

The security problem is straightforward: the cache stores LLM outputs that often contain sensitive retrieved content. If user A asks "What is our salary band for senior engineers?" and that query hits the vector store and retrieves a sensitive HR document, the response (including the retrieved content) may be stored in cache. If user B sends a sufficiently similar query, they may receive the cached response, which contains sensitive information they do not have authorisation to see.

Three things every RAG security review should check on the LLM cache:

1. Cache key design. If the cache key is based purely on the query string, slightly different phrasings of the same question produce different cache keys. This means sensitive responses get cached multiple times under different keys. It also means an attacker can iterate through variations to find a cached sensitive response.

2. Cache entry scope. Is the cache global (shared across all users) or per-user? A global cache in a multi-tenant system leaks responses across tenant boundaries. Per-user caches are safer but more expensive to maintain.

3. Cache entry lifetime. How long do cached entries persist? A cached response containing sensitive data that was valid at time T may no longer be valid at time T+30 days if the underlying document has been updated or the user's permissions have changed.

Vector databases influence what information AI models see and how they behave. Weak access control means any component with query access can retrieve any document, regardless of classification or ownership. In a multi-tenant environment this is a data leakage path. In an agentic environment it is a privilege escalation path.

There are two main access control models relevant to vector databases, and they work best in combination.

API keys are the most common authentication mechanism for vector databases and they are also one of the most common causes of unauthorized access. The problem is not usually that teams use API keys, it is how they use them.

Encryption for vector databases operates at three distinct layers. Most organisations implement the first two. Very few implement the third, which is why data breaches at the vector layer are still possible even in systems that appear well-secured.

Encryption is only as strong as the key management around it. If encryption keys are stored alongside the data they protect, an attacker who accesses the storage layer gets both the data and the keys. Effective key management separates key storage from data storage and enforces rotation, auditing, and access controls on keys independently.

In practice this means using a dedicated key management service: AWS KMS, Google Cloud KMS, Azure Key Vault, or any provider that supports the KMIP standard. Keys should be rotated on a schedule, and key usage should be logged so you can detect if a key is used in unexpected ways.

Where you deploy a vector database changes your attack surface. The same vector database product can have very different security properties depending on whether it runs as a managed cloud service, inside a private cloud, or on-premises. This is not a minor operational detail. It determines who is responsible for each security control and where the gaps are most likely to appear.

Three security questions that every deployment decision must answer:

Network exposure. Is the vector database endpoint publicly reachable, or only accessible from within a private network? A public endpoint without proper authentication exposes the entire retrieval layer to the internet. This is one of the most common misconfiguration categories in vector database deployments.

Tenant isolation. In a shared deployment (multiple teams or customers using the same vector database instance), are namespace boundaries enforced at the database level or only at the application level? Application-level isolation fails if a bug or injection attack bypasses the application logic.

Monitoring coverage. Can you detect misuse of the deployment? A vector database with no query logging or anomaly detection can be exfiltrated over weeks through low-volume systematic queries without triggering any alerts.

Every component of a standard RAG pipeline operates on data in plaintext. Documents are ingested in plaintext. Embeddings are generated from plaintext and stored as plaintext vectors. Queries are sent to the embedding model in plaintext. Retrieved documents are injected into the LLM prompt in plaintext. The LLM response contains plaintext summaries of retrieved content.

This means that any component in the pipeline that is compromised exposes the underlying data. A misconfigured vector database leaks every embedding in it. A compromised orchestration layer sees every query and every retrieved document. A cache with weak access control exposes historical responses.

VectaX from Mirror Security addresses the plaintext problem using Similarity-Preserving Search. This is the key property that makes encrypted vector search possible: the encrypted embeddings maintain enough mathematical structure that similarity comparisons still work, even though the actual values are encrypted. You can run a similarity search against encrypted vectors without ever decrypting them.

The access control gap is addressed through VectaX's built-in RBAC, which enforces permissions at role, group, and department levels at query time, not just at ingestion time. An analyst in the finance department retrieving from a shared vector store cannot retrieve documents that belong to the legal department's namespace, even if they know exactly how to phrase the query.

Module 5 covers the cryptographic foundations of this in detail: how Fully Homomorphic Encryption (FHE) enables encrypted inference, what format-preserving encryption (FPE) does to metadata, and how these combine into a complete secure RAG architecture.