One key per classification: what foundation4 encrypts, and what it doesn't

Security reviews of retrieval systems tend to stall on one question: what exactly is encrypted? "Everything" is never the true answer, and a vendor who says it leaves the reviewer to find the gaps alone. This post is the honest map for foundation4: what is encrypted, with which key, where that key lives, and what is not encrypted and why. The limits are the useful part.

The short version

Stored text is encrypted with a separate key for each classification. Credentials are encrypted or hashed. Vectors, full-text index entries and metadata are not encrypted, because PostgreSQL reads them to answer searches and filters. The classification keys live in the same database as the text they protect. Database access, storage encryption and backup handling therefore belong to the operator.

The map

foundation4 encrypts selected data at the application layer, before it reaches PostgreSQL or the Redis-compatible cache. Storage-level encryption, such as encrypted volumes or an encrypted database service, is separate. The operator configures it in the platform. This table follows the Data protection page.

Data	Where	Protection
Fragment text	PostgreSQL, embeddings schema	ChaCha20-Poly1305, one key for each classification
Classification keys	PostgreSQL, the same database as the text	None at the application layer
Vectors	PostgreSQL	None at the application layer
Full-text index entries	PostgreSQL	None at the application layer
Metadata	PostgreSQL, a copy on each document and fragment	None at the application layer
Identifiers, states and configuration	PostgreSQL	None at the application layer
LLM API keys	PostgreSQL, public schema	Fernet, keyed by the application secret
API key verification results	Cache, 15 minute expiry	Fernet, keyed by the application secret
Execution traces	Cache, 60 minutes	Fernet, keyed by the application secret
API key secrets	PostgreSQL, public schema	Argon2id hash
Frequently read objects	Cache, minutes	None at the application layer

How fragment text is encrypted

foundation4 splits each document into fragments. The worker encrypts the text of every fragment with ChaCha20-Poly1305 before it writes the fragment to PostgreSQL. This is authenticated encryption: it hides the text and detects changes to it.

Keys. Each classification of a pipeline has a 256-bit key. The key is generated from the operating system's random number generator when the classification is created. Adding a classification that already exists keeps the existing key.

Nonces. Each fragment is encrypted with a new random 96-bit nonce, stored next to the ciphertext.

Use. The worker encrypts each fragment with the key of its document's classification. The API server reads the key in the same database query that reads the fragment. It decrypts the text wherever a request uses it, such as search results and fragment lists requested with contents=true.

Integrity. The Poly1305 tag detects altered ciphertext. A fragment that fails the check makes the request return HTTP 500 with the message Decryption error. Tampered text is refused, not served.

Lifetime. A key exists as long as its classification. Removing a classification deletes the key together with every document and fragment that carries it. There is no operation that replaces the key of an existing classification. If your policy requires key rotation, plan for that limit.

What the keys do, and what they don't

The keys separate the ciphertext of each classification. They do not decide which fragments a request can read. The API server reads the key of any fragment a request returns. Which fragments come back depends on the permissions of the API key on the pipeline and on the classifications the search request names. In other words, results are scoped to each reader by the classifications and metadata your application supplies. The Access control page covers that side.

What is not encrypted, and why

Some data has to stay readable to PostgreSQL, because the database compares it directly. A database cannot filter or rank on values it cannot read.

• Vectors. Nearest-neighbor search compares vectors inside PostgreSQL. Vectors are computed from the fragment text.
• Full-text index entries. For pipelines with full-text search, PostgreSQL stores the normalized words of each fragment with their positions. Full-text queries match against these entries. Full-text search is chosen per pipeline at creation, so a pipeline without it has no such entries.
• Metadata. Filters compare metadata values inside PostgreSQL. Each document and each fragment carries a copy of the document's metadata.
• Identifiers and states. Document identifiers, external identifiers, versions, expiry times, processing statuses and classification names.
• Configuration. Names, descriptions and settings of pipelines, models, text splitters, taxonomies, API keys and permissions, and LLM registrations apart from the API key.

One practical rule follows: sensitive values belong in the text, not in metadata. Put the room identifier or the project code you filter on in metadata. Keep the content itself in the text.

One more item to note. Embedding model settings, including any credentials for a remote embedding endpoint, are stored as plain configuration. Any API key with read permission on embedding models can read them, so grant that permission with care.

Where the keys live

The classification keys are stored as raw bytes in the classifications table of each pipeline, in the same database as the text. They are not encrypted with the application secret, and foundation4 does not use an external key management service.

Here is what that design protects against, and what it does not.

• It protects fragment text in a copy of the fragment tables that excludes the classifications tables. An export of the text tables alone gives ciphertext.
• It does not protect fragment text from a party that can read both. That includes a database role with read access to the embeddings schema, a backup of the whole database, and anyone with full access to the running database.

The encryption is one layer. Database access control is the layer that carries the weight, and it is yours to set.

The application secret and the master key

The application secret is a Fernet key: 32 random bytes in URL-safe base64. Fernet encrypts with AES using a 128-bit key in CBC mode, and signs the result with HMAC-SHA256. It protects three things: LLM API keys, API key verification results in the cache, and execution traces. It does not protect fragment text or API key secrets.

No endpoint returns the API key of an LLM. Reading a registration with read permission on the LLM returns the registration without it:

{
  "id": "3f0c9b6e-2d4a-4c1e-9b7f-5a8d2e6c1f04",
  "name": "internal-general",
  "description": "General-purpose model on the internal model server",
  "endpoint": "<model server URL>",
  "model": "example-model-8b-instruct"
}

foundation4 has no operation that re-encrypts stored data under a new application secret. A changed secret therefore has the same effect as a lost one. Each LLM's API key must be set again with PATCH /llms/{id}. Cached verification results are discarded and computed again. Traces written before the change return Decryption error until they expire. Treat the application secret as a root credential.

API key secrets are generated from 36 random bytes and returned once, when the key is created. foundation4 stores an Argon2id hash with a random salt. No endpoint returns the secret or the hash, and a lost secret cannot be recovered. The key is replaced instead.

The master key is the administration key of the installation. The installation stores an Argon2id hash of its secret in PostgreSQL. The API server and the workers verify the master key at startup. Its identifier and secret are fixed after the first installation, so the operator records both in the organization's secret store. The Secrets and keys page has the procedures.

In transit: why verify-full matters

The connection to PostgreSQL carries the classification keys with each read of fragment text. For pipelines with full-text search, it also carries the fragment text and the query text of full-text searches. Text that is encrypted at rest crosses this connection next to its key. That connection needs TLS with a checked certificate.

The sslmode parameter of the database URL decides this:

sslmode	Encryption	Server certificate check
disable	None	None
prefer (default)	When the server offers TLS	None
require	Always	None
verify-ca	Always	Signed by a trusted certificate authority
verify-full	Always	Signed by a trusted certificate authority, and issued for the host name in the URL

Only verify-ca and verify-full check the server certificate. A production deployment uses verify-full, with a private certificate authority supplied where needed. A database session with administrative rights can then confirm that every foundation4 connection uses TLS:

SELECT a.usename, s.ssl, s.version
FROM pg_stat_ssl s JOIN pg_stat_activity a USING (pid)
WHERE a.datname = '<database>';

Every row should show ssl as t. The database server can also reject unencrypted connections outright. The Database and Security hardening pages give the steps.

The other connections:

• Clients. The API server serves plain HTTP inside the cluster. TLS for client applications and browsers terminates at the ingress controller or the Gateway. Every request carries an API key secret, so every client path uses TLS.
• Inside the namespace. Connections to the cache, to NATS JetStream and to the gRPC sidecar run without TLS. Network policies limit them to the pods that use them. The charts ship none for the foundation4 pods, so the operator applies them.
• Model servers. Requests to embedding and LLM servers carry fragment text, queries and prompts. They use TLS when the endpoint URL starts with https://. Content leaves your perimeter only if an administrator registers a model server outside it.

What a database backup contains

A backup of the PostgreSQL database holds everything: the encrypted fragment text together with the classification keys, the vectors, full-text index entries, metadata and configuration, the API key hashes, and the LLM API keys encrypted with the application secret. It carries the same sensitivity as the database. Inside a full backup, the fragment encryption gives no protection.

The LLM API keys in a backup can only be decrypted with the application secret in effect when the backup was taken. Keep that secret with each backup set, but stored apart from the backup files.

Deletion has limits too. Deleting a document removes its versions, fragments, vectors and full-text entries from the database. It changes no existing backup. Deleted content stays in older backups and archived write-ahead logs until backup retention removes them. Expired versions also keep their encrypted text until the document is deleted. Backup retention is part of any erasure plan.

What the operator owns

Asset	Controls
PostgreSQL access	A dedicated database, roles limited to foundation4 and the database administrators, network access limited to the foundation4 pods
Storage and backups	Storage-level encryption, restricted access to backup files, a retention period that matches deletion obligations
Application secret and master key	Kubernetes Secrets encrypted at rest and readable only by operators, with copies in the organization's secret store apart from the backups
NATS JetStream and the cache	Network policies that admit only foundation4 pods. NATS JetStream holds submitted text for up to 24 hours, so its volumes get storage-level encryption
Transport	TLS at the ingress or Gateway, verify-full to PostgreSQL, https:// model endpoints on shared networks
Logs and trace exports	Access limited to the operators of the installation
Client API keys	One key per client application, kept in that application's secret store

Why we write it down

A reviewer needs to know where each protection ends, not only where it starts. foundation4 encrypts stored text with a separate key for each classification and detects tampering. It keeps vectors, index entries and metadata readable, because that is how the database scopes each search. The full detail is on the Data protection and Security at a glance pages.

To walk through these controls against your own environment, request a briefing.