Tracing: see exactly which documents your agent retrieved for an answer

An agent gives an answer that looks wrong. Before you change the prompt, swap the model or re-split your documents, you need to know one thing: what was the model given? There are three usual causes. Search returned the wrong passages, or none. The prompt put good passages together badly. Or the model had the right text and still answered poorly. Each cause has a different fix, and guessing wastes time.

foundation4 answers that question with an execution trace. You ask for one when you run an agent, and foundation4 records the fragments it retrieved and the messages it sent to the model. This post explains what a trace contains, how to read one, and how long it stays available.

What an agent is in foundation4

An agent is a stored prompt template that combines retrieval from a pipeline with a call to a registered LLM. The template is an ordered list of messages with named placeholders. A query placeholder holds a value your application supplies, such as the user's question. A retrieval placeholder runs a search with that value and inserts the text of the fragments it finds.

The agent stores no pipeline and no model. Each execution names both in request headers, so one agent can serve several pipelines and several models. The Agents and prompt templates page describes the full definition.

One detail matters for tracing. foundation4 inserts only the text of the retrieved fragments into the prompt, in rank order. The inserted text carries no fragment identifiers, scores or metadata. The model therefore sees words, not sources. The trace is where the sources live.

Turn tracing on for an execution

Tracing is off by default. You turn it on for each execution by setting tracing to true in the body of POST /agents/{id}/execute:

curl -X POST "$FOUNDATION4_URL/agents/$AGENT_ID/execute" \
  -H "x-api-key: $FOUNDATION4_API_KEY" \
  -H "x-api-key-secret: $FOUNDATION4_API_SECRET" \
  -H "x-pipeline-id: $PIPELINE_ID" \
  -H "x-llm-id: $LLM_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": {"question": "How do I rotate the database credentials?"},
    "classification": ["internal"],
    "stream": false,
    "tracing": true,
    "temperature": 0.2
  }'

The answer comes back in the body as usual. A successful execution also returns the trace identifier in the x-foundation4ai-tracing-id response header. foundation4 stores the trace after retrieval and after the values are inserted into the templates, and before the call to the model.

You then read the trace with the same API key:

curl "$FOUNDATION4_URL/tracing/$TRACING_ID" \
  -H "x-api-key: $FOUNDATION4_API_KEY" \
  -H "x-api-key-secret: $FOUNDATION4_API_SECRET"

The endpoint is described on Get an execution trace.

What a trace contains

A trace has three fields.

Field	Content
traces	For each retrieval placeholder, the fragments it retrieved, in rank order, each with its metadata, text and score
prompt	The messages as sent to the LLM, each with its role and template, after the input values and fragment text were inserted. Messages with include set to false are absent
inputs	The input values of the execution request, by query placeholder name

Together they cover the whole path from question to model input. inputs is what your application asked. traces is what search found. prompt is what the model actually read.

The trace does not contain the model's answer. Your application already has the answer from the execution response, so keep the two together if you want to compare them later.

A worked example

The following trace comes from the docs. It is an execution of a support-answers agent whose similarity search returned one fragment:

{
  "traces": {
    "context": [
      {
        "id": "0192f8c4-5d6e-7f80-9a1b-2c3d4e5f6a7b",
        "classification": "internal",
        "metadata": {"title": "Database credential rotation"},
        "document_id": "0192f8c3-1a2b-7c3d-8e4f-5a6b7c8d9e0f",
        "page_content": "Database credentials rotate every 30 days. An administrator starts an early rotation from the Credentials page of the operations console.",
        "version": 1790740800000000,
        "expired_at": null,
        "score": 0.1834,
        "order_id": 0
      }
    ]
  },
  "prompt": [
    {
      "role": "system",
      "template": "Answer using only the reference text. If the reference text does not contain the answer, say that the answer is not available."
    },
    {
      "role": "user",
      "template": "Reference text:\nDatabase credentials rotate every 30 days. An administrator starts an early rotation from the Credentials page of the operations console.\n\nQuestion: How do I rotate the database credentials?"
    }
  ],
  "inputs": {"question": "How do I rotate the database credentials?"}
}

Read it in three steps.

Step 1: did retrieval find anything?

Look under each placeholder in traces. Here the context placeholder holds one fragment. An empty list would mean the search found no fragment for the classifications and the filter of the request.

An empty list points away from the model. Either the content is not in the pipeline, the reader's classifications do not cover it, or the filter excluded it. Rewording the prompt will not fix any of those. The fragment's document_id, version and classification tell you which document and which version was used, so you can check that the right content was in scope.

Step 2: how close was the match?

The score of each fragment is its cosine distance from the search text, so a lower score is a closer match. The fragments appear in rank order, and order_id gives each one's position.

If the top fragments are about a different subject, or the scores are far from the question, the problem is retrieval quality. Typical fixes sit in the pipeline: how documents are split into fragments, which embedding model is used, or the placeholder's k and threshold parameters. The Search and retrieval page describes the scores.

Step 3: what did the model actually read?

The prompt field shows the exact text the model received. In this example the fragment text was inserted under "Reference text", followed by the question. The reference text says an administrator starts an early rotation from the Credentials page.

Now compare that with the answer. If the answer says something the prompt does not support, or says the answer is not available when it plainly is, the retrieval did its job. The problem is in the template or the model. You might tighten the system message, label the sections more clearly, or try a different registered model.

That is the core use of a trace. In the docs' words, the prompt messages show the exact text that the LLM received, so a missing fact in the answer can be traced to the retrieval or to the model.

Before you run the model: the retrieval dry run

If you suspect retrieval, you do not need to call a model at all. POST /agents/{id}/search runs the agent's retrieval placeholders and returns, for each placeholder, the fragments it would retrieve, with scores. It takes the same pipeline header and the same prompt, classification and filters fields as an execution. This lets you test a fix to content or filters without spending model calls.

Who can read a trace, and for how long

A trace holds the decrypted text of the retrieved fragments. foundation4 limits it in three ways.

Only the key that ran the execution. A trace can be read only with the API key that ran the execution. Any other key receives HTTP 404 with the message Trace with id <id> not found. The endpoint needs no permission on an object, because the key itself is the check.

Encrypted while stored. The trace is kept in the deployment's Redis-compatible cache, encrypted with the deployment's application secret. The Data protection page lists it alongside the other values encrypted with that secret.

Held for 60 minutes. The cache keeps the trace for 60 minutes after the execution. Reading it does not extend that period. After 60 minutes, a request for the trace receives the same HTTP 404.

A trace is a short-lived debugging and display record. It is not a history of every execution. If your team needs to keep what an agent retrieved, your application reads the trace within the hour and stores what it needs under its own retention rules.

The prompt field has one more use. It is the text foundation4 sent to the model server, so a reviewer can see exactly what content reached the model for each answer.

How this differs from OpenTelemetry traces and request IDs

foundation4 also provides Prometheus metrics and OpenTelemetry tracing, and every response from an API operation carries an x-request-id header. Those signals describe requests: the route, the status, the duration, and the span in your distributed trace. They carry no fragments and no prompt text. An agent execution trace is a separate record about content: which fragments one answer was built from. The two meet in your application. If it keeps both the x-request-id and the x-foundation4ai-tracing-id headers of an execution response, it can connect the agent trace to the OpenTelemetry trace of the same request. Another post covers request IDs and OpenTelemetry in detail, and the docs page on signals lists every signal.

Showing sources to your users

The same trace that helps you debug also lets your application show users where an answer came from. The knowledge base agent solution uses it this way:

1. Store the article title and link in each document's metadata.
2. Run the agent with tracing set to true.
3. Read the trace with the identifier from x-foundation4ai-tracing-id.
4. List the title and url from the metadata of each fragment below the answer.

Because the model receives fragment text and no metadata, it can name a source article only if the fragment text contains the title. The solution therefore starts each document with its article title. That way the answer can cite the article in its own words, and the trace supplies the link.

Remember the access rule here. Only the key that ran the execution reads the trace, so the component that calls the agent is the one that fetches the sources.

Agent limits worth knowing

A few points from the feature status page shape how you use agents today:

• Agents retrieve with similarity and maximal marginal relevance (MMR) placeholders. Full-text and hybrid placeholders are planned.
• Agents are stateless. Each execution is independent, and your application supplies any conversation history in the input values.
• Agents call the model through the OpenAI Responses API and need an API key on the registered LLM. Any non-empty value works for a model server without authentication. A model server that implements only the Chat Completions API does not work with agents in the current release.

Summary

When an agent answer looks wrong, a trace tells you what the model was given. traces shows what search found and how close it was. prompt shows the exact text the model read. inputs shows what your application asked. With those three, you can tell a retrieval problem from a prompt or model problem before you change anything.

To see agent tracing on your own content, request a briefing.