Skip to content
Nasir Nasir-Ameen
Notes

AI Document Q&A Agent

Retrieval is unfinished when citations disappear during synthesis

Keeping chunk, source and page identifiers attached while an optional model turns retrieved text into an answer.
6 min read

A retrieval system can find exactly the right passage and still produce an answer nobody can check, if the link back to the source is lost on the way to the screen.

Where the link usually breaks

The common shape is: find some passages, paste them into a prompt, and ask a model to answer. The model returns fluent text. Which sentence came from which passage is now a matter of trust.

Keeping the source attached

In my document Q&A agent, a document is split into bounded chunks, and each chunk keeps its source and its position. Retrieval returns passages together with those identifiers, as structured data.

The answer is built from that structure. Citations are tied to the passages that supported the response, and the synthesis step is not allowed to introduce a source of its own.

The model is optional

By default the agent produces answers without calling a language model at all, using a deterministic synthesiser. A model can be switched on, and it consumes the same retrieval result under the same contract.

That was deliberate. It means the system runs and is tested without a paid service, and it means changing the synthesiser cannot change whether an answer is traceable.

Not rescanning everything

Passages are indexed when a document is ingested, so a second question about the same documents uses the index instead of reading every chunk again. Storage is bounded so a large upload cannot grow without limit.

What is tested

Chunking, retrieval and citation construction each have their own tests, and none of them need a model to be available.

A useful system of this kind does not merely sound grounded. It keeps the path back to the material that made the answer possible.