Skip to content

Citations

A list of source documents next to an answer is not a citation. It says the answer came from somewhere in these files, which is exactly the part a reader cannot check without reading the files. Cairn cites a span, and verifies it.

What a citation is here

Every answer is a set of claims, and every claim carries citations. A citation names a chunk and a character range inside it, plus the quoted text itself:

{
  "text": "A customer has 14 days to request a credit for a damaged delivery.",
  "citations": [
    {
      "chunk_id": "7c1f2a9b4e6d8035",
      "start": 112,
      "end": 178,
      "quote": "A customer has 14 days to request a credit for a damaged delivery"
    }
  ],
  "supported": true,
  "support_score": 0.86
}

The offsets are relative to the chunk's text, and the chunk itself carries start and end into its document, so any citation resolves to an absolute position in the source file and to the page it falls on. That chain is why parsers record page offsets and why chunkers must satisfy document.text[chunk.start:chunk.end] == chunk.text exactly.

Verification, not trust

supported is filled in by the verifier, never by the model. Two checks have to pass.

The quote must exist. The cited text must be a verbatim substring of the chunk it names. A model that paraphrases while claiming to quote, or invents a chunk id, fails here and the claim is marked unsupported.

The quote must support the claim. A real quote from a real chunk can still be attached to a claim it does not back. The default check is lexical: the share of the claim's content tokens (lowercased, stopwords removed, numbers kept) that appear in the quote, with a threshold of 0.5. Numbers are kept deliberately, because a claim that says fourteen days cited to a quote that says thirty is the failure mode that matters most. When a judge provider is configured and is not the stub, an entailment check through that provider replaces the lexical test.

A claim that fails either check is not hidden. It appears in the answer with supported: false, and it moves two numbers in the report.

The two numbers

Metric What it counts What it catches
citation_validity verified claims over all claims how much of what the system asserts is actually evidenced
faithfulness answers where every claim verified, over answered how often a whole answer is clean

They are different questions. A system can hold high citation validity while almost every answer contains one bad claim, and faithfulness is what exposes that. An answer with no claims at all is not faithful, because an answer without evidence has offered nothing to check.

Citation validity is one of the things the gate watches. A change that improves the wording while quietly citing less is a regression, and it is refused with the numbers in the reason.

Reading them

cairn ask "What temperature range must frozen goods be held at?"

Each claim is printed with its verdict, its source and its page, and the quote itself, so the check a reader would do by hand has already been done and its result is on the screen.

Why span level

Three reasons, in the order they matter.

  1. A reader can check it in seconds. A file name asks for a search; a quote with a page asks for a glance.
  2. The harness can check it automatically. Verification is only possible because the citation is precise enough to test. Document-level citations cannot be verified by anything except a person.
  3. It constrains generation. A model asked for verbatim quotes with chunk ids produces different, more grounded text than one asked to mention its sources.

Limits

Verification proves the quote exists and overlaps the claim. It does not prove the document is true, that the passage was the right one to use, or that something important elsewhere in the corpus was missed. Recall metrics speak to the third; nothing here speaks to the first two. A verified citation means the system did not make it up, which is a floor, not a ceiling.