Yesterday's RAG answer was a paragraph. Today: the same answer with citations — [chunk-0] markers showing which sources the model used. Auditable, reproducible, and a forcing function on the model: if it can't cite, it shouldn't claim.
context_block = "\n".join(f"[{cid}] {entry['text']}" for cid, entry in retrieved.items())
prompt = f'''Use the context to answer the question. Cite sources as [chunk-N] in your answer.
Context:
{context_block}
Question: {query}
Answer:'''The trick is just labeling the chunks before stuffing?
Half of it. The other half is the prompt instruction — "cite sources as [chunk-N]". With both, the model includes markers like Paris is the capital of France [chunk-0] and you can grep for them.
And we verify the citation?
Regex on the output. If \[chunk-\d+\] doesn't appear, the answer didn't cite — flag it. Production: extract the citation and check the cited chunk actually contains the claimed fact (advanced verification, beyond this lesson).
def rag_with_citations(query, retrieved_items):
# retrieved_items: list of (chunk_id, text)
context_block = "\n".join(f"[{cid}] {text}" for cid, text in retrieved_items)
prompt = f'''Answer the question using only the context. Cite sources as [chunk-N] inline.
Context:
{context_block}
Question: {query}
Answer:'''
return Agent(model).run_sync(prompt).output[chunk-0] marker prefixes each chunk in the context blockMiss either, and citations don't appear.
import re
cited_ids = set(re.findall(r'\[(chunk-\d+)\]', answer))
provided_ids = {cid for cid, _ in retrieved_items}
stray = cited_ids - provided_ids
if stray:
raise ValueError(f"answer cited chunks not in context: {stray}")This catches the failure mode "model fabricates a citation it didn't actually receive" — the regex pulls citation markers from the answer and confirms they all appeared in the retrieved chunks. Beyond this lesson, but worth knowing the pattern.
We assert the answer contains at least one [chunk-N] marker — minimum bar for "the model is citing". Deeper checks (correct citation, claim-supports-citation) are eval-suite territory (L11).
Yesterday's RAG answer was a paragraph. Today: the same answer with citations — [chunk-0] markers showing which sources the model used. Auditable, reproducible, and a forcing function on the model: if it can't cite, it shouldn't claim.
context_block = "\n".join(f"[{cid}] {entry['text']}" for cid, entry in retrieved.items())
prompt = f'''Use the context to answer the question. Cite sources as [chunk-N] in your answer.
Context:
{context_block}
Question: {query}
Answer:'''The trick is just labeling the chunks before stuffing?
Half of it. The other half is the prompt instruction — "cite sources as [chunk-N]". With both, the model includes markers like Paris is the capital of France [chunk-0] and you can grep for them.
And we verify the citation?
Regex on the output. If \[chunk-\d+\] doesn't appear, the answer didn't cite — flag it. Production: extract the citation and check the cited chunk actually contains the claimed fact (advanced verification, beyond this lesson).
def rag_with_citations(query, retrieved_items):
# retrieved_items: list of (chunk_id, text)
context_block = "\n".join(f"[{cid}] {text}" for cid, text in retrieved_items)
prompt = f'''Answer the question using only the context. Cite sources as [chunk-N] inline.
Context:
{context_block}
Question: {query}
Answer:'''
return Agent(model).run_sync(prompt).output[chunk-0] marker prefixes each chunk in the context blockMiss either, and citations don't appear.
import re
cited_ids = set(re.findall(r'\[(chunk-\d+)\]', answer))
provided_ids = {cid for cid, _ in retrieved_items}
stray = cited_ids - provided_ids
if stray:
raise ValueError(f"answer cited chunks not in context: {stray}")This catches the failure mode "model fabricates a citation it didn't actually receive" — the regex pulls citation markers from the answer and confirms they all appeared in the retrieved chunks. Beyond this lesson, but worth knowing the pattern.
We assert the answer contains at least one [chunk-N] marker — minimum bar for "the model is citing". Deeper checks (correct citation, claim-supports-citation) are eval-suite territory (L11).
Create a free account to get started. Paid plans unlock all tracks.