Standards question

Ask engineering standards in plain language; every answer cites document and page. Built by Alvin Alias.

Recall@3 0.923 / faithfulness 0.928 graded on 31 labeled queries · 778 pages

checking server

AA-2026-03

Ask about valves, heat transfer, NASA reviews, process safety, or DOE efficiency rules

Rotates questions from this system's own graded exam. The labeled source prints next to whatever the answer cites. Two of them it should refuse.

Ask your own question
Calibration record · retrieval and generation graded separately
Recall@3 0.923 · retrieval, 31 labeled queries
MRR 0.817
Faithfulness 0.928 · generation graded against retrieved text
Relevancy 0.960
Refusal 5 of 5 out-of-corpus traps declined
2,091 chunks at size 300, chosen by ablation
How it works · hybrid retrieval, rerank, citation-constrained answers

dense and BM25 retrieval fused with RRF because engineering acronyms defeat dense search alone; optional cross-encoder rerank; the generator is constrained to retrieved excerpts and every response carries document and page; retrieval and generation are graded separately because a fluent answer can still be ungrounded.

Notes · when it declines and why first run may wait
  1. This runs on a free tier that sleeps between visitors. First start takes 30 to 60 seconds; runs after that are quick.
  2. It answers only from the six documents on the shelf and is graded on declining everything else; a refusal on an off-shelf question is the system working.
  3. Graded questions come from the same 31-query set the published metrics use.