Catalog record · Risk Management
Stage as of Oct 27, 2026 · historical report, not live reconfirmation.
Source-linked AI-policy guidance with exact-quote validation
Primary workflow source. A named organization's workflow supported by a reviewed primary source. Stage is reported separately; this label does not establish measured outcomes or independent validation.
Last researched Oct 29, 2026. Source access and reported stage are recorded below; deployment evidence does not establish ROI.
What the research supports
Defra opened Ask the toolkit on 27 October. Claude Sonnet 4.6 receives the entire small guidance corpus, returns structured answers with source links, and the application removes quotations not found in the named page. It has no browsing/tools or agent loop; changes to pages, prompts or model trigger fixed-question regression tests.
- Reported stage
- Pilot
- Technology
- GenAI/LLM
- Business function
- Not assessed
- Impact (legacy label)
- Not labeled
- Catalog date · unconfirmed
- Not reported
- Imported
- Oct 29, 2026
- Last researched
- Oct 29, 2026
Measures & evidence
No quantified measure recorded. This is a gap in the evidence, not a finding of no value.
Documented vendor relationships
No vendor relationship has been linked to this specific workflow in the curated overlay. This does not mean it was built without vendors.
See the company’s documented relationshipsDeployment evidence & limits
Public Beta; opened 2026-10-27, no launch usage evidence Status as of 2026-10-27.
Stage is as reported in the cited evidence. Current operating status has not been independently reconfirmed; the research date is an editorial review date.
Tests met 4 of 6 gates; grounded answers scored 87–90/100 and expected-rule quoting 16–19/22, failing their perfect-score gates. Human agreement with the AI judge was uneven. No formal fairness assessment or representative public/Welsh test; no usage data at launch. A correct quotation does not establish that the selected rule applies to the question. Small-corpus full-context handling may not transfer to large, permissioned bank policy libraries. No FS adoption or ROI evidenced.
Potential FS application · inferred
Inferred: bounded internal policy or model-risk guidance assistant; exact quotation checks support traceability of policy obligations.
Inferred: bounded internal policy or model-risk guidance assistant; exact quotation checks support traceability of policy obligations.
Original industry: Public administration. Transfer limits: A correct quotation does not establish that the selected rule applies to the question. Small-corpus full-context handling may not transfer to large, permissioned bank policy libraries. No FS adoption or ROI evidenced.
Source register
- DEFRA: Ask the toolkitgov.uk · opens in a new tab
Defra / GOV.UK · issuer · full-text · Published 2026-10-27
Research lanes: public_science_transfer · Batch global-discovery-2026-10-29. Coverage and remaining gaps