Accuracy Checklist

Whether your agent is used for customer support, decision-making, or content generation, inaccuracies will result in poor decisions, a poor user experience, reputational damage and perhaps litigation.

Check your proposed or existing agent for Accuracy best practice...

---

Does your agent...
minimize hallucinations by using retrieval-augmented generation (RAG)?
fact-check its outputs against known-to-be-reliable data or knowledge sources?
cross-check answers and decisions using at least one more model?
provide sources, references, or logic supporting responses that need evidence?
apply constraints on what topics or domains it will accept as input?
apply constraints on what topics or domains it will permit as output?
check that output remains consistent with an ongoing conversation or task?
only make high-risk decisions when there's a human in the loop?
declare uncertainly or lack of knowledge when it doesn't know?
detect and reject misleading, ambiguous, or adversarial queries?
have access to real-time data (or not need it)?
collect user feedback and use it to improve accuracy over time?
get frequent comprehensive accuracy evaluations?
get frequent bias evaluations?
have processes that track changes in its knowledge, data and code?

Accuracy Risk: 100

Unless you've addressed most of the items in the above checklist, the risk that your agent will provide a poor user experience and damage your reputation is high. Answer the above questions by checking the boxes - and get an assessment of your agent's accuracy risk.

---

Our understanding of AI agent accuracy risks and how to mitigate them is evolving quickly. Be sure to check this list regularly for new knowledge.