How to reduce incorrect answers and AI hallucinations

How can we prevent AI from making up data or responding too confidently?

No model completely eliminates incorrect answers, but Soportered can reduce them by connecting the AI with authoritative sources, defining when it should refrain, showing references and evaluating the system with real queries from the company. For sensitive tasks we can also incorporate human review and limits that prevent a response from being used as an automatic decision.

Risks to review

• Evaluate only easy questions or questions prepared by those who implemented the system.
• Accept fluent answers as correct without comparing them to an authoritative source.
• Change model, instructions or documents without repeating the tests.
• Automate actions based on a response that was not validated.

How Soportered can help

1. Soportered can build an evaluation with real, ambiguous questions with no known answer.
2. We can link responses to authoritative business sources and check their citations.
3. We can configure prudent responses when information is insufficient or contradictory.
4. We can compare versions of the model and detect when a change reduces quality.
5. We can create indicators for correct, partial, incorrect answers and abstentions.
6. We can incorporate human approval before using results in critical tasks.

When to evaluate this solution

• There is no test set or those responsible for the content.
• Responses will be used for customers, security, finance, or compliance.
• We want to compare various models and costs with reproducible results.