After the PoC, define how to evaluate
Move beyond plausible answers with questions, documents, and edge cases that reflect the work people actually need to do.
A sample article prepared to introduce this blog.
From demo questions to real work
Answering a few prepared questions is different from helping real users. People ask about missing information, leave out constraints, and combine requests. An evaluation set should reflect those situations.
Collect common questions, hard-to-find documents, and requests restricted by permissions. Write down the expected behavior and supporting evidence for each case. That makes evaluation more useful than matching answer text alone.
Evaluate answers and actions separately
Writing fluent text and choosing the correct tool are different capabilities. Check whether retrieval found the right material, whether evidence supports the answer, and whether execution needs confirmation.
A safe refusal can be the right behavior without completing the task. Record task completion, appropriate deferral or refusal, and evidence quality separately to make the remaining gaps visible.
Keep comparisons stable
Changing the questions and scoring rules while changing the model makes improvement difficult to interpret. Compare quality, cost, and latency on the same set, and inspect edge cases separately.
Add questions discovered in production to the next evaluation set while retaining earlier versions. Evaluation is a recurring check that changes preserve useful behavior, rather than a one-time approval.