PracticeWorking with AI: Delegation & Oversight
Fix the claim
The 96% green light
The situation
Tidewater sells helpdesk software to business customers. An AI model drafts Tidewater's own support replies, and today a person checks every draft before it is sent. The model was tested offline on past emails. Maya, the product lead, wants to know which replies, if any, can go out without that check.
Maya · Product lead: “Can we put this sentence on the launch page?”
“Our test shows the AI can resolve 96% of all customer requests without a human check.”
What you can see (3) · tap a card to read it
›E01 · Headline
An offline test used 200 historical English email requests from existing customers. Reviewers judged 192 proposed replies acceptable. No test reply was sent to a customer. “Acceptable” means correct issue identification and a supported response with no prohibited account change or unsupported promise.
›E03 · Boundaries
The test excluded live chat, multilingual requests and a forthcoming new product channel.
›E04 · Time
On the tested standard requests, reviewers recorded approximately two minutes less handling time when using an AI draft. The study did not measure unreviewed live outcomes.
Step 1 of 2
Which replacement can the team support with this test?
Your first choice is kept. Changing your mind later counts as a retry.