PracticeWorking with AI: Delegation & Oversight
The boundary changes
The 96% green light
The situation
Tidewater sells helpdesk software to business customers. An AI model drafts Tidewater's own support replies, and today a person checks every draft before it is sent. The model was tested offline on past emails. Maya, the product lead, wants to know which replies, if any, can go out without that check.
Maya · Product lead: “Routing has been checked separately now. What policy do you want for the existing channel?”
The aim: same-day service within 40 reviews a day, and Maya accepts a small, watched risk in standard replies to get it; exceptions must never go out unchecked. The existing channel gets 100 requests a day, about 20 of them exceptions. Reviewing every exception plus 10 standard drafts uses about 30 of the 40 review slots. Anything over capacity waits in the queue; nothing sends on its own. The bounded policy has a stop rule: any sampled wrong promise, wrong account change or known routing failure pauses that group, which then waits for review.
What you can see (5) · tap a card to read it
›E01 · Headline
An offline test used 200 historical English email requests from existing customers. Reviewers judged 192 proposed replies acceptable. No test reply was sent to a customer. “Acceptable” means correct issue identification and a supported response with no prohibited account change or unsupported promise.
›E02 · Composition
180 standard requests: 177 acceptable, 3 unacceptable. 20 account-specific exceptions: 15 acceptable, 5 unacceptable. 177 + 15 = 192 acceptable of 200. Reviewers sorted the requests into standard and exception after the test.
›E03 · Boundaries
The test excluded live chat, multilingual requests and a forthcoming new product channel.
›E07 · Routing check
For the existing English-email channel only, a separately checked routing rule found all 40 exception requests in a recent 200-request review sample. It is good evidence, but it can't prove the rule will never misroute.
›E08 · Operating envelope
Existing channel: 100 requests a day, with 20 expected exceptions. Staff can review 40 replies a day. A scoped policy reviews exceptions plus 10 sampled standard drafts before send; ordinary unmatched requests queue for review. Manual review of all 100 exceeds same-day capacity. These daily counts are planning estimates; real days will vary.
Step 1 of 2
Which policy do you recommend for the existing channel?
Your first choice is kept. Changing your mind later counts as a retry.