Write22 of 26 · about 9 min

Knowing when to hand it to a person

Tom Brennan, five months in, will stop looking at a suggestion that is wrong twice. That is not impatience: a suggestion he cannot trust costs him more than no suggestion, because he has to check it anyway.

So the system needs to be allowed to say nothing. Routing a low-confidence case to a human is not a failure of the model, it is the feature that makes the rest of the output usable.

The threshold is a product decision with a measurable consequence, and it belongs in the charter, not in a constant somebody picked. Choosing it means deciding what a wrong routing costs compared to a human glance.

Example

if confidence >= THRESHOLD and queue in QUEUES:
    return "route", queue
return "human", "below the threshold agreed in the charter"

The reason travels with the decision. "Warranty, 87%" told Tom nothing; "below the threshold" tells him what happened.

Your task

Write decide(case, threshold) returning a tuple (action, detail). Route only when the queue is known AND the confidence is at or above the threshold; otherwise hand to a human with a reason. Then print the decisions at a threshold of 0.75, and the share of cases routed, as a percentage with no decimals.

Stuck?

Explain it

The sponsor wants the routed share higher. What do you need to know before lowering the threshold?

Not graded — this is for you.

Where this goes

D4 requires the threshold to be stated in the charter and justified with measured numbers, and D7 produces the curve that justifies it.

CASES = [
    {"ticket_id": "TCK-0001", "queue": "Warranty", "confidence": 0.93},
    {"ticket_id": "TCK-0002", "queue": "Technical", "confidence": 0.58},
    {"ticket_id": "TCK-0003", "queue": "Billing", "confidence": 0.75},
    {"ticket_id": "TCK-0004", "queue": "Warrantee", "confidence": 0.81},
    {"ticket_id": "TCK-0005", "queue": "Parts & Returns", "confidence": 0.99},
]

QUEUES = ["Warranty", "Technical", "Billing", "Parts & Returns", "Account / Other"]
THRESHOLD = 0.75


def decide(case, threshold):
    return ("human", "not implemented")


routed = 0
for case in CASES:
    action, detail = decide(case, THRESHOLD)
    if action == "route":
        routed += 1
    print(f'{case["ticket_id"]} {action} {detail}')
print(f"routed: {round(100 * routed / len(CASES))}%")

Ctrl+Enter checks

This is a teaching runtime for a subset of Python, running in your browser. Integers are exact and the errors are written in plain English, but the standard library is a small subset and anything missing is refused by name rather than approximated. Your project runs on real Python.

Press Run to see what your program does, or Check when you think it is right. Everything runs here in your browser.