Write21 of 26 · about 11 min

Model output that is nearly JSON

A model asked for JSON returns JSON almost always. The interesting engineering is the rest of the time: a missing comma, a field left out, a queue name that is nearly one of yours.

Three different failures hide behind "invalid output", and they need different answers. **Unparseable** text may be worth one repair attempt. **Parseable but schema-invalid** — a missing required field — is worth one re-prompt. **Parseable, valid, but semantically wrong** — "Warrantee" — cannot be repaired by asking again, because the model believes it.

Whatever you choose, the loop needs a hard cap and a defined end state. An unbounded repair loop against a paid API is the most expensive bug in this field.

The outputs below are recorded fixtures. Nothing calls a model.

Example

QUEUES = {"Warranty", "Technical", "Billing", "Parts & Returns", "Account / Other"}

# valid   -> use it
# unparseable -> repair once, then give up
# wrong queue -> a human decides; do not re-prompt

Your task

Write classify(raw) returning one of the strings ok:<queue>, unparseable, missing:<field> or unknown-queue:<value>. Then print <ticket> <result> for each fixture, and a summary count of how many were usable.

Stuck?

Explain it

Why is re-prompting a good answer for unparseable output and a bad one for "Warrantee"?

Not graded — this is for you.

Where this goes

D4 is the triage service. Its rubric requires a hard attempt cap, a defined end state for each failure class, and that the caps are demonstrated rather than asserted.

import json

QUEUES = ["Warranty", "Technical", "Billing", "Parts & Returns", "Account / Other"]
REQUIRED = ["queue", "confidence"]

outputs = json.loads(open("fixtures.json").read())["outputs"]


def classify(raw):
    return "unparseable"


usable = 0
for ticket_id in sorted(outputs):
    result = classify(outputs[ticket_id])
    if result.startswith("ok:"):
        usable += 1
    print(ticket_id, result)
print("usable:", usable, "of", len(outputs))

Files you can open: fixtures.json

Ctrl+Enter checks

This is a teaching runtime for a subset of Python, running in your browser. Integers are exact and the errors are written in plain English, but the standard library is a small subset and anything missing is refused by name rather than approximated. Your project runs on real Python.

Press Run to see what your program does, or Check when you think it is right. Everything runs here in your browser.