What causes wrong parameter extraction, and how can it be improved?

What you need to know

Causes and fixes

Two layers of validation

• Shape — strict mode ("strict": true on the tool) guarantees the JSON matches the schema: right fields, types and enum values.
• Meaning — your handler checks what a schema cannot: the date is in the future, the account belongs to the user, the amount is within limits.

1def validate_transfer(args, user, today):2    errors = []3    if args["account_id"] not in user.account_ids:4        errors.append(f"account_id {args['account_id']} is not one of the user's accounts: "5                      f"{', '.join(user.account_ids)}")6    if args["amount_inr"] > user.daily_limit_inr:7        errors.append(f"amount_inr exceeds the daily limit of {user.daily_limit_inr}")8    if args.get("scheduled_date") and args["scheduled_date"] < today.isoformat():9        errors.append(f"scheduled_date is in the past; today is {today.isoformat()}")10    return errors   # non-empty -> return an is_error tool_result with these lines

Each message says what is wrong and what is valid, so the model usually fixes it in one turn.

Log every validation failure

Record tool name, field and error for every rejected call. Sorted by count, that log is your to-do list: the top field is the next description to fix.

A real-life example

A bank assistant's create_transfer tool had a 6% rejection rate. The validation log showed:

• 41% — amount given in lakhs or thousands ("send 2 lakh" became 2).
• 27% — payee_id invented from a name, not taken from list_payees.
• 18% — scheduled_date a year off when the user said "on the 5th" in late December.

Fixes: the field was renamed amount_inr with the description "full amount in rupees; 2 lakh = 200000"; the payee_id description says "must come from list_payees; call it first if needed"; the system prompt includes today's date and time zone (IST). Strict mode was switched on. Rejections dropped to 0.7%, and the remaining ones were caught and fixed by the model after one error message.

Follow-up questions to expect

• "Doesn't strict mode solve this?" — It solves shape errors only. A perfectly formatted date can still be the wrong date.
• "Should the model ask the user when unsure?" — Yes. Give it an ask_user tool and say in descriptions that it should ask rather than guess required IDs.
• "How do you test argument quality?" — Labelled requests with expected arguments; compare with normalisation (dates, case, whitespace) and score per field.