Enterprise AI Solutions Architecture

Course Content

Enterprise AI Solutions Architecture

13 sections · 29 lessons

Tool Permissions and Blast Radius


In a demo before the pilot, an engineer gave the assistant a convenient tool: update_customer_record(customer_id, field, value). It could change any field on any customer. It made the demo impressive; the assistant could "fix" a wrong phone number on request. It also meant that a single successful prompt injection, hidden in a customer's email pasted into case notes, could change the payment date on a different customer's loan.

Tools are how a model's words become actions. Whatever a tool can do, a confused or manipulated model can make it do. So tool design is security design, and it deserves the same care as a firewall rule.

This lesson introduces blast radius: the worst harm a single wrong or malicious call can cause, and how far that harm can spread before anyone notices. It then applies least privilege to tools and produces the tool permission matrix, part B of MER-06.

Blast radius: effect times scope times reach3321821241111———not builtEffectScopeReachScoreupdate_customer_recordcreate_letter_draftget_case_notessend_letter
Binding the customer ID in code and creating drafts instead of writes is most of the difference between 18 and 4.

Why a workflow still needs a tool matrix

Meridian's release 1 is a set of workflows: code decides which functions to call, not the model. It is fair to ask why the design needs a tool matrix at all. There are three reasons. First, code still passes model output into some functions; the letter body the model wrote goes into the draft-creation call. Second, section 7 will expose these functions as shared services that other assistants in the bank can call, some of them agents. Third, the future plan set-up capability at L3 will give a model real choices. Designing the tools safely now means none of those changes needs a redesign.

Measuring blast radius

Score every tool on three dimensions, each from 1 to 3, and multiply. The product gives a rough tier that decides how much protection the tool needs.

Dimension123
EffectRead onlyCreates a draft or pending itemChanges a record or sends externally
ScopeOne customer or case, bound by codeAny one customer the model namesMany customers or the whole system
ReachInternal, seen by staffAffects a customer after human actionReaches a customer or third party directly

A score of 1 to 3 is low, 4 to 9 is medium and 12 or more is high. The demo's update_customer_record scores 3 for effect, 3 for scope (any customer) and 2 for reach: 18, high. A read-only policy search scores 1 × 1 × 1 = 1. The formula is crude on purpose. Its value is that it forces a conversation about each dimension, and it makes the worst tools stand out.

Two other properties matter even though they are not in the score. Reversibility: can the effect be undone completely? And detection time: how long before someone would notice a bad call? A wrong change that is found in minutes is very different from one found at the next monthly statement.

Least privilege, applied to tools

Six rules turn the principle of least privilege into tool design.

  1. Narrow tools beat general ones. get_arrears(case) instead of run_query(sql). A general tool's blast radius is everything it can reach.
  2. Bind identifiers in code. The customer ID comes from the open case, never from the model's arguments. The model can influence what is read about the customer, not which customer.
  3. Separate read from write. Different tools, different credentials. A read tool that is misused leaks; a write tool that is misused damages. Keep them apart so one cannot become the other.
  4. Cap volume. Maximum calls per request, maximum records per call, maximum drafts per case per day.
  5. Prefer drafts to writes. A tool that creates a draft or a pending change moves the effect behind a human, outside the model's reach.
  6. Do not build what must never happen. No send_letter, no waive_fees, no update_payment_schedule. A missing tool is the strongest control there is, because there is nothing to misuse.

Here is how Meridian's orchestrator binds context in code. The model never sees or supplies the customer ID.

Python
from dataclasses import dataclass@dataclass(frozen=True)class CaseContext:    case_id: str    customer_id: str        # taken from Atlas, never from model output    user_id: str    obo_token: str          # the staff member's delegated tokenclass ToolDenied(Exception):    passALLOWED_TEMPLATES = {"HARDSHIP-PLAN-V7", "HARDSHIP-HOLIDAY-V4"}MAX_NOTES = 200def get_case_notes(ctx: CaseContext, atlas, months: int = 12) -> list[dict]:    if not 1 <= months <= 60:        raise ToolDenied("months must be between 1 and 60")    notes = atlas.notes(customer_id=ctx.customer_id, months=months, token=ctx.obo_token)    return notes[:MAX_NOTES]def create_letter_draft(ctx: CaseContext, docgen, template_id: str, body: str, figures: dict) -> str:    if template_id not in ALLOWED_TEMPLATES:        raise ToolDenied(f"template not allowed: {template_id}")    return docgen.create_draft(        case_id=ctx.case_id, customer_id=ctx.customer_id, template_id=template_id,        body=body, figures=figures, status="DRAFT",        created_by=f"assistant-for:{ctx.user_id}", token=ctx.obo_token,    )

CaseContext is built by the orchestrator from the Atlas case and the staff member's sign-on, and it is frozen so nothing downstream can change it. Each tool takes the context plus only the arguments the model may influence: how many months of notes, which approved template, what the letter body says. Every call carries the staff member's delegated token, so Atlas and DocGen enforce their own rules too. The draft is created with status DRAFT and records who it was created for.

The maker-checker principle

Banks have long used maker-checker, also called the four-eyes principle: one person prepares a change and a different person approves it. It maps cleanly onto AI. An AI may be a maker; it must never be the checker, and it must never be both. When Meridian eventually builds plan set-up at L3, the model will create a pending change request inside Ledger's existing maker-checker workflow, and a named human will approve it using the same screen and controls they use today. The bank does not need a new approval system for AI; it needs AI to enter the one it already trusts.

The tool permission matrix is part B of MER-06.

ToolEffectScopeReachScoreCredentialLimitsApproval
search_policyReadDocuments, filtered by roleInternal1Staff delegated40 resultsNone
get_account_factsReadOne customer, boundInternal1Staff delegated1 call per requestNone
get_case_notesReadOne customer, boundInternal1Staff delegated200 notes, 60 monthsNone
get_hardship_figuresRead and computeOne customer, boundInternal1Staff delegated3 calls per draftNone
create_letter_draftDraftOne case, boundAfter human action4Staff delegated, draft scope only5 drafts per case per dayHuman approves in DocGen
send_letterNot built
update_payment_scheduleNot built

Check your understanding

0 of 3 answered

1.Why does Meridian's orchestrator take the customer ID from the Atlas case instead of letting the model pass it?

2.A tool creates drafts for one case, bound by code, which reach a customer only after human action. Using the lesson's scoring, what is its blast radius score?

3.Under the maker-checker principle, which role may an AI play in a future plan set-up capability?