Enterprise AI Solutions Architecture

Course Content

Enterprise AI Solutions Architecture

13 sections · 29 lessons

Defending the Design in Review


It is the Thursday afternoon from the first lesson. The Chief Risk Officer, the CISO, the head of model risk, the data protection officer and a finance partner sit around the table, with Dana Whitfield as business owner. They have forty minutes, and they have read the dossier. Their job is to find what is wrong with it. Your job is not to sell the design. It is to make it easy for sceptical, busy people to reach a sound decision, and to be the person in the room who knows most precisely where the design is weak.

Defending a design is a skill, and it can be prepared. Almost every hard question in a review is predictable from who is asking it. Almost every good answer is a pointer to something that already exists. And the answers that lose reviews are also predictable: vague reassurance, a number without a range, and a bluff when the honest answer is "we don't know yet".

This lesson prepares Meridian's defence, shows what good answers sound like, and ends with the review's outcome recorded as the last design-record document.

Answering a reviewer's hardest questionReviewer asksYou know the answerYou do not know yetFirst sentence,then evidence IDRange,residual and ownerSay so: what,who, by whenOffer it asa condition
Both branches end in something specific, owned and checkable; a bluff on one question costs credibility on every other.

Prepare by reviewer

Each reviewer has a concern, and each concern has a hardest version. Write them down beforehand, with where the answer lives.

ReviewerMain concernHardest likely questionWhere the answer lives
Chief Risk OfficerCustomer harm"What is the worst this can do to a customer in difficulty, and how would we know?"Register R-01 to R-03; invariants; SEV1 runbook
CISOAttack and leakage"Where can an attacker's words reach the model, and what happens when the model obeys them?"MER-10; tool matrix; output filter
Head of model riskEvidence and change"How will we know it has got worse, and who decides a change is material?"MER-08 objectives; change classification; challenge set
Data protection officerPersonal data"What personal data leaves the bank, and is anything kept?"MER-04; contract terms; audit store design
Finance partnerReal return"What happens to the case if adoption is 70%?"MER-11 sensitivity table
Head of ServicingAccepting residual risk"What exactly am I accepting?"Residual risk page, with ratings and indicators

Rehearse with a colleague playing each role, and ask them to be unkind. The first rehearsal usually reveals one question you cannot answer from a document. Better there than in the room.

What good answers sound like

Answers that lose reviews

  • "The model is very accurate"
  • "We've told it not to do that"
  • "Security is built in throughout"
  • A confident guess when you are not sure

Answers that win them

  • "93% sampled correctness, range 91.5 to 94.2, objective and fallback in MER-08"
  • "No send tool exists; the access test in the release suite proves it"
  • "Threat T-02 is closed by two independent controls in code"
  • "We don't know yet; here is how and when we will"

Five habits produce answers like those on the right.

  1. Answer the question in the first sentence, then give the evidence by ID or number.
  2. Give ranges, not points. "93%" invites "on how many?"; "93%, range 91.5 to 94.2 on 1,400 graded answers" closes it.
  3. Say "we don't know yet" when it is true, followed by what you would need, by when, and a condition the board could set. Bluffing on one question costs you credibility on every other answer.
  4. Do not defend what is not yours. Residual risk is accepted by the business owner. Policy correctness belongs to the policy team. Say who owns it and what evidence they have seen.
  5. Name your weaknesses before they are found. R-01's medium residual and the letters' NPV are already on page one.

A sample exchange

Here is part of Meridian's actual review, lightly edited.

CRO: Your own register says R-01 stays at medium. So staff will sometimes tell customers the wrong thing. Why should I accept that?

Architect: Because today they already do, more often. The mystery shop found 18% of staff answers wrong or incomplete. In the pilot, using the same method, staff with the assistant were at 7%. We are not asking you to accept a new risk; we are asking you to replace a larger, unmeasured one with a smaller, measured one. If sampled correctness falls below 85% over a week, answers switch to search mode automatically. Dana has seen these numbers and accepts the residual.

CISO: What if the provider is breached?

Architect: Under the contract, the provider keeps no prompts or outputs once a request is processed, so a breach would expose requests in flight during the incident, not a store of history. That is why R-04 is rated low, not zero. The alternative is self-hosting generation, which the topology decision prices at over ten times the cost, with lower quality on our evaluation. I recommend accepting it, with an annual review of the provider's security assurance as a condition.

Head of model risk: Your judge could drift and start passing bad answers.

Architect: It could. Analysts re-grade 10% of judged items every week, and if the false pass rate goes above 5%, the judge is recalibrated before it gates another release. Your own challenge set is re-run at every revalidation, and it is the one the team has never seen.

Notice that none of the answers claims the risk is gone. Each states the size of what remains, the evidence, the control and who owns it.

When you cannot answer

Every serious review contains at least one question the design cannot fully answer yet. What matters is not avoiding those questions but answering them in a way that keeps the board's trust in everything else.

Turning objections into conditions

Approval with conditions is the normal, healthy outcome of a serious review. A condition converts an objection into a specific, owned, dated action. The review's outcome is recorded as the final design-record document.

YAML
id: MER-13-outcomereview: AI review board, 2026-09-17, 38 minutes, 14 questionsdecision: stage 1 approved with conditions; stage 2 referred to risk committeeconditions:  C-1: quarterly board report on objectives, invariants, incidents, adoption, drills  C-2: provider B contract signed before release 1 (fallback must be real)  C-3: retention rule for unapproved drafts agreed with DPO before letter pilot  C-4: legal classification memo reviewed every 6 months and on capability change  C-5: independent red team before stage 2 pilot  C-6: annual review of provider security assuranceowners: {C-1: service owner, C-2: procurement, C-3: DPO, C-4: legal,         C-5: CISO, C-6: procurement}next_review: gate 1, month 10

After approval, the design record does not go into a drawer. It becomes the system's living documentation. Each material change updates the affected documents, and the traceability matrix is re-run. When a new architect inherits the system in two years, the first thing they read is this design record, and every decision still has its reason next to it.

That table is the whole course in miniature. Every row you can fill is a decision already made, measured and owned; every empty row is where your next week of design work should go.

Check your understanding

0 of 3 answered

1.The CRO asks why she should accept a residual risk that staff will sometimes give wrong answers. What is the strongest reply?

2.The data protection officer asks a question the design does not answer. What should the architect do?

3.Why is "approved with conditions" a good outcome rather than a partial failure?