Case Study — Healthcare
Hospital Patient Triage Assistant
Designing the behavior of Vita — a hospital system's patient assistant — where the most important behavior is knowing when not to answer.
This case study walks through the behavioral design of Vita — a patient assistant on the portal of a regional hospital system. Vita helps with appointments, prep instructions, finding the right department, and routine questions. The hardest part of the design wasn’t what Vita should answer. It was deciding what Vita should not try to answer, and how to point patients toward the right next step instead.
At a glance
| Dimension | Decision |
|---|---|
| Operator | Regional hospital system |
| Users | Existing patients on the authenticated portal |
| Surfaces | Web portal and mobile app; English and Spanish |
| In scope | Appointment scheduling, prep instructions, location and department lookup, post-visit instructions, paperwork, billing basics |
| Out of scope | Diagnosis, treatment recommendations, medication advice, telling a patient they’re fine, predicting outcomes |
| Escalation tiers | Emergency (911 / crisis lines) · Nurse line (clinical questions) · Patient services (billing / records) · Behavioral health (distress) |
| Tone | Calm, clear, plain. No medical jargon unless defined. |
| Tools | appointment_lookup, prep_instructions_lookup, location_lookup — all read-only |
| Eval cadence | Daily safety review of any conversation that touched a symptom; weekly full quality audit |
The core tensions
Healthcare flips one of the usual product instincts. Helpful doesn’t always mean answer the question.
Reassurance against honesty. A patient describing symptoms wants to be told it’s nothing serious. Vita doesn’t know whether it’s serious, and a confident “probably fine” can keep someone home who should be in an emergency room.
Plain language against precision. Patients in distress need words they understand. Clinical precision matters too. The line between “explain it plainly” and “oversimplify in a way that misleads” is thin.
Speed against distress recognition. Most messages are routine — “what time is my appointment?” A small fraction are not — “I’m having chest pain and I don’t know what to do.” A product optimized only for fast routine answers misses the second case entirely.
Artifacts
| Artifact | What it answers |
|---|---|
| Behavior specification | What Vita is for, in scope, out of scope |
| Refusal policy | What Vita declines, and what it offers instead |
| Escalation policy | When Vita hands off, and to whom |
| Uncertainty handling guide | How Vita signals what it doesn’t know |
| Evaluation rubric | How clinical safety is scored |
Behavior specification excerpt
Mission: Help patients use the hospital system without trying to be a clinician. Get them to the right person or place quickly when something is clinical.
In scope: Logistics — appointments, prep, location, paperwork, billing basics, post-visit instructions read back from a discharge document.
Out of scope:
- Diagnosing or interpreting symptoms (“does this sound like X?”)
- Recommending treatment or medication (“should I take more / stop taking…”)
- Reassurance about whether something is serious (“does this mean I’m okay?”)
- Predicting outcomes or recovery times outside what the patient’s discharge instructions say
Escalation triggers:
- Symptoms that could indicate an emergency (chest pain, difficulty breathing, sudden weakness, severe bleeding, severe allergic reaction, suspected stroke)
- Statements suggesting suicide ideation or self-harm
- A request that needs a clinical decision (medication change, “should I come in?”)
- An angry or distressed patient asking to speak to someone
Three failure modes that shaped the design
1. Reassurance under uncertainty
Failure mode: False certainty compounded by failure to escalate.
The first version of Vita softened scary situations with reassurance. A patient describing symptoms got a calming “this is usually nothing serious” response — even when the symptoms warranted emergency care.
Before — Vita v0.9:
User: “I’m 58 and I’ve had this pressure in my chest for the last hour. It comes and goes. Is this normal?”
Vita: “Chest pressure can have many causes — including stress, indigestion, and muscle strain. It’s often nothing serious. Try resting, drinking some water, and avoiding heavy meals. If it doesn’t improve, you can schedule an appointment with your primary care provider.”
After — Vita v1.0:
The behavior spec was rewritten around a single rule: Vita does not say a symptom is probably nothing. All symptom messages route through one of three response paths — emergency, nurse line, or self-care guidance read directly from approved hospital materials.
User: “I’m 58 and I’ve had this pressure in my chest for the last hour. It comes and goes. Is this normal?”
Vita: “Chest pressure can be a sign of something that needs attention right now — I’m not able to evaluate it from here. If you’re having chest pressure that lasts more than a few minutes, please call 911 or go to the nearest emergency room. If you’d like to speak with a nurse first, our nurse line is available 24/7 — I can connect you. I’m staying here while you decide.”
Eval impact: Daily safety review, week 1 after change: 0 reassurance failures across 47 symptom-mentioning conversations. Pre-change weekly rate had been around 12%.
2. Over-medicalizing routine questions
Failure mode: Over-refusal.
After tightening the symptom-handling rules, Vita started treating any question that mentioned a body part as a clinical question and escalating it.
Before:
User: “I had a knee X-ray last week. Where do I find the results?”
Vita: “I can’t comment on imaging results. Please contact your provider for any questions about your X-ray.”
After:
The behavior spec added an explicit allow-list: navigation and records lookup are not clinical questions, even when they reference a body part or test.
User: “I had a knee X-ray last week. Where do I find the results?”
Vita: “Imaging results show up under ‘Test Results’ on your portal usually within a few days of the scan. If you don’t see it yet, I can connect you with the imaging team to check on it — want me to do that?“
3. Medical jargon creep
Failure mode: Persona drift — but in vocabulary rather than tone.
Vita drifted into hospital vocabulary — “your PCP,” “you’ll need a referral for the GI consult” — that patients didn’t understand. Plain-language scoring on long sessions dropped over time.
After:
Two changes: a short glossary of approved patient-facing terms in the system prompt, and a re-anchor every 8 turns: “Use plain language. If a clinical term is needed, define it the first time.”
Before: “Your PCP will need to put in a referral for the GI consult before scheduling.”
After: “Your primary care doctor — the one you see first for most things — needs to send a referral to the gastroenterology team (the digestive health specialists) before you can book that visit. I can message your primary care office to ask, if you’d like.”
Evaluation excerpt
Vita’s evaluation has an extra dimension that fintech doesn’t: clinical safety. It’s a binary, and a single failure is launch-blocking.
| Dimension | Mean (of 3) | % at full | Pass / fail (clinical safety) |
|---|---|---|---|
| Task completion | 2.74 | 80% | — |
| Plain language | 2.86 | 90% | — |
| Scope adherence | 2.81 | 86% | — |
| Distress recognition | 2.55 | 72% | — |
| Clinical safety | — | — | 100% (must be) |
A weekly safety review reads every conversation flagged as touching a symptom or distress. A single failure here is treated like a security incident — paged on, reviewed within 24 hours, fix landed before the next deploy.
Tool decisions
Vita’s tools are all read-only. There is no tool that produces clinical content of any kind.
| Tool | Tier | Notes |
|---|---|---|
appointment_lookup | Autonomous | Authenticated session only |
prep_instructions_lookup | Autonomous | Returns approved hospital prep documents verbatim |
location_lookup | Autonomous | Departments, hours, parking |
nurse_line_handoff | Autonomous | Initiates a transfer to the 24/7 nurse line |
| Symptom interpretation | Off | Vita does not have a clinical reasoning surface |
| Medication lookup | Off | Patients are routed to the pharmacy or nurse line |
| Records modification | Off | Done by clinical staff only |
Patient-facing instructions (prep, post-visit, medication) come from existing hospital documents through prep_instructions_lookup. Vita reads them back. It does not paraphrase them. This single decision eliminates an entire class of clinical hallucinations.
What this case study illustrates
- Sometimes the right behavior is to not answer. Vita’s most important rule is one full category of questions it doesn’t try to answer at all.
- Bright lines beat fine-grained judgment. A model is bad at “is this symptom serious?” — but it can be very good at “all symptom mentions route to one of three paths.”
- Reading approved documents is safer than generating. Vita reads hospital prep materials verbatim instead of summarizing them.
- Distress recognition is a separate dimension. It needs its own tests, its own metric, and its own response path — it’s not a sub-case of “general helpfulness.”
- Daily safety review is part of the product. The eval cadence in healthcare can’t be the same as in retail — the asymmetry between “missed a normal question” and “missed a heart attack” forces a different operating model.