Principles 4 & 5: Evaluate Quality and Divide Labor
Learning Objectives
|
Principle 4: Evaluate Quality
Evaluate Quality means treating AI clinical output as testable — not final. Rather than accepting the first response, a professional prompt engineer identifies errors, rates outputs against a rubric, and runs controlled tests to understand which variables drive good or bad clinical results.
Mindset Shift Casual user: Runs prompt once. Output ‘looks fine.’ Uses it. Prompt Engineer: Runs prompt 3–5 times. Rates against rubric. Identifies failure point. Fixes one variable. Re-runs.
In healthcare, ‘looks fine’ is not enough. Clinical accuracy and patient safety require systematic evaluation. |
Clinical Output Evaluation Rubric
Evaluation Criterion | Score (1–5) | What to Check |
Clinical Accuracy | _ / 5 | Diagnoses, medications, dosages, lab values — all correct? |
Format Compliance | _ / 5 | Matches requested format exactly (SOAP, table, numbered list)? |
Tone & Language | _ / 5 | Appropriate for audience (clinical EHR vs. patient-facing)? |
Completeness | _ / 5 | All required sections and fields present? |
Safety & Compliance | _ / 5 | No PHI exposed, no dangerous clinical errors, HIPAA-safe? |
TOTAL | _ / 25 | Score below 20/25 = revise the prompt before clinical use |
Principle 5: Divide Labor
Divide Labor means decomposing a complex, multi-output clinical task into a sequence of focused, single-output prompts — where the output of each step feeds as input into the next. This is prompt chaining.
Chain Type | Description | Healthcare Application |
Sequential | Step A output → Step B input | Table note → Handoff summary → Patient letter |
Retrieval | AI output used to query data | Diagnosis list → search clinical guidelines (UpToDate) |
Function Call | AI output triggers system action | Medication list → pharmacy dispensing system import |
Cross-Modal | Text feeds image/document pipeline | Diagnosis → patient education infographic generation |
When NOT to Divide Labor
|
Real-Life Healthcare Example — Both Principles
Maria Gonzalez (Patient Thread — Evaluate Quality and Divide Labor): Complete Clinical Workflow Chain Maria has been admitted. The team now needs to build the complete documentation workflow. All 5 prompts are chained — output from each step feeds into the next.
STEP 1 — Intake Summary (G – Give Direction +P – Provide Examples +S – Specify Format applied): Prompt: ‘You are an Internal Medicine physician. Summarize the ED triage notes for Maria Gonzalez, 62F, admitted with HbA1c 10.2%, eGFR 52, BP 158/94. Return as: Chief Complaint | Vital Signs | Key Lab Values | Initial Impression. Max 5 bullet points per section.’ Evaluate: Score on rubric. Must score 20+/25 before moving to Step 2.
STEP 2 — Differential Diagnosis (using Step 1 output as input): Prompt: ‘Based on this patient summary: [paste Step 1 output]. List a differential diagnosis with 3 most likely conditions ranked by probability. Format: Diagnosis | Rationale | Priority (High/Med/Low).’
STEP 3 — Numbered List Note (using Step 2 output as input): Prompt: ‘Using the patient summary and differential above, generate a complete numbered format.
STEP 4 — Nurse Handoff (using Step 3 numbered format as input): Prompt: ‘Convert this SOAP note into an SBAR handoff note for the night nursing team. Situation | Background | Assessment | Recommendation. Max 150 words.’ Evaluate: Score handoff on rubric — especially Tone (must be nursing-appropriate, not physician-formal).
STEP 5 — Patient Discharge Letter (using Step 3 Plan as input): Prompt: ‘You are a bilingual Patient Educator. Using the Treatment Plan from Maria’s Numbered list note, write a plain-language discharge letter for Maria. Grade 6 reading level. Include Spanish translations for key medical terms. Sections: Diagnosis | Medications | Warning Signs | Next Appointment.’ |