Learning Objectives
|
Why This Lesson Is Non-Negotiable
Using AI in healthcare is powerful — but it carries serious patient safety and legal responsibilities. This lesson is not optional. Every principle learned in this course must be applied within these safety boundaries.
HIPAA Prompt Safety: What NEVER Goes in a Prompt
NEVER Include in a Prompt | Safe Alternative to Use Instead |
Patient full name (e.g., Maria Gonzalez) | Use: ‘Patient A’ or ’62-year-old female patient’ |
Date of birth, Social Security Number, MRN | Use: approximate age and generic demographics |
Specific hospital name tied to a real patient | Use: ‘regional hospital’ or ‘City General’ without patient ID |
Real physician names linked to patient records | Use: ‘attending physician’ generically |
Insurance plan details and claim numbers | Use: ‘standard insurance coverage’ generically |
NOTE: In this course, ‘Maria Gonzalez’ is a fictional training case only — not a real patient. In real clinical practice, always use de-identified or fictional patient data when prompting public AI tools.
AI Hallucination — The Biggest Clinical Risk
What Is an AI Hallucination? An AI hallucination occurs when the model produces confident, fluent, plausible-sounding information that is factually wrong. In healthcare, this can mean:
– Recommending a drug that is contraindicated for a patient’s condition – Citing a clinical guideline that does not exist or is outdated – Stating an incorrect medication dosage with the same confidence as a correct one – Generating a lab value that was not in the prompt and is clinically wrong
AI hallucinations do not look like errors — they look like correct, well-formatted clinical output. This is what makes them dangerous. Always verify clinical AI output against authoritative sources. |
5 Responsible Prompting Rules
- Always treat AI clinical output as a DRAFT — never as a final clinical decision.
- Always verify AI-generated diagnoses, medications, and dosages against current clinical guidelines.
- Never enter real patient PHI into a public AI tool (ChatGPT, Claude, Gemini, Copilot).
- Report AI output errors through your hospital’s quality assurance and incident reporting system.
- Know and follow your hospital’s AI use policy before deploying any AI prompt in a clinical workflow.
Real-Life Healthcare Example — Safety
Maria Gonzalez (Patient Thread — Lesson 3.5): Hallucination Risk Scenario You ask AI to generate Maria’s discharge medication list. The AI produces a clean, well-formatted table — it looks perfect.
BUT: On review, a clinical pharmacist notices the AI has listed Metformin at 2000mg twice daily. Maria has CKD Stage 2. Metformin at this dose is contraindicated with her eGFR of 52. Standard guidance: Metformin should be reduced or held at eGFR < 45; caution at eGFR 45–60.
The AI produced a plausible, correctly formatted medication entry — with a dangerous dosage. It did not hallucinate the drug name. It hallucinated the dose for this patient’s renal status.
Lesson: Never use AI-generated medication lists without clinical pharmacist review. Always specify patient renal function in the prompt: ‘Patient has CKD Stage 2, eGFR 52. Recommend renally-adjusted dosing for all medications.’
Even with this addition — VERIFY before use. AI is a drafting tool, not a prescribing authority. |