Clinician AI
Feedback System
Partner with UPMC clinicians, we aim to develop a clinician feedback interface for evaluating AI-generated Prograf dosage recommendations.
Our goal is to:
Develop a clinician feedback interface that displays essential information for evaluating AI-generated Prograf dosage recommendations, enhancing model reliability and facilitate clinical adoption.
Lead Designer:
ux design
User Testing
Research
Team
Saera Kim
Ananya Chaudhary
Guanjie Cheng
Xinfei Cen
Tools
Figma, Adobe CC, Miro
Duration
Spring 2025
4 months
Project Introduction
Clinicians + AI: Closing the Gap in Clinical Dosing
Artificial intelligence holds promise for improving patient care—but many models stall before making it into real clinical settings. Clinician concerns about reliability, interpretability, and workflow fit often block adoption.
Our project centers on empowering clinicians to co-refine AI-generated dosing suggestions. We partnered with UPMC’s kidney transplant team to design a feedback interface for Prograf (an immunosuppressant drug). Clinicians review AI-recommended doses and provide structured feedback, enabling the model to learn and improve in real-world use.
Why It’s Hard: The Implementation Gap
AI research often remains decoupled from real-world clinical demands.
Models trained on retrospective datasets fail to capture workflow constraints, emergent risk factors, or nuanced clinician intuition.
In high-stakes areas like transplant dosing, clinicians must consider many dynamic variables (infection risk, prior patient response, comorbidities) that AI may not account for.
Without transparency, control, or adaptability, many clinicians distrust AI recommendations.
What We Aimed to Build
Primary Goal
To build a feedback-driven clinician interface that lets doctors evaluate and fine-tune AI-recommended Prograf doses—thus improving model trust and readiness for clinical deployment.
Subsidiary Objectives
Understand clinicians’ existing workflows & pain points
Map the mental model of how they choose Prograf dosing
Identify which clinical data/context is essential for safe dose adjustment
Design feedback interactions that minimize cognitive load
Structure the interface to support asynchronous, thoughtful feedback
Stakeholders / Partners
Academic & Technical: Holly Wiberg, Woody Zhu, Satyam Verma (AI / modeling leads)
Clinical: Dr. Amit Tevar (Transplant Director), UPMC kidney transplant clinical team
This cross-disciplinary collaboration ensures we ground the tool in real clinical realities.
Research
Understanding Clinicians, Workflows, and AI Gaps
Mixed-method research process combining expert interviews, think-aloud sessions, and literature review.
Our focus: uncover how clinicians make dosing decisions, what information they rely on, and how AI recommendations fit, or conflict, with real workflows.
Primary Research: Interviews with 1 doctor, 2 clinicians, and 1 medical student to trace their real-time dosing logic.
Secondary Research: Six academic studies on clinician feedback systems, decision support, and usability in AI healthcare tools.
Synthesis: Collaborative affinity mapping to extract insights and define design opportunities.
What We Learned About Clinical Decision-Making
Our findings revealed that dosing decisions are highly dynamic and intuitive, shaped by both data and experience.
Dosing is a feedback loop.
Clinicians adjust doses based on patient trends over time (e.g., creatinine, urine output) —much like “tasting and adjusting a recipe.”Clinicians input orders, not explanations.
Current systems lack space for reasoning, orders are entered quickly via structured fields. Any new system must balance structured familiarity with flexibility for rationale.Dosage decisions are subjective.
Experience and patient variability mean each clinician may decide differently. This highlighted the need for a tool that supports transparency and nuance, not uniformity.
Prototyping
Entering the prototyping phase, we focused on a key question:
“How might clinicians best document their rationale when accepting, rejecting, or modifying AI-suggested Prograf dosages?”
Mapping the Clinician Journey
We created storyboards to visualize how clinicians might interact with the AI feedback system during their workflow — from reviewing AI recommendations to providing feedback and updating patient records.
This helped us identify the key interaction moments where context, clarity, and reasoning support were critical.
Defining the Feedback Spectrum
Through storyboarding and early prototyping, we identified a spectrum of feedback methods—from open, flexible input to tightly structured formats.
Each option reflected a trade-off between expressiveness and machine readability.
We mapped six approaches along this continuum to test how clinicians naturally communicate their reasoning while ensuring the AI model could interpret it effectively.
Lo-Fi Prototype Testing
We conducted think-aloud testing with nursing practitioners from inpatient and outpatient settings.
Clinicians walked through mock patient cases, reviewed AI-generated Prograf doses, and verbalized how they’d accept, adjust, or reject recommendations.
What We Explored:
How feedback structure impacts cognitive flow
Whether context and reasoning are preserved
How decision patterns differ between care settings
Key Takeaways:
Inpatient clinicians favor rapid, structured feedback.
Outpatient clinicians prefer flexibility to explain nuanced decisions.
Feedback must adapt to both pace and context of each environment.
Lo-fi Prototype Insights + Refinements
Mid-fi Prototype Insights + Refinements
We shifted our goal to a more pressing need after refining our feedback interaction model:
"How can we display patient data to enable clinicians to make informed adjustments to AI-suggested Prograf dosages with ease?"
The shift in focus was driven by usability feedback indicating that timely access to patient data is essential for safe and effective dosage decisions.
We created four mid-fidelity interfaces — table and graph versions for both inpatient and outpatient settings — to test how clinicians interpret information differently.
Table View: prioritizes precision and quick scanning
Graph View: emphasizes visual patterns and longitudinal trends
Inpatient Mode: highlights granular, real-time metrics
Outpatient Mode: emphasizes longer-term adherence and history
Each variation explored how data representation affects confidence, speed, and reasoning.
Finale
Final prototype- A cleaner, more integrated experience that minimizes cognitive load and enables fast, high-quality feedback.
Success Metrics
We evaluated usability by measuring time-to-decision for dose evaluation versus the current EHR workflow.
Result-
~50% Faster Decision Time
Clinicians reached decisions in 0:47s (vs. 1:33s) while maintaining clinical accuracy using our software. This indicates improved efficiency and lower mental effort when reviewing cases.
Although decisions were made 50% faster, speed is not the primary goal for clinicians—accuracy remains the priority. The design emphasizes clarity and efficiency, reducing cognitive load and encouraging sustained engagement. Faster, lower-effort decisions help minimize fatigue and build long-term trust in AI-supported workflows.
The platform allows clinicians to provide structured feedback on AI-generated dosing recommendations, improving model accuracy and confidence in AI-assisted care. This feedback loop enables continuous model refinement and supports the broader adoption of AI in clinical practice.
