Using AI as a Reviewer in Clinical Workflows

This study could change how every frontline clinic in the world delivers care. Penda Health and OpenAI revealed that an AI tool called AI Consult, embedded into real clinical workflows in Kenya, reduced diagnostic errors by 16% and treatment errors by 13%—across nearly 40,000 live patient visits. This is what it looks like when AI becomes a real partner in care. The clinical error rate went down and clinician confidence went up. 🤨 But this isn’t just about numbers. It’s a rare glimpse into something more profound: what happens when technology meets clinicians where they are—and earns their trust. 🦺 Clinicians described AI Consult not as a replacement, but as a safety net. It didn’t demand attention constantly. It didn’t override judgment. It whispered—quietly highlighting when something was off, offering feedback, improving outcomes. And over time, clinicians adapted. They made fewer mistakes even before AI intervened. 🚦 The tool was designed not just to be intelligent, but to be invisible when appropriate, and loud only when necessary. A red-yellow-green interface kept autonomy in the hands of the clinician, while surfacing insights only when care quality or safety was at risk. 📈 Perhaps most strikingly, the tool seemed to be teaching, not just flagging. As clinicians engaged, they internalized better practices. The "red alert" rate dropped by 10%—not because the AI got quieter, but because the humans got better. 🗣️ This study invites us to reconsider how we define “care transformation.” It's not just about algorithms being smarter than us. It's about designing systems that are humble enough to support us, and wise enough to know when to speak. 🤫 The future of medicine might not be dramatic robot takeovers or AI doctors. It might be this: thousands of quiet, careful nudges. A collective step away from the status quo, toward fewer errors, more reflection, and ultimately, more trust in both our tools and ourselves. #AIinHealthcare #PrimaryCare #CareTransformation #ClinicalDecisionSupport #HealthTech #LLM #DigitalHealth #PendaHealth #OpenAI #PatientSafety

AI in Clinical Data Management (CDM) AI is transforming Clinical Data Management by automating manual and error-prone tasks — from CRF design to data cleaning, query handling, and database lock. Here’s how AI fits across the CDM lifecycle: 1. Protocol → Study Build (Setup Phase) AI Use Cases: Protocol Parsing: Automatically extract study design elements (arms, visits, endpoints, assessments). Auto-generate eCRFs: Create CDASH-compliant CRFs and edit checks from protocols and standards. CDISC Mapping: Suggest CDASH/SDTM mappings for CRF fields. Data Dictionary Creation: Build metadata tables with variable names, labels, datatypes, and units. Approach/Tools: LLMs fine-tuned on CDISC/CDASH, NLP (spaCy, transformers) for structured extraction, integration with EDC APIs (Rave, Veeva, Oracle). 2. During Data Collection AI Use Cases: Smart Data Entry: Auto-fill/validate fields using context (e.g., logical consistency, range checks). Dynamic Edit Checks: Predict likely invalid data (e.g., male + pregnancy test). Duplicate/Outlier Detection: Flag duplicate IDs or implausible lab values in real time. Techniques: ML models for anomaly detection and predictive validation integrated into data pipelines (Python, R, SAS). 3. Data Cleaning & Query Management AI Use Cases: Automated Query Generation: Identify discrepancies and raise queries. Query Prioritization: Rank by data quality impact or timeline risk. Resolution Suggestions: NLP models draft likely responses. Outlier/Trend Analysis: Detect abnormal site or patient patterns. Tools: ML models trained on query logs; Power BI/Tableau dashboards highlighting predicted risks. 4. Data Review & Reconciliation AI Use Cases: Cross-System Reconciliation: Compare EDC, IVRS, and lab data. SDTM Validation: Ensure CDASH → SDTM mappings meet CDISC rules. Medical Coding: NLP suggests MedDRA/WHODrug terms. Signal Detection: Identify unusual subject or visit patterns. Techniques: Python/Pandas for rule-based checks; transformers (BERT, BioBERT) for coding normalization. 5. Database Lock & Post-Lock Activities AI Use Cases: Final QC Automation: Validate edit checks and metadata completeness. Timeline Prediction: Forecast lock delays using site/query metrics. Regulatory Submission: Auto-generate define.xml, reviewer guides, annotated CRFs. 6. Operational Metrics & Oversight AI Use Cases: Automated Dashboards: Track data entry lag, query closure, CRF completion. Risk-Based Monitoring: Predict high-risk sites/patients for focused review. Resource Forecasting: Estimate workload based on query trends. Tools: Power BI, Streamlit, Python ML models via Flask/FastAPI. 7. Emerging & Advanced Applications Generative AI Assistants: Natural language queries like “Show subjects missing visit dates last 7 days.” Automated SDTM/ADaM Drafting: LLMs create SAS-ready datasets and define.xml templates. AI Audit Companion: Summarize audit logs and deviations. Knowledge Bots: Train on SOPs, standards, and prior study documents.

Clinicians working collaboratively with a custom GPT diagnostic assistant outperformed those using traditional resources, highlighting the power of AI-human synthesis for complex diagnostic reasoning. 1️⃣ A randomized trial with 70 US physicians tested two workflows using a GPT-based assistant: AI-first (AI suggests before clinician) and AI-second (clinician suggests before AI). 2️⃣ Both AI workflows significantly outperformed conventional tools, 85% (AI-first) and 82% (AI-second) vs. 75% with traditional resources. 3️⃣ Diagnostic accuracy gains were strongest in “clinically actionable decisions”, final diagnosis and next steps, especially when AI was used first (8.9% higher than AI-second). 4️⃣ AI support narrowed the spread of low scores, helping raise the floor of performance without sacrificing clinician autonomy. 5️⃣ When clinicians went first, the AI sometimes anchored its “independent” response to their input, despite explicit instructions not to, suggesting the model was influenced by human suggestions. 6️⃣ Clinicians were more likely to anthropomorphize and interact conversationally with the AI when it came second, indicating workflow order influences trust and engagement. 7️⃣ Using AI first was slightly faster per case (by ~1.5 minutes), though not significantly so unless strict protocol adherence was enforced. 8️⃣ Despite high performance, the AI’s recommendations occasionally worsened clinician answers (~8% of cases), pointing to real-world safety considerations. 9️⃣ Post-study, clinician willingness to use AI in complex reasoning increased significantly, with nearly all participants reporting high satisfaction and perceived value. 🔟 The study shows that workflow design, not just model capability, is crucial to unlocking AI’s potential in diagnostic support. ✍🏻 Selin Everett, Bryan Bunning, Priyank Jain, Ivan Lopez, Anup Agarwal, Manisha Desai, Robert Gallo, Ethan Goh, MD, Vinay Kadiyala, Zahir Kanjee, Jacob Koshy, Andrew Olson, Adam Rodman, Kevin Schulman, Eric Strong, Jonathan Chen, Eric Horvitz. From Tool to Teammate: A Randomized Controlled Trial of Clinician-AI Collaborative Workflows for Diagnosis. medRxiv. 2025. DOI: 10.1101/2025.06.07.25329176

Could AI drafts—even imperfect ones—be a time-saver for radiologists when interpreting CT scans? Our pilot study using simulated AI reports found a 24% faster workflow, with accuracy intact. Q: What makes this study's approach unique? A: Instead of building an AI system, we used GPT-4 to simulate what AI-generated draft reports might look like. We deliberately introduced 1-3 errors in half the drafts to study how radiologists would handle imperfect AI assistance - a "Wizard of Oz" approach to prototype the future workflow. Q: How was the simulation study structured? A: We conducted a 3-reader crossover study with 20 chest CT cases. Each case was read twice: once with standard templates, and once with our simulated AI drafts. This controlled design let us directly compare the workflows. Q: What efficiency gains did you see with the simulated drafts? A: Median reporting time dropped from 573 to 435 seconds (p=0.003) - a 24% reduction. Two readers showed major improvements (717→398s and 361→322s), while one showed an increase (947→1015s). Q: Did the intentionally flawed drafts impact accuracy? A: Surprisingly, even with deliberately introduced errors in half the simulated drafts, the AI-assisted workflow showed slightly fewer clinically significant errors (0.27±0.52) compared to standard workflow (0.38±0.78). While not statistically significant, this suggests radiologists maintained their vigilance even with imperfect drafts. Q: How did radiologists respond to working with these simulated drafts? A: All 3 readers found the prototype system easy to use and well-integrated into their workflow. Two reported somewhat less mental effort, while one reported significantly reduced effort. Their likelihood to recommend it varied (scores of 5, 9, and 10 out of 10). Q: What's next? A: While these simulation results are encouraging, these are small scale pilot studies setting the stage for deeper validation. Link to short paper: https://lnkd.in/d-4aTJ69 Congratulations to stellar team of Julián Nicolás Acosta, Siddhant Dogra, Subathra Adithan, Kay Wu, MD 💫, Michael Moritz, Stephen Kwak

Did you see the recent news??? Microsoft recently unveiled its latest AI Diagnostic Orchestrator (MAI DxO), reporting an impressive 85.5% accuracy on 304 particularly complex cases from the New England Journal of Medicine, compared to just ~20% for physicians under controlled conditions . These results—quadrupling the diagnostic accuracy of human clinicians and more cost-effective than standard pathways — have gotten a lot of buzz. They may mark a significant milestone in clinical decision support and raise both enthusiasm but also caution. Some perspective as we continue to determine the role of AI in healthcare. 1. Validation Is Essential Promising results in controlled settings are just the beginning. We urge Microsoft and others to pursue transparent, peer reviewed clinical studies, including real-world trials comparing AI-assisted workflows against standard clinician performance—ideally published in clinical journals. 2. Recognize the value of Patient–Physician Relations Even the most advanced AI cannot replicate the human touch—listening, interpreting, and guiding patients through uncertainty. Physicians must retain control, using AI as a tool, not a crutch. 3. Acknowledge Potential Bias AI is only as strong as its training data. We must ensure representation across demographics and guard against replicating systemic biases. Transparency in model design and evaluation standards is non-negotiable. 4. Regulatory & Liability Frameworks As AI enters clinical care, we need clear pathways from FDA approval to liability guidelines. The AMA is actively engaging with regulators, insurers, and health systems to craft policies that ensure safety, data integrity, and professional accountability. 5. Prioritize Clinician Wellness Tools that reduce diagnostic uncertainty and documentation burden can strengthen clinician well-being. But meaningful adoption requires integration with workflow, training, and ongoing support. We need to look at this from a holistic perspective. We need to promote an environment where physicians, patients, and AI systems collaborate, Let’s convene cross sector partnerships across industry, academia, and government to champion AI that empowers clinicians, enhances patient care, and protects public health. Let’s embrace innovation—not as a replacement for human care, but as its greatest ally. #healthcare #ai #innovation #physicians https://lnkd.in/ew-j7yNS

From CommunityLIVE by Hyland: AI that helps clinicians and patients, today. I sat down with Dr Shankar Sridharan, baby heart doctor, Chief Clinical Information Officer at Great Ormond Street Hospital, and National Clinical Lead for AI at NHS England. We spoke about taking AI out of pilots and into frontline care. Scale and design - The NHS has run the largest generative AI program with ambient voice across nine care settings so far. - 17,000 patients included with a staged, scientific approach. - Phase zero in the innovation lab, then professional actors and doctors in a safe test EHR, then real clinicians with real patients, then scale. Measured outcomes - 25% increase in direct care time. - In Accident and Emergency, documentation time reduced by 51%. - Independent analysis by York Health Economics Consortium shows each A&E doctor can see at least one extra patient per shift. - At NHS scale, that is 9,279 additional patients per day, a capacity benefit north of 650 million with a further documentation benefit of 160 million. Why now - Algorithmic AI needed high digital maturity and often lived in imaging. - Generative AI can work with unstructured data and needs operationalisation more than heavy new infrastructure. How to operationalise - People, process, technology in that order. - Train clinicians. Integrate into workflow. Measure time to decision, accuracy, and rework, not just model scores. Governance and assurance - Build at the speed of trust. Clear governance for data, cyber, and clinical safety, and assurance so the public can see how risk is managed. - Move beyond pilotitis Tiny pilots with no owner and no scale plan slow us down. Create a national plan with strategic delivery, then expand by use case. - What good looks like in practice Ambient voice reduces documentation load so clinicians focus on patients. Better throughput and clinicians who feel less drained at the end of a shift. The next step - Use AI as the tenth voice in multidisciplinary teams. - Not to replace clinicians, but to widen perspective, surface risks, and nudge better decisions. As Shankar put it, the clinician is the superhero. AI is the cape. This is how AI earns trust in healthcare. Results first. Safety always. Scale with discipline. Full interview link in the comments. #data #ai #CommunityLIVE25 #hyland #theravitshow

Clinicians like me can interact with AI in two fundamentally different ways. In one paradigm, AI generates output for us to review and verify. In the other, we do the work and AI checks it for us. Both models have value. But the key is choosing the right one for the task at hand. That choice depends on several factors, including the clinician's capabilities, their preferences, and the reliability of the AI output. When I’m reviewing records or drafting (straightforward) notes, I often want AI to go first — I’ll check its work. But when I’m making decisions or replying to patient messages, I want to lead — and have AI check me. When those roles are mismatched, AI becomes a burden instead of a help. Here’s a non-clinical example, since (unlike EHRs) I can share the image below: I almost never use Copilot to draft emails. Yet every time I open a message, Copilot auto-suggests a few replies, unnecessarily taking up screen space and interrupting my flow. It’s the wrong paradigm, forced into the wrong moment. Stop showing it to me, please. Conversely, what stood out to me in the recent Penda Health / OpenAI study was how the AI reduced errors by %13-16% withOUT inserting itself into every clinical encounter. Instead of continually intruding onto the EHR screen, it operated more like a background safety net — surfacing only when it mattered. More health tech should be designed with that kind of discernment — and less like Copilot emails, intrusively barging onto our screens and into our minds whether useful or not. It's not just about what AI can do, but whether — and when — it should show up at all.

Case Tuesday: Workflow Optimization It’s a typical morning in a large hospital radiology department: Dozens of CT, MRI, and X-ray studies waiting to be read Some cases are routine follow-ups, others are life-threatening emergencies Radiologists must balance speed, accuracy, and communication with busy clinical teams The challenge: Critical cases can get buried in a growing worklist Reporting bottlenecks delay clinical decision-making Communication gaps slow down patient care across departments #AI is stepping in to help beyond diagnosis itself: Automatically prioritizing urgent cases (e.g., suspected stroke, hemorrhage, PE) Routing studies to the right subspecialist radiologist Streamlining reporting and reducing administrative burden Integrating with hospital systems to ensure faster handoff to clinicians The radiologist is still at the center of the process but now supported by an ecosystem that reduces friction and ensures patients get the right care, faster. The impact: More efficient workflows that free up radiologists to focus on complex cases Shorter turnaround times for critical findings Improved collaboration across multidisciplinary teams As Chief Medical Officer at GE HealthCare, I see workflow optimization as the “hidden superpower” of AI; less visible than detecting a tumor or a clot, but just as vital for ensuring patients get the care they need at the moment they need it. Where do you see the greatest potential for AI in radiology helping with diagnosis, or transforming the workflow itself? #CaseTuesday #RadiologyWorkflow #AIinHealthcare #FutureOfMedicine #GEHealthcare

Utah may have just shown us where clinical AI starts to earn trust. Not in the most dramatic parts of medicine, but in the routine work that quietly consumes physician time every day. Prescription renewals are not flashy, but they are exactly where access, safety, liability, and workflow all collide. The early Doctronic pilot data is still preliminary. It is Phase One, the patient numbers are limited, and Utah’s Office of AI Policy says an independent review is still underway. But the signal is interesting. In 72% of cases, the AI recommended prescription renewal, and every one of those cases still went to a human physician for review. The reviewing physician agreed with the AI’s recommendation 91% of the time. When there was disagreement, the issue was usually that the physician wanted more information, such as labs or more symptom detail. After a second physician reviewed those cases, 97% were ultimately considered appropriate by at least one reviewing physician. The other side of the pilot may be even more important. In 28% of cases, the AI did not recommend renewal and escalated the patient to a physician. Physicians agreed that escalation was appropriate 69% of the time, while 31% were considered overly cautious. That is probably the right failure mode for an early clinical AI system. Overcaution can create friction. But undercaution can create harm. The challenge is finding the line where AI improves access without lowering the standard of care. This is what I think healthcare needs to debate more honestly. We should not compare AI to an imaginary version of medicine where every human decision is perfect. We should compare it to real-world care, where clinicians disagree, patients wait, refills get delayed, and risk is already being managed under time pressure. Prescription renewal may sound boring. But boring workflows are where trust gets built. If AI can help physicians handle low-complexity care faster, with clear escalation rules and transparent auditing, that is a serious healthcare delivery story. The key question is not whether AI should be involved. The key question is what oversight, documentation, and clinical boundaries need to exist before this scales. Curious how others see this, especially physicians, pharmacists, telehealth clinicians, and healthcare operators. Would you be comfortable with an AI-assisted prescription renewal workflow if every approval still required physician review?

Can an # AI #Doctor partner with clinicians? Can we please move past the AI versus doctor/clinician comparisons in taking board exams.. solving diagnostically challenging cases... providing more empathetic on-line responses to patients...? and instead focus on improving patient care and their outcomes? The authors, Hashim Hayat, Adam Oskowitz et. al. at the University of California, San Francisco, of a recent study may be hinting at this: envisioning an agentic model (Doctronic) “used in sequence with a clinician” to expand access while letting doctors focus on high‑touch, high‑complexity care and supporting the notion that AI’s “main utility is augmenting throughput” rather than replacing clinicians (https://lnkd.in/e-y3CnuF)  In their study: ▪️ >100 cooperating LLM agents handled history evaluation, differential diagnosis, and plan development autonomously. ▪️ Performance was assessed with predefined LLM‑judge prompts plus human review. ▪️ Primary diagnosis matched clinicians in 81 % of cases and ≥1 of the top‑4 matched in 95 %—with no fabricated diagnoses or treatments. ▪️AI and clinicians produced clinically compatible care plans in 99.2 % of cases (496 / 500).  ▪️In discordant outputs, expert reviewers judged the AI superior 36 % of the time vs. 9 % for clinicians (remainder equivalent). Some key #healthcare AI concepts to consider: 🟢 Cognitive back‑up, in this study, the model identified overlooked guideline details (seen in the 36 % of discordant cases; the model used guidelines and clinicians missed). 🟢 Clinicians sense nuances that AI cannot perceive (like body‑language, social determinants). 🟢 Workflow relief , Automating history‑taking and structured documentation, which this study demonstrates is feasible, returns precious time to bedside interactions. 🟢 Safety net through complementary error profiles – Humans misdiagnose for different reasons than #LLMs; so using both enables cross‑checks that neither party could execute alone and may have a synergistic effect. Future research would benefit from designing trials that directly quantify team performance (clinician/team alone vs. clinician/team + AI) rather than head‑to‑head contests, aligning study structure with the real clinical objective—better outcomes through collaboration. Ryan McAdams, MD Scott J. Campbell MD, MPH George Ferzli, MD, MBOE, EMBA Brynne Sullivan Ameena Husain, DO Alvaro Moreira Kristyn Beam Spencer Dorn Hansa Bhargava MD Michael Posencheg Bimal Desai MD, MBI, FAAP, FAMIA Jeffrey Glasheen, MD Thoughts? #UsingWhatWeHaveBetter