AI in Healthcare: Do AI Tools Actually Improve Patient Outcomes?

The conversation about artificial intelligence in healthcare has been buzzing nationally, but here in Chicago, it’s hitting close to home—especially as I walk past the sleek glass towers of the Illinois Medical District on Roosevelt Road, where some of the city’s most advanced hospitals are piloting these very tools. The question isn’t just whether AI can transcribe a doctor’s notes faster; it’s whether these systems are actually making patients healthier, or if we’re just automating inefficiencies without measuring the real impact.

That concern echoes what Jenna Wiens of the University of Michigan and Anna Goldenberg of the University of Toronto raised in their recent Nature Medicine paper: the rapid deployment of AI in clinical settings is outpacing rigorous evaluation of whether these tools improve patient outcomes. While ambient AI scribes—like those from Abridge or Microsoft DAX—are winning praise from doctors for reducing burnout by handling visit documentation, there’s still a critical gap in understanding how they affect clinical decision-making. Does relying on an AI-generated summary change how a physician interprets a patient’s story? Could it subtly alter the cognitive process of diagnosis, especially for younger clinicians still developing their clinical intuition?

Chicago’s own institutions are deeply involved in this transition. Rush University Medical Center has been testing ambient AI tools in its outpatient clinics, aiming to cut down on after-hours charting. Similarly, Northwestern Medicine has integrated AI-assisted radiology tools to flag potential abnormalities in chest X-rays, speeding up initial reads. But as Wiens points out, accuracy in isolation doesn’t equal better care. A tool that quickly flags a lung nodule is only valuable if it leads to timely, appropriate follow-up—not if it causes alarm fatigue or leads to unnecessary invasive procedures because clinicians over-trust its output.

The University of Illinois Hospital & Health Sciences System has similarly joined the growing number of facilities using AI for predictive analytics, particularly in managing sepsis risk in emergency departments. These systems analyze vital signs and lab results in real time to alert clinicians to subtle deterioration. Yet, as the January 2025 study from the University of Minnesota noted—cited in the original piece—only about two-thirds of hospitals using such tools evaluate their accuracy, and even fewer assess them for bias. In a city as diverse as Chicago, where patient populations vary widely across neighborhoods, that’s a serious concern. An algorithm trained primarily on data from one demographic may miss critical signals in another, potentially exacerbating existing disparities in care.

There’s also the quieter, less discussed effect: how these tools influence medical education. At the University of Illinois Chicago College of Medicine, students are now encountering AI-augmented workflows in their training. If they grow accustomed to relying on AI-generated summaries or risk scores, will they develop the same depth of pattern recognition and clinical reasoning as physicians trained in the pre-AI era? Wiens’ concern about unintended consequences isn’t theoretical—it’s already unfolding in lecture halls and simulation labs across the city.

Despite these unknowns, the belief in AI’s potential remains strong among Chicago’s healthcare leaders. As Wiens herself said, she doesn’t want to halt adoption—she wants smarter, more measured integration. The goal isn’t to reject these tools, but to ensure they’re evaluated not just for technical accuracy, but for their real-world effect on diagnosis, treatment plans, and patient survival and quality of life.

Given my background in biotechnology and health policy, if this trend impacts you in Chicago, here are the three types of local professionals you need to understand as AI reshapes clinical workflows:

First, gaze for clinical informatics specialists who work directly within hospital systems like Rush or Northwestern. These aren’t just IT staff—they’re often nurses or physicians with advanced training in how technology integrates into patient care. The best ones don’t just install software; they design workflow studies, track outcome metrics, and advocate for rigorous evaluation of AI tools before they go live. Ask if they’ve led studies measuring changes in patient safety events or clinician cognitive load after implementation.

Second, seek out health equity analysts or bias auditors—often found in academic settings like the University of Illinois School of Public Health or independent consultancies focused on algorithmic fairness. These professionals specialize in testing whether AI models perform consistently across different racial, age, and socioeconomic groups. In a city with stark health disparities between neighborhoods like Englewood and Lincoln Park, their work is essential to ensure predictive tools don’t inadvertently worsen outcomes for vulnerable populations.

Third, consider medical educators or curriculum designers at Chicago’s medical schools who are actively studying how AI affects clinical training. The most valuable ones aren’t just teaching how to use the latest scribe tool—they’re researching whether reliance on AI affects diagnostic accuracy in simulations, or how it changes patient-doctor communication. They should be able to point to ongoing studies or partnerships with local hospitals assessing these second-order effects.

Ready to find trusted professionals? Browse our complete directory of top-rated biotechnologyandhealthappthecheckup experts in the Chicago area today.