What if your medical diagnosis depended on how well you type, or the color of your skin? Imagine trusting an AI generated newscast about your health—only to discover it might be ignoring your pain just because of your gender or ethnicity.

Across hospitals and clinics worldwide, artificial intelligence is revolutionizing healthcare, with big players like Microsoft, Amazon, OpenAI, and Google racing to roll out tools that promise to lighten the load for doctors and speed up patient care. Advanced large language models (LLMs) like ChatGPT, Gemini, and those from start-ups Nabla and Heidi are now generating clinical notes, summarizing visits, and even suggesting treatments. But behind this digital leap lurks a disturbing reality: multiple studies now show that these AI systems can actually make things worse for women and minorities, downplaying their symptoms and sometimes recommending less care.

New research from MIT, UCL, and other top universities has found that medical AI models such as OpenAI’s GPT-4, Meta’s Llama 3, and specialized tools like Palmyra-Med tend to suggest lower levels of care for women, sometimes even telling them to just self-treat at home. The same research revealed a lack of compassion in AI responses to Black and Asian patients seeking mental health help. In some cases, these models gave vastly different advice based purely on the perceived race or gender of the patient. One MIT study even found that patients who wrote messages with typos or informal language were up to 9% more likely to be advised against seeking medical care—putting non-native English speakers and tech newcomers at a clear disadvantage.

This isn’t just a technical hiccup. It’s a mirror of the real-world biases baked into the data these AIs are trained on—data scraped from the internet, where stereotypes and inequality are rampant. As Professor Travis Zack from UCSF points out, “If you’re in any situation where a Reddit subforum is advising your health decisions, that’s not a safe place to be.”

It’s not only American hospitals facing this problem. In the UK, Google’s Gemma model, used by over half the local authorities, has been shown in LSE research to downplay women’s issues compared to men’s in social work case notes. Meanwhile, projects like the NHS-backed Foresight AI, trained on anonymized health data from 57 million Brits, promise a more accurate picture—but have triggered privacy concerns, leading to a government review. Europe’s Delphi-2M system aims to predict disease decades ahead but raises similar questions about data protection and accuracy.

Big tech companies insist they’re tackling the problem: OpenAI says it’s improved newer versions of GPT-4 and now works with clinicians to stress-test models for fairness and safety. Google claims to take model bias “extremely seriously,” developing tools to scrub sensitive data and prevent discrimination. Some researchers, like those at Open Evidence, are turning to more reliable data sources—like peer-reviewed medical journals and FDA guidelines—to train their AI, and ensuring every AI output is backed by solid citations.

But there’s still a long road ahead. As AI generated newscasts about healthcare become more common, the risk of “hallucinations”—AI making up answers—remains a serious threat, especially when lives are on the line. Yet, experts like MIT’s Marzyeh Ghassemi remain hopeful. She believes that if we focus AI on closing the dangerous health gaps that have existed for decades, the technology could bring massive benefits—if we don’t let old biases slip in through the back door.

It’s a high-stakes race: can AI generated newscast about medical bias shine a light before it’s too late? Or will these digital doctors simply reinforce the same old inequalities with a high-tech twist?