5 Questions as AI-Powered Tools Inundate Healthcare

AI-powered clinical decision support systems (CDSS) facilitate evidence-based decision-making by synthesizing medical data and offering real-time insights to clinicians. This assists healthcare providers in making well-informed, timely decisions regarding patient care, thereby reducing errors, improving treatment efficacy, and leading to better overall health outcomes.

Recent examples of CDSS successes include:

• In a busy obstetrical unit, clinician researchers at Maimonides Medical Center in New York recently used Medical Brain, an AI-based CDSS, to reduce the incidence of certain predefined adverse clinician events, which they termed “Red Never Events.” Researchers tracked the incidence of three essential actions not taken: (1) lack of administering antibiotics for group B strep positive patients in labor, (2) lack of administering magnesium sulfate for seizure prophylaxis in patients with severe preeclampsia, and (3) failure to discontinue Pitocin in laboring patients with a non-reassuring fetal heart rate tracing. Over four phases of rollout, from 2018 through 2022, the implementation of Medical Brain helped them achieve a 90 percent drop in the tracked RNEs.
• Researchers at Mount Sinai have developed a new tool for the interpretation of electrocardiograms (ECGs). The ECG is valued as a tool that is noninvasive, low-cost, and applicable to many patient presentations. However, the ECG’s usefulness as a diagnostic tool has been limited by the difficulty of interpreting some of the patterns it uncovers. Now, a new AI model for ECG analysis, developed by researchers at Mount Sinai, allows the ECG to be interpreted as language. This approach can enable more accurate diagnosis, especially for rare conditions. So far, HeartBEiT is outpacing prior machine-learning models for ECG interpretation.

By analyzing vast amounts of data from clinical trials, patient records, and scientific literature, AI can pinpoint diagnostic criteria, identify potential drug candidates, assist with population health studies, and streamline the research process overall.

Expedited disease diagnosis: AI has the potential to significantly impact health on a global scale by facilitating expedited disease diagnosis. For instance, early on in our experience with COVID-19, we were able to convert anecdotal reports of anosmia, a loss of smell, to a recognized early symptom of the virus through machine analysis of large numbers of medical records. This recognition helped prevent the spread of the virus, and it continues to assist clinicians with diagnosis and differentiating COVID-19 from the flu.

Risk stratification: AI can also assist clinicians in selecting tailored medical interventions, such as when addressing population health concerns by helping clinicians risk stratify groups of patients. AI-powered risk stratification opportunities include everything from helping hospitals identify patients at higher risk of readmission to helping radiologists identify patients whose injury patterns raise flags for intimate partner violence, clarifying their decisions regarding treatment options and when to proceed with tough conversations with patients.

Outbreak response: One prominent application for AI is to enable effective epidemiological surveillance to address health challenges and disparities worldwide. By identifying patterns and trends in population health data, AI-driven tools present an opportunity for predicting and addressing public health crises, which can help public health officials monitor and respond to outbreaks of infectious diseases or other health emergencies. This rapid response capability can save lives by mitigating the impact of epidemics and facilitating the timely allocation of resources where they are needed most.

Telemedicine services are particularly vital for underserved populations and remote areas with limited access to medical professionals. When appropriate, AI-powered diagnostics and treatment recommendations can extend the reach of medical expertise.

• Example: Video visits for pregnant patients: Around the country, various large healthcare systems have trialed programs for low-risk pregnant patients that replace some in-person prenatal care visits with videoconference consultations with providers. Outcomes overall are positive. Postnatal care also presents opportunities for telemedicine visits with medical professionals, such as for lactation support. For those integrating telemedicine visits into prenatal and postnatal care, AI can assist with risk stratification to determine which patients are eligible for more virtual visits, vs. in-person visits.

When patients do need to be seen in person, AI-driven tools can help with triage to determine how quickly they need to be seen, and by whom.

• Example: Faster triage for mass casualty events: One dramatic example of AI in action is the use of machine learning to develop a model for faster triage during response to mass casualty events by firefighters and other responders, with a goal of maximizing the number of survivors who can be assisted by a presumably small number of fast-working, early-arriving clinician responders.

Whether for everyday situations or life-threatening emergencies, the goal is to use AI-powered tools to supplement in-person care. Remote monitoring is an area of telehealth where AI shows promise. Examples include:

• Example: Remote monitoring for infection prevention: AI-powered applications may assist certain patients with COVID-19 in self-isolating at home more safely. A smart-phone app and biosensor that monitor physiological parameters like respiration rate, pulse rate, and cough sounds can assist with risk stratification by identifying patients who are at risk for rapid deterioration—and who need to be seen promptly.
• Example: Determining level of care: AI-powered tools are being developed to identify those patients with diabetes and some existing renal impairment who show greater propensity for rapid deterioration, and who may benefit from more aggressive intervention.

While advancements in technology offer significant potential for improving healthcare and education, some innovations may further widen the digital divide. Access to and utilization of technology and AI-driven resources may not be uniform across populations that include different ages, socioeconomic groups, and geographic locations, leading to e-health disparities.

Further, healthcare information technology (HIT) and AI can inadvertently incorporate biases, negatively impacting underserved populations:

• Example: Skin tone and healthcare devices: Many medical professionals are committed to combatting expressions of racism when they encounter those in their professional setting—yet racism can sneak into their practice in the very devices they are using for care. For instance, during the COVID-19 pandemic, it was discovered that pulse oximeters did not accurately capture blood oxygen saturation measurements in individuals with darker skin, leading to poorer outcomes.
• Example: Race and risk calculators: Race-based and gender-based biases can also linger in medical pathways and algorithms, increasing risks for certain populations. For instance, the algorithm to proceed with vaginal birth after cesarean section (VBAC) formerly factored in race, disproportionally considering women of color at higher risk for complications when undergoing VBAC. As a result, they were less likely to be provided the opportunity to trial VBAC delivery and benefit from the potential advantages associated with vaginal delivery, such as lower rates of surgical complications, faster recovery, and fewer complications. As a consequence of this bias, BIPOC U.S. women faced disproportionately higher rates of cesarean sections compared to their white counterparts. In recognition of the biases being promoted by this algorithm and its undesirable effects on quality of care, the algorithm was recently revised to remove the race adjustment.

To counteract these biases and ensure the equitable distribution of technology's benefits, concerted efforts should focus on expanding access to technology infrastructure, investing in digital literacy programs, and fostering a diverse and inclusive approach to the development and implementation of digital tools in healthcare and education. Engaging stakeholders from various sociodemographic backgrounds in the design and evaluation of digital applications can help ensure that the resulting technologies are both effective and equitable. Many frameworks for participatory design are currently being tested, and attention paid to successes in this area will no doubt be fruitful.

Plausible vs. factual: While ChatGPT is adept at understanding natural language and responding to queries, there are instances where it may generate inaccurate information, which can include instances of AI "hallucination." These hallucinations occur when the AI system generates information that appears plausible but is not based on actual facts or evidence. Although AI-driven tools are capable of providing valuable information and guidance, they are not infallible. There is the possibility of generating inaccurate or outdated information, which underscores the importance of students and educators verifying the information provided, developing critical thinking skills, and cross-referencing with other reliable sources.

Potential for bias: A further concern is the potential amplification of pre-existing biases. AI systems, including ChatGPT, are trained on vast datasets that may contain implicit biases present in the source material. This can lead to AI-generated responses reflecting and possibly amplifying these biases.

• Example: Gender bias: A notable example of AI exhibiting implicit bias occurred in 2018 with the release of Amazon's AI recruiting tool. The tool was designed to review resumes and rank job applicants according to their qualifications. However, it was discovered that the AI system displayed a significant gender bias, favoring male applicants over female applicants for technical roles. This bias arose because the AI had been trained on resumes submitted to Amazon over a ten-year period, during which the majority of applicants for technical positions were male. Consequently, the AI system learned to associate male candidates with desirable qualifications and implicitly downgraded resumes containing words associated with female applicants, such as attending an all-women's college.

Example: Nonrepresentative patient data: Likewise, in healthcare, if a diagnostic tool is trained on a dataset from predominantly male and/or Caucasian patients, it may not be reliable for use with patients who are women and/or people of color. Therefore, it is crucial for users to be aware of this potential shortcoming and actively work to counteract it by maintaining a critical mindset and engaging in thoughtful discussions with fellow students, educators, and healthcare professionals.