What is a medical Artificial Intelligence (AI) application?

From the Guidelines

Medical AI apps should be used with caution, prioritizing FDA-approved or CE-marked apps that have strong privacy policies and come from reputable developers, as they can provide convenient health monitoring but have limitations in accuracy and cannot replace professional medical care. When considering the use of medical AI apps, it is essential to weigh the potential benefits against the risks, including errors at both the individual and societal level, as highlighted in a recent scoping review 1. This review emphasizes the need for a deeper understanding of how consumers interact with direct-to-consumer health care AI apps, beyond merely considering the system's technical specifications.

Key Considerations for Medical AI Apps

• Ensure the app is FDA-approved or CE-marked to guarantee a certain level of safety and efficacy
• Choose apps from reputable developers with strong privacy policies to protect user data
• Be aware of the app's limitations in accuracy and the potential for overdiagnosis or underdiagnosis, as seen in examples such as Google's AI-powered dermatology assist app 1
• Use medical AI apps as a complement to, rather than a replacement for, professional medical care

Recommended Medical AI Apps

• Established apps like Ada Health, K Health, or Babylon for symptom checking
• Specialized apps like Eko for heart monitoring or SkinVision for skin cancer screening
• Apps that provide preliminary health assessments, medication reminders, and health tracking, but always verify any diagnosis or treatment recommendation with a qualified medical provider before taking action, as recommended by recent studies 1

Limitations and Future Directions

• Medical AI apps have limitations in accuracy and cannot replace the nuanced judgment of healthcare professionals
• There is a need for further research on the design and development of medical AI apps that incorporate consumer behavior insights and prioritize user experience, as highlighted in the scoping review 1
• Future studies should focus on systematically summarizing the potential barriers faced by medical AI apps and providing design recommendations for future research, to ensure the safe and effective integration of these apps into healthcare practice.

From the Research

Medical AI App Development and Implementation

• The development and implementation of medical AI apps require a structured approach to ensure safe and effective deployment in healthcare settings 2.
• A key challenge in the successful deployment of AI in healthcare is the need for health system-level governance of AI applications, which is critical for patient safety, accountability, and clinician trust 3.
• The implementation of AI in healthcare practice has been described and researched in the literature, with a focus on establishing the effectiveness of interventions and technical and computational aspects of AI systems 4.

Governance and Regulation of Medical AI Apps

• Effective governance of medical AI apps is essential to ensure that they are used in a way that is safe, equitable, and respectful of patient rights 3.
• Medical regulators have a key role to play in setting standards for doctors who use AI in patient care, and in guiding the profession on emerging and vexed areas of practice such as AI 5.
• The development of guidelines and regulations for the use of AI in healthcare is critical to ensure that AI is used in a way that is transparent, accountable, and respectful of patient rights 5.

Physician Competencies in AI-Assisted Clinical Settings

• The use of AI in clinical settings requires physicians to have a range of competencies, including critical thinking, technical skills, and digital literacy 6.
• There is a need for further research and normative analysis on the responsible use of AI in clinical settings, including the development of clear guidelines on physician competencies and the establishment of a regulatory framework for the use of AI in healthcare 6.
• The connection between trust and efficiency in patient care is an important area of research, and further study is needed to understand how physicians can build trust in AI and use it effectively in clinical settings 6.