Evaluating AI in real world clinical settings
  1. Macquarie University
  2. Faculty of Medicine, Health and Human Sciences
  3. Departments and schools
  4. Australian Institute of Health Innovation
  5. Our research centres
  6. Health Informatics
  7. Our research
  8. Evaluating AI in real world clinical settings
Professor Farah Magrabi

Project information

Find out about this project's description, goals, sponsors and contacts.

Four researchers looking at images on screens

Project sponsor: The National Health and Medical Research Council Centre for Research Excellence in Digital Health (APP1134919)

About the project

In healthcare, AI promises to transform clinical decision-making processes as it has the potential to harness the vast amounts of genomic, biomarker, and phenotype data that are being generated across the health system to improve the safety and quality of care decisions.

Today, AI has been successfully incorporated into variety of clinical decision support systems for detecting clinical findings in medical imaging, suggesting diagnoses and recommending treatments in data-intensive specialties like radiology, pathology and ophthalmology.

Future systems are expected to be increasingly more autonomous, going beyond making recommendations about possible clinical actions to autonomously performing certain tasks such as triaging patients and screening referrals.

Little is known about the use of the present generation of AI in clinical settings, with evaluation of these systems primarily focusing on examining the performance of algorithms within laboratory settings. To date, there have been limited published observational studies in the contemporary literature that have investigated the use of AI-based decision support systems within a clinical setting or within standard of care.

Project goals

The goals of this project are to:

  • investigate approaches to safely implement and evaluate the impact of AI systems on clinical decision-making, care delivery and patient outcomes
  • develop requirements for safe implementation and use of AI in real-world clinical settings.

We welcome enquiries about approaches for safe and effective integration of AI in real-world clinical settings, and are happy to assist individuals and organisations study the impact of AI on decision-making, care delivery and patient outcomes.

Contact: farah.magrabi@mq.edu.au

  1. Magrabi F, Ammenwerth E, McNair JB, De Keizer NF, Hypponen H, Nykanen P, et al. Artificial Intelligence in Clinical Decision Support: Challenges for Evaluating AI and Practical Implications. Yearbook of medical informatics. 2019;28(1):128-34.
  2. Coiera E. Assessing Technology Success and Failure Using Information Value Chain Theory. Stud Health Technol Inform. 2019;263:35-48.
  3. Coiera E. The Last Mile: Where Artificial Intelligence Meets Reality. J Med Internet Res. 2019;21(11):e16323.