An official website of the United States government
The NIH conducted planning activities to inform the Precision Medicine with AI: Integrating Imaging with Multimodal Data (PRIMED-AI) program concept. This concept will aim to harness advancements in artificial intelligence (AI) technologies to enable integration of clinical imaging data with a variety of other health data to support the clinical decision-making process. Such a program would transform disease prevention, detection, diagnosis, and treatment, ultimately improving patient outcomes. The NIH Council of Councils approved PRIMED-AI as a Common Fund program in April 2025 - view the videocast (starting at 04:00:30).
Strategic planning activities focused on a Landscape Analysis, Request for Information, and Strategic Planning Workshop.
Landscape Analysis
The objectives of the PRIMED-AI landscape analysis were to survey, explore, and identify applications of AI, machine learning (ML), and deep learning (DL) in biomedical and clinical research related to the integration of medical imaging and non-imaging multimodal data for precision medicine. Current activity and near-term trends in this landscape were analyzed for their ability to deliver innovative solutions in patient care and improve patient outcomes. This analysis informed the development of the PRIMED-AI program concept.
Interest, investment, and expertise exist, but are siloed.
- There is interest in AI, clinical imaging, and precision medicine projects that aligns with the PRIMED-AI concept, but efforts are scattered and often siloed. The scope of NIH-supported projects in these areas has often focused on a single data type in isolation, such as radiological scans, retinal photographs, or pathology images (1).
- The number of related projects supported by the NIH has remained largely unchanged (2).
- The NIH currently has no coordinated investment in this area; additional funding could enable significant strategic progress on precision medicine by providing paths for interdisciplinary team science to overcome current limitations in growth and scope.
AI/ML infrastructure and capacity are rapidly building.
- Due to the complexity of human health data, AI/ML is essential for processing large, diverse datasets to identify patterns and make advanced predictions that unlock potential to directly benefit both individual patients and their caregivers (3).
- The integration of AI with multimodal data analysis in medical imaging has the potential to significantly enhance precision medicine (4).
- A paradigm shift toward AI-supported analysis is needed to promote integration of medical imaging with multimodal health data in enabling novel precision medicine strategies.
Investment in the PRIMED-AI concept is timely.
- The AI medical imaging market is growing rapidly, driven by technological advancements and an increasing demand for clinical imaging (5). As a result, investments in AI initiatives like those proposed for PRIMED-AI are highly advantageous and well-positioned for impact and success.
- The global market for AI deployment in healthcare is ripe. Internationally, AI approaches applied to medical imaging are advancing through a variety of initiatives and investments (6).
- Existing regulatory frameworks are not yet fully equipped to handle the complexities of AI. PRIMED-AI would be uniquely positioned to provide continuity through partnerships across diverse communities and influence the regulatory space (3).
There are many potential federal and non-federal partners that could enhance the PRIMED-AI concept.
- There are multiple NIH and ARPA-H programs that align with the proposed timeline of PRIMED-AI and could be leveraged for infrastructure, tools, and data to support PRIMED-AI awardees. These programs could complement each other and support an NIH-wide initiative on AI that is “bigger than the sum of its parts.”
- The implementation and adoption of PRIMED-AI deliverables would be improved through collaborative partnerships such as with FDA, FNIH, AHRQ, and ARPA-H.
- Private partners should be evaluated for potential engagement in academic-industrial partnerships with awardees, transition partners for PRIMED-AI developments, and community stakeholders to inform the rapidly developing AI space.
NIH is well-positioned to lead this potential initiative.
- While the NIH is broadly investing in research related to AI/ML, clinical imaging, and precision medicine, a coordinated programmatic effort that catalyzes an intersection of these three topics does not currently exist.
- The PRIMED-AI concept is synergistic, disease agnostic, and would advance the missions of multiple NIH Institutes, Centers, and Offices, 18 of which are represented in the PRIMED-AI working group.
- The PRIMED-AI concept addresses obstacles for the precision medicine field that are uniquely suited to team science solutions at the NIH-wide level.
References for Landscape Analysis
- Analysis of internal administrative data.
- Estimates of Funding for Various Research, Condition, and Disease Categories (RCDC) .
- Labkoff, S., Oladimeji, B., Kannry, J., Solomonides, A., Leftwich, R., Koski, E., Joseph, A. L., Lopez-Gonzalez, M., Fleisher, L. A., Nolen, K., Dutta, S., Levy, D. R., Price, A., Barr, P. J., Hron, J. D., Lin, B., Srivastava, G., Pastor, N., Luque, U. S., Bui, T. T. T., Singh R., Williams T., Weiner M. G., Naumann T., Sittig D. F., Jackson G. P., Quintana, Y. (2024). Toward a responsible future: recommendations for AI-enabled clinical decision support . Journal of the American Medical Informatics Association: JAMIA, 31(11), 2730–2739.
- Poalelungi, D. G., Musat, C. L., Fulga, A., Neagu, M., Neagu, A. I., Piraianu, A. I., & Fulga, I. (2023). Advancing Patient Care: How Artificial Intelligence Is Transforming Healthcare . Journal of Personalized Medicine, 13(8), 1214.
- ARPA-H Investor Catalyst Hub. (2024). Medical Imaging Data Marketplace Survey Report . Advanced Research Projects Agency for Health (ARPA-H).
- Contreras, B. (2023). AI in medical imaging market expected to increase to $14.2 billion by 2032 . Managed Healthcare Executive.
Request for Information
The Common Fund issued a Request for Information (RFI, NOT-RM-24-011) to identify high priority challenges and opportunities in developing trustworthy, cost-effective, accessible, ethical, and sustainable precision medicine AI algorithms that integrate medical images with other patient-related data to support clinical decision making. Community input was requested on:
- Imaging and related multimodal data integration approaches
- Developing AI-based clinical decision support tools that leverage clinical imaging and multimodal data integration
- Demonstrating clinical utility of multimodal algorithms for precision medicine.
The goal of the PRIMED-AI program concept is to determine how to use AI to bring together medical imaging with multiple, diverse types of health data (or "multimodal data") in a way that supports precision medicine. The approach to the concept was informed by extensive input from the public and the research community, through responses to a Request for Information (RFI) and several informational calls. These inputs underscored that standardized imaging protocols, high-quality data curation, and robust ethical considerations to ensure the reliability, reproducibility, and fairness of AI-driven healthcare tools would be necessary for PRIMED-AI to achieve its vision. Collaborative efforts involving academia, industry, and healthcare providers, alongside innovative AI methods, are essential for developing robust AI models that can be used in a variety of situations. The following core concepts will guide future PRIMED-AI funding announcements:
- Standardization and Data Integration: Emphasizing standardized imaging protocols, such as those needed for MRI sequences, and meticulous metadata retention to improve the reliability of AI models.
- AI-Based Clinical Decision Support Tools: Addressing the need to harmonize machine learning platforms and address bias and privacy concerns through advanced AI techniques.
- Demonstrating Clinical Utility: Highlighting the importance of diverse datasets and partnerships with Electronic Health Record (EHR) vendors and others to measure the impact of AI tools on patient outcomes.
- Ethical Considerations: Focusing on data privacy, bias mitigation, and interdisciplinary collaboration to ensure the equitable and ethical deployment of multimodal AI tools in precision medicine.
- Innovative Methods: Adopting cutting-edge AI approaches such as federated learning to enhance data privacy, model transparency, and overall AI robustness.
Full Report of RFI Responses
Strategic Planning Workshop
As part of strategic planning activities, the NIH Common Fund hosted a virtual workshop to engage subject matter experts, community members, and NIH staff in a discussion of high-priority areas for the NIH to consider when developing the PRIMED-AI program concept. The primary focus was to identify opportunities and challenges to enable precision medicine with artificial intelligence (AI) through the integration of clinical imaging and multimodal data. The session topics included:
- Developing algorithms that leverage multimodal data to solve clinical needs
- Accessing and preparing AI-ready imaging and multimodal data to enable interoperability
- Validating and implementing clinical imaging and multimodal AI
- Ethical considerations for the use of imaging based, multimodal AI clinical decision support tools
- These topics were explored in a series of informational presentations and discussion sessions featuring various expert speakers and panelists with a variety of expertise. The discussion sessions were guided by questions from the panelists and from participants.
This summary represents the opinions and perspectives of the workshop participants, which do not necessarily reflect the perspectives of NIH, the federal government, or the goals or structure of the potential PRIMED-AI program.
On March 11-12, 2025, the National Institutes of Health (NIH) Office of Strategic Coordination –The Common Fund convened the Precision Medicine with AI: Integrating Imaging with Multimodal Data (PRIMED-AI) workshop to identify key opportunities, complexities, and challenges in the emerging space of AI for precision medicine. Experts across NIH, academia, and industry gathered to share perspectives in four sessions, summarized below. The workshop aimed to chart a course for the integration of AI with imaging and other diverse health data types ("multimodal data") to advance precision medicine.
Session 1, Developing Algorithms that Leverage Multimodal Data to Solve Clinical Needs, emphasized the crucial relationship between data and metadata for interpreting AI model-identified outputs. Multimodal health data includes both imaging data (e.g., CT, MRI, and others) and non-imaging data (e.g., electronic health records, medical reports, and others), spanning multiple formats and domains. Co-chairs and panelists discussed the roles that AI models can play in detecting and understanding biological processes as well as the role of collecting and preparing data to ensure that models can perform these functions accurately. Developing robust AI models often requires curation and harmonization of noisy data as well as training of the algorithm with high quality data, both of which are informed by metadata. Co-chairs and panelists also discussed ways in which AI may prove clinically beneficial, such as identifying disease subtypes. Key discussions centered on the need for well-annotated, high-quality multimodal data to build AI models capable of addressing specific clinical needs and uncovering valuable biological insights.
Session 2, Accessing and Preparing AI-Ready Imaging and Multimodal Data to Enable Interoperability, focused on the collection, curation, and provenance of datasets for use in AI development. AI model development requires large quantities of deidentified, harmonized, and labeled data. Co-chairs and panelists highlighted concerns and potential best practices around data sharing and accessibility to meet these needs and to maintain sustainable health databases by evaluating dataset representativeness and quality. The session underscored the critical importance of establishing robust mechanisms for accessing, preparing, and sharing large, high-quality, AI-ready datasets while ensuring data privacy, interoperability, and quality.
Session 3, Validating and Implementing Clinical Imaging and Multimodal AI, examined the AI implementation landscape, including the processes already in place to validate and monitor AI models in clinical settings and processes that could be developed in the future to support AI model implementation. Co-chairs and panelists considered essential issues including who (e.g., AI developers, federal agencies, independent governance groups, others) should play a role in AI model monitoring and what metrics should be used to evaluate AI model implementation beyond accuracy. Generalizability, fit-for-purpose, and clinical benefit are important metrics to consider when determining the value of the implemented AI model. Discussions in this session revolved around the crucial steps for validating and implementing AI models in clinical practice.
Session 4, Ethical Considerations for the Use of Imaging-Based, Multimodal AI Clinical Decision Support Tools, highlighted patient privacy and concerns related to underlying assumptions for AI development and implementation as well as technical strategies and frameworks for accountability to mitigate these risks and ensure transparent, trustworthy AI development. The chair and panelists raised concerns about the relationships between patient consent and data collection for AI model development. This session addressed the significant ethical implications of using imaging-based, multimodal AI in clinical decision support.
In conclusion, workshop participants emphasized the paramount importance of developing AI models that aim to provide clinical benefit and further patient health. Developing clinically useful AI models will essentially require representative health data across a variety of health conditions, disease states, and populations coupled with supportive data sharing practices and policies.