An official website of the United States government
Event Details
Description of Event
The National Institutes of Health’s Office of Data Science Strategy hosted a virtual workshop on assessing dataset and data resource value and reach on Feb. 19, 2020.
The goal of this workshop was to discuss core metrics, use cases, and best practices to better understand data usage and impact. The workshop focused on two types of data resources – repositories and knowledgebases – and brought together managers of diverse biomedical data resources to discuss community-supported best practices for data metrics.
NIH-funded data resources (repositories and knowledgebases) use varied approaches to measure use and utility (community value/impact) of the resource itself, and/or the use and utility of specific datasets held by the resource. Data resource managers, funders, and users are interested in understanding the impact and value of data resources and the data they host. This workshop included speakers representing each of these stakeholders to discuss approaches to and values in evaluating research data and its infrastructure.
Some of the questions that participants considered included:
- What are the long-term positive or negative consequences of having evaluation metrics for research data?
- Are there existing standards or methodologies for assessing research data value and reach?
- How might different stakeholders (data resource users, managers, or funders) use data metrics?
NIH Data Science IdeaScale was used to gather input from the community. The agenda and presentations are below, along with recordings of each presentation and panel discussion. A
| Time | Presentation |
|---|---|
| 9:00 a.m. – 9:10 a.m. EST | |
| 9:10 a.m. – 9:40 a.m. EST | Dr. Stefanie Haustein,University of Ottawa, |
| Session 1: Evaluating and measuring data use and utility Session chair: Daniella Lowenberg Presentations (15 minutes + 5 minutes speaker transition time): | |
| 9:40 a.m. – 9:55 a.m. EST | Daniella Lowenberg,California Digital Library, |
| 10:00 a.m. – 10:15 a.m. EST | Dr. Valerie Schneider,National Institutes of Health, |
| 10:20 a.m. – 10:35 a.m. EST | Dr. Susan Redline,Harvard University, |
| 10:40 a.m. – 10:55 a.m. EST | Dr. Regina Bures,National Institutes of Health, |
| Panel Discussion: 11:00 a.m. – 12:00 p.m. - Questions from participants, plus framing questions Framing questions:What types of indicators have been or could be gathered to measure research data or data repository/knowledgebase value and reach?How does the maturity of a data repository/knowledgebase or the community it represents impact the evaluation of data metrics?How can repositories use existing initiatives and experts around standardized data usage, data citation, and data metrics to responsibly report on data value?View Panel Discussion | |
| 12:00 p.m. – 1:00 p.m. EST | Lunch Break |
| Session 2: Stakeholder use cases for data usage and utility metrics Session chair: Dr. Warren Kibbe Presentations (15 minutes + 5 minutes speaker transition time): | |
| 1:00 p.m. – 1:15 p.m. EST | Sean Coady,National Institutes of Health, |
| 1:20 p.m. – 1:35 p.m. EST | Dr. Robert Moritz,Institute for Systems Biology, |
| 1:40 p.m. – 1:55 p.m. EST | Dr. Brian Byrd,University of Michigan, |
| Panel Discussion: 2:00 p.m. – 3:00 p.m. - presentation Q&A, discussion guided by framing questions Framing questions:What use cases and questions should all data repositories/knowledgebases have in common around dataset or repository impact?What metrics to evaluate reach and impact do funders, repository managers, data generators and data users want access to?How could established data metrics help in incentivizing researchers to publish data, or influence broader compliance with policy implementations or promotion and tenure processes?View Panel Discussion | |
| 3:00 p.m. – 3:10 p.m. EST | |
| 3:10 p.m. – 3:30 p.m. EST | |
NIH Virtual Workshop on Data Metrics