Make Data Count

https://doi.org/10.60804/DBT4-CZ71  Make Data Count is a community hub for the development of metrics that can help us understand how data is used in research and policy activities. We reached out to Make Data Count community members for their views on data metrics and data evaluation. In this post, we share their perspectives on why understanding the use and reach of open data is important, and on their interest in Make Data Count.

Community perspectives on data metrics and data evaluation

The Make Data Count Summit 2024 took place in London earlier this month. Over two days, we explored what areas of research infrastructure, practices, evidence and evaluation we need to focus on to advance data metrics and our understanding of the reach and impact of open data.

Evaluation, context and getting practical on data metrics: the Make Data Count Summit 2024

DOI 10.60804/6a1z-rr79   At the recent Make Data Count Summit 2024, we had the opportunity to explore the insights that the Data Citation Corpus provides for institutions and funders. A large-scale open aggregate of data citations, the Data Citation Corpus currently stores 5 million citations collected via persistent identifier metadata and text mining of article content to identify mentions to datasets.

The Data Citation Corpus in practice: Perspectives from a funder and an institution

Kicking off the year, we had the pleasure of announcing the first release of the Data Citation Corpus. In response to the multiple expressions of interest in learning more about the project and in using the data file for the corpus, we held a webinar dedicated to the Data Citation Corpus.

‘I would like to better identify citations for data’: Community feedback and use cases for the first release of the Data Citation Corpus

First-of-its-kind aggregation brings together millions of data citations to advance understanding of data usage


 https://doi.org/10.60804/r14z-mw10

DataCite, in partnership with the Chan Zuckerberg Initiative (CZI), is delighted to announce the first release of the Data Citation Corpus.

DataCite launches first release of the Data Citation Corpus

This post has been cross-posted on the DataCite blog. A critical piece of open data infrastructure that has received insufficient attention is the evaluation of data usage. We still lack a clear understanding and a body of evidence on how data are being accessed, utilized, and incorporated into research activities….

A critical piece of open data infrastructure that has received insufficient attention is the evaluation of data usage. We still lack a clear understanding and a body of evidence on how data are being accessed, utilized, and incorporated into research activities.

Make Data Count Summit: The Time Is Now to Advance Data Metrics

Make Data Count Team

Make Data Count (MDC) is convening a two-day summit dedicated to the evaluation of open data usage, reach, and impact.

Announcing the Inaugural MDC Summit

Daniella Lowenberg

Join us for a 3-part series on key issues and opportunities for the community to prioritize on our journey to open data metrics.   Watch the recordings of FORAGE, EXPLORE, and BEGIN!

MDC Virtual Events: Spring, 2022

aninkov

Joint Declaration of Data Citation Principles

antonninkov/ISSI2021: Datasets on DataCite - an Initial Bibliometric Investigation

Datasets on DataCite - an Initial Bibliometric Investigation

By Kathleen Gregory and Anton Ninkov

 Data citations hold great promise for a variety of stakeholders. Unfortunately, due in part to a lack of metadata, i.e. about disciplinary domains, many of those promises remain out of reach. Metadata providers – repositories, publishers and researchers – play a key role in improving the current situation.

The potentials of data citations are many.

Data citations in context: We need disciplinary metadata to move forward

Daniella Lowenberg, Rachael Lammey, Matthew B. Jones, John Chodacki, Martin Fenner

DOI: 10.5281/zenodo.4701079

Data Citation: Let’s Choose Adoption Over Perfection

By: John Chodacki, Martin Fenner, Daniella Lowenberg Today, Zenodo announced their intentions to remove the altmetrics.com badges from their landing pages–and we couldn’t be more energized by their commitment to

 open

infrastructure, supporting their mission to make scientific information open and free.

 “We strongly believe that metadata about records including citation data &amp;

Open Metrics Require Open Infrastructure

Since 2014, the Make Data Count (MDC) initiative has focused on building the social and technical infrastructure for the development of research data metrics.

Igniting Change: Our Next Steps Towards Open Data Metrics

The Make Data Count team has been working on various infrastructure and outreach projects focused on how to measure the reach and impact of research data.

Open Data Metrics: Lighting the Fire

Following advice from our workshop attendees at RDA13, we invite you to join us for our spring webinar. Join us on

 May 8th at 8am PST/3pm GMT

as we demo our new aggregation services at DataCite and DataONE. This webinar is intended to spotlight the features and services we can build off of our central infrastructure such as aggregated usage and citations. This webinar will be recorded and posted on our website.

Save The Date: MDC Spring Webinar

Join us on

 March 26th at 8:00am PST/4:00pm GMT

for a webinar on repository implementation of our COUNTER Code of Practice for Research Data and Make Data Count recommendations. This webinar will feature a panel of developers from repositories that have implemented or about to release standardized data metrics: Dataverse, Dryad, and Zenodo. We will interview each repository on their implementation process.

Repository Implementation Webinar: March 26, 2019

When: April 1st, 10:00am-12:00pm Where: Loews Hotel, Philadelphia.

When:

April 1st, 10:00am-12:00pm

 Where:

Loews Hotel, Philadelphia.

Save the Date: Make Data Count Pre-RDA13 Workshop

Crossposted from DataONE blog: https://www.dataone.org/news/new-usage-metrics

Publications have long maintained a citation standard for research papers, ensuring credit for cited work and ideas. Tracking use of data collections, however, has remained a challenge. DataONE is pleased to share our latest effort in overcoming this barrier and in demonstrating data reuse with new data usage and citation metrics.

DataONE Implements New Usage and Citation Metrics to Make Your Data Count

With Make Data Count now in its second year, the focus is shifting from building infrastructure to driving adoption of our open data-level metrics infrastructure.

Make Data Count &amp; Scholix Join FORCE(2018)s

Crossposted from COUNTER on September 13, 2018

There is a need for the consistent and credible reporting of research data usage. Such usage metrics are required as an important component in understanding how publicly available research data are being reused.

COUNTER Code of Practice for Research Data Usage Metrics Release 1

It’s been two exciting months since we released the first iteration of our data-level-metrics infrastructure.  We are energized by the interest garnered and questions we’ve received and we wanted to share a couple of highlights!

 July Webinar

Soon after launch we hosted a webinar on “How-To” make your data count. Thank you to the 100 attendees that joined us for asking such thoughtful questions.

Make Data Count Summer 2018 Update

One year into our Sloan funded Make Data Count project, we are proud to release Version 1 of standardized data usage and citation metrics! As a community that values research data it is important for us to have a standard and fair way to compare metrics for data sharing. We know of and are involved in a variety of initiatives around data citation infrastructure and best practices; including Scholix, Crossref and DataCite Event Data.

It&#8217;s Time to Make Your Data Count!

Many publishers have implemented open data policies and have publicly declared  their support of data as a valuable component of the research process. But to give credit to researchers and incentivize behavior for data publishing, the community needs to promote proper citation of data.

Publishers: Make Your Data Citations Count!

The Make Data Count team is rapidly approaching the first release of standardized and comparable data level metrics (DLMs) on California Digital Library’s Dash and DataONE repositories. Resources on this release will be available shortly, but in the meantime the team would like to share updates on work completed in winter and our spring roadmap.

Make Data Count Update: Spring, 2018

Kicking off Love Data Week 2018, the Make Data Count (MDC) team is pleased to announce that the first iteration of our Code of Practice for Research Data Usage Metrics Release 1 has now been posted as a preprint. Beginning in June, members from the MDC team and COUNTER began conversations around what a standard for data usage metrics may look like. By September we were able to release an initial draft outline for community feedback.

Code of practice for research data usage metrics release 1

For the past few months, we have worked to garner interest and facilitate discussion about data usage metrics within the community. Internally, we are working to drive development toward comparable, standardized data usage metrics and data citations on repository interfaces. We are excited to share our progress and we want to thank those who have given us feedback or have gotten involved along the way!

Make Data Count Winter 2018 Update

The Make Data Count (MDC) project is moving ahead with full force and the team wanted to take a moment to update the research stakeholder community on our project resources and roadmap. In September, the MDC team sat down and mapped out the project plan for our two-year grant.

Make Data Count Update: November, 2017

Following our draft update and executive summary, Make Data Count and COUNTER are proud to release our first draft of a

 Code of Practice for Research Data

. This first iteration is meant to be a draft, and our goal is to receive input and feedback from the community. We ask that you please comment on and mark up the document with questions, suggestions, and/or overall feedback.

COUNTER Code of Practice for Research Data Draft 1

As a research and scholarly communications community, we value methods to gauge the impact of research outputs, and we do this in the forms of citations and downloads. But, until now this has been limited to traditional journal publications, and scholarly research is much more than an article. Foremost, data play a major role in the research process and deserve to be valued as a first-class scholarly output.

COUNTER Code of Practice for Research Data Draft Update

COUNTER is a non-profit organization supported by a global community of library, publisher and vendor members, who contribute to the development of the Code of Practice through working groups and outreach. COUNTER and MDC  are collaborating to develop and publish a Code of Practice for Research Data. This set of recommendations will focus on how data usage should be measured and reported.

COUNTER Code of Practice for Research Data

The Alfred P. Sloan Foundation has made a 2-year, $747K award to the California Digital Library, DataCite and DataONE to support collection of usage and citation metrics for data objects. Building on pilot work, this award will result in the launch of a new service that will collate and expose data level metrics.

Make Data Count:  Building a System to Support Recognition of Data as a First Class Research Output

Making Data Count (MDC) team members are at the center of many initiatives that focus on aspects of metrics, including DLM. We leverage existing channels to build a new data usage standard and to promote integration and adoption amongst data centers and data consumers.  If you want to get in contact and start collaborating with us, please: Join our mailing list Follow us on Twitter Contact us directly!

How to engage with MDC?

Journal articles are the currency of scholarly research. As a result, we as a community, use sophisticated methods to gauge the impact of research and measure the attention it receives by analyzing article citations, article page views and downloads, and social media metrics. While imprecise, these metrics offer us a way to identify relationships and better understand relative impact.

Understanding the problem

Blog post by Iratxe Puebla and Maria Alejandra Tejada Gómez.

DOI: 10.60804/zb5q-qy55 The Make Data Count Summit last September brought together researchers, infrastructure providers and representatives across institutions, funding and policy areas to discuss data metrics and the responsible evaluation of data usage.

‘Policies that make data count’: data evaluation in funder and policy-making processes

The Generalist Repository Ecosystem Initiative (GREI) has as one of its objectives the implementation of open metrics. A consistent approach to data citations is an important step to drive meaningful metrics that provide visibility on data usage, signal the added value of data repositories and enable reporting on the reach...

Why data citations?

Data citations are a useful measure to gain understanding on the use of research data. Data citations recognize the individual(s) or organization(s) that collected and shared the data, and researcher surveys regularly show that researchers value receiving citations to their dataset (see for example The State of Open Data report, or research by Kathleen Gregory and colleagues).

GREI recommendations to support consistent practices to collect, expose and aggregate citations to open data

The COUNTER Code of Practice for Research Data provided a key milestone in data evaluation practices by making it possible to report comparable usage counts across platforms. Over the last months, Make Data Count and COUNTER have collaborated to explore what a suitable direction would be for an update to the Code of Practice for Research Data, and we are now sharing our proposal to merge the Code of Practice for Research Data with COUNTER R5.1 for public consultation.

https://doi.org/10.60804/8hvp-e561


 This blog post is cross-posted on the COUNTER site.

The COUNTER Code of Practice for Research Data provided a key milestone in data evaluation practices by making it possible to report comparable usage counts across platforms.

Bringing the COUNTER Code of Practice for Research Data and COUNTER Release 5.1 together to enhance normalization

DOI:10.60804/AGQD-PF50  We are thrilled to share that Make Data Count has been selected by SCOSS as essential open infrastructure for participation in its next funding cycle.

SCOSS selects Make Data Count as infrastructure essential to open science

Redivis

DOI 10.60804/1516-e177

 Blog post by Ian Mathews, CEO at Redivis, Zach Chandler, Director of Open Scholarship Strategy at Stanford University, Erin DeLaney, Head of Design at Redivis


 
 Analyze data citations for any repository by forking
 
 
 this analytical workflow
 
 
 on Redivis!
 

At Redivis, we believe that data-driven research should be accessible, reproducible, and as open as

Analyzing the Data Citation Corpus on Redivis

https://doi.org/10.60804/fkfy-1162  Make Data Count is a community hub for the development of metrics that can help us understand how data is used in research and policy activities. Several Make Data Count community members have given us their insights on the Data Citation Corpus, our project to build a central open aggregate of data citations.

Community perspectives on the Data Citation Corpus

This week we are thrilled to welcome Clare Dean as Communications Consultant for Make Data Count. In this role, Clare will be supporting Make Data Count’s outreach and community engagement activities. Clare brings a wealth of experience working with organizations in scholarly communications and open science initiatives, but best to let her tell you about it. Here’s a brief introduction to Clare, in her own words. Welcome Clare!

Clare Dean

https://doi.org/10.60804/9aqp-4273

This week we are thrilled to welcome Clare Dean as Communications Consultant for Make Data Count. In this role, Clare will be supporting Make Data Count’s outreach and community engagement activities.

Welcoming Clare Dean as Communications Consultant for Make Data Count

Make Data Count is pleased to announce the Make Data Count Summit 2024, a two-day meeting dedicated to responsible data metrics and the evaluation of open data usage and impact.

https://doi.org/10.60804/zhtp-vs98

Make Data Count is pleased to announce the Make Data Count Summit 2024, a two-day meeting dedicated to responsible data metrics and the evaluation of open data usage and impact. The Make Data Count Summit 2024 will be held in London, UK on September 5-6 and will provide a forum for discussions about evidence-based data metrics and responsible assessment of open data.

Join us for the Make Data Count Summit 2024

*Blog post by Iratxe Puebla and [Maria Alejandra Tejada Gómez.
]{style="font-weight: 400;"}*[[DOI:
10.60804/zb5q-qy55](https://doi.org/10.60804/zb5q-qy55){target="_blank"
rel="noopener"}]{style="font-weight: 400;"}

[The ]{style="font-weight: 400;"}[[Make Data Count
Summit]{style="font-weight: 400;"}](https://makedatacount.org/read-our-blog/evaluation-context-and-getting-practical-on-data-metrics-the-make-data-count-summit-2024/)[
last September brought together researchers, infrastructure providers
and representatives across institutions, funding and policy areas to
discuss data metrics and the responsible evaluation of data usage.
Policy makers and funding agencies play a major role in driving forward
open science practices and the adoption of open data as well as data
evaluation, and so one of the sessions at the Summit focused on funder
and policy-making processes. ]{style="font-weight: 400;"}[Mercè Crosas
(Barcelona Supercomputing Center & CODATA) facilitated a discussion with
Dagmar Meyer (European Research Council), Anna Diaz Font (Medical
Research Foundation) and Maria Alejandra Tejada Gómez (Universidad
Javeriana)]{style="font-weight: 400;"}[ about data evaluation and the
role of data metrics]{style="font-weight: 400;"}[ to measure the impact
of open data and inform decision-making in national
contexts.]{style="font-weight: 400;"}

### [The need for data evaluation as part of open data policies]{style="font-weight: 400;"}

[A number of research funders support or require data sharing.
]{style="font-weight: 400;"}[Dagmar Meyer and Anna Diaz Font both noted
that the European Research Council and Medical Research Foundation
stipulate as part of their policies that data from the funded research
should be made available. Maria Alejandra Tejada Gómez shared that the
policies approved under the latest National Development Plan in Colombia
also expect access to the results and data from publicly-funded
research. The panelists noted that an evaluation of data availability
would fit within processes to check for policy compliance, but
acknowledged that such compliance checks are currently
rare.]{style="font-weight: 400;"}

[As for any other evaluation process, there is a need to consider what
is the purpose and scope of a data evaluation framework: is this at the
researcher level or at program or national level? Is it to check for
compliance or to inform funding or policy decisions? The metrics to
incorporate into the evaluation will depend on the type of evaluation to
undertake. When considering national policies, the framework will then
influence policies at institutional level, where requirements for data
management plans or data sharing are already implemented or in process.
The institutional policies need to align to the broader national
regulation for open data.]{style="font-weight: 400;"}

 

![](https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-600x355.jpg){.size-medium
.wp-image-940 .aligncenter fetchpriority="high" decoding="async"
width="600" height="355"
srcset="https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-600x355.jpg 600w, https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-1024x606.jpg 1024w, https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-768x455.jpg 768w, https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-1536x909.jpg 1536w, https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-2048x1212.jpg 2048w, https://makedatacount.org/wp-content/uploads/2024/12/Policies-that-make-data-count-session-1568x928.jpg 1568w"
sizes="(max-width: 600px) 100vw, 600px"}

 

### 

### [Current practices, challenges and opportunities]{style="font-weight: 400;"}

[While the policies in place at different funders indicate that
researchers may report any output as part of their grant applications or
reports, in practice, few funders currently complete evaluations of data
impact. This is also the case at institutional level, where policies
have so far focused on the aspects of data up to the stage of sharing,
with less attention beyond that point.]{style="font-weight: 400;"}

[The discussion explored the challenges around the implementation of
data evaluation in funder processes. The panelists raised a few elements
that could do with further definition. Conceptually evaluation of data
could be viewed in different ways: on the one hand, we have the aspect
of the quality of the data and its accompanying metadata -- e.g. use of
persistent identifiers (PIDs), alignment to FAIR and CARE principles,
compliance with privacy and security requirements -- and whether the
data shared is reproducible and reusable; on the other hand, there can
be an evaluation of the reuse and impact of the data, that is, assessing
whether others have reused the data and for what purposes. These two
axes of evaluation will require a different combination of metrics, with
the former possibly relying more on qualitative information and the
latter more on quantitative metrics such as data citations, downloads,
or the number of access requests. The metrics to implement may also
differ depending on whether the evaluation is related to an individual
researcher or rather a research program or a collection of
outputs.]{style="font-weight: 400;"}

[In addition to defining the metrics for each type of evaluation, there
were also questions raised about the information required to provide
adequate context so that panels would trust and incorporate it into
their reviews. And as a complementary point, it would be important to
have common ground on what the sources of data usage information would
be, what tools are available to access the information, and what are the
remaining gaps to address.]{style="font-weight: 400;"}

[An aspect that garnered substantial comments was that of the resourcing
needed to implement data evaluation in meaningful ways. The panel
recognized that in order to complete evaluations of data reach and
impact, there would need to be additional resources dedicated to curate
and preserve the data to make that evaluation possible, to sustain the
peer evaluation of data (with review panels incorporating the relevant
skills to complete this type of assessment), and to gather and interpret
quantitative measures collected through automated means or external
sources. Ensuring a comprehensive view on the usage of data also relies
on sustaining the infrastructure necessary to collect this information
-e.g. repositories, data storage, interoperable systems and
integrations-, and to make it openly
available.]{style="font-weight: 400;"}

[While this may seem challenging, there is a consensus that data is
critical to address global challenges. We saw the critical role of open
data in responding to the COVID-19 pandemic, and we should thus leverage
the potential of data to accelerate research progress to respond to
societal needs. To this end, there are a number of ways in which we can
take steps to make data evaluation a reality. Many institutions have
developed repositories and work with infrastructure providers that
supply interoperable systems. It will be important to sustain this
existing infrastructure while emphasizing interoperability to gain
economies of scale. There are also tools and services that provide open
information about data usage; while the usage measures may not be mature
for their use in all areas of evaluation, we can already put the
information available to use where this is meaningful. For example, the
number of data citations is scaling and in combination with other
information that funders collect, this can already provide insights into
the reach and impact of datasets at program
level.]{style="font-weight: 400;"}

### [Resourcing data evaluation processes is key]{style="font-weight: 400;"}

[The discussion highlighted the importance that funding and
policy-making agencies place on open data. The discussion touched on the
need to identify the right information for a responsible and equitable
evaluation of researchers and of research programs, and to resource
data-evaluation efforts. Importantly, all panelists highlighted that we
need a better understanding of how open data practices are evolving, and
of the impact and return-on-investment of open data policies. And so,
the time has come to match ]{style="font-weight: 400;"}[our recognition
that data evaluation is important with the investment in resources
necessary to make it happen.]{style="font-weight: 400;"}

 

*We thank Anna Diaz Font (Medical Research Foundation), Mercè Crosas
(Barcelona Supercomputing Center & CODATA), and Dagmar Meyer (European
Research Council) for their contributions during the Make Data Count
Summit and insightful discussions that fed into this post.*


[*https://doi.org/10.60804/zhtp-vs98*](https://doi.org/10.60804/zhtp-vs98){target="_blank"
rel="noopener"}

Make Data Count is pleased to announce the [Make Data Count Summit
2024](https://summit.makedatacount.org/){data-type="link"
data-id="https://summit.makedatacount.org/"}, a two-day meeting
dedicated to responsible data metrics and the evaluation of open data
usage and impact.

The Make Data Count Summit 2024 will be held in London, UK on September
5-6 and will provide a forum for discussions about evidence-based data
metrics and responsible assessment of open data. The Summit will bring
together representatives of the research and policy communities, data
administrators, funders, policymakers, publishers and infrastructure
providers.

Our [Make Data Count Summit in
Washington](https://makedatacount.org/make-data-count-summit-the-time-is-now-to-advance-data-metrics/)
last year highlighted that there are strong foundations for data
metrics, but also a need to raise awareness and pursue conversations
about data evaluation across sectors. By hosting this second Make Data
Count Summit next September, we want to provide a dedicated venue for
continued engagement with this topic. 

The event will incorporate facilitated discussions, panels and
brainstorming sessions. The Make Data Count Summit will foster a common
understanding of existing data metrics and their use, showcase
developments in infrastructure and practices supporting data metrics,
and engage groups across policy, research, and research-supporting
organizations in the next steps toward evidence-based metrics and the
meaningful assessment of research data. 

For more details about the event, as well as registration details, visit
[summit.makedatacount.org](https://summit.makedatacount.org/){data-type="link"
data-id="https://summit.makedatacount.org/"}.

::: {.wp-block-spacer style="height: 14px;" aria-hidden="true"}
 
:::

:::: {.wp-block-buttons .is-content-justification-center .is-layout-flex .wp-container-core-buttons-is-layout-1 .wp-block-buttons-is-layout-flex}
::: {.wp-block-button}
[Register for the Make Data count
Summit](https://events.wellcome.org/event/dd26426b-7709-46a9-baad-aa19dcec4ff3/summary){.wp-block-button__link
.has-background .wp-element-button style="background-color: #073763;"
target="_blank" rel="noopener"}
:::
::::

::: {.wp-block-spacer style="height: 40px;" aria-hidden="true"}
 
:::

*Note this is an in-person only event. If you are interested in keeping
up-to-date with the discussions but cannot attend in person, do sign up
for the [Make Data Count
newsletter](https://cdlib.us13.list-manage.com/subscribe?u=7d7129c053f7aa9fc40f662b2&id=2751dee8f9){data-type="link"
data-id="https://cdlib.us13.list-manage.com/subscribe?u=7d7129c053f7aa9fc40f662b2&id=2751dee8f9"
target="_blank" rel="noopener"}.*


### **First-of-its-kind aggregation brings together millions of data citations to advance understanding of data usage** {#first-of-its-kind-aggregation-brings-together-millions-of-data-citations-to-advance-understanding-of-data-usage .wp-block-heading style="text-align: left;"}

[*https://doi.org/10.60804/r14z-mw10*](https://doi.org/10.60804/r14z-mw10){target="_blank"
rel="noopener"}

DataCite, in partnership with the Chan Zuckerberg Initiative (CZI), is
delighted to announce the first release of the [Data Citation
Corpus](https://makedatacount.org/data-citation/){data-type="link"
data-id="https://makedatacount.org/data-citation/"}. A major milestone
in the Make Data Count initiative, the release makes eight million data
citations openly available and usable for the first time via an
interactive dashboard and public data file. We invite the community to
engage with the data and provide feedback on this collaborative effort.

As highlighted by Make Data Count, the lack of a centralized resource
for citations to datasets has hindered the evaluation of how open data
is being used. To address this gap, DataCite, with funding from the
[Wellcome Trust](https://wellcome.org/){data-type="link"
data-id="https://wellcome.org/" target="_blank" rel="noopener"}, has
developed an innovative aggregation that brings together for the first
time data citations from diverse sources into a comprehensive and
publicly accessible resource for the global community.

> "There is a pressing need to understand how open data is used, but we
> have lacked a resource to access this information in a centralized and
> open manner. The Data Citation Corpus will allow the community to gain
> access to critical insights on data usage." said Iratxe Puebla,
> Director of Make Data Count. "We are thrilled to share the progress
> from our collaboration with CZI to bring together citations from
> different sources, and look forward to working with others in the
> community to expand the breadth and coverage of the corpus."

The first release of the corpus includes data citations in DataCite and
Crossref metadata as well as asserted data citations contributed by CZI,
available to the community via a data citation store and dashboard
developed by Coko. Leveraging accession numbers from Europe PMC, CZI
applied a machine-learning model to a large set of full-text articles
and preprints to extract mentions to datasets. **This has enabled the
first-ever aggregation of citations for datasets with DOIs and accession
numbers into a single corpus,** enabling a more complete picture of data
usage**.**

> "As an organization that invests in research data and reference
> datasets, we believe it is critical to understand how data is shared
> and reused to enable new scientific discoveries," said Patricia
> Brennan, Vice President of Science Technology at the Chan Zuckerberg
> Initiative. "DataCite has been a leader in this space, providing
> critical infrastructure for data citation and for tracking its reuse.
> We're proud to support them in their vision to build a comprehensive
> global corpus of actionable data citations."

The interactive
[dashboard](http://corpus.datacite.org/dashboard){target="_blank"
rel="noopener"} of the corpus allows users to visualize and report on
citations by a variety of facets, such as funder, data repository, or
the journal where the article citing the data is published.

::: {.wp-block-spacer style="height: 13px;" aria-hidden="true"}
 
:::

::: {.wp-block-image}
<figure class="aligncenter size-large is-resized">
<img
src="https://makedatacount.org/wp-content/uploads/2024/09/Screenshot-2024-01-19-at-14.20.28-1-1024x303.png"
class="wp-image-1049" style="width: 892px; height: auto;"
decoding="async" />
</figure>
:::

::: {.wp-block-spacer style="height: 32px;" aria-hidden="true"}
 
:::

A complete data file of all of the citations is also available for
additional analysis and evaluation. Request the data file via this
[form](https://forms.gle/JeiGC6updejQemAW6){target="_blank"
rel="noopener"}.

Forthcoming releases will focus on addressing existing metadata gaps,
for example, related to the disciplinary information for the datasets,
and on incorporating feedback from early adopters. DataCite will also
pursue new collaborations with additional citation aggregators to expand
the breadth and scale of data citations in the corpus. 

Community input is an integral part of this project and DataCite invites
researchers, institutions, funders and infrastructure providers to
provide feedback on the first release of the corpus and future
development work. Please join us for an online webinar on February 22 to
learn more about the first release of the corpus and how to use it.
[Register
now](https://datacite.zoom.us/webinar/register/WN_fm2O1BqxRomjXzkgPWLhaQ){data-type="link"
data-id="https://datacite.zoom.us/webinar/register/WN_fm2O1BqxRomjXzkgPWLhaQ"
target="_blank" rel="noopener"} to participate in this interactive
session.

::: {.wp-block-spacer style="height: 27px;" aria-hidden="true"}
 
:::

:::: {.wp-block-buttons .is-content-justification-center .is-layout-flex .wp-container-core-buttons-is-layout-4 .wp-block-buttons-is-layout-flex}
::: {.wp-block-button}
[Register for the
webinar](https://datacite.zoom.us/webinar/register/WN_fm2O1BqxRomjXzkgPWLhaQ){.wp-block-button__link
.has-background .has-text-align-center .wp-element-button
style="background-color: #073763;" target="_blank" rel="noopener"}
:::
::::

::: {.wp-block-spacer style="height: 43px;" aria-hidden="true"}
 
:::

## **About DataCite** {#about-datacite .wp-block-heading}

[DataCite](https://datacite.org/){target="_blank" rel="noopener"} is a
global community that shares a common interest: to ensure that research
outputs and resources are openly available and connected so that their
reuse can advance knowledge across and between disciplines, now and in
the future. 

## **About Chan Zuckerberg Initiative** {#about-chan-zuckerberg-initiative .wp-block-heading}

The [Chan Zuckerberg
Initiative](https://chanzuckerberg.com/){target="_blank" rel="noopener"}
was founded in 2015 to help solve some of society's toughest challenges
--- from eradicating disease and improving education, to addressing the
needs of our communities. Through collaboration, providing resources and
building technology, our mission is to help build a more inclusive, just
and healthy future for everyone.

> *This post has been cross-posted on the [DataCite
> blog](https://doi.org/10.5438/fz2h-4092){data-type="link"
> data-id="https://doi.org/10.5438/fz2h-4092" target="_blank"
> rel="noopener"}.*

A critical piece of open data infrastructure that has received
insufficient attention is the evaluation of data usage. We still lack a
clear understanding and a body of evidence on how data are being
accessed, utilized, and incorporated into research activities.

While interest in this topic is increasing, there has so far not been a
dedicated event for discussions on the evaluation of data usage, and on
the development of the data metrics required to support such evaluation
across both research and government. To address this need, we hosted the
[Make Data Count Summit](https://summit.makedatacount.org/) on 12-13
September 2023, as a forum to bring diverse stakeholders together to
tackle nuanced issues about the importance of open data metrics. The
event brought together over 120 attendees in Washington DC, including
representatives from research institutions, funders and government,
researchers, publishers, and infrastructure providers with the goal to
hone in on actionable items for agencies and institutions to advance
data metrics and the evaluation of data usage.

### **Strong Foundations for Data Metrics** {#strong-foundations-for-data-metrics .wp-block-heading}

The first day of the event included presentations from ongoing efforts
toward data metrics. [Daniella
Lowenberg](https://orcid.org/0000-0003-2255-1869){target="_blank"
rel="noopener"} (University of California, Office of the President)
provided an overview of the work of Make Data Count since 2014. The
lessons learnt from the initiative's work on developing standards and
engaging the community have paved a renewed focus on supporting open
infrastructure and on building evidence on data usage practices to
continue to refine data metrics for diverse uses. Her takeaway: it's
time for a new focus in the open data world, and that focus is undivided
attention to the development of open data metrics.

[Matt Buys](https://orcid.org/0000-0001-7234-3684){target="_blank"
rel="noopener"} from DataCite and [Carly
Strasser](https://orcid.org/0000-0001-9592-2339){target="_blank"
rel="noopener"} from the [Chan Zuckerberg
Initiative](https://chanzuckerberg.com){target="_blank" rel="noopener"}
provided an update on the ongoing collaborative project to build an
[Open Global Data Citation
Corpus](https://makedatacount.org/data-citation/). This momentous
project seeks to aggregate citations to data from a variety of sources,
including citations from DataCite metadata as well as those from other
sources, such as data mentions extracted from full-text articles through
machine learning and identifiers from EMBL-EBI. The goal is that once
completed, the corpus will provide data usage information to the
community, openly, and at a scale not possible before.

::: {.wp-block-spacer style="height:12px" aria-hidden="true"}
:::

::: {.wp-block-image}
<figure class="aligncenter size-large is-resized">
<img
src="https://makedatacount.org/wp-content/uploads/2024/09/F52Az8gWEAE5qQs-2-1024x618.jpeg"
class="wp-image-1014" style="width:597px;height:360px" loading="lazy"
decoding="async" width="597" height="360" />
<figcaption>Matt Buys (DataCite) and Carly Strasser (Chan Zuckerberg
Initiative) present the collaborative project to build an Open Global
Data Citation Corpus.</figcaption>
</figure>
:::

::: {.wp-block-spacer style="height:11px" aria-hidden="true"}
:::

[Julia Lane](https://orcid.org/0000-0002-8139-2960){target="_blank"
rel="noopener"} from New York University presented her vision for data
as a public asset and her work on the ['Democratizing Data'
project](https://issues.org/democratizing-government-data-lane/){target="_blank"
rel="noopener"}, which has developed algorithms to identify mentions to
data as part of full-text articles. This project is mining the content
of articles in Scopus to surface mentions to data and visualize those
through dashboards for stakeholders to explore.

In the panel discussion 'Policy and Administrative Priorities for Data
Metrics in the US', panelists from different agencies discussed recent
developments in the United States seeking to open data, such as the
[Evidence-act](https://www.congress.gov/bill/115th-congress/house-bill/4174){target="_blank"
rel="noopener"} and last year's [OSTP
memo](https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-OSTP-Public-access-Memo.pdf){data-type="link"
data-id="https://www.whitehouse.gov/wp-content/uploads/2022/08/08-2022-OSTP-Public-access-Memo.pdf"
target="_blank" rel="noopener"}. These policies have provided important
impetus for not only opening up administrative and research data but
also for agencies to consider what their priorities should be for
understanding and evaluating the use of data that has been opened up.

### **Data Metrics Must Be Embedded Across the Ecosystem and Supported by Evidence** {#data-metrics-must-be-embedded-across-the-ecosystem-and-supported-by-evidence .wp-block-heading}

On the second day, Nancy Potok, CEO at NAPx Consulting and former Chief
Statistician of the United States provided an overview on the
foundations and lessons learnt from the five years since the US
Evidence-act, which promoted the release of data to make it more
accessible to the public, as well as the use of data to inform policy
development. 

The subsequent sessions explored the needs for data metrics across
different areas of the research process, including funding agencies,
institutions, and scholarly communications. Institutional processes for
tenure and promotion were highlighted as an area of particular
importance in order to drive awareness among researchers and adoption of
data evaluation and data metrics.

We also heard the latest evidence on data usage practices and data
citation from a panel of bibliometricians, who highlighted the
discoverability of datasets and metadata completeness as areas of
improvement. The panelists called for further research to inform
meaningful data metrics so that we avoid the pitfalls of defaulting to
oversimplified and opaque metrics.

::: {.wp-block-spacer style="height:18px" aria-hidden="true"}
:::

::: {.wp-block-image}
<figure class="aligncenter size-large is-resized">
<img
src="https://makedatacount.org/wp-content/uploads/2024/09/Make-Data-Count-Summit-1024x629.png"
class="wp-image-1008" style="width:553px;height:340px" loading="lazy"
decoding="async" width="553" height="340" />
<figcaption>Dr Stefanie Haustein (University of Ottawa and
ScholCommLab) introduces the session dedicated to bibliometrics studies
on data reuse</figcaption>
</figure>
:::

::: {.wp-block-spacer style="height:18px" aria-hidden="true"}
:::

### **Prioritizing Data Metrics Now** {#prioritizing-data-metrics-now .wp-block-heading}

During the Summit, we invited attendees to share their experiences and
suggestions in two breakout group discussions. While these discussions
highlighted that data usage evaluation is a complex subject that will
require many nuanced conversations across sectors, the message was also
clear that we must iterate in incremental steps, and not let perfect be
the enemy of good as we drive the conversations forward.

A few of the topics highlighted during the breakout conversations
include:

- Data metrics are nuanced, we will need to provide clear information
 and resources for a wide range of stakeholders so that they have the
 information they need to join the conversation and also to lead it
 within their communities
- There is a need to raise awareness and engagement on data metrics
 across all institutional levels -- from individual researchers and
 administrators to institutional leaders and program managers.
- Data metrics must be anchored on transparent information and built
 upon consistent practices, while also being mindful of domain-specific
 needs

We thank all the attendees for their engagement during the different
sessions. It is clear that there is a shared interest in driving the
evaluation of data usage and an understanding that we should
collectively work towards meaningful evidence-based metrics. The Make
Data Count initiative will be taking forward these conversations and we
invite everyone interested to collaborate with us, as we all move
forward to advance meaningful data metrics.

<figure class="wp-block-table">
<table class="has-background" style="background-color:#d1e5f4">
<tbody>
<tr class="odd">
<td class="has-text-align-center" data-align="center">You can
access the slides for the talks presented at the Summit on <a
href="https://doi.org/10.5281/zenodo.8370593" target="_blank"
rel="noopener">Zenodo</a>.</td>
</tr>
</tbody>
</table>
</figure>