Make Data Count

https://doi.org/10.60804/DBT4-CZ71  Make Data Count is a community hub for the development of metrics that can help us understand how data is used in research and policy activities. We reached out to Make Data Count community members for their views on data metrics and data evaluation. In this post, we share their perspectives on why understanding the use and reach of open data is important, and on their interest in Make Data Count.

Community perspectives on data metrics and data evaluation

The Make Data Count Summit 2024 took place in London earlier this month. Over two days, we explored what areas of research infrastructure, practices, evidence and evaluation we need to focus on to advance data metrics and our understanding of the reach and impact of open data.

Evaluation, context and getting practical on data metrics: the Make Data Count Summit 2024

DOI 10.60804/6a1z-rr79   At the recent Make Data Count Summit 2024, we had the opportunity to explore the insights that the Data Citation Corpus provides for institutions and funders. A large-scale open aggregate of data citations, the Data Citation Corpus currently stores 5 million citations collected via persistent identifier metadata and text mining of article content to identify mentions to datasets.

The Data Citation Corpus in practice: Perspectives from a funder and an institution

Kicking off the year, we had the pleasure of announcing the first release of the Data Citation Corpus. In response to the multiple expressions of interest in learning more about the project and in using the data file for the corpus, we held a webinar dedicated to the Data Citation Corpus.

‘I would like to better identify citations for data’: Community feedback and use cases for the first release of the Data Citation Corpus

First-of-its-kind aggregation brings together millions of data citations to advance understanding of data usage


 https://doi.org/10.60804/r14z-mw10

DataCite, in partnership with the Chan Zuckerberg Initiative (CZI), is delighted to announce the first release of the Data Citation Corpus.

DataCite launches first release of the Data Citation Corpus

This post has been cross-posted on the DataCite blog. A critical piece of open data infrastructure that has received insufficient attention is the evaluation of data usage. We still lack a clear understanding and a body of evidence on how data are being accessed, utilized, and incorporated into research activities….

A critical piece of open data infrastructure that has received insufficient attention is the evaluation of data usage. We still lack a clear understanding and a body of evidence on how data are being accessed, utilized, and incorporated into research activities.

Make Data Count Summit: The Time Is Now to Advance Data Metrics

Make Data Count Team

Make Data Count (MDC) is convening a two-day summit dedicated to the evaluation of open data usage, reach, and impact.

Announcing the Inaugural MDC Summit

Daniella Lowenberg

Join us for a 3-part series on key issues and opportunities for the community to prioritize on our journey to open data metrics.   Watch the recordings of FORAGE, EXPLORE, and BEGIN!

MDC Virtual Events: Spring, 2022

aninkov

Joint Declaration of Data Citation Principles

antonninkov/ISSI2021: Datasets on DataCite - an Initial Bibliometric Investigation

Datasets on DataCite - an Initial Bibliometric Investigation

By Kathleen Gregory and Anton Ninkov

 Data citations hold great promise for a variety of stakeholders. Unfortunately, due in part to a lack of metadata, i.e. about disciplinary domains, many of those promises remain out of reach. Metadata providers – repositories, publishers and researchers – play a key role in improving the current situation.

The potentials of data citations are many.

Data citations in context: We need disciplinary metadata to move forward

Daniella Lowenberg, Rachael Lammey, Matthew B. Jones, John Chodacki, Martin Fenner

DOI: 10.5281/zenodo.4701079

Data Citation: Let’s Choose Adoption Over Perfection

By: John Chodacki, Martin Fenner, Daniella Lowenberg Today, Zenodo announced their intentions to remove the altmetrics.com badges from their landing pages–and we couldn’t be more energized by their commitment to

 open

infrastructure, supporting their mission to make scientific information open and free.

 “We strongly believe that metadata about records including citation data &amp;

Open Metrics Require Open Infrastructure

Since 2014, the Make Data Count (MDC) initiative has focused on building the social and technical infrastructure for the development of research data metrics.

Igniting Change: Our Next Steps Towards Open Data Metrics

The Make Data Count team has been working on various infrastructure and outreach projects focused on how to measure the reach and impact of research data.

Open Data Metrics: Lighting the Fire

Following advice from our workshop attendees at RDA13, we invite you to join us for our spring webinar. Join us on

 May 8th at 8am PST/3pm GMT

as we demo our new aggregation services at DataCite and DataONE. This webinar is intended to spotlight the features and services we can build off of our central infrastructure such as aggregated usage and citations. This webinar will be recorded and posted on our website.

Save The Date: MDC Spring Webinar

Join us on

 March 26th at 8:00am PST/4:00pm GMT

for a webinar on repository implementation of our COUNTER Code of Practice for Research Data and Make Data Count recommendations. This webinar will feature a panel of developers from repositories that have implemented or about to release standardized data metrics: Dataverse, Dryad, and Zenodo. We will interview each repository on their implementation process.

Repository Implementation Webinar: March 26, 2019

When: April 1st, 10:00am-12:00pm Where: Loews Hotel, Philadelphia.

When:

April 1st, 10:00am-12:00pm

 Where:

Loews Hotel, Philadelphia.

Save the Date: Make Data Count Pre-RDA13 Workshop

Crossposted from DataONE blog: https://www.dataone.org/news/new-usage-metrics

Publications have long maintained a citation standard for research papers, ensuring credit for cited work and ideas. Tracking use of data collections, however, has remained a challenge. DataONE is pleased to share our latest effort in overcoming this barrier and in demonstrating data reuse with new data usage and citation metrics.

DataONE Implements New Usage and Citation Metrics to Make Your Data Count

With Make Data Count now in its second year, the focus is shifting from building infrastructure to driving adoption of our open data-level metrics infrastructure.

Make Data Count &amp; Scholix Join FORCE(2018)s

Crossposted from COUNTER on September 13, 2018

There is a need for the consistent and credible reporting of research data usage. Such usage metrics are required as an important component in understanding how publicly available research data are being reused.

COUNTER Code of Practice for Research Data Usage Metrics Release 1

It’s been two exciting months since we released the first iteration of our data-level-metrics infrastructure.  We are energized by the interest garnered and questions we’ve received and we wanted to share a couple of highlights!

 July Webinar

Soon after launch we hosted a webinar on “How-To” make your data count. Thank you to the 100 attendees that joined us for asking such thoughtful questions.

Make Data Count Summer 2018 Update

One year into our Sloan funded Make Data Count project, we are proud to release Version 1 of standardized data usage and citation metrics! As a community that values research data it is important for us to have a standard and fair way to compare metrics for data sharing. We know of and are involved in a variety of initiatives around data citation infrastructure and best practices; including Scholix, Crossref and DataCite Event Data.

It&#8217;s Time to Make Your Data Count!

Many publishers have implemented open data policies and have publicly declared  their support of data as a valuable component of the research process. But to give credit to researchers and incentivize behavior for data publishing, the community needs to promote proper citation of data.

Publishers: Make Your Data Citations Count!

The Make Data Count team is rapidly approaching the first release of standardized and comparable data level metrics (DLMs) on California Digital Library’s Dash and DataONE repositories. Resources on this release will be available shortly, but in the meantime the team would like to share updates on work completed in winter and our spring roadmap.

Make Data Count Update: Spring, 2018

Kicking off Love Data Week 2018, the Make Data Count (MDC) team is pleased to announce that the first iteration of our Code of Practice for Research Data Usage Metrics Release 1 has now been posted as a preprint. Beginning in June, members from the MDC team and COUNTER began conversations around what a standard for data usage metrics may look like. By September we were able to release an initial draft outline for community feedback.

Code of practice for research data usage metrics release 1

For the past few months, we have worked to garner interest and facilitate discussion about data usage metrics within the community. Internally, we are working to drive development toward comparable, standardized data usage metrics and data citations on repository interfaces. We are excited to share our progress and we want to thank those who have given us feedback or have gotten involved along the way!

Make Data Count Winter 2018 Update

The Make Data Count (MDC) project is moving ahead with full force and the team wanted to take a moment to update the research stakeholder community on our project resources and roadmap. In September, the MDC team sat down and mapped out the project plan for our two-year grant.

Make Data Count Update: November, 2017

Following our draft update and executive summary, Make Data Count and COUNTER are proud to release our first draft of a

 Code of Practice for Research Data

. This first iteration is meant to be a draft, and our goal is to receive input and feedback from the community. We ask that you please comment on and mark up the document with questions, suggestions, and/or overall feedback.

COUNTER Code of Practice for Research Data Draft 1

As a research and scholarly communications community, we value methods to gauge the impact of research outputs, and we do this in the forms of citations and downloads. But, until now this has been limited to traditional journal publications, and scholarly research is much more than an article. Foremost, data play a major role in the research process and deserve to be valued as a first-class scholarly output.

COUNTER Code of Practice for Research Data Draft Update

COUNTER is a non-profit organization supported by a global community of library, publisher and vendor members, who contribute to the development of the Code of Practice through working groups and outreach. COUNTER and MDC  are collaborating to develop and publish a Code of Practice for Research Data. This set of recommendations will focus on how data usage should be measured and reported.

COUNTER Code of Practice for Research Data

The Alfred P. Sloan Foundation has made a 2-year, $747K award to the California Digital Library, DataCite and DataONE to support collection of usage and citation metrics for data objects. Building on pilot work, this award will result in the launch of a new service that will collate and expose data level metrics.

Make Data Count:  Building a System to Support Recognition of Data as a First Class Research Output

Making Data Count (MDC) team members are at the center of many initiatives that focus on aspects of metrics, including DLM. We leverage existing channels to build a new data usage standard and to promote integration and adoption amongst data centers and data consumers.  If you want to get in contact and start collaborating with us, please: Join our mailing list Follow us on Twitter Contact us directly!

How to engage with MDC?

Journal articles are the currency of scholarly research. As a result, we as a community, use sophisticated methods to gauge the impact of research and measure the attention it receives by analyzing article citations, article page views and downloads, and social media metrics. While imprecise, these metrics offer us a way to identify relationships and better understand relative impact.

Understanding the problem

Blog post by Iratxe Puebla and Maria Alejandra Tejada Gómez.

DOI: 10.60804/zb5q-qy55 The Make Data Count Summit last September brought together researchers, infrastructure providers and representatives across institutions, funding and policy areas to discuss data metrics and the responsible evaluation of data usage.

‘Policies that make data count’: data evaluation in funder and policy-making processes

The Generalist Repository Ecosystem Initiative (GREI) has as one of its objectives the implementation of open metrics. A consistent approach to data citations is an important step to drive meaningful metrics that provide visibility on data usage, signal the added value of data repositories and enable reporting on the reach...

Why data citations?

Data citations are a useful measure to gain understanding on the use of research data. Data citations recognize the individual(s) or organization(s) that collected and shared the data, and researcher surveys regularly show that researchers value receiving citations to their dataset (see for example The State of Open Data report, or research by Kathleen Gregory and colleagues).

GREI recommendations to support consistent practices to collect, expose and aggregate citations to open data

The COUNTER Code of Practice for Research Data provided a key milestone in data evaluation practices by making it possible to report comparable usage counts across platforms. Over the last months, Make Data Count and COUNTER have collaborated to explore what a suitable direction would be for an update to the Code of Practice for Research Data, and we are now sharing our proposal to merge the Code of Practice for Research Data with COUNTER R5.1 for public consultation.

https://doi.org/10.60804/8hvp-e561


 This blog post is cross-posted on the COUNTER site.

The COUNTER Code of Practice for Research Data provided a key milestone in data evaluation practices by making it possible to report comparable usage counts across platforms.

Bringing the COUNTER Code of Practice for Research Data and COUNTER Release 5.1 together to enhance normalization

DOI:10.60804/AGQD-PF50  We are thrilled to share that Make Data Count has been selected by SCOSS as essential open infrastructure for participation in its next funding cycle.

SCOSS selects Make Data Count as infrastructure essential to open science

Redivis

DOI 10.60804/1516-e177

 Blog post by Ian Mathews, CEO at Redivis, Zach Chandler, Director of Open Scholarship Strategy at Stanford University, Erin DeLaney, Head of Design at Redivis


 
 Analyze data citations for any repository by forking
 
 
 this analytical workflow
 
 
 on Redivis!
 

At Redivis, we believe that data-driven research should be accessible, reproducible, and as open as

Analyzing the Data Citation Corpus on Redivis

https://doi.org/10.60804/fkfy-1162  Make Data Count is a community hub for the development of metrics that can help us understand how data is used in research and policy activities. Several Make Data Count community members have given us their insights on the Data Citation Corpus, our project to build a central open aggregate of data citations.

Community perspectives on the Data Citation Corpus

This week we are thrilled to welcome Clare Dean as Communications Consultant for Make Data Count. In this role, Clare will be supporting Make Data Count’s outreach and community engagement activities. Clare brings a wealth of experience working with organizations in scholarly communications and open science initiatives, but best to let her tell you about it. Here’s a brief introduction to Clare, in her own words. Welcome Clare!

Clare Dean

https://doi.org/10.60804/9aqp-4273

This week we are thrilled to welcome Clare Dean as Communications Consultant for Make Data Count. In this role, Clare will be supporting Make Data Count’s outreach and community engagement activities.

Welcoming Clare Dean as Communications Consultant for Make Data Count

Make Data Count is pleased to announce the Make Data Count Summit 2024, a two-day meeting dedicated to responsible data metrics and the evaluation of open data usage and impact.

https://doi.org/10.60804/zhtp-vs98

Make Data Count is pleased to announce the Make Data Count Summit 2024, a two-day meeting dedicated to responsible data metrics and the evaluation of open data usage and impact. The Make Data Count Summit 2024 will be held in London, UK on September 5-6 and will provide a forum for discussions about evidence-based data metrics and responsible assessment of open data.

Join us for the Make Data Count Summit 2024

[DOI
10.60804/1516-e177](https://doi.org/10.60804/1516-e177){target="_blank"
rel="noopener"}

**Blog post by [Ian
Mathews](https://orcid.org/0000-0002-3436-6639){.c-link target="_blank"
rel="noopener noreferrer"
stringify-link="https://orcid.org/0000-0002-3436-6639"
sk="tooltip_parent"}, CEO at Redivis, [Zach
Chandler](https://orcid.org/0000-0003-2402-9839){.c-link target="_blank"
rel="noopener noreferrer"
stringify-link="https://orcid.org/0000-0003-2402-9839"
sk="tooltip_parent"}, Director of Open Scholarship Strategy at Stanford
University, [Erin
DeLaney](https://orcid.org/0000-0003-4127-5601){.c-link target="_blank"
rel="noopener noreferrer"
stringify-link="https://orcid.org/0000-0003-4127-5601"
sk="tooltip_parent"}, Head of Design at Redivis**

#### *[Analyze data citations for any repository by forking ]{style="font-weight: 400;"}*[*[this analytical workflow]{style="font-weight: 400;"}*](https://redivis.com/projects/hzan-6y0nc4vt8){target="_blank" rel="noopener"}*[ on Redivis!]{style="font-weight: 400;"}*

[At
]{style="font-weight: 400;"}[[Redivis]{style="font-weight: 400;"}](https://redivis.com){target="_blank"
rel="noopener"}[, we believe that data-driven research should be
accessible, reproducible, and as open as possible. We also believe that
there is a significant underinvestment in dataset curation across the
academic enterprise. We argue that this is due to the reality that
dataset curators too often go unacknowledged for their substantial
efforts and contributions to our collective research endeavors. Redivis
is a platform that provides researchers and universities with the tools
to host, curate, and analyze data, and we are deeply committed to
ensuring our community receives credit for their
work.]{style="font-weight: 400;"}

[We've been incredibly excited by the ongoing work at Make Data Count,
and its development of the Data Citation Corpus as a central aggregate
of citations to research datasets. We were enthusiastic to see the
]{style="font-weight: 400;"}[[public release of the Data Citation
Corpus]{style="font-weight: 400;"}](https://zenodo.org/records/13376773){target="_blank"
rel="noopener"}[ in the spring, and the latest v2.0 release in August.
This dataset is a major step forward in ensuring attribution and credit
for dataset curators. So of course, the first thing we did was to load
the Data Citation Corpus into Redivis so that we could explore it
further!]{style="font-weight: 400;"}

**Exploring the Data Citation Corpus**

[The Data Citation Corpus is distributed as 6 CSV or JSON files
totalling approximately 1.5GB. Not huge by big data standards, but
sufficiently large that we want to make sure to use the right tooling to
explore the data. In our case, we loaded the CSVs into a
]{style="font-weight: 400;"}[[single
table]{style="font-weight: 400;"}](https://redivis.com/datasets/am5t-e9jvcn6s5/tables/h08j-egn4kbpr3){target="_blank"
rel="noopener"}[ of over 5 million records on Redivis, which
automatically combined the files, inferred data types, and generated
various summary statistics. ]{style="font-weight: 400;"}

[We endeavor to make Redivis as open as possible -- anyone with a
browser can search for and explore public datasets, read documentation,
and browse the data and summary statistics. Some actions, like running
custom queries and uploading data, require a free account, available to
anyone around the world. Explore the Data Citation Corpus table for
yourself below!]{style="font-weight: 400;"}

 

 

[We can see that there are 1,692 distinct repositories represented in
these 5,256,114 citations, with the European Nucleotide Archive
representing almost half of the citations. We also observe that most
citations have occurred over the past several decades, though there are
older citations in our database.]{style="font-weight: 400;"}

[Once we've logged in, we can run custom queries to ask more nuanced
questions of the Data Citation Corpus. We previously saw that the
European Nucleotide Archive was the most cited repository -- let's
explore where those citations are coming from. By running a quick query
to filter our records and looking at the summary statistics for the
"journal" variable, we can see that 9,998 distinct journals cited the
European Nucleotide Archive's datasets, with
]{style="font-weight: 400;"}*[PLOS ONE ]{style="font-weight: 400;"}*[at
the top with over 100,000 citations.]{style="font-weight: 400;"}

![Figure
1](https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23%E2%80%AFAM-600x368.png){.alignnone
.wp-image-882 loading="lazy" decoding="async" width="975" height="598"
srcset="https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM-600x368.png 600w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM-1024x628.png 1024w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM-768x471.png 768w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM-1536x942.png 1536w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM-1568x962.png 1568w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM-620x380.png 620w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.09.23 AM.png 1754w"
sizes="auto, (max-width: 975px) 100vw, 975px"}

** 
The Institutional Use Case: Counting data citations at Stanford**

[So far, we've only scratched the surface of the analytical capabilities
of Redivis. To dive in further, let's use a real-world use case:
]{style="font-weight: 400;"}[[Zach
Chandler]{style="font-weight: 400;"}](https://profiles.stanford.edu/zach-chandler){target="_blank"
rel="noopener"}[ is the Director of Open Scholarship Strategy at
Stanford, and wants to better understand how datasets from research
groups at his institution are being cited in the literature. By using
the Data Citation Corpus on Redivis, Zach can develop a more
comprehensive picture of the research impact of Stanford's
datasets.]{style="font-weight: 400;"}

[To do so, Zach creates a ]{style="font-weight: 400;"}[[new analytical
workflow]{style="font-weight: 400;"}](https://redivis.com/projects/hzan-6y0nc4vt8){target="_blank"
rel="noopener"}[, where he can reference and combine any dataset on
Redivis as he builds out his analysis. These workflows are a performant,
in-browser interface where researchers can transform data with SQL (or a
SQL-generating GUI), and perform further analysis via Jupyter notebooks
running Python, R, Stata, or SAS. Workflows are also fully reproducible
and shareable, allowing researchers to contribute and build off of each
others' work.]{style="font-weight: 400;"}

 

![Figure
2](https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33%E2%80%AFAM-600x367.png){.wp-image-885
.aligncenter loading="lazy" decoding="async" width="881" height="539"
srcset="https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM-600x367.png 600w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM-1024x626.png 1024w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM-768x469.png 768w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM-1536x938.png 1536w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM-1568x958.png 1568w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM-620x380.png 620w, https://makedatacount.org/wp-content/uploads/2024/11/Screenshot-2024-11-14-at-5.12.33 AM.png 1748w"
sizes="auto, (max-width: 881px) 100vw, 881px"}

 

[Here, Zach starts with the Data Citation Corpus, and uses the point and
click "transform" interface to filter down to citations of datasets in
the
]{style="font-weight: 400;"}[[OpenNeuro]{style="font-weight: 400;"}](https://openneuro.org/){target="_blank"
rel="noopener"}[ repository -- a free and open platform for sharing
neuroimaging data, born out of the Stanford Center for Reproducible
Neuroscience. He then joins these citations with metadata from the
]{style="font-weight: 400;"}[[DataCite public data
file]{style="font-weight: 400;"}](https://redivis.com/datasets/7wec-6vgw8qaaq/tables){target="_blank"
rel="noopener"}[, a dataset that contains information about every
dataset that has been issued a DOI through DataCite. This allows us to
bring in metadata about the authors of the cited datasets, so that we
can appropriately give them credit.]{style="font-weight: 400;"}

[After a bit more data wrangling, Zach has created two tables: 1) a list
of OpenNeuro citations, linked to the citing publication; and 2) a list
of Open Neuro citations ]{style="font-weight: 400;"}*[by
author]{style="font-weight: 400;"}*[, which we can then use to link
dataset citations to the individuals whose data curations made the final
publication possible.]{style="font-weight: 400;"}

[This all comes together in a PYTHON notebook where we can calculate
some final counts and visualize our results:]{style="font-weight: 400;"}

 

 

**Complete your own analysis of the Data Citation Corpus**

[The
]{style="font-weight: 400;"}[[workflow]{style="font-weight: 400;"}](https://redivis.com/projects/hzan-6y0nc4vt8){target="_blank"
rel="noopener"}[ that we've developed for this use case is public for
anyone to view or copy for further exploration. The OpenNeuro DOI prefix
is parameterized within the workflow -- you can simply swap it out for
any other DOI prefix and re-run the analysis to search for the citations
of interest to you! ]{style="font-weight: 400;"}

[Of course, counting citations is just one of many things we can do with
the Data Citation Corpus -- particularly when we link it to other
datasets, such as the DataCite public data file. We've wondered how the
completeness of DOI metadata might influence a dataset's propensity to
be cited, or what sort of variability there is between scientific
disciplines in citing datasets. You could even use GPU-backed notebooks
on Redivis to train machine learning models that assess metadata
quality. ]{style="font-weight: 400;"}

[We invite you to explore the ]{style="font-weight: 400;"}[[Data
Citation
Corpus]{style="font-weight: 400;"}](https://redivis.com/datasets/am5t-e9jvcn6s5/tables){target="_blank"
rel="noopener"}[ on Redivis yourself to better understand this dataset
and the powerful information it contains. ]{style="font-weight: 400;"}

[As a final note, we ask for everyone to remember how important
persistent identifiers (PIDs) are for the research ecosystem. If
]{style="font-weight: 400;"}[[ROR
identifiers]{style="font-weight: 400;"}](https://ror.org/){target="_blank"
rel="noopener"}[ were more widely adopted, this kind of analysis in our
example would be much easier to do in the future. Ditto for
]{style="font-weight: 400;"}[[ORCID]{style="font-weight: 400;"}](https://orcid.org/){target="_blank"
rel="noopener"}[. As a community, let's all work to use persistent
identifiers, so that everyone can get credited for their important
contributions that drive our collective knowledge
forward.]{style="font-weight: 400;"}

<figure class="wp-block-table mt-4">
<table class="has-fixed-layout">
<tbody>
<tr class="odd">
<td>Explore the Data Citation Corpus data file: <a
href="https://zenodo.org/doi/10.5281/zenodo.11196858" target="_blank"
rel="noopener">https://zenodo.org/doi/10.5281/zenodo.11196858</a></td>
</tr>
</tbody>
</table>
</figure>

Kicking off the year, we had the pleasure of announcing the [first
release of the Data Citation
Corpus](https://makedatacount.org/first-release-of-the-open-global-data-citation-corpus/).
In response to the multiple expressions of interest in learning more
about the project and in using the data file for the corpus, we held a
webinar dedicated to the Data Citation Corpus.

With over 300 attendees, and plenty of follow on, the discussions we are
having with community members are helping us inform the next steps for
the project and prioritize areas that the community has signaled as
requiring further work. Drawing on these conversations, we wanted to
highlight some of the themes that have arisen and address questions
posed during the webinar. We also share the link to the webinar
recording below for those who missed it or want to review details.

### **Scope of the first release** {#scope-of-the-first-release .wp-block-heading}

The data file for the first release of the Data Citation Corpus includes
1.3 million data citations from DataCite Event Data, and eight million
data mentions identified by the Chan Zuckerberg Initiative (CZI) by
mining the full text of articles. Anyone interested in the data file can
contact us via [this
form](https://docs.google.com/forms/d/e/1FAIpQLSd1l7ovTQs3EMw9mz4HFaVB2SuUQ8Z8FldoCDgvD74GV-vh0Q/viewform){target="_blank"
rel="noopener"}; we are happy to share the data file, and are just using
this format to be able to follow up with users with further information
or regarding possible collaborations. There is also a public dashboard
that provides high-level visualization of the contents of the data file,
accessible at
[http://corpus.datacite.org/dashboard](http://corpus.datacite.org/dashboard){target="_blank"
rel="noopener"}.

The data file covers data-paper links identified through 1) [DataCite
Event
Data](https://support.datacite.org/docs/eventdata-guide){target="_blank"
rel="noopener"}, based on the [metadata relationships that designate
data
citations](https://support.datacite.org/docs/contributing-citations-and-references){target="_blank"
rel="noopener"} and 2) CZI's machine learning approach, where a mention
of a dataset identifier found in the text of the article is designated
as a citation.

By design, the Data Citation Corpus is focused on citations to data. We
recognize the importance of understanding usage and citations to all
kinds of open outputs (including samples, software, protocols and other)
but the scope of the corpus lies with datasets.

The relationship between the dataset and the article relies on the
identifier for the dataset. We recognize that open datasets can vary in
how they are created and shared, some include a single file and some
multiple components. The community has raised questions about how to
best handle citations to individual datasets and collections of data,
and how to best propagate citations from collections to their individual
components. While the corpus does not address this use case at this
time, it is something we will continue to explore as part of the corpus
development.

### **Coverage & quality** {#coverage-quality .wp-block-heading}

The text mining completed by CZI involved a list of over 40 repositories
(see slide 8 [here](https://zenodo.org/records/10691948){target="_blank"
rel="noopener"} for the repository list). The repositories were selected
due to their standing in the community and because curated terms exist
for their accession numbers, but it is important to bear in mind that
the group focuses on life sciences disciplines. CZI completed text
mining on five million open access papers available via Europe PMC (as
the open licenses for those articles permit content mining), and thus,
the literature included also falls mostly within the life sciences. We
thus acknowledge that the text mining completed includes a slice of the
literature. One of the challenges in data citation that had been
expressed over the years was the separate handling of DOIs and accession
numbers; the collaboration between CZI and DataCite provides a
proof-of-principle to address this challenge and, for the first time,
bring together citations to data with accession numbers and DOIs. As we
move ahead, it is in our pipeline for the Data Citation Corpus to expand
coverage of the disciplines represented.

During the webinar, Ana-Maria Istrate (CZI) also touched on the fact
that the machine-learning model employed to search the article text
would have picked some false positives, for example, if a string
matching an accession number was also used to designate a grant number
or another research entity. This is another aspect of the data citations
that we will continue to work on, and we are happy to collaborate with
groups interested in looking at improvements to the model, or in
completing further curation of the citations included.

::: {.wp-block-spacer style="height: 28px;" aria-hidden="true"}
 
:::

::: {.wp-block-image}
<figure class="aligncenter size-large is-resized">
<img
src="https://makedatacount.org/wp-content/uploads/2024/09/Screenshot-2024-02-28-at-08.06.03-1024x630.png"
class="wp-image-1158" style="width: 695px; height: auto;"
decoding="async" />
</figure>
:::

::: {.wp-block-spacer style="height: 44px;" aria-hidden="true"}
 
:::

We also discussed metadata elements for the data citations, and some
known metadata gaps. While we took steps to identify metadata for
affiliation, funder and subject information where available, those
metadata are not yet available for many citations, and subject area in
particular is only recorded for under 1% of the data-paper links. This
is not a problem specific to the Data Citation Corpus, and rather
relates to the broader challenge around metadata completeness for both
datasets and articles, but we recognize it as a priority area for
additional work. We would like to explore approaches to infer discipline
information for the datasets, or leverage AI or other approaches to
enrich the discipline-level categorization for the citations.

### **Uses for the Data Citation Corpus** {#uses-for-the-data-citation-corpus .wp-block-heading}

We have received over 70 requests for the data file of the corpus, from
groups and individuals in a variety of roles. Those requesting the file
have expressed a common interest in better understanding open data
practices and the reach of open data, but their specific use cases are
tailored to their professional needs:

- [Researchers]{style="text-decoration: underline;"}: Researchers are
 interested in using the corpus for bibliometric analyses, to study
 practices around data sharing and reuse, and correlations between
 specific parameters of datasets (e.g. metadata quality, whether are
 data are associated with an article) and the level of reuse.
- [Librarians]{style="text-decoration: underline;"}: Many of the
 librarians seek to identify citations for data hosted by their
 institutions and are exploring ways to integrate data citations into
 scholarship assessment reports.
- [Infrastructure Providers]{style="text-decoration: underline;"}: A key
 focus for this group is to improve data citation coverage in existing
 platforms and services, and they are looking to check for additional
 citations in the corpus to complement those they already store or
 expose. Infrastructure providers also seek to leverage data citations
 to enhance their search and discovery features for digital objects. 
- [Institutional Administrators]{style="text-decoration: underline;"}:
 This group's main interest is to analyze data citations from their
 institutions to evaluate research impact, and in turn incentivize open
 science practices.
- [Publishers]{style="text-decoration: underline;"}: Publisher
 representatives are looking to identify citations for data, compare
 those to their existing indices, and analyze bibliographic patterns,
 especially in the context of open access publications.
- [Data Repositories]{style="text-decoration: underline;"}: These seek
 to compare citations for their datasets with those found in other
 sources, to showcase the impact and relevance of their datasets.

There are also additional nuanced use cases by other community members
and we are keen to hear about the different ways in which the Data
Citation Corpus can serve different data evaluation needs. While it is
still early days, it is encouraging to see that the value of the Data
Citation Corpus is understood across so many sectors of the scholarly
community.  

We are still at an early stage and much work lies ahead to enhance the
Data Citation Corpus as a tool that addresses the needs of these diverse
communities, but we are encouraged by the response we have received so
far. We are confident that the corpus aligns with a recognized need to
better understand the use and reach of datasets. As we work to improve
the current data file and incorporate additional data sources, we will
continue to actively engage with the community and report on the many
ways the Data Citation Corpus is being used.

*If you would like to request access to the current data file (and be on
the mailing list for further iterations of the corpus as they become
available), please complete* [*this
form*](https://docs.google.com/forms/d/e/1FAIpQLSd1l7ovTQs3EMw9mz4HFaVB2SuUQ8Z8FldoCDgvD74GV-vh0Q/viewform){target="_blank"
rel="noopener"}*.  *

<figure class="wp-block-table">
<table class="has-background" style="background-color: #abe5f2;">
<tbody>
<tr class="odd">
<td class="has-text-align-center" data-align="center">You can
find the slides from the talks at Zenodo (<a
href="https://zenodo.org/records/10689709" target="_blank"
rel="noopener">here</a> &amp; <a
href="https://zenodo.org/search?q=metadata.subjects.subject%3A%22Unlocking%20the%20Power%20of%20Data%20Citation%3A%20First%20Release%20of%20the%20Data%20Citation%20Corpus%22&amp;l=list&amp;p=1&amp;s=10&amp;sort=bestmatch"
target="_blank" rel="noopener">here</a>) and watch the recording of the
webinar on <a
href="https://www.youtube.com/watch?v=yrHAHbS6lvU&amp;t=30s"
target="_blank" rel="noopener">Youtube</a>.</td>
</tr>
</tbody>
</table>
</figure>

### **First-of-its-kind aggregation brings together millions of data citations to advance understanding of data usage** {#first-of-its-kind-aggregation-brings-together-millions-of-data-citations-to-advance-understanding-of-data-usage .wp-block-heading style="text-align: left;"}

[*https://doi.org/10.60804/r14z-mw10*](https://doi.org/10.60804/r14z-mw10){target="_blank"
rel="noopener"}

DataCite, in partnership with the Chan Zuckerberg Initiative (CZI), is
delighted to announce the first release of the [Data Citation
Corpus](https://makedatacount.org/data-citation/){data-type="link"
data-id="https://makedatacount.org/data-citation/"}. A major milestone
in the Make Data Count initiative, the release makes eight million data
citations openly available and usable for the first time via an
interactive dashboard and public data file. We invite the community to
engage with the data and provide feedback on this collaborative effort.

As highlighted by Make Data Count, the lack of a centralized resource
for citations to datasets has hindered the evaluation of how open data
is being used. To address this gap, DataCite, with funding from the
[Wellcome Trust](https://wellcome.org/){data-type="link"
data-id="https://wellcome.org/" target="_blank" rel="noopener"}, has
developed an innovative aggregation that brings together for the first
time data citations from diverse sources into a comprehensive and
publicly accessible resource for the global community.

> "There is a pressing need to understand how open data is used, but we
> have lacked a resource to access this information in a centralized and
> open manner. The Data Citation Corpus will allow the community to gain
> access to critical insights on data usage." said Iratxe Puebla,
> Director of Make Data Count. "We are thrilled to share the progress
> from our collaboration with CZI to bring together citations from
> different sources, and look forward to working with others in the
> community to expand the breadth and coverage of the corpus."

The first release of the corpus includes data citations in DataCite and
Crossref metadata as well as asserted data citations contributed by CZI,
available to the community via a data citation store and dashboard
developed by Coko. Leveraging accession numbers from Europe PMC, CZI
applied a machine-learning model to a large set of full-text articles
and preprints to extract mentions to datasets. **This has enabled the
first-ever aggregation of citations for datasets with DOIs and accession
numbers into a single corpus,** enabling a more complete picture of data
usage**.**

> "As an organization that invests in research data and reference
> datasets, we believe it is critical to understand how data is shared
> and reused to enable new scientific discoveries," said Patricia
> Brennan, Vice President of Science Technology at the Chan Zuckerberg
> Initiative. "DataCite has been a leader in this space, providing
> critical infrastructure for data citation and for tracking its reuse.
> We're proud to support them in their vision to build a comprehensive
> global corpus of actionable data citations."

The interactive
[dashboard](http://corpus.datacite.org/dashboard){target="_blank"
rel="noopener"} of the corpus allows users to visualize and report on
citations by a variety of facets, such as funder, data repository, or
the journal where the article citing the data is published.

::: {.wp-block-spacer style="height: 13px;" aria-hidden="true"}
 
:::

::: {.wp-block-image}
<figure class="aligncenter size-large is-resized">
<img
src="https://makedatacount.org/wp-content/uploads/2024/09/Screenshot-2024-01-19-at-14.20.28-1-1024x303.png"
class="wp-image-1049" style="width: 892px; height: auto;"
decoding="async" />
</figure>
:::

::: {.wp-block-spacer style="height: 32px;" aria-hidden="true"}
 
:::

A complete data file of all of the citations is also available for
additional analysis and evaluation. Request the data file via this
[form](https://forms.gle/JeiGC6updejQemAW6){target="_blank"
rel="noopener"}.

Forthcoming releases will focus on addressing existing metadata gaps,
for example, related to the disciplinary information for the datasets,
and on incorporating feedback from early adopters. DataCite will also
pursue new collaborations with additional citation aggregators to expand
the breadth and scale of data citations in the corpus. 

Community input is an integral part of this project and DataCite invites
researchers, institutions, funders and infrastructure providers to
provide feedback on the first release of the corpus and future
development work. Please join us for an online webinar on February 22 to
learn more about the first release of the corpus and how to use it.
[Register
now](https://datacite.zoom.us/webinar/register/WN_fm2O1BqxRomjXzkgPWLhaQ){data-type="link"
data-id="https://datacite.zoom.us/webinar/register/WN_fm2O1BqxRomjXzkgPWLhaQ"
target="_blank" rel="noopener"} to participate in this interactive
session.

::: {.wp-block-spacer style="height: 27px;" aria-hidden="true"}
 
:::

:::: {.wp-block-buttons .is-content-justification-center .is-layout-flex .wp-container-core-buttons-is-layout-4 .wp-block-buttons-is-layout-flex}
::: {.wp-block-button}
[Register for the
webinar](https://datacite.zoom.us/webinar/register/WN_fm2O1BqxRomjXzkgPWLhaQ){.wp-block-button__link
.has-background .has-text-align-center .wp-element-button
style="background-color: #073763;" target="_blank" rel="noopener"}
:::
::::

::: {.wp-block-spacer style="height: 43px;" aria-hidden="true"}
 
:::

## **About DataCite** {#about-datacite .wp-block-heading}

[DataCite](https://datacite.org/){target="_blank" rel="noopener"} is a
global community that shares a common interest: to ensure that research
outputs and resources are openly available and connected so that their
reuse can advance knowledge across and between disciplines, now and in
the future. 

## **About Chan Zuckerberg Initiative** {#about-chan-zuckerberg-initiative .wp-block-heading}

The [Chan Zuckerberg
Initiative](https://chanzuckerberg.com/){target="_blank" rel="noopener"}
was founded in 2015 to help solve some of society's toughest challenges
--- from eradicating disease and improving education, to addressing the
needs of our communities. Through collaboration, providing resources and
building technology, our mission is to help build a more inclusive, just
and healthy future for everyone.