Comparative Analysis of Academic Databases

What is the comparative coverage and retrieval performance of multidisciplinary academic databases (OpenAlex, Semantic Scholar, Dimensions) versus traditional subscription databases (Web of Science, Scopus, EMBASE) across different academic disciplines?

Multidisciplinary databases such as OpenAlex demonstrate higher retrieval rates and better coverage of open access and social sciences content compared to traditional databases, whereas subscription services like Scopus and Web of Science excel in metadata accuracy and citation tracking for STEM disciplines.

Abstract

Multidisciplinary academic databases tend to retrieve a broader range of records than traditional subscription services. Several studies report that OpenAlex, for example, returns up to 100% more records than Scopus and 27.9–29.3% more than Web of Science in specific comparisons. Open sources consistently capture more open access, non‐English, and social sciences and humanities content, as highlighted by studies that observed better coverage in the Global South and for emerging fields. In contrast, traditional databases such as Web of Science, Scopus, and EMBASE consistently deliver higher metadata quality, more accurate document type classification, and more reliable citation tracking, particularly in STEM and health-related literature.

Methods

We analyzed 23 sources from an initial pool of 500, using 7 screening criteria. Each paper was reviewed for 8 key aspects that mattered most to the research question.

Records from Elicit search

• n = 0 Papers screened using: Database Comparison, Performance Metrics, Study Design, Disciplinary Scope, Database Focus, Empirical Comparison, Publication Type
• n = 500 Papers screened out
• n = 477 Papers included for extraction
• n = 23

Paper search

We performed a semantic search across over 138 million academic papers from the Elicit search engine, which includes all of Semantic Scholar and OpenAlex.

We retrieved 500 papers most relevant to the query for screening.

Screening

We screened in sources based on their abstracts that met these criteria:

• Database Comparison: Direct comparison of multidisciplinary academic databases (OpenAlex, Semantic Scholar, Dimensions) with traditional subscription databases (Web of Science, Scopus, EMBASE).
• Performance Metrics: Measurement of coverage metrics (e.g., indexed publications, disciplinary breadth) and retrieval performance metrics (e.g., recall, precision).
• Study Design: Empirical studies providing quantitative evidence about database performance.
• Disciplinary Scope: Analysis across multiple academic disciplines or discipline-specific comparisons.
• Database Focus: Inclusion of specified databases (OpenAlex, Semantic Scholar, Dimensions, Web of Science, Scopus, EMBASE).
• Empirical Comparison: Providing empirical comparisons rather than solely descriptive database features.
• Publication Type: Research articles with original empirical data.

Data extraction

We extracted each data column from each paper, focusing on:

• Database Comparison
• Study Methodology
• Disciplines Examined
• Time Period
• Coverage Performance
• Retrieval Performance
• Disciplinary Differences
• Contextual Factors

Results

Coverage Performance

Metadata Quality and Retrieval Accuracy

• OpenAlex shows variable metadata quality with issues in author disambiguation, affiliation matching, and document type classification.
• Retrieval completeness for systematic reviews is high (up to 98%).
• Semantic Scholar and Dimensions perform well in author disambiguation and thematic classification.
• Traditional Databases (Web of Science, Scopus) maintain higher metadata quality and more accurate document classifications.

Citation Tracking Performance

• OpenAlex provides more citations than Scopus and Web of Science in several studies.
• Cross-referencing multiple databases improves citation tracking completeness.

Conclusion

Open multidisciplinary databases are improving in coverage but still lag behind traditional databases in metadata quality and document type accuracy. The choice of database has significant implications for coverage, retrieval performance, and the reliability of bibliometric analyses.