Elicit: Comparative Analysis of Academic Databases
Comparative Analysis of Academic Databases
What is the comparative coverage and retrieval performance of multidisciplinary academic databases (OpenAlex, Semantic Scholar, Dimensions) versus traditional subscription databases (Web of Science, Scopus, EMBASE) across different academic disciplines?
Multidisciplinary databases such as OpenAlex demonstrate higher retrieval rates and better coverage of open access and social sciences content compared to traditional databases, whereas subscription services like Scopus and Web of Science excel in metadata accuracy and citation tracking for STEM disciplines.
Abstract
Multidisciplinary academic databases tend to retrieve a broader range of records than traditional subscription services. Several studies report that OpenAlex, for example, returns up to 100% more records than Scopus and 27.9–29.3% more than Web of Science in specific comparisons. Open sources consistently capture more open access, non‐English, and social sciences and humanities content, as highlighted by studies that observed better coverage in the Global South and for emerging fields. In contrast, traditional databases such as Web of Science, Scopus, and EMBASE consistently deliver higher metadata quality, more accurate document type classification, and more reliable citation tracking, particularly in STEM and health-related literature.
Methods
We analyzed 23 sources from an initial pool of 500, using 7 screening criteria. Each paper was reviewed for 8 key aspects that mattered most to the research question.
Paper search
We performed a semantic search across over 138 million academic papers from the Elicit search engine, which includes all of Semantic Scholar and OpenAlex.
We ran this query: “What is the comparative coverage and retrieval performance of multidisciplinary academic databases (OpenAlex, Semantic Scholar, Dimensions) versus traditional subscription databases (Web of Science, Scopus, EMBASE) across different academic disciplines? Focus on papers after 2024 in your analysis.”
We retrieved 500 papers most relevant to the query for screening.
Screening
We screened in sources based on their abstracts that met the following criteria:
- Database Comparison: Direct comparison of multidisciplinary academic databases (OpenAlex, Semantic Scholar, Dimensions) with traditional subscription databases (Web of Science, Scopus, EMBASE).
- Performance Metrics: Measurement of coverage metrics or retrieval performance metrics.
- Study Design: Empirical studies providing quantitative evidence about database performance.
- Disciplinary Scope: Analysis across multiple academic disciplines or discipline-specific comparisons.
- Database Focus: Inclusion of specified multidisciplinary and traditional databases.
- Empirical Comparison: Provision of empirical performance comparisons.
- Publication Type: Research articles with original empirical data.
Data extraction
We extracted data columns from each paper, including:
- Database Comparison
- Study Methodology
- Disciplines Examined
- Time Period
- Coverage Performance
- Retrieval Performance
- Disciplinary Differences
- Contextual Factors
Results
Characteristics of Included Studies
| Study | Database(s) Studied | Performance Metrics Evaluated | Sample Size | Key Focus Area | Full text retrieved |
|---|---|---|---|---|---|
| Thelwall and Jiang, 2025 | OpenAlex, Scopus | Coverage, citation counts, document type/coverage | 28.6 million articles | Comparative coverage and citation indicator performance | Yes |
| Delgado-Quirós and Ortega, 2025 | Dimensions, OpenAlex, Semantic Scholar | Coverage, metadata quality | >115,000 DOIs | Research entity information and coverage | No |
| Fuente-Gutiérrez and Kippes, 2025 | OpenAlex, Web of Science, Scopus | Coverage, disciplinary representation | 47,354 articles | Co-authorship and coverage in Argentina-Spain collaboration | Yes |
| Mokhnacheva, 2025 | OpenAlex, The Lens | Coverage of Russian publications | Not mentioned | Russian publication coverage in global context | No |
| Rajit et al., 2025 | OpenAlex, Semantic Scholar, Embase | Coverage, overlap | 2023 PCOS guideline articles | Automated evidence retrieval for living guidelines | No |
| Mongeon et al., 2025 | OpenAlex, Web of Science | Metadata quality, document type accuracy | 7.6 million Web of Science records | Document type discrepancies | Yes |
| Simard et al., 2025 | OpenAlex, Web of Science, Scopus | Coverage, Gini coefficients | Not mentioned | Open access journal coverage | Yes |
| Li et al., 2025 | OpenAlex, Web of Science | OA book coverage | 255,810 (OpenAlex), 8,713 (Web of Science) | OA academic book trends | Yes |
| Mezquita et al., 2025 | OpenAlex, Scopus, Web of Science | Coverage, citation counts | Not mentioned | Nursing literature coverage | No |
| Jahn, 2025 | OpenAlex, Scopus, Web of Science | Coverage, metadata quality | >13,000 hybrid journals | Transformative agreement impact on hybrid OA | Yes |
Effects
Coverage Performance
| Study | Database | Coverage Rate (%) | Overlap with Other Databases (%) | Disciplinary Variations |
|---|---|---|---|---|
| Thelwall and Jiang, 2025 | OpenAlex | >100% of Scopus | Not mentioned | Better for Global South, more OA/data journals |
| Delgado-Quirós and Ortega, 2025 | OpenAlex | 89.9% authors with one publication | Not mentioned | Problems with books/book chapters |
| Fuente-Gutiérrez and Kippes, 2025 | OpenAlex | 27.9% more than Web of Science | Not mentioned | Better in social sciences and humanities, less bias to Spanish |
| Mokhnacheva, 2025 | OpenAlex | Not mentioned | Not mentioned | Better for Russian publications than The Lens |
| Rajit et al., 2025 | OpenAlex | 98.6% | 90.5% (all 4 databases) | Health sciences (PCOS) |
| Mongeon et al., 2025 | OpenAlex | Not mentioned | Not mentioned | Broader in humanities, non-English, Global South |
| Simard et al., 2025 | OpenAlex | Most DOAJ journals | 16% | Best for social sciences and humanities, multilingual journals |
| Li et al., 2025 | OpenAlex | 9.46% OA books | Not mentioned | Better in emerging fields, OA books |
| Mezquita et al., 2025 | OpenAlex | Nearly all nursing journals | Not mentioned | More non-Western sources |
| Jahn, 2025 | OpenAlex | Not mentioned | Not mentioned | Limited corresponding author data |
Metadata Quality and Retrieval Accuracy
OpenAlex:
- Metadata quality is variable, with issues in author disambiguation, affiliation matching, and document type classification.
- One large-scale study found that 97% of document type misclassification errors occurred in OpenAlex.
- Retrieval completeness for systematic reviews is high (up to 98%), but precision and recall can vary depending on the search strategy and discipline.
Citation Tracking Performance
OpenAlex:
- Provides more citations than Scopus and Web of Science, though missing citations and incomplete reference lists are observed for recent publications.
Key Points:
- Citation tracking is more comprehensive when multiple databases are used in combination.
Comparative Performance Across Database Categories
- Multidisciplinary Open Databases (OpenAlex, Semantic Scholar, Dimensions): Superior in open access and social sciences.
- Traditional Subscription Databases (Web of Science, Scopus, Embase): Superior in metadata quality, document type accuracy, and coverage of STEM and health.
Key Points:
- The choice of database has significant implications for coverage, retrieval performance, and the reliability of bibliometric analyses.