Core Diagnostic Finding: A
404 Resource not foundresponse from Crossref does not prove a paper is fabricated. Live API probes confirm that authentic DataCite DOIs such as10.48550/arXiv.1706.03762return HTTP404from Crossref while resolving HTTP200in OpenAlex—and invisible PDF characters like Soft Hyphens (U+00AD) or En-Dashes (U+2013) quietly trigger identical Crossref404failures.
Live API Proof: Why Prefix Ownership Determines 404 vs 200
Many researchers assume every valid DOI lives inside Crossref. In reality, the international DOI system operates across multiple independent Registration Agencies (RAs), including Crossref, DataCite, mEDRA, and JaLC. You can verify which agency owns any prefix by querying the official DOI RA router (https://doi.org/doiRA/{prefix}).
Executing live HTTP requests across three benchmark DOIs demonstrates exactly why single-registry checkers falsely reject authentic literature:
1. Official Prefix Ownership Check (doi.org/doiRA)
10.48550 (arXiv preprints) --> [{"DOI": "10.48550", "RA": "DataCite"}]
10.1038 (Springer Nature) --> [{"DOI": "10.1038", "RA": "Crossref"}]
10.99999 (Synthetic dummy) --> [{"DOI": "10.99999", "status": "DOI does not exist"}]
2. Crossref vs OpenAlex Resolution on Vaswani et al. (2017) "Attention Is All You Need"
GET https://api.crossref.org/works/10.48550/arXiv.1706.03762
--> HTTP 404 (680 ms): "Resource not found."
GET https://api.openalex.org/works/https://doi.org/10.48550/arXiv.1706.03762
--> HTTP 200 OK (727 ms): id="https://openalex.org/W2626778328", year=2017, source="arXiv (Cornell University)"
Any verification tool that stops after querying Crossref flags Attention Is All You Need—one of the most cited machine learning papers in history—as unresolved. Only secondary fallback resolution across OpenAlex or the DataCite API prevents false alarms on arXiv, Zenodo datasets, Figshare supplements, and institutional repositories (see arXiv metadata guidelines).
Six Technical Reasons Real Papers Fail Automated DOI Checks
1. DataCite, Zenodo, Software, and Preprint Prefixes
Besides arXiv (10.48550), software releases archived on Zenodo (10.5281), Dryad datasets (10.5061), and institutional theses register through DataCite rather than Crossref. Always confirm RA ownership via doi.org/doiRA/ or test the DOI in a multi-index validator such as CitePure’s DOI Checker before questioning a co-author’s bibliography.
2. Invisible PDF Soft Hyphens (U+00AD) and En-Dashes (U+2013)
Copying formatted references out of multi-column academic PDFs or rich-text word processors is the single most common source of false DOI errors. Word processors routinely autocorrect sequences around digits into typographic En-Dashes (U+2013), while PDF typesetting engines inject invisible Soft Hyphens (U+00AD) at column wraps.
Look at our live byte-level extraction and Crossref API tests against 10.1038/s41586-020-2012-7:
| Copy-Paste Corruption Type | Raw Input String | Extracted Pattern | Live Crossref API Response |
|---|---|---|---|
Clean ASCII Hyphen (U+002D) |
10.1038/s41586-020-2012-7 |
Exact identifier match | HTTP 200 OK |
PDF Soft Hyphen (U+00AD) |
10.1038/s41586020-2012-7 |
Invisible u00AD retained inside suffix |
HTTP 404 Resource not found. |
Auto-Replaced En-Dash (U+2013) |
10.1038/s41586–020–2012–7 |
Typographic – replaces ASCII - |
HTTP 404 Resource not found. |
| PDF Column Line-Break Space | 10.1038/ s41586-020-2012-7 |
Regex stops at whitespace; returns null |
Skipped (falls back to title search) |
URL-Encoded Slash (%2F) |
10.48550%2FarXiv.1706.03762 |
Fails literal 10.xxxx/ delimiter check |
HTTP 404 Resource not found. |
| Trailing Sentence Period / Paren | (see https://doi.org/10.1038/s41586-020-2012-7). |
Trailing ). stripped cleanly |
HTTP 200 OK |
3. Reproducible Regex Pre-Sanitizer for Corrupted DOI Lists
Before submitting a manuscript bibliography or running batch checks, normalize copy-pasted strings to eliminate zero-width artifacts, typographic dashes, stray spaces after 10.xxxx/, and encoded slashes:
// Sanitize PDF/Word Unicode corruption before DOI extraction
function sanitizeForDoi(raw) {
return String(raw || '')
.replace(/[u00ADu200Bu200Cu200DuFEFF]/g, '') // Strip invisible soft hyphens & zero-width chars
.replace(/[u2010-u2015u2212]/g, '-') // Convert typographic dashes/minus to ASCII '-'
.replace(/(10.d{4,9})%2F/gi, '$1/') // Decode URL-encoded prefix separator
.replace(/(10.d{4,9}/)s+/g, '$1') // Rejoin line-break space right after prefix slash
.trim();
}
For non-technical authors, pasting dirty references into the CitePure Citation Corrector normalizes legacy doi: and http://dx.doi.org/ strings automatically.
4. Publisher Deposit Lag on OnlineFirst / Ahead-of-Print Proofs
Editorial production systems frequently mint an identifier inside a publisher proof PDF and webpage days or weeks before batch XML metadata completes ingestion into Crossref and OpenAlex REST caches. During that deposit lag window, clicking the publisher landing page works while automated API endpoints temporarily return 404.
5. Books, Chapters Without Chapter DOIs, and Institutional Grey Literature
Monographs cataloged strictly via ISBN, OECD and WHO policy briefs, statutes, and archival collections rarely hold article-level Crossref deposits unless the book publisher paid for chapter-level DOI registration. Returning Not found prevents false-positive fuzzy matches to similarly titled journal reviews.
6. Translated Titles or Heavy Subtitle Truncation Without DOIs
When no DOI is present and a paper published in German, Japanese, Spanish, or Chinese appears in an English bibliography via an ad-hoc translated title—or when citation formatting drops subtitles completely—token overlap can slip below the minimum similarity threshold unless the original-language title was deposited alongside the English translation. Locate verified English or bilingual DOIs cleanly via the Academic Source Finder or review exact thresholds in our Methodology.
Leave a Reply