How to Audit ChatGPT Citations Step by Step

4-Step Audit Rule: Never audit an LLM bibliography line by line in isolation. First reconcile in-text numbered markers against the reference section, screen every DOI against authoritative registries before checking non-DOI titles, triage Mismatch entries separately from unindexed Not found records, and archive a timestamped evidence report.

Four Specific Failure Fingerprints in LLM Reference Drafts

Students and co-authors rarely fabricate citations intentionally. Problems typically enter manuscripts when an assistant summarizes background literature, suggests five supporting citations under tight prompting, or reorganizes an outline across multiple chat turns. Across real-world audit runs, LLM bibliographies repeatedly exhibit four structural fingerprints:

1. Orphan In-Text Brackets and Trailing Ghost Entries: Reordering paragraphs in chat shifts citations smoothly in prose—leaving markers like [8] referencing nothing while trailing list entries never appear in the body.
2. Real Publisher Prefixes + Synthetic Suffixes: Models memorize legitimate DOI prefixes such as 10.1038/ (Springer Nature) or 10.48550/ (arXiv/DataCite), then synthesize realistic-looking suffixes that return HTTP 404.
3. Author-Year Drift Under Paraphrased Topics: Combining a widely known first author with an adjacent co-author or shifting a 2018 landmark paper to 2022–2024 breaks downstream metadata reconciliation.
4. Venue Upgrade Hallucinations: Turning an arXiv preprint or workshop extended abstract into a finished journal citation in Nature Medicine or Lancet Digital Health—complete with fabricated volume and issue numbers.

Step 1: Reconcile Whole-Draft Markers Before Checking Databases

Before spending time looking up individual titles, validate structural correspondence inside the manuscript body. When auditing numbered formats such as IEEE or Vancouver brackets, expanded ranges like [1-3] must map cleanly onto bibliography entries without exceeding total list length.

When CitePure’s deterministic reconciler parses a draft containing prose markers [1, 2], range [1-3], and a trailing stray marker [8] against a four-item bibliography, its range expander resolves [1, 2, 3, 8], immediately catching both out-of-range marker [8] and unused bibliography entry [4] in milliseconds—without touching any external server.

Reconciler Input Sample:
  In-text prose  : "...trials [1, 2]... transformer models [1-3] accelerated screening [8]."
  References list: [1] Vaswani 2017  [2] LeCun 2015  [3] Zhou 2020  [4] Silver 2016

Deterministic Output:
  marks  : [1, 2, 3, 8]     // [1-3] expanded deterministically to 1, 2, 3
  over   : [8]              // Cited marker > 4 reference items
  unused : [4]              // Silver (2016) present in bibliography but never cited in prose

Methodological note: If you test only a short excerpt with fewer prose sentences than citations (meaning ≥ 60% of lines look like references), paste an extra body paragraph above the References heading so draft-mode detection engages automatically, or review entries directly in the Verification Studio.

Step 2: Run Crossref-First DOI Lookups, Then Non-DOI Search

Because direct identifier resolution avoids ambiguous title collisions, citations carrying DOIs and citations lacking DOIs should follow separate resolution paths:

• With a DOI string: Query Crossref’s /works/{doi} endpoint first. If Crossref returns HTTP 404 (common with DataCite-registered preprints such as arXiv), fall back automatically to OpenAlex’s DOI endpoint (/works/https://doi.org/{doi}). For rapid single-entry forensics, paste suspicious identifiers into our DOI Resolver & Chimera Checker.
• Without a DOI string: Query OpenAlex full-text work search alongside Crossref bibliographic search, spacing sequential calls cleanly (CitePure uses a 180 ms client throttle plus a 900 ms automatic backoff on HTTP 429 rate limits) and re-ranking candidates by composite title, year, author surname, and journal agreement rather than relying on raw API search rank.

Step 3: Triage Every Result Into Verified, Mismatch, or Not Found

Treating all non-green statuses identically leads either to false accusations against legitimate preprints or to overlooking subtly corrupted references. Apply strict action criteria to each classification:

Step 4: Repair Valid Entries and Document Proof Before Submission

Once genuine references are separated from hallucinations, address remaining edge cases in two passes:

1. Fix broken metadata formatting cleanly: When a legitimate paper fails automated parsing due to deprecated http://dx.doi.org/ prefixes, misplaced publisher cities, or malformed years, normalize the string in our Citation Corrector against official Crossref fields.
2. Archive a pre-submission evidence bundle: Before submitting to a thesis advisor or journal portal, export CitePure’s standalone HTML Verification Certificate (or .docx source evidence report from Source Finder). Preserving matched DOIs, exact composite scores, and UTC verification timestamps creates an auditable integrity log without creating cloud accounts or uploading manuscript text.