AI Hallucinations in Academic Writing
AI Hallucinations in Academic Writing:
What They Are and How to Catch Them
From fabricated citations at NeurIPS to a $440,000 Deloitte scandal — the crisis is bigger, and closer to home, than most researchers want to admit.
There’s a moment every researcher dreads. You’re fact-checking a paper — maybe your own, maybe a student’s — and you track down a citation. You type the DOI into the browser. Nothing. You run the title through Google Scholar. Nothing. You try the journal’s own search. The paper, the study, the expert it quotes with such confidence — doesn’t exist. Welcome to the age of AI hallucinations in academic writing. It’s not an edge case anymore.
What AI Hallucinations Actually Are (and Aren’t)
The term “hallucination” gets thrown around a lot, sometimes lazily. It isn’t a bug in the traditional software sense — a miscalculated variable or a crashed function. It’s something stranger. When a large language model generates text, it doesn’t retrieve facts from a database. It predicts the next most-likely token, again and again, based on patterns learned from billions of words. That process produces fluent, confident-sounding prose. It also, unfortunately, produces fluent, confident-sounding fiction.
The term itself is contested. Journalist Benj Edwards at Ars Technica has argued the word is misleading because it implies the model has something like perception that breaks down. Some researchers prefer “confabulation” — a term borrowed from neuropsychology, referring to the brain’s tendency to fill gaps with plausible-sounding invented memories, without any intention to deceive. That framing is actually quite apt. The AI isn’t lying. It doesn’t have intentions. It’s just filling in the pattern.
AI hallucination in this context means: any AI-generated output that is presented as factual but is partially or wholly fabricated — including invented citations, non-existent authors, false statistics, fictitious court rulings, or distorted summaries of real research.
What makes this particularly troubling for academic writing is the confidence level. The model doesn’t hedge. It doesn’t say “I think there might be a paper by…” It generates a full citation — author names, journal title, volume, issue, page numbers, DOI — with the same flat certainty it uses to tell you what year the French Revolution started. That’s what catches people.
Why Academic Writing Is Especially Vulnerable
Not all hallucinations are equally dangerous. If an AI invents a recipe ingredient, someone might have a bad dinner. If it invents a legal precedent, someone might go to prison. Academic writing sits somewhere that makes hallucinations particularly insidious, for several reasons.
First, there’s the sheer volume of AI use in research workflows right now. Students use it for literature reviews. Researchers use it to draft introductions. Postdocs use it to summarize papers they haven’t fully read. The AI tools are deeply woven into the writing process — not always disclosed — and most users aren’t running systematic verification checks.
Second, citation verification is genuinely tedious. A literature review might cite 60 or 80 sources. Checking every one by hand would take hours. So people spot-check, and AI hallucinations are very good at not being the ones that get spot-checked, because they look entirely normal.
Third, and this is the subtle one: AI hallucinations are worse in niche or poorly-documented fields. If you ask ChatGPT for citations on Keynesian economics, you’ll probably get real papers — there’s an enormous amount of training data. Ask it for citations on, say, the psychology of religious extremism in rural communities, and you’re in much more dangerous territory. The model interpolates from what it knows and generates something plausible. Plausible is not the same as real.
Using AI in research isn’t the problem. The problem is using it for citation generation without verification — or assuming that because the AI sounds confident, the source exists. Disclosure and verification are what matter.
The Five Main Hallucination Types in Research Writing
Hallucinations don’t all look the same. Knowing what varieties exist makes them easier to catch. Based on current literature and real-world cases, five main types show up repeatedly.
| Type | What it looks like | Detection difficulty | Risk level |
|---|---|---|---|
| Ghost citation | Fully fabricated paper — plausible title, real-looking authors, correct journal format, nonexistent DOI | Medium | High |
| Chimera citation | Real paper, but with elements blended from other papers — wrong authors added, altered title, misattributed findings | Hard | High |
| Misrepresented source | Real paper exists, but the AI’s summary contradicts or exaggerates its actual findings | Very hard | High |
| Statistical fabrication | Specific numbers, percentages, or effect sizes that sound precise but have no source or are distorted | Medium | Medium–High |
| Authority attribution | Real expert’s name attached to a quote or claim they never made; real institution attached to nonexistent study | Hard | Very High |
The chimera citation is arguably the most dangerous type, precisely because the base paper is real. If you search the DOI, you’ll find something. It’s only when you compare what the AI said about the paper versus what the paper actually says that the problem emerges. And that requires actually reading the source, which — let’s be honest — a lot of people skip.
Scale of the Problem: Hard Numbers from 2025–2026
These numbers deserve some context, not to minimize them but to read them correctly. The Deakin study found that hallucination rates varied enormously by topic maturity — ChatGPT was 94% accurate for well-documented conditions like major depression, but fabrication rates shot to nearly 30% for niche topics with less training data. That’s a crucial finding. It means you can’t generalize across domains. A historian citing AI on World War I may have very different risks than a psychologist citing AI on emerging treatment protocols.
The NeurIPS 2025 findings, though, are particularly sobering. These weren’t undergraduate essays. They were papers submitted to one of the most rigorous AI conferences in the world, each reviewed by three to five experts who — theoretically — understand the systems better than almost anyone else. The hallucinations still got through.
Real Cases That Broke Into the Open
Abstract statistics are one thing. What makes the scale of this problem viscerally clear are the real incidents — the ones where the hallucination wasn’t caught before it became public.
A $440,000 Government Report Built on Invented Research
Deloitte Australia was commissioned by the Australian government’s Department of Employment and Workplace Relations to produce a 237-page audit of its automated welfare compliance system. The contract was worth A$440,000. The report was published in July 2025.
It unraveled when Dr. Chris Rudge, a health law researcher at the University of Sydney, started reading it. The first error that caught his eye was a citation attributing a book to a colleague, Professor Lisa Burton Crawford — a book that didn’t exist, on a topic outside her expertise entirely. “I instantaneously knew it was either hallucinated by AI or the world’s best kept secret,” he told the Associated Press.
Further investigation revealed up to 20 errors: references to nonexistent academic papers, citations from scholars who never wrote the attributed works, and a fabricated quote attributed to Federal Justice Jennifer Davies (whose name was also misspelled). Deloitte eventually confirmed the firm had used Azure OpenAI GPT-4o and agreed to a partial refund. Australian Senator Barbara Pocock’s summary was pointed: “The kinds of things that a first-year university student would be in deep trouble for.”
Key lesson: AI tools used without human verification at the citation level can contaminate even professionally produced, publicly funded reports.
100+ Fabricated Citations at the World’s Most Prestigious AI Conference
GPTZero, a Canadian AI detection startup, analyzed 4,841 papers accepted and presented at NeurIPS (Neural Information Processing Systems) 2025. They found more than 100 hallucinated citations spread across at least 53 published papers — roughly 1% of all accepted submissions, which sounds small until you consider that each paper represents months or years of work and was reviewed by multiple domain experts.
The hallucinations were sophisticated. In some cases, an AI model blended elements from multiple real papers — believable titles, plausible author names, legitimate-looking journal venues — creating chimera citations that required deep subject-matter knowledge to catch. Some fabricated citations even included fake co-authors appended to real papers, a pattern that “no human would reasonably make,” according to GPTZero CTO Alex Cui.
Key lesson: Even expert peer reviewers in the very field generating the hallucinations are not a reliable safety net.
56% Error Rate in Mental Health Literature Reviews
Researchers at Deakin University’s School of Psychology tested ChatGPT (GPT-4o) by asking it to generate literature reviews across three mental health topics with varying levels of research documentation: major depressive disorder (well-documented), binge eating disorder (moderately documented), and body dysmorphic disorder (less documented).
Of 176 citations produced, only 77 (43.8%) were both real and accurate. 19.9% were completely fabricated. Of the remainder that were real, 45.4% contained errors — wrong publication dates, incorrect page numbers, broken DOIs. Topic mattered enormously: depression citations were 94% real; BDD citations had a fabrication rate near 29%. Among fabricated citations that included DOIs, 64% linked to real but completely unrelated papers — making the errors harder to detect without careful reading.
Key lesson: GPT-4o, one of the most capable models available, still can’t be trusted for citation generation — particularly in niche fields.
“You cannot trust the recommendations when the very foundation of the report is built on a flawed, originally undisclosed, and non-expert methodology.”— Dr. Chris Rudge, Deputy Director of Health Law, University of Sydney, responding to Deloitte’s claim that their report’s “substance” remained intact despite fabricated citations (October 2025)
Red Flags: How to Spot a Hallucination Before It Spreads
There’s no single foolproof signal, but certain patterns appear repeatedly in documented hallucinations. Train yourself to notice them.
- The DOI goes somewhere wrong. The DOI is valid and resolves, but the paper it links to is completely different from what the AI described. This is the chimera citation pattern — and it’s insidious because you did check, just not carefully enough.
- The author publishes in one field, the citation is in another. Like Lisa Burton Crawford being cited for a book on welfare policy when her entire body of work is in constitutional law. Real researchers specialize — when the citation topic doesn’t match the scholar’s known expertise, investigate.
- The journal exists but the issue doesn’t. The AI correctly names a real peer-reviewed journal, but the volume and issue number it specifies don’t exist. Check the journal’s archive directly.
- The statistics are oddly precise and unsourced. “Studies show that 73.2% of…” where no specific study is named, or the named study doesn’t contain those numbers when you actually look.
- The quote is attributed to a court ruling or government document. These are particularly high-risk. The Deloitte case featured a completely invented judicial quote. Legal and official documents are harder to verify quickly, which is exactly why AI tends to hallucinate them more confidently.
- The citation is from a very recent paper, but the AI was trained before that date. Temporal impossibility is an obvious red flag that sometimes gets missed.
- Multiple citations cluster around the same unusual topic. If the AI generates five citations all specifically relevant to a niche research question, that level of convenient specificity should trigger suspicion, not relief.
Detection Tools Worth Actually Using
The toolkit for catching hallucinations has developed rapidly over the past 18 months. Some of these are purpose-built for academic contexts; others are broader platforms being adapted for research use.
Purpose-built for academic papers. Ingests a PDF and searches across the open web and major academic databases to verify each citation — checking authors, titles, publication venues, and links. The tool that caught 100+ fabrications at NeurIPS 2025.
Free academic databases for manual cross-referencing. Search by DOI, title, or author. OpenAlex has indexed over 250 million scholarly works and is freely accessible. Good for bulk verification when you can script queries.
CrossRef maintains the authoritative registry for DOIs. If a DOI doesn’t resolve in CrossRef, it doesn’t exist. Simple, fast, and definitive for the “does this paper exist?” question.
Searches through peer-reviewed literature and grounds responses in actual papers. Useful for generating literature summaries with real citations — essentially an AI assistant that’s connected to a real database rather than generating from pattern completion.
AI assistants that cite real-time web sources. Not perfect, and still hallucinate, but they ground outputs in live web search results. Better for fact-checking than for generating new citations.
Tracks how papers have been cited by others — positively, negatively, or neutrally. Useful not just for verification but for understanding a paper’s standing in its field. If a claimed seminal paper has zero citations in Scite, that’s a red flag.
No tool currently catches misrepresented sources reliably — cases where the paper is real but the AI’s description of its findings is wrong. That still requires a human to open and read the source document. The tool verification above handles ghost and chimera citations, not conceptual distortions.
A Practical Verification Workflow for Researchers
There’s no magic here. Verification is just disciplined process. The goal is to make it systematic enough that it takes minimal cognitive overhead while still being thorough.
Never use AI to generate citations directly
Use AI to help you find literature (with Consensus, Semantic Scholar, or similar grounded tools), but never to produce a formatted reference list. The citation generation step is where hallucinations are most concentrated. Generate your own citations from databases, then use AI to help draft the prose around them.
Verify every DOI, not a sample
If the AI did contribute to a reference list, verify every single citation — not spot-check. Use CrossRef or Semantic Scholar for bulk DOI resolution. It takes less time than it sounds if you do it systematically. Paste all DOIs into CrossRef’s bulk lookup at once.
Check author-topic alignment
For citations involving specific claims from specific scholars, run a quick Google Scholar search on the cited author. Do they work in that area at all? Have they published in that journal before? Red flags here don’t prove fabrication but they justify deeper investigation.
Read the abstract of every cited paper
This is the misrepresentation check. Does the abstract actually support the claim you’re making? This step catches chimera citations and distorted summaries that tool-based verification misses entirely. Yes, it takes time. It’s also what academic writing is supposed to involve.
Run a pre-submission hallucination check
Before submitting to a journal or conference, run the manuscript through GPTZero’s Hallucination Check or equivalent. This should be standard practice in any AI-assisted research workflow — the same way spell-check is standard. It takes minutes and the downside of skipping it is severe.
Disclose AI involvement fully
Most journals now have AI disclosure requirements, but disclosure matters beyond compliance. It creates accountability — for you and for whoever reviews your work. The Deloitte report’s failure to disclose Azure OpenAI use was part of what turned a technical error into a professional scandal.
A word on effort levels: the workflow above sounds demanding because it is. But that effort is proportional to what’s at stake. For a 10-page student essay with 15 citations, a full verification sweep takes about 30 minutes. For a doctoral dissertation, it’s a multi-day project. In both cases, the alternative is staking your academic reputation on text that an AI hallucinated at 8,000 tokens per second.
What Universities and Journals Are Doing — and Not Doing
Institutions are responding, but unevenly. Some are ahead of the problem; many are scrambling.
What’s actually happening
Most major journals now require disclosure of AI use in manuscript preparation. Nature, Science, The Lancet, JMIR — all have updated their editorial policies since 2023. NeurIPS itself introduced reviewer guidance on hallucination flagging for its 2025 program, though as the GPTZero findings showed, guidance and reliable detection are different things.
The University of Mississippi ran a 2024 study finding that many student-submitted citations were “partially or completely fabricated.” The response at that institution, and several others, has been to integrate AI literacy training into research methods courses — covering not just how to use AI tools, but specifically how they fail and what hallucinations look like in practice.
Where the gaps remain
The honest answer is that institutional policy hasn’t caught up with actual behavior. Students and researchers are using AI tools in research workflows at a scale that outpaces any enforcement or verification infrastructure universities currently have. The Deakin study from November 2025 reinforced something that’s been true for two years: even GPT-4o, with all its improvements over earlier models, still fabricates more than half of academic citations in niche domains. The models haven’t solved this. Better prompt engineering doesn’t solve this. Disclosure policies don’t prevent it — they just create a paper trail when things go wrong.
When a graduate student submits a dissertation containing fabricated citations generated by an AI tool, who bears responsibility? The student, for insufficient verification? The university, for inadequate AI literacy training? The AI developer, for producing a tool that generates fake citations with no warning? This remains genuinely unresolved — and universities need clearer frameworks before the next major case lands on their desk.
Where Things Are Heading in 2026 and Beyond
There’s some reason for cautious optimism here, though “cautious” is doing a lot of work in that sentence.
On the technical side, Retrieval-Augmented Generation (RAG) architectures are making significant inroads in research-focused tools. Rather than generating citations from pattern completion, RAG systems retrieve actual papers from live academic databases and use those documents as the basis for responses. Consensus AI, Elicit, and similar tools use this approach. It doesn’t eliminate hallucination — RAG systems still sometimes misrepresent what the retrieved documents say — but it substantially reduces ghost citations.
Model calibration is also improving. Research teams at several major labs are working on training models to express uncertainty rather than confabulate — to say “I’m not certain a paper on this exact topic exists” rather than inventing one. This is harder than it sounds, partly because confident-sounding outputs were what users originally rewarded during RLHF training, and retraining that preference takes time.
More encouraging is the emergence of citation verification as a standard step in peer review. GPTZero’s hallucination detection tools are being piloted by some conferences and journals as part of pre-review screening. If this becomes standard infrastructure — the way plagiarism detection (Turnitin, iThenticate) became standard over the last two decades — it would materially reduce the number of fabricated citations reaching publication.
The broader picture is this: AI tools are not going to exit research workflows. That ship has sailed. The question is whether the academic community builds verification habits and institutional infrastructure fast enough to prevent hallucinations from quietly corrupting the scholarly record at scale. Given the NeurIPS 2025 findings — hallucinations in papers at the world’s premier AI conference, reviewed by AI experts — the answer right now is “not yet.” But “not yet” is more hopeful than “never,” and the tools and awareness are genuinely developing.
The researchers who’ll navigate this era best aren’t the ones who refuse to use AI or the ones who use it uncritically. They’re the ones who understand exactly how these models fail, build that understanding into their workflow, and treat verification not as a burden but as the basic professional standard it has always been, just with a new class of failure mode to account for.
More from Trendix.tech
Primary Sources & Further Reading
All data cited in this article comes from peer-reviewed publications, published institutional reports, or credible primary journalism. Links verified as of May 2026.
- Linardon et al. (2025). “Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models.” JMIR Mental Health, 12, e80371. DOI: 10.2196/80371
- Ansari, S. (2026). “Compound Deception in Elite Peer Review: A Failure Mode Taxonomy of 100 Fabricated Citations at NeurIPS 2025.” arXiv:2602.05930
- Walters & Wilder (2023). “Fabrication and errors in the bibliographic citations generated by ChatGPT.” Scientific Reports. DOI: 10.1038/s41598-023-41032-5
- GPTZero (2026, January). “GPTZero finds 100 new hallucinations in NeurIPS 2025 accepted papers.” gptzero.me/news/neurips
- Deloitte Australia / DEWR (2025). “Targeted Compliance Framework Assurance Review” — revised version, October 2025. With disclosure of Azure OpenAI GPT-4o usage.
- arXiv:2602.17671 (2026). “AI Hallucination from Students’ Perspective: A Thematic Analysis.” arxiv.org
- Harvard Kennedy School Misinformation Review (2025). “New sources of inaccuracy? A conceptual framework for studying AI hallucinations.” misinforeview.hks.harvard.edu
- MacDonald (2024). False citation rates in psychology subfields. Mind Pad, Canadian Psychological Association.
External Sources & Corroborating Coverage
The claims in this article are drawn from peer-reviewed studies, institutional reports, and primary journalism. Below are direct links to the key sources, plus additional coverage from outlets that independently reported the same incidents.
Links verified as of May 2026. Some sources (e.g., GPTZero) have commercial interests in AI detection; we include them because they published primary data, but readers should evaluate all sources critically — including this one.
0 responses to “AI Hallucinations in Academic Writing: What They Are and How to Catch Them”