Prashant Garg, Thiemo Fetzer (2024), preprint (OSF Preprints 2024-11-15; SSRN Dec 2024; arXiv from 2025-01-12, econ.GN) — ARXIV-2501.06873
Key takeaways
- Turning 44,852 NBER/CEPR economics working papers (1980–2023) into LLM-extracted “claim graphs” (JEL-concept nodes, edges labelled by evidentiary basis) shows the share of causally identified claims rising from 7.7% (1990) to 31.7% (2020). Causal narrative structure and causal novelty predict top-five publication and citations, while non-causal complexity and novelty do not.
Seeded from the abstract and a skim, before the reading in NOTE-083. What follows is what the work says about itself.
As economics grows, a bottleneck is representing what papers claim in a form that can be compared and aggregated. The authors introduce evidence-annotated claim graphs: each paper becomes a directed network of standardized economic concepts, with edges labelled by the kind of evidence, including whether a causal-inference design supports them. A multi-stage AI workflow builds these graphs for 44,852 papers from 1980–2023. Causal edges rise from 7.7% in 1990 to 31.7% in 2020. Causal structure and causal novelty are positively associated with top-five publication and long-run citations, while their non-causal counterparts are weakly or negatively related.
Standing in the record
Filed from the survey of 2026-09-25 of work the anthology set aside as out of scope (tier B): 45 seconds of active reading over 5 sessions in the papers-feed tracker. It was filed Deferred, unread. NOTE-083 is the close reading of 2026-09-25, and it placed the work: Proposed — a promising, openly released claim-extraction infrastructure whose headline associations rest on edges never validated against human labels and on regressions with year fixed effects only. An edge-level human-annotated audit, and outcome regressions with field and method controls on a common sample, would settle it.
When seeded, before the reading in NOTE-083, the entry gave a priority for a deeper reading: low — 45 s over 5 sessions is a bounce (“drop for now” triage). The skim covers the method and headline numbers, and the only thing to revisit is the validation design, if LLM-based literature extraction becomes a topic.
What the seeding asked a deeper reading to check (NOTE-083 is that reading):
- A concrete example of using LLMs for large-scale structured extraction from scientific literature, the same pattern as this record’s own reading notes and relation graphs. The validation design (repeated passes, edge-overlap thresholds) is the transferable part.
- Validation is mostly self-consistency, and external checks cover under 2% of papers. Check how precision and recall change across EO thresholds.
- The associations with publication and citations are correlational, so check the controls.
Access when seeded: arXiv abs page (v1 12 Jan 2025; v2 24 Feb 2026) and the full v2 PDF (71 pp.), which I read: abstract, introduction and positioning, method section heads, the validation section (§2.4) and conclusion. Crossref shows earlier same-titled versions by the same authors: OSF preprint 10.31219/osf.io/u4vgs (posted 2024-11-15) and SSRN 10.2139/ssrn.5045487 / 10.2139/ssrn.5009061 (created Dec 2024). The published date is the OSF posting. doi is set to none because those DOIs belong to preprint servers, and arXiv is the preferred identifier. The v1 abstract was not compared with v2, which reads as a reframing (“evidence-annotated claim graphs”).