Hybrid Fact-Checking that Integrates Knowledge Graphs, Large Language Models, and Search-Based Retrieval Agents Improves Interpretable Claim Verification
2511.03217v1
cs.CL, cs.AI, cs.CY, cs.IR, 68T50, I.2.7; H.3.3
2025-11-07
Авторы:
Shaghayegh Kolli, Richard Rosenbaum, Timo Cavelius, Lasse Strothe, Andrii Lata, Jana Diesner
Abstract
Large language models (LLMs) excel in generating fluent utterances but can
lack reliable grounding in verified information. At the same time,
knowledge-graph-based fact-checkers deliver precise and interpretable evidence,
yet suffer from limited coverage or latency. By integrating LLMs with knowledge
graphs and real-time search agents, we introduce a hybrid fact-checking
approach that leverages the individual strengths of each component. Our system
comprises three autonomous steps: 1) a Knowledge Graph (KG) Retrieval for rapid
one-hop lookups in DBpedia, 2) an LM-based classification guided by a
task-specific labeling prompt, producing outputs with internal rule-based
logic, and 3) a Web Search Agent invoked only when KG coverage is insufficient.
Our pipeline achieves an F1 score of 0.93 on the FEVER benchmark on the
Supported/Refuted split without task-specific fine-tuning. To address Not
enough information cases, we conduct a targeted reannotation study showing that
our approach frequently uncovers valid evidence for claims originally labeled
as Not Enough Information (NEI), as confirmed by both expert annotators and LLM
reviewers. With this paper, we present a modular, opensource fact-checking
pipeline with fallback strategies and generalization across datasets.