Guarding truthfulness: Detecting and correcting false information

Huang, Kung-Hsiang

Permalink

https://hdl.handle.net/2142/125546 Copy

Description

Title

Guarding truthfulness: Detecting and correcting false information

Author(s)

Huang, Kung-Hsiang

Issue Date

2024-06-28

Director of Research (if dissertation) or Advisor (if thesis)

Ji, Heng

Doctoral Committee Chair(s)

Ji, Heng

Committee Member(s)

• Zhai, ChengXiang
• McKeown, Kathleen
• Peng, Hao
• Zhao, Han
• Joty, Shafiq

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• fact-checking
• factual inconsistency
• fake news detection
• factual error correction

Abstract

The proliferation of false information in today’s digital age is becoming a global epidemic, leading to societal instability and discord. Large language models, such as ChatGPT, have unintentionally escalated the problem by occasionally generating misleading content, promoting harmful views, and eroding faith in public conversation. Based on these realities, this thesis sets out to establish effective strategies to counter and curtail the spread of false information. We approach the mitigation of false news from two distinct angles: detection and correction. Robustly detecting false information is a stepping stone towards combating false information and preventing the further spread of inaccurate content. We have contributed to the field by proposing two approaches that address two important challenges: the gap between machine-generated fake news and human-written fake news, and the scarcity of fact-checking annotated data in low-resource languages. First, we generate more realistic fake news by explicitly incorporating propaganda techniques and adopting self-critical sequence training to instruct the generator not to produce factual content. We show that detectors trained on our generated data are more effective in identifying human-written fake news. Second, for the challenge of data scarcity, we leverage cross-lingual retrieval techniques to identify relevant passages across languages by learning to retrieve from different languages via pseudo-feedback, thereby enabling cross-lingual fact-checking. Our method outperforms existing models and alleviates the data scarcity issues. Upon identifying false information, the intuitive next step is to automatically correct the information based on the supporting or refuting evidence retrieved. To this end, we devised an interpretable factual error correction framework, which remediates factual errors in a zero-shot manner. This framework, inspired by how humans correct errors, asks questions about the input claim, looks for answers in the evidence, and reconstructs a factual statement. By breaking this task into several sub-tasks that have much richer resources, we eliminate the need for training data in factual error correction. More importantly, the decomposability of our framework offers natural interpretability, as the questions and answers generated explicitly illuminate which parts of the original claim were incorrect and why. We demonstrate that our approach outperforms fully-supervised approaches that have been fine-tuned. Additionally, we extended the scope of factual error correction to visual data, focusing on chart understanding. Recognizing the inherent complexity of interpreting graphical information, we proposed a methodology for converting charts into tables for enhanced error correction. Furthermore, we introduced the Multi-Agent Debate Refinement (MADR) framework to tackle unfaithfulness in explanations generated by large language models for fact-checking. This iterative, multi-agent strategy significantly enhanced the faithfulness and reliability of these explanations.

Graduation Semester

2024-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/125546

Copyright and License Information

Copyright 2024 Kung-Hsiang Huang

Owning Collections