AAAI 2025 Tutorial:
Hallucinations in Large Multimodal Models
About this tutorial
Large Language Models (LLMs) have made significant strides in generating human-like text, but their tendency to hallucinate—producing factually incorrect or fabricated information—remains a pressing issue. This tutorial provides a comprehensive exploration of hallucinations in LLMs, introducing participants to the key concepts and challenges in this domain. We will cover the types of hallucination, including Factual Mirage and Silver Lining, and present the latest approaches for benchmarking, detection, and mitigation. The motivation for understanding hallucination is particularly critical from a multimodal standpoint, as Vision-Language Models (VLMs) can exacerbate the problem by blending hallucinated text with misleading images or video. The tutorial will offer practical techniques to reduce hallucinations using both black-box and gray-box methods. Designed for researchers and professionals in generative AI, this tutorial bridges the gap between emerging research and practical solutions, providing attendees with valuable insights and tools to enhance the factual accuracy of LLM outputs. Participants will gain a deeper understanding of the complexities surrounding LLM hallucination and be equipped with strategies to drive future innovations in the field.
Schedule
Our tutorial will be held on February 26. Slides are available here.
| Time | Section | Presenter |
|---|---|---|
| 08:30—08:45 | Section 1: Introduction to hallucination in LLMs | Amitava |
| 08:45—09:45 | Section 2: Hallucination Detection/Mitigationn | Aman/Vipula |
| 09:45—10:00 | Section 3: Open Challenges | Amitava |
| 10:00—10:15 | Q & A Session |
Reading List
- A Survey of Hallucination in Large Foundation Models (Rawte et al., 2023)
- Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models (Shi et al., 2023)
- Cognitive Mirage: A Review of Hallucinations in Large Language Models (Ye et al., 2023)
- Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization (Cao et al., 2023)
- The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (Rawte et al., 2023)
- CHAIN-OF-VERIFICATION REDUCES HALLUCINATION IN LARGE LANGUAGE MODELS (Dhuliawala et al., 2023)
- Halo: Estimation and reduction of hallucinations in open-source weak large language models (Elaraby et al., 2023)
- When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization (Ladhak et al., 2023)
- HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (Li et al., 2023)
- A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation (Varshney et al., 2023)
- How Language Model Hallucinations Can Snowball (Zhang et al., 2023)
- SELFCHECKGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models (Manakaul et al., 2023)
- SELF-CONTRADICTORY HALLUCINATIONS OF LLMS: EVALUATION, DETECTION AND MITIGATION (Mündler et al., 2023)
BibTeX
@article{ hallucination-llm-tutorial,
author = { Rawte, Vipula and Chadha, Aman and Sheth, Amit and Das, Amitava },
title = { AAAI 2025 Tutorial: Hallucination in Large Multimodal Models },
journal = { AAAI 2025 },
year = { 2025 },
}