Document Type
Conference Proceeding
Publication Date
10-21-2025
Department
Department of Computer Science
Abstract
Empowered by large language models (LLMs), intelligent agents have become a popular paradigm for interacting with open environments to facilitate AI deployment. However, hallucinations generated by LLMs - where outputs are inconsistent with facts - pose a significant challenge, undermining the credibility of intelligent agents. Only if hallucinations can be mitigated, the intelligent agents can be used in real-world without any catastrophic risk. Therefore, effective detection and mitigation of hallucinations are crucial to ensure the dependability of agents. Unfortunately, the related approaches either depend on white-box access to LLMs or fail to accurately identify hallucinations. To address the challenge posed by hallucinations of intelligent agents, we present HalMit, a novel black-box watchdog framework that models the generalization bound of LLM-empowered agents and thus detect hallucinations without requiring internal knowledge of the LLM's architecture. Specifically, a probabilistic fractal sampling technique is proposed to generate a sufficient number of queries to trigger the incredible responses in parallel, efficiently identifying the generalization bound of the target agent. Experimental evaluations demonstrate that HalMit significantly outperforms existing approaches in hallucination monitoring. Its black-box nature and superior performance make HalMit a promising solution for enhancing the dependability of LLM-powered systems.
Publication Title
Frontiers in Artificial Intelligence and Applications
ISBN
9781643686318
Recommended Citation
Liu, S.,
Liu, W.,
Xu, Z.,
Wang, X.,
Chen, B.,
&
Li, T.
(2025).
Towards Mitigation of Hallucination for LLM-Empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor.
Frontiers in Artificial Intelligence and Applications,
413, 1019-1026.
http://doi.org/10.3233/FAIA250910
Retrieved from: https://digitalcommons.mtu.edu/michigantech-p2/2227
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Version
Publisher's PDF
Publisher's Statement
© 2025 The Authors. Publisher’s version of record: https://doi.org/10.3233/FAIA250910