Adaptive Planning: Comparing Human and AI Responses in Premortem Planning

Document Type

Conference Proceeding

Publication Date

12-31-2024

Department

Department of Psychology and Human Factors

Abstract

Premortems are a structured analytic technique designed to evaluate a plan before it is implemented. They have been used to improve estimates and generate alternative predicted outcomes. One psychological mechanism that underlies the Premortem is prospective hindsight with people generating reasons for a plan’s failure. The reasons reduce people’s confidence and overconfidence in their plans relative to other plan critiquing methods. In this case study, we qualitatively compare student-generated responses for evaluating a plan and a large language model-generated (e.g., Chat GPT-4) evaluation of the same plan. To the extent that the reasons for failure generated by the large language model are similar or different to the human participants is the first step in a conceptual evaluation. The results have implications for a variety of structured analytic techniques to improve human-AI teaming in decision making. Furthermore, they provide initial suggestions for where adaptive systems could support collaborative intelligence.

Publication Title

HCI International 2024 – Late Breaking Papers

Link to Full Text

Share

COinS