Skip to main content
OSTI.GOVU.S. Department of Energy Office of Scientific and Technical Information
U.S. Department of Energy
Office of Scientific and Technical Information
 

Mitigative Strategies for Recovering From Large Language Model Trust Violations

Journal Article · · Journal of Cognitive Engineering and Decision Making
 [1];  [1];  [1]
  1. Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
+ Show Author Affiliations
In this study, we investigated strategies to address trust issues arising from errors in large language models (LLMs). The study examined the impact of confidence scores, system capability explanations, and user feedback on trust restoration post-error. 68 participants viewed the responses of an LLM to 20 general trivia questions, with an error introduced on the third trial. Each participant was presented with one mitigation strategy. Participants rated their overall trust in the model and the reliability of the answer. Results showed an immediate drop in trust after the error; however, there were no differences across the three strategies in trust recovery. Further, all conditions had a logarithmic trend in trust recovery following error. Differences in overall trust were predicted by perceived reliability of the answer, suggesting that participants were evaluating results critically and using that to inform their trust in the model. Qualitative data supported this finding; participants expressed lasting distrust despite the LLM’s later accuracy. Results showcase the need to prioritize accuracy in LLM deployment, because early errors may irrevocably damage user trust calibration and later adoption.
Research Organization:
Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
Sponsoring Organization:
USDOE; USDOE Laboratory Directed Research and Development (LDRD) Program; USDOE Office of Science (SC), Office of Workforce Development for Teachers & Scientists (WDTS)
Grant/Contract Number:
AC05-76RL01830
OSTI ID:
2479660
Alternate ID(s):
OSTI ID: 2560812
Report Number(s):
PNNL-SA--193711
Journal Information:
Journal of Cognitive Engineering and Decision Making, Journal Name: Journal of Cognitive Engineering and Decision Making Journal Issue: 1 Vol. 19; ISSN 1555-3434
Publisher:
Sage PublicationsCopyright Statement
Country of Publication:
United States
Language:
English

References (48)

A systematic review of algorithm aversion in augmented decision making journal October 2019
Trust, self-confidence, and operators' adaptation to automation journal January 1994
Timing Is Key for Robot Trust Repair conference January 2015
Explainable AI: The New 42? book January 2018
Examining Science Education in ChatGPT: An Exploratory Study of Generative Artificial Intelligence journal March 2023
Machine learning and deep learning journal April 2021
The role of trust in automation reliance journal June 2003
Evaluating XAI: A comparison of rule-based and example-based explanations journal February 2021
Explaining black-box classifiers using post-hoc explanations-by-example: The effect of explanations and error-rates in XAI user studies journal May 2021
Artificial intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry journal January 2021
The effects of personality and locus of control on trust in humans versus artificial intelligence journal August 2020
How transparency modulates trust in artificial intelligence journal April 2022
What influences algorithmic decision-making? A systematic literature review on algorithm aversion journal February 2022
How should intelligent agents apologize to restore trust? Interaction effects between anthropomorphism and apology attribution on trust repair journal August 2021
Trust in deliberation: The consequences of deliberative decision strategies for medical decisions. journal November 2015
Algorithm aversion: People erroneously avoid algorithms after seeing them err. journal January 2015
Large language models in medicine journal July 2023
From ‘automation’ to ‘autonomy’: the importance of trust repair in human–machine interaction journal April 2018
Trust, control strategies and allocation of function in human-machine systems journal October 1992
Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation journal March 1996
Measuring trust inside organisations journal September 2006
Stubborn Reliance on Human Nature in Employee Selection: Statistical Decision Aids Are Evolutionarily Novel journal September 2008
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
  • Ribeiro, Marco Tulio; Singh, Sameer; Guestrin, Carlos
  • Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD '16 https://doi.org/10.1145/2939672.2939778
conference January 2016
Understanding the Effect of Accuracy on Trust in Machine Learning Models conference May 2019
Do I trust my machine teammate? conference March 2019
Interpreting Interpretability: Understanding Data Scientists' Use of Interpretability Tools for Machine Learning conference April 2020
The relationship between trust in AI and trustworthy machine learning technologies conference January 2020
Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making conference January 2020
How do visual explanations foster end users' appropriate trust in machine learning? conference March 2020
To Trust or to Think
  • Buçinca, Zana; Malaya, Maja Barbara; Gajos, Krzysztof Z.
  • Proceedings of the ACM on Human-Computer Interaction, Vol. 5, Issue CSCW1 https://doi.org/10.1145/3449287
journal April 2021
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions journal November 2024
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust journal September 2014
Individual Differences in the Calibration of Trust in Automation journal December 2014
Measuring Individual Differences in the Perfect Automation Schema journal April 2015
Trusting Automation: Designing for Responsivity and Resilience journal April 2021
Overcoming Algorithm Aversion: People Will Use Imperfect Algorithms If They Can (Even Slightly) Modify Them journal March 2018
Overtrusting robots: Setting a research agenda to mitigate overtrust in automation journal October 2021
Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information journal December 2006
Trust in Automation: Designing for Appropriate Reliance journal January 2004
Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance journal October 2019
An Integrative Model of Organizational Trust journal July 1995
Generative AI in the Workplace: Employee Perspectives of ChatGPT Benefits and Organizational Policies preprint March 2023
Measurement of Trust in Automation: A Narrative Review and Reference Guide journal October 2021
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health journal April 2023
Repairing and Enhancing Trust:Approaches to Reducing Organizational Trust Deficits journal January 2010
The Role Of Causal Attribution Dimensions In Trust Repair journal January 2009
The Repair of Trust: A Dynamic Bilateral Perspective and Multilevel Conceptualization journal July 2009
Human Trust in Artificial Intelligence: Review of Empirical Research journal July 2020

Similar Records

Role of Uncertainty Quantification in the Explainability of Large Language Models for the Nuclear Industry
Conference · Tue Jun 17 20:00:00 EDT 2025 · OSTI ID:3028731
Trust and Public Participation in Risk Policy Issues
Conference · Tue Nov 30 23:00:00 EST 1999 · OSTI ID:15004140

Related Subjects

97 MATHEMATICS AND COMPUTING