Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation

Lebiere, Christian; Blaha, Leslie M.; Fallon, Corey K.; Jefferson, Brett

doi:10.3389/frobt.2021.652776

Title: Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation

Journal Article · Mon May 24 00:00:00 EDT 2021 · Frontiers in Robotics and AI

DOI:https://doi.org/10.3389/frobt.2021.652776· OSTI ID:1813592

Lebiere, Christian ^[1]; Blaha, Leslie M. ^[2]; Fallon, Corey K. ^[3];

^[3]

Carnegie Mellon Univ., Pittsburgh, PA (United States)
Air Force Research Lab., Pittsburgh, PA (United States). 711th Human Performance Wing
Pacific Northwest National Lab. (PNNL), Richland, WA (United States)

Trust calibration for a human–machine team is the process by which a human adjusts their expectations of the automation’s reliability and trustworthiness; adaptive support for trust calibration is needed to engender appropriate reliance on automation. Herein, we leverage an instance-based learning ACT-R cognitive model of decisions to obtain and rely on an automated assistant for visual search in a UAV interface. This cognitive model matches well with the human predictive power statistics measuring reliance decisions; we obtain from the model an internal estimate of automation reliability that mirrors human subjective ratings. The model is able to predict the effect of various potential disruptions, such as environmental changes or particular classes of adversarial intrusions on human trust in automation. Finally, we consider the use of model predictions to improve automation transparency that account for human cognitive biases in order to optimize the bidirectional interaction between human and machine through supporting trust calibration. The implications of our findings for the design of reliable and trustworthy automation are discussed.

View Accepted Manuscript (DOE)

Cite

Export

Save

Research Organization:: Pacific Northwest National Lab. (PNNL), Richland, WA (United States)

Sponsoring Organization:: USDOE; US Air Force Office of Scientific Research (AFOSR)

Grant/Contract Number:: AC05-76RL01830; FA9550-18-1-025

OSTI ID:: 1813592

Report Number(s):: PNNL-SA-159195

Journal Information:: Frontiers in Robotics and AI, Vol. 8; ISSN 2296-9144

Publisher:: Frontiers Media S.A.Copyright Statement

Country of Publication:: United States

Language:: English

References (18)

Automation Bias, Accountability, and Verification Behaviors Mosier, Kathleen L.; Skitka, Linda J.; Burdick, Mark D. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 40, Issue 4 https://doi.org/10.1177/154193129604000413	journal	October 1996
Follower reactions to leader trust violations: A grounded theory of violation types, likelihood of recovery, and recovery process Grover, Steven L.; Hasel, Markus C.; Manville, Caroline European Management Journal, Vol. 32, Issue 5 https://doi.org/10.1016/j.emj.2014.01.002	journal	October 2014
A Behavioral Decision Theory Approach to Modeling Dyadic Trust in Superiors and Subordinates Butler, John K.; Cantrell, R. Stephen Psychological Reports, Vol. 55, Issue 1 https://doi.org/10.2466/pr0.1984.55.1.19	journal	August 1984
Anesthesia Alarms in Context: An Observational Study Seagull, F. Jacob; Sanderson, Penelope M. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 43, Issue 1 https://doi.org/10.1518/001872001775992453	journal	March 2001
A Functional Model of Sensemaking in a Neurocognitive Architecture Lebiere, Christian; Pirolli, Peter; Thomson, Robert Computational Intelligence and Neuroscience, Vol. 2013 https://doi.org/10.1155/2013/921695	journal	January 2013
Understanding dynamics between initial trust and usage intentions of mobile banking Kim, Gimun; Shin, BongSik; Lee, Ho Geun Information Systems Journal, Vol. 19, Issue 3 https://doi.org/10.1111/j.1365-2575.2007.00269.x	journal	May 2009
An Integrative Model Of Organizational Trust Mayer, Roger C.; Davis, James H.; Schoorman, F. David Academy of Management Review, Vol. 20, Issue 3 https://doi.org/10.5465/amr.1995.9508080335	journal	July 1995
A Capacity Coefficient Method for Characterizing the Impacts of Automation Transparency on Workload Efficiency Fallon, Corey K.; Blaha, Leslie M.; Jefferson, Brett Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 63, Issue 1 https://doi.org/10.1177/1071181319631436	journal	November 2019
No Effect of Cue Format on Automation Dependence in an Aided Signal Detection Task Bartlett, Megan L.; McCarley, Jason S. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 61, Issue 2 https://doi.org/10.1177/0018720818802961	journal	October 2018
Factors affecting the adoption of e-learning systems in Qatar and USA: Extending the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) El-Masri, Mazen; Tarhini, Ali Educational Technology Research and Development, Vol. 65, Issue 3 https://doi.org/10.1007/s11423-016-9508-8	journal	January 2017
System reliability, performance and trust in adaptable automation Chavaillaz, Alain; Wastell, David; Sauer, Jürgen Applied Ergonomics, Vol. 52 https://doi.org/10.1016/j.apergo.2015.07.012	journal	January 2016
Stress states, personality and cognitive functioning: A review of research with the Dundee Stress State Questionnaire Matthews, Gerald Personality and Individual Differences, Vol. 169 https://doi.org/10.1016/j.paid.2020.110083	journal	February 2021
Toward Personalized Deceptive Signaling for Cyber Defense Using Cognitive Models Cranford, Edward A.; Gonzalez, Cleotilde; Aggarwal, Palvi Topics in Cognitive Science, Vol. 12, Issue 3 https://doi.org/10.1111/tops.12513	journal	July 2020
Instance-based learning: Integrating sampling and repeated decisions from experience. Gonzalez, Cleotilde; Dutt, Varun Psychological Review, Vol. 118, Issue 4 https://doi.org/10.1037/a0024558	journal	January 2011
What did you expect? CREAM analysis of hazardous incidents occurring on approach to rail signals Phillips, Ross Owen; Sagberg, Fridulv Safety Science, Vol. 66 https://doi.org/10.1016/j.ssci.2014.02.003	journal	July 2014
Complacency and Automation Bias in the Use of Imperfect Automation Wickens, Christopher D.; Clegg, Benjamin A.; Vieane, Alex Z. Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 57, Issue 5 https://doi.org/10.1177/0018720815581940	journal	April 2015
Instance-based learning in dynamic decision making Gonzalez, Cleotilde; Lerch, Javier F.; Lebiere, Christian Cognitive Science, Vol. 27, Issue 4 https://doi.org/10.1207/s15516709cog2704_2	journal	July 2003
Adaptive Cyber Deception: Cognitively Informed Signaling for Cyber Defense Cranford, Edward; Gonzalez, Cleotilde; Aggarwal, Palvi Proceedings of the Annual Hawaii International Conference on System Sciences https://doi.org/10.24251/HICSS.2020.232	conference	January 2020

Similar Records

Trust in automation and meta-cognitive accuracy in NPP operating crews

Conference · Sat Jul 01 00:00:00 EDT 2006 · OSTI ID:1813592

Skraaning, Jr, G; Miberg Skjerve, A B

Cognitive Model of Trust Dynamics Predicts Human Behavior within and between Two Games of Strategic Interaction with Computerized Confederate Agents

Journal Article · Fri Feb 12 00:00:00 EST 2016 · Frontiers in Psychology · OSTI ID:1813592

Collins, Michael G.; Juvina, Ion; Gluck, Kevin A.

Toward designing for trust in database automation

Conference · Sat Jul 01 00:00:00 EDT 2006 · OSTI ID:1813592

Duez, P P; Jamieson, G A

Related Subjects

60 APPLIED LIFE SCIENCES
Cognitive architectures
ACT-R
Trust in automation
Automation transparency
Trust calibration
Human-machine teaming

Title: Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation

Citation Formats

References (18)

Similar Records

Related Subjects