“Shoulda, Coulda, Woulda”: Conceptualizing the Differences in Trust Between Human-Human Teaming and Human-Machine Teaming
Intelligent decision support systems (IDSSs) are machine teammates designed to facilitate better human decision-making in high-consequence domains such as health care, power grid operations, and fraud detection. IDSSs identify patterns in datasets and provide intelligent decision-making recommendations to human teammates. However, previous research indicates that humans often trust IDSS recommendations less than the recommendations from their human teammates, even when the machine teammate is more accurate. To conceptualize why trust differs, we review the literature surrounding trust, error, and predictability. Then, we compile and compare participant trust ratings and decision-making in an abridged systematic review of previous studies manipulating teammate type, error rate, and error type. Finally, we conduct a content analysis of participants’ qualitative responses to trust queries from a survey on generative language models. Results suggest that humans may trust IDSS teammates less than other human teammates because of differences in (1) interaction complexity, (2) blame attribution, and (3) swift trust. We conclude that human factors practitioners should collaborate with data scientists and domain experts to build and maintain trust in IDSSs by anthropomorphizing algorithms, matching mental models, and considering individual differences.
- Research Organization:
- Pacific Northwest National Laboratory (PNNL), Richland, WA (United States)
- Sponsoring Organization:
- USDOE Office of Science (SC), Office of Workforce Development for Teachers & Scientists (WDTS)
- DOE Contract Number:
- AC05-76RL01830
- OSTI ID:
- 1999810
- Report Number(s):
- PNNL-34522
- Country of Publication:
- United States
- Language:
- English
Similar Records
Human-Centric Contingency Analysis Metrics for Evaluating Operator Performance and Trust
Explainable Artificial Intelligence Technology for Predictive Maintenance