Automation Failures on Tasks Easily Performed by Operators Undermine Trust in Automated Aids
- 1 June 2006
- journal article
- research article
- Published by SAGE Publications in Human Factors: The Journal of the Human Factors and Ergonomics Society
- Vol. 48 (2) , 241-256
- https://doi.org/10.1518/001872006777724408
Abstract
Objective: We tested the hypothesis that automation errors on tasks easily performed by humans undermine trust in automation. Background: Research has revealed that the reliability of imperfect automation is frequently misperceived. We examined the manner in which the easiness and type of imperfect automation errors affect trust and dependence. Method: Participants performed a target detection task utilizing an automated aid. In Study 1, the aid missed targets either on easy trials (easy miss group) or on difficult trials (difficult miss group). In Study 2, we manipulated both easiness and type of error (miss vs. false alarm). The aid erred on either difficult trials alone (difficult errors group) or on difficult and easy trials (easy miss group; easy false alarm group). Results: In both experiments, easy errors led to participants mistrusting and disagreeing more with the aid on difficult trials, as compared with those using aids that generated only difficult errors. This resulted in a downward shift in decision criterion for the former, leading to poorer overall performance. Misses and false alarms led to similar effects. Conclusion: Automation errors on tasks that appear “easy” to the operator severely degrade trust and reliance. Application: Potential applications include the implementation of system design solutions that circumvent the negative effects of easy automation errors.Keywords
This publication has 25 references indexed in Scilit:
- Trust in Automation: Designing for Appropriate RelianceHuman Factors: The Journal of the Human Factors and Ergonomics Society, 2004
- Why People Fail to Recognize Their Own IncompetenceCurrent Directions in Psychological Science, 2003
- Predicting Misuse and Disuse of Combat Identification SystemsMilitary Psychology, 2001
- The Effects of Correlation and Response Bias in Alerted Monitor DisplaysHuman Factors: The Journal of the Human Factors and Ergonomics Society, 1997
- Humans and Automation: Use, Misuse, Disuse, AbuseHuman Factors: The Journal of the Human Factors and Ergonomics Society, 1997
- Alarm effectiveness in driver-centred collision-warning systemsErgonomics, 1997
- Human probability matching behaviour in response to alarms of varying reliabilityErgonomics, 1995
- System operator response to warnings of danger: A laboratory investigation of the effects of the predictive value of a warning on human response time.Journal of Experimental Psychology: Applied, 1995
- Stalking the Elusive Schema: Insights from Developmental and Social-Psychological Analyses of Gender SchemasSocial Cognition, 1986
- Memory for actions in scripted activities as a function of typicality, retention interval, and retrieval taskMemory & Cognition, 1981