Title: OUT-OF-THE-LOOP PILOTS: Study of an applied phenomenon through performance-monitoring EEG measures.
Abstract: Event Abstract Back to Event OUT-OF-THE-LOOP PILOTS: Study of an applied phenomenon through performance-monitoring EEG measures. Bertille Somon1, 2*, Aurélie CAMPAGNE2, Arnaud Delorme3, 4 and Bruno Berberian1 1 Office National d'Études et de Recherches Aérospatiales, Salon-de-Provence, Information Processing and Systems Department, France 2 UMR5105 Laboratoire de Psychologie et NeuroCognition (LPNC), France 3 University of California, San Diego, Swartz Center for Computational Neuroscience, United States 4 UMR5549 Centre de Recherche Cerveau et Cognition (CerCo), France Automation has increasingly invaded our daily-life in the past decades and even more radical changes are anticipated in the forthcoming years. Due to the modifications it has provoked in the way we perform tasks and activities in interaction with highly automated systems, new problematics have emerged. One of them is the Out-Of-The-(control-)Loop performance problem (OOTL; Endsley & Kiris, 1995). This phenomenon is vastly studied in aeronautics and has been well described in terms of effects and consequences. One of its operational manifestations is a difficulty from the operator to monitor correctly highly automated and highly reliable systems, and to detect system errors. The brain’s error detection system, called performance monitoring, is a well-studied mechanism in Cognitive Neuroscience. Its neural correlates are relatively well-known in terms of electrophysiology and neuroanatomy. Thus, we propose to use theories and knowledges from the cognitive neuroscience literature (Gehring, Liu, Orr, & Carp, 2011; Weller, Schwarz, Kunde, & Pfister, 2018), to better understand, characterize, and compensate this applied problem (Somon, Campagne, Delorme, & Berberian, 2017). In order to measure the neural correlates of performance monitoring during human-machine interactions, we had to face several challenges. Most neuroimaging studies have evaluated self-performance monitoring (i.e., when we are the one performing the error) and have used standardized laboratory tasks (i.e., in a very controlled environment with simple repetitive button-press tasks). Hence three experiments were designed to progress on our understanding of such activity in a supervisory context. A first experiment aimed at exploring whether the performance monitoring activity is present during supervisory conditions, and, if so, whether it is impacted by task difficulty. Participants had to supervise another agent performing a laboratory task (i.e., a vertically oriented arrowhead version of the Eriksen Flanker task) with two levels of difficulty (easy and difficult). Interestingly, the agent could be either another human being, or an automated system. We recorded the electroencephalographic (EEG) signature of the detection of other’s errors on 17 participants (12 men; 27.5 years ±4.78 years) with a 72-active-electrodes actiCAP (Brain Products GmbH). Then data were analyzed (i) using a classical statistical analysis of variance on the mean amplitude of the event-related-potentials (ERPs) related to performance monitoring at locations defined in literature (i.e., FCz, CPz; Weller et al., 2018) and (ii) using the non-parametric cluster-based permutation test (Maris & Oostenveld, 2007). This latter innovative technique in the context of performance monitoring allowed detecting ERPs without preconceived notions about their temporo-spatial characteristics. It also highlighted a significant effect of both the type of agent being supervised and the level of difficulty on the error-related activity. Data analysis revealed a P2-N2-P3 complex after error detection only. This complex was significantly decreased with increasing level of difficulty, but also when supervising an automated system compared to a human agent. Based on Human-Machine Interactions studies, the latter can be explained by a higher disengagement of the participant during the task when interacting with an automated system. In a second experiment, we addressed the issue of the ecological setting of the supervision, in order to move towards a neuroergonomic study of the OOTL. Consequently, we measured system performance monitoring, and system error detection, in a more applied context. We recorded the brain activity of 20 participants (13 men, 27.75 years ±1.42 years) with a 65-active-electrodes actiCAP (Brain Products GmbH), during supervision of an air traffic collision avoidance simulator. Given the dynamic aspect of this task, a more appropriate data analysis than the classical analysis of variance of the ERP amplitudes was performed: a trial-by-trial time-frequency analysis. This technique decomposes, at the trial level, the spectral brain activity across time. Based on this measure, we identified specific supervision activity related to error detection in more ecological settings. A significant suppression of alpha activity (8-13Hz) was observed in the time-window extending from 150 to 400ms following an erroneous response of the simulator. However, no effect of the task difficulty was showed on supervision brain activity. A coming third experiment, based on all previous results intends to assess how the error detection process during ecological system supervision can be degraded by the OOTL performance problem. In this study, the OOTL will be induced by manipulating complacency and trust towards the automated system. We will record EEG activity for both in- and out-of-the-loop conditions in order to compare them. Several analyses are envisioned: (i) a cluster-based permutation test performed on trial-by-trial time-frequency data, and (ii) a more global analysis of the spectral brain activity at the scale of the experiment. These analyses will allow the exploration of both error- and correct-related activity and the assessment of the impact of the OOTL phenomenon on performance monitoring processes across time. In conclusion, it appears that the cerebral correlates of the performance monitoring function obtained from laboratory studies cannot be transposed easily to real-life settings. Consequently, the development of new ways to measure and analyze this process in an ecological context is required. To this purpose, the general aim of our research is to move slowly and step by step towards the neuroergonomic study of everyday life situations. Recent developments of lighter and more wearable devices for measuring brain activity can allow the implementation of this kind of study. Likewise, the use of pioneering signal processing and data analysis techniques, which are less constraining, allows the analysis of various data without a priori on the latency, localization or frequency band of the data and is therefore particularly suitable in dynamic task contexts. References Endsley, M. R., & Kiris, E. O. (1995). The Out-of-the-Loop Performance Problem and Level of Control in Automation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(2), 381–394. https://doi.org/10.1518/001872095779064555 Gehring, W. J., Liu, Y., Orr, J. M., & Carp, J. (2011). The Error-Related Negativity (ERN/Ne). https://doi.org/10.1093/oxfordhb/9780195374148.013.0120 Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods, 164(1), 177–190. https://doi.org/10.1016/j.jneumeth.2007.03.024 Somon, B., Campagne, A., Delorme, A., & Berberian, B. (2017). Performance Monitoring Applied to System Supervision. Frontiers in Human Neuroscience, 11. https://doi.org/10.3389/fnhum.2017.00360 Weller, L., Schwarz, K. A., Kunde, W., & Pfister, R. (2018). My mistake? Enhanced error processing for commanded compared to passively observed actions. Psychophysiology, e13057. https://doi.org/10.1111/psyp.13057 Keywords: Out-of-the-loop, Electroencephalography, Performance monitoring, system supervision, N2-P3 complex, cluster-based permutation tests, Time-frequency analysis (TFA), Ecological Validity Conference: 2nd International Neuroergonomics Conference, Philadelphia, PA, United States, 27 Jun - 29 Jun, 2018. Presentation Type: Oral Presentation Topic: Neuroergonomics Citation: Somon B, CAMPAGNE A, Delorme A and Berberian B (2019). OUT-OF-THE-LOOP PILOTS: Study of an applied phenomenon through performance-monitoring EEG measures.. Conference Abstract: 2nd International Neuroergonomics Conference. doi: 10.3389/conf.fnhum.2018.227.00038 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 06 Mar 2018; Published Online: 27 Sep 2019. * Correspondence: Miss. Bertille Somon, Office National d'Études et de Recherches Aérospatiales, Salon-de-Provence, Information Processing and Systems Department, Salon-de-Provence, 13661, France, [email protected] Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Bertille Somon Aurélie CAMPAGNE Arnaud Delorme Bruno Berberian Google Bertille Somon Aurélie CAMPAGNE Arnaud Delorme Bruno Berberian Google Scholar Bertille Somon Aurélie CAMPAGNE Arnaud Delorme Bruno Berberian PubMed Bertille Somon Aurélie CAMPAGNE Arnaud Delorme Bruno Berberian Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.