Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction

Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with... applied sciences Review Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction 1 1 , 2 1 , 2 , Huashuo Liu , Fei Wang and Dan Zhang * Department of Psychology, School of Social Sciences, Tsinghua University, Beijing 100084, China Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China * Correspondence: dzhang@tsinghua.edu.cn Abstract: The increasingly central role of robotic agents in daily life requires effective human–robot interaction (HRI). For roboticists to optimize interaction design, it is crucial to understand the potential effects of robotic agents on human performance. Yet a systematic specification of contributing factors is lacking, and objective measures of HRI performance are still limited. In these regards, the findings of research on human–human interaction can provide valuable insights. In this review, we break down the complex effects of robotic agents on interacting humans into some basic building blocks based on human–human interaction findings, i.e., the potential effects of physical presence, motor actions, and task co-representation in HRI. For each effect, we advise on future directions regarding its implication. Furthermore, we propose that the neural correlates of these effects could support real- time evaluation and optimization of HRI with electroencephalograph (EEG)-based brain–computer interface (BCI). Keywords: human–robot interaction; human–human interaction; social facilitation; motor priming; joint action; brain–computer interface; electroencephalography 1. Introduction Citation: Liu, H.; Wang, F.; Zhang, D. As we embrace an era of human–robot interaction (HRI), robotic agents are becoming Inspiring Real-Time Evaluation and ubiquitous in our daily life, such as in education [1], rehabilitation [2], and elderly care [3]. Optimization of Human–Robot To achieve effective HRI, not only do we need to improve robot performance, but it is also Interaction with Psychological Findings from Human–Human crucial to understand the potential effects of robotic agents on human performance [4,5]. Ef- Interaction. Appl. Sci. 2023, 13, 676. forts have been made to explore how humans interact with robots in recent HRI research [6]. https://doi.org/10.3390/app13020676 Two of the primary issues in this field, however, are the lack of a systematic specification of critical factors contributing to effective interaction [7], as well as the limited objective Academic Editor: Ashley Colley measures of HRI performance [7,8]. To address these issues, it might be helpful to draw on Received: 19 November 2022 the established theories and findings from human–human interaction research, which has Revised: 17 December 2022 been intensively investigated for hundreds of years, as a considerable portion of them have Accepted: 3 January 2023 been proved to dominate aspects of HRI similarly [9]. In this review, we summarize the Published: 4 January 2023 psychological effects that a robotic agent interacting with a human might induce and, more importantly, introduce the approach of integrating electroencephalography (EEG)-based brain–computer interface (BCI) in HRI application, which can be promising for real-time measurement and optimization of these effects. Due to the limited practices carried out in Copyright: © 2023 by the authors. this area so far, we do not intend to conduct a systematic review on existing applications Licensee MDPI, Basel, Switzerland. along this line. Rather, we aim to introduce the related findings from human–human stud- This article is an open access article ies, discuss their future implication in HRI, and illustrate how to implement EEG-based distributed under the terms and BCI in different fields of HRI. More specifically, we first break down the complex psycho- conditions of the Creative Commons logical influence of interacting agents into some of their basic building blocks based on Attribution (CC BY) license (https:// classical human–human interaction findings, i.e., the potential effects of physical presence, creativecommons.org/licenses/by/ motor actions, and task co-representation. For each effect, we briefly introduce the classical 4.0/). Appl. Sci. 2023, 13, 676. https://doi.org/10.3390/app13020676 https://www.mdpi.com/journal/applsci Appl. Sci. 2023, 13, x FOR PEER REVIEW 2 of 12 Appl. Sci. 2023, 13, 676 2 of 12 their basic building blocks based on classical human–human interaction findings, i.e., the potential effects of physical presence, motor actions, and task co-representation. For each effect, we briefly introduce the classical findings in human–human studies (for compre- findings in human–human studies (for comprehensive information we refer the readers to hensive information we refer the readers to related systematic reviews) and give advice related systematic reviews) and give advice on future directions concerning its implication on future directions concerning its implication for HRI. Furthermore, we believe these for HRI. Furthermore, we believe these findings can not only guide HRI design but also be findings can not only guide HRI design but also be integrated into application systems to integrated into application systems to improve task performance more directly. Therefore, improve task performance more directly. Therefore, we propose that the neural correlates we propose that the neural correlates of these effects hold the potential to support real-time of these effects hold the potential to support real-time evaluation and optimization of HRI evaluation and optimization of HRI based on the BCI technique (see Figure 1). We then based on the BCI technique (see Figure 1). We then provide a general guideline for devel- provide a general guideline for developing these BCIs, which is further illustrated with oping these BCIs, which is further illustrated with examples from different fields of HRI. examples from different fields of HRI. Figure 1. Potential psychological effects of a robotic agent on an interacting human partner in Figure 1. Potential psychological effects of a robotic agent on an interacting human partner in hu- human-robot interaction (HRI) with brain-computer interface (BCI)-based real-time evaluation and man-robot interaction (HRI) with brain-computer interface (BCI)-based real-time evaluation and optimization. optimization. 2. The Effect of Physical Presence in Social Interaction 2. The Effect of Physical Presence in Social Interaction 2.1. Social Facilitation 2.1. Social Facilitation The effect of another individual’s physical presence on the performer in human– The effect of another individual’s physical presence on the performer in human–hu- human interaction, which is “the fundamental form of interindividual influence” ([10], man interaction, which is “the fundamental form of interindividual influence” ([10], p. p. 269), is mainly discussed in the frame of the social facilitation effect. The investigation of 269), is mainly discussed in the frame of the social facilitation effect. The investigation of social facilitation can be traced back to the early days of modern psychology, and the initial social facilitation can be traced back to the early days of modern psychology, and the ini- findings are somewhat contradictory: some studies [11–13] reveal improved performance, tial findings are somewhat contradictory: some studies [11–13] reveal improved perfor- while others [14–16] indicate an impaired one when working in the presence of others. mance, while others [14–16] indicate an impaired one when working in the presence of Zajonc’s influential paper [10] provides an explanation of the phenomenon based on the others. Zajonc’s influential paper [10] provides an explanation of the phenomenon based drive theory, which states that the presence of an audience might increase one’s arousal level on the drive theory, which states that the presence of an audience might increase one’s (drive), eliciting the dominant response, which is often correct in simple tasks but incorrect arousal level (drive), eliciting the dominant response, which is often correct in simple in complex tasks. Although various explanations (e.g., [17,18]; see [19] for a review) have tasks but incorrect in complex tasks. Although various explanations (e.g., [17,18]; see [19] been proposed to account for social facilitation, most theories relate its mechanism to the for a review) have been proposed to account for social facilitation, most theories relate its individuals’ arousal level [20]. mechanism to the individuals’ arousal level [20]. Subsequent studies further reveal that the social facilitation effect is modulated by Subsequent studies further reveal that the social facilitation effect is modulated by multiple factors, such as the type of task, as well as the characteristic of the audience. multiple factors, such as the type of task, as well as the characteristic of the audience. For For example, a familiar or practiced task showed improvement in the audience condition, example, a familiar or practiced task showed improvement in the audience condition, whereas learners of a task showed no improvement [21]. When the presence of another whereas learners of a task showed no improvement [21]. When the presence of another individual is felt to be reassuring, the level of arousal (and thus the social facilitation individual is felt to be reassuring, the level of arousal (and thus the social facilitation ef- effect) might decrease [10]. Moreover, expert observers would elicit larger facilitation than fect) might decrease [10]. Moreover, expert observers would elicit larger facilitation than peer observers [22]. With the numerous empirical studies on social facilitation in various peer observers [22]. With the numerous empirical studies on social facilitation in various domains, human performance could be improved with the intentional control and design of potential social facilitation based on the specific task. Appl. Sci. 2023, 13, 676 3 of 12 2.2. The Implication of Social Facilitation for HRI As one of the classic and well-established findings in social psychology, the social facilitation effect reported in human–human interaction has important consequences in many real-world social scenarios. Meanwhile, some HRI studies have examined the effect of robotic presence on human performance. While Schermerhorn et al. [23] and Riether et al. [24] successfully replicated the social facilitation effect with robotic presence, Irfan et al. [25] reported a number of failed attempts, and therefore call for caution when applying results from interpersonal psychology to HRI. As some of the investigations on social facilitation were conducted in the early years, their research methodologies might not fully meet the higher standards required by the scientific communities today. What’s more, unique issues exist regarding the effect of robotic presence on humans in HRI, which cannot be addressed by simply transferring previous human–human interaction findings, such as the anthropomorphic appearance of the robot and the enhanced sensing and memory abilities permitted by electronic sensors and storage, etc. Therefore, it is urgent to conduct state-of-the-art research on social facilitation by robotic presence tailored to the specific factors of concern in HRI. Still, the established paradigms for investigating social facilitation and the lessons on controlling confounding effects from social psychology could be an important reference. The potential effect of robotic presence, whether similar to findings from human–human interaction, should be taken into consideration when designing HRI, as improper robotic presence might be unnecessary or even counterproductive for task performance. 3. The Effect of Motor Actions in Social Interaction 3.1. Motor Priming While the presence of another individual has an impact on the observee, humans also actively encode the actions of others as observers. The most significant discovery regarding the effect of another agent’s motor actions on humans is the so-called motor resonance, i.e., the phenomenon that the perceiver ’s corresponding motor system is activated automat- ically during action perception [26–28]. Interestingly, action observation would influence action execution per se. Due to the motor resonance mechanism which activates the neural substrates of the observed action automatically, the execution of a similar action with the observed one is facilitated or ‘primed,’ which is called motor priming [29]. This priming effect has been found to increase with the similarity between observed and executed ac- tions [30]. However, if the required action to perform is incompatible with the observed one, the prepotent response to perform the primed action (i.e., the observed one) needs to be inhibited, resulting in the interference of action execution. Since the motor priming effect can be investigated empirically, it provides an objective measure of the magnitude of motor resonance. With its potential to alter motor behavior, the motor priming effect has been increas- ingly utilized in a number of real-world application scenarios. For example, some motor priming paradigms have been proved to successfully improve functional recovery after injury [31]. 3.2. The Implication of Motor Priming for HRI Given the increasing presence of embodied robots in our lives, the study of motor priming in HRI is particularly important in a variety of contexts, as priming can be advanta- geous if used properly, but can also be detrimental if not well-controlled. However, mixed findings have been reported regarding whether motor priming exists when actions are performed by robotic agents. While some studies did not find motor priming for robotic agents [32–34], more recent studies suggest that human actions can also be primed by robotic actions [35–39]. It is necessary to clarify these divergencies for a unifying theory on motor priming in HRI. We therefore suggest a more in-depth exploration of motor priming in HRI with strict control for related factors. Some possible accounts for the mixed findings regarding motor Appl. Sci. 2023, 13, 676 4 of 12 priming in HRI have been proposed. Chaminade and Cheng [40] hypothesize that motor resonance is primarily a perceptual (automatic) process in human–human interaction, but may be more susceptible to the attention of the observer in HRI. Therefore, in order to activate motor resonance in HRI, a clear instruction of paying attention to the relationship between the agents and the objects [41] is needed so that the robot’s movement is viewed as transitive goal-directed action. Langer and Levy-Tzedek [42] propose that the seemingly contradictory findings might be explained by different stimulus presentations, i.e., the embodiment of the robot, as well as how its actions are observed (e.g., through static or dynamic images). They assume that generally, actions performed by an embodied and physically present robot would more probably elicit motor resonance and priming in human subjects. A direct test of these hypotheses with well-controlled experiment conditions regarding (but not limited to) these factors is needed in future HRI research. 4. The Effect of Task Co-Representation in Social Interaction 4.1. Joint Action The effect of task co-representation, which occurs when a corresponding task repre- sentation is activated by the awareness of another ’s task [43], is primarily investigated in the context of joint action where multiple individuals coordinate to achieve a shared goal. These studies modified classical interference tasks such as the Simon task [44] or the Eriksen flanker task [45] to joint versions, in which the two individuals are responsible for one of the two responses each. A joint spatial compatibility effect similar to that in the original tasks has been observed [46–49]. This joint compatibility effect implies that individuals in joint action incorporate another person’s task into their own, thus representing their own and others’ actions in a functionally equivalent way. Task co-representation leads an actor to prepare for and predict the actions of the other actor, which starts even before the initiation of actual movements (i.e., in the action planning phase). With electrophysiological evidence for predictive shared motor planning, Kourtis et al. [50] show that people in joint action represent each other ’s action in advance in order to facilitate coordination. They further report similar motor activation during joint action planning with unimanual movements required for each actor and during individual action planning with bimanual movements [51], implying that participants in joint action engage in motor predictions concerning both their own and the co-actor ’s contributions. Joint actions are pervasive in everyday life. Extending from the effect of physical presence or separate motor actions of another agent, findings of task co-representation in joint actions offer valuable insights into the rich space for the cognitive processes that facilitate humans to collaborate with each other. 4.2. The Implication of Joint Action for HRI Equipped with multiple cognitive mechanisms to form task co-representation, humans are born experts in joint actions. Curioni, Knoblich and Sebanz [43] propose that joint action in humans could be a model for HRI to inspire the design of robotic agents with necessary interactive tools for collaboration with humans. Empirical HRI studies considering task co-representation in joint actions, however, are still limited. Huang et al. [52] identified the coordination methods in a human–human handover task and implemented these strategies in a human–robot condition. They found that if the robot could monitor their human partner and adapt their movement accordingly, the user experience and task performance could be improved. Curioni, Knoblich and Sebanz [43] commented that this result demonstrated the benefits of incorporating the awareness of the partner ’s task into the artificial agent’s planning in HRI, which not only improves efficiency but also fosters desirability of the interaction. On the one hand, we suggest that future HRI studies pay special attention to factors influencing the formation of task co-representation for human actors in human– robot joint actions. On the other hand, roboticists should consider combining specialized mechanisms to include their human partner ’s actions in robots’ own action planning. Appl. Sci. 2023, 13, 676 5 of 12 5. Real-Time Evaluation of HRI with BCI As mentioned above, the optimal level of these psychological effects in interaction often varies depending on specific HRI scenarios. Take the motor priming effect as an example. Motor priming can be used to facilitate the execution of desired movements for human actors. As a preliminary application of robotic motor priming, in a clinical trial [53], adding a robotic device to prime the required movements has been shown to facilitate rehabilitation compared to a task-oriented approach alone. However, sometime this priming effect can also be detrimental. For example, for the robotic “nurses” designed to hand surgical tools to a surgeon, if the velocity of their movement is too fast, the surgeon might be primed by this action, hence endangering the safety of the patient [42]. Therefore, it is of vital importance to measure these critical psychological effects in HRI in a real-time and quantitative way and find their optimal level. A promising technique that bears such potential is the brain–computer interface (BCI; [54]). Compared to traditional HRI evaluation methods such as questionnaires applied during or after the interaction, BCI can provide a non-disruptive and real-time evaluation of implicit cognitive processes throughout the interaction [55]. Specifically, due to its non-invasive and portable characteristics, electroencephalography (EEG)-based BCI has become the most popular form among other neural recording modalities [56]. Endeavors have been made using EEG-based BCI to evaluate interactive systems (e.g., [57]). In these applications, the measurement of human mental states such as workload, attention, and emotions has been shown to benefit a lot from the EEG technique [55]. We propose that the psychological effect of robotic agents on humans in HRI could also be quantified with BCI. 5.1. EEG Signatures of the Three Psychological Effects in HRI To achieve this goal, a critical step is to identify the EEG signatures of these effects, which social neuroscientific studies have shed light on. Starting with the neuroscientific findings related to social facilitation, on the one hand, as arousal is a key concept in most social facilitation theories, i.e., the presence of others could increase individuals’ arousal level thus facilitating their dominant response [20], an EEG approach which can provide direct information about arousal level [58,59] bears the potential to measure the extent of the social facilitation effect [60]. Indeed, EEG indicators of arousal such as alpha power, which is negatively linked to arousal level [61–64], and beta power, which is positively correlated with arousal [65,66], have already been used in studies on social facilitation [20]. On the other hand, neural correlates of some cognitive processes that may contribute to social facilitation, such as performance monitoring, have been intensively investigated with EEG. Specifically, EEG allows us to identify the event-related potentials (ERPs) instantly following specific events, making it ideal for exploring the fast-changing internal processes involved in social interaction. EEG studies on performance monitoring have paid particular attention to the error-related negativity (ERN or Ne; [67–69]), which is induced by an erroneous response. Generally, the ERN is believed to reflect how much one cares about the consequence of his action [70]. Enhanced ERNs are reported for errors committed in the presence of peers [71] or under social evaluation [72,73], proposed to arise from the greater threatening value of errors induced by concerns about being judged by others [74]. The neural substrates of motor resonance recognized by brain imaging studies [75–77] are called the mirror system. Multiple brain regions involved in motor resonance, such as the ventral premotor cortex and the pars opercularis of the posterior inferior frontal gyrus (Brodmann area 44), as well as the rostral inferior parietal lobule, are recognized by brain imaging studies [75–77]. In EEG research, the event-related synchronization and desynchronization of the mu (8–12 Hz) rhythm have been linked to the activity of the mirror system [78,79]. The power of the mu rhythm decreases when subjects perform or observe an action, known as mu suppression (see [80] for a review). In the context of motor priming, an EEG study [81] revealed a more significant mu suppression when the relevance of observed movement to the required action to perform is higher. Appl. Sci. 2023, 13, 676 6 of 12 The neural processes involved in joint action (mostly with joint interference paradigms) have also been investigated with EEG. The amplitude of the P3 component is larger for their own No-Go trials in a joint action setting than in an individual condition, suggesting an increased need for response inhibition [82,83]. Another ERP component related to the joint action task is the stimulus-locked N2 reflecting response conflict. Ruissen and Bruijn [84] reported that after nasal oxytocin administration, which has been reported to facilitate social behavior by enhancing self–other integration, participants exhibit larger N2 for Go trials in the joint Simon task compared to the individual setting. Peterburs et al. [85] investigated differences in the processing of social (i.e., indicating that a co-actor should respond) and nonsocial No-Go (i.e., signaling that neither actor nor co-actor should respond) cues in a joint flanker task. The conflict-sensitive N2 showed a compatibility effect only for social No-Go cues, and the P3b component related to response discrimination and evaluation was larger for social relative to non-social No-Go. We summarize the neural correlates related to the potential psychological effects in Table 1. Note that, however, one should be careful to generalize these neural signatures to tasks different from the ones reported in original studies, as many of them depend on task types, as well as the cognitive processes involved. Table 1. Psychological effects in social interaction and their potential EEG signatures. Psychological Cognitive Processes Involved Related Factor References Effects (And Their Potential EEG Signatures *) Arousal level [61–66] (alpha and beta power) Physical presence Social facilitation Performance monitoring [71–73] (error-related negativity, ERN) Sensorimotor mirroring Motor priming Motor action [80,81] (event-related mu suppression) Response conflict [84,85] (N2 component) Stimulus classification, response discrimination Task [50,82,83,85] Joint action (P3 component) co-representation Action planning (contingent negative variation, CNV; [50,51] movement-related potential, MRP) * Note that the EEG signatures of these psychological effects are contingent on specific tasks and cognitive processes involved in HRI. 5.2. Developing BCI Systems for HRI Evaluation and Optimization In order to evaluate these psychological effects based on human neural activity in HRI, a passive BCI (as opposed to an active BCI designed for controlling external devices with the user ’s brain signal) [86] could be utilized. To implement it in a specific HRI scenario, one should first specify the effect of interest in the given task. Although multiple psychological effects might play a role in the same task, in most cases, the target effect would be more focused. For example, researchers might be most interested in measuring the extent of the motor priming effect in a task where a robot and a human are both involved in intricate physical movements. Therefore, the specification of the effect of interest depends on the purpose and type of different HRI tasks and scenarios. Once the goal of the BCI system is determined, researchers could identify potential neural signatures associated with the target effect by means of a literature search on related neuroscience studies. Special attention should be paid to the paradigm reported in original studies, as the neural correlates of the desired effect could vary across different paradigms. Pilot studies with a similar setting to the real HRI application, if possible, are recommended to confirm the validity of the chosen neural signatures. Appl. Sci. 2023, 13, x FOR PEER REVIEW 7 of 12 psychological effects might play a role in the same task, in most cases, the target effect would be more focused. For example, researchers might be most interested in measuring the extent of the motor priming effect in a task where a robot and a human are both in- volved in intricate physical movements. Therefore, the specification of the effect of interest depends on the purpose and type of different HRI tasks and scenarios. Once the goal of the BCI system is determined, researchers could identify potential neural signatures associated with the target effect by means of a literature search on re- lated neuroscience studies. Special attention should be paid to the paradigm reported in original studies, as the neural correlates of the desired effect could vary across different paradigms. Pilot studies with a similar setting to the real HRI application, if possible, are recommended to confirm the validity of the chosen neural signatures. Appl. Sci. 2023, 13, 676 7 of 12 A typical BCI system consists of multiple component stages such as data acquisition, preprocessing, dimensionality reduction, feature extraction, classification, and applica- tion [87]. To construct a BCI based on the selected features, we recommend classical hand- A typical BCI system consists of multiple component stages such as data acquisition, books on BCI application (e.g., [88,89]) for technical details. The performance of the BCI preprocessing, dimensionality reduction, feature extraction, classification, and applica- prototype tion sh [87 oul ]. d To th constr en be uct evalua a BCI tbased ed in on simu thelated selected HRI featur scena es,rios we rby ecommend comparin classical g the BCI handbooks on BCI application (e.g., [88,89]) for technical details. The performance of the output to other validated measures of the target effect, as well as by evaluating the usa- BCI prototype should then be evaluated in simulated HRI scenarios by comparing the bility of the system [90]. If the system performance could not meet certain criterions, one BCI output to other validated measures of the target effect, as well as by evaluating the might search for other neural signatures or adjust the algorithm of the system modules. usability of the system [90]. If the system performance could not meet certain criterions, An iterative process is often needed in the development of BCI systems. one might search for other neural signatures or adjust the algorithm of the system modules. The implementation of the BCI in real HRI scenarios enables a real-time evaluation of the An iterative process is often needed in the development of BCI systems. The implementation of the BCI in real HRI scenarios enables a real-time evaluation effect of interest. The robotic agents could be further designed to adapt to the BCI output in a of the effect of interest. The robotic agents could be further designed to adapt to the BCI timely manner to optimize task performance and user experience. The general steps for de- output in a timely manner to optimize task performance and user experience. The general veloping BCIs for HRI evaluation and optimization are schematized in Figure 2. steps for developing BCIs for HRI evaluation and optimization are schematized in Figure 2. Figure 2. General steps for developing BCIs for HRI evaluation and optimization. Figure 2. General steps for developing BCIs for HRI evaluation and optimization. 5.3. Application of BCI in Different Fields of HRI To illustrate the application of BCI combined with the potential psychological effects in HRI, we next describe a few typical scenarios from different HRI fields for exemplary purposes. Appl. Sci. 2023, 13, 676 8 of 12 The increasing use of robotic assistance in medicine calls attention to their critical influences on humans, which might even endanger the life of patients under extreme circumstances. The extent of the motor priming effect of a robotic nurse’s movement on a human surgeon could be tracked with a BCI approach. A high degree of motor priming effect indicates that the surgeon might be seriously affected by the robotic movement [42], therefore the velocity of the robot’s movement could be set to an appropriate range, match- ing the habit of the human surgeon in this situation, rather than as fast as possible, in order to avoid detrimental consequences. Another field with extensive application of robotics is the manufacturing industry. Indeed, joint action for humans and industrial robots has not only been investigated in studies [91,92], but also become a reality in factories [93]. When an industrial robot and a human worker assembly capital goods together, it is important for both of them to form a unified representation of their shared task, which could be monitored be means of a BCI. If the BCI outcome indicates insufficient task co-representation for the human worker (e.g., weak motor planning EEG activities before the movement of the robot), the robot should be adjusted to display more predictable behaviors in line with the human partner ’s expectation, which has been proved to facilitate joint action performance in HRI [43]. Meanwhile, the social facilitation effect induced by robotic presence is ubiquitous in most fields of HRI, including the aforementioned scenarios. With a BCI to evaluate the magnitude of such effect in real time, the electronic monitoring function of the robot could be emphasized when beneficial to human performance and weakened when detrimental, resulting in improved productivity. 6. Conclusions In this article, we review human–human interaction studies investigating three pri- mary psychological effects which may also play an essential role in HRI, i.e., the potential effects of physical presence, motor actions, and task co-representation. For each effect, we discuss its implication for HRI and provide suggestions for future studies. Importantly, we propose that the neural signatures related to these effects could support real-time evalua- tion and optimization of HRI with EEG-based BCI. A general procedure for developing such BCIs is introduced, the implementation of which is further illustrated with examples from different fields of HRI. Therefore, we believe that psychological findings can not only serve as guidelines for HRI design, but also be integrated into application systems for a direct enhancement of HRI performance. Theoretical and technical challenges remain to be solved in future studies, such as the development of more portable EEG acquisition devices for better BCI user experience and the limited examination of the ecological validity of the neural signatures of the psychological effects in different HRI tasks. Author Contributions: Conceptualization, H.L., F.W. and D.Z.; investigation and visualization, H.L.; writing—original draft preparation, H.L.; writing—review and editing, F.W. and D.Z.; supervision and funding acquisition, F.W and D.Z. All authors have read and agreed to the published version of the manuscript. Funding: This work was funded by the National Natural Science Foundation of China, grant number 61977041, and the Tsinghua University Spring Breeze Fund, grant number 2021Z99CFY037. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: No new data were created or analyzed in this study. Data sharing is not applicable to this article. Conflicts of Interest: The authors declare no conflict of interest. Appl. Sci. 2023, 13, 676 9 of 12 References 1. Toh, L.P.E.; Causo, A.; Tzuo, P.-W.; Chen, I.M.; Yeo, S.H. A review on the use of robots in education and young children. J. Educ. Technol. Soc. 2016, 19, 148–163. 2. Li, B.; Li, G.; Sun, Y.; Jiang, G.; Kong, J.; Jiang, D. A review of rehabilitation robot. In Proceedings of the 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China, 19–21 May 2017; pp. 907–911. 3. Abdi, J.; Al-Hindawi, A.; Ng, T.; Vizcaychipi, M.P. Scoping review on the use of socially assistive robot technology in elderly care. BMJ Open 2018, 8, e018815. [CrossRef] 4. Licklider, J.C.R. Man-Computer Symbiosis. Ire. T. Hum. Fact. Electf. 1960, 1, 4–11. [CrossRef] 5. Henschel, A.; Hortensius, R.; Cross, E.S. Social Cognition in the Age of Human–Robot Interaction. Trends Neurosci. 2020, 43, 373–384. [CrossRef] 6. Bethel, C.L.; Murphy, R.R. Review of Human Studies Methods in HRI and Recommendations. Int. J. Soc. Robot. 2010, 2, 347–359. [CrossRef] 7. Wiese, E.; Metta, G.; Wykowska, A. Robots As Intentional Agents: Using Neuroscientific Methods to Make Robots Appear More Social. Front. Psychol. 2017, 8, 1663. [CrossRef] 8. Kompatsiari, K.; Pérez-Osorio, J.; Tommaso, D.D.; Metta, G.; Wykowska, A. Neuroscientifically-Grounded Research for Improved Human-Robot Interaction. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 3403–3408. 9. Krämer, N.C.; von der Pütten, A.; Eimler, S. Human-Agent and Human-Robot Interaction. In Human-Computer Interaction: The Agency Perspective; Zacarias, M., de Oliveira, J.V., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 215–240. 10. Zajonc, R.B. Social Facilitation. Science 1965, 149, 269–274. [CrossRef] 11. Triplett, N. The Dynamogenic Factors in Pacemaking and Competition. Am. J. Psychol. 1898, 9, 507–533. [CrossRef] 12. Meumann, E. Haus-und Schularbeit Experimente an Kindern der Volksschule; Kessinger Publishing: Whitefish, MT, USA, 1904. 13. Dashiell, J.F. An experimental analysis of some group effects. J. Abnorm. Soc. Psychol. 1930, 25, 190–199. [CrossRef] 14. Moore, H.T. Laboratory Tests of Anger, Fear and Sex Interest. Am. J. Psychol. 1917, 28, 390–395. [CrossRef] 15. Ekdahl, A.G. Effects of attitude on free word association time. Genet. Psychol. Monogr. 1929, 5, 253–338. 16. Burri, C. The influence of an audience upon recall. J. Educ. Psychol. 1931, 22, 683–690. [CrossRef] 17. Bond, C.F. Social facilitation: A self-presentational view. J. Personal. Soc. Psychol. 1982, 42, 1042–1050. [CrossRef] 18. Baron, R.S. Distraction-Conflict Theory: Progress and Problems. Adv. Exp. Soc. Psychol. 1986, 19, 1–40. [CrossRef] 19. Strauss, B. Social facilitation in motor tasks: A review of research and theory. Psychol. Sp. Exerc. 2002, 3, 237–256. [CrossRef] 20. Liu, N.; Yu, R.; Yang, L.; Lin, X. Gender composition mediates social facilitation effect in co-action condition. Sci. Rep. 2017, 7, 15073. [CrossRef] 21. Dube, S.K.; Tatz, S.J. Audience effects in tennis performance. Percept. Mot. Ski. 1991, 73, 844–846. [CrossRef] 22. Oviatt, D.P. Social Facilitation and Motor Performance: A Meta-Analysis. Master ’s Thesis, University of Maryland, College Park, MD, USA, 2005. 23. Schermerhorn, P.; Scheutz, M.; Crowell, C.R. Robot social presence and gender: Do females view robots differently than males? In Proceedings of the Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, Amsterdam, The Netherlands, 12–15 March 2008; pp. 263–270. 24. Riether, N.; Hegel, F.; Wrede, B.; Horstmann, G. Social facilitation with social robots? In Proceedings of the 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boston, MA, USA, 5–8 March 2012; pp. 41–47. 25. Irfan, B.; Kennedy, J.; Lemaignan, S.; Papadopoulos, F.; Senft, E.; Belpaeme, T. Social Psychology and Human-Robot Interaction: An Uneasy Marriage. In Proceedings of the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 5–8 March 2018; pp. 13–20. 26. Buccino, G.; Binkofski, F.; Fink, G.R.; Fadiga, L.; Fogassi, L.; Gallese, V.; Seitz, R.J.; Zilles, K.; Rizzolatti, G.; Freund, H.J. Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study: Cortical activation during action observation. Eur. J. Neurosci. 2001, 13, 400–404. [CrossRef] 27. Buccino, G.; Binkofski, F.; Riggio, L. The mirror neuron system and action recognition. Brain Lang 2004, 89, 370–376. [CrossRef] 28. Fadiga, L.; Craighero, L.; Olivier, E. Human motor cortex excitability during the perception of others’ action. Curr. Opin. Neurobiol. 2005, 15, 213–218. [CrossRef] 29. Wearden, J.H.; Edwards, H.; Fakhri, M.; Percival, A. Why “sounds are judged longer than lights”: Application of a model of the internal clock in humans. Q. J. Exp. Psychol. Sect. B 1998, 51, 97–120. 30. Brass, M.; Bekkering, H.; Wohlschläger, A.; Prinz, W. Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and Imitative Cues. Brain Cogn. 2000, 44, 124–143. [CrossRef] [PubMed] 31. Madhavan, S.; Stoykov, M.E. Editorial: Motor Priming for Motor Recovery: Neural Mechanisms and Clinical Perspectives. Front. Neurol. 2017, 8, 448. [CrossRef] [PubMed] 32. Castiello, U.; Lusher, D.; Mari, M.; Edwards, M.; Humphreys, G.W. Observing a human and a robotic hand grasping an object: Differential motor priming effects. In Common Mechanisms in Perception and Action, Attention and Performance XIX; Prinz, W., Hommel, B., Eds.; Oxford University Press: New York, NY, USA, 2002; pp. 315–333. 33. Tai, Y.F.; Scherfler, C.; Brooks, D.J.; Sawamoto, N.; Castiello, U. The Human Premotor Cortex Is Mirror Only for Biological Actions. Curr. Biol. 2004, 14, 117–120. [CrossRef] [PubMed] Appl. Sci. 2023, 13, 676 10 of 12 34. Kilner, J.M.; Paulignan, Y.; Blakemore, S.J. An Interference Effect of Observed Biological Movement on Action. Curr. Biol. 2003, 13, 522–525. [CrossRef] [PubMed] 35. Pierno, A.C.; Mari, M.; Lusher, D.; Castiello, U. Robotic movement elicits visuomotor priming in children with autism. Neuropsy- chologia 2008, 46, 448–454. [CrossRef] 36. Eizicovits, D.; Edan, Y.; Tabak, I.; Levy-Tzedek, S. Robotic gaming prototype for upper limb exercise: Effects of age and embodiment on user preferences and movement. Restor. Neurol. Neurosci. 2018, 36, 261–274. [CrossRef] 37. Kashi, S.; Levy-Tzedek, S. Smooth leader or sharp follower? Playing the mirror game with a robot. Restor. Neurol. Neurosci. 2018, 36, 147–159. [CrossRef] 38. Vannucci, F.; Sciutti, A.; Lehman, H.; Sandini, G.; Nagai, Y.; Rea, F. Cultural differences in speed adaptation in human-robot interaction tasks. Paladyn J. Behav. Robot. 2019, 10, 256–266. [CrossRef] 39. Press, C.; Bird, G.; Flach, R.; Heyes, C. Robotic movement elicits automatic imitation. Cogn. Brain Res. 2005, 25, 632–640. [CrossRef] 40. Chaminade, T.; Cheng, G. Social cognitive neuroscience and humanoid robotics. J. Physiol. 2009, 103, 286–295. [CrossRef] [PubMed] 41. Gazzola, V.; Rizzolatti, G.; Wicker, B.; Keysers, C. The anthropomorphic brain: The mirror neuron system responds to human and robotic actions. Neuroimage 2007, 35, 1674–1684. [CrossRef] 42. Langer, A.; Levy-Tzedek, S. Priming and Timing in Human-Robot Interactions. In Modelling Human Motion: From Human Perception to Robot Design; Noceti, N., Sciutti, A., Rea, F., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 335–350. 43. Curioni, A.; Knoblich, G.; Sebanz, N. Joint Action in Humans: A Model for Human-Robot Interactions. In Humanoid Robotics: A Reference; Goswami, A., Vadakkepat, P., Eds.; Springer: Dordrecht, The Netherlands, 2016; pp. 1–19. 44. Simon, J.R.; Small, A.M. Processing auditory information: Interference from an irrelevant cue. J. Appl. Psychol. 1969, 53, 433–435. [CrossRef] 45. Eriksen, B.A.; Eriksen, C.W. Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept. Psychophys. 1974, 16, 143–149. [CrossRef] 46. Sebanz, N.; Knoblich, G.; Prinz, W. Representing others’ actions: Just like one’s own? Cognition 2003, 88, B11–B21. [CrossRef] [PubMed] 47. Atmaca, S.; Sebanz, N.; Knoblich, G. The joint flanker effect: Sharing tasks with real and imagined co-actors. Exp. Brain Res. 2011, 211, 371–385. [CrossRef] [PubMed] 48. Ciardo, F.; Lugli, L.; Nicoletti, R.; Rubichi, S.; Iani, C. Action-space coding in social contexts. Sci. Rep. 2016, 6, 22673. [CrossRef] 49. Ciardo, F.; Wykowska, A. Response Coordination Emerges in Cooperative but Not Competitive Joint Task. Front. Psychol. 2018, 9, 1919. [CrossRef] 50. Kourtis, D.; Sebanz, N.; Knoblich, G. Predictive representation of other people’s actions in joint action planning: An EEG study. Soc. Neurosci. 2013, 8, 31–42. [CrossRef] 51. Kourtis, D.; Knoblich, G.; Wozniak, ´ M.; Sebanz, N. Attention allocation and task representation during joint action planning. J. Cogn. Neurosci. 2014, 26, 2275–2286. [CrossRef] 52. Huang, C.-M.; Cakmak, M.; Mutlu, B. Adaptive Coordination Strategies for Human-Robot Handovers. Robot. Sci. Syst. 2015, 11, 1–11. 53. Hsieh, Y.W.; Wu, C.Y.; Wang, W.E.; Lin, K.C.; Chang, K.C.; Chen, C.C.; Liu, C.T. Bilateral robotic priming before task-oriented approach in subacute stroke rehabilitation: A pilot randomized controlled trial. Clin. Rehabil. 2017, 31, 225–233. [CrossRef] [PubMed] 54. Gao, X.; Wang, Y.; Chen, X.; Gao, S. Interface, interaction, and intelligence in generalized brain–computer interfaces. Trends Cogn. Sci. 2021, 25, 671–684. [CrossRef] [PubMed] 55. Frey, J.; Mühl, C.; Lotte, F.; Hachet, M. Review of the Use of Electroencephalography as an Evaluation Method for Human- Computer Interaction. In Proceedings of the PhyCS-International Conference on Physiological Computing Systems, Lisbonne, Portugal, 7–9 January 2014; pp. 214–223. 56. Rashid, M.; Sulaiman, N.; PP Abdul Majeed, A.; Musa, R.M.; Ab Nasir, A.F.; Bari, B.S.; Khatun, S. Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review. Front. Neurorobotics 2020, 14, 25. [CrossRef] [PubMed] 57. Cano, S.; Soto, J.; Acosta, L.; Peñeñory, V.M.; Moreira, F. Using Brain-Computer Interface to evaluate the User eXperience in interactive systems. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2022, 1–9. [CrossRef] 58. Klimesch, W. Alpha-band oscillations, attention, and controlled access to stored information. Trends Cogn. Sci. 2012, 16, 606–617. [CrossRef] 59. Orekhova, E.V.; Stroganova, T.A.; Posikera, I.N. Alpha activity as an index of cortical inhibition during sustained internally controlled attention in infants. Clin. Neurophysiol. 2001, 112, 740–749. [CrossRef] 60. Liu, N.; Yu, R. Influence of social presence on eye movements in visual search tasks. Ergonomics 2017, 60, 1667–1681. [CrossRef] 61. Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. 1999, 29, 169–195. [CrossRef] 62. Baek, H.; Min, B.-K. Blue light aids in coping with the post-lunch dip: An EEG study. Ergonomics 2015, 58, 803–810. [CrossRef] Appl. Sci. 2023, 13, 676 11 of 12 63. Barry, R.J.; Clarke, A.R.; McCarthy, R.; Selikowitz, M.; Rushby, J.A.; Ploskova, E. EEG differences in children as a function of resting-state arousal level. Clin. Neurophysiol. 2004, 115, 402–408. [CrossRef] [PubMed] 64. Tran, Y.; Craig, A.; McIsaac, P. Extraversion–introversion and 8–13 Hz waves in frontal cortical regions. Pers. Indiv. Differ. 2001, 30, 205–215. [CrossRef] 65. Lazzaro, I.; Gordon, E.; Li, W.; Lim, C.L.; Plahn, M.; Whitmont, S.; Clarke, S.; Barry, R.J.; Dosen, A.; Meares, R. Simultaneous EEG and EDA measures in adolescent attention deficit hyperactivity disorder. Int. J. Psychophysiol. 1999, 34, 123–134. [CrossRef] [PubMed] 66. Mann, C.A.; Lubar, J.F.; Zimmerman, A.W.; Miller, C.A.; Muenchen, R.A. Quantitative analysis of EEG in boys with attention- deficit-hyperactivity disorder: Controlled study with clinical implications. Pediatr. Neurol. 1992, 8, 30–36. [CrossRef] [PubMed] 67. Ullsperger, M.; Danielmeier, C.; Jocham, G. Neurophysiology of performance monitoring and adaptive behavior. Physiol. Rev. 2014, 94, 35–79. [CrossRef] [PubMed] 68. Weinberg, A.; Dieterich, R.; Riesel, A. Error-related brain activity in the age of RDoC: A review of the literature. Int. J. Psychophysiol. 2015, 98, 276–299. [CrossRef] [PubMed] 69. Gehring, W.J.; Goss, B.; Coles, M.G.H.; Meyer, D.E.; Donchin, E. The Error-Related Negativity. Perspect. Psychol. Sci. 2018, 13, 200–204. [CrossRef] 70. Alanis, J.C.G.; Baker, T.E.; Peper, M.; Chavanon, M.-L. Social context effects on error-related brain activity are dependent on interpersonal and achievement-related traits. Sci. Rep. 2019, 9, 1728. [CrossRef] 71. Kim, E.Y.; Iwaki, N.; Uno, H.; Fujita, T. Error-Related Negativity in Children: Effect of an Observer. Dev. Neuropsychol. 2005, 28, 871–883. [CrossRef] 72. Hajcak, G.; Moser, J.S.; Yeung, N.; Simons, R.F. On the ERN and the significance of errors. Psychophysiology 2005, 42, 151–160. [CrossRef] 73. Riesel, A.; Weinberg, A.; Endrass, T.; Kathmann, N.; Hajcak, G. Punishment has a lasting impact on error-related brain activity. Psychophysiology 2012, 49, 239–247. [CrossRef] [PubMed] 74. Van Meel, C.S.; Van Heijningen, C.A. The effect of interpersonal competition on monitoring internal and external error feedback. Psychophysiology 2010, 47, 213–222. [CrossRef] [PubMed] 75. Decety, J.; Grèzes, J.; Costes, N.; Perani, D.; Jeannerod, M.; Procyk, E.; Grassi, F.; Fazio, F. Brain activity during observation of actions. Influence of action content and subject’s strategy. Brain 1997, 120, 1763–1777. [CrossRef] [PubMed] 76. Iacoboni, M.; Woods, R.P.; Brass, M.; Bekkering, H.; Mazziotta, J.C.; Rizzolatti, G. Cortical mechanisms of human imitation. Science 1999, 286, 2526–2528. [CrossRef] [PubMed] 77. Rizzolatti, G.; Fadiga, L.; Matelli, M.; Bettinardi, V.; Paulesu, E.; Perani, D.; Fazio, F. Localization of grasp representations in humans by PET: 1. Observation versus execution. Exp. Brain Res. 1996, 111, 246–252. [CrossRef] 78. Muthukumaraswamy, S.D.; Johnson, B.W.; McNair, N.A. Mu rhythm modulation during observation of an object-directed grasp. Cogn. Brain Res. 2004, 19, 195–201. [CrossRef] 79. Oberman, L.M.; Hubbard, E.M.; McCleery, J.P.; Altschuler, E.L.; Ramachandran, V.S.; Pineda, J.A. EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Cogn. Brain Res. 2005, 24, 190–198. [CrossRef] 80. Pineda, J.A. The functional significance of mu rhythms: Translating seeing and hearing into doing. Brain Res. Rev. 2005, 50, 57–68. [CrossRef] 81. Silas, J.; Levy, J.P.; Holmes, A. Sensitivity of mu rhythm modulation to the relevance of an observed movement but not to goal congruency. Int. J. Psychophysiol. 2012, 85, 168–173. [CrossRef] 82. Sebanz, N.; Bekkering, H.; Knoblich, G. Joint action: Bodies and minds moving together. Trends Cogn. Sci. 2006, 10, 70–76. [CrossRef] 83. Sebanz, N.; Knoblich, G.; Prinz, W.; Wascher, E. Twin Peaks: An ERP Study of Action Planning and Control in Coacting Individuals. J. Cogn. Neurosci. 2006, 18, 859–870. [CrossRef] [PubMed] 84. Ruissen, M.I.; Bruijn, E.R.A.d. Is it me or is it you? Behavioral and electrophysiological effects of oxytocin administration on self-other integration during joint task performance. Cortex 2015, 70, 146–154. [CrossRef] [PubMed] 85. Peterburs, J.; Liepelt, R.; Voegler, R.; Ocklenburg, S.; Straube, T. It’s not me, it’s you-Differential neural processing of social and non-social nogo cues in joint action. Soc. Neurosci. 2017, 14, 114–124. [CrossRef] 86. Han, Y.; Ziebell, P.; Riccio, A.; Halder, S. Two sides of the same coin: Adaptation of BCIs to internal states with user-centered design and electrophysiological features. Brain Comput. Interfaces 2022, 9, 102–114. [CrossRef] 87. Singh, H.; Daly, I. Translational Algorithms: The Heart of a Brain Computer Interface. In Brain-Computer Interfaces: Current Trends and Applications; Hassanien, A.E., Azar, A.T., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 97–121. 88. Nam, C.S.; Nijholt, A.; Lotte, F. Brain–Computer Interfaces Handbook; CRC Press: Boca Raton, FL, USA, 2018. 89. Clerc, M.; Bougrain, L.; Lotte, F. Brain-Computer Interfaces 1: Methods and Perspectives; John and Wiley and Sons: Hoboken, NJ, USA, 2016. 90. Kübler, A.; Holz, E.M.; Riccio, A.; Zickler, C.; Kaufmann, T.; Kleih, S.C.; Staiger-Sälzer, P.; Desideri, L.; Hoogerwerf, E.-J.; Mattia, D. The User-Centered Design as Novel Perspective for Evaluating the Usability of BCI-Controlled Applications. PLoS ONE 2014, 9, e112392. [CrossRef] Appl. Sci. 2023, 13, 676 12 of 12 91. Lenz, C.; Nair, S.; Rickert, M.; Knoll, A.; Rosel, W.; Gast, J.; Bannat, A.; Wallhoff, F. Joint-action for humans and industrial robots for assembly tasks. In Proceedings of the RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication, Munich, Germany, 1–3 August 2008; pp. 130–135. 92. Nikolaidis, S.; Lasota, P.; Rossano, G.; Martinez, C.; Fuhlbrigge, T.; Shah, J. Human-robot collaboration in manufacturing: Quantitative evaluation of predictable, convergent joint action. In Proceedings of the IEEE ISR, Seoul, Republic of Korea, 24–26 October 2013; pp. 1–6. 93. Hentout, A.; Aouache, M.; Maoudj, A.; Akli, I. Human–robot interaction in industrial collaborative robotics: A literature review of the decade 2008–2017. Adv. Robot. 2019, 33, 764–799. [CrossRef] Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Applied Sciences Multidisciplinary Digital Publishing Institute

Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction

Applied Sciences , Volume 13 (2) – Jan 4, 2023

Loading next page...
 
/lp/multidisciplinary-digital-publishing-institute/inspiring-real-time-evaluation-and-optimization-of-human-ndash-robot-Znf06U7cbu

References (85)

Publisher
Multidisciplinary Digital Publishing Institute
Copyright
© 1996-2023 MDPI (Basel, Switzerland) unless otherwise stated Disclaimer Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. Terms and Conditions Privacy Policy
ISSN
2076-3417
DOI
10.3390/app13020676
Publisher site
See Article on Publisher Site

Abstract

applied sciences Review Inspiring Real-Time Evaluation and Optimization of Human–Robot Interaction with Psychological Findings from Human–Human Interaction 1 1 , 2 1 , 2 , Huashuo Liu , Fei Wang and Dan Zhang * Department of Psychology, School of Social Sciences, Tsinghua University, Beijing 100084, China Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China * Correspondence: dzhang@tsinghua.edu.cn Abstract: The increasingly central role of robotic agents in daily life requires effective human–robot interaction (HRI). For roboticists to optimize interaction design, it is crucial to understand the potential effects of robotic agents on human performance. Yet a systematic specification of contributing factors is lacking, and objective measures of HRI performance are still limited. In these regards, the findings of research on human–human interaction can provide valuable insights. In this review, we break down the complex effects of robotic agents on interacting humans into some basic building blocks based on human–human interaction findings, i.e., the potential effects of physical presence, motor actions, and task co-representation in HRI. For each effect, we advise on future directions regarding its implication. Furthermore, we propose that the neural correlates of these effects could support real- time evaluation and optimization of HRI with electroencephalograph (EEG)-based brain–computer interface (BCI). Keywords: human–robot interaction; human–human interaction; social facilitation; motor priming; joint action; brain–computer interface; electroencephalography 1. Introduction Citation: Liu, H.; Wang, F.; Zhang, D. As we embrace an era of human–robot interaction (HRI), robotic agents are becoming Inspiring Real-Time Evaluation and ubiquitous in our daily life, such as in education [1], rehabilitation [2], and elderly care [3]. Optimization of Human–Robot To achieve effective HRI, not only do we need to improve robot performance, but it is also Interaction with Psychological Findings from Human–Human crucial to understand the potential effects of robotic agents on human performance [4,5]. Ef- Interaction. Appl. Sci. 2023, 13, 676. forts have been made to explore how humans interact with robots in recent HRI research [6]. https://doi.org/10.3390/app13020676 Two of the primary issues in this field, however, are the lack of a systematic specification of critical factors contributing to effective interaction [7], as well as the limited objective Academic Editor: Ashley Colley measures of HRI performance [7,8]. To address these issues, it might be helpful to draw on Received: 19 November 2022 the established theories and findings from human–human interaction research, which has Revised: 17 December 2022 been intensively investigated for hundreds of years, as a considerable portion of them have Accepted: 3 January 2023 been proved to dominate aspects of HRI similarly [9]. In this review, we summarize the Published: 4 January 2023 psychological effects that a robotic agent interacting with a human might induce and, more importantly, introduce the approach of integrating electroencephalography (EEG)-based brain–computer interface (BCI) in HRI application, which can be promising for real-time measurement and optimization of these effects. Due to the limited practices carried out in Copyright: © 2023 by the authors. this area so far, we do not intend to conduct a systematic review on existing applications Licensee MDPI, Basel, Switzerland. along this line. Rather, we aim to introduce the related findings from human–human stud- This article is an open access article ies, discuss their future implication in HRI, and illustrate how to implement EEG-based distributed under the terms and BCI in different fields of HRI. More specifically, we first break down the complex psycho- conditions of the Creative Commons logical influence of interacting agents into some of their basic building blocks based on Attribution (CC BY) license (https:// classical human–human interaction findings, i.e., the potential effects of physical presence, creativecommons.org/licenses/by/ motor actions, and task co-representation. For each effect, we briefly introduce the classical 4.0/). Appl. Sci. 2023, 13, 676. https://doi.org/10.3390/app13020676 https://www.mdpi.com/journal/applsci Appl. Sci. 2023, 13, x FOR PEER REVIEW 2 of 12 Appl. Sci. 2023, 13, 676 2 of 12 their basic building blocks based on classical human–human interaction findings, i.e., the potential effects of physical presence, motor actions, and task co-representation. For each effect, we briefly introduce the classical findings in human–human studies (for compre- findings in human–human studies (for comprehensive information we refer the readers to hensive information we refer the readers to related systematic reviews) and give advice related systematic reviews) and give advice on future directions concerning its implication on future directions concerning its implication for HRI. Furthermore, we believe these for HRI. Furthermore, we believe these findings can not only guide HRI design but also be findings can not only guide HRI design but also be integrated into application systems to integrated into application systems to improve task performance more directly. Therefore, improve task performance more directly. Therefore, we propose that the neural correlates we propose that the neural correlates of these effects hold the potential to support real-time of these effects hold the potential to support real-time evaluation and optimization of HRI evaluation and optimization of HRI based on the BCI technique (see Figure 1). We then based on the BCI technique (see Figure 1). We then provide a general guideline for devel- provide a general guideline for developing these BCIs, which is further illustrated with oping these BCIs, which is further illustrated with examples from different fields of HRI. examples from different fields of HRI. Figure 1. Potential psychological effects of a robotic agent on an interacting human partner in Figure 1. Potential psychological effects of a robotic agent on an interacting human partner in hu- human-robot interaction (HRI) with brain-computer interface (BCI)-based real-time evaluation and man-robot interaction (HRI) with brain-computer interface (BCI)-based real-time evaluation and optimization. optimization. 2. The Effect of Physical Presence in Social Interaction 2. The Effect of Physical Presence in Social Interaction 2.1. Social Facilitation 2.1. Social Facilitation The effect of another individual’s physical presence on the performer in human– The effect of another individual’s physical presence on the performer in human–hu- human interaction, which is “the fundamental form of interindividual influence” ([10], man interaction, which is “the fundamental form of interindividual influence” ([10], p. p. 269), is mainly discussed in the frame of the social facilitation effect. The investigation of 269), is mainly discussed in the frame of the social facilitation effect. The investigation of social facilitation can be traced back to the early days of modern psychology, and the initial social facilitation can be traced back to the early days of modern psychology, and the ini- findings are somewhat contradictory: some studies [11–13] reveal improved performance, tial findings are somewhat contradictory: some studies [11–13] reveal improved perfor- while others [14–16] indicate an impaired one when working in the presence of others. mance, while others [14–16] indicate an impaired one when working in the presence of Zajonc’s influential paper [10] provides an explanation of the phenomenon based on the others. Zajonc’s influential paper [10] provides an explanation of the phenomenon based drive theory, which states that the presence of an audience might increase one’s arousal level on the drive theory, which states that the presence of an audience might increase one’s (drive), eliciting the dominant response, which is often correct in simple tasks but incorrect arousal level (drive), eliciting the dominant response, which is often correct in simple in complex tasks. Although various explanations (e.g., [17,18]; see [19] for a review) have tasks but incorrect in complex tasks. Although various explanations (e.g., [17,18]; see [19] been proposed to account for social facilitation, most theories relate its mechanism to the for a review) have been proposed to account for social facilitation, most theories relate its individuals’ arousal level [20]. mechanism to the individuals’ arousal level [20]. Subsequent studies further reveal that the social facilitation effect is modulated by Subsequent studies further reveal that the social facilitation effect is modulated by multiple factors, such as the type of task, as well as the characteristic of the audience. multiple factors, such as the type of task, as well as the characteristic of the audience. For For example, a familiar or practiced task showed improvement in the audience condition, example, a familiar or practiced task showed improvement in the audience condition, whereas learners of a task showed no improvement [21]. When the presence of another whereas learners of a task showed no improvement [21]. When the presence of another individual is felt to be reassuring, the level of arousal (and thus the social facilitation individual is felt to be reassuring, the level of arousal (and thus the social facilitation ef- effect) might decrease [10]. Moreover, expert observers would elicit larger facilitation than fect) might decrease [10]. Moreover, expert observers would elicit larger facilitation than peer observers [22]. With the numerous empirical studies on social facilitation in various peer observers [22]. With the numerous empirical studies on social facilitation in various domains, human performance could be improved with the intentional control and design of potential social facilitation based on the specific task. Appl. Sci. 2023, 13, 676 3 of 12 2.2. The Implication of Social Facilitation for HRI As one of the classic and well-established findings in social psychology, the social facilitation effect reported in human–human interaction has important consequences in many real-world social scenarios. Meanwhile, some HRI studies have examined the effect of robotic presence on human performance. While Schermerhorn et al. [23] and Riether et al. [24] successfully replicated the social facilitation effect with robotic presence, Irfan et al. [25] reported a number of failed attempts, and therefore call for caution when applying results from interpersonal psychology to HRI. As some of the investigations on social facilitation were conducted in the early years, their research methodologies might not fully meet the higher standards required by the scientific communities today. What’s more, unique issues exist regarding the effect of robotic presence on humans in HRI, which cannot be addressed by simply transferring previous human–human interaction findings, such as the anthropomorphic appearance of the robot and the enhanced sensing and memory abilities permitted by electronic sensors and storage, etc. Therefore, it is urgent to conduct state-of-the-art research on social facilitation by robotic presence tailored to the specific factors of concern in HRI. Still, the established paradigms for investigating social facilitation and the lessons on controlling confounding effects from social psychology could be an important reference. The potential effect of robotic presence, whether similar to findings from human–human interaction, should be taken into consideration when designing HRI, as improper robotic presence might be unnecessary or even counterproductive for task performance. 3. The Effect of Motor Actions in Social Interaction 3.1. Motor Priming While the presence of another individual has an impact on the observee, humans also actively encode the actions of others as observers. The most significant discovery regarding the effect of another agent’s motor actions on humans is the so-called motor resonance, i.e., the phenomenon that the perceiver ’s corresponding motor system is activated automat- ically during action perception [26–28]. Interestingly, action observation would influence action execution per se. Due to the motor resonance mechanism which activates the neural substrates of the observed action automatically, the execution of a similar action with the observed one is facilitated or ‘primed,’ which is called motor priming [29]. This priming effect has been found to increase with the similarity between observed and executed ac- tions [30]. However, if the required action to perform is incompatible with the observed one, the prepotent response to perform the primed action (i.e., the observed one) needs to be inhibited, resulting in the interference of action execution. Since the motor priming effect can be investigated empirically, it provides an objective measure of the magnitude of motor resonance. With its potential to alter motor behavior, the motor priming effect has been increas- ingly utilized in a number of real-world application scenarios. For example, some motor priming paradigms have been proved to successfully improve functional recovery after injury [31]. 3.2. The Implication of Motor Priming for HRI Given the increasing presence of embodied robots in our lives, the study of motor priming in HRI is particularly important in a variety of contexts, as priming can be advanta- geous if used properly, but can also be detrimental if not well-controlled. However, mixed findings have been reported regarding whether motor priming exists when actions are performed by robotic agents. While some studies did not find motor priming for robotic agents [32–34], more recent studies suggest that human actions can also be primed by robotic actions [35–39]. It is necessary to clarify these divergencies for a unifying theory on motor priming in HRI. We therefore suggest a more in-depth exploration of motor priming in HRI with strict control for related factors. Some possible accounts for the mixed findings regarding motor Appl. Sci. 2023, 13, 676 4 of 12 priming in HRI have been proposed. Chaminade and Cheng [40] hypothesize that motor resonance is primarily a perceptual (automatic) process in human–human interaction, but may be more susceptible to the attention of the observer in HRI. Therefore, in order to activate motor resonance in HRI, a clear instruction of paying attention to the relationship between the agents and the objects [41] is needed so that the robot’s movement is viewed as transitive goal-directed action. Langer and Levy-Tzedek [42] propose that the seemingly contradictory findings might be explained by different stimulus presentations, i.e., the embodiment of the robot, as well as how its actions are observed (e.g., through static or dynamic images). They assume that generally, actions performed by an embodied and physically present robot would more probably elicit motor resonance and priming in human subjects. A direct test of these hypotheses with well-controlled experiment conditions regarding (but not limited to) these factors is needed in future HRI research. 4. The Effect of Task Co-Representation in Social Interaction 4.1. Joint Action The effect of task co-representation, which occurs when a corresponding task repre- sentation is activated by the awareness of another ’s task [43], is primarily investigated in the context of joint action where multiple individuals coordinate to achieve a shared goal. These studies modified classical interference tasks such as the Simon task [44] or the Eriksen flanker task [45] to joint versions, in which the two individuals are responsible for one of the two responses each. A joint spatial compatibility effect similar to that in the original tasks has been observed [46–49]. This joint compatibility effect implies that individuals in joint action incorporate another person’s task into their own, thus representing their own and others’ actions in a functionally equivalent way. Task co-representation leads an actor to prepare for and predict the actions of the other actor, which starts even before the initiation of actual movements (i.e., in the action planning phase). With electrophysiological evidence for predictive shared motor planning, Kourtis et al. [50] show that people in joint action represent each other ’s action in advance in order to facilitate coordination. They further report similar motor activation during joint action planning with unimanual movements required for each actor and during individual action planning with bimanual movements [51], implying that participants in joint action engage in motor predictions concerning both their own and the co-actor ’s contributions. Joint actions are pervasive in everyday life. Extending from the effect of physical presence or separate motor actions of another agent, findings of task co-representation in joint actions offer valuable insights into the rich space for the cognitive processes that facilitate humans to collaborate with each other. 4.2. The Implication of Joint Action for HRI Equipped with multiple cognitive mechanisms to form task co-representation, humans are born experts in joint actions. Curioni, Knoblich and Sebanz [43] propose that joint action in humans could be a model for HRI to inspire the design of robotic agents with necessary interactive tools for collaboration with humans. Empirical HRI studies considering task co-representation in joint actions, however, are still limited. Huang et al. [52] identified the coordination methods in a human–human handover task and implemented these strategies in a human–robot condition. They found that if the robot could monitor their human partner and adapt their movement accordingly, the user experience and task performance could be improved. Curioni, Knoblich and Sebanz [43] commented that this result demonstrated the benefits of incorporating the awareness of the partner ’s task into the artificial agent’s planning in HRI, which not only improves efficiency but also fosters desirability of the interaction. On the one hand, we suggest that future HRI studies pay special attention to factors influencing the formation of task co-representation for human actors in human– robot joint actions. On the other hand, roboticists should consider combining specialized mechanisms to include their human partner ’s actions in robots’ own action planning. Appl. Sci. 2023, 13, 676 5 of 12 5. Real-Time Evaluation of HRI with BCI As mentioned above, the optimal level of these psychological effects in interaction often varies depending on specific HRI scenarios. Take the motor priming effect as an example. Motor priming can be used to facilitate the execution of desired movements for human actors. As a preliminary application of robotic motor priming, in a clinical trial [53], adding a robotic device to prime the required movements has been shown to facilitate rehabilitation compared to a task-oriented approach alone. However, sometime this priming effect can also be detrimental. For example, for the robotic “nurses” designed to hand surgical tools to a surgeon, if the velocity of their movement is too fast, the surgeon might be primed by this action, hence endangering the safety of the patient [42]. Therefore, it is of vital importance to measure these critical psychological effects in HRI in a real-time and quantitative way and find their optimal level. A promising technique that bears such potential is the brain–computer interface (BCI; [54]). Compared to traditional HRI evaluation methods such as questionnaires applied during or after the interaction, BCI can provide a non-disruptive and real-time evaluation of implicit cognitive processes throughout the interaction [55]. Specifically, due to its non-invasive and portable characteristics, electroencephalography (EEG)-based BCI has become the most popular form among other neural recording modalities [56]. Endeavors have been made using EEG-based BCI to evaluate interactive systems (e.g., [57]). In these applications, the measurement of human mental states such as workload, attention, and emotions has been shown to benefit a lot from the EEG technique [55]. We propose that the psychological effect of robotic agents on humans in HRI could also be quantified with BCI. 5.1. EEG Signatures of the Three Psychological Effects in HRI To achieve this goal, a critical step is to identify the EEG signatures of these effects, which social neuroscientific studies have shed light on. Starting with the neuroscientific findings related to social facilitation, on the one hand, as arousal is a key concept in most social facilitation theories, i.e., the presence of others could increase individuals’ arousal level thus facilitating their dominant response [20], an EEG approach which can provide direct information about arousal level [58,59] bears the potential to measure the extent of the social facilitation effect [60]. Indeed, EEG indicators of arousal such as alpha power, which is negatively linked to arousal level [61–64], and beta power, which is positively correlated with arousal [65,66], have already been used in studies on social facilitation [20]. On the other hand, neural correlates of some cognitive processes that may contribute to social facilitation, such as performance monitoring, have been intensively investigated with EEG. Specifically, EEG allows us to identify the event-related potentials (ERPs) instantly following specific events, making it ideal for exploring the fast-changing internal processes involved in social interaction. EEG studies on performance monitoring have paid particular attention to the error-related negativity (ERN or Ne; [67–69]), which is induced by an erroneous response. Generally, the ERN is believed to reflect how much one cares about the consequence of his action [70]. Enhanced ERNs are reported for errors committed in the presence of peers [71] or under social evaluation [72,73], proposed to arise from the greater threatening value of errors induced by concerns about being judged by others [74]. The neural substrates of motor resonance recognized by brain imaging studies [75–77] are called the mirror system. Multiple brain regions involved in motor resonance, such as the ventral premotor cortex and the pars opercularis of the posterior inferior frontal gyrus (Brodmann area 44), as well as the rostral inferior parietal lobule, are recognized by brain imaging studies [75–77]. In EEG research, the event-related synchronization and desynchronization of the mu (8–12 Hz) rhythm have been linked to the activity of the mirror system [78,79]. The power of the mu rhythm decreases when subjects perform or observe an action, known as mu suppression (see [80] for a review). In the context of motor priming, an EEG study [81] revealed a more significant mu suppression when the relevance of observed movement to the required action to perform is higher. Appl. Sci. 2023, 13, 676 6 of 12 The neural processes involved in joint action (mostly with joint interference paradigms) have also been investigated with EEG. The amplitude of the P3 component is larger for their own No-Go trials in a joint action setting than in an individual condition, suggesting an increased need for response inhibition [82,83]. Another ERP component related to the joint action task is the stimulus-locked N2 reflecting response conflict. Ruissen and Bruijn [84] reported that after nasal oxytocin administration, which has been reported to facilitate social behavior by enhancing self–other integration, participants exhibit larger N2 for Go trials in the joint Simon task compared to the individual setting. Peterburs et al. [85] investigated differences in the processing of social (i.e., indicating that a co-actor should respond) and nonsocial No-Go (i.e., signaling that neither actor nor co-actor should respond) cues in a joint flanker task. The conflict-sensitive N2 showed a compatibility effect only for social No-Go cues, and the P3b component related to response discrimination and evaluation was larger for social relative to non-social No-Go. We summarize the neural correlates related to the potential psychological effects in Table 1. Note that, however, one should be careful to generalize these neural signatures to tasks different from the ones reported in original studies, as many of them depend on task types, as well as the cognitive processes involved. Table 1. Psychological effects in social interaction and their potential EEG signatures. Psychological Cognitive Processes Involved Related Factor References Effects (And Their Potential EEG Signatures *) Arousal level [61–66] (alpha and beta power) Physical presence Social facilitation Performance monitoring [71–73] (error-related negativity, ERN) Sensorimotor mirroring Motor priming Motor action [80,81] (event-related mu suppression) Response conflict [84,85] (N2 component) Stimulus classification, response discrimination Task [50,82,83,85] Joint action (P3 component) co-representation Action planning (contingent negative variation, CNV; [50,51] movement-related potential, MRP) * Note that the EEG signatures of these psychological effects are contingent on specific tasks and cognitive processes involved in HRI. 5.2. Developing BCI Systems for HRI Evaluation and Optimization In order to evaluate these psychological effects based on human neural activity in HRI, a passive BCI (as opposed to an active BCI designed for controlling external devices with the user ’s brain signal) [86] could be utilized. To implement it in a specific HRI scenario, one should first specify the effect of interest in the given task. Although multiple psychological effects might play a role in the same task, in most cases, the target effect would be more focused. For example, researchers might be most interested in measuring the extent of the motor priming effect in a task where a robot and a human are both involved in intricate physical movements. Therefore, the specification of the effect of interest depends on the purpose and type of different HRI tasks and scenarios. Once the goal of the BCI system is determined, researchers could identify potential neural signatures associated with the target effect by means of a literature search on related neuroscience studies. Special attention should be paid to the paradigm reported in original studies, as the neural correlates of the desired effect could vary across different paradigms. Pilot studies with a similar setting to the real HRI application, if possible, are recommended to confirm the validity of the chosen neural signatures. Appl. Sci. 2023, 13, x FOR PEER REVIEW 7 of 12 psychological effects might play a role in the same task, in most cases, the target effect would be more focused. For example, researchers might be most interested in measuring the extent of the motor priming effect in a task where a robot and a human are both in- volved in intricate physical movements. Therefore, the specification of the effect of interest depends on the purpose and type of different HRI tasks and scenarios. Once the goal of the BCI system is determined, researchers could identify potential neural signatures associated with the target effect by means of a literature search on re- lated neuroscience studies. Special attention should be paid to the paradigm reported in original studies, as the neural correlates of the desired effect could vary across different paradigms. Pilot studies with a similar setting to the real HRI application, if possible, are recommended to confirm the validity of the chosen neural signatures. Appl. Sci. 2023, 13, 676 7 of 12 A typical BCI system consists of multiple component stages such as data acquisition, preprocessing, dimensionality reduction, feature extraction, classification, and applica- tion [87]. To construct a BCI based on the selected features, we recommend classical hand- A typical BCI system consists of multiple component stages such as data acquisition, books on BCI application (e.g., [88,89]) for technical details. The performance of the BCI preprocessing, dimensionality reduction, feature extraction, classification, and applica- prototype tion sh [87 oul ]. d To th constr en be uct evalua a BCI tbased ed in on simu thelated selected HRI featur scena es,rios we rby ecommend comparin classical g the BCI handbooks on BCI application (e.g., [88,89]) for technical details. The performance of the output to other validated measures of the target effect, as well as by evaluating the usa- BCI prototype should then be evaluated in simulated HRI scenarios by comparing the bility of the system [90]. If the system performance could not meet certain criterions, one BCI output to other validated measures of the target effect, as well as by evaluating the might search for other neural signatures or adjust the algorithm of the system modules. usability of the system [90]. If the system performance could not meet certain criterions, An iterative process is often needed in the development of BCI systems. one might search for other neural signatures or adjust the algorithm of the system modules. The implementation of the BCI in real HRI scenarios enables a real-time evaluation of the An iterative process is often needed in the development of BCI systems. The implementation of the BCI in real HRI scenarios enables a real-time evaluation effect of interest. The robotic agents could be further designed to adapt to the BCI output in a of the effect of interest. The robotic agents could be further designed to adapt to the BCI timely manner to optimize task performance and user experience. The general steps for de- output in a timely manner to optimize task performance and user experience. The general veloping BCIs for HRI evaluation and optimization are schematized in Figure 2. steps for developing BCIs for HRI evaluation and optimization are schematized in Figure 2. Figure 2. General steps for developing BCIs for HRI evaluation and optimization. Figure 2. General steps for developing BCIs for HRI evaluation and optimization. 5.3. Application of BCI in Different Fields of HRI To illustrate the application of BCI combined with the potential psychological effects in HRI, we next describe a few typical scenarios from different HRI fields for exemplary purposes. Appl. Sci. 2023, 13, 676 8 of 12 The increasing use of robotic assistance in medicine calls attention to their critical influences on humans, which might even endanger the life of patients under extreme circumstances. The extent of the motor priming effect of a robotic nurse’s movement on a human surgeon could be tracked with a BCI approach. A high degree of motor priming effect indicates that the surgeon might be seriously affected by the robotic movement [42], therefore the velocity of the robot’s movement could be set to an appropriate range, match- ing the habit of the human surgeon in this situation, rather than as fast as possible, in order to avoid detrimental consequences. Another field with extensive application of robotics is the manufacturing industry. Indeed, joint action for humans and industrial robots has not only been investigated in studies [91,92], but also become a reality in factories [93]. When an industrial robot and a human worker assembly capital goods together, it is important for both of them to form a unified representation of their shared task, which could be monitored be means of a BCI. If the BCI outcome indicates insufficient task co-representation for the human worker (e.g., weak motor planning EEG activities before the movement of the robot), the robot should be adjusted to display more predictable behaviors in line with the human partner ’s expectation, which has been proved to facilitate joint action performance in HRI [43]. Meanwhile, the social facilitation effect induced by robotic presence is ubiquitous in most fields of HRI, including the aforementioned scenarios. With a BCI to evaluate the magnitude of such effect in real time, the electronic monitoring function of the robot could be emphasized when beneficial to human performance and weakened when detrimental, resulting in improved productivity. 6. Conclusions In this article, we review human–human interaction studies investigating three pri- mary psychological effects which may also play an essential role in HRI, i.e., the potential effects of physical presence, motor actions, and task co-representation. For each effect, we discuss its implication for HRI and provide suggestions for future studies. Importantly, we propose that the neural signatures related to these effects could support real-time evalua- tion and optimization of HRI with EEG-based BCI. A general procedure for developing such BCIs is introduced, the implementation of which is further illustrated with examples from different fields of HRI. Therefore, we believe that psychological findings can not only serve as guidelines for HRI design, but also be integrated into application systems for a direct enhancement of HRI performance. Theoretical and technical challenges remain to be solved in future studies, such as the development of more portable EEG acquisition devices for better BCI user experience and the limited examination of the ecological validity of the neural signatures of the psychological effects in different HRI tasks. Author Contributions: Conceptualization, H.L., F.W. and D.Z.; investigation and visualization, H.L.; writing—original draft preparation, H.L.; writing—review and editing, F.W. and D.Z.; supervision and funding acquisition, F.W and D.Z. All authors have read and agreed to the published version of the manuscript. Funding: This work was funded by the National Natural Science Foundation of China, grant number 61977041, and the Tsinghua University Spring Breeze Fund, grant number 2021Z99CFY037. Institutional Review Board Statement: Not applicable. Informed Consent Statement: Not applicable. Data Availability Statement: No new data were created or analyzed in this study. Data sharing is not applicable to this article. Conflicts of Interest: The authors declare no conflict of interest. Appl. Sci. 2023, 13, 676 9 of 12 References 1. Toh, L.P.E.; Causo, A.; Tzuo, P.-W.; Chen, I.M.; Yeo, S.H. A review on the use of robots in education and young children. J. Educ. Technol. Soc. 2016, 19, 148–163. 2. Li, B.; Li, G.; Sun, Y.; Jiang, G.; Kong, J.; Jiang, D. A review of rehabilitation robot. In Proceedings of the 2017 32nd Youth Academic Annual Conference of Chinese Association of Automation (YAC), Hefei, China, 19–21 May 2017; pp. 907–911. 3. Abdi, J.; Al-Hindawi, A.; Ng, T.; Vizcaychipi, M.P. Scoping review on the use of socially assistive robot technology in elderly care. BMJ Open 2018, 8, e018815. [CrossRef] 4. Licklider, J.C.R. Man-Computer Symbiosis. Ire. T. Hum. Fact. Electf. 1960, 1, 4–11. [CrossRef] 5. Henschel, A.; Hortensius, R.; Cross, E.S. Social Cognition in the Age of Human–Robot Interaction. Trends Neurosci. 2020, 43, 373–384. [CrossRef] 6. Bethel, C.L.; Murphy, R.R. Review of Human Studies Methods in HRI and Recommendations. Int. J. Soc. Robot. 2010, 2, 347–359. [CrossRef] 7. Wiese, E.; Metta, G.; Wykowska, A. Robots As Intentional Agents: Using Neuroscientific Methods to Make Robots Appear More Social. Front. Psychol. 2017, 8, 1663. [CrossRef] 8. Kompatsiari, K.; Pérez-Osorio, J.; Tommaso, D.D.; Metta, G.; Wykowska, A. Neuroscientifically-Grounded Research for Improved Human-Robot Interaction. In Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 1–5 October 2018; pp. 3403–3408. 9. Krämer, N.C.; von der Pütten, A.; Eimler, S. Human-Agent and Human-Robot Interaction. In Human-Computer Interaction: The Agency Perspective; Zacarias, M., de Oliveira, J.V., Eds.; Springer: Berlin/Heidelberg, Germany, 2012; pp. 215–240. 10. Zajonc, R.B. Social Facilitation. Science 1965, 149, 269–274. [CrossRef] 11. Triplett, N. The Dynamogenic Factors in Pacemaking and Competition. Am. J. Psychol. 1898, 9, 507–533. [CrossRef] 12. Meumann, E. Haus-und Schularbeit Experimente an Kindern der Volksschule; Kessinger Publishing: Whitefish, MT, USA, 1904. 13. Dashiell, J.F. An experimental analysis of some group effects. J. Abnorm. Soc. Psychol. 1930, 25, 190–199. [CrossRef] 14. Moore, H.T. Laboratory Tests of Anger, Fear and Sex Interest. Am. J. Psychol. 1917, 28, 390–395. [CrossRef] 15. Ekdahl, A.G. Effects of attitude on free word association time. Genet. Psychol. Monogr. 1929, 5, 253–338. 16. Burri, C. The influence of an audience upon recall. J. Educ. Psychol. 1931, 22, 683–690. [CrossRef] 17. Bond, C.F. Social facilitation: A self-presentational view. J. Personal. Soc. Psychol. 1982, 42, 1042–1050. [CrossRef] 18. Baron, R.S. Distraction-Conflict Theory: Progress and Problems. Adv. Exp. Soc. Psychol. 1986, 19, 1–40. [CrossRef] 19. Strauss, B. Social facilitation in motor tasks: A review of research and theory. Psychol. Sp. Exerc. 2002, 3, 237–256. [CrossRef] 20. Liu, N.; Yu, R.; Yang, L.; Lin, X. Gender composition mediates social facilitation effect in co-action condition. Sci. Rep. 2017, 7, 15073. [CrossRef] 21. Dube, S.K.; Tatz, S.J. Audience effects in tennis performance. Percept. Mot. Ski. 1991, 73, 844–846. [CrossRef] 22. Oviatt, D.P. Social Facilitation and Motor Performance: A Meta-Analysis. Master ’s Thesis, University of Maryland, College Park, MD, USA, 2005. 23. Schermerhorn, P.; Scheutz, M.; Crowell, C.R. Robot social presence and gender: Do females view robots differently than males? In Proceedings of the Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, Amsterdam, The Netherlands, 12–15 March 2008; pp. 263–270. 24. Riether, N.; Hegel, F.; Wrede, B.; Horstmann, G. Social facilitation with social robots? In Proceedings of the 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Boston, MA, USA, 5–8 March 2012; pp. 41–47. 25. Irfan, B.; Kennedy, J.; Lemaignan, S.; Papadopoulos, F.; Senft, E.; Belpaeme, T. Social Psychology and Human-Robot Interaction: An Uneasy Marriage. In Proceedings of the Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 5–8 March 2018; pp. 13–20. 26. Buccino, G.; Binkofski, F.; Fink, G.R.; Fadiga, L.; Fogassi, L.; Gallese, V.; Seitz, R.J.; Zilles, K.; Rizzolatti, G.; Freund, H.J. Action observation activates premotor and parietal areas in a somatotopic manner: An fMRI study: Cortical activation during action observation. Eur. J. Neurosci. 2001, 13, 400–404. [CrossRef] 27. Buccino, G.; Binkofski, F.; Riggio, L. The mirror neuron system and action recognition. Brain Lang 2004, 89, 370–376. [CrossRef] 28. Fadiga, L.; Craighero, L.; Olivier, E. Human motor cortex excitability during the perception of others’ action. Curr. Opin. Neurobiol. 2005, 15, 213–218. [CrossRef] 29. Wearden, J.H.; Edwards, H.; Fakhri, M.; Percival, A. Why “sounds are judged longer than lights”: Application of a model of the internal clock in humans. Q. J. Exp. Psychol. Sect. B 1998, 51, 97–120. 30. Brass, M.; Bekkering, H.; Wohlschläger, A.; Prinz, W. Compatibility between Observed and Executed Finger Movements: Comparing Symbolic, Spatial, and Imitative Cues. Brain Cogn. 2000, 44, 124–143. [CrossRef] [PubMed] 31. Madhavan, S.; Stoykov, M.E. Editorial: Motor Priming for Motor Recovery: Neural Mechanisms and Clinical Perspectives. Front. Neurol. 2017, 8, 448. [CrossRef] [PubMed] 32. Castiello, U.; Lusher, D.; Mari, M.; Edwards, M.; Humphreys, G.W. Observing a human and a robotic hand grasping an object: Differential motor priming effects. In Common Mechanisms in Perception and Action, Attention and Performance XIX; Prinz, W., Hommel, B., Eds.; Oxford University Press: New York, NY, USA, 2002; pp. 315–333. 33. Tai, Y.F.; Scherfler, C.; Brooks, D.J.; Sawamoto, N.; Castiello, U. The Human Premotor Cortex Is Mirror Only for Biological Actions. Curr. Biol. 2004, 14, 117–120. [CrossRef] [PubMed] Appl. Sci. 2023, 13, 676 10 of 12 34. Kilner, J.M.; Paulignan, Y.; Blakemore, S.J. An Interference Effect of Observed Biological Movement on Action. Curr. Biol. 2003, 13, 522–525. [CrossRef] [PubMed] 35. Pierno, A.C.; Mari, M.; Lusher, D.; Castiello, U. Robotic movement elicits visuomotor priming in children with autism. Neuropsy- chologia 2008, 46, 448–454. [CrossRef] 36. Eizicovits, D.; Edan, Y.; Tabak, I.; Levy-Tzedek, S. Robotic gaming prototype for upper limb exercise: Effects of age and embodiment on user preferences and movement. Restor. Neurol. Neurosci. 2018, 36, 261–274. [CrossRef] 37. Kashi, S.; Levy-Tzedek, S. Smooth leader or sharp follower? Playing the mirror game with a robot. Restor. Neurol. Neurosci. 2018, 36, 147–159. [CrossRef] 38. Vannucci, F.; Sciutti, A.; Lehman, H.; Sandini, G.; Nagai, Y.; Rea, F. Cultural differences in speed adaptation in human-robot interaction tasks. Paladyn J. Behav. Robot. 2019, 10, 256–266. [CrossRef] 39. Press, C.; Bird, G.; Flach, R.; Heyes, C. Robotic movement elicits automatic imitation. Cogn. Brain Res. 2005, 25, 632–640. [CrossRef] 40. Chaminade, T.; Cheng, G. Social cognitive neuroscience and humanoid robotics. J. Physiol. 2009, 103, 286–295. [CrossRef] [PubMed] 41. Gazzola, V.; Rizzolatti, G.; Wicker, B.; Keysers, C. The anthropomorphic brain: The mirror neuron system responds to human and robotic actions. Neuroimage 2007, 35, 1674–1684. [CrossRef] 42. Langer, A.; Levy-Tzedek, S. Priming and Timing in Human-Robot Interactions. In Modelling Human Motion: From Human Perception to Robot Design; Noceti, N., Sciutti, A., Rea, F., Eds.; Springer International Publishing: Cham, Switzerland, 2020; pp. 335–350. 43. Curioni, A.; Knoblich, G.; Sebanz, N. Joint Action in Humans: A Model for Human-Robot Interactions. In Humanoid Robotics: A Reference; Goswami, A., Vadakkepat, P., Eds.; Springer: Dordrecht, The Netherlands, 2016; pp. 1–19. 44. Simon, J.R.; Small, A.M. Processing auditory information: Interference from an irrelevant cue. J. Appl. Psychol. 1969, 53, 433–435. [CrossRef] 45. Eriksen, B.A.; Eriksen, C.W. Effects of noise letters upon the identification of a target letter in a nonsearch task. Percept. Psychophys. 1974, 16, 143–149. [CrossRef] 46. Sebanz, N.; Knoblich, G.; Prinz, W. Representing others’ actions: Just like one’s own? Cognition 2003, 88, B11–B21. [CrossRef] [PubMed] 47. Atmaca, S.; Sebanz, N.; Knoblich, G. The joint flanker effect: Sharing tasks with real and imagined co-actors. Exp. Brain Res. 2011, 211, 371–385. [CrossRef] [PubMed] 48. Ciardo, F.; Lugli, L.; Nicoletti, R.; Rubichi, S.; Iani, C. Action-space coding in social contexts. Sci. Rep. 2016, 6, 22673. [CrossRef] 49. Ciardo, F.; Wykowska, A. Response Coordination Emerges in Cooperative but Not Competitive Joint Task. Front. Psychol. 2018, 9, 1919. [CrossRef] 50. Kourtis, D.; Sebanz, N.; Knoblich, G. Predictive representation of other people’s actions in joint action planning: An EEG study. Soc. Neurosci. 2013, 8, 31–42. [CrossRef] 51. Kourtis, D.; Knoblich, G.; Wozniak, ´ M.; Sebanz, N. Attention allocation and task representation during joint action planning. J. Cogn. Neurosci. 2014, 26, 2275–2286. [CrossRef] 52. Huang, C.-M.; Cakmak, M.; Mutlu, B. Adaptive Coordination Strategies for Human-Robot Handovers. Robot. Sci. Syst. 2015, 11, 1–11. 53. Hsieh, Y.W.; Wu, C.Y.; Wang, W.E.; Lin, K.C.; Chang, K.C.; Chen, C.C.; Liu, C.T. Bilateral robotic priming before task-oriented approach in subacute stroke rehabilitation: A pilot randomized controlled trial. Clin. Rehabil. 2017, 31, 225–233. [CrossRef] [PubMed] 54. Gao, X.; Wang, Y.; Chen, X.; Gao, S. Interface, interaction, and intelligence in generalized brain–computer interfaces. Trends Cogn. Sci. 2021, 25, 671–684. [CrossRef] [PubMed] 55. Frey, J.; Mühl, C.; Lotte, F.; Hachet, M. Review of the Use of Electroencephalography as an Evaluation Method for Human- Computer Interaction. In Proceedings of the PhyCS-International Conference on Physiological Computing Systems, Lisbonne, Portugal, 7–9 January 2014; pp. 214–223. 56. Rashid, M.; Sulaiman, N.; PP Abdul Majeed, A.; Musa, R.M.; Ab Nasir, A.F.; Bari, B.S.; Khatun, S. Current Status, Challenges, and Possible Solutions of EEG-Based Brain-Computer Interface: A Comprehensive Review. Front. Neurorobotics 2020, 14, 25. [CrossRef] [PubMed] 57. Cano, S.; Soto, J.; Acosta, L.; Peñeñory, V.M.; Moreira, F. Using Brain-Computer Interface to evaluate the User eXperience in interactive systems. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2022, 1–9. [CrossRef] 58. Klimesch, W. Alpha-band oscillations, attention, and controlled access to stored information. Trends Cogn. Sci. 2012, 16, 606–617. [CrossRef] 59. Orekhova, E.V.; Stroganova, T.A.; Posikera, I.N. Alpha activity as an index of cortical inhibition during sustained internally controlled attention in infants. Clin. Neurophysiol. 2001, 112, 740–749. [CrossRef] 60. Liu, N.; Yu, R. Influence of social presence on eye movements in visual search tasks. Ergonomics 2017, 60, 1667–1681. [CrossRef] 61. Klimesch, W. EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Res. Rev. 1999, 29, 169–195. [CrossRef] 62. Baek, H.; Min, B.-K. Blue light aids in coping with the post-lunch dip: An EEG study. Ergonomics 2015, 58, 803–810. [CrossRef] Appl. Sci. 2023, 13, 676 11 of 12 63. Barry, R.J.; Clarke, A.R.; McCarthy, R.; Selikowitz, M.; Rushby, J.A.; Ploskova, E. EEG differences in children as a function of resting-state arousal level. Clin. Neurophysiol. 2004, 115, 402–408. [CrossRef] [PubMed] 64. Tran, Y.; Craig, A.; McIsaac, P. Extraversion–introversion and 8–13 Hz waves in frontal cortical regions. Pers. Indiv. Differ. 2001, 30, 205–215. [CrossRef] 65. Lazzaro, I.; Gordon, E.; Li, W.; Lim, C.L.; Plahn, M.; Whitmont, S.; Clarke, S.; Barry, R.J.; Dosen, A.; Meares, R. Simultaneous EEG and EDA measures in adolescent attention deficit hyperactivity disorder. Int. J. Psychophysiol. 1999, 34, 123–134. [CrossRef] [PubMed] 66. Mann, C.A.; Lubar, J.F.; Zimmerman, A.W.; Miller, C.A.; Muenchen, R.A. Quantitative analysis of EEG in boys with attention- deficit-hyperactivity disorder: Controlled study with clinical implications. Pediatr. Neurol. 1992, 8, 30–36. [CrossRef] [PubMed] 67. Ullsperger, M.; Danielmeier, C.; Jocham, G. Neurophysiology of performance monitoring and adaptive behavior. Physiol. Rev. 2014, 94, 35–79. [CrossRef] [PubMed] 68. Weinberg, A.; Dieterich, R.; Riesel, A. Error-related brain activity in the age of RDoC: A review of the literature. Int. J. Psychophysiol. 2015, 98, 276–299. [CrossRef] [PubMed] 69. Gehring, W.J.; Goss, B.; Coles, M.G.H.; Meyer, D.E.; Donchin, E. The Error-Related Negativity. Perspect. Psychol. Sci. 2018, 13, 200–204. [CrossRef] 70. Alanis, J.C.G.; Baker, T.E.; Peper, M.; Chavanon, M.-L. Social context effects on error-related brain activity are dependent on interpersonal and achievement-related traits. Sci. Rep. 2019, 9, 1728. [CrossRef] 71. Kim, E.Y.; Iwaki, N.; Uno, H.; Fujita, T. Error-Related Negativity in Children: Effect of an Observer. Dev. Neuropsychol. 2005, 28, 871–883. [CrossRef] 72. Hajcak, G.; Moser, J.S.; Yeung, N.; Simons, R.F. On the ERN and the significance of errors. Psychophysiology 2005, 42, 151–160. [CrossRef] 73. Riesel, A.; Weinberg, A.; Endrass, T.; Kathmann, N.; Hajcak, G. Punishment has a lasting impact on error-related brain activity. Psychophysiology 2012, 49, 239–247. [CrossRef] [PubMed] 74. Van Meel, C.S.; Van Heijningen, C.A. The effect of interpersonal competition on monitoring internal and external error feedback. Psychophysiology 2010, 47, 213–222. [CrossRef] [PubMed] 75. Decety, J.; Grèzes, J.; Costes, N.; Perani, D.; Jeannerod, M.; Procyk, E.; Grassi, F.; Fazio, F. Brain activity during observation of actions. Influence of action content and subject’s strategy. Brain 1997, 120, 1763–1777. [CrossRef] [PubMed] 76. Iacoboni, M.; Woods, R.P.; Brass, M.; Bekkering, H.; Mazziotta, J.C.; Rizzolatti, G. Cortical mechanisms of human imitation. Science 1999, 286, 2526–2528. [CrossRef] [PubMed] 77. Rizzolatti, G.; Fadiga, L.; Matelli, M.; Bettinardi, V.; Paulesu, E.; Perani, D.; Fazio, F. Localization of grasp representations in humans by PET: 1. Observation versus execution. Exp. Brain Res. 1996, 111, 246–252. [CrossRef] 78. Muthukumaraswamy, S.D.; Johnson, B.W.; McNair, N.A. Mu rhythm modulation during observation of an object-directed grasp. Cogn. Brain Res. 2004, 19, 195–201. [CrossRef] 79. Oberman, L.M.; Hubbard, E.M.; McCleery, J.P.; Altschuler, E.L.; Ramachandran, V.S.; Pineda, J.A. EEG evidence for mirror neuron dysfunction in autism spectrum disorders. Cogn. Brain Res. 2005, 24, 190–198. [CrossRef] 80. Pineda, J.A. The functional significance of mu rhythms: Translating seeing and hearing into doing. Brain Res. Rev. 2005, 50, 57–68. [CrossRef] 81. Silas, J.; Levy, J.P.; Holmes, A. Sensitivity of mu rhythm modulation to the relevance of an observed movement but not to goal congruency. Int. J. Psychophysiol. 2012, 85, 168–173. [CrossRef] 82. Sebanz, N.; Bekkering, H.; Knoblich, G. Joint action: Bodies and minds moving together. Trends Cogn. Sci. 2006, 10, 70–76. [CrossRef] 83. Sebanz, N.; Knoblich, G.; Prinz, W.; Wascher, E. Twin Peaks: An ERP Study of Action Planning and Control in Coacting Individuals. J. Cogn. Neurosci. 2006, 18, 859–870. [CrossRef] [PubMed] 84. Ruissen, M.I.; Bruijn, E.R.A.d. Is it me or is it you? Behavioral and electrophysiological effects of oxytocin administration on self-other integration during joint task performance. Cortex 2015, 70, 146–154. [CrossRef] [PubMed] 85. Peterburs, J.; Liepelt, R.; Voegler, R.; Ocklenburg, S.; Straube, T. It’s not me, it’s you-Differential neural processing of social and non-social nogo cues in joint action. Soc. Neurosci. 2017, 14, 114–124. [CrossRef] 86. Han, Y.; Ziebell, P.; Riccio, A.; Halder, S. Two sides of the same coin: Adaptation of BCIs to internal states with user-centered design and electrophysiological features. Brain Comput. Interfaces 2022, 9, 102–114. [CrossRef] 87. Singh, H.; Daly, I. Translational Algorithms: The Heart of a Brain Computer Interface. In Brain-Computer Interfaces: Current Trends and Applications; Hassanien, A.E., Azar, A.T., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 97–121. 88. Nam, C.S.; Nijholt, A.; Lotte, F. Brain–Computer Interfaces Handbook; CRC Press: Boca Raton, FL, USA, 2018. 89. Clerc, M.; Bougrain, L.; Lotte, F. Brain-Computer Interfaces 1: Methods and Perspectives; John and Wiley and Sons: Hoboken, NJ, USA, 2016. 90. Kübler, A.; Holz, E.M.; Riccio, A.; Zickler, C.; Kaufmann, T.; Kleih, S.C.; Staiger-Sälzer, P.; Desideri, L.; Hoogerwerf, E.-J.; Mattia, D. The User-Centered Design as Novel Perspective for Evaluating the Usability of BCI-Controlled Applications. PLoS ONE 2014, 9, e112392. [CrossRef] Appl. Sci. 2023, 13, 676 12 of 12 91. Lenz, C.; Nair, S.; Rickert, M.; Knoll, A.; Rosel, W.; Gast, J.; Bannat, A.; Wallhoff, F. Joint-action for humans and industrial robots for assembly tasks. In Proceedings of the RO-MAN 2008-The 17th IEEE International Symposium on Robot and Human Interactive Communication, Munich, Germany, 1–3 August 2008; pp. 130–135. 92. Nikolaidis, S.; Lasota, P.; Rossano, G.; Martinez, C.; Fuhlbrigge, T.; Shah, J. Human-robot collaboration in manufacturing: Quantitative evaluation of predictable, convergent joint action. In Proceedings of the IEEE ISR, Seoul, Republic of Korea, 24–26 October 2013; pp. 1–6. 93. Hentout, A.; Aouache, M.; Maoudj, A.; Akli, I. Human–robot interaction in industrial collaborative robotics: A literature review of the decade 2008–2017. Adv. Robot. 2019, 33, 764–799. [CrossRef] Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Journal

Applied SciencesMultidisciplinary Digital Publishing Institute

Published: Jan 4, 2023

Keywords: human–robot interaction; human–human interaction; social facilitation; motor priming; joint action; brain–computer interface; electroencephalography

There are no references for this article.