Skip to main content
SearchLoginLogin or Signup

The Relative Contribution of Five Key Perceptual Cues and Their Interaction to the Sense of Embodiment

Volume 3, Issue 1, Spring 2022. DOI: 10.1037/tmb0000068

Published onMar 10, 2022
The Relative Contribution of Five Key Perceptual Cues and Their Interaction to the Sense of Embodiment
·

Abstract

A range of perceptual cues drive the Sense of Embodiment (SoE) with an external object, such as a virtual arm that looks like one’s own or can be controlled like one’s own. Since most experiments test one or two cues at a time, it is difficult to establish their relative contribution and possible interaction. This work aims at investigating the importance of five key perceptual cues (field of view, visuo-proprioceptive synchronicity, tactile feedback, visual human likeness, and connectedness) and their potential interaction in a single, full-factorial experiment. Participants touched a target dot, which changed position after a hit, for 1 min in a virtual environment seen through a head-mounted display. Participants’ arm and hand motions were mapped to a virtual arm and hand. All perceptual cues had two levels: SoE supportive and SoE suppressive. Twenty-eight participants completed the task in all possible combinations. We recorded Task Performance and self-ratings of Sense of Ownership and Sense of Agency. Results showed that visuo-proprioceptive synchronicity had the largest effect on all three measures. The relative importance of the remaining four cues differed for the three dependent variables. The cues did not have significant interactive effects. We conclude that when designing an interface for maximum supportive embodiment, visuo-proprioceptive synchronicity is the most important perceptual cue. The extent to which other supportive cues can improve the embodiment experience depends on the considered variables, but generally, the more other supportive cues can be added, the better.

Keywords: embodiment, perceptual cues, ownership, agency, virtual reality

Disclosures: Due to the agreement with the Ethics committee, data cannot be shared with third parties. However, analytic methods are shared in the article. Concerning the sources of funding, we declare that there are no perceived or potential conflicts of interest. We also disclose any prior uses of data reported in the manuscript.

Correspondence concerning this article should be addressed to Sara Falcone, Department of Human Media Interaction, University of Twente, Drienerlolaan 5, 7522 NB, Enschede, The Netherlands s.falcone@utwente.nl


Sense of Embodiment (SoE) refers to the experience that the external body, body part, or object (such as a rubber hand, a mannequin, a robotic device, or a virtual avatar) is perceived as one’s own (Kilteni et al., 2012). It is an important concept in the fields of virtual reality (VR) and telerobotics, where operators control a virtual or robotic limb or hand as if it is their own.

SoE has been studied originally and extensively using the paradigm of the Rubber Hand illusion (RHI; Botvinick & Cohen, 1998). The RHI is a perceptual illusion in which individuals experience a fake model hand as being part of their own body: A sense of ownership is created. To do that, the experimenter provides simultaneous tactile stimulation to both the real and fake hand. Finally, the experimenter introduces a threat (e.g., stubbing the fake hand with a knife or hitting it with a hammer) to break the illusion and record the participants’ reaction. If the illusion happened, the participants will be scared by the threat. The RHI demonstrates that the combination of visual and tactile signals strongly influences the subjective experience of body ownership. Since the original RHI studies, numerous studies have found that is possible to induce a strong SoE over virtual and real extracorporeal objects such as fake limbs, robotic hands and arms, mannequins, virtual bodies, and even empty volumes of space and invisible bodies (Caspar et al., 2015; Guterstam et al., 2015; Kondo et al., 2018; Van Der Hoort & Ehrsson, 2016). These objects include ones that, in contrast to the original paradigm, can be controlled by the user, such as is the case in telerobotics.

The most used and acknowledged definitions of SoE vary in the context and field of application (embodiment illusion, VR, telerobotics), in the components which are attributed to the embodiment and the way in which they are related, and in the effect that the SoE is said to have on individuals (De Vignemont, 2011; Kilteni et al., 2012; Longo et al., 2008; Metzinger, 2009). SoE can be seen as an overarching construct that encompasses different components. These include the sense of ownership (Krom et al., 2019), sense of agency (Newport and Preston, 2010; Newport et al., 2010), and sense of self-location (Arzy et al., 2006). The sense of ownership can be described as the feeling of self-attribution of an external object or device. For example, if an operator is teleoperating a robotic arm, the level of the sense of ownership is defined by the degree to which the operators experience the robotic arm as being part of their body (Krom et al., 2019). The sense of agency is defined as the feeling of being able to interact with the environment through an external device. This component is characterized by the trust that operators put in the fact that their intended actions are mirrored by the controlled device and by the sense of control that they have over it (Newport & Preston, 2010). The sense of self-location refers to the volume of space where one feels located. Operators should be aware of the remote environment and their position in it. They should feel confident in estimating distance and position of objects, and (if possible) in navigating in the remote environment (Arzy et al., 2006).

The relation between the SoE components and how to disentangle them is still an open debate. Most of the literature reports a strong correlation between the sense of ownership and agency (Gallagher, 2000; Pyasik et al., 2018; Tsakiris et al., 2006). The two senses have common spatiotemporal constraints of the integration processes and they affect each other at the level that is very difficult to disentangle their effects on the embodiment experience. However, there is also evidence that these two components involve different cognitive processes (Kalckert & Ehrsson, 2012; Synofzik et al., 2008; Tsakiris et al., 2010). These cognitive and neuroscientific studies sustain that although both components appear as phenomenally uniform and strongly correlated, they are complex crossmodal phenomena of heterogeneous functional and representational levels.

Toet et al. (2020) discuss two cognitive models that are currently used to describe the sensory processes underlying embodiment. One model builds on Bayesian perceptual learning (Armel & Ramachandran, 2003), and postulates that multisensory brain areas, such as the premotor cortex and posterior parietal cortex (Ehrsson et al., 2004, 2005), integrate signals from different modalities that co-occur with a high probability in near-personal space (Ehrsson et al., 2004; Makin et al., 2008). Two perceptions from different modalities are “bound” when they co-occur with a high probability. For example, in an RHI setup, if participants observe the fake hand being touched by a brush in the same position as they feel their real hand being touched, they incorporate this in their bodily representation and expect new visual feedback to co-occur with coherent tactile feedback as well. According to this theory, the perceptual cues interact and influence each other. This means that the manipulation of one cue changes the perception of the other cues involved in the experience and, as consequence, affects the SoE.

Another neuroscientific framework to understand SoE is the predictive encoding theory (Friston, 2012; Friston & Kiebel, 2009; Hohwy, 2013). In contrast with the Bayesian perceptual learning process, predictive encoding postulates that the brain produces models at each level of perceptual and cognitive processing to predict what information it should receive from the level below it (i.e., top-down). Then, the brain compares the bottom-up sensory information with the predictions from the model. The discrepancies between both (the prediction errors) are the only elements that are passed to higher levels, where they are used to update the model or resolved by activating a different model. Both these actions (model updates and activation) are aimed at minimizing or suppressing prediction errors at a lower level (Friston & Kiebel, 2009; Friston, 2011). This theory relates to the SoE in the sense that a high error is associated with a low SoE (Apps & Tsakiris, 2014). According to this theory, the model can update each perceptual cue individually and there is no interaction effect among the cues.

Understanding SoE is particularly important if SoE determines performance in teleoperation. Teleoperation systems have been developed to allow human operators to perform complex tasks in unpredictable or hazardous environments, such as the inspection of space- and deep-sea structures, demining operations, and minimally invasive surgery (Diolaiti & Melchiorri, 2002; Okamura, 2004; Sheridan, 1995). Teleoperation aims to replicate human manipulative skills and dexterity at a remote workplace over an arbitrary distance and at an arbitrary scale (Niemeyer et al., 2016). If operators feel as if they are present in the remote or virtual world, this may increase control, enhance task performance, and reduce cognitive workload (Sanchez-Vives & Slater, 2005). The idea is that operators should ideally have the (illusory) experience that the avatar’s body and hands are their own body and hands, not or hardly noticing that the operation is being mediated, increasing the teleoperation system transparency (Cabrera & Wachs, 2017). The role of SoE in affecting teleoperation performance (Niemeyer et al., 2016) gained attention in the last decade, when studies on the embodiment illusion and experience started to be designed and developed (Ehrsson, 2007; Moseley et al., 2008; Petkova & Ehrsson, 2008). Some studies try to demonstrate that a high level of SoE can improve telepresence and teleoperation tasks performance (Marasco et al., 2018; Sanchez-Vives & Slater, 2005; Schiefer et al., 2015). The study from Sanchez-Vives and Slater (2005) supported the investigation of telepresence beyond only the domain of computer science and other technologically oriented disciplines, but also as a mainstream part of neuroscience. They sustain that studies on perception, way-finding, self-representation, and sense of self, will also contribute to the understanding of telepresence. Moreover, they stated that the concept of presence is sufficiently similar to consciousness that it may help to transform research within that domain. This is, indeed, the direction that this line of research is taking. Schiefer et al. (2015) wanted to assess the effect of sensory feedback on task performance in individuals with limb loss. Sensory feedback by peripheral nerve stimulation improved object discrimination and manipulation, SoE, and confidence. An embodiment survey showed an improved sense of integration of the prosthesis in self-body image with sensory feedback. Even Marasco et al. (2018) tested the importance of the sensory feedback. The authors developed an automated neural-machine interface that vibrates the muscles used for the control of prosthetic hands. This system stimulates kinesthetic sense in amputees, allowing them to control prosthetic hand movements in the absence of visual feedback and increasing their sense of agency. This approach resulted in a promising strategy for improving motor performance.

We are interested in the key perceptual cues that cause the SoE and affect the task performance. Mostly using the classic RHI paradigm (Botvinick & Cohen, 1998), several cues that affect SoE and task performance have been identified. However, it is unclear what the relative importance is of, for instance, visuo-tactile and visuomotor synchronicity between the real and the virtual arm (Kilteni et al., 2015), likeness of the fake hand (Haans et al., 2008), the position of the real limb with respect to the virtual one (Tsakiris & Haggard, 2005), viewing mode (direct view, VR, Augmented Reality), and point of view (Škola & Liarokapis, 2016) on the strength of the embodiment illusion.

In the present study, we use a pointing task with a virtual hand to experimentally explore the effect of five important perceptual cues, selected from the literature review by Toet et al. (2020) on SoE and task performance: Field of view, visuo-proprioceptive synchronicity, tactile feedback, visual human-likeness, and connectedness. To our knowledge, testing the effect of five perceptual cues in a single experiment, therewith allowing direct comparison and exploration of interactions, has not been done before. There are more cues reported to affect SoE (Toet et al., 2020), but including more cues would have made the experimental design and potential interactions too complex, and the duration of the experiment too long. The cues were also selected such that they were compatible and independently modifiable within the task at hand. Below is a description and background of the selected cues.

  1. The field of view is the open, observable area that individuals can see without head or eye movements, directly or via an optical device (Fribourg et al., 2020), such as a VR headset. Normally, humans have a slightly over 210-degree forward-facing horizontal arc of their visual field, that is, without eye movements (with eye movements included it is slightly larger). It was demonstrated that a reduced field of view affects the sense of agency and can create movement impairments (Wenk et al., 2021).

  2. Visuo-proprioceptive synchronicity refers to the synchronicity between visual and proprioceptive cues detected by the operator. Since normally, proprioception and visual information are aligned in time and space, asynchronicity of the two could break the embodiment illusion (Kondo et al., 2018; Krom et al., 2019). However, even if the visuo-proprioceptive synchronicity seems to improve the subjective embodiment perception, the importance and weight of this cue on the SoE is still unclear (Carey et al., 2019).

  3. Tactile feedback refers to the availability of tactile information. In teleoperation and VR settings, operators lack natural tactile feedback because their hands are not actually touching any object and the current haptic technology is still limited. Tactile feedback can be provided artificially using very small forces or cues (such as vibration), that are mostly only felt through mechanoreceptors in the skin (Krogmeier et al., 2019). The effect of tactile feedback and visuo-tactile synchronization on SoE was extensively studied and explored. Currently, there is still an open debate between who sustains the importance of this perceptual cue (Fröhner et al., 2018) and who considers it as not essential to the embodiment experience (Krom et al., 2019).

  4. The arm’s visual human-likeness refers to the human-likeness and appearance fidelity of the embodied objects such as a robotic hand. The more the object one is controlling is similar to the real body, the higher will be the operator’s SoE (Mick et al., 2020; Shin et al., 2021). Several studies have been conducted on the effect of the anthropomorphism of the avatar on the self-identification and likeness of the user or operator (Kao, 2019; Kulms & Kopp, 2019), leading to design anthropomorphic systems in, for example robotics, VR, mixed reality systems to increase the user’s experience (Liarokapis et al., 2013; Mohd Tuah et al., 2016). 5 Finally, connectedness of the arm to the body refers to the perception that the embodied object is an extension of the operators’ body and (visually) connected to it as a continuum of their own body (Linebarger & Kessler, 2002). In Perez-Marcos et al. (2012), the authors investigate the importance of four factors on the SoE in a virtual RHI: Visuo-tactile synchronicity while stroking the virtual and the real arms, body continuity, alignment between the real and virtual arms, and the distance between them. The results show that the subjective illusion of ownership over the virtual arm and the time to evoke this illusion are strongly affected by not only synchronous visuo-tactile stimulation but also by connectivity of the virtual arm with the rest of the virtual body.

In our study, these perceptual cues were presented in two levels: One to support the SoE (referred to as SoE supportive), and one to suppress or break the SoE (referred to as SoE suppressive) in the context of controlling a virtual arm. While there is an extensive body of literature on the effects of each of these single cues on embodiment, their relative contribution as well as their potential interaction are still unknown. By including all five cues in a full-factorial design, we can measure the relative contribution of each to the SoE and task performance, and test for possible interactions between cues.

Research Questions and Hypotheses

This study was led by three research questions (RQs): (a) What is the ranking of the perceptual cues for two tested SoE components (sense of ownership and agency) and task performance, and is the order consistent over the different dependent variables recorded to estimate the SoE? (b) Are there interaction effects between the perceptual cues? Is a simple additive model sufficient or do we need a more complex model to test the effect of the perceptual cues together? (c) Are the SoE and task performance related? If so, this may imply that a higher SoE leads to higher task performance. Associated with these RQs are the following hypotheses (H):

Hypothesis 1: (a) We hypothesize that the perceptual cues have a different weight in affecting the sense of ownership, the sense of agency, and the task performance. Specifically, (b) the sense of ownership will be mainly affected by the visual human-likeness, the visuo-proprioceptive synchronicity, the field of view, and the connectedness, which are the cues most strongly connected to the appearance and veridicity of the external device in relation to the human operator’s perception of the remote body and environment. (c) The sense of agency will be mainly affected by the visuo-proprioceptive synchronicity, tactile feedback, and field of view, since this component is related to the mimicking of the actions by the external device and the feeling of control. Finally, (d) the task performance will be directly affected by the visuo-proprioceptive synchronicity and the field of view, since this component is related to the efficiency and effectiveness in accomplishing the task, and indirectly affected by all the other cues, namely, through increasing SoE.

Regarding RQ2, we expect that (H2) we will not find a significant interaction between the perceptual cues.

Finally, (H3) we expect that the SoE and task performance are related.

Method

Participants

Twenty-eight right-handed participants (16 females and 12 males, between 19 and 49 years old) were recruited from the Netherlands Organization for Applied Scientific Research (TNO) participant pool. The sample size was determined on the basis of previous similar studies that we found in the literature (Marasco et al., 2018; Slater et al., 2008, 2009; Tsakiris et al., 2010). Because the questionnaire was administrated in its original language (English), participants could only join if they could read, speak, and understand English. The study was approved by the Institutional Review Board of TNO (reference number: 2020-012). Participants were paid 30€ and their travel costs were reimbursed.

Materials and Task

Participants viewed a virtual scene through a head-mounted display (HMD) Vive Pro Eye. The Vive Pro Eye offers a 110° field of view, a maximum refresh rate of 90 frames per second, and a combined resolution of 2,160 × 1,200 pixels (1,080 × 1,200 pixels per eye). A Vive Tracker was placed on the floor to determine the center of the half-circle range, and another was strapped to the right wrist of the participant. The experiment was run on a Lenovo Legion T730-28ICO 90JF with a GEFORCE RTX 2080 Super graphics card and an Intel Core i9 processor. The project was created in Unity 2019.2.17f1 and Visual Studio 2019. The scene was visualized using SteamVR 1.15.19 and the SteamVR Unity Plugin 2.6.1. The “VR Hands and FP Arms Pack” by NatureManufacture was used for the arm and hand. The “Final IK” package by Rootmotion was used to allow the arm segments to move naturally. To receive the tactile feedback, participants used the Elitac Tactile Display, which is a glove that contains tactors to provide tactile stimulation to different parts of the hand. During the experiment, we activated one tactor placed on the tip of the right index finger for tactile feedback, with no offset at an intensity of 10 out of 15 on a logarithmic scale and a duration of 200 ms. The virtual environment consisted of a white grid with nine dots, of which eight were black and one was red. Participants were asked to touch the red dot. Upon touching, the red dot turned black and a random other black dot turned red. Participants were asked to touch as many red dots as they could in 1 min.

Questionnaire

We administered a shortened version of the Embodiment questionnaire, in English, from Peck and Gonzalez-Franco (2021) to measure the sense of ownership and sense of agency. Because participants had to complete the questionnaire 32 times, we reduced the number of questions from 16 to 5. We removed questions that were somewhat repetitive and addressed similar aspects of the sense of ownership and agency. We kept the questions that were easy to understand and had a clear relation to our experimental setup. For the sense of ownership, we administered the following questions: (a) I felt as if the virtual hand was my hand; (b) It seemed as if the virtual hand replaced my real hand; (c) It seemed as if the touch I felt was caused by the virtual hand touching the virtual target; (d) At some point it felt that the virtual hand resembled my own hand. For the sense of agency, we administered the following question: (e) It felt like I could control the virtual hand as if it was my own hand. Participants rated sense of ownership and agency on a 7-point Likert scale (from 1 = strongly disagree, to 7 = strongly agree). The order of the questions was randomized for each trial.

Design

Each perceptual cue had two levels: (a) supportive, in which we set the perceptual cue such that strong SoE was expected; (b) suppressive, in which we set the perceptual cue such that weak SoE was expected. The supportive and suppressive settings of the perceptual cues were as follows: (a) the field of view in the supportive condition allowed participants to have a human-like range of view (approximately 90 degrees temporally to central fixation, 50 degrees superiorly and nasally, and 60 degrees inferiorly); while in the suppressive condition, we narrowed the range and participants had the experience of observing the environment through a 14 × 4 cm casing (approximately 90° temporally to central fixation, 25° superiorly and nasally, and 30° inferiorly). (b) The visuo-proprioceptive synchronicity was supportive when the movement of the real and virtual hand was congruent, and suppressive when we added 20 ms delay1 to both the visual and tactile feedback. (c) The tactile feedback, in the supportive condition, was characterized by a vibration that the participants felt every time that they touched the red dot. To create the suppressive level of this cue, we removed the tactile feedback. (d) When the visual human-likeness was in its supportive condition, participants controlled a realistic human virtual hand; in the suppressive condition they controlled a blue shiny virtual hand. Finally, (e) when the connectedness was in the supportive condition, the virtual hand was attached to a virtual arm. In the suppressive condition, instead, participants had to accomplish the task by manipulating a floating hand.

Suppressive and Supportive conditions in the VR environment

Participants performed the 32 trials of the same task, one for each possible combination of the perceptual cues, that is, 2(supportive/ suppressive)5(perceptualcues) = 32 trials, where one trial is a 1 min pointing task. The order of trials was randomized per participant.

Procedure

Participants were asked to sit on a chair, wear the sensory glove, the tracker on the wrist, and place their right arm on a supportive platform on their right. The supportive platform was used to make the task less strenuous for the participants. Before asking to wear the VR headset, we instructed them on the procedure and the task. Next, we calibrated the position of the virtual hand and the target according to the participants’ height. During the calibration process, they experienced controlling the virtual hand. Participants performed the 32 1-min trials. After each trial, they answered the short embodiment questionnaire. Participants continued to the next trial whenever they indicated they were ready. At the end of the experiment, we asked participants for feedback on their experience and their opinion on the experiment.

Analysis

For each participant and each of the 32 trials, sense of ownership was defined as the average score for the four sense of ownership questions. For the sense of agency, we used the score on the sense of agency question. Task performance was defined as the number of touched red dots. For determining the ranking of the five perceptual cues in order of importance with respect to their effect on the three dependent variables (RQ1), and for determining whether interactions occurred (RQ2), we used a linear mixed-effects model, applying a pairwise comparison between the independent variables. The five perceptual cues (and their pairwise interactions) were treated as fixed effects, whereas participants were treated as random effect. We used a step function to remove the nonsignificant (p value > .05) interactions between the factors (perceptual cues) from the model. We removed, each time that we updated the model, the interaction with the highest p value, until all the included perceptual cues had a significant effect on the dependent variable. This operation was done for each dependent variable (sense of ownership, sense of agency, and performance as defined by the number of touched dots) separately, and resulted in a ranking of the cues in terms of t statistics. To determine whether SoE and task performance are related (RQ3), we performed Pearson’s correlations between the sense of ownership, agency, and task performance.

Figure 1

The Sense of Ownership Scores for All the Perceptual Cues in Each Condition, Supportive and Suppressive
Note. On each box, the central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the + symbol.

Results

Figures 1, 2, and 3 show the sense of ownership, sense of agency, and task performance for each of the (supportive and suppressive) levels and each of the five cues. Figures 1 and 2 indicate that the manipulations worked for all the perceptual cues in the expected direction: When the cues were in the supportive condition, SoE seemed to be higher and task performance seemed to be improved compared to the suppressive condition. Figure 3 indicates that, except for the visuo-proprioceptive synchronicity, effects on task performance were small and less consistent.

Figure 2

The Sense of Agency Scores for All the Perceptual Cues in Each Condition, Supportive and Suppressive
Note. On each box, the central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the + symbol.

Figure 3

The Task Performance Scores for All the Perceptual Cues in Each Condition, Supportive and Suppressive
Note. On each box, the central mark indicates the median, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the + symbol.

The results of the linear mixed-effects model (Table 1) confirmed these impressions. A significant effect was found for all the perceptual cues on the sense of ownership and agency, with the only exception of the human-likeness on the sense of agency (t = −0.511, p = .610). For task performance, a significant effect was found of the visuo-proprioceptive synchronicity (t = −7.067, p < .001) and connectedness (t = −2.320, p = .021).

Table 1
Overview of the Pairwise Comparison Between the Levels Supportive and Suppressive for the Perceptual Cues for Sense of Ownership, Sense of Agency, and Task Performance

Dependent variable

Independent variable

df

Std error

t value

p value

Sense of ownership

Field of view

863

0.245

−2.959

.003

Human likeness

863

0.245

−4.234

<.001

Tactile feedback

863

0.245

−12.866

<.001

Connectedness

863

0.245

3.251

.001

Visuo-proprioceptive synchronicity

863

0.245

−19.003

<.001

Sense of agency

Field of view

864

0.074

−2.914

.004

Human likeness

864

0.074

−0.511

.610

Tactile feedback

864

0.074

−7.962

<.001

Connectedness

864

0.074

2.433

.015

Visuo-proprioceptive synchronicity

864

0.074

−26.709

<.001

Task performance

Field of view

865

1.470

−1.911

.056

Human likeness

865

1.470

0.784

.433

Tactile feedback

865

1.470

1.082

.280

Connectedness

865

1.470

−2.320

.021

Visuo-proprioceptive synchronicity

865

1.470

−7.067

<.001

In Table 2, we report the rank order of the perceptual cues based on the t values, separately for sense of ownership, sense of agency and task performance, and allowed to answer our first RQ (What is the ranking of the perceptual cues for sense of ownership, sense of agency and task performance, and is the order consistent over the different dependent variables recorded to estimate the SoE?). Table 2 indicates the relative importance of the perceptual cues for obtaining a high SoE experience and task performance. For all dependent variables, modulating visuo-proprioceptive synchronicity had the strongest effect.

Table 2
The Ranking (Based on t values) of the Perceptual Cues for Sense of Ownership, Sense of Agency, and Task Performance

Ranking

Sense of ownership

Sense of agency

Task performance

1

Visuo-proprioceptive synchronicity

Visuo-proprioceptive synchronicity

Visuo-proprioceptive synchronicity

2

Tactile feedback

Tactile feedback

Connectedness

3

Human-likeness

Field of view

Field of view (no effect)

4

Connectedness

Connectedness

Tactile feedback (no effect)

5

Field of view

Human-likeness (no effect)

Human-likeness (no effect)

The linear mixed-effects model indicated that none of the perceptual cues showed significant interaction effects on any dependent variable, answering our second RQ as to whether there are interaction effects between the perceptual cues?. Table 3 indicates the order of removal of the interactive and main effects from the different models.

Table 3
The Order of Removal of the Interaction and Main Effects From the Different Models

Dependent variable

Interaction and main effect

Removal order

Ownership

Field of view—visuo-proprioceptive synchronicity

1

Field of view—connectedness

2

Agency

Field of view—human-likeness

1

Field of view—tactile feedback

2

Human likeness—connectedness

3

Field of view—visuo-proprioceptive synchronicity

4

Human-likeness—tactile feedback

5

Field of view—connectedness

6

Human-likeness—visuo-proprioceptive synchronicity

7

Tactile feedback—visuo-proprioceptive synchronicity

8

Tactile feedback—connectedness

9

Connectedness—visuo-proprioceptive synchronicity

10

Human-likeness

11

Task performance

Connectedness—visuo-proprioceptive synchronicity

1

Human-likeness—tactile feedback

2

Field of view—connectedness

3

Field of view—human-likeness

4

Tactile feedback—visuo-proprioceptive synchronicity

5

Human-likeness—connectedness

6

Tactile feedback—connectedness

7

Field of view—tactile feedback

8

Human-likeness—visuo-proprioceptive synchronicity

9

Field of view—visuo-proprioceptive synchronicity

10

Field of view

11

Tactile feedback

12

Human-likeness

13

To answer our third RQ (Are the SoE and task performance related?), we performed a Pearson’s correlation analysis between the sense of ownership, agency, and task performance. SoE and task performance were found to be weak, but significantly correlated for both sense of ownership (R2 = 0.13, df = 26, p < .001) and sense of agency (R2 = 0.18, df = 26, p < .001). A strong correlation was found between sense of ownership and sense of agency (R2 = 0.75, df = 26, p < .001).

Discussion

Our study resulted in rank orders of perceptual cues importance for the sense of ownership, sense of agency, and task performance. We found significant effects of the perceptual cues on each dependent variable, with some exceptions (RQ1). We did not observe any significant interaction effect among the independent variables, supporting the hypothesis that a linear, additive model can adequately describe the combined results of the five perceptual cues (RQ2). Furthermore, we found a weak but significant correlation between SoE and task performance (RQ3).

To discuss our findings in detail, on the basis of the results reported in Table 2 we accepted H1a (perceptual cues have a different weight in affecting the sense of ownership, agency, and task performance). We observed a different effect of each perceptual cue on the dependent variables. H1b (the perceptual cues which mostly affect sense of ownership are visual human-likeness, visuo-proprioceptive synchronicity, field of view, and connectedness), H1c (the perceptual cues which mostly affect the sense of agency are visuo-proprioceptive synchronicity, tactile feedback, and field of view), and H1d (the perceptual cues which mostly affect task performance are visuo-proprioceptive synchronicity and the field of view) were partly accepted. We found a significant effect of all perceptual cues on the sense of ownership and agency, with the only exception of the human-likeness on the sense of agency (as expected). However, the weight (based on the t value) that each cue had in affecting the dependent variable was slightly different than hypothesized—visuo-proprioceptive synchronicity was the cue which affected all dependent variables the most. As for task performance, we found a significant effect of the visuo-proprioceptive synchronicity and connectedness, but not of field of view.

We did not find an interaction effect among the independent variables and the additive model that we adopted allowed us to test the independent variables. Based on the result, we accept both H2a (we do not expect an interaction between the perceptual cues) and H2b (an additive model is enough to test this combination of perceptual cues together).

Finally, our results were consistent with H3 (SoE and task performance are related). The correlations between the dependent variables were all significant, although the correlation between the SoE components and task performance was weak.

Visuo-proprioceptive synchronicity was the cue that most strongly affected the sense of ownership and agency. When this cue was in the suppressive condition, participants had more difficulties in accomplishing the task and experiencing embodiment, as also reported in other studies (Debarba et al., 2015; Kokkinara & Slater, 2014).

However, while visuo-proprioceptive synchronicity has been found to be the most important of the tested perceptual cues, note that a weak SoE was still obtained in the suppressive condition (median sense of ownership score of 3, and a median sense of agency score of 2.5). For both SoE measures, visuo-proprioceptive synchronicity is followed by tactile feedback, demonstrating the importance of the information provided by the tactile sense in establishing SoE. Then, the ranking differentiates, showing a difference in the sense of ownership and agency. For the sense of ownership, the third rank is occupied by human-likeness, which is a coherent result with respect to our initial hypotheses, considering that the sense of ownership focuses on the sense of self-attribution of the external embodiment. For sense of agency, instead, we found the field of view to be more important. This can be explained by the fact that in the suppressive condition, the field size was only 14 × 4 cm. This made the control of the virtual arm and hand more complicated and reduced the possibility to observe the movement of the virtual arm, reducing the level of perceived agency. Ranking fourth for both sense of ownership and agency is connectedness. Connectedness provides more (visual) information on arm posture and position, supporting the experience of the external arm and hand as attached to one’s own shoulder. This may have facilitated the perception of the joints in space and the control of the virtual hand. For the sense of ownership, ranking last, but still significant, is the field of view. The field of view in the suppressive condition degraded the perception of the environment but only had a small effect on the embodiment perception, probably due to the point of view that was not changed. The first-person perspective helps the operator in having an immersive perception of the embodiment, especially when it is realized using a VR headset (Slater et al., 2010). For the sense of agency, ranking last and nonsignificant is human-likeness. The finding that human-likeness did not affect sense of agency, but did have a significant effect on sense of ownership which is in line with previous studies (Gallagher, 2000; Synofzik et al., 2008; Tsakiris et al., 2006; Tsakiris et al., 2010). Pyasik et al. (2018) argue that the individual spatiotemporal constraints for the integration of sensory-related signals, which are unconscious, are common to both the sense of ownership and the sense of agency, whereas their subjective and conscious experience would rely on additional processes specific for each sense. We assessed SoE through a questionnaire (Peck & Gonzalez-Franco, 2021) in this experiment. Although intended to measure perceptual experience, we cannot rule out that the questionnaire data include (cognitive) bias. Combining questionnaires with implicit measures such as skin conductance response (Petkova & Ehrsson, 2008), heart rate (Slater et al., 2010), or pupil dilatation (Falcone et al., 2021), can make the results more robust and provide more insight on the discrepancies in sense of ownership and agency.

For task performance, there are just two cues that affected it, firstly, the visuo-proprioceptive synchronicity and secondly, the connectedness, where performance was better in the suppressive rather than the supportive condition. The effect of visuo-proprioceptive synchronicity was consistent with participants always reporting difficulties in accomplishing the task while this cue was presented in the suppressive condition. If anything, we would have expected connectedness to increase proprioceptive information, therewith supporting task performance. The opposite finding may have been caused by a less cluttered display when only the disconnected hand is presented rather than the whole arm. Field of view, tactile feedback, and human likeness did not have a significant effect on task performance. The lack of an effect of field of view could be explained by the fact that even if it could affect the SoE, especially the sense of agency as reported in Wenk et al. (2021), the reduction of this cue in the suppressive condition did not hamper task execution. This could also be related to both the design of the task, which was simple to accomplish, and the small workspace in which participants had to operate the virtual arm.

While there is evidence that sense of ownership and agency involve different cognitive processes (Kalckert & Ehrsson, 2012; Synofzik et al., 2008; Tsakiris et al., 2010), in correspondence to most of the literature, we found a strong correlation between the sense of body ownership and sense of agency (Gallagher, 2000; Pyasik et al., 2018; Tsakiris et al., 2006). The exact relation between the sense that one’s body is one’s own (body ownership) and the sense that one controls one’s own bodily actions (agency) has been the focus of much speculation but remains unclear. Tsakiris et al. (2010), discuss two models to describe the relationship between the sense of ownership and sense of agency. First, an additive model, in which agency and body ownership are strongly related, because the ability to control actions is a powerful cue to body ownership; plus possible additional subcomponents unique to ownership and agency. An alternative independence model, sustains that agency and body ownership are qualitatively different experiences triggered by different inputs and recruiting distinct brain networks. A network of sensorimotor transformations and motor control, and a set of hetero-modal association cortices implicated in various cognitive functions. We still do not know the exact functions and contributions of these brain regions to the sense of agency. We found correlations between task performance and sense of ownership and between task performance and agency to be significant but weak with an explained variance below 5%. Previous studies reported that a higher SoE resulted in a much larger increase in task performance (Marasco et al., 2018; Sanchez-Vives & Slater, 2005; Schiefer et al., 2015). The tasks in those studies required a major interaction with the environment, which was usually better characterized than ours. An interesting possibility could be that a high SoE reduces the cognitive workload of the operator and has a larger effect when the task demands are high, for instance, when operating an avatar or teleoperation system that is more complexly designed and that can achieve more complicated tasks as is the case in the previous studies. Following this line of reasoning, a higher SoE could also affect the learning speed: Reducing the cognitive workload of the operator may speed up learning to perform the task. This prediction could be tested in a more complex task with novice users.

Future Works

In future work, we want to redesign the task in order to include the assessment of the sense of self-location, in order to have a complete picture of the SoE. Moreover, we want to extend the perceptual cues to test, such as head movement control and point of view. To include these variables, we need to redesign the setup and the experiment. To circumvent possible relations between task performance and SoE as subjectively reported through post hoc knowledge of the own performance, implicit (unconscious) measures of SoE would be of great value (Verhagen et al., 2020). To this end, adding physiological measures such as skin conductance, heart rate, and pupil dilation may be of interest. Finally, and as mentioned above, we are interested in investigating how SoE may affect learning of a different and more complex task. However, in this case, findings in Brouwer et al. (2014)rather indicate that using physiology to monitor learning is not as straightforward as one might expect.

Conclusions

Our full-factorial experiment resulted in a rank order of five different perceptual cues with respect to their effect on sense of ownership, sense of agency, and task performance. Different rankings were found, but visuo-proprioceptive synchronicity affected all three outcome measures most strongly. We did not observe an interaction effect between the perceptual cues and we found a weak relation between the level of SoE and task performance. These findings can help to decide on choosing the factors to optimize in a system in order to achieve a high sense of ownership, agency, and task performance.


Comments
0
comment

No comments here

Why not start the discussion?