Every time we perform an action, our brain receives a large amount of sensory information about both the environment and our own bodies. All this information is used to continuously update our model of the world and act accordingly. Sensory-motor coordination—literally the coordination of our sensory and motor systems—allows us to create a mental representation of our own bodies. When this coordination is disrupted, as is the case with amputees, our ability to interact with the environment is reduced. The lack of sensory feedback from that limb means our representation of our bodies is no longer complete, and this can producing a ‘phantom limb’ effect. This is generally experienced as a false sensation in the missing limb, which can include severe pain.¹

Though motor control can, to an extent be restored through myoelectric-controlled prosthetic devices,³ providing rich feedback to the user is still an open challenge. The major problem for amputees using such devices is a lack of awareness of the artificial limb: it has not been ‘incorporated’ into their body representation or body schema.⁴ A side-effect of this is that users experience an increased cognitive load while using the prosthesis, which in turn results in them abandoning it.⁵ The principal goal of our research is to find ways to include rich sensory feedback in prosthetic devices that would aid their incorporation of the user's body representation or schema. This should reduce the cognitive load and increase the device's functional repertoire.

There have been several attempts to provide tactile feedback in prosthetic applications.⁶ Our studies build upon two approaches for conveying information to the prosthetic users: using electrical stimulation of the skin, and audio feedback modulation. In terms of the former, we developed a high frequency electrical stimulator² to convey somatosensory feedback. Depending on the waveform and intensity provided to the stimulator, we can generate a wide range of sensations. Skin stimulation faces the challenge of habituation, where the perception of the stimuli received is reduced over time. However, by using high frequency electrical stimulation, we can reduce these habituation effects, and transmit information for more extended periods of time² as can be seen in Figure 1.

We also explored the use of audio feedback to convey proprioceptive information. The results of the study presented similar responses between both audio and visual feedback in the performance between groups.⁷

Shown is a comparison of stimulation methods tested to see how best to overcome habituation. We tested the recognition rate of five stimulation frequencies over a period of five trials. We utilized two stimulation methods: a) presents a drop in the recognition rate, whereas b) presents a more stable recognition rate for the same period of time.²

To evaluate the effects of the interaction between sensory and motor modalities we performed some experiments using an electromyography- (EMG-) controlled prosthetic hand with an amputee patient. Using an fMRI scanner, we recorded the activation of the motor and somatosensory cortices during her interaction with the robot hand. The participant controlled it using the EMG signals from her right arm (stump), whilst the electrical stimulation was administered in the contralateral upper arm.^8,9

Our studies presented an interesting reaction of the brain when the stimulation occurs as an isolated event. In such case, the parietal lobe perceives the stimulus presented to the body as an uncorrelated event, without a strong mapping to the somatosensory area as can be seen in Figure 2. However, when electrical stimulation was used to provide haptic feedback while grasping an object with the robot hand—even though the stimulus was provided to the arm opposite to the one used to control the robot hand—we detected activation in the somatosensory area of the left hemisphere. This can be seen in Figure 3.

With passive electrical stimulation we detected a spread activation over the parietal and occipital lobe of the patient.

With active electrical stimulation the motor and the somatosensory cortices presented an increment of activity in the left hemisphere of the amputee's brain.

The results presented here provide insights to the mechanisms responsible for generating body image representations. The production of this tactile illusion points to a correlative behavior of the brain in processing active motor commands and their corresponding feedback. This behavior can be used to shape the interaction between an amputee and a prosthetic device, promoting the ‘incorporation’ of the latter in the mental body representation of the amputee. In our future work, we will explore the benefits from using a multisensory feedback to promote this incorporation, which we believe will reduce the currently constant requirement of conscious mental activity in controlling a prosthetic device.

2. H. Flor, T. Elbert, W. Mïhlnickel, C. Pantev, C. Wienbruch and E. Taub, Cortical reorganization and phantom phenomena in congenital and traumatic upper-extremity amputees, Exp. Brain. Res. 119 (2), pp. 205-212 Mar, 1998.
   
   
3. A. Hernandez Arieta, Development of a multi-channel functional electrical stimulation system for prosthetic applications of limbs October, 2007. PhD thesis, The University of Tokyo
   
   
4. M. Asghari Oskoei and H. Hu, Myoelectric control systems– survey, Biomed. Sig. Proc. and Control 2 (4), pp. 275-294, 2007. http://www.sciencedirect.com/science/article/B7XMN-4PKP4MN-1/2/7f6f6653d2da796b0325e86cdfd50e1f.
   
   
5. A. Karl, N. Birbaumer, W. Lutzenberger, L. G. Cohen and H. Flor, Reorganization of motor and somatosensory cortex in upper extremity amputees with phantom limb pain., J. Neurosci. 21 (10), pp. 3 May, 2001.
   
   
6. E. A. Biddiss and T. T. Chau, Upper limb prosthesis use and abandonment: A survey of the last 25 years, Prosthetics and Orthotics Int'l 31 (3), pp. 236-257, 2007.
   
   
7. Y. Visell, Tactile sensory substitution: Models for enaction in HCI, Interacting with Computers 21 (1–2), pp. 38-53, 2009. Special issue: Enactive Interfaces http://www.sciencedirect.com/science/article/B6V0D-4TB7792-1/2/e62be0c137b2cdf45735f54a7110c683.
   
   
8. J. González, W. Yu and A. Hernandez-Arieta, Multichannel audio biofeedback for dynamical coupling between prosthetic hands and their users, Industrial Robot: An In'l J. 37 (2), pp. 148-156, 2010.
   
   
9. A. Hernandez Arieta, K. Dermitzakis, D. Damian, M. Lungarella and R. Pfeifer, Sensory-motor coupling in rehabilitation robotics, Handbook of Service Robotics, pp. 21-36, I-Tech Education and Publishing, 2008.
   
   
10. R. Kato, H. Yokoi, A. Hernandez Arieta, W. Yu and T. Arai, Mutual adaptation among man and machine by using f-MRI analysis, Robot. Auton. Syst. 57 (2), pp. 161-166, 2009.