Multichannel haptic feedback unlocks prosthetic hand dexterity

Abd, Moaed A.; Ingicco, Joseph; Hutchinson, Douglas T.; Tognoli, Emmanuelle; Engeberg, Erik D.

doi:10.1038/s41598-022-04953-1

Multichannel haptic feedback unlocks prosthetic hand dexterity

Journal Article · Thu Feb 10 23:00:00 EST 2022 · Scientific Reports

DOI:https://doi.org/10.1038/s41598-022-04953-1· OSTI ID:1844822

Abd, Moaed A.; Ingicco, Joseph; Hutchinson, Douglas T.; Tognoli, Emmanuelle; Engeberg, Erik D.

Abstract

Loss of tactile sensations is a major roadblock preventing upper limb-absent people from multitasking or using the full dexterity of their prosthetic hands. With current myoelectric prosthetic hands, limb-absent people can only control one grasp function at a time even though modern artificial hands are mechanically capable of individual control of all five digits. In this paper, we investigated whether people could precisely control the grip forces applied to two different objects grasped simultaneously with a dexterous artificial hand. Toward that end, we developed a novel multichannel wearable soft robotic armband to convey artificial sensations of touch to the robotic hand users. Multiple channels of haptic feedback enabled subjects to successfully grasp and transport two objects simultaneously with the dexterous artificial hand without breaking or dropping them, even when their vision of both objects was obstructed. Simultaneous transport of the objects provided a significant time savings to perform the deliveries in comparison to a one-at-a-time approach. This paper demonstrated that subjects were able to integrate multiple channels of haptic feedback into their motor control strategies to perform a complex simultaneous object grasp control task with an artificial limb, which could serve as a paradigm shift in the way prosthetic hands are operated.

View Journal Article

Sponsoring Organization:: USDOE

OSTI ID:: 1844822

Alternate ID(s):: OSTI ID: 1903818

Journal Information:: Scientific Reports, Journal Name: Scientific Reports Journal Issue: 1 Vol. 12; ISSN 2045-2322

Publisher:: Nature Publishing GroupCopyright Statement

Country of Publication:: United Kingdom

Language:: English

References (64)

Human development of grip force modulation relating to cyclic movement-induced inertial loads R., Blank; A., Breitenbach; M., Nitschke Experimental Brain Research, Vol. 138, Issue 2 https://doi.org/10.1007/s002210000622	journal	May 2001
Anthropomorphic Control of a Dexterous Artificial Hand via Task Dependent Temporally Synchronized Synergies Kent, Benjamin A.; Lavery, John; Engeberg, Erik D. Journal of Bionic Engineering, Vol. 11, Issue 2 https://doi.org/10.1016/S1672-6529(14)60044-5	journal	June 2014
Estimating the Prevalence of Limb Loss in the United States: 2005 to 2050 Ziegler-Graham, Kathryn; MacKenzie, Ellen J.; Ephraim, Patti L. Archives of Physical Medicine and Rehabilitation, Vol. 89, Issue 3 https://doi.org/10.1016/j.apmr.2007.11.005	journal	March 2008
Myoelectric control systems—A survey Asghari Oskoei, Mohammadreza; Hu, Huosheng Biomedical Signal Processing and Control, Vol. 2, Issue 4 https://doi.org/10.1016/j.bspc.2007.07.009	journal	October 2007
Prosthetic hand control: A multidisciplinary review to identify strengths, shortcomings, and the future Kumar, Dinesh K.; Jelfs, Beth; Sui, Xiaohong Biomedical Signal Processing and Control, Vol. 53 https://doi.org/10.1016/j.bspc.2019.101588	journal	August 2019
Toward improved control of prosthetic fingers using surface electromyogram (EMG) signals Khushaba, Rami N.; Kodagoda, Sarath; Takruri, Maen Expert Systems with Applications, Vol. 39, Issue 12 https://doi.org/10.1016/j.eswa.2012.02.192	journal	September 2012
Dual arm manipulation—A survey Smith, Christian; Karayiannidis, Yiannis; Nalpantidis, Lazaros Robotics and Autonomous Systems, Vol. 60, Issue 10 https://doi.org/10.1016/j.robot.2012.07.005	journal	October 2012
Multi-fingered haptic palpation using pneumatic feedback actuators Li, Min; Luo, Shan; Nanayakkara, Thrishantha Sensors and Actuators A: Physical, Vol. 218 https://doi.org/10.1016/j.sna.2014.08.003	journal	October 2014
Restoration of sensory information via bionic hands Bensmaia, Sliman J.; Tyler, Dustin J.; Micera, Silvestro Nature Biomedical Engineering https://doi.org/10.1038/s41551-020-00630-8	journal	November 2020
A cutaneous mechanoneural interface for neuroprosthetic feedback S. Srinivasan, Shriya; M. Herr, Hugh Nature Biomedical Engineering https://doi.org/10.1038/s41551-020-00669-7	journal	February 2021
Consumer design priorities for upper limb prosthetics Biddiss, Elaine; Beaton, Dorcas; Chau, Tom Disability and Rehabilitation: Assistive Technology, Vol. 2, Issue 6 https://doi.org/10.1080/17483100701714733	journal	January 2007
A survey on what Australians with upper limb difference want in a prosthesis: justification for using soft robotics and additive manufacturing for customized prosthetic hands Stephens-Fripp, Benjamin; Jean Walker, Mary; Goddard, Eliza Disability and Rehabilitation: Assistive Technology, Vol. 15, Issue 3 https://doi.org/10.1080/17483107.2019.1580777	journal	March 2019
Individual finger control of a modular prosthetic limb using high-density electrocorticography in a human subject Hotson, Guy; McMullen, David P.; Fifer, Matthew S. Journal of Neural Engineering, Vol. 13, Issue 2 https://doi.org/10.1088/1741-2560/13/2/026017	journal	February 2016
Testing a Prosthetic Haptic Feedback Simulator With an Interactive Force Matching Task Chatterjee, Aniruddha; Chaubey, Pravin; Martin, Jay JPO Journal of Prosthetics and Orthotics, Vol. 20, Issue 2 https://doi.org/10.1097/01.JPO.0000311041.61628.be	journal	January 2008
Survey of Upper-Extremity Prosthesis Users in Sweden and the United Kingdom Kyberd, Peter J.; Wartenberg, Constanze; Sandsjö, Leif JPO Journal of Prosthetics and Orthotics, Vol. 19, Issue 2 https://doi.org/10.1097/JPO.0b013e3180459df6	journal	January 2007
Upper-Limb Prosthetics: Critical Factors in Device Abandonment Biddiss, Elaine; Chau, Tom American Journal of Physical Medicine & Rehabilitation, Vol. 86, Issue 12 https://doi.org/10.1097/PHM.0b013e3181587f6c	journal	January 2007
Direction of Slip Detection for Adaptive Grasp Force Control with a Dexterous Robotic Hand Abd, Moaed A.; Gonzalez, Iker J.; Colestock, Thomas C. 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) https://doi.org/10.1109/AIM.2018.8452704	conference	July 2018
Armband with Soft Robotic Actuators and Vibrotactile Stimulators for Bimodal Haptic Feedback from a Dexterous Artificial Hand Abd, Moaed A.; Bornstein, Michael; Tognoli, Emmanuelle 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM) https://doi.org/10.1109/AIM.2018.8452709	conference	July 2018
Performance Evaluation of Pattern Recognition Algorithms for Upper Limb Prosthetic Applications Marinelli, A.; Semprini, M.; Canepa, M. 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob) https://doi.org/10.1109/BioRob49111.2020.9224466	conference	November 2020
HapWRAP: Soft Growing Wearable Haptic Device Agharese, Nathaniel; Cloyd, Tyler; Blumenschein, Laura H. 2018 IEEE International Conference on Robotics and Automation (ICRA) https://doi.org/10.1109/ICRA.2018.8460891	conference	May 2018
Towards the Control of Individual Fingers of a Prosthetic Hand Using Surface EMG Signals Tenore, Francesco; Ramos, Ander; Fahmy, Amir 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society https://doi.org/10.1109/IEMBS.2007.4353752	conference	August 2007
Classification of Finger Movements for the Dexterous Hand Prosthesis Control With Surface Electromyography Al-Timemy, Ali H.; Bugmann, Guido; Escudero, Javier IEEE Journal of Biomedical and Health Informatics, Vol. 17, Issue 3 https://doi.org/10.1109/JBHI.2013.2249590	journal	May 2013
A Soft Pneumatic Actuator as a Haptic Wearable Device for Upper Limb Amputees: Toward a Soft Robotic Liner Huaroto, Juan Julian; Suarez, Etsel; Krebs, Hermano Igo IEEE Robotics and Automation Letters, Vol. 4, Issue 1 https://doi.org/10.1109/LRA.2018.2874379	journal	January 2019
Hybrid Force–Velocity Sliding Mode Control of a Prosthetic Hand Engeberg, E. D.; Meek, S. G.; Minor, M. A. IEEE Transactions on Biomedical Engineering, Vol. 55, Issue 5 https://doi.org/10.1109/TBME.2007.914672	journal	May 2008
Decoding of Individuated Finger Movements Using Surface Electromyography Tenore, Francesco V. G.; Ramos, Ander; Fahmy, Amir IEEE Transactions on Biomedical Engineering, Vol. 56, Issue 5 https://doi.org/10.1109/TBME.2008.2005485	journal	May 2009
The GRASP Taxonomy of Human Grasp Types Feix, Thomas; Romero, Javier; Schmiedmayer, Heinz-Bodo IEEE Transactions on Human-Machine Systems, Vol. 46, Issue 1 https://doi.org/10.1109/THMS.2015.2470657	journal	February 2016
Bioinspired Sinusoidal Finger Joint Synergies for a Dexterous Robotic Hand to Screw and Unscrew Objects With Different Diameters Karnati, Nareen; Kent, Benjamin A.; Engeberg, Erik D. IEEE/ASME Transactions on Mechatronics, Vol. 18, Issue 2 https://doi.org/10.1109/TMECH.2012.2222907	journal	April 2013
Closing the Loop Between Body Compensations and Upper Limb Prosthetic Movements: A Feasibility Study Legrand, Mathilde; Jarrasse, Nathanael; Montalivet, Etienne de IEEE Transactions on Medical Robotics and Bionics, Vol. 3, Issue 1 https://doi.org/10.1109/TMRB.2020.3048251	journal	February 2021
A Review of the Performance of Extrinsically Powered Prosthetic Hands Damerla, Revanth; Qiu, Yi; Sun, Taylor M. IEEE Transactions on Medical Robotics and Bionics, Vol. 3, Issue 3 https://doi.org/10.1109/TMRB.2021.3100612	journal	August 2021
Online Myoelectric Control of a Dexterous Hand Prosthesis by Transradial Amputees Cipriani, Christian; Antfolk, Christian; Controzzi, Marco IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 19, Issue 3 https://doi.org/10.1109/TNSRE.2011.2108667	journal	June 2011
Control of Upper Limb Prostheses: Terminology and Proportional Myoelectric Control—A Review Fougner, Anders; Stavdahl, Øyvind; Kyberd, Peter J. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 20, Issue 5 https://doi.org/10.1109/TNSRE.2012.2196711	journal	September 2012
Haptic Feedback Enhances Grip Force Control of sEMG-Controlled Prosthetic Hands in Targeted Reinnervation Amputees Kim, Keehoon; Colgate, J. Edward IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 20, Issue 6 https://doi.org/10.1109/TNSRE.2012.2206080	journal	November 2012
Artificial Redirection of Sensation From Prosthetic Fingers to the Phantom Hand Map on Transradial Amputees: Vibrotactile Versus Mechanotactile Sensory Feedback Antfolk, Christian; D'Alonzo, Marco; Controzzi, Marco IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 21, Issue 1 https://doi.org/10.1109/TNSRE.2012.2217989	journal	January 2013
Closed-Loop Control of Grasping With a Myoelectric Hand Prosthesis: Which Are the Relevant Feedback Variables for Force Control? Ninu, Andrei; Dosen, Strahinja; Muceli, Silvia IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 22, Issue 5 https://doi.org/10.1109/TNSRE.2014.2318431	journal	September 2014
Short- and Long-Term Learning of Feedforward Control of a Myoelectric Prosthesis with Sensory Feedback by Amputees Strbac, Matija; Isakovic, Milica; Belic, Minja IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 25, Issue 11 https://doi.org/10.1109/TNSRE.2017.2712287	journal	November 2017
Model-Based Control of Individual Finger Movements for Prosthetic Hand Function Blana, Dimitra; Van Den Bogert, Antonie J.; Murray, Wendy M. IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 28, Issue 3 https://doi.org/10.1109/TNSRE.2020.2967901	journal	March 2020
Tactile Sensing—From Humans to Humanoids Dahiya, R. S.; Metta, G.; Valle, M. IEEE Transactions on Robotics, Vol. 26, Issue 1 https://doi.org/10.1109/TRO.2009.2033627	journal	February 2010
Robotic Hand Acceleration Feedback to Synergistically Prevent Grasped Object Slip Kent, Benjamin A.; Engeberg, Erik D. IEEE Transactions on Robotics, Vol. 33, Issue 2 https://doi.org/10.1109/TRO.2016.2633574	journal	April 2017
Bellowband: A Pneumatic Wristband for Delivering Local Pressure and Vibration Young, Eric M.; Memar, Amirhossein H.; Agarwal, Priyanshu 2019 IEEE World Haptics Conference (WHC) https://doi.org/10.1109/WHC.2019.8816075	conference	July 2019
Controlling sensation intensity for electrotactile stimulation in human-machine interfaces Akhtar, Aadeel; Sombeck, Joseph; Boyce, Brandon Science Robotics, Vol. 3, Issue 17 https://doi.org/10.1126/scirobotics.aap9770	journal	April 2018
Learning ambidextrous robot grasping policies Mahler, Jeffrey; Matl, Matthew; Satish, Vishal Science Robotics, Vol. 4, Issue 26 https://doi.org/10.1126/scirobotics.aau4984	journal	January 2019
Soft robot perception using embedded soft sensors and recurrent neural networks Thuruthel, Thomas George; Shih, Benjamin; Laschi, Cecilia Science Robotics, Vol. 4, Issue 26 https://doi.org/10.1126/scirobotics.aav1488	journal	January 2019
See, feel, act: Hierarchical learning for complex manipulation skills with multisensory fusion Fazeli, N.; Oller, M.; Wu, J. Science Robotics, Vol. 4, Issue 26 https://doi.org/10.1126/scirobotics.aav3123	journal	January 2019
Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand George, J. A.; Kluger, D. T.; Davis, T. S. Science Robotics, Vol. 4, Issue 32 https://doi.org/10.1126/scirobotics.aax2352	journal	July 2019
The Hannes hand prosthesis replicates the key biological properties of the human hand Laffranchi, M.; Boccardo, N.; Traverso, S. Science Robotics, Vol. 5, Issue 46 https://doi.org/10.1126/scirobotics.abb0467	journal	September 2020
Neuroengineering challenges of fusing robotics and neuroscience Cheng, Gordon; Ehrlich, Stefan K.; Lebedev, Mikhail Science Robotics, Vol. 5, Issue 49 https://doi.org/10.1126/scirobotics.abd1911	journal	December 2020
PneuHaptic: delivering haptic cues with a pneumatic armband He, Liang; Xu, Cheng; Xu, Ding Proceedings of the 2015 ACM International Symposium on Wearable Computers - ISWC '15 https://doi.org/10.1145/2802083.2802091	conference	January 2015
Survey of upper limb prosthesis users in Sweden, the United Kingdom and Canada Kyberd, Peter J.; Hill, Wendy Prosthetics & Orthotics International, Vol. 35, Issue 2 https://doi.org/10.1177/0309364611409099	journal	January 2011
More with less: A comparative kinematical analysis of Django Reinhardt’s adaptations to hand injury Wininger, Michael; Williams, David J. Prosthetics & Orthotics International, Vol. 39, Issue 3 https://doi.org/10.1177/0309364614523173	journal	January 2015
Electromyogram synergy control of a dexterous artificial hand to unscrew and screw objects Kent, Benjamin A.; Karnati, Nareen; Engeberg, Erik D. Journal of NeuroEngineering and Rehabilitation, Vol. 11, Issue 1 https://doi.org/10.1186/1743-0003-11-41	journal	January 2014
A real-time comparison between direct control, sequential pattern recognition control and simultaneous pattern recognition control using a Fitts’ law style assessment procedure Wurth, Sophie M.; Hargrove, Levi J. Journal of NeuroEngineering and Rehabilitation, Vol. 11, Issue 1 https://doi.org/10.1186/1743-0003-11-91	journal	January 2014
EMG Biofeedback for online predictive control of grasping force in a myoelectric prosthesis Dosen, Strahinja; Markovic, Marko; Somer, Kelef Journal of NeuroEngineering and Rehabilitation, Vol. 12, Issue 1 https://doi.org/10.1186/s12984-015-0047-z	journal	June 2015
Shoulder kinematics plus contextual target information enable control of multiple distal joints of a simulated prosthetic arm and hand Mick, Sébastien; Segas, Effie; Dure, Lucas Journal of NeuroEngineering and Rehabilitation, Vol. 18, Issue 1 https://doi.org/10.1186/s12984-020-00793-0	journal	January 2021
The SoftHand Pro: Functional evaluation of a novel, flexible, and robust myoelectric prosthesis Godfrey, Sasha Blue; Zhao, Kristin D.; Theuer, Amanda PLOS ONE, Vol. 13, Issue 10 https://doi.org/10.1371/journal.pone.0205653	journal	October 2018
A national study of Veterans with major upper limb amputation: Survey methods, participants, and summary findings Resnik, Linda; Ekerholm, Sarah; Borgia, Matthew PLOS ONE, Vol. 14, Issue 3 https://doi.org/10.1371/journal.pone.0213578	journal	March 2019
Sensory feedback in upper limb prosthetics Antfolk, Christian; D’Alonzo, Marco; Rosén, Birgitta Expert Review of Medical Devices, Vol. 10, Issue 1 https://doi.org/10.1586/erd.12.68	journal	January 2013
Mechanical design and performance specifications of anthropomorphic prosthetic hands: A review Belter, Joseph T.; Segil, Jacob L.; Dollar, Aaron M. The Journal of Rehabilitation Research and Development, Vol. 50, Issue 5 https://doi.org/10.1682/JRRD.2011.10.0188	journal	January 2013
Differences in myoelectric and body-powered upper-limb prostheses: Systematic literature review Carey, Stephanie L.; Lura, Derek J.; Highsmith, M. Jason Journal of Rehabilitation Research and Development, Vol. 52, Issue 3 https://doi.org/10.1682/JRRD.2014.08.0192	journal	January 2015
Myoelectric control of prosthetic hands: state-of-the-art review Geethanjali, Purushothaman Medical Devices: Evidence and Research, Vol. Volume 9 https://doi.org/10.2147/MDER.S91102	journal	January 2016
Prosthesis rejection in acquired major upper-limb amputees: a population-based survey Østlie, Kristin; Lesjø, Ingrid Marie; Franklin, Rosemary Joy Disability and Rehabilitation: Assistive Technology, Vol. 7, Issue 4 https://doi.org/10.3109/17483107.2011.635405	journal	November 2011
Bayesian Exploration for Intelligent Identification of Textures Fishel, Jeremy A.; Loeb, Gerald E. Frontiers in Neurorobotics, Vol. 6 https://doi.org/10.3389/fnbot.2012.00004	journal	January 2012
Improving Fine Control of Grasping Force during Hand–Object Interactions for a Soft Synergy-Inspired Myoelectric Prosthetic Hand Fu, Qiushi; Santello, Marco Frontiers in Neurorobotics, Vol. 11 https://doi.org/10.3389/fnbot.2017.00071	journal	January 2018
Literature Review on Needs of Upper Limb Prosthesis Users Cordella, Francesca; Ciancio, Anna Lisa; Sacchetti, Rinaldo Frontiers in Neuroscience, Vol. 10 https://doi.org/10.3389/fnins.2016.00209	journal	May 2016
Control Capabilities of Myoelectric Robotic Prostheses by Hand Amputees: A Scientific Research and Market Overview Atzori, Manfredo; Müller, Henning Frontiers in Systems Neuroscience, Vol. 9 https://doi.org/10.3389/fnsys.2015.00162	journal	November 2015

Similar Records

Haptic perception with an articulated, sensate robot hand

Technical Report · Wed Feb 28 23:00:00 EST 1990 · OSTI ID:6909453

Soft Robotic Glove System for Wearable Haptic Teleoperation - 18373

Conference · Sun Jul 01 00:00:00 EDT 2018 · OSTI ID:22977681

Multichannel haptic feedback unlocks prosthetic hand dexterity

Citation Formats

References (64)

Similar Records

Related Subjects