A geometry-aware and customizable multimodal sensing system for texture and material identification in prosthetics
Source abstract
Abstract
The restoration of natural somatosensation remains largely absent in upper-limb prostheses. While many current electronic skins target to replicate human tactile sensing, they are still limited by coarse resolution, planar form factors, and high fabrication cost. Here, we present a geometry-aware and customizable multimodal sensing system for prosthetics, functioning as a sensing component for mimicking human skin. The system integrates high-resolution pressure and temperature mapping in conformal, free-form modules. A scan-model-print workflow converts a patient’s prosthesis topography into 3D sensor curvature for seamless surface conformity. Self-aligning snap-fit interconnects enable on-demand reconfiguration and routing across sensor modules, while interlaced pressure and temperature matrices maximize spatial utilization. Compared with commercial glove sensors, the system achieves ~10-fold finer spatial resolution for pressure and denser temperature mapping. A multi-head neural network performs real-time calibration and denoising, reducing per-pixel error and improving signal fidelity. With high-rate pressure sampling and sensitive temperature readout, the sensing system enables accurate texture and material recognition. This sensing system serves as a bridge that brings natural tactile sensing on artificial prostheses, providing a platform for future development of a full bionic skin with both sensing and haptic stimulation functions for personalized neuroprosthetics.
Source: openaire · Quality: candidate unverified · Source record ↗
References (55)
Show source-provided references
1. Restoration of sensory information via bionic hands10.1038/s41551-020-00630-8 ↗
2. Sensory Feedback for Upper-Limb Prostheses: Opportunities and Barriers10.1109/TNSRE.2022.3159186 ↗
3. Pursuing prosthetic electronic skin10.1038/nmat4671 ↗
4. Electronic Skin: Opportunities and Challenges in Convergence with Machine Learning10.1146/annurev-bioeng-103122-032652 ↗
5. Literature Review on Needs of Upper Limb Prosthesis Users10.3389/fnins.2016.00209 ↗
6. Home Use of a Neural-connected Sensory Prosthesis Provides the Functional and Psychosocial Experience of Having a Hand Again10.1038/s41598-018-26952-x ↗
7. Biomimetic sensory feedback through peripheral nerve stimulation improves dexterous use of a bionic hand10.1126/scirobotics.aax2352 ↗
8. Neural feedback strategies to improve grasping coordination in neuromusculoskeletal prostheses10.1038/s41598-020-67985-5 ↗
9. Sensory feedback by peripheral nerve stimulation improves task performance in individuals with upper limb loss using a myoelectric prosthesis10.1088/1741-2560/13/1/016001 ↗
10. Mechanotactile Sensory Feedback Improves Embodiment of a Prosthetic Hand During Active Use10.3389/fnins.2020.00263 ↗
11. Biomimetic Intraneural Sensory Feedback Enhances Sensation Naturalness, Tactile Sensitivity, and Manual Dexterity in a Bidirectional Prosthesis10.1016/j.neuron.2018.08.033 ↗
12. Multichannel haptic feedback unlocks prosthetic hand dexterity10.1038/s41598-022-04953-1 ↗
13. Restoring sensation to prosthetics10.1038/s41593-024-01621-7 ↗
14. Upper limb prostheses: bridging the sensory gap10.1177/17531934221131756 ↗
15. Sensory Feedback in Hand Prostheses: A Prospective Study of Everyday Use10.3389/fnins.2020.00663 ↗
16. E-Skin Bimodal Sensors for Robotics and Prosthesis Using PDMS Molds Engraved by Laser10.3390/s19040899 ↗
17. High-resolution imaging of skin deformation shows that afferents from human fingertips signal slip onset10.7554/eLife.64679 ↗
18. Recent Development of Flexible Tactile Sensors and Their Applications10.3390/s22010050 ↗
19. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides10.1126/scirobotics.aai7529 ↗
20. Structural Electronic Skin for Conformal Tactile Sensing10.1002/advs.202370227 ↗
21. Self-rerouting sensor network for electronic skin resilient to severe damage10.1038/s41467-025-56596-1 ↗
22. Artificial intelligence-powered electronic skin10.1038/s42256-023-00760-z ↗
23. Overview of 3D Printing Multimodal Flexible Sensors10.1021/acsami.4c14219 ↗
24. Fully 3D printed flexible, conformal and multi-directional tactile sensor with integrated biomimetic and auxetic structure10.1038/s44172-023-00131-x ↗
25. Bioinspired cilia-based electronic skin for multimodal mechanical sensing via additive manufacturing10.20517/ss.2025.03 ↗
26. Electronic skins and machine learning for intelligent soft robots10.1126/scirobotics.aaz9239 ↗
27. 3D-printed epifluidic electronic skin for machine learning–powered multimodal health surveillance10.1126/sciadv.adi6492 ↗
28. Photoelectric dual mode sensing system based on one-step fabricated heterojunction artificial synapses device10.1016/j.mser.2025.101021 ↗
29. Bifunctional optoelectronic devices based on Ruddlesden-Popper Sr2TiO4 for emulating neuromorphic attention mechanisms10.1016/j.optlastec.2025.113491 ↗
30. 2D MoTe2/MoS2−x Ox Van der Waals Heterostructure for Bimodal Neuromorphic Optoelectronic Computing10.1002/aelm.202300388 ↗
31. Photoelectric Dual-Mode and Nonvolatile Modulation Based on Ferroelectric Memristors for Reservoir Computing10.1021/acssuschemeng.4c05355 ↗
32. A multifunctional electronic skin based on patterned metal films for tactile sensing with a broad linear response range10.1126/sciadv.abl8313 ↗
33. Flexible and self-powered temperature–pressure dual-parameter sensors using microstructure-frame-supported organic thermoelectric materials10.1038/ncomms9356 ↗
34. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics10.1126/scirobotics.aau6914 ↗
35. Mutually Noninterfering Flexible Pressure–Temperature Dual-Modal Sensors Based on Conductive Metal–Organic Framework for Electronic Skin10.1021/acsnano.1c07388 ↗
36. Multimodal Sensors with Decoupled Sensing Mechanisms10.1002/advs.202202470 ↗
37. Learning the signatures of the human grasp using a scalable tactile glove10.1038/s41586-019-1234-z ↗
38. A Flexible, Large-Scale Sensing Array with Low-Power In-Sensor Intelligence
39. Tekscan Grip™ System (Grip Sensor Model 4256E). https://www.tekscan.com/products-solutions/systems/grip-system.
40. TE Connectivity: (2018) GA10K3MCD1 MICRO BetaCHIP (MCD) Thermistor Probes Data Sheet. https://www.te.com/commerce/DocumentDelivery/DDEController?Action=srchrtrv&DocNm=GA10K3MCD1&DocType=Data+Sheet&DocLang=English&PartCntxt=GA10K3MCD1&DocFormat=pdf.
41. A natural biomimetic prosthetic hand with neuromorphic tactile sensing for precise and compliant grasping10.1126/sciadv.adr9300 ↗
42. Flexible Three-Dimensional Force Tactile Sensor Based on Velostat Piezoresistive Films10.3390/mi15040486 ↗
43. Multisensory electronic skin with decoupled pressure–temperature-sensing capabilities for similar object recognition10.1073/pnas.2519693122 ↗
44. Energy autonomous electronic skin with direct temperature-pressure perception10.1016/j.nanoen.2022.107273 ↗
45. Versatile adhesive skin enhances robotic interactions with the environment10.1126/sciadv.adt4765 ↗
46. Digital manufacturing of personalised footwear with embedded sensors10.1038/s41598-023-29261-0 ↗
47. Robotic Snap-fit Assembly with Success Identification Based on Force Feedback
48. The Sensory Neurons of Touch10.1016/j.neuron.2013.07.051 ↗
49. Nonlinear Approximation and (Deep) ReLU Networks10.1007/s00365-021-09548-z ↗
50. GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks
51. Improved Adam Optimizer for Deep Neural Networks
52. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
53. SGDR: Stochastic Gradient Descent with Warm Restarts
54. Peripheral thermosensation in mammals10.1038/nrn3784 ↗
55. Hastie · 2009