Nondestructive identification of softness via bioinspired multisensory electronic skins integrated on a robotic hand
Source abstract
Abstract
Abstract Tactile sensing is essentially required for dexterous manipulation in robotic applications. Mimicking human perception of softness identification in a non-invasive fashion, thus achieving satisfactory interaction with fragile objects remains a grand challenge. Here, a scatheless measuring methodology based on the multisensory electronic skins to quantify the elastic coefficient of soft materials is reported. This recognition approach lies in the preliminary classification of softness by piezoelectric signals with a modified machine learning algorithm, contributing to an appropriate contact force assignment for subsequent quantitative measurements via strain sensing feedback. The integration of multifunctional sensing system allows the manipulator to hold capabilities of self-sensing and adaptive grasping motility in response to objects with the various softness (i.e., kPa-MPa). As a proof-of-concept demonstration, the biomimetic manipulator cooperates with the robotic arm to realize the intelligent sorting of oranges varying in freshness, paving the way for the development of microsurgery robots, human-machine interfacing, and advanced prosthetics.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Advanced Sensor and Energy Harvesting Materials · Tactile and Sensory Interactions · Interactive and Immersive Displays
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (66)
Show source-provided references
1. Tian, L. et al. Large-area MRI-compatible epidermal electronic interfaces for prosthetic control and cognitive monitoring. Nat. Biomed. Eng. 3, 194–205 (2019).10.1038/s41551-019-0347-x ↗
2. Chortos, A., Liu, J. & Bao, Z. Pursuing prosthetic electronic skin. Nat. Mater. 15, 937–950 (2016).10.1038/nmat4671 ↗
3. Delmas, P., Hao, J. & Rodat-Despoix, L. Molecular mechanisms of mechanotransduction in mammalian sensory neurons. Nat. Rev. Neurosci. 12, 139–153 (2011).10.1038/nrn2993 ↗
4. Lai, Y. et al. Actively perceiving and responsive soft robots enabled by self-powered, highly extensible, and highly sensitive triboelectric proximity- and pressure-sensing skins. Adv. Mater. 30, 1801114 (2018).10.1002/adma.201801114 ↗
5. Wang, Z. et al. Signal filtering enabled by spike voltage‐dependent plasticity in metalloporphyrin‐based memristors. Adv. Mater. 33, 2104370 (2021).10.1002/adma.202104370 ↗
6. Hartmann, F., Drack, M. & Kaltenbrunner, M. Meant to merge: Fabrication of stretchy electronics for robotics. Sci. Robot. 3, eaat9091 (2018).10.1126/scirobotics.aat9091 ↗
7. Wang, C. et al. Soft ultrathin electronics innervated adaptive fully soft robots. Adv. Mater. 30, 1706695 (2018).10.1002/adma.201706695 ↗
8. Lee, Y. et al. Stretchable organic optoelectronic sensorimotor synapse. Sci. Adv. 4, eaat7387 (2018).10.1126/sciadv.aat7387 ↗
9. Kim, J. et al. Stretchable silicon nanoribbon electronics for skin prosthesis. Nat. Commun. 5, 5747 (2014).10.1038/ncomms6747 ↗
10. Yang, Y. et al. A laser-engraved wearable sensor for sensitive detection of uric acid and tyrosine in sweat. Nat. Biotechnol. 38, 217–224 (2020).10.1038/s41587-019-0321-x ↗
11. Wang, Y. et al. All-weather, natural silent speech recognition via machine-learning-assisted tattoo-like electronics. npj Flex. Electron. 5, 20 (2021).10.1038/s41528-021-00119-7 ↗
12. Kaltenbrunner, M. et al. An ultra-lightweight design for imperceptible plastic electronics. Nature 499, 458–463 (2013).10.1038/nature12314 ↗
13. Ning, X. et al. Mechanically active materials in three-dimensional mesostructures. Sci. Adv. 4, eaat8313 (2018).10.1126/sciadv.aat8313 ↗
14. He, K. et al. An artificial somatic reflex arc. Adv. Mater. 32, 1905399 (2020).10.1002/adma.201905399 ↗
15. Wang, R. et al. Tensile and torsional elastomer fiber artificial muscle by entropic elasticity with thermo-piezoresistive sensing of strain and rotation by a single electric signal. Mater. Horiz. 7, 3305–3315 (2020).10.1039/D0MH01003K ↗
16. Ha, M. et al. Reconfigurable magnetic origami actuators with on-board sensing for guided assembly. Adv. Mater. 33, 2008751 (2021).10.1002/adma.202008751 ↗
17. Parida, K., Bhavanasi, V., Kumar, V., Bendi, R. & Lee, P. Self-powered pressure sensor for ultra-wide range pressure detection. Nano Res. 10, 3557–3570 (2017).10.1007/s12274-017-1567-6 ↗
18. Jang, K. et al. Soft network composite materials with deterministic and bio-inspired designs. Nat. Commun. 6, 6566 (2015).10.1038/ncomms7566 ↗
19. Shi, C. et al. Heterogeneous integration of rigid, soft, and liquid materials for self-healable, recyclable, and reconfigurable wearable electronics. Sci. Adv. 6, eabd0202 (2020).10.1126/sciadv.abd0202 ↗
20. Ha, T. et al. A chest-laminated ultrathin and stretchable e-tattoo for the measurement of electrocardiogram, seismocardiogram, and cardiac time intervals. Adv. Sci. 6, 1900290 (2019).10.1002/advs.201900290 ↗
21. Wang, C. et al. Programmable and scalable transfer printing with high reliability and efficiency for flexible inorganic electronics. Sci. Adv. 6, eabb2393 (2020).10.1126/sciadv.abb2393 ↗
22. Zhou, H. et al. Stretchable piezoelectric energy harvesters and self-powered sensors for wearable and implantable devices. Biosens. Bioelectron. 168, 112569 (2020).10.1016/j.bios.2020.112569 ↗
23. Zhang, Y. et al. Climbing-inspired twining electrodes using shape memory for peripheral nerve stimulation and recording. Sci. Adv. 5, eaaw1066 (2019).10.1126/sciadv.aaw1066 ↗
24. Zhang, C. et al. 3D printed, solid-state conductive ionoelastomer as a generic building block for tactile applications. Adv. Mater. 34, e2105996 (2021).10.1002/adma.202105996 ↗
25. Qiu, Y. et al. Bioinspired, multifunctional dual-mode pressure sensors as electronic skin for decoding complex loading processes and human motions. Nano Energy 78, 105337 (2020).10.1016/j.nanoen.2020.105337 ↗
26. Liu, Z. et al. An ultrasensitive contact lens sensor based on self‐assembly graphene for continuous intraocular pressure monitoring. Adv. Funct. Mater. 31, 2010991 (2021).10.1002/adfm.202010991 ↗
27. Zhai, Z., Wang, Y., Lin, K., Wu, L. & Jiang, H. In situ stiffness manipulation using elegant curved origami. Sci. Adv. 6, eabe2000 (2020).10.1126/sciadv.abe2000 ↗
28. Wang, C. et al. Stretchable, multifunctional epidermal sensor patch for surface electromyography and strain measurements. Adv. Intell. Syst. 3, 2100031 (2021).10.1002/aisy.202100031 ↗
29. Han, X. et al. Flexible polymer transducers for dynamic recognizing physiological signals. Adv. Funct. Mater. 26, 3640–3648 (2016).10.1002/adfm.201600008 ↗
30. Wang, F. et al. Flexible doppler ultrasound device for the monitoring of blood flow velocity. Sci. Adv. 7, eabi9283 (2021).10.1126/sciadv.abi9283 ↗
31. Qu, C. et al. Bioinspired flexible volatile organic compounds sensor based on dynamic surface wrinkling with dual-signal response. Small 15, e1900216 (2019).10.1002/smll.201900216 ↗
32. Su, Y., Dagdeviren, C. & Li, R. Measured output voltages of piezoelectric devices depend on the resistance of voltmeter. Adv. Funct. Mater. 25, 5320–5325 (2015).10.1002/adfm.201502280 ↗
33. Qiu, Y. et al. A biomimetic drosera capensis with adaptive decision‐predation behavior based on multifunctional sensing and fast actuating capability. Adv. Funct. Mater. 32, 2110296 (2022).10.1002/adfm.202110296 ↗
34. Jin, Y. et al. Identifying human body states by using a flexible integrated sensor. npj Flex. Electron. 4, 28 (2020).10.1038/s41528-020-00090-9 ↗
35. Wang, Q. et al. Self‐healable multifunctional electronic tattoos based on silk and graphene. Adv. Funct. Mater. 29, 1808695 (2019).10.1002/adfm.201808695 ↗
36. Qiu, Y. et al. The frequency-response behaviour of flexible piezoelectric devices for detecting the magnitude and loading rate of stimuli. J. Mater. Chem. C. 9, 584–594 (2021).10.1039/D0TC02949A ↗
37. Bai, N. et al. Graded intrafillable architecture-based iontronic pressure sensor with ultra-broad-range high sensitivity. Nat. Commun. 11, 209 (2020).10.1038/s41467-019-14054-9 ↗
38. Wang, X. et al. Somatosensory, light-driven, thin-film robots capable of integrated perception and motility. Adv. Mater. 32, 2000351 (2020).10.1002/adma.202000351 ↗
39. Cheng, L. et al. A highly sensitive piezoresistive sensor with interlocked graphene microarrays for meticulous monitoring of human motions. J. Mater. Chem. C. 8, 11525–11531 (2020).10.1039/D0TC02539A ↗
40. Li, S. et al. Contact-resistance-free stretchable strain sensors with high repeatability and linearity. ACS Nano 16, 541–553 (2022).10.1021/acsnano.1c07645 ↗
41. Zhao, Y. et al. Somatosensory actuator based on stretchable conductive photothermally responsive hydrogel. Sci. Robot. 6, eabd5483 (2021).10.1126/scirobotics.abd5483 ↗
42. Li, D., Yang, S., Chen, X., Lai, W. & Huang, W. 3D Wearable fabric‐based micro‐supercapacitors with ultra‐high areal capacitance. Adv. Funct. Mater. 31, 2107484 (2021).10.1002/adfm.202107484 ↗
43. Zhu, M. et al. Haptic-feedback smart glove as a creative human-machine interface (HMI) for virtual/augmented reality applications. Sci. Adv. 6, eaaz8693 (2020).10.1126/sciadv.aaz8693 ↗
44. Dhong, C. et al. Role of indentation depth and contact area on human perception of softness for haptic interfaces. Sci. Adv. 5, eaaw8845 (2019).10.1126/sciadv.aaw8845 ↗
45. Saal, H., Delhaye, B., Rayhaun, B. & Bensmaia, S. Simulating tactile signals from the whole hand with millisecond precision. Proc. Natl Acad. Sci. Usa. 114, E5693–E5702 (2017).10.1073/pnas.1704856114 ↗
46. Wang, Y. et al. Hierarchically patterned self-powered sensors for multifunctional tactile sensing. Sci. Adv. 6, eabb9083 (2020).10.1126/sciadv.abb9083 ↗
47. Ma, X. et al. Sensor embedded soft fingertip for precise manipulation and softness recognition. IEEE Robot. Autom. Lett. 6, 8734–8741 (2021).10.1109/LRA.2021.3115156 ↗
48. Pang, K. et al. Hydroplastic foaming of graphene aerogels and artificially intelligent tactile sensors. Sci. Adv. 6, eabd4045 (2020).10.1126/sciadv.abd4045 ↗
49. Sundaram, S. et al. Learning the signatures of the human grasp using a scalable tactile glove. Nature 569, 698–702 (2019).10.1038/s41586-019-1234-z ↗
50. Chun, S. et al. Self-powered pressure- and vibration-sensitive tactile sensors for learning technique-based neural finger skin. Nano Lett. 19, 3305–3312 (2019).10.1021/acs.nanolett.9b00922 ↗
51. Lu, H., Hong, Y., Yang, Y., Yang, Z. & Shen, Y. Battery-less soft millirobot that can move, sense, and communicate remotely by coupling the magnetic and piezoelectric effects. Adv. Sci. 7, 2000069 (2020).10.1002/advs.202000069 ↗
52. Yu, X. et al. Needle-shaped ultrathin piezoelectric microsystem for guided tissue targeting via mechanical sensing. Nat. Biomed. Eng. 2, 165–172 (2018).10.1038/s41551-018-0201-6 ↗
53. Kim, K. et al. Tactile avatar: tactile sensing system mimicking human tactile cognition. Adv. Sci. 8, 2002362 (2021).10.1002/advs.202002362 ↗
54. Yan, Y. et al. Soft magnetic skin for super-resolution tactile sensing with force self-decoupling. Sci. Robot. 6, eabc8801 (2021).10.1126/scirobotics.abc8801 ↗
55. Cui, Z. et al. Haptically quantifying young’s modulus of soft materials using a self-locked stretchable strain sensor. Adv. Mater. e2104078 (2021).10.1002/adma.202104078 ↗
56. Song, E. et al. Miniaturized electromechanical devices for the characterization of the biomechanics of deep tissue. Nat. Biomed. Eng. 5, 759–771 (2021).10.1038/s41551-021-00723-y ↗
57. Beker, L. et al. A bioinspired stretchable membrane-based compliance sensor. Proc. Natl Acad. Sci. Usa. 117, 11314–11320 (2020).10.1073/pnas.1909532117 ↗
58. Lee, G. et al. Fingerpad-inspired multimodal electronic skin for material discrimination and texture recognition. Adv. Sci. 8, 2002606 (2021).10.1002/advs.202002606 ↗
59. Li, G., Liu, S., Wang, L. & Zhu, R. Skin-inspired quadruple tactile sensors integrated on a robot hand enable object recognition. Sci. Robot. 5, eabc8134 (2020).10.1126/scirobotics.abc8134 ↗
60. Sengupta, D., Romano, J. & Kottapalli, A. G. P. Electrospun bundled carbon nanofibers for skin-inspired tactile sensing, proprioception and gesture tracking applications. npj Flex. Electron. 5, 29 (2021).10.1038/s41528-021-00126-8 ↗
61. Johansson, R. & Flanagan, J. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10, 345–359 (2009).10.1038/nrn2621 ↗
62. Park, J., Kim, M., Lee, Y., Lee, H. & Ko, H. Fingertip skin-inspired microstructured ferroelectric skins discriminate static/dynamic pressure and temperature stimuli. Sci. Adv. 1, e1500661 (2015).10.1126/sciadv.1500661 ↗
63. Boutry, C. et al. A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci. Robot. 3, eaau6914 (2018).10.1126/scirobotics.aau6914 ↗
64. Chun, K., Son, Y., Jeon, E., Lee, S. & Han, C. A self-powered sensor mimicking slow- and fast-adapting cutaneous mechanoreceptors. Adv. Mater. 30, 1706299 (2018).10.1002/adma.201706299 ↗
65. Zhang, Z. et al. Deep learning-enabled triboelectric smart socks for IoT-based gait analysis and VR applications. npj Flex. Electron. 4, 29 (2020).10.1038/s41528-020-00092-7 ↗
66. Mahler, J. et al. Learning ambidextrous robot grasping policies. Sci. Robot. 4, eaau4984 (2019).10.1126/scirobotics.aau4984 ↗