专题:学科融合与设计创新

无障碍设计研究:面向听障人群的振动触觉音乐体验

  • 王韫 ,
  • 李子晋
展开
  • 1. 北京航空航天大学机械工程及自动化学院,北京 100191
    2. 清华大学美术学院,北京 100084
    3. 清华大学未来实验室,北京 100084
    4. 中央音乐学院音乐人工智能与音乐信息科技系,北京 100031
王韫,讲师,研究方向为感知觉体验设计、跨学科设计思维教育,电子信箱:wang_yun@buaa.edu.cn

收稿日期: 2022-03-15

  修回日期: 2022-07-19

  网络出版日期: 2023-05-22

基金资助

文化和旅游部重点实验室项目(2022DMKLB003)

ccessibility design research: Vibrotactile music experience for hearing impaired people

  • WANG Yun ,
  • LI Zijin
Expand
  • 1. School of Mechanical Engineering and Automation, Beihang University, Beijing 100191, China
    2. Academy of Arts & Design, Tsinghua University, Beijing 100084, China
    3. The Future Laboratory, Tsinghua University, Beijing 100084, China
    4. Department of Music AI and Information Technology, Central Conservatory of Music, Beijing 100031, China

Received date: 2022-03-15

  Revised date: 2022-07-19

  Online published: 2023-05-22

摘要

虽然听觉通道受损,聋人群体也有欣赏音乐的需求、能力和权利。当前,现代化听觉辅助技术仍然无法很好地满足听障人群对于听觉艺术的感知和欣赏需求,不利于这一群体的社会交往和自我发展。聚焦听障人群的音乐活动需求和偏好,提出通过触觉来感受声波的振动是聋人参与音乐活动的常见形式。综述了触觉刺激对听觉信息的表征优势,振动触觉的音乐表达能力、局限性和设计方法,并梳理了可穿戴和静态振动触觉音乐设备的研究发展。触觉具备认知负担小、时序性强、自然感强、沉浸感强等优势,通过感官代偿设计,在补偿听觉损失、增益听觉感知的同时,还可以承载一定的情感传达作用。然而,振动触觉也具有分辨率较低、难以感知高频段信息等局限性。展望了未来的体验设计研发的方向和趋势。

本文引用格式

王韫 , 李子晋 . 无障碍设计研究:面向听障人群的振动触觉音乐体验[J]. 科技导报, 2023 , 41(8) : 74 -82 . DOI: 10.3981/j.issn.1000-7857.2023.08.007

Abstract

Hearing impaired people have the need, ability and right to enjoy music, however modern hearing aid technologies still unable to meet their needs for music perception and appreciation, which is not conducive to their social interaction and self-development. This paper firstly focuses on the needs and preferences of hearing impaired people in music activities, and emphasizes that the common way for deaf people to participate in music activities is to feel the vibration of sound waves through tactile channel. Secondly, it summarizes the advantages of tactile stimulation in representing auditory information, as well as the expressiveness competence, limitations and design methods of vibrotactile music, and reviews related works of wearable and static vibrotactile music devices. Tactile channel has advantages in several aspects such as cognitive load, temporal sequence, naturalness and immersiveness etc, not only it can compensate for hearing loss and enhance hearing perception, but also carry emotional communication. It also has limitations such as low resolution and difficulty in conveying high-frequency information. Finally, this paper proposes several research directions of UX design in the future, to provide reference for researchers in related fields.

参考文献

[1] World Health Organization. World report on hearing[EB/OL]. (2021-03-03) [2022-03-25]. https://www. who. int/news-room/events/detail/2021/03/03/default-calendar/launch-of-the-world-report-on-hearing.
[2] Gugenheimer J, Plaumann K, Schaub F, et al. The impact of assistive technology on communication quality between deaf and hearing individuals[C]//Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing(CSCW'17), New York: ACM, 2017: 669-682.
[3] 陈四海, 魏姗 . 辅助技术的社会伦理挑战及应对[J]. 伦理学研究, 2020(2): 114-120.
[4] Evelyn Glennie[EB/OL]. [2022-03-25]. https://www. evelyn.co.uk.
[5] Mandy Harvey[EB/OL]. [2022-03-25]. https://mandyharveymusic.com.
[6] Troi Lee[EB/OL]. [2022-03-25]. https://www. deafrave.com.
[7] Sean Forbes[EB/OL]. [2022-03-25]. https://www.deafandloud.com/bio.
[8] Fulford R, Ginsborg J, Goldbart J. Learning not to listen: The experiences of musicians with hearing impairments [J]. Music Education Research, 2011(4): 447-464.
[9] Maler A. Music and deafness in the nineteenth-century U.S. imagination[J]. Journal of the Society for American Music, 2022(2): 184-205.
[10] 连赟 . 中国特殊音乐教育: 历史与现状研究[D]. 南京:南京艺术学院, 2010.
[11] 中华人民共和国教育部 . 聋校义务教育数学课程标准(2016年版)[S]. 北京: 北京师范大学, 2018.
[12] 杨晴 . 听障学生节奏感的训练与培养[J]. 现代特殊教育, 2019(5): 68-71.
[13] Fletcher M D. Can haptic stimulation enhance music perception in hearing-impaired listeners[J]. Frontiers in Neuroscience, 2021(15): e723877.
[14] Looi V, Rutledge K, Prvan T. Music appreciation of adult hearing aid users and the impact of different levels of hearing loss[J]. Ear Hear, 2019(3): 529-544.
[15] Jacobson D, Haptic or touch-based knowledge[J]. International Encyclopedia of Human Geography, 2020, doi: 10.1016/B978-008044910-4.00451-X.
[16] Lakatos P, Chen C M, O'connell M N, et al. Neuronal oscillations and multisensory interaction in primary auditory cortex[J]. Neuron, 2006(2): 279-292.
[17] Young G W, Murphy D, Weeter J, Haptics in music:The effects of vibrotactile stimulus in low frequency auditory difference detection tasks[J]. IEEE Transactions on Haptics, 2017(1): 135-139.
[18] Levänen S, Hamdorf D. Feeling vibrations: Enhanced tactile sensitivity in congenitally deaf humans[J]. Neuroscience Letters. 2001(1): 75-77.
[19] Haynes A C, Lawry J, Kent C, et al. Feel music: Enriching our emotive experience of music through audio-tactile mappings[J]. Multimodal Technologies and Interaction, 2021(6): 29.
[20] Good A, Reed M J, Russo F A. Compensatory plasticity in the deaf brain: Effects on perception of music[J]. Brain sciences, 2014(4): 560-574.
[21] 路璐, 田丰, 戴国忠, 等 . 融合触、听、视觉的多通道认知和交互模型[J]. 计算机辅助设计与图形学学报, 2014(4): 654-661.
[22] Birnbaum D, Wanderley M. A systematic approach to musical vibrotactile feedback[C]//Proceedings of the International Computer Music Conference, San Francisco:ICMC, 2007.
[23] Musical Vibration[EB/OL]. [2022-03-25]. http://www.musicalvibrations.com.
[24] Hopkins C, Maté -Cid S, Fulford R et al. Vibrotactile presentation of musical notes to the glabrous skin for adults with normal hearing or a hearing impairment: Thresholds, dynamic range and high-frequency perception[J]. PLoS One, 2016(5): e0155807.
[25] Mazzoni A, Bryan-Kinns N. How does it feel like? An exploratory study of a prototype system to convey emotion through haptic wearable devices[C]//Proceedings of the International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN). New York: ACM,2015: 64-68.
[26] Sharp A, Bacon B A, Champoux F. Enhanced tactile identification of musical emotion in the deaf[J]. Experimental Brain Research, 2020(5): 1229-1236.
[27] 熊兴福, 李姝瑶 . 感官代偿设计在产品中的应用[J]. 包装工程, 2009(10): 131-132, 139.
[28] Remache-Vinueza B, Trujillo-León A, Zapata M, et al. Audio-tactile rendering: A review on technology and methods to convey musical information through the sense of touch[J]. Sensors, 2021(19): 6575.
[29] Gunther E, Davenport G, O'Modhrain S. Cutaneous grooves: Composing for the sense of touch[C]//Proceedings of the 2002 conference on New interfaces for musical expression (NIME '02). New York: ACM, 2002: 1-6.
[30] Karam M, Russo F, Branje C, et al. Towards a model human cochlea: Sensory substitution for crossmodal audio-tactile displays[C]//Proceedings of Graphics Interface 2008 (GI '08). Canada: Canadian Information Processing Society, 2008: 267-274.
[31] Hattwick I, Franco I, Giordano M, et al. Composition techniques for the ilinx vibrotactile garment[C]//Proceedings of the International Computer Music Conference, Denton: ICMC, 2015: 420-423.
[32] Vest: A sensory substitution neuroscience project[EB/OL]. (2016-04-01)[2022-03-25]. https://www.kickstarter. com/projects/324375300/vest-a-sensory-substitution-neuroscience-project.
[33] Hashizume S, Sakamoto S, Suzuki K, et al. Livejacket: Wearable music experience device with multiple speakers[C]//Proceedings of the International Conference on Distributed, Ambient, and Pervasive Interactions, Cham, Switzerland: Springer, 2018: 359-371.
[34] SubPac M2[EB/OL]. [2022-03-25]. https://subpac. com/subpac-m2.
[35] Trivedi U, Alqasemi R, Dubey R V. Wearable musical haptic sleeves for people with hearing impairment[C]//Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments. New York: ACM, 2019: 146-151.
[36] La Versa B, Diamanti L, Peruzzi I, et al. Muvib: Music and vibration[C]//Proceedings of the 2014 ACM International Symposium on Wearable Computers Adjunct Program—ISWC'14 Adjunct. New York: ACM, 2014: 65-70.
[37] Petry B, Illandara T, Nanayakkara S. MuSS-bits: Sensordisplay blocks for deaf people to explore musical sounds [C]//Proceedings of the 28th Australian Conference on Computer-Human Interaction (OzCHI '16). New York: ACM, 2016: 72-80.
[38] Yamazaki Y, Mitake H, Oda R, et al. Hapbeat: Single DOF wide range wearable haptic display[C]//Proceedings of the SIGGRAPH Emerging Technologies (SIGGRAPH '17). New York: ACM, 2017: 1-2.
[39] M: NI(Vibrotextile™)[EB/OL]. [2022-03-25]. https://www.notimpossible.com/projects/music-not-impossible.
[40] 杜熙茹, 谷海玲, 王巧珍, 等 . 听障学生音乐节拍训练无线控制系统设计[J]. 电子设计工程, 2018(20): 178-182.
[41] Darrow A A. The effect of vibrotactile stimuli via the SOMATRONTM on the identification of pitch change by hearing impaired children[J]. Journal of Music Therapy. 1992(29): 103-112.
[42] Nanayakkara S, Taylor E, Wyse L, et al. An enhanced musical experience for the deaf: Design and evaluation of a music display and a haptic chair[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). New York: ACM, 2009: 337-346.
[43] Maria K, Carmen B, Gabe N, et al. The emoti-chair: An interactive tactile music exhibit[C]//Proceedings of the CHI Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). New York: ACM, 2010: 3069-3074.
[44] Jack R, Mcpherson A, Stockman T. Designing tactile musical devices with and for deaf users: A case study[C]//Proceedings of the International Conference on the Multimedia Experience of Music. Sheffield: ICMEM, 2015:1-7.
[45] Hayes L. Skin music (2012): An audio-haptic composition for ears and body[C]//Proceedings of the SIGCHI Conference on Creativity and Cognition (C&C '15). New York: ACM, 2015: 359-360. 
[46] Alves A F, Lima B F, Candido L S A, et al. Auris System: Providing vibrotactile feedback for hearing impaired population[J]. BioMed Research International.2017(1): 1-9.
[47] Fontana F, Camponogara I, Cesari P, et al. An Exploration on whole-body and foot-based vibrotactile sensitivity to melodic consonance[C]//Proceedings of the International Conference on Systems, Man, and Cybernetics, Hamburg, Germany: IEEE, 2016: 143-150.
[48] Eitan Z, Rothschild I. How music touches: Musical parameters and listeners' audio-tactile metaphorical mappings[J]. Psychology of Music, 2011, 39(4): 449-467.
文章导航

/