专题:学科融合与设计创新

未来人居中的智能美妆:数字化妆镜交互技术及其应用

  • 孙喆 ,
  • 席雪宁 ,
  • 徐迎庆
展开
  • 1. 清华大学美术学院,北京 100084
    2. 清华大学未来实验室,北京 100084
孙喆,硕士研究生,研究方向为智能美妆、计算机视觉、智能硬件,电子信箱:sunz15@tsinghua.org.cn

收稿日期: 2021-11-27

  修回日期: 2022-07-21

  网络出版日期: 2023-05-22

基金资助

国家自然科学基金项目(62172252)

Makeup system in smart home: Interactive technology in digital mirror for makeup and its application

  • SUN Zhe ,
  • XI Xuening ,
  • XU Yingqing
Expand
  • 1. Academy of Arts & Design, Tsinghua University, Beijing 100084, China
    2. The Future Laboratory, Tsinghua University, Beijing 100084, China

Received date: 2021-11-27

  Revised date: 2022-07-21

  Online published: 2023-05-22

摘要

要随着人工智能、物联网等技术的发展,家居智能化成为新趋势。在购物消费、日常娱乐中,美妆行业表现出了巨大的市场潜力,美妆领域的智能化亦成为一个有趣的研究方向。围绕未来人居中的智能美妆系统,从基于镜面的人机交互设计、智能美妆相关算法、数字化妆镜系统相关交互技术3个方面,综述了国内外相关研究进展,分析了未来数字化妆镜系统的应用场景,总结了数字化妆系统的搭建思路与交互方案。

本文引用格式

孙喆 , 席雪宁 , 徐迎庆 . 未来人居中的智能美妆:数字化妆镜交互技术及其应用[J]. 科技导报, 2023 , 41(8) : 53 -64 . DOI: 10.3981/j.issn.1000-7857.2023.08.005

Abstract

Along with the development of technologies such as artificial intelligence(AI) and the internet of things(IoT), here comes the trend of smart home. Makeup industry shows convincing market potential in both consumption and entertainment. Digital makeup has become an interesting research direction. This article focused on makeup system in smart home scenario. The research of related works was divided into three parts: (1) Human-computer interaction design based on mirror; (2) Algorithms related to digital makeup; (3) Hardware technologies in digital mirror for makeup. Based on the survey, application scenarios of the digital makeup mirror system were analyzed and forecasted. This review ought to provide a summary of the development strategy and interface design of digital makeup system for further investigation in this field.

参考文献

[1] Besserer D, Bäurle J, Nikic A, et al. Fitmirror: A smart mirror for positive affect in everyday user morning routines[C]//Proceedings of the Workshop on Multimodal Analyses Enabling Artificial Agents in Human-Machine Interaction. New York: Association for Computing Machinery, 2016: 48-55.
[2] Hippocrate A A E, Luhanga E T, Masashi T, et al. Smart gyms need smart mirrors: Design of a smart gym concept through contextual inquiry[C]//Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. New York: Association for Computing Machinery, 2017: 658-661.
[3] Mohamed A S A, Wahab M N A, Suhaily S S, et al. Smart mirror design powered by raspberry PI[C]//Proceedings of the 2018 Artificial Intelligence and Cloud Computing Conference. New York: Association for Computing Machinery, 2018: 166-173.
[4] Salgian A, Vickerman D, Vassallo D. A smart mirror for music conducting exercises[C]//Proceedings of the on Thematic Workshops of ACM Multimedia 2017. New York: Association for Computing Machinery, 2017: 544-549.
[5] Dang C T, Aslan I, Lingenfelser F, et al. Towards somaesthetic smarthome designs: Exploring potentials and limitations of an affective mirror[C]//Proceedings of the 9th International Conference on the Internet of Things. New York: Association for Computing Machinery, 2019: 1-8.
[6] Chu M, Dalal B, Walendowski A, et al. Countertop responsive mirror: Supporting physical retail shopping for sellers, buyers and companions[C]//Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2010: 2533-2542.
[7] Parlangeli O, Guidi S, Marchigiani E, et al. Shopping online and online design: The role of prospective memory in the use of online product configurators[C]//Proceedings of the 13th Biannual Conference of the Italian SIGCHI Chapter: Designing the Next Interaction. New York: Association for Computing Machinery, 2019: 1-7.
[8] Kim H, Huh B K, Im S H, et al. Finding satisfactory transparency: An empirical study on public transparent displays in a shop context[C]//Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2015: 1151-1156.
[9] Weißker T, Berst A, Hartmann J, et al. The massive mobile multiuser framework: Enabling ad-hoc realtime interaction on public displays with mobile devices[C]//Proceedings of the 5th ACM International Symposium on Pervasive Displays. New York: Association for Computing Machinery, 2016: 168-174.
[10] Newn J, Velloso E, Carter M, et al. Multimodal segmentation on a large interactive tabletop: Extending interaction on horizontal surfaces with gaze[C]//Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces. New York: Association for Computing Machinery, 2016: 251-260.
[11] Hansen T R, Eriksson E, Lykke-Olesen A. Mixed interaction space: Designing for camera based interaction with mobile devices[C]//CHI’05 Extended Abstracts on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2005: 1933-1936.
[12] Bohari U, Chen T J. To draw or not to draw: Recognizing stroke-hover intent in non-instrumented gesturefree mid-air sketching[C]//23rd International Conference on Intelligent User Interfaces. New York: Association for Computing Machinery, 2018: 177-188.
[13] Ishak E W, Feiner S K. Interacting with hidden content using content-aware free-space transparency[C]//Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology. New York: Association for Computing Machinery, 2004: 189-192.
[14] Kubo Y, Takada R, Shizuki B, et al. Exploring context-aware user interfaces for smartphone-smartwatch cross-device interaction[J]. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2017, 1(3): 69:1-69.
[15] Chen X, Marquardt N, Tang A, et al. Extending a mobile device’s interaction space through body-centric interaction[C]//Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services. New York: Association for Computing Machinery, 2012: 151-160.
[16] Loorak M H, Zhou W, Trinh H, et al. Hand-over-face input sensing for interaction with smartphones through the built-in camera[C]//Proceedings of the 21st International Conference on Human-Computer Interaction with Mobile Devices and Services. New York: Association for Computing Machinery, 2019: 1-12.
[17] Mistry P, Maes P. Mouseless: A computer mouse as small as invisible[C]//CHI’11 Extended Abstracts on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2011: 1099-1104.
[18] Lee S S, Chae J, Kim H, et al. Towards more natural digital content manipulation via user freehand gestural interaction in a living room[C]//Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing. New York: Association for Computing Machinery, 2013: 617-626.
[19] Yeo H S, Feng W, Huang M X. WATouCH: Enabling direct input on non-touchscreen using smartwatch’s photoplethysmogram and IMU sensor fusion[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2020: 1-10.
[20] Bostan I, Buruk O T, Canat M, et al. Hands as a controller: User preferences for hand specific on-skin gestures[C]//Proceedings of the 2017 Conference on Designing Interactive Systems. New York: Association for Computing Machinery, 2017: 1123-1134.
[21] Sharma R, Patterh M. Face recognition using face alignment and PCA techniques: A literature survey[J]. IOSR Journal of Computer Engineering (IOSR-JCE), 2015, 17(4): 17-30.
[22] Jin X, Tan X. Face alignment in-the-wild: A survey[J].Computer Vision and Image Understanding, 2017, 162: 1-22.
[23] Wang N, Gao X, Tao D, et al. Facial feature point detection: A comprehensive survey[J]. Neurocomputing, 2018, 275: 50-65.
[24] Tan S, Chen D, Guo C, et al. A robust shape reconstruction method for facial feature point detection[J]. Computational Intelligence and Neuroscience, 2017, 2017: 1-11.
[25] Wu W, Qian C, Yang S, et al. Look at boundary: A boundary-aware face alignment algorithm[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: Association for Computing Machinery, 2018: 2129-2138.
[26] Zhang J, Hu H, Shen G. Joint stacked hourglass network and salient region attention refinement for robust face alignment[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2020, 16(1): 1-18.
[27] Chang H, Lu J, Yu F, et al. Pairedcyclegan: Asymmetric style transfer for applying and removing makeup[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. New York: Association for Computing Machinery, 2018: 40-48.
[28] Bao R, Yu H, Liu S, et al. Automatic makeup based on generative adversarial nets[C]//Proceedings of the 10th International Conference on Internet Multimedia Computing and Service. New York: Association for Computing Machinery, 2018: 1-5.
[29] Li Y, Huang H, Cao J, et al. Disentangled representation learning of makeup portraits in the wild[J]. International Journal of Computer Vision, 2020, 128(8-9): 2166-2184.
[30] Li T, Qian R, Dong C, et al. BeautyGAN: Instance-level facial makeup transfer with deep generative adversarial network[C]//Proceedings of the 26th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2018: 645-653.
[31] Park J, Kim H, Ji S, et al. An automatic virtual makeup scheme based on personal color analysis[C]//Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication. New York: Association for Computing Machinery, 2018: 1-7.
[32] Evangelista B, Meshkin H, Kim H, et al. Realistic AR makeup over diverse skin tones on mobile[C]//Siggraph Asia 2018 Posters. New York: Association for Computing Machinery, 2018: 1-2.
[33] Liu L, Xing J, Liu S, et al. Wow! You are so beautiful today[J]. ACM Transactions on Multimedia Computing, Communications, and Applications, 2014, 11(s1): 1-20.
[34] Ou X, Liu S, Cao X, et al. Beauty eMakeup: A deep makeup transfer system[C]//Proceedings of the 24th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2016: 701-702.
[35] Nguyen T V, Liu L. Smart mirror: Intelligent makeup recommendation and synthesis[C]//Proceedings of the 25th ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2017: 1253-1254.
[36] 朱淼良, 姚远, 蒋云良 . 增强现实综述[J]. 中国图象图形学报, 2004(7): 3-10.
[37] Bermano A H, Billeter M, Iwai D, et al. Makeup Lamps: Live augmentation of human faces via projection[J]. Computer Graphics Forum, 2017, 36(2): 311-323.
[38] Treepong B, Mitake H, Hasegawa S. Makeup creativity enhancement with an augmented reality face makeup system[J]. Computers in Entertainment, 2018, 16(4): 6-17.
[39] Nakagawa M, Tsukada K, Siio I. Smart makeup system: Supporting makeup using lifelog sharing[C]//Proceedings of the 13th International Conference on Ubiquitous Computing. New York: Association for Computing Machinery, 2011: 483-484.
[40] Truong A, Chi P, Salesin D, et al. Automatic generation of two-level hierarchical tutorials from instructional makeup videos[C]//Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. Yokohama: ACM, 2021: 1-16.
[41] Hung M H, Yang J, Hsieh C H. A new virtual makeup system based on golden sample search[C]//Proceedings of the 2020 4th International Conference on Electronic Information Technology and Computer Engineering. Xiamen: ACM, 2020: 350-354.
[42] Rahman A S M M, Tran T T, Hossain S A, et al. Augmented rendering of makeup features in a smart interactive mirror system for decision support in cosmetic products selection[C]//2010 IEEE/ACM 14th International Symposium on Distributed Simulation and Real Time Applications. New York: IEEE, 2010: 203-206.
[43] Nishimura A, Siio I. IMake: Eye makeup design generator[C]//Proceedings of the 11th Conference on Advances in Computer Entertainment Technology. New York: Association for Computing Machinery, 2014: 1-6.
[44] Beaudouin-Lafon M. Designing interaction, not interfaces[C]//Proceedings of the Working Conference on Advanced Visual Interfaces. New York: Association for Computing Machinery, 2004: 15-22.
[45] Edelberg J, Kilrain J. Design systems: Consistency, efficiency & collaboration in creating digital products[C]//Proceedings of the 38th ACM International Conference on Design of Communication. New York: Association for Computing Machinery, 2020: 1-3.
[46] Johnston V, Black M, Wallace J, et al. A framework for the development of a dynamic adaptive intelligent user interface to enhance the user experience[C]//Proceedings of the 31st European Conference on Cognitive Ergonomics. New York: Association for Computing Machinery, 2019: 32-35.
[47] Jacob R J K, Girouard A, Hirshfield L M, et al. Reality-based interaction: Unifying the new generation of interaction styles[C]//CHI’07 Extended Abstracts on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2007: 2465-2470.
[48] O’hara K, Harper R, Mentis H, et al. On the natural⁃ness of touchless: Putting the“interaction”back into NUI[J]. ACM Transactions on Computer-Human Interaction, 2013, 20(1): 5-25.
[49] Lenz E, Hassenzahl M, Diefenbach S. How performing an activity makes meaning[C]//Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. New York: Association for Computing Machinery, 2019: 1-6.
[50] Catarci T, Amendola M, Bertacchini F, et al. Digital interaction: Where are we going? [C]//Proceedings of the 2018 International Conference on Advanced Visual Interfaces. New York: Association for Computing Machinery, 2018: 1-5.
文章导航

/