专题论文

图像美学质量评价技术发展趋势

  • 金鑫 ,
  • 周彬 ,
  • 邹冬青 ,
  • 李晓东 ,
  • 孙红波 ,
  • 吴乐
展开
  • 1. 北京电子科技学院计算机科学与技术系, 北京 100070;
    2. 北京航空航天大学计算机学院, 北京 100191;
    3. 北京市商汤科技开发有限公司, 北京 100084
金鑫,讲师,研究方向为计算美学、虚拟现实、计算机视觉,电子信箱:jinxin@besti.edu.cn

收稿日期: 2018-04-23

  修回日期: 2018-05-03

  网络出版日期: 2018-05-19

基金资助

国家自然科学基金项目(61402021)

Image aesthetic quality assessment: A survey

  • JIN Xin ,
  • ZHOU Bin ,
  • ZOU Dongqing ,
  • LI Xiaodong ,
  • SUN Hongbo ,
  • WU Le
Expand
  • 1. Department of Computer Science and Technology, Beijing Electronics Science and Technology Institute, Beijing 100070, China;
    2. School of Computer Science and Engineering, Beihang University, Beijing 100191, China;
    3. Sense Time, Beijing 100084, China

Received date: 2018-04-23

  Revised date: 2018-05-03

  Online published: 2018-05-19

摘要

图像质量评价是利用计算机模拟人类视觉系统自动评价图像的失真程度,而图像美学质量评价是利用计算机模拟人类对美的感知与认知,自动评价图像的"美感",分析图像在构图、颜色、光影、景深、虚实等美学因素影响下所形成的美感刺激,这是计算美学与计算机视觉、心理学、虚拟现实等领域的学科交叉新方向,其在摄影摄像、电影电视、电子商务、服装设计、平面设计、文稿设计、工业设计、汽车行业、建筑行业、美容美妆等多个行业具有良好的应用前景。本文回顾了图像美学质量评价的发展历史,从基于人工设计美学特征的方法、基于美学特征深度学习的方法、图像美学质量评价的新任务、图像美学质量评价数据集构建等方面综述了该领域关键技术的发展情况,并展望了图像美学质量评价及其关键技术的发展趋势。

本文引用格式

金鑫 , 周彬 , 邹冬青 , 李晓东 , 孙红波 , 吴乐 . 图像美学质量评价技术发展趋势[J]. 科技导报, 2018 , 36(9) : 36 -45 . DOI: 10.3981/j.issn.1000-7857.2018.09.005

Abstract

Image quality assessment (IQA) is to measure the perceived image degradation while image aesthetic quality assessment (IAQA) is to automatically give a review of aesthetics of an image. The composition, color, illumination, depth of field, blurness, sharpness, etc. are analyzed by means of computer to analog the aesthetic perception and cognition of human beings. IAQA is a novel interdisciplinary field of computer aesthetics and computer vision, psychology, and virtual reality. IAQA can be used for photography, movie and television, ecommerce, fashion design, graphic design, document typesetting, industry design, automobile industry, architectural design, beauty makeup, etc. In this article, we give a brief history of IAQA. Then we review the methods based on hand-crafted aesthetic feature design, the methods based on deep aesthetic feature learning, new topics in IAQA, and dataset collection for IAQA. Finally, we give the future directions of IAQA.

参考文献

[1] Tong H, Li M, Zhang H, et al. Classification of digital photos taken by photographers or home users[C]//Proceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing. Heidelberg:Springer-Verlag, 2004:198-205
[2] Ke Y, Tang X, Jing F. The design of high-level features for photo quality assessment[C]//Proceedings of the 19th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2006:419-426.
[3] Datta R, Joshi D, Li J, et al. Studying aesthetics in photographic images using a computational approach[C]//Proceedings of the 9th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2006:288-301.
[4] Luo Y, Tang X. Photo and video quality evaluation:Focusing on the subject[C]//Proceedings of the 10th European Conference on Computer Vision (ECCV). Heidelberg:Springer-Verlag, 2008:386-399.
[5] Wong L K, Low K L. Saliency-enhanced image aesthetics class prediction[C]//Proceedings of the 16th IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2009:997-1000.
[6] Li C, Chen T. Aesthetic visual quality assessment of paintings[J]. IEEE Journal of Selected Topics in Signal Processing, 2009, 3(2):236-252.
[7] Nishiyama M, Okabe T, Sato I, et al. Aesthetic quality classification of photographs based on color harmony[C]//Proceedings of the 24th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2011:33-40.
[8] Dhar S, Ordonez V, Berg T L. High level describable attributes for predicting aesthetics and interestingness[C]//Proceedings of the 24th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2011:1657-1664.
[9] Marchesotti L, Perronnin F, Larlus D, et al. Assessing the aesthetic quality of photographs using generic image descriptors[C]//Proceedings of the 13th IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2011:1784-1791.
[10] Hurter B. The best of photographic lighting techniques and images for digital photographers[M]. 2nd ed. New York:Amherst Media, 2007.
[11] Hunter F, Biver S, Fuqua P. Light:Science and magic:An introduction to photographic lighting[M]. 3rd ed. New York:Focal Press, 2007.
[12] Grey C. Master lighting guide for portrait photographers[M]. New York:Amherst Media, 2004.
[13] Prakel D. Basics photography:Lighting[M]. Switzerland:AVA Publishing, 2007.
[14] Jin X, Zhao M, Chen X, et al. Learning artistic lighting template from portrait photographs[C]//Proceedings of the 11th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2010:101-114.
[15] Chen X, Jin X, Wu H, et al. Learning templates for artistic portrait lighting analysis[J]. IEEE Transactions on Image Processing, 2015, 24(2):608-618.
[16] Luo W, Wang X, Tang X. Content-based photo quality assessment[C]//Proceedings of the 13th IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2011:2206-2213.
[17] Tang X, Luo W, Wang X. Content-based photo quality assessment[J]. IEEE Transactions on Multimedia, 2013, 15(8):1930-1943.
[18] Gray D, Yu K, Xu W, et al. Predicting facial beauty without landmarks[C]//Proceedings of the 11th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2010:434-447.
[19] Li C, Gallagher A, Loui A C, et al. Aesthetic quality assessment of consumer photos with faces[C]//Proceedings of the IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2010:3221-3224.
[20] Khan S S, Vogel D. Evaluating visual aesthetics in photographic portraiture[C]//Proceedings of the 8th Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging. New York:ACM, 2012:55-62.
[21] Lu X, Lin Z, Jin H, et al. RAPID:Rating pictorial aesthetics using deep learning[C]//Proceedings of the ACM International Conference on Multimedia. New York:ACM, 2014:457-466.
[22] Kao Y, Wang C, Huang K. Visual aesthetic quality assessment with a regression model[C]//Proceedings of 2015 IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2015. 1583-1587.
[23] Lu X, Lin Z, Shen X, et al. Deep multi-patch aggregation network for image style, aesthetics, and quality estimation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2015:990-998.
[24] Dong Z, Tian X. Multi-level photo quality assessment with multi-view features[J]. Neurocomputing, 2015, 168:308-319.
[25] Wang W, Zhao M, Wang L, et al. A multi-scene deep learning model for image aesthetic evaluation[J]. Signal Processing Image Communication, 2016, 47(C):511-518.
[26] Kao Y, Huang K, Maybank S. Hierarchical aesthetic quality assessment using deep convolutional neural networks[J]. Signal Processing Image Communication, 2016, 47:500-510.
[27] Kong S, Shen X, Lin Z, et al. Photo aesthetics ranking network with attributes and content adaptation[C]//Proceedings of 14th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2016:662-679.
[28] Jin X, Chi J, Peng S, et al. Deep image aesthetics classification using inception modules and fine-tuning connected layer[C]//Proceedings of the 8th International Conference on Wireless Communications and Signal Processing. Piscataway NJ:IEEE, 2016, doi:10.1109/WCSP.2016.7752571.
[29] Ma S, Liu J, Chen C W. A-Lamp:Adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2017:722-731.
[30] Jin X, Wu L, Li X D, et al. ILGNet:Inception modules with connected local and global features for efficient image aesthetic quality classification using domain adaptation[J]. arXiv.org, 2018, arXiv:1610.02256v3.
[31] Jin B, Segovia M V O, Süsstrunk S. 2016. Image aesthetic predictors based on weighted CNNs[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2016:2291-2295.
[32] Wang Z, Liu D, Chang S, et al. Image aesthetics assessment using deep chatterjee's machine[C]//Proceedings of 2017 International Joint Conference on Neural Networks. Piscataway NJ:IEEE, 2017:941-948.
[33] Hou L, Yu C P, Samaras D. Squared earth mover's distancebased loss for training deep neural networks[J]. arXiv.org, 2017, arXiv:1611.05916
[34] Deng Y, Chen C L, Tang X. Image aesthetic assessment:An experimental survey[J]. IEEE Signal Processing Magazine, 2016, 34(4):80-106.
[35] Murray N, Marchesotti L, Perronnin F. AVA:A large-scale database for aesthetic visual analysis[C]//Proceedings of the 25th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2012:2408-2415.
[36] Wu O, Hu W M, Gao J. Learning to predict the perceived visual quality of photos[C]//Proceedings of 2011 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2011:225-232.
[37] Park T S, Zhang B T. Consensus analysis and modeling of visual aesthetic perception[J]. IEEE Transactions on Affective Computing, 2015, 6(3):272-285.
[38] Kim W H, Choi J H, Lee J S. Subjectivity in aesthetic quality assessment of digital photographs:Analysis of user comments[C]//Proceedings of the 23rd ACM international conference on Multimedia. New York:ACM, 2015:983-986.
[39] Kim W H, Choi J H, Lee J S. Objectivity and subjectivity in aesthetic quality assessment of digital photographs[J]. IEEE Transactions on Affective Computing, 2018, doi:10.1109/TAFFC.2018.2809752.
[40] Jin X, Wu L, Li X, et al. Predicting aesthetic score distribution through cumulative Jensen-Shannon divergence[C]//Proceedings of AAAI Conference on Artificial Intelligence (AAAI). New York:AAAI, 2018.
[41] Chang K Y, Lu K H, Chen C S. Aesthetic critiques generation for photos[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway NJ:IEEE, 2017:3534-3543.
[42] Collomosse J, Tu B, Wilber M, et al. Sketching with Style:Visual search with sketches and aesthetic context[C]//Proceedings of IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2017:2679-2687.
[43] Cui C, Fang H, Deng X, et al. Distribution-oriented aesthetics assessment for image search[C]//Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2017:1013-1016.
[44] Wang W, Shen J. Deep cropping via attention box prediction and aesthetics assessment[C]//2017 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2017:2205-2213.
[45] Ren J, Shen X, Lin Z, et al. Personalized image aesthetics[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2017:638-647.
[46] Schwarz K, Wieschollek P, Lensch H P A. Will people like your image?[J]. arXiv.org, 2018, arXiv:1611.05203.
[47] Wang W S, Yang S, Zhang W S, et al. Neural aesthetic image reviewer[J]. arXiv.org, 2018, arXiv:1802.10240.
[48] Yanulevskaya V, Uijlings J, Bruni E, et al. In the eye of the beholder:Employing statistical analysis and eye tracking for analyzing abstract paintings[C]//Proceedings of the 20th ACM International Conference on Multimedia. New York:ACM, 2012:349-358.
[49] Agrawal A, Premachandran V, Kakarala R. Rating image aesthetics using a crowd sourcing approach[C]//Workshops of the 6th Pacific-Rim Symposium on Image and Video Technology. New York:Springer-Verlag New York Inc., 2013, 8334:24-32.
[50] Datta R, Wang J Z. ACQUINE:Aesthetic quality inference engine-real time automatic ratings of photo aesthetics[C]//Proceedings of the ACM International Conference on Multimedia Information Retrieval. New York:ACM, 2010:421-424.
文章导航

/