Spescial Issues

Image aesthetic quality assessment: A survey

  • JIN Xin ,
  • ZHOU Bin ,
  • ZOU Dongqing ,
  • LI Xiaodong ,
  • SUN Hongbo ,
  • WU Le
Expand
  • 1. Department of Computer Science and Technology, Beijing Electronics Science and Technology Institute, Beijing 100070, China;
    2. School of Computer Science and Engineering, Beihang University, Beijing 100191, China;
    3. Sense Time, Beijing 100084, China

Received date: 2018-04-23

  Revised date: 2018-05-03

  Online published: 2018-05-19

Abstract

Image quality assessment (IQA) is to measure the perceived image degradation while image aesthetic quality assessment (IAQA) is to automatically give a review of aesthetics of an image. The composition, color, illumination, depth of field, blurness, sharpness, etc. are analyzed by means of computer to analog the aesthetic perception and cognition of human beings. IAQA is a novel interdisciplinary field of computer aesthetics and computer vision, psychology, and virtual reality. IAQA can be used for photography, movie and television, ecommerce, fashion design, graphic design, document typesetting, industry design, automobile industry, architectural design, beauty makeup, etc. In this article, we give a brief history of IAQA. Then we review the methods based on hand-crafted aesthetic feature design, the methods based on deep aesthetic feature learning, new topics in IAQA, and dataset collection for IAQA. Finally, we give the future directions of IAQA.

Cite this article

JIN Xin , ZHOU Bin , ZOU Dongqing , LI Xiaodong , SUN Hongbo , WU Le . Image aesthetic quality assessment: A survey[J]. Science & Technology Review, 2018 , 36(9) : 36 -45 . DOI: 10.3981/j.issn.1000-7857.2018.09.005

References

[1] Tong H, Li M, Zhang H, et al. Classification of digital photos taken by photographers or home users[C]//Proceedings of the 5th Pacific Rim Conference on Advances in Multimedia Information Processing. Heidelberg:Springer-Verlag, 2004:198-205
[2] Ke Y, Tang X, Jing F. The design of high-level features for photo quality assessment[C]//Proceedings of the 19th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2006:419-426.
[3] Datta R, Joshi D, Li J, et al. Studying aesthetics in photographic images using a computational approach[C]//Proceedings of the 9th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2006:288-301.
[4] Luo Y, Tang X. Photo and video quality evaluation:Focusing on the subject[C]//Proceedings of the 10th European Conference on Computer Vision (ECCV). Heidelberg:Springer-Verlag, 2008:386-399.
[5] Wong L K, Low K L. Saliency-enhanced image aesthetics class prediction[C]//Proceedings of the 16th IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2009:997-1000.
[6] Li C, Chen T. Aesthetic visual quality assessment of paintings[J]. IEEE Journal of Selected Topics in Signal Processing, 2009, 3(2):236-252.
[7] Nishiyama M, Okabe T, Sato I, et al. Aesthetic quality classification of photographs based on color harmony[C]//Proceedings of the 24th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2011:33-40.
[8] Dhar S, Ordonez V, Berg T L. High level describable attributes for predicting aesthetics and interestingness[C]//Proceedings of the 24th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2011:1657-1664.
[9] Marchesotti L, Perronnin F, Larlus D, et al. Assessing the aesthetic quality of photographs using generic image descriptors[C]//Proceedings of the 13th IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2011:1784-1791.
[10] Hurter B. The best of photographic lighting techniques and images for digital photographers[M]. 2nd ed. New York:Amherst Media, 2007.
[11] Hunter F, Biver S, Fuqua P. Light:Science and magic:An introduction to photographic lighting[M]. 3rd ed. New York:Focal Press, 2007.
[12] Grey C. Master lighting guide for portrait photographers[M]. New York:Amherst Media, 2004.
[13] Prakel D. Basics photography:Lighting[M]. Switzerland:AVA Publishing, 2007.
[14] Jin X, Zhao M, Chen X, et al. Learning artistic lighting template from portrait photographs[C]//Proceedings of the 11th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2010:101-114.
[15] Chen X, Jin X, Wu H, et al. Learning templates for artistic portrait lighting analysis[J]. IEEE Transactions on Image Processing, 2015, 24(2):608-618.
[16] Luo W, Wang X, Tang X. Content-based photo quality assessment[C]//Proceedings of the 13th IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2011:2206-2213.
[17] Tang X, Luo W, Wang X. Content-based photo quality assessment[J]. IEEE Transactions on Multimedia, 2013, 15(8):1930-1943.
[18] Gray D, Yu K, Xu W, et al. Predicting facial beauty without landmarks[C]//Proceedings of the 11th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2010:434-447.
[19] Li C, Gallagher A, Loui A C, et al. Aesthetic quality assessment of consumer photos with faces[C]//Proceedings of the IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2010:3221-3224.
[20] Khan S S, Vogel D. Evaluating visual aesthetics in photographic portraiture[C]//Proceedings of the 8th Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging. New York:ACM, 2012:55-62.
[21] Lu X, Lin Z, Jin H, et al. RAPID:Rating pictorial aesthetics using deep learning[C]//Proceedings of the ACM International Conference on Multimedia. New York:ACM, 2014:457-466.
[22] Kao Y, Wang C, Huang K. Visual aesthetic quality assessment with a regression model[C]//Proceedings of 2015 IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2015. 1583-1587.
[23] Lu X, Lin Z, Shen X, et al. Deep multi-patch aggregation network for image style, aesthetics, and quality estimation[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2015:990-998.
[24] Dong Z, Tian X. Multi-level photo quality assessment with multi-view features[J]. Neurocomputing, 2015, 168:308-319.
[25] Wang W, Zhao M, Wang L, et al. A multi-scene deep learning model for image aesthetic evaluation[J]. Signal Processing Image Communication, 2016, 47(C):511-518.
[26] Kao Y, Huang K, Maybank S. Hierarchical aesthetic quality assessment using deep convolutional neural networks[J]. Signal Processing Image Communication, 2016, 47:500-510.
[27] Kong S, Shen X, Lin Z, et al. Photo aesthetics ranking network with attributes and content adaptation[C]//Proceedings of 14th European Conference on Computer Vision. Heidelberg:Springer-Verlag, 2016:662-679.
[28] Jin X, Chi J, Peng S, et al. Deep image aesthetics classification using inception modules and fine-tuning connected layer[C]//Proceedings of the 8th International Conference on Wireless Communications and Signal Processing. Piscataway NJ:IEEE, 2016, doi:10.1109/WCSP.2016.7752571.
[29] Ma S, Liu J, Chen C W. A-Lamp:Adaptive layout-aware multi-patch deep convolutional neural network for photo aesthetic assessment[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2017:722-731.
[30] Jin X, Wu L, Li X D, et al. ILGNet:Inception modules with connected local and global features for efficient image aesthetic quality classification using domain adaptation[J]. arXiv.org, 2018, arXiv:1610.02256v3.
[31] Jin B, Segovia M V O, Süsstrunk S. 2016. Image aesthetic predictors based on weighted CNNs[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Piscataway NJ:IEEE, 2016:2291-2295.
[32] Wang Z, Liu D, Chang S, et al. Image aesthetics assessment using deep chatterjee's machine[C]//Proceedings of 2017 International Joint Conference on Neural Networks. Piscataway NJ:IEEE, 2017:941-948.
[33] Hou L, Yu C P, Samaras D. Squared earth mover's distancebased loss for training deep neural networks[J]. arXiv.org, 2017, arXiv:1611.05916
[34] Deng Y, Chen C L, Tang X. Image aesthetic assessment:An experimental survey[J]. IEEE Signal Processing Magazine, 2016, 34(4):80-106.
[35] Murray N, Marchesotti L, Perronnin F. AVA:A large-scale database for aesthetic visual analysis[C]//Proceedings of the 25th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway NJ:IEEE, 2012:2408-2415.
[36] Wu O, Hu W M, Gao J. Learning to predict the perceived visual quality of photos[C]//Proceedings of 2011 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2011:225-232.
[37] Park T S, Zhang B T. Consensus analysis and modeling of visual aesthetic perception[J]. IEEE Transactions on Affective Computing, 2015, 6(3):272-285.
[38] Kim W H, Choi J H, Lee J S. Subjectivity in aesthetic quality assessment of digital photographs:Analysis of user comments[C]//Proceedings of the 23rd ACM international conference on Multimedia. New York:ACM, 2015:983-986.
[39] Kim W H, Choi J H, Lee J S. Objectivity and subjectivity in aesthetic quality assessment of digital photographs[J]. IEEE Transactions on Affective Computing, 2018, doi:10.1109/TAFFC.2018.2809752.
[40] Jin X, Wu L, Li X, et al. Predicting aesthetic score distribution through cumulative Jensen-Shannon divergence[C]//Proceedings of AAAI Conference on Artificial Intelligence (AAAI). New York:AAAI, 2018.
[41] Chang K Y, Lu K H, Chen C S. Aesthetic critiques generation for photos[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV). Piscataway NJ:IEEE, 2017:3534-3543.
[42] Collomosse J, Tu B, Wilber M, et al. Sketching with Style:Visual search with sketches and aesthetic context[C]//Proceedings of IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2017:2679-2687.
[43] Cui C, Fang H, Deng X, et al. Distribution-oriented aesthetics assessment for image search[C]//Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2017:1013-1016.
[44] Wang W, Shen J. Deep cropping via attention box prediction and aesthetics assessment[C]//2017 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2017:2205-2213.
[45] Ren J, Shen X, Lin Z, et al. Personalized image aesthetics[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway NJ:IEEE, 2017:638-647.
[46] Schwarz K, Wieschollek P, Lensch H P A. Will people like your image?[J]. arXiv.org, 2018, arXiv:1611.05203.
[47] Wang W S, Yang S, Zhang W S, et al. Neural aesthetic image reviewer[J]. arXiv.org, 2018, arXiv:1802.10240.
[48] Yanulevskaya V, Uijlings J, Bruni E, et al. In the eye of the beholder:Employing statistical analysis and eye tracking for analyzing abstract paintings[C]//Proceedings of the 20th ACM International Conference on Multimedia. New York:ACM, 2012:349-358.
[49] Agrawal A, Premachandran V, Kakarala R. Rating image aesthetics using a crowd sourcing approach[C]//Workshops of the 6th Pacific-Rim Symposium on Image and Video Technology. New York:Springer-Verlag New York Inc., 2013, 8334:24-32.
[50] Datta R, Wang J Z. ACQUINE:Aesthetic quality inference engine-real time automatic ratings of photo aesthetics[C]//Proceedings of the ACM International Conference on Multimedia Information Retrieval. New York:ACM, 2010:421-424.
Outlines

/