Research of machine vision technology in agricultural application: Today and the future

CHEN Bingqi; WU Zhaoheng; LI Hongye; WANG Jin

doi:10.3981/j.issn.1000-7857.2018.11.006

PDF(2169 KB)

Science & Technology Review ›› 2018, Vol. 36 ›› Issue (11) : 54-65. DOI: 10.3981/j.issn.1000-7857.2018.11.006

Exclusive

Research of machine vision technology in agricultural application: Today and the future

Author information -

College of Engineering, China Agricultural University, Beijing 100083, China

Abstract

Machine vision technology has been widely applied to various fields of agricultural production. In this paper we summarize the outstanding research results at home and abroad and elaborate the main forms of machine vision application in agriculture at this stage. We list the research results of machine vision in crop selection and quality inspection, plant growth information monitoring, field vision navigation, and other application directions. By analyzing its innovative image processing algorithms and the composition of machine vision systems, we show that the current application of machine vision in agriculture is still having problems such as poor reliability, high cost, and low level of intelligence. We analyze its subjective and objective reasons and put forward corresponding suggestions. Combining the current research and application of machine vision in various fields, we look forward to the future development direction of machine vision in agricultural applications. We believe that machine vision systems based on embedded processing modules and multi-technology integration will become the main development trend in the future. The deep learning model represented by convolution neural network will also become the core technology of future image recognition, which can greatly help solve the existing problems of machine vision in agricultural applications.

Key words

machine vision / agriculture / nondestructive testing / navigation

Cite this article

EndNote

Ris (Procite)

Bibtex

Download Citations

CHEN Bingqi, WU Zhaoheng, LI Hongye, WANG Jin. Research of machine vision technology in agricultural application: Today and the future[J]. Science & Technology Review, 2018, 36(11): 54-65 https://doi.org/10.3981/j.issn.1000-7857.2018.11.006

References

[1] Haralick R M, Shanmugam K, Dinstein I. Textural features for image classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1973, 3(6):610-621.
[2] Taylor R W, Rehkugler G E, Throop J A. Apple bruise detection using a digital line scan camera system[C]. National Conference on Agricultural Electronics Applications. St. Joseph:American Society of Agricultural Engineers, 1984.
[3] 刘禾, 汪懋华. 用计算机图像技术进行苹果坏损自动检测的研究[J]. 农业机械学报, 1998(4):81-86. Liu He, Wang Maohua. Research on automatic detection of apple damage using computer image technology[J]. Journal of Agricultural Machinery, 1998(4):81-86.
[4] 郑文钟. 国内外智能化农业机械装备发展现状[J]. 现代农机, 2015(6):4-8. Zheng Wenzhong. Domestic and foreign intelligent agricultural machinery and equipment development status[J]. Modern Agricultural Machinery, 2015(6):4-8.
[5] 王福娟. 机器视觉技术在农产品分级分选中的应用[J]. 农机化研究, 2011, 33(5):249-252. Wang Fujuan. Application of computer vision technology in farm produce classification and selection[J]. Journal of Agricultural Mechanization Research, 2011, 33(5):249-252.
[6] 陈丽, 王兰英. 概率神经网络在玉米叶部病害识别中的应用[J]. 农机化研究, 2011, 33(6):145-148. Chen Li, Wang Lanying. Probabilistic neural network application in maize leaf disease identification[J]. Journal of Agricultural Mechanization Research, 2011, 33(6):145-148.
[7] 慕军营, 戚树腾, 陈军, 等. 自动导航系统在农业中的应用及果园适用性分析[J]. 农机化研究, 2014(7):8-12. Mu Junying, Qi Shuteng, Chen Jun, et al. Application of automatic navigation system in agriculture machinery and analysis of applicability in orchard[J]. Journal of Agricultural Mechanization Research, 2014(7):8-12.
[8] 张五一, 赵强松, 王东云. 机器视觉的现状及发展趋势[J]. 中原工学院学报, 2008, 19(1):9-12. Zhang Wuyi, Zhao Qiangsong, Wang Dongyun. Application of image segmentation based on improved watershed segmentation algorithm[J]. Journal of Central Plains Institute of Technology, 2008, 19(1):9-12.
[9] 章炜. 机器视觉技术发展及其工业应用[J]. 红外, 2005, 27(2):11-17. Zhang Wei. Development of machine vision technology and its industrial application[J]. Infrared, 2005, 27(2):11-17.
[10] 成芳, 应义斌. 机器视觉技术在农作物种子质量检验中的应用研究进展[J]. 农业工程学报, 2001, 17(6):175-179. Cheng Fang, Ying Yibin. Application of machine vision tech-nology to quality evaluation of ag ricultural seed resources[J]. Journal of Agricultural Engineering, 2001, 17(6):175-179.
[11] Galloway M M. Textural analysis using gray level run length[J]. Computer Graphics and Image Processing, 1975, 4(2):172-179.
[12] Zayas I, Pomeranz Y, Lai F S. Discrimination between Arthur and Arkan wheats by image analysis[J]. Cereal Chemistry, 1985, 62(6):478-480.
[13] Jansen P I. Seed production quality in Trifolium balansae and T.resupinatum:The effect of temperature[J]. Seed Science & Technology, 1995(23):353-364.
[14] Granitto P M, Navone H D, Verdes P F, et al. Weed seeds identification by machine vision[J]. Computers & Electronics in Agriculture, 2002, 33(2):91-103.
[15] Yan X, Wang J, Liu S, et al. Purity identification of maize seed based on color characteristics[C]. International Conference on Computer and Computing Technologies in Agriculture. Berlin Heidelberg:Springer, 2010:620-628.
[16] 陈兵旗, 高振江, 宋同珍, 等. 棉种图像精选方案与算法研究[J]. 农业机械学报, 2010, 41(1):167-171. Chen Bingqi, Gao Zhenjiang, Song Tongzhen, et al. Scheme and algorithm of cottonseed selection based on image processing[J]. Journal of Agricultural Mechanization, 2010, 41(1):167-171.
[17] Hong P T T, Hai T T T, Lan L T, et al. Comparative study on vision based rice seed varieties identification[C]. Seventh International Conference on Knowledge and Systems Engineering. Piscataway, NJ:IEEE, 2015:377-382.
[18] 李振, 廖同庆, 冯青春, 等. 基于图像处理技术的黄瓜种子活力指数检测系统设计[J]. 种子, 2015, 34(6):111-115. Li Zhen, Liao Tongqing, Feng Qingchun, et al. Asystem design in cucumber seed vigor index detection based on image Processing[J]. Seed, 2015, 34(6):111-115.
[19] 王侨, 陈兵旗, 朱德利, 等. 基于机器视觉的定向播种用玉米种粒精选装置研究[J]. 农业机械学报, 2017, 48(2):27-37. Wang Qiao, Chen Bingqi, Zhu Deli, et al. Machine visionbased selection machine of corn seed used for directional seeding[J]. Journal of Agricultural Mechanization, 2017, 48(2):27-37.
[20] 陈兵旗, 孙旭东, 韩旭, 等. 基于机器视觉的水稻种子精选技术[J]. 农业机械学报, 2010, 41(7):168-173. Chen Bingqi, Sun Xudong, Han Xu, et al. Rice seed selection based on machine vision[J]. Journal of Agricultural Mechanization, 2010, 41(7):168-173.
[21] Dubey B P, Bhagwat S G, Shouche S P, et al. Potential of artificial neural networks in varietal identification using morphometry of wheat grains[J]. Biosystems Engineering, 2006, 95(1):61-67.
[22] Arefi A, Motlagh A M, Teimourlou R F. Wheat class identification using computer vision system and artificial neural networks[J]. International Agrophysics, 2011, 25(4):319-325.
[23] 李景彬, 陈兵旗, 邵鲁浩, 等. 基于BP神经网络的脱绒棉种品种识别[J]. 农业工程学报, 2012, 28(2):265-269. Li Jingbin, Chen Bingqi, Shao Luhao, et al. Variety identification of delinted cotton seeds based on BP neural network[J]. Journal of Agricultural Engineering, 2012, 28(2):265-269.
[24] 饶秀勤. 基于机器视觉的水果品质实时检测与分级生产线的关键技术研究[D]. 杭州:浙江大学, 2007. Rao Xiuqin. The key technologies of real-time detection and grading of production line fruit quality based on machine vision[D]. Hangzhou:Zhejiang University, 2007.
[25] 周竹, 黄懿, 李小昱, 等. 基于机器视觉的马铃薯自动分级方法[J]. 农业工程学报, 2012, 28(7):178-183. Zhou Zhu, Huang Yi, Li Xiaoyu, et al. Automatic detecting and grading method of potato based on machine vision[J]. Journal of Agricultural Engineering, 2012, 28(7):178-183.
[26] Hasankhani R, Navid H. Potato sorting based on size and color in machine vision system[J]. Journal of Agricultural Science, 2012, 4(5):235-244.
[27] Sofu M M, Er O, Kayacan M C. Design of an automatic apple sorting system using machine vision[J]. Computers & Electronics in Agriculture, 2016, 127:395-405.
[28] 王侨, 陈兵旗, 杨曦, 等. 用于定向播种的玉米种穗图像精选方法[J]. 农业工程学报, 2015, 31(1):170-177. Wang Qiao, Chen Qiaoqi, Yang Xi, et al. Corn ears image selection method for directional seeding[J]. Journal of Agricultural Engineering, 2015, 31(1):170-177.
[29] 刘长青, 陈兵旗. 基于机器视觉的玉米果穗参数的图像测量方法[J]. 农业工程学报, 2014, 30(6):131-138. Liu Changqing, Chen Bingqi. Method of image detection for ear of corn based on machine vision[J]. Journal of Agricultural Engineering, 2014, 30(6):131-138.
[30] 陈兵旗, 刘长青. 玉米粒在穗计数方法:CN102750584A[P]. 2012-10-24. Chen Bingqi, Liu Changqing. Method for counting kernels in the ear of corn:CN102750584A[P]. 2012-10-24.
[31] 毕昆, 王纪华, 王成,等. 基于机器视觉的玉米果穗性状参数自动检测装置:CN101881598A[P]. 2010-11-10. Bi Kun, Wang Jihua, Wang Cheng, et al.Automatic detection device for maize ear traits based on machine vision:CN101881598A[P]. 2010-11-10.
[32] 周金辉, 马钦, 朱德海, 等. 基于机器视觉的玉米果穗产量组分性状测量方法[J]. 农业工程学报, 2015, 31(3):221-227. Zhou Jinhui, Ma Qin, Zhu Dehai, et al. Measurement method for yield component traits of maize based on machine vision[J]. Journal of agricultural engineering, 2015, 31(3):221-227.
[33] Karayel D, Wiesehoff M. Laboratory measurement of seed drill seed spacing and velocity of fall of seeds usinghighspeed camera system[J]. Computers and Electronics in Agriculture, 2006, 50(2):89-96.
[34] 马旭, 王剑平, 胡少兴, 等. 用图像处理技术检测精密排种器性能[J]. 农业机械学报, 2001, 32(4):34-37. Ma Xu, Wang Jianping, Hu Shaoxing, et al. Detection of a precision seedmeter performance using image processing technology[J]. Journal of Agricultural Mechanization, 2001, 32(4):34-37.
[35] Yazgi A, Degirmencioglu A. Optimization of the seed spacinguniformity performance of a vacuum-type precision seederusing response surface methodology[J]. Biosystems Engineering, 2007, 97(3):347-356.
[36] 赵郑斌, 刘昱程, 刘忠军, 等. 基于机器视觉的穴盘精密播种性能检测系统[J]. 农业机械学报, 2014(增刊1):24-28. Zhao Zhengbin, Liu Yucheng, Liu Zhongjun, et al. Performance detection system of tray precision seeder based on machine vision[J]. Journal of Agricultural Mechanization, 2014(Suppl 1):24-28.
[37] 秦忠连. 基于图像处理的排种器性能检测方法研究[D]. 北京:中国农业大学, 2008. Qin zhonglian. Based on image processing metering device performance detection method[D]. Beijing:China Agricultural University, 2008.
[38] 宁纪锋, 何东健, 杨蜀秦. 玉米籽粒的尖端和胚部的计算机视觉识别[J]. 农业工程学报, 2004, 20(3):117-119. Ning Jifeng, He Dongjian, Yang Shuqin. Identification of tip cap and germ surface of corn kernel using computer vision[J]. Journal of Agricultural Engineering, 2004, 20(3):117-119.
[39] 刘长青, 陈兵旗, 张新会,等. 玉米定向精播种粒形态与品质动态检测方法[J]. 农业机械学报, 2015, 46(9):47-54. Liu Changqing, Chen Bingqi, Zhang Xinhui, et al. Dynamic detection of corn seeds for directional precision seeding[J]. Journal of Agricultural Mechanization, 2015, 46(9):47-54.
[40] 王侨, 陈兵旗, 寇春荣,等. 基于机器视觉的玉米种粒定向定位摆放装置研制[J]. 农业工程学报, 2017, 33(11):19-28. Wang Qiao, Chen Bingqi, Kou Chunrong, et al. Development of corn seed directional positioning machine based on machine vision[J]. Journal of Agricultural Engineering, 2017, 33(11):19-28.
[41] 陈兵旗, 郭学梅, 李晓华. 基于图像处理的小麦病害诊断算法[J]. 农业机械学报, 2009, 40(12):190-195. Chen Bingqi, Guo Xuemei, Li Xiaohua. Image diagnosis algorithm of diseased wheat[J]. Journal of Agricultural Mechanization, 2009, 40(12):190-195.
[42] 韩瑞珍, 何勇. 基于计算机视觉的大田害虫远程自动识别系统[J]. 农业工程学报, 2013, 29(3):156-162. Han Ruizhen, He Yong. Computer vision-based remote automatic identification system of field pest[J]. Journal of Agricultural Engineering,2013, 29(3):156-162.
[43] 张芳. 基于计算机视觉的农作物病害检测系统的研究[D]. 西安:陕西科技大学, 2013. Zhang Fang. Study of crop disease detection system based on computer vision[D]. Xi'an:Shanxi University of Science and Technology, 2013.
[44] 马稚昱, 清水浩, 辜松. 基于机器视觉的菊花生长自动无损监测技术[J]. 农业工程学报, 2010, 26(9):203-209. Ma Zhiyu, Qing Shuihao, Gu Song. Chrysanthemum growth automatic nondestructive monitoring technology based on machine vision[J]. Journal of Agricultural Engineering, 2010, 26(9):203-209.
[45] 陈兵旗, 何醇, 马彦平, 等. 大田玉米长势的三维图像监测与建模[J]. 农业工程学报, 2011, 27(增刊1):366-372. Chen Bingqi, He Qiu, Ma Yanping, et al. 3D image monitoring and modeling of for corn growth in field condition[J]. Journal of Agricultural Engineering, 2011, 27(suppl 1):366-372.
[46] Jin J, Tang L. Corn plant sensing using real-time stereo vision[J]. Journal of Field Robotics, 2009, 26(6/7):591-608.
[47] Liu Y, Chen B, Qiao J. Development of a machine vision algorithm for recognition of peach fruit in a natural scene[J]. Transactions of the Asabe, 2011, 54(2):695-702.
[48] Silwal A, Gongal A, Karkee M. Identification of red apples in field environment with over the row machine vision system[J]. Agricultural Engineering International:CIGR Journal, 2014, 16(4):66-75.
[49] 赵晓霞, 张铁中, 陈兵旗, 等. 自然环境下桃子图像分割算法[J]. 农机化研究, 2009, 31(2):107-111. Zhao Xiaoxia, Zhang Tiezhong, Chen Bingqi, et al. Study on the segmentation algorithm for peach in natural scene[J]. Agricultural Mechanization Research, 2009, 31(2):107-111.
[50] 司永胜, 乔军, 刘刚, 等. 苹果采摘机器人果实识别与定位方法[J]. 农业机械学报, 2010, 41(9):148-153. Si Yongsheng, Qiao Jun, Liu Gang, et al. Recognition and location of fruits for apple harvesting robot[J]. Journal of Agricultural Mechanization, 2010, 41(9):148-153.
[51] 彭辉, 吴鹏飞, 翟瑞芳, 等. 基于视差图像的重叠果实图像分割算法[J]. 农业机械学报, 2012, 43(6):167-173. Peng Hui, Wu Pengfei, Zhai Ruifang, et al. Parallax images based overlapping fruit image segmentation algorithm[J]. Journal of Agricultural Mechanization, 2012, 43(6):167-173.
[52] Linker R, Cohen O, Naor A. Determination of the number of green apples in RGB images recorded in orchards[J]. Computers & Electronics in Agriculture, 2012, 81(1):45-57.
[53] Waxman A M, Lemoigne J, Davis L S, et al. A visual navigation system for autonomous land vehicles[J]. IEEE Journal on Robotics & Automation, 1987, 3(2):124-141.
[54] Ishikawa S, Kuwamoto H, Ozawa S. Visual navigation of an autonomous vehicle using white line recognition[J]. Intelligence IEEE Transactions on Pattern Analysis & Machine, 1988, 10(5):743-749.
[55] 周俊, 姬长英, 刘成良. 农用轮式移动机器人视觉导航系统[J]. 农业机械学报, 2005, 36(3):90-94. Zhou Jun, Ji Changying, Liu Chengliang. Visual navigation system of agricultural wheeled mobile robot[J]. Journal of Agricultural Mechanization, 2005, 36(3):90-94.
[56] 周俊, 姬长英. 农业机器人视觉导航中多分辨率路径识别[J]. 农业机械学报, 2003, 34(6):120-123. Zhou Jun, Ji Changying. Multi-resolution road recognition for visual navigation[J]. Journal of Agricultural Mechanization, 2003, 34(6):120-123.
[57] 陈娇, 姜国权, 杜尚丰, 等. 基于垄线平行特征的视觉导航多垄线识别[J]. 农业工程学报, 2009, 25(12):107-113. Chen Jiao, Jiang Guoquan, Du Shangfeng, et al. Crop rows detection based on parallel characteristic of crop rows using visual navigation[J]. Journal of Agricultural Engineering, 2009, 25(12):107-113.
[58] 陈兵旗, 东城清秀, 渡边兼五, 等. 插秧机器人视觉系统研究(第4报)——水泥田埂的检测[J]. 日本农业机械学报, 1999, 61(3):57-64. Chen Bingqi, Dongcheng Qingxiu, Watanabe Kenzo, et al. Research on transplanting robot vision system (4th report):Testing of cement field rakes[J]. Journal of Japan Agricultural Machinery Journal, 1999, 61(3):57-64.
[59] 陈兵旗, 东城清秀, 渡边兼五, 等. 插秧机器人视觉系统研究(第3报)——土田埂的检测[J]. 日本农业机械学报, 1998, 60(5):63-74. Chen Bingqi, Dongcheng Qingxiu, Watanabe Kenzo, et al. Research on transplanting robot vision system(3rd report):Detection of soil ridges[J]. Journal of Japan Agricultural Machinery Journal, 1998, 60(5):63-74.
[60] 陈兵旗, 渡边兼五, 东城清秀, 等. 插秧机器人视觉系统研究(第2报)——基于Hough变换的目标苗列线检测[J]. 日本农业机械学报, 1997, 59(3):23-28. Chen Bingqi, Watanabe Kenzo, Dongcheng Qingxiu, et al. Research on transplanting robot vision system(2nd report):Target line detection based on Hough Transform[J]. Journal of Japan Agricultural Machinery Journal, 1997, 59(3):23-28.
[61] 渡边兼五, 陈兵旗, 东城清秀, 等. 插秧机器人视觉系统研究(第1报)——基于图像处理的目标苗检测[J]. 日本农业机械学会杂志, 1997, 59(2):49-55. Watanabe Kenzo, Chen Bingqi, Dongcheng Qingxiu, et al. Research on the transplanting robot vision system(first report):Target seedling detection based on image processing[J]. Journal of Japan Agricultural Machinery Journal, 1997, 59(2):23-28.
[62] 陈兵旗. 由插秧机器人实现降低生产成本和提高生产率的研究[J]. 日本饭岛纪念食品科学振兴财团年报, 1997:312-316. Chen Bingqi. Reduce production costs and improve productivity by planting robots[J]. Japan Iijima Memorial Food Learn to Revitalize the Foundation Annual Report, 1997:312-316.
[63] Chen B Q, Tojo S, Watanabe K. Detection algorithm for traveling route in paddy field for automated managing machines[J]. Transaction of the ASAE, 2002, 45(1):239-246.
[64] 毛可骏, 周平, 赵匀, 等. 基于机器视觉的自主插秧机导航信息的提取研究[J]. 农机化研究, 2009, 31(5):63-66. Mao Kejun, Zhou Ping, Zhao Yun, et al. The study for the navigation information about automatic navigation of unmanned rice mechanical transplanter based on machine vision[J]. Agricultural Mechanization Research, 2009, 31(5):63-66.
[65] Han Y H, Wang Y M,Kang F. Navigation line detection based on support vector machine for automatic agriculture vehicle[C]. International Conference on Automatic Control and Artificial Intelligence. Stevenage:The Institution of Engineering and Technology, 2012:1381-1385.
[66] 张方明, 陈舒杰, 等. 水田作业轮式自动农业机器人的路径规划方法[C]. 第十届长三角科技论坛农机化分论坛, 2013. Zhang Fangming, Chen Shujie, et al. The study for the navigation information about automatic navigation of unmanned rice mechanical transplanter based on machine vision[C]. The 10th Yangtze River Delta Forum on Agricultural Mechanization, 2013.
[67] 赵颖, 陈兵旗, 王书茂. 基于机器视觉的耕作机器人行走目标直线的检测[J]. 农业机械学报, 2006, 37(4):81-86. Zhao Ying, Chen Bingqi, Wang Shumao. Fast detection of furrows based on machine vision on autonomous mobile robot[J]. Journal of Agricultural Mechanization, 2006, 37(4):81-86.
[68] 李景彬, 陈兵旗, 刘阳, 等. 采棉机视觉导航路线图像检测方法[J]. 农业工程学报, 2013, 29(11):11-19. Li Jingbin, Chen Bingqi, Liu Yang, et al. Detection for navigation route for cotton harvester based on machine vision[J]. Journal of Agricultural Engineering, 2013, 29(11):11-19.
[69] 张磊, 王书茂, 陈兵旗, 等. 基于双目视觉的农田障碍物检测[J]. 中国农业大学学报, 2007, 12(4):70-74. Zhang Lei, Wang Shumao, Chen Bingqi, et al. Detection of obstacles in farmland based on binocular vision[J]. Journal of China Agricultural University, 2007, 12(4):70-74.
[70] 苟琴, 耿楠, 张志毅. 基于视差图的未知环境下农田障碍物检测方法[J]. 计算机工程与设计, 2013, 34(2):707-710. Gou Qin, Geng Nan, Zhang ZhiYi. Detection of obstacles in farmland based on binocular vision[J]. Computer Engineering and Design, 2013, 34(2):707-710.
[71] 李权. 基于双目视觉的障碍物检测研究[D]. 南京:南京农业大学, 2014. Li Quan. Research on obstacle detection based on binocular vision[D]. Nanjing:Nanjing Agricultural University, 2014.
[72] Cherubini A, Chaumette F. A redundancy-based approach for obstacle avoidance in mobile robot navigation[C]//IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, NJ:IEEE, 2010:5700-5705.
[73] Hanawa K, Yamashita T, Matsuo Y, et al. Development of a stereo vision system to assist the operation of agricultural tractors[J]. Japan Agricultural Research Quarterly, 2012, 46(4):287-293.
[74] English A, Ross P, Ball D, et al. Vision based guidance for robot navigation in agriculture[C]//2014 IEEE International Conference on Robotics and Automation. Piscataway, NJ:IEEE, 2014:1693-1698.
[75] 农用拖拉机田间视觉导航系统:201504[R]. 石河子:新疆生产建设兵团农业机械检验测试中心, 2015. Agricultural tractor field visual navigation system:201504[R]. Xinjiang:Xinjiang Production and Construction Corps Agricultural Machinery Testing and Testing Center, 2015.