CN114220033A - Wheat imperfect grain identification method combining image enhancement and CNN - Google Patents

Wheat imperfect grain identification method combining image enhancement and CNN Download PDF

Info

Publication number
CN114220033A
CN114220033A CN202010918351.6A CN202010918351A CN114220033A CN 114220033 A CN114220033 A CN 114220033A CN 202010918351 A CN202010918351 A CN 202010918351A CN 114220033 A CN114220033 A CN 114220033A
Authority
CN
China
Prior art keywords
image
frequency part
neural network
layer
low
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010918351.6A
Other languages
Chinese (zh)
Other versions
CN114220033B (en
Inventor
何小海
贺杰安
吴晓红
吴小强
卿粼波
滕奇志
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sichuan University
Original Assignee
Sichuan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sichuan University filed Critical Sichuan University
Priority to CN202010918351.6A priority Critical patent/CN114220033B/en
Publication of CN114220033A publication Critical patent/CN114220033A/en
Application granted granted Critical
Publication of CN114220033B publication Critical patent/CN114220033B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

Aiming at the imperfect wheat grain identification task, the invention designs a method based on a convolutional neural network and image processing. The performance improvement of the classification network and the data separability are the key of the classification accuracy. Aiming at the different points of the wheat image and the image of the traditional image classification task in the practical application scene, the invention provides an image detail enhancement method aiming at the wheat image so as to enhance the separability of data; and the method is expanded into a feature layer, and corresponding detail enhancement degree can be learned through training and detail enhancement is performed on an output feature map of a neural network hidden layer. The invention provides a novel and practical solution for the imperfect wheat grain recognition task, and has wide application prospect.

Description

Wheat imperfect grain identification method combining image enhancement and CNN
Technical Field
The invention relates to the field of computer vision, in particular to a traditional image processing method and application of a deep neural network.
Background
The imperfect wheat grain identification task aims at identifying imperfect grains of wheat grains according to images, and generally imperfect grain types can be divided into damaged grains, worm-eaten grains, scab grains, sprouting grains and mildewed grains besides perfect grains. Current research mainly uses deep neural networks to classify wheat images. In practical application, the wheat image has a single image background or no background, most pixels of various wheat grains (including imperfect grains, perfect grains and different imperfect grain types) are very similar, and more pixels of various wheat grains are classified by fine parts of the image. Most of the current research does not take advantage of this feature.
Disclosure of Invention
The invention provides a method for identifying imperfect wheat grains by combining image enhancement and CNN based on the characteristics of wheat images. And enhancing image details based on an image processing technology to enhance the separability of various types of data. Taking the enhanced image as the input of the neural network, and then enhancing the performance of the classification network by using BN (batch normalization) technology. The invention realizes the purpose through the following technical scheme:
a wheat imperfect grain identification method combining image enhancement and CNN comprises the following steps:
the method comprises the following steps: a data set is established.
Step two: and building a Detail Enhancement (Detail Enhancement) layer.
Step three: and adding a DE layer in the deep neural network classifier.
Step four: and training a classifier by using a classification neural network added with a DE layer to obtain a class corresponding to the image.
The image preprocessing and data set building as step one is described as follows:
(1) the wheat grain image collected under the actual background is a black background, however, the proportion of the wheat grain to the whole image is not large, and after the wheat grain outline is found out, the image is cut by taking the external rectangle with the maximum outline as a boundary.
(2) To facilitate neural network input, the image is resized to a fixed size (227 x 227) and the foreground is centered.
(3) And (4) looking at the pictures by the wheat professional quality testing personnel, marking the categories of all the pictures, and establishing a data set.
(4) The data set comprises 6000 images, 5000 images are taken as a training set to train the neural network, and the remaining 1000 images are taken as a test set to verify the generalization performance of the neural network.
As the image detail enhancement in step two, it is explained as follows:
(1) the basic image layer of the image is solved by constraining the L0 gradient of the image in the horizontal and vertical directions by using an alternating minimization algorithm. Convention I is the original image, S is the smooth image to be solved,
Figure BDA0002664608350000011
and
Figure BDA0002664608350000012
representing the partial derivatives of the solved S along the horizontal and vertical directions (this partial derivative is solved by forward difference), so that the derivative at any point p in the image can be recorded as
Figure BDA0002664608350000021
For the two-dimensional image S, the L0 norm of the horizontal direction and the vertical direction needs to be constrained, and the specific constraint terms are:
Figure BDA0002664608350000022
wherein, "# { }" indicates that the pixels p satisfying the condition within the braces are counted. Assuming that I is an original image and S is a base layer meeting constraint conditions, solving an objective function as follows:
Figure BDA0002664608350000023
a non-negative parameter λ is introduced, controlling the weight by which I is smoothed. It is converted to the unconstrained form:
Figure BDA0002664608350000024
solving this equation is difficult because C (S) is not convexly non-conductive. For this purpose, an auxiliary variable h is introducedpAnd vpAnd solving by adopting an alternating minimization algorithm. Solving the target transformation:
Figure BDA0002664608350000025
(4) where β is a hyperparameter and h is a constraintpAnd vpCorresponding to the original image gradient
Figure BDA0002664608350000026
And
Figure BDA0002664608350000027
the similarity of (c). The basic layer S of the image can be obtained by solving the formula, and different smoothness degrees can be obtained by adjusting the parameters lambda and beta. The solution of equation (4) can be obtained using an alternating minimization algorithm:
Figure BDA0002664608350000028
Figure BDA0002664608350000029
and (5) alternately iterating according to the formulas (5) and (6) to obtain the smoothed base layer.
(2) After the base layer S is solved, the image I is transformed as follows: d ═ S + γ (I-S), where γ is a constant greater than 1, controlling the degree of detail highlighted.
(3) The whole detail enhancement layer accepts input features x, learnable parameters lambda, beta, gamma
Figure BDA00026646083500000210
Adding a DE layer in the deep neural network classifier as step three, which is described as follows:
(1) the DE operation enables each data Feature to be more specialized, the contribution to the correct category can be effectively enhanced, the DE layer can be used as an input layer to directly enhance details of an input image, a Feature Map can also be added into a hidden layer to enhance the Feature Map, and when the Feature Map is not a two-dimensional matrix, the parameters lambda, beta and gamma are only required to be set as corresponding vectors.
(2) The DE layer needs to be operated before the nonlinear activation, in order to avoid gradient explosion, the BN (batch normalization) layer is added to the network while the DE layer is used, and the BN operation is an important research result in the deep learning field and is also an operation widely known in the field, and is not described herein additionally.
Drawings
FIG. 1 is a six-class wheat grain image of the data set of the present invention.
Fig. 2 is an illustration of a detail enhancement layer proposed by the present invention as an input layer.
Fig. 3 is a sample of the detail enhancement layer before and after six types of wheat grain images are input.
Fig. 4 is a structure of two adjacent layers of a neural network proposed by the present invention to incorporate DE.
Detailed Description
The invention will be further described with reference to the accompanying drawings in which:
fig. 3 shows examples of six types of wheat grain images before and after enhancement by the detail enhancement algorithm of the present invention, and fig. 4 shows two adjacent layers of the deep neural network using BN according to the present invention. In order to verify the effectiveness of the method, typical classification networks LeNet-5, ResNet-34 and VGG-16 are respectively used for comparing and using images before and after enhancement as network input and whether the network is added with a BN layer, namely four groups of comparison tests of each classical classification network total 12 groups of experiments. As the ResNet-34 network adds the BN layer, the ResNet-34 experiment can compare the ResNet-34 with the BN layer removed (ResNet34-BN), the ResNet-34 with the enhanced image used for removing BN (ResNet34-BN + DE), the ResNet and the ResNet with the enhanced image used for removing BN (ResNet34+ DE) in four groups. The experimental data are shown in table 1.
TABLE 1 LeNet-5, VGG-16, ResNet-34 comparative experiments
Figure BDA0002664608350000031
The following conclusions are easily drawn from table 1:
(1) the three classification networks of the experiment all obtain the highest test set accuracy and lower training set accuracy on the method (combining BN and DE) provided by the invention. (2) Using the enhanced image as input, a > 1% test set accuracy boost was obtained over three different networks. When a BN layer is added in the network, the accuracy of the test set of all three networks is improved by more than 5%. (3) The three networks all obtain the maximum test set accuracy on the method provided by the invention, and the test set accuracy is improved by more than 7% compared with the baseline. The effectiveness of the method provided by the invention is verified.

Claims (4)

1. A wheat imperfect grain identification method combining image enhancement and CNN is characterized by comprising the following steps:
the method comprises the following steps: smoothing a matrix (including but not limited to an input image) based on an L0 norm constraint, and extracting a low-frequency part with slow change as a base layer;
step two: the learnable self-adaptive high-frequency component enhancement method is characterized in that the original matrix and the smoothed result are subtracted to obtain a high-frequency part with violent change, the high-frequency part is superposed with a low-frequency part after the part of the image layer is enhanced, and parameters for controlling the enhancement degree are subjected to self-adaptive learning by a deep neural network;
step three: and (4) putting the Detail Enhancement layer (DE) packaged in the step two into a neural network, and optimizing control parameters while training the neural network, so that the high-frequency or low-frequency part is not adaptively enhanced according to the category without being input, and higher out-of-category difference is obtained.
2. The method of claim 1, wherein the method comprises the steps of: in the first step, a matrix (including but not limited to an input image) is smoothed based on L0 norm constraint, the number of unequal adjacent pixels in the image is constrained, a slowly-changing low-frequency part is extracted as a base layer, and the low-frequency component of the image meeting the constraint condition is solved by constructing L0 gradient alternate minimization of the transverse direction and the longitudinal direction of the image.
3. The method of claim 1, wherein the method comprises the steps of: and in the second step, the self-adaptive high-frequency component enhancement method is characterized in that the difference is made between the original matrix and the result after smoothing to obtain a high-frequency part with violent change, the high-frequency part is superposed with the low-frequency part after the image layer of the part is enhanced, wherein the parameter for controlling the enhancement degree is self-adaptively learned by a deep neural network, the learnable hyper-parameter lambda and beta are given to control the smoothing degree of the two-dimensional matrix, the parameter gamma is used for controlling the enhancement degree of the high-frequency component and is used as a characteristic extraction layer of the deep neural network to self-adaptively enhance local characteristics, and for the high-dimensional matrix of the hidden layer, the parameters lambda, beta and gamma only need to be expanded into corresponding vectors.
4. The method of claim 1, wherein the method comprises the steps of: and step three, putting the Detail Enhancement layer (DE) packaged in the step two into a neural network, training the neural network, and optimizing control parameters at the same time, so that the high-frequency part or the low-frequency part is not input and adaptively enhanced according to the category, and the network back propagation adaptively selects the enhanced amplitude of the high-frequency part or the low-frequency part in the training stage to obtain higher out-of-category difference.
CN202010918351.6A 2020-09-03 2020-09-03 Wheat imperfect grain identification method combining image enhancement and CNN Active CN114220033B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010918351.6A CN114220033B (en) 2020-09-03 2020-09-03 Wheat imperfect grain identification method combining image enhancement and CNN

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010918351.6A CN114220033B (en) 2020-09-03 2020-09-03 Wheat imperfect grain identification method combining image enhancement and CNN

Publications (2)

Publication Number Publication Date
CN114220033A true CN114220033A (en) 2022-03-22
CN114220033B CN114220033B (en) 2023-07-18

Family

ID=80695580

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010918351.6A Active CN114220033B (en) 2020-09-03 2020-09-03 Wheat imperfect grain identification method combining image enhancement and CNN

Country Status (1)

Country Link
CN (1) CN114220033B (en)

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101452575A (en) * 2008-12-12 2009-06-10 北京航空航天大学 Image self-adapting enhancement method based on neural net
CN101706954A (en) * 2009-11-13 2010-05-12 北京中星微电子有限公司 Image enhancement method and device thereof as well as image low frequency component computing method and device thereof
CN106716479A (en) * 2014-03-31 2017-05-24 英皇创新有限公司 A computer implemented method of deriving performance from a local model
CN108875747A (en) * 2018-06-15 2018-11-23 四川大学 A kind of wheat unsound grain recognition methods based on machine vision
EP3543916A1 (en) * 2018-03-20 2019-09-25 Idemia Identity & Security France Methods for learning of parameters of a convolutional neural network, and classification of input data
CN110333195A (en) * 2019-07-15 2019-10-15 北华航天工业学院 Water content in plant leaf detection method and device
CN110929610A (en) * 2019-11-12 2020-03-27 上海五零盛同信息科技有限公司 Plant disease identification method and system based on CNN model and transfer learning
CN111161189A (en) * 2019-12-31 2020-05-15 杭州电子科技大学 Single image re-enhancement method based on detail compensation network
AU2020100953A4 (en) * 2020-06-05 2020-07-16 D, Vijayakumar DR Automated food freshness detection using feature deep learning

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101452575A (en) * 2008-12-12 2009-06-10 北京航空航天大学 Image self-adapting enhancement method based on neural net
CN101706954A (en) * 2009-11-13 2010-05-12 北京中星微电子有限公司 Image enhancement method and device thereof as well as image low frequency component computing method and device thereof
CN106716479A (en) * 2014-03-31 2017-05-24 英皇创新有限公司 A computer implemented method of deriving performance from a local model
EP3543916A1 (en) * 2018-03-20 2019-09-25 Idemia Identity & Security France Methods for learning of parameters of a convolutional neural network, and classification of input data
CN108875747A (en) * 2018-06-15 2018-11-23 四川大学 A kind of wheat unsound grain recognition methods based on machine vision
CN110333195A (en) * 2019-07-15 2019-10-15 北华航天工业学院 Water content in plant leaf detection method and device
CN110929610A (en) * 2019-11-12 2020-03-27 上海五零盛同信息科技有限公司 Plant disease identification method and system based on CNN model and transfer learning
CN111161189A (en) * 2019-12-31 2020-05-15 杭州电子科技大学 Single image re-enhancement method based on detail compensation network
AU2020100953A4 (en) * 2020-06-05 2020-07-16 D, Vijayakumar DR Automated food freshness detection using feature deep learning

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
YIN SHEN 等: "Image Recognition Method Based on an Improved Convolutional Neural Network to Detect Impurities in Wheat" *
王志鹏 等: "小麦不完善粒智能识别研究" *
贺杰安 等: "结合图像增强和卷积神经网络的小麦不完善粒识别" *
陈赛赛: "小麦质量指标机器视觉技术研究" *

Also Published As

Publication number Publication date
CN114220033B (en) 2023-07-18

Similar Documents

Publication Publication Date Title
CN108537743B (en) Face image enhancement method based on generation countermeasure network
CN111881714B (en) Unsupervised cross-domain pedestrian re-identification method
CN108648191B (en) Pest image recognition method based on Bayesian width residual error neural network
CN110084318B (en) Image identification method combining convolutional neural network and gradient lifting tree
US20190228268A1 (en) Method and system for cell image segmentation using multi-stage convolutional neural networks
CN107463920A (en) A kind of face identification method for eliminating partial occlusion thing and influenceing
CN112598658A (en) Disease identification method based on lightweight twin convolutional neural network
CN106303233A (en) A kind of video method for secret protection merged based on expression
CN110705591A (en) Heterogeneous transfer learning method based on optimal subspace learning
CN106709528A (en) Method and device of vehicle reidentification based on multiple objective function deep learning
CN111274883B (en) Synthetic sketch face recognition method based on multi-scale HOG features and deep features
Zhu et al. A-pixelhop: A green, robust and explainable fake-image detector
CN113205026A (en) Improved vehicle type recognition method based on fast RCNN deep learning network
CN110991554B (en) Improved PCA (principal component analysis) -based deep network image classification method
CN110717451A (en) Medicinal plant leaf disease image identification method based on deep learning
CN109145135A (en) A kind of human face portrait aging method based on principal component analysis
Ticay-Rivas et al. Pollen classification based on geometrical, descriptors and colour features using decorrelation stretching method
Woźniak et al. Basic concept of cuckoo search algorithm for 2D images processing with some research results: An idea to apply cuckoo search algorithm in 2d images key-points search
CN111611963B (en) Face recognition method based on neighbor preservation canonical correlation analysis
CN114220033A (en) Wheat imperfect grain identification method combining image enhancement and CNN
CN109325416A (en) A kind of high-definition image fast face recognition method based on PCA and SRC
Li et al. Criminal investigation image classification based on spatial cnn features and elm
CN110399885B (en) Image target classification method based on local geometric perception
Osterloff et al. Ranking color correction algorithms using cluster indices
Malathy et al. Improved face recognition using convolutional neural network with unaided learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant