CN112232343A - Neural network and method for recognizing grain mildewed grains - Google Patents

Neural network and method for recognizing grain mildewed grains Download PDF

Info

Publication number
CN112232343A
CN112232343A CN202010914811.8A CN202010914811A CN112232343A CN 112232343 A CN112232343 A CN 112232343A CN 202010914811 A CN202010914811 A CN 202010914811A CN 112232343 A CN112232343 A CN 112232343A
Authority
CN
China
Prior art keywords
layer
output data
network
convolution
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010914811.8A
Other languages
Chinese (zh)
Other versions
CN112232343B (en
Inventor
杨东
姜俊伊
李倩倩
毕文雅
石天玉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Academy of National Food and Strategic Reserves Administration
Original Assignee
Academy of National Food and Strategic Reserves Administration
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Academy of National Food and Strategic Reserves Administration filed Critical Academy of National Food and Strategic Reserves Administration
Priority to CN202010914811.8A priority Critical patent/CN112232343B/en
Publication of CN112232343A publication Critical patent/CN112232343A/en
Application granted granted Critical
Publication of CN112232343B publication Critical patent/CN112232343B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/02—Agriculture; Fishing; Mining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/68—Food, e.g. fruit or vegetables

Abstract

The embodiment of the invention discloses a neural network and a method for identifying grain mildewed grains, which comprises a first convolution layer and a second convolution layer, wherein the first convolution layer is used for performing first convolution operation on spectral image data xi (QxQx256) to obtain first layer network output data; the maximum pooling layer is used for pooling the second-layer network output data, and the attention mechanism module is used for performing kernel mildew map feature mining on the pooled data to obtain fourth-layer network output data; the third convolution layer is used for carrying out third convolution operation on the fourth layer network output data to obtain fifth layer network output data; the fourth convolution layer is used for performing fourth convolution operation on the fifth layer network output data to obtain sixth layer network output data; the system comprises a first full-connection layer, a second full-connection layer and a first classifier which are connected in series, wherein the first classifier outputs an identification result.

Description

Neural network and method for recognizing grain mildewed grains
Technical Field
The invention relates to the technical field of food science. And more particularly, to a grain mildew granule recognition neural network, a training method, a recognition method, a computer device, and a computer-readable storage medium.
Background
The quality safety problem of corn which is one of the three grains is concerned by the national people in the process of harvesting and storing, and the quality evaluation result is directly related to the modes of pricing and classified storage, such as the corn warehousing time and the like. The mildewed grains are an important index for measuring the quality of raw grains and are one of key work contents of the grain warehouse-in and warehouse-out inspection process. The safety problem caused by the mildew of the corn not only brings economic loss to the later-stage storage and processing of the seeds, but also the mistaken eating of the mildewed corn can harm the health of people and livestock. In addition, the mildew grains are individually listed as detection items in the GB1353-2018 corn standard which is exported in 2018, so that the detection strength of the mildew grains is enhanced. Therefore, the method can quickly and effectively detect the quantity of the mildewed grains and take counter measures in advance, and is a key means for preventing the polluted grains from entering a consumption link as raw materials.
At present, most of corn purchasing sites adopt a uniformly approved artificial sensory evaluation method in the grain industry to effectively detect mildewed grains and imperfect grains of corn (the detection of one sample by human eyes needs 30-40 minutes). However, the manual quality inspection depends on experienced professionals, has the problems of strong subjectivity, poor reproducibility, long detection period and the like, is difficult to meet the requirement of on-site rapid batch detection, and seriously influences the working efficiency of the quality inspection of the grain storage site. Therefore, the development of a rapid, nondestructive, intelligent and convenient field operation detection method is urgently needed to replace manual quality inspection to complete the identification work of the mildewed grains.
Disclosure of Invention
The invention aims to provide a neural network and a neural method for identifying grain mildewed grains, which are used for solving at least one of the problems in the prior art.
In order to achieve the purpose, the invention adopts the following technical scheme:
in a first aspect, the invention provides a grain mildew recognition neural network, which comprises
The first convolution layer is used for performing first convolution operation on spectral image data xi (QxQx256) to obtain first layer network output data, wherein the spectral image data is obtained by extracting pixel point spectral images in a region of interest of the QxQ size of a grain sample, the QxQ is an image scale, and 256 is the number of spectral bands;
the second convolution layer is used for carrying out second convolution operation on the first layer network output data to obtain second layer network output data;
the maximum pooling layer is used for pooling the second-layer network output data to obtain third-layer network output data xi (Q '× Q' × 128), wherein Q '× Q' is the image scale after pooling processing;
the attention mechanism module is used for performing grain mildew map feature mining on the third-layer network output data to obtain fourth-layer network output data;
the third convolution layer is used for carrying out third convolution operation on the fourth layer network output data to obtain fifth layer network output data;
the fourth convolution layer is used for performing fourth convolution operation on the fifth layer network output data to obtain sixth layer network output data;
the system comprises a first full connection layer, a second full connection layer and a first classifier which are connected in series, wherein the first full connection layer receives the output data of a sixth layer network, and the first classifier outputs an identification result.
In one embodiment, the attention mechanism module includes
A spectral domain attention mechanism network comprising
The global uniform pooling layer is used for pooling the third-layer network output data to obtain global uniform pooled output data;
the fifth convolution layer is used for performing fifth convolution operation on the global uniform pooling output data to obtain fifth convolution output data, wherein the convolution kernel size is 1 multiplied by 128/r, and r is the reduction rate;
a sixth convolution layer, configured to perform a sixth convolution operation on the fifth convolution output data to obtain sixth convolution output data, where a convolution kernel size is 1 × 1 × 128;
an upsampling layer, configured to perform upsampling on the sixth convolution output data to obtain a weight vector Pb (Q' × 128);
a first multiplication weighting module for weighting the vector Pb to each band of the input third-layer network output data xi (Q '× Q' × 128) by scaling operation to obtain the output result under the action of the attention machine in the spectral domain
Figure BDA0002664641860000021
An image domain attention mechanism network comprising
A seventh convolutional layer and a first reset module connected in series, wherein the seventh convolutional layer is configured to receive the third layer network output data and output the first reset output data by the first reset module, and a convolutional kernel of the seventh convolutional layer has a size of 3 × 3 × 128/r;
an eighth convolutional layer and a second reset module connected in series, wherein the eighth convolutional layer is configured to receive the third output data and output a second reset output data by the second reset module, and the eighth convolutional layer convolutional kernel has a size of 3 × 3 × 128/r;
the second multiplication weighting module is used for weighting the first reset output data to second reset output data to obtain second multiplication weighted output data;
the second classifier is used for performing classification calculation on the second multiplication weighted output data to obtain second classification output data;
a ninth convolutional layer and a third reset module connected in series, wherein the ninth convolutional layer is configured to receive the third output data and output a third reset output data by the third reset module, and the ninth convolutional layer convolutional kernel has a size of 1 × 1 × 128/r;
a third multiplication weighting module, configured to weight the second classification output data to third replacement output data to obtain a feature vector pi (Q' × 128);
a fourth reset module and a tenth convolutional layer connected in series, the fourth reset module to receive a feature vector pi (Q' × 128), the tenth convolutional layer to output tenth convolutional output data;
a first adding and weighting module, configured to weight the tenth convolution output data onto the third output data xi (Q' × 128), and obtain an output result under the action of the image domain attention machine
Figure BDA0002664641860000031
A second additive weighting module for outputting the output result under the action of the attention of the spectral domain
Figure BDA0002664641860000032
Output results under a sum-image domain attention mechanism
Figure BDA0002664641860000033
And performing addition weighting to obtain the fourth output data.
In a specific embodiment, the spectral image data is obtained by collecting a spectral image of corn kernels by using a visible/near-infrared hyperspectral imaging system in a wave band of 400-1000 nm.
In one embodiment, r is 2.
In one particular embodiment, the qxq is 21 x 21.
In a second aspect, the invention provides a method for training a neural network for recognizing grain mildewed grains, which comprises the following steps of
S200, preparing a training set: marking a sample with a kernel mildew area marked by an artificial quality inspection result in advance to form an interested area of Q multiplied by Q;
s204, extracting pixel point spectral image data xi (QxQx256) in the region;
s206, taking image data xi (Q multiplied by 256) as the input of the network;
and S208, taking the cross entropy function as a cost function of the network, optimizing network parameters by adopting a gradient descent method, and obtaining optimal network parameters according to the cost function, the accuracy and/or the F-Score result change trend.
In one embodiment, the expression of the first classifier is
Figure BDA0002664641860000034
N of the sample spectral image data xiIs the output characteristic, x, of the second fully-connected layer in the networki＝[xi1,xi2,…,xim]Where m is the number of output channels, f (w)jxi+bj) For the excitation function of hidden layer neurons, L is the number of neurons, wjRepresents a correspondence xiWeight value of j-th neuron of data, bjFor the jth hidden layer neuron bias term, ρjTo connect the output weight of the jth hidden layer, OiAnd dividing the output result into mildewed and healthy grains.
In one embodiment, the cost function used in the network training is
Figure BDA0002664641860000041
Wherein N represents the number of samples and,
Figure BDA0002664641860000042
in order to predict the value of the network,
Figure BDA0002664641860000043
is the true value of the sample. The learning rate in the network training process is initialized to 0.001 and the attenuation factor is 0.88.
In one embodiment, the accuracy is
Figure BDA0002664641860000044
F-Score is calculated by the precision P and the recall ratio R;
Figure BDA0002664641860000045
Figure BDA0002664641860000046
wherein TP is the number of true samples to be judged as true samples; FN is a true sample and is judged as the number of false samples; TN is judged as the number of the pseudo samples; FP is the number of samples that are discriminated as being true.
In a particular embodiment, comprises
Acquiring spectral image data of a region of interest of a sample to be identified; and
and inputting the image data into the network to obtain a recognition result.
In a third aspect, the present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the method as provided in the first aspect of the present application.
In a fourth aspect, the present invention also provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the method as provided in the first aspect of the present application when executing the program.
The invention has the following beneficial effects:
the invention provides a grain mildewed grain identification neural network and identification method for grain industry, which are oriented to the grain industry, have strong subjectivity, waste time and labor, and solve the problems of poor characterization capability and limited batch detection of a rapid detection model based on a traditional feature learning method.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
FIG. 1 illustrates a grain mildew particle recognition neural network model architecture diagram according to one embodiment of the present application.
Fig. 2 shows a flow chart of a grain mildew grain identification method according to an embodiment of the application.
FIG. 3 illustrates a spectral domain attention mechanism block diagram according to one embodiment of the present application.
FIG. 4 illustrates an image domain attention mechanism block diagram according to one embodiment of the present application.
Fig. 5 shows graphs of distributions of grain mildew kernel recognition neural models Acc and Cost for different corn varieties, (a) kyoto 968, (b) zheng 958, (c) xiaoyu 335, (d) jiyue 816 according to an embodiment of the present application.
FIG. 6 shows a schematic diagram of a computer device suitable for use to implement embodiments of the present application
Detailed Description
In order to more clearly illustrate the invention, the invention is further described below with reference to preferred embodiments and the accompanying drawings. Similar parts in the figures are denoted by the same reference numerals. It is to be understood by persons skilled in the art that the following detailed description is illustrative and not restrictive, and is not to be taken as limiting the scope of the invention.
First embodiment
As shown in figure 1, the invention provides a convolution recognition neural network for grain mildew grains, which comprises
A first winding layer for windingPerforming a first convolution operation on spectral image data xi (QxQx256) to obtain first-layer network output data, wherein the spectral image data is obtained by extracting pixel point spectral images in an interested area of the QxQ size of a grain sample, QxQ is an image scale, namely pixel point multiplication, and QxQ is Q2256 is the number of spectral bands in an image area consisting of pixel points;
the second convolution layer is used for carrying out second convolution operation on the first output data to obtain second-layer network output data;
the maximum pooling layer is used for pooling the second-layer network output data to obtain third-layer network output data xi (Q '× Q' × 128), wherein Q '× Q' is the image scale after pooling processing;
the attention mechanism module is used for performing kernel mildew characteristic mining on the third-layer network output data to obtain fourth-layer network output data;
the third convolution layer is used for carrying out third convolution operation on the fourth output data to obtain fifth-layer network output data;
the fourth convolution layer is used for performing fourth convolution operation on the fifth output data to obtain sixth-layer network output data;
the system comprises a first full connection layer, a second full connection layer and a first classifier which are connected in series, wherein the first full connection layer receives the output data of a sixth layer network, and the first classifier outputs an identification result.
In one embodiment, 4 sets of cascaded convolutional layers (Conv 1-Conv 4) all using convolution kernels of 3 × 3 size, with channel numbers of 64, 128, 256, 512; after each convolution operation, overfitting is avoided and the generalization capability of the network is improved through Batch normalization processing, and a ReLU function is further adopted for nonlinear transformation.
In one particular embodiment, a Kernel Extreme Learning Machine (KELM) is employed as the first classifier. The expression of the first classifier is
Figure BDA0002664641860000061
N of the sample spectral image data xiIs the output characteristic, x, of the second fully-connected layer in the networki＝[xi1,xi2,…,xim]Where m is the number of output channels, f (w)jxi+bj) For the excitation function of hidden layer neurons, L is the number of neurons, wjRepresents a correspondence xiWeight value of j-th neuron of data, bjFor the jth hidden layer neuron bias term, ρjTo connect the output weight of the jth hidden layer, OiAnd dividing the output result into mildewed and healthy grains. The cost function E of the ELM classifier is expressed as:
Figure BDA0002664641860000062
where s ═ 1,2, …, L, (wj, bj ═ 1,2, …, L), N of the sample spectral image data xiIs the output characteristic, x, of the second fully-connected layer in the networki＝[xi1,xi2,…,xim]，OiAnd (3) for outputting results including mildewed and healthy grains, yi is corresponding sample class mark data, and a Radial Basis Function (RBF) is introduced to improve the fitting and generalization capability of the ELM classifier to form a KELM model.
In one embodiment, the attention mechanism module includes a spectral domain attention mechanism network as shown in FIG. 3 and an image domain attention mechanism network as shown in FIG. 4.
A spectral domain attention mechanism network comprising
The global uniform pooling layer is used for pooling the third-layer network output data to obtain global uniform pooled output data;
the fifth convolution layer is used for performing fifth convolution operation on the global uniform pooling output data to obtain fifth convolution output data, wherein the convolution kernel size is 1 multiplied by 128/r, and r is the reduction rate;
a sixth convolution layer, configured to perform a sixth convolution operation on the fifth convolution output data to obtain sixth convolution output data, where a convolution kernel size is 1 × 1 × 128;
an upsampling layer, configured to perform upsampling on the sixth convolution output data to obtain a weight vector Pb (Q' × 128);
a first multiplication weighting module for weighting the vector Pb to each band of the input third-layer network output data xi (Q '× Q' × 128) by scaling operation to obtain the output result under the action of the attention machine in the spectral domain
Figure BDA0002664641860000071
An image domain attention mechanism network comprising
A seventh convolutional layer and a first reset module connected in series, wherein the seventh convolutional layer is configured to receive the third layer network output data and output the first reset output data by the first reset module, and a convolutional kernel of the seventh convolutional layer has a size of 3 × 3 × 128/r;
an eighth convolutional layer and a second reset module connected in series, wherein the eighth convolutional layer is configured to receive the third output data and output a second reset output data by the second reset module, and the eighth convolutional layer convolutional kernel has a size of 3 × 3 × 128/r;
the second multiplication weighting module is used for weighting the first reset output data to second reset output data to obtain second multiplication weighted output data;
the second classifier is used for performing classification calculation on the second multiplication weighted output data to obtain second classification output data;
a ninth convolutional layer and a third reset module connected in series, wherein the ninth convolutional layer is configured to receive the third output data and output a third reset output data by the third reset module, and the ninth convolutional layer convolutional kernel has a size of 1 × 1 × 128/r;
a third multiplication weighting module, configured to weight the second classification output data to third replacement output data to obtain a feature vector pi (Q' × 128); the response feature vector pi may be expressed as:
Figure BDA0002664641860000072
the weight coefficients of the function g (.) are characterized by a convolution of 1 × 1 × 128/r. The function f (.) is implemented by a gaussian function:
Figure BDA0002664641860000073
theta and
Figure BDA0002664641860000074
the weight coefficients are characterized by a convolution of 3 x 128/r, respectively. And c, (x) adopting a softmax (.) function, and adjusting the dimension of the output characteristic response result pi through convolution of 1 × 1 × 128 to obtain a characteristic vector pi (Q '× Q' × 128).
A fourth reset module and a tenth convolutional layer connected in series, the fourth reset module to receive a feature vector pi (Q' × 128), the tenth convolutional layer to output tenth convolutional output data;
a first adding and weighting module, configured to weight the tenth convolution output data onto the third output data xi (Q' × 128), and obtain an output result under the action of the image domain attention machine
Figure BDA0002664641860000081
A second additive weighting module for outputting the output result under the action of the attention of the spectral domain
Figure BDA0002664641860000082
Output results under a sum-image domain attention mechanism
Figure BDA0002664641860000083
And performing addition weighting to obtain the fourth output data.
In one embodiment, r is 1,2, 4, and 8, respectively, and the optimal reduction rate r in this example is determined to be 2 according to the network convergence trend and the stability result.
Second embodiment
A flow chart of a grain mildew identification method is shown in fig. 2.
The invention provides a method for training a neural network for identifying grain mildewed grains, which comprises the following steps of
S200, preparing a training set: marking a sample with a kernel mildew area marked by an artificial quality inspection result in advance to form an interested area of Q multiplied by Q;
in one embodiment, the corn kernel sample is collected and stored in a laboratory for later use at 4 ℃, a professional quality inspector performs artificial sensory judgment (national standard for corn mildew detection) on whether the corn kernel sample is mildewed or not, and a kernel mildewed area is marked to provide basis for the division of the hyperspectral image interesting area.
Classifying the samples into healthy grains and mildewed grains according to the manual quality inspection result, randomly selecting one part of samples for training a deep neural network and constructing a mildewed grain identification model, and using the other part of samples as independent samples for model verification.
In one embodiment, the corn varieties include Jingke 968, Zhengdan 958, Xiuyu 335, Ji 816, 800-1000 varieties per variety were selected as representative samples. The number distribution of the samples of the training set and the independent test set of corn kernels of different varieties is shown in table 1:
TABLE 1
Figure BDA0002664641860000084
2/3 samples were further randomly selected for each breed training set sample for construction of the discriminative model, with the remaining 1/3 samples for testing of the model.
And collecting hyperspectral image data of the corn grains by using a visible/near-infrared hyperspectral imaging system in a 400-1000 nm wave band.
S204, extracting pixel point spectral image data xi (QxQx256) in the region;
s206, taking image data xi (Q multiplied by 256) as the input of the network;
in a specific embodiment, Q is 9, 13, 17, 21, and 25, respectively, the images of the regions with different sizes are input into the network for training, and the optimal image region-of-interest size in this example is determined to be 21 × 21 according to the network convergence trend and the stability result.
And S208, taking the cross entropy function as a cost function of the network, optimizing network parameters by adopting a gradient descent method, and obtaining optimal network parameters according to the cost function, the accuracy and/or the F-Score result change trend.
In one embodiment, the network model cost function is represented as:
Figure BDA0002664641860000091
wherein N represents the number of samples and,
Figure BDA0002664641860000092
in order to predict the value of the network,
Figure BDA0002664641860000093
is the true value of the sample. The learning rate in the network training process is initialized to 0.001 and the attenuation factor is 0.88.
Acc formula:
Figure BDA0002664641860000094
wherein TP is the number of true samples to be judged as true samples; FN is a true sample and is judged as the number of false samples; TN is judged as the number of the pseudo samples; FP is a false sample and is judged to be in the iterative training process of the number of the true samples.
In a specific embodiment, the training results of the grain mildew identification neural network Cost and Acc of the four corn samples are shown in fig. 5, wherein the iteration times epoch of the grain mildew identification neural network networks are respectively 450, 382, 402 and 380, and the Kyoco 968, Zhengdan 958, Xiuyu 335 and Ji 816 are respectively; the cost function cost values of the training set respectively converge to 0.1989, 0.2011, 0.186 and 0.1969, and the discrimination accuracy Acc is 0.9391, 0.9058, 0.9466 and 0.9248; the cost function cost values of the test set respectively converge to 0.2092, 0.2196, 0.2019 and 0.2054, and the discrimination accuracy Acc is 0.9153, 0.8943, 0.9223 and 0.9156 respectively. Therefore, the established grain mildewed grain identification neural network is ideal for the whole training result of the hyperspectral data of the four corn grains.
The F-Score is calculated by the accuracy P and the recall ratio R, and the model performance evaluation index calculation formula is as follows:
Figure BDA0002664641860000095
Figure BDA0002664641860000101
wherein TP is the number of true samples to be judged as true samples; FN is a true sample and is judged as the number of false samples; TN is judged as the number of the pseudo samples; FP is the number of samples that are discriminated as being true.
In one particular embodiment, the end result is: the discrimination result of the grain mildew grain recognition neural network model on the corn mildew grains is shown in the following table 2: aiming at four corn kernel samples in a training set, the model discrimination accuracy (Acc) is 93-96%, the F-score is in the range of 95-97%, the test concentration Acc is 90-93%, the F-score is in the range of 92-96%, and the model has considerable discrimination accuracy and stability. The determination result of the corn mildew grains of the Yu 335 variety is optimal, the training set and the test set Acc are respectively 95.69% and 94.62%, and the F-score respectively reaches 96.80% and 95.74%. Therefore, the grain mildew identification neural network model has excellent identification capability on whether the corn grains are mildewed or not.
TABLE 2
Figure BDA0002664641860000102
In one embodiment, independent sample testing is included
And inputting the independent sample data into the established grain mildewed grain identification neural network model to obtain a judgment result of whether grains are mildewed or not, and completing the test of the model performance.
The results of the model independent sample testing are shown in table 3: the result of the test model Acc passing through the independent sample is 88-92%, the F-score is in the range of 90-94%, and the overall discrimination capability of the model is good. Therefore, the grain mildew grain identification neural network model established by the method can be used for quick nondestructive batch detection of the corn mildew grains.
TABLE 3
Figure BDA0002664641860000103
In one particular embodiment, the method comprises:
acquiring spectral image data of a region of interest of a sample to be identified; and inputting the image data into the network to obtain a recognition result.
The invention provides a neural network and a method for identifying grain mildewed grains, which construct a qualitative identification model suitable for on-site rapid batch detection of the mildewed grains by focusing characteristic information of local maps of the mildewed grains by introducing an attention mechanism, improve quality inspection efficiency, facilitate price per quality of the grain industry, guarantee grain benefits of farmers and national grain safety, and powerfully promote the development of grain purchasing quality inspection business in China.
Third embodiment
Fig. 6 shows a schematic structural diagram of a computer device according to another embodiment of the present application. The computer device 50 shown in fig. 6 is only an example, and should not bring any limitation to the function and the scope of use of the embodiments of the present application. As shown in fig. 6, computer device 50 is embodied in the form of a general purpose computing device. The components of computer device 50 may include, but are not limited to: one or more processors or processing units 500, a system memory 516, and a bus 501 that couples various system components including the system memory 516 and the processing unit 500.
Bus 501 represents one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example, such architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, micro-channel architecture (MAC) bus, enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.
Computer device 50 typically includes a variety of computer system readable media. Such media may be any available media that is accessible by computer device 50 and includes both volatile and nonvolatile media, removable and non-removable media.
The system memory 516 may include computer system readable media in the form of volatile memory, such as Random Access Memory (RAM)504 and/or cache memory 506. The computer device 50 may further include other removable/non-removable, volatile/nonvolatile computer system storage media. By way of example only, storage system 508 may be used to read from and write to non-removable, nonvolatile magnetic media (not shown in FIG. 6, and commonly referred to as a "hard disk drive"). Although not shown in FIG. 6, a magnetic disk drive for reading from and writing to a removable, nonvolatile magnetic disk (e.g., a "floppy disk") and an optical disk drive for reading from or writing to a removable, nonvolatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 501 by one or more data media interfaces. Memory 516 may include at least one program product having a set (e.g., at least one) of program modules that are configured to carry out the functions of embodiment one.
A program/utility 510 having a set (at least one) of program modules 512 may be stored, for example, in memory 516, such program modules 512 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which examples or some combination thereof may comprise an implementation of a network environment. Program modules 512 generally perform the functions and/or methodologies of the embodiments described herein.
Computer device 50 may also communicate with one or more external devices 70 (e.g., keyboard, pointing device, display 60, etc.), with one or more devices that enable a user to interact with the computer device 50, and/or with any devices (e.g., network card, modem, etc.) that enable the computer device 50 to communicate with one or more other computing devices. Such communication may occur via input/output (I/O) interfaces 502. Also, computer device 50 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN) and/or a public network, such as the Internet) through network adapter 514. As shown in FIG. 6, network adapter 514 communicates with the other modules of computer device 50 via bus 501. It should be appreciated that although not shown in FIG. 6, other hardware and/or software modules may be used in conjunction with computer device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
The processor unit 500 executes programs stored in the system memory 516 to execute various functional applications and data processing, for example, to implement a method for establishing and identifying a grain mildew granule identification neural network provided in one embodiment of the present application.
Aiming at the existing problems, the establishment method of the grain mildewed grain identification neural network and the computer equipment of the identification method are formulated, the attention mechanism is introduced to focus the local map characteristic information of the mildewed grains, a qualitative discrimination model suitable for the field rapid batch detection of the mildewed grains is constructed, the quality inspection efficiency is improved, the grain industry can conveniently quote according to the quality, the grain benefit of farmers and the national grain safety are guaranteed, and the method has a wide application prospect.
Fourth embodiment
Another embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor implements the method provided by the first embodiment. In practice, the computer-readable storage medium may take any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium.
A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present embodiment, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing. Computer program code for carrying out operations for aspects of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).
In the description of the present invention, it should be noted that, in the description of the present invention, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
It should be understood that the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention, and it will be obvious to those skilled in the art that other variations or modifications may be made on the basis of the above description, and all embodiments may not be exhaustive, and all obvious variations or modifications may be included within the scope of the present invention.

Claims (12)

1. A grain mildew particle recognition neural network is characterized by comprising
The first convolution layer is used for performing first convolution operation on spectral image data xi (QxQx256) to obtain first layer network output data, wherein the spectral image data is obtained by extracting pixel point spectral images in a region of interest of the QxQ size of a grain sample, the QxQ is an image scale, and 256 is the number of spectral bands;
the second convolution layer is used for carrying out second convolution operation on the first layer network output data to obtain second layer network output data;
the maximum pooling layer is used for pooling the second-layer network output data to obtain third-layer network output data xi (Q '× Q' × 128), wherein Q '× Q' is the image scale after pooling processing;
the attention mechanism module is used for performing grain mildew map feature mining on the third-layer network output data to obtain fourth-layer network output data;
the third convolution layer is used for carrying out third convolution operation on the fourth layer network output data to obtain fifth layer network output data;
the fourth convolution layer is used for performing fourth convolution operation on the fifth layer network output data to obtain sixth layer network output data;
the system comprises a first full connection layer, a second full connection layer and a first classifier which are connected in series, wherein the first full connection layer receives the output data of a sixth layer network, and the first classifier outputs an identification result.
2. The network of claim 1, wherein the attention mechanism module comprises
A spectral domain attention mechanism network comprising
The global uniform pooling layer is used for pooling the third-layer network output data to obtain global uniform pooled output data;
the fifth convolution layer is used for performing fifth convolution operation on the global uniform pooling output data to obtain fifth convolution output data, wherein the convolution kernel size is 1 multiplied by 128/r, and r is the reduction rate;
a sixth convolution layer, configured to perform a sixth convolution operation on the fifth convolution output data to obtain sixth convolution output data, where a convolution kernel size is 1 × 1 × 128;
an upsampling layer, configured to perform upsampling on the sixth convolution output data to obtain a weight vector Pb (Q' × 128);
a first multiplication weighting module for weighting the vector Pb to each band of the input third-layer network output data xi (Q '× Q' × 128) by scaling operation to obtain the output result under the action of the attention machine in the spectral domain
Figure FDA0002664641850000011
An image domain attention mechanism network comprising
A seventh convolutional layer and a first reset module connected in series, wherein the seventh convolutional layer is configured to receive the third layer network output data and output the first reset output data by the first reset module, and a convolutional kernel of the seventh convolutional layer has a size of 3 × 3 × 128/r;
an eighth convolutional layer and a second reset module connected in series, wherein the eighth convolutional layer is configured to receive the third output data and output a second reset output data by the second reset module, and the eighth convolutional layer convolutional kernel has a size of 3 × 3 × 128/r;
the second multiplication weighting module is used for weighting the first reset output data to second reset output data to obtain second multiplication weighted output data;
the second classifier is used for performing classification calculation on the second multiplication weighted output data to obtain second classification output data;
a ninth convolutional layer and a third reset module connected in series, wherein the ninth convolutional layer is configured to receive the third output data and output a third reset output data by the third reset module, and the ninth convolutional layer convolutional kernel has a size of 1 × 1 × 128/r;
a third multiplication weighting module, configured to weight the second classification output data to third replacement output data to obtain a feature vector pi (Q' × 128);
a fourth reset module and a tenth convolutional layer connected in series, the fourth reset module to receive a feature vector pi (Q' × 128), the tenth convolutional layer to output tenth convolutional output data;
a first adding and weighting module, configured to weight the tenth convolution output data onto the third output data xi (Q' × 128), and obtain an output result under the action of the image domain attention machine
Figure FDA0002664641850000021
A second additive weighting module for outputting the output result under the action of the attention of the spectral domain
Figure FDA0002664641850000022
Output results under a sum-image domain attention mechanism
Figure FDA0002664641850000023
And performing addition weighting to obtain the fourth output data.
3. The network of claim 1, wherein the spectral image data is obtained by collecting spectral images of corn kernels by a visible/near-infrared hyperspectral imaging system in a 400-1000 nm band.
4. The network of claim 2, wherein r is 2.
5. The network of any of claims 1-4, wherein QxQ is 21 x 21.
6. A method of training the network of any one of claims 1-5, comprising
S200, preparing a training set: marking a sample with a kernel mildew area marked by an artificial quality inspection result in advance to form an interested area of Q multiplied by Q;
s204, extracting pixel point spectral image data xi (QxQx256) in the region;
s206, taking spectral image data xi (Q multiplied by 256) as the input of the network;
and S208, taking the cross entropy function as a cost function of the network, optimizing network parameters by adopting a gradient descent method, and obtaining optimal network parameters according to the cost function and the change trend of the identification accuracy result.
7. The method of claim 6, wherein the first classifier is expressed as
Figure FDA0002664641850000031
N of the sample spectral image data xiIs the output characteristic, x, of the second fully-connected layer in the networki＝[xi1,xi2,…,xim]Where m is the number of output channels, f (w)jxi+bj) For the excitation function of hidden layer neurons, L is the number of neurons, wjRepresents a correspondence xiWeight value of j-th neuron of data, bjFor the jth hidden layer neuron bias term, ρjTo connect the output weight of the jth hidden layer, OiAnd dividing the output result into mildewed and healthy grains.
8. The method of claim 6,
the cost function used in the network training is
Figure FDA0002664641850000032
Wherein N represents the number of samples and,
Figure FDA0002664641850000033
in order to predict the value of the network,
Figure FDA0002664641850000034
for the true value of the sample, the learning rate in the network training process is initialized to 0.001, and the attenuation factor is 0.88.
9. The method of claim 6, wherein the accuracy is
Figure FDA0002664641850000035
F-Score is calculated by the precision P and the recall ratio R;
Figure FDA0002664641850000036
Figure FDA0002664641850000037
wherein TP is the number of true samples to be judged as true samples; FN is a true sample and is judged as the number of false samples; TN is judged as the number of the pseudo samples; FP is the number of samples that are discriminated as being true.
10. A method of identifying grain mildew granules using the method of any one of claims 6 to 9, comprising
Acquiring spectral image data of a region of interest of a sample to be identified; and
and inputting the image data into the network to obtain a recognition result.
11. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 6-9 or 10.
12. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 6-9 or 10 when executing the program.
CN202010914811.8A 2020-09-03 2020-09-03 Grain mildew grain identification neural network and identification method Active CN112232343B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010914811.8A CN112232343B (en) 2020-09-03 2020-09-03 Grain mildew grain identification neural network and identification method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010914811.8A CN112232343B (en) 2020-09-03 2020-09-03 Grain mildew grain identification neural network and identification method

Publications (2)

Publication Number Publication Date
CN112232343A true CN112232343A (en) 2021-01-15
CN112232343B CN112232343B (en) 2023-11-21

Family

ID=74117017

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010914811.8A Active CN112232343B (en) 2020-09-03 2020-09-03 Grain mildew grain identification neural network and identification method

Country Status (1)

Country Link
CN (1) CN112232343B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113109240A (en) * 2021-04-08 2021-07-13 国家粮食和物资储备局标准质量中心 Method and system for determining imperfect grains of grains implemented by computer
CN115201144A (en) * 2022-09-14 2022-10-18 武汉工程大学 Quantitative detection method, system and medium for amino acid and protein of rapeseed

Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191350A1 (en) * 2008-04-28 2011-08-04 Agency For Science, Technology And Research Method and system for concentration detection
CN107169090A (en) * 2017-05-12 2017-09-15 深圳市唯特视科技有限公司 A kind of special object search method of utilization content rings around information extraction characterization image
CN107578060A (en) * 2017-08-14 2018-01-12 电子科技大学 A kind of deep neural network based on discriminant region is used for the method for vegetable image classification
CN109376804A (en) * 2018-12-19 2019-02-22 中国地质大学（武汉） Based on attention mechanism and convolutional neural networks Classification of hyperspectral remote sensing image method
CN109993220A (en) * 2019-03-23 2019-07-09 西安电子科技大学 Multi-source Remote Sensing Images Classification method based on two-way attention fused neural network
CN110084794A (en) * 2019-04-22 2019-08-02 华南理工大学 A kind of cutaneum carcinoma image identification method based on attention convolutional neural networks
CN110188635A (en) * 2019-05-16 2019-08-30 南开大学 A kind of plant pest recognition methods based on attention mechanism and multi-level convolution feature
CN110473164A (en) * 2019-05-31 2019-11-19 北京理工大学 A kind of image aesthetic quality evaluation method based on attention mechanism
CN110516596A (en) * 2019-08-27 2019-11-29 西安电子科技大学 Empty spectrum attention hyperspectral image classification method based on Octave convolution
CN110689056A (en) * 2019-09-10 2020-01-14 Oppo广东移动通信有限公司 Classification method and device, equipment and storage medium
CN110991511A (en) * 2019-11-26 2020-04-10 中原工学院 Sunflower crop seed sorting method based on deep convolutional neural network
CN111178120A (en) * 2018-12-25 2020-05-19 中国科学院合肥物质科学研究院 Pest image detection method based on crop identification cascade technology
CN111274869A (en) * 2020-01-07 2020-06-12 中国地质大学（武汉） Method for classifying hyperspectral images based on parallel attention mechanism residual error network
CN111353531A (en) * 2020-02-25 2020-06-30 西安电子科技大学 Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism
CN111563520A (en) * 2020-01-16 2020-08-21 北京航空航天大学 Hyperspectral image classification method based on space-spectrum combined attention mechanism

Patent Citations (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110191350A1 (en) * 2008-04-28 2011-08-04 Agency For Science, Technology And Research Method and system for concentration detection
CN107169090A (en) * 2017-05-12 2017-09-15 深圳市唯特视科技有限公司 A kind of special object search method of utilization content rings around information extraction characterization image
CN107578060A (en) * 2017-08-14 2018-01-12 电子科技大学 A kind of deep neural network based on discriminant region is used for the method for vegetable image classification
CN109376804A (en) * 2018-12-19 2019-02-22 中国地质大学（武汉） Based on attention mechanism and convolutional neural networks Classification of hyperspectral remote sensing image method
CN111178120A (en) * 2018-12-25 2020-05-19 中国科学院合肥物质科学研究院 Pest image detection method based on crop identification cascade technology
CN109993220A (en) * 2019-03-23 2019-07-09 西安电子科技大学 Multi-source Remote Sensing Images Classification method based on two-way attention fused neural network
CN110084794A (en) * 2019-04-22 2019-08-02 华南理工大学 A kind of cutaneum carcinoma image identification method based on attention convolutional neural networks
CN110188635A (en) * 2019-05-16 2019-08-30 南开大学 A kind of plant pest recognition methods based on attention mechanism and multi-level convolution feature
CN110473164A (en) * 2019-05-31 2019-11-19 北京理工大学 A kind of image aesthetic quality evaluation method based on attention mechanism
CN110516596A (en) * 2019-08-27 2019-11-29 西安电子科技大学 Empty spectrum attention hyperspectral image classification method based on Octave convolution
CN110689056A (en) * 2019-09-10 2020-01-14 Oppo广东移动通信有限公司 Classification method and device, equipment and storage medium
CN110991511A (en) * 2019-11-26 2020-04-10 中原工学院 Sunflower crop seed sorting method based on deep convolutional neural network
CN111274869A (en) * 2020-01-07 2020-06-12 中国地质大学（武汉） Method for classifying hyperspectral images based on parallel attention mechanism residual error network
CN111563520A (en) * 2020-01-16 2020-08-21 北京航空航天大学 Hyperspectral image classification method based on space-spectrum combined attention mechanism
CN111353531A (en) * 2020-02-25 2020-06-30 西安电子科技大学 Hyperspectral image classification method based on singular value decomposition and spatial spectral domain attention mechanism

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
ABHIJIT GUHA ROY等: "Concurrent Spatial and Channel ‘Squeeze & Excitation’ in Fully Convolutional Networks", 《MICCAI 2018：INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION》, pages 421 - 429 *
H. SUN,X.ZHANG,X.LU AND S.WU: "spectral-spatial attention Network for Hyperspectral Image Classification", SPECTRAL-SPATIAL ATTENTION NETWORK FOR HYPERSPECTRAL IMAGE CLASSIFICATION, vol. 58, no. 5, pages 3232 - 3245, XP011784747, DOI: 10.1109/TGRS.2019.2951160 *
HAO SUN等: "Spectral–Spatial Attention Network for Hyperspectral Image Classification", 《IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING》, vol. 58, no. 5, pages 3232, XP011784747, DOI: 10.1109/TGRS.2019.2951160 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113109240A (en) * 2021-04-08 2021-07-13 国家粮食和物资储备局标准质量中心 Method and system for determining imperfect grains of grains implemented by computer
CN115201144A (en) * 2022-09-14 2022-10-18 武汉工程大学 Quantitative detection method, system and medium for amino acid and protein of rapeseed
CN115201144B (en) * 2022-09-14 2022-12-09 武汉工程大学 Quantitative detection method, system and medium for amino acid and protein of rapeseed

Also Published As

Publication number Publication date
CN112232343B (en) 2023-11-21

Similar Documents

Publication Publication Date Title
Bock et al. From visual estimates to fully automated sensor-based measurements of plant disease severity: status and challenges for improving accuracy
CN110309867B (en) Mixed gas identification method based on convolutional neural network
CN113259331B (en) Unknown abnormal flow online detection method and system based on incremental learning
CN113360701B (en) Sketch processing method and system based on knowledge distillation
CN112232343A (en) Neural network and method for recognizing grain mildewed grains
CN106570631B (en) P2P platform-oriented operation risk assessment method and system
CN111368656A (en) Video content description method and video content description device
Popa et al. Composite financial performance index prediction–a neural networks approach
CN113919886A (en) Data characteristic combination pricing method and system based on summer pril value and electronic equipment
Yipeng et al. Determination of wheat kernels damaged by fusarium head blight using monochromatic images of effective wavelengths from hyperspectral imaging coupled with an architecture self-search deep network
CN115423594A (en) Enterprise financial risk assessment method, device, equipment and storage medium
CN104504399B (en) A kind of multispectral data supervised classification method of the linear Correlation Information Entropy of combination
Wu et al. Rapid nondestructive detection of peanut varieties and peanut mildew based on hyperspectral imaging and stacked machine learning models
Matavulj et al. Advanced CNN architectures for pollen classification: Design and comprehensive evaluation
CN114048817A (en) Deep learning input set priority test method based on variation strategy
CN108663334A (en) The method for finding soil nutrient spectral signature wavelength based on multiple Classifiers Combination
Pujari et al. Detection and classification of fungal disease with radon transform and support vector machine affected on cereals
Liao et al. A hybrid CNN-LSTM model for diagnosing rice nutrient levels at the rice panicle initiation stage
CN116091103A (en) Method, device, electronic equipment and medium for measuring and calculating periodic environment remediation
CN115345488A (en) Wetland ecosystem service evaluation method, device, server and medium
CN115062815A (en) Method and system for measuring enterprise production and operation conditions by using thermal radiation data
CN107179297A (en) A kind of intelligent category of annatto authentication method and its platform
CN112819256A (en) Convolution time sequence room price prediction method based on attention mechanism
Lutsenko et al. Intellectual analysis of spectral images of plants
Singh et al. Rice kernels classification with deep learning using a modified dataset mimicking real-world conditions

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant