CN115965953A - Grain variety classification method based on hyperspectral imaging and deep learning - Google Patents
Grain variety classification method based on hyperspectral imaging and deep learning Download PDFInfo
- Publication number
- CN115965953A CN115965953A CN202310009003.0A CN202310009003A CN115965953A CN 115965953 A CN115965953 A CN 115965953A CN 202310009003 A CN202310009003 A CN 202310009003A CN 115965953 A CN115965953 A CN 115965953A
- Authority
- CN
- China
- Prior art keywords
- grain variety
- hyperspectral
- variety classification
- grain
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/10—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in agriculture
Abstract
The invention discloses a grain variety classification method based on hyperspectral imaging and deep learning, which belongs to the field of grain variety classification and comprises the following steps: acquiring a multi-channel hyperspectral image of a grain seed, preprocessing the multi-channel hyperspectral image to obtain a preprocessed hyperspectral image dataset, and acquiring a first characteristic map based on the preprocessed hyperspectral image dataset; the preprocessed hyperspectral image data set comprises a plurality of spliced images; constructing a grain variety classification network model, wherein the grain variety classification network model comprises a grain variety classification module; guiding and training a grain variety classification network model based on a grain variety classification module to obtain an optimized grain variety classification network model; and inputting the spliced image into an optimized grain variety classification network model to obtain a grain variety classification result. The invention provides a simple, efficient, lossless, economical and automatic grain variety classification method by utilizing hyperspectral imaging and deep learning technology.
Description
Technical Field
The invention belongs to the field of grain variety classification, and particularly relates to a grain variety classification method based on hyperspectral imaging and deep learning.
Background
Classification or identification of seed varieties originated in the middle of the 19 th century. Subsequently, a series of conventional rice seed variety classification methods have appeared, which mainly include morphological identification methods, physiological and biochemical identification methods, molecular biological identification methods, and the like, and these methods have advantages and have great differences in practical applications. With the development of hybridization technology, more and more mixed characteristics appear among different crop varieties, which brings great challenges to the traditional seed variety identification method.
Compared with the traditional method, the emerging machine learning method can quickly and reliably learn from a large amount of data, and has the advantages of no loss and low cost. While most machine learning methods have good performance, they rely heavily on artificially generated features designed for a particular task, limiting the applicability of these methods in complex/difficult situations. Furthermore, the function of such features may not be sufficient to distinguish subtle variations between different varieties or dramatic variations between the same variety. Therefore, automatically extracting more discriminative features is considered as a key to hyperspectral image classification. Compared with a manual generation method, the deep learning method can effectively and automatically extract the distinguishing features from the hyperspectral image. At present, a hyperspectral image classification method based on a depth network can be generally divided into three categories, including a spectral feature network, a spatial feature network and a spectral-spatial feature network. In general, models that comprehensively consider spatial and spectral information have better classification performance. Recently, many have attempted to apply attention to deep neural networks, greatly improving their performance. Therefore, it is of great significance to introduce an attention mechanism in the deep learning model.
Since the acquisition of hyperspectral data is expensive and time consuming, the number of images acquired at the end is not sufficient to support network training. Data augmentation, which is considered a viable approach to solving this problem, is typically addressed by generating new samples from existing samples.
Aiming at the problems, the invention provides a method for classifying grain varieties based on hyperspectral imaging and deep learning technology so as to solve the problems in the prior art.
Disclosure of Invention
The invention aims to provide a grain variety classification method based on hyperspectral imaging and deep learning, and aims to solve the problems in the prior art.
In order to achieve the aim, the invention provides a grain variety classification method based on hyperspectral imaging and deep learning, which comprises the following steps:
acquiring a multi-channel hyperspectral image of a grain seed, preprocessing the multi-channel hyperspectral image to obtain a preprocessed hyperspectral image dataset, and acquiring a first characteristic map based on the preprocessed hyperspectral image dataset; wherein the preprocessed hyperspectral image dataset comprises a plurality of stitched images;
constructing a grain variety classification network model, wherein the grain variety classification network model comprises a grain variety classification module; guiding and training the grain variety classification network model based on the grain variety classification module to obtain a trained grain variety classification model;
inputting the spliced image into the trained grain variety classification model to obtain a grain variety classification result.
Optionally, before the preprocessing of the multi-channel hyperspectral image,
and performing spectrum calibration and spectrum data baseline correction on the multichannel hyperspectral image based on the white polytetrafluoroethylene material plate and a multivariate scattering correction algorithm.
Optionally, the process of preprocessing the multi-channel hyperspectral image includes:
decomposing the multi-channel hyperspectral images to obtain a plurality of single-channel hyperspectral images, and discarding images corresponding to unclear spectral bands based on the single-channel hyperspectral images to obtain preprocessed hyperspectral images;
dividing the preprocessed hyperspectral images into a plurality of image subsets according to category labels, wherein each image subset comprises a plurality of single-channel hyperspectral images;
and cutting and splicing a plurality of single-channel hyperspectral images in each image subset to obtain a plurality of spliced images, wherein subimages of the spliced images come from the same wave band.
Optionally, the process of inputting the first feature map into the grain variety classification module to obtain the weighted feature map includes:
extracting texture information of the first feature map based on multiple edge detection operators to obtain a gradient map corresponding to each edge detection operator;
obtaining a second feature map based on a plurality of gradient maps;
obtaining a weight map based on the second feature map;
and obtaining a weighted feature map based on the first feature map and the weight map.
Optionally, the grain variety classification module adopts a mixed gradient domain attention module with a pyramid structure;
wherein the mixed gradient domain attention module of the pyramid structure comprises a pyramid module.
Optionally, extracting texture information of the first feature map based on multiple edge detection operators to obtain gradient maps corresponding to the edge detection operators, and after cascading the multiple gradient maps, inputting the gradient maps into a convolutional layer for feature extraction and channel recovery to obtain the second feature map;
wherein the plurality of edge detection operators include Sobel, scharr, laplace, roberts, and Prewitt.
Optionally, the second feature map is encoded based on the pyramid module to obtain a weight map; wherein the pyramid module comprises: seven convolutional layers, three largest pooling layers and three deconvolution layers; the number of convolution kernels of the convolution layer is equal to the number of channels of the second feature map, and the size of the kernels is 3 x 3; the kernel size of the maximum pooling layer is set to be 2 multiplied by 2, and the filling is set to be 1; the kernel size in the deconvolution layer is set to 2 x 2, with a step size set to 2.
An important feature of the pyramid module is that the module redistributes the proper weight of the feature map by encoding the gradient map extracted by the five classical image operators to effectively extract the detail features while reducing noise and interference information.
The invention has the technical effects that:
the invention adopts the design of introducing external prior knowledge into the attention mechanism, so that the model learning process is greatly influenced towards a beneficial direction. The invention can redistribute the learning weight for the characteristic diagram in the network learning process, so that the network can pay more attention to the edge and texture information of the grain seeds in the training process.
The invention realizes a simple, high-efficiency, lossless and automatic grain variety identification method based on hyperspectral imaging and deep learning technologies, enriches the current grain variety identification technical system, contributes to further strengthening the supervision of the seed market and ensures the legal rights and interests of breeding enterprises and rice farmers.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 is a flow chart of a method in an embodiment of the invention;
FIG. 2 is a diagram illustrating a step two data enhancement strategy according to an embodiment of the present invention;
FIG. 3 is a network architecture diagram of a step four hybrid gradient domain attention module in an embodiment of the present invention;
FIG. 4 is a flowchart of a verification experiment in an embodiment of the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
Example one
As shown in fig. 1 to 3, the present embodiment provides a grain variety classification method based on hyperspectral imaging and deep learning, and specifically provides a method for classifying rice seed varieties based on hyperspectral imaging and deep learning technology, including the following steps:
step one, acquiring a hyperspectral image data set of rice seeds, carrying out data preprocessing, converting a multichannel hyperspectral image into a series of single-channel images, and discarding partial unclear images according to priori knowledge.
Step two, as shown in fig. 2, according to a data enhancement strategy, each hyperspectral image with 200 seeds is randomly divided into 260 × 260 image blocks, and then every four image blocks are randomly spliced into an image. The method greatly relieves the problem of insufficient training set caused by hyperspectral imaging characteristics.
And step three, introducing an edge detection operator. And receiving a characteristic diagram with the size of BxC xHxW from a certain layer of the network as input, detecting and retaining edges by using Sobel, scharr, laplace, roberts and Prewitt edge detection operators, and extracting texture information.
(1) Sobel operator. It is mainly used as a discrete differential operator for edge detection. The approximate calculation rules for size and direction are as follows:
where Gx and Gy denote images with horizontal and vertical edge detection, respectively. The Sobel operator is more suitable for images with larger gray gradient and larger noise.
(2) Scharr operator. By increasing the distance between pixel values, weak edge information is effectively extracted, which is an implementation of enhancing the difference of Sobel operators. The Scharr operator and Sobel operator differ only in their convolution kernels.
(3) And (4) Laplace operator. The operator is isotropic and can sharpen boundaries and lines in any direction. It is typically used to determine whether an edge pixel is located in a bright or dark region of an image. It is a second derivative operator that produces steep zero crossings at edges, defined as follows:
where f denotes the digital image, x denotes the horizontal direction and y denotes the vertical direction.
(4) Roberts operator. The objective is to detect edges using local difference operators, which is defined as follows:
the operator has better detection effect on the vertical edge than the oblique edge.
(5) Prewitt operator. The operator performs neighborhood convolution on an image in image space using two directional templates. One directional template is used to detect horizontal edges and the other is used to detect vertical edges.
The form is defined as follows:
where d refers to the first derivative operation.
And step four, providing a mixed domain gradient attention module with a pyramid structure. The gradient map G with 5 channels of C obtained in the third step of connection 1 ，G 2 ，…，G 5 And obtaining an output with 5C channels, inputting the output into the convolutional layer for feature extraction and channel recovery, and generating a feature map with C channels, which is named as G. With the multi-resolution technique, a pyramid module is used to encode the feature map G. The module consists of seven convolutional layers, three max pooling layers, and three anti-convolutional layers. The network architecture of the module is shown in fig. 3.
An important feature of the attention module is that the module re-assigns the appropriate weights of the feature maps by encoding the gradient maps extracted by the five classical image operators to effectively extract detail features while reducing noise and interference information. Since the input and weighted feature maps are the same size and a dot product operation is used, this module can be viewed as an attention module based on mixed gradient domain design and can be combined with multiple networks in a low cost manner to achieve better classification performance.
And step five, after the input feature map is subjected to edge detection operators and a mixed gradient domain attention module to extract edge and texture features, the result obtained by multiplying the output weight map and the input image is continuously trained by a backbone network to obtain a final classification result.
Example two
The embodiment provides a verification test of a rice variety classification method based on hyperspectral imaging and deep learning technology, as shown in fig. 4, the verification test includes:
step one, selecting a data set. There are six major classes of rice seeds selected. Each type of rice seed comprises 50 hyperspectral images, wherein 35 images are randomly selected for training, and the rest 15 images are used for testing. 7000 training images and 3000 testing images with the size of each type of rice seeds being 520 x 520 are finally obtained by respectively adopting the data enhancement method provided by the invention on the training set and the testing set.
Step two, setting up an experiment. Implemented in the Pythrch 1.8.0 framework of Windows 10 machines and trained and tested on a platform equipped with 11 th generation Intel (R) Core (TM) i7-11700K@3.60GHz CPU, NVIDIA Geforce RTX 3060Ti (12 GB) GPU and 32GB RAM. CUDA and CUDNN are also used for acceleration. Furthermore, using the SGD optimizer, the raw learning rate was 0.04 and the basic cross entropy was used as a loss function.
And thirdly, evaluating indexes.
The classification precision of each rice seed; overall classification accuracy; a Kappa coefficient; macro-F1.
And step four, evaluating the classification result. The rice seed amplification hyperspectral image set generated based on the proposed data enhancement method adopts seven classical network structures to quantitatively evaluate the effectiveness of the proposed mixed gradient domain attention module and the edge detection operator in rice seed classification. According to the classification results of seven network structures under the conditions of adding and not adding the attention module and the edge detection operator, including classification accuracy (AA), overall classification accuracy (OA), macro-F1 and Kappa coefficient (Kappa) of each type of rice seeds, it is found that after the edge detection operator is introduced and the mixed gradient domain attention module is added, for the rice seed hyperspectral image set obtained by the method, each baseline model is remarkably improved in overall accuracy, macro-F1 and Kappa coefficient.
The above description is only for the preferred embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.
Claims (7)
1. A grain variety classification method based on hyperspectral imaging and deep learning is characterized by comprising the following steps:
acquiring a multi-channel hyperspectral image of a grain seed, preprocessing the multi-channel hyperspectral image to obtain a preprocessed hyperspectral image dataset, and acquiring a first characteristic map based on the preprocessed hyperspectral image dataset; wherein the preprocessed hyperspectral image dataset comprises a plurality of stitched images;
constructing a grain variety classification network model, wherein the grain variety classification network model comprises a grain variety classification module; guiding and training the grain variety classification network model based on the grain variety classification module to obtain a trained grain variety classification model;
inputting the spliced image into the trained grain variety classification model to obtain a grain variety classification result.
2. The grain variety classification method based on hyperspectral imaging and deep learning according to claim 1, characterized in that before preprocessing the multichannel hyperspectral image,
and performing spectrum calibration and spectrum data baseline correction on the multichannel hyperspectral images based on the white polytetrafluoroethylene material plate and a multivariate scattering correction algorithm.
3. The grain variety classification method based on hyperspectral imaging and deep learning according to claim 2, wherein the preprocessing process of the multichannel hyperspectral image comprises the following steps:
decomposing the multi-channel hyperspectral images to obtain a plurality of single-channel hyperspectral images, and discarding images corresponding to unclear spectral bands based on the plurality of single-channel hyperspectral images to obtain preprocessed hyperspectral images;
dividing the preprocessed hyperspectral images into a plurality of image subsets according to category labels, wherein each image subset comprises a plurality of single-channel hyperspectral images;
and cutting and splicing a plurality of single-channel hyperspectral images in each image subset to obtain a plurality of spliced images, wherein subimages of the spliced images come from the same wave band.
4. The grain variety classification method based on hyperspectral imaging and deep learning according to claim 1, wherein the process of inputting the first feature map into the grain variety classification module to obtain the weighted feature map comprises:
extracting texture information of the first feature map based on multiple edge detection operators to obtain a gradient map corresponding to each edge detection operator;
obtaining a second feature map based on a plurality of gradient maps;
obtaining a weight map based on the second feature map;
and obtaining a weighted feature map based on the first feature map and the weight map.
5. The grain variety classification method based on hyperspectral imaging and deep learning according to claim 4 is characterized in that the grain variety classification module adopts a mixed gradient domain attention module with a pyramid structure;
wherein the mixed gradient domain attention module of the pyramid structure comprises a pyramid module.
6. The grain variety classification method based on hyperspectral imaging and deep learning according to claim 4 is characterized in that texture information of the first feature map is extracted based on multiple edge detection operators to obtain gradient maps corresponding to the edge detection operators, and the gradient maps are cascaded and then input to a convolutional layer for feature extraction and channel recovery to obtain a second feature map;
wherein the plurality of edge detection operators include Sobel, scharr, laplace, roberts, and Prewitt.
7. The grain variety classification method based on hyperspectral imaging and deep learning according to claim 5, wherein the second feature map is encoded based on the pyramid module to obtain a weight map;
wherein the pyramid module comprises: seven convolution layers, three maximum pooling layers and three deconvolution layers; the number of convolution kernels of the convolution layer is equal to the number of channels of the second feature map, and the size of the kernels is 3 x 3; the kernel size of the maximum pooling layer is set to be 2 multiplied by 2, and the filling is set to be 1; the kernel size in the deconvolution layer is set to 2 x 2, with a step size set to 2.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310009003.0A CN115965953B (en) | 2023-01-04 | 2023-01-04 | Grain Variety Classification Method Based on Hyperspectral Imaging and Deep Learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310009003.0A CN115965953B (en) | 2023-01-04 | 2023-01-04 | Grain Variety Classification Method Based on Hyperspectral Imaging and Deep Learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115965953A true CN115965953A (en) | 2023-04-14 |
| CN115965953B CN115965953B (en) | 2023-08-22 |
Family
ID=87357774
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310009003.0A Active CN115965953B (en) | 2023-01-04 | 2023-01-04 | Grain Variety Classification Method Based on Hyperspectral Imaging and Deep Learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115965953B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110852227A (en) * | 2019-11-04 | 2020-02-28 | 中国科学院遥感与数字地球研究所 | Hyperspectral image deep learning classification method, device, equipment and storage medium |
| CN111914907A (en) * | 2020-07-13 | 2020-11-10 | 河海大学 | Hyperspectral image classification method based on deep learning space-spectrum combined network |
| CN113705526A (en) * | 2021-09-07 | 2021-11-26 | 安徽大学 | Hyperspectral remote sensing image classification method |
| WO2022073452A1 (en) * | 2020-10-07 | 2022-04-14 | 武汉大学 | Hyperspectral remote sensing image classification method based on self-attention context network |
| CN114882249A (en) * | 2022-03-28 | 2022-08-09 | 吉林工程技术师范学院 | Hyperspectrum-based seed variety quality detection method and system |
| CN115564996A (en) * | 2022-09-29 | 2023-01-03 | 安徽大学 | Hyperspectral remote sensing image classification method based on attention union network |
-
2023
- 2023-01-04 CN CN202310009003.0A patent/CN115965953B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110852227A (en) * | 2019-11-04 | 2020-02-28 | 中国科学院遥感与数字地球研究所 | Hyperspectral image deep learning classification method, device, equipment and storage medium |
| CN111914907A (en) * | 2020-07-13 | 2020-11-10 | 河海大学 | Hyperspectral image classification method based on deep learning space-spectrum combined network |
| WO2022073452A1 (en) * | 2020-10-07 | 2022-04-14 | 武汉大学 | Hyperspectral remote sensing image classification method based on self-attention context network |
| CN113705526A (en) * | 2021-09-07 | 2021-11-26 | 安徽大学 | Hyperspectral remote sensing image classification method |
| CN114882249A (en) * | 2022-03-28 | 2022-08-09 | 吉林工程技术师范学院 | Hyperspectrum-based seed variety quality detection method and system |
| CN115564996A (en) * | 2022-09-29 | 2023-01-03 | 安徽大学 | Hyperspectral remote sensing image classification method based on attention union network |
Non-Patent Citations (1)
| Title |
|---|
| 刘启超;肖亮;刘芳;徐金环;: "SSCDenseNet:一种空-谱卷积稠密网络的高光谱图像分类算法", 电子学报, no. 04 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115965953B (en) | 2023-08-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107292343B (en) | Hyperspectral remote sensing image classification method based on six-layer convolutional neural network and spectrum-space information combination | |
| CN105354865B (en) | The automatic cloud detection method of optic of multispectral remote sensing satellite image and system | |
| CN108052980B (en) | Image-based air quality grade detection method | |
| CN108109140A (en) | Low Grade Gliomas citric dehydrogenase non-destructive prediction method and system based on deep learning | |
| CN110702648B (en) | Fluorescent spectrum pollutant classification method based on non-subsampled contourlet transformation | |
| CN107590785A (en) | A kind of Brillouin spectrum image-recognizing method based on sobel operators | |
| Markkandeyan et al. | Deep learning based semantic segmentation approach for automatic detection of brain tumor | |
| CN114387454A (en) | Self-supervision pre-training method based on region screening module and multi-level comparison | |
| CN114241309A (en) | Rice sheath blight identification method and system based on ShuffleNet V2-Unet | |
| CN117607120A (en) | Food additive Raman spectrum detection method and device based on improved Resnext model | |
| CN115965953B (en) | Grain Variety Classification Method Based on Hyperspectral Imaging and Deep Learning | |
| CN111127407B (en) | Fourier transform-based style migration forged image detection device and method | |
| CN110866552A (en) | Hyperspectral image classification method based on full convolution space propagation network | |
| CN110992267A (en) | Abrasive particle identification method based on DPSR and Lightweight CNN | |
| CN113850284B (en) | Multi-operation detection method based on multi-scale feature fusion and multi-branch prediction | |
| CN113192076B (en) | MRI brain tumor image segmentation method combining classification prediction and multi-scale feature extraction | |
| CN113269701B (en) | Low-illumination image enhancement method based on attention guidance and kernel selection mechanism | |
| CN103761530A (en) | Hyperspectral image unmixing method based on relevance vector machine | |
| CN111709427A (en) | Fruit segmentation method based on sparse convolution kernel | |
| CN114821146A (en) | Enhanced weak supervision-based fine-grained Alzheimer's disease classification method | |
| Jizat et al. | Effect of Image Compression using Fast Fourier Transformation and Discrete Wavelet Transformation on Transfer Learning Wafer Defect Image Classification | |
| CN117173485B (en) | Intelligent classification system method and system for lung cancer tissue pathological images | |
| Silva et al. | An Evaluation of Image Preprocessing in Skin Lesions Detection | |
| CN115841594B (en) | Attention mechanism-based coal gangue hyperspectral variable image domain data identification method | |
| CN112733769B (en) | Hyperspectral image classification method based on multiband entropy rate superpixel segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |