NL2025810A

NL2025810A - Method for classifying and evaluating nitrogen content level of brassica rapa subsp. oleifera (brsro) canopy

Info

Publication number: NL2025810A
Application number: NL2025810A
Authority: NL
Inventors: He Yong; Cen Haiyan; Liu Fei; Fang Hui; Zhu Yueming; Ma Zhihong
Original assignee: Univ Zhejiang
Priority date: 2019-08-19
Filing date: 2020-06-11
Publication date: 2021-02-24
Also published as: CN110555395A; NL2025810B1

Abstract

The present invention discloses a method for classifying and evaluating a nitrogen content level of a Brassica rapa subsp. Oleifera (BRSRO) canopy, and belongs to the technical field of intelligent agriculture. The method includes: obtaining hyperspectral image samples of a BRSRO plant with different nitrogen gradients at different angles, and marking the nitrogen gradients of all hyperspectral images; performing spectral correction on the acquired hyperspectral image samples, segmenting the corrected hyperspectral image samples to generate hyperspectral data of BRSRO canopy, and saving' in .mat format; randomly selecting' the obtained hyperspectral data, averaging a spectrum of a region that is randomly selected many times, generating a data set, and inputting to a stacked autoencoder (SAE) neural network for training; forming a prediction data set of the BRSRO canopy based on different nitrogen gradients under the best characteristic unit obtained after the training of the SAE neural network, and constructing a prediction model; and performing category determination of a characteristic and a nitrogen gradient based on the prediction model, and completing the evaluation of a nitrogen content in the plant canopy represented by the characteristic.

Description

METHOD FOR CLASSIFYING AND EVALUATING NITROGEN CONTENT LEVEL OF BRASSICA RAPA SUBSP. OLEIFERA (BRSRO) CANOPY

TECHNICAL FIELD The present invention relates to the technical field of intelligent agriculture, and in particular to a method for classifying and evaluating a nitrogen content level of a Brassica rapa subsp. Oleifera (BRSRO) canopy.

BACKGROUND Nitrogen is an important component of proteins, chlorophylls, some vitamins and intracellular enzymes. Nitrogen deficiency affects the formation of these substances, resulting in poor nutritional growth and poor reproductive growth factors that make up the yield. In the absence of other limiting factors, sufficient nitrogen supply enables the Brassica rapa plants to grow tall, flourish and bloom to yield more pods. Since the pods are also the site for photosynthesis to produce dry matter of seeds, the connection between nitrogen supply and yield is obvious. With the wide application of spectroscopy in smart agriculture, the acquisition of phenotypic information of crop nitrogen content and multiple physiological divisions based on spectroscopy has become an effective means to achieve data acquisition for intelligent agriculture. The realization of non-destructive and rapid level determination through the hyperspectral imaging (HSI) technique of BRSRO canopy is faced with the curse of dimensionality in the hyperspectral data and has poor results like conventional methods. In recent years, various sensors and equipment have been developed for rapid and non-destructive evaluation of nitrogen in staple crops. Among them, the HSI-based monitoring and evaluation of nitrogen has been widely used.

HSI provides hundreds of continuous waveband reflection signals for each pixel of the tested sample, which provides rich data information for evaluating the nitrogen content and other physiclogical indicators of crops. However, the high dimensionality and high resolution of the data imply a dimensional disaster for data analysis and processing, which greatly restricts the efficiency of data mining algorithms. It is one of the problems in the research and application of agricultural big data to effectively extract the professional characteristics of data while reducing the dimensionality thereof.

In recent years, deep learning (DL) algorithms have been used for large-scale data processing, and have significantly improved the accuracy of data analysis in a variety of classification and regression tasks. It is a hotspot for the research and application of big data. Stacked autoencoders (SAEs) are commonly used for reducing data dimensionality or generating raw data for other classification/clustering methods. SAEs show their advantages compared with traditional techniques such as principal component analysis (PCA). The data compression and characteristic extraction based on the SAEs provide a new method of big data processing for smart agriculture.

SUMMARY An objective of the present invention is to provide a method for classifying and evaluating a nitrogen content level of a Brassica rapa subsp. Oleifera (BRSRO) canopy. The present invention realizes the dimensionality reduction and effective characteristic extraction of a hyperspectral image of BRSRO canopy and the rapid evaluation of the nitrogen content level of the BRSRO canopy based on a characteristic.

To achieve the above objective, the present invention provides a method for classifying and evaluating a nitrogen content level of a BRSRO canopy provided by the present invention, including the following steps: 1} obtaining hyperspectral image samples of a BRSRO plant with different nitrogen gradients at different angles, and marking the nitrogen gradients of all hyperspectral images; 2) performing spectral correction on the acquired hyperspectral image samples, segmenting the corrected hyperspectral image samples to generate hyperspectral data of BRSRO canopy, and saving in .mat format; 3) randomly selecting the hyperspectral data obtained in step 2), averaging a spectrum of a region that is randomly selected many times, generating a data set, inputting to a stacked autoencoder (SAE) neural network for training, and outputting a spectral characteristic of the BRSRO canopy under the best characteristic unit, which provides a basis for evaluating a nitrogen content of the canopy of the imaged plant; 4) processing to-be-evaluated data according to the above steps, forming a prediction data set of the BRSRO canopy based on different nitrogen gradients under the best characteristic unit obtained after the training of the SAE neural network, and constructing a prediction model for a nitrogen level of the BRSRO based on the spectral characteristic by using a classification and regression trees (CART) algorithm; and 5) processing to-be-tested hyperspectral data of the BRSRO canopy according to steps 2) and 3), performing category determination of a characteristic and a nitrogen gradient based on the prediction model, and completing the evaluation of the nitrogen content in the plant canopy represented by the characteristic.

In the above technical solution, the hyperspectral images of the BRSRO canopy are acquired at multiple imaging angles to explore the sensitivity of the angles for the rapid evaluation of the nitrogen content level of the BRSRO canopy. The neural network model is derived through training of the deep learning (DL) network, which is used to obtain the spectral characteristics of the spectral images of the BRSRO plant at the best angle, thereby obtaining the evaluation result of the nitrogen content level. The present invention greatly improves the evaluation efficiency.

Preferably, in step 1), the hyperspectral images of the canopy of the BRSRO plant are acquired at angles of 0°, 15° and 25°. In an actual application scenario, 25° is usually selected as the hyperspectral imaging angle, which is helpful for the SAE neural network to capture the nitrogen- related spectral characteristics.

Preferably, in step 2), a standard white board and a dark noise are used to perform the spectral correction on the generated hyperspectral images according to the following equation: kB lc = WIE x 100% In the equation, lc is a corrected hyperspectral image; [, is an acquired original hyperspectral image; B is a dark current image; W is a white image taken from the standard white board, and W represents a stable high-reflectance standard under a halogen lamp.

Preferably, the SAE neural network includes an input layer, a hidden layer and an output layer; the input layer has a 1 x N-dimensional training spectrum, and input S is defined as a set of S = {s{(l), s(2), s(3), ..., s{(n)}, where n=1, 2, 3, ... N; the hidden layer includes an encoding process and a decoding process; a first encoding layer encodes input data s{n) as the next encoding layer, that is, e(s), which is calculated as follows: e(s) = Relu(ws + be) In the equation, ws is an encoding weight matrix; be. is an encoding deviation vector used to all encoding processes until the final encoding process produces the input extracted characteristic; Relu is an activation function; this function is used to all encoding and decoding calculation processes for activation, and the encoded characteristic is decoded as follows:

d(s) = Relu(wdefs) + ba) In the equation, wa is a decoding weight matrix, and by is a decoding deviation vector.

5 Preferably, in step 3), when the hyperspectral data is randomly selected, an average spectrum selected in a random region of 4*4 pixels is used as a sample. In order to avoid overfitting of the evaluation model, the hyperspectral images obtained in step 1) are separately processed based on the total number thereof according to step 2). Then, the random selection is performed multiple times according to the method of step 3). The average spectrum of the selection region is calculated, and no less than 4,000 spectral samples are generated as a data set.

Preferably, in step 3), the sample is used as a 1 x 120 vector; the vector is input to the SAE neural network to be trained for 200 times and finally encoded into different characteristic units, such as 1 x 100, 1 x 80, 1 x 60, 1 x 40 , 1 x 20 and 1 = 5.

Preferably, in step 3), the number of 1 x 100 characteristic units is selected as an evaluation criteria for the nitrogen level of BRSRO canopy.

Compared with the prior art, the present invention has the following beneficial effects; The classification and evaluation method of the present invention is based on the DL method to conduct experiments and data analysis on the nitrogen content levels of the BRSRO canopy under three hyperspectral imaging angles. The present invention obtains the best imaging angle, and directly acquires the spectral images of the to-be-tested plant under the best imaging angle, which greatly improves the evaluation efficiency.

BRIEF DESCRIPTION OF DRAWINGS FIG. 1 shows a hyperspectral imaging system according to an example of the present invention.

FIG. 2 shows a hyperspectral imaging angle adjuster according to an example of the present invention.

FIG. 3 is a flowchart of hyperspectral image preprocessing according to an example of the present invention.

FIG. 4 is a structural diagram of a stacked autoencoder (SAE) neural network according to an example of the present invention.

FIG. 5 shows different data characteristics extracted from 5 to 100 units of the same spectral data according to an example of the present invention, where (a), (b), (cc), (d), (e) and {£) correspond to different signals extracted by a characteristic unit from a wavelength.

FIG. 6 1s a diagram showing a classification and evaluation result of four «classification algorithms on different numbers of characteristics according to an example of the present invention.

FIG. 7 is a diagram showing a classification result of data taken under three camera angles by four models of different characteristic units according to an example of the present invention.

DETAILED DESCRIPTION To make the objectives, technical solutions and advantages of the present invention clearer, the present invention is described in more detail with reference to the examples and accompanying drawings.

Example This example provides a method for classifying and evaluating a nitrogen content level of a BRSRO canopy. The method is realized by using a hyperspectral imaging device with an adjustable imaging angle. It is based on a deep learning (DL) method to conduct experiments and data analysis on the nitrogen content levels of the BRSRO canopy under three hyperspectral imaging angles and obtain the best imaging angle.

Referring to FIG. 1, the hyperspectral imaging device with an adjustable imaging angle is composed of a sCMOS camera, an imaging spectrometer, a high-resolution lens, two 150 W halogen tungsten lamps and a conveyor belt operated by a stepper motor. The hyperspectral imaging device adopts line scan imaging, and is also provided with an imaging angle adjuster. Referring to FIG. 2, the imaging angle adjuster supports hyperspectral imaging of BRSRO canopy in the range of 0° to 90°.

In this example, the method for «classifying and evaluating a nitrogen content level of a BRSRO canopy includes the following steps: A hyperspectral image acquired by the above imaging device was first preprocessed as follows: In order to eliminate the interference of an environmental factor and a lighting factor, white and dark images were used to correct the hyperspectral image according to equation (1): Ie = 222 x 100% (1) In the equation, lc is a corrected hyperspectral image; I, is an acquired original hyperspectral image; B is a dark current image; W is a white image taken from a standard white board, and W represents a stable high-reflectance standard under the halogen lamp (reaching 99% reflectance range).

After the correction, region of interest (ROI) selection and plant canopy segmentation were performed, as shown in FIG. 3. In this step, a ROI mask range was manvally constructed by using ENVI software. The ROI in each hyperspectral image represented the entire canopy, so the mask range was about 3.5-1.0, covering all spectral images with total pixels. Then a canopy data cube in the hyperspectral image was generated and saved as a .mat format file. The spectra from all bands in the .mat file were randomly selected as an input of a stacked autoencoder (SAE) neural network designed for characteristic extraction and data dimension derivation. A spectral resolution was set to

120 and an average spectrum was set to a 1 x 120 vector as a training input and a decoding output of the SAE network.

A spectral data dimensionality reduction and characteristic extraction structure based on SAE is shown in FIG. 4. The basic structure of the SAE included an input layer, several hidden layers and an output layer. In FIG.

4, the hidden layer represents a spectral characteristic extracted from the input spectrum by an encoding layer, and its reliability is verified by back propagation by a decoding layer on a right side of the neural network.

The input layer had a 1 x N-dimensional training spectrum, and input S was defined as a set of S = {s{1), s(2}), s(3)}, ..., s{n)}, where n = 1, 2, 3, ... N. The hidden layer included an encoding process and a decoding process. A first encoding layer encoded input data s(n) as the next encoding layer, that is, ets), which was calculated according to Equation (2): e(s) = Relu(ws + be) (2) In the equation, ws is an encoding weight matrix; b. is an encoding deviation vector used to all encoding processes until the final encoding process produces the input extracted characteristic; Relu is an activation function.

This function was used to all encoding and decoding calculation processes for activation, and the encoded characteristic was decoded according to Equation (3): d(s) = Relu(wae(s) + ba (3) In the equation, wa is a decoding weight matrix, and ba is a decoding deviation vector. Since the SAE neural network was designed to reproduce input data sm as d(s™) through multiple hidden layers, all coded hidden layers were expected to extract the most representative characteristic of the input. Thus, the extracted characteristics would become more abstract as the encoding layer deepened. It was expected that all decoding hidden layers reproduced the input data through feedback and penalty functions throughout the decoding process.

During spectrum extraction, random spectrum extraction was performed on 2*2 pixels of the spectral image, and a total of 48,000 spectral data were selected. Four types of machine learning (ML) classifiers were used to classify characteristic units obtained from BRSRO samples of five nitrogen gradients. The results show that the algorithm proposed by the present invention realized effective characteristic extraction, and is applicable for hyperspectral imaging data analysis and numerical control evaluation to achieve effective dimensionality reduction of hyperspectral data while achieving effective characteristic extraction.

In the SAE network of five characteristic units, the spectral samples were first input as 1 x 120 vectors. The vectors were then encoded into 1 x 100, 1 x 80, 1 x 60, 1 x 40, 1 x 20 and 1 x 5 networks. After 200 trainings, the encoded characteristics were saved as comma separated value (CSV) files for classification and evaluation supported by ML. The constructed neural network used the Python program based on Keras (an application program interface, API), and all encoding and decoding activation functions used Relu.

In the training process, each spectral sample was trained 200 times to ensure that the extracted characteristic of the original input was decoded into the input spectrum with minimum loss. Referring to FIG. 5, the SAE accurately reproduced the spectral input, thereby ensuring that the results were reliable for the evaluation of the nitrogen level of the BRSRO canopy.

After the characteristics were extracted from the spectral data, the encoded characteristic data were saved as common format CSV files. In order to check the characteristic extraction effect, the present invention used four classification models provided by the scikit-learn tool, including classification and regression trees (CART), naive

Bayes (NB), AdaBoost and random forest. The characteristic data set was divided into a training data set (80%) and a test data set (20%) for the classification of the nitrogen content levels of the BRSRO canopy.

Since all individual characteristics always belonged to one of the following 4 categories: true positive (TP), true negative (IN), false positive (FP) and false negative (FN), the total test characteristics were set to TP + FP + TN + FN. A confusion matrix was used to summarize the performance of the classification models. Pp N

EE NEN | Therefore, the evaluation of the nitrogen levels of the BRSRO canopy was achieved through the characteristic classification results, and the overall accuracy (ACC) is as follows: acc = EIN 100% TP+FP+TN+FN In most cases, the average size of the hyperspectral imaging data was about 100 MB per file. After image processing and characteristic extraction, the encoded characteristics only required 400 kb of space for 100 characteristic units, and 256 times of dimensionality reduction were achieved. The extracted characteristics were effectively used in the evaluation and analysis of the nitrogen levels of the BRSRO canopy. In order to evaluate the hyperspectral imaging data at different imaging angles and explore a method suitable for classifying the nitrogen content in the BRSRO canopy, the experiment designed five nitrogen gradients to obtain the BRSRO samples, and the data were acquired at three imaging angles to form a corresponding data set.

The evaluation results of the nitrogen level of the BRSRO canopy based on characteristics are shown in FIG. 6. The figure clearly shows the classification results of nitrogen in the BRSRO canopy. When the extracted spectral characteristic units decreased, the classification accuracy decreased accordingly. The results show that the hyperspectral characteristic extraction units in the range of 60-100 all realized the accurate evaluation of the nitrogen content in the BRSRO canopy. However, as the number of characteristics increased, the classification accuracy increased accordingly.

In order to verify the classification accuracy of the nitrogen content levels of the BRSRO canopy at different angles, the present invention used four classifiers to evaluate the hyperspectral imaging data at three hyperspectral imaging angles (0°, 15°, 25°). These angles showed the differences in the classification of the nitrogen content levels of the BRSRO canopy. As shown in FIG. 7, in a particular classifier, the classification accuracy of the nitrogen content level in the BRSRO canopy increased with the increase of characteristic units, but different angles showed corresponding differences. FIG. 7 shows that among the three angles, 25° maintained excellent performance.

Through the adjustment of the hyperspectral imaging angles, this example realized the dimensional compression and characteristic extraction of the hyperspectral imaged by the SAEs-based DL method and the evaluation of the nitrogen content levels of the BRSRO canopy based on the extracted characteristics. The analysis of the data at the three angles shows that the characteristics extracted from the hyperspectral data at 25° had the best classification effect on the nitrogen content levels of the BRSRO canopy.

Finally, the hyperspectral images of the BRSRO plant to be tested were acquired at the optimal angle of 25°. The spectral characteristics were obtained by using the trained network model, and the nitrogen content levels of the BRSRO canopy were matched according to the spectral characteristics, thereby rapidly obtaining the nitrogen content levels of the BRSRO canopy.

Claims

CONCLUSIONS

A method for classifying and evaluating a nitrogen content level of a foliage of Brassica rapa subsp. Oleifera (BRSRO), comprising the following steps: 1) obtaining hyperspectral image samples from a BRSRO plant with different nitrogen gradients at different angles, and marking the nitrogen gradients of all hyperspectral images; 2) performing spectral correction on the obtained hyperspectral image samples, segmenting the corrected hyperspectral image samples to generate hyper-spectral data of BRSRO foliage, and saving in .mat format; 3) randomly selecting the hyperspectral data obtained in step 2), averaging a spectrum of a region that has been randomly selected many times, generating a data set, entering into a stacked autoencoder (SAE) neural network for training, and outputting a spectral characteristic of the BRSRO-1of under the best characteristic unit, providing a basis for evaluating a nitrogen content of the Foliage of the imaged plant; 4) processing the data to be evaluated according to the steps above, forming a prediction data set of the BRSRO foliage based on different hydrogen gradients under the best characteristic unit obtained after the training of the SAE neural network, and constructing a prediction model for the nitrogen level of the BRSRO foliage based on the spectral characteristic using a classification and regression trees (CART) algorithm;

5) processing the hyperspectral data of the BRSRO foliage to be tested according to steps 2) and 3), performing a trait categorization and nitrogen gradient based on the prediction model, and completing the evaluation of the nitrogen content in the foliage represented by the trait.

A method for classifying and evaluating a nitrogen content level of a BRSRO foliage according to claim 1, wherein in step 1) the hyperspectral images of the foliage of the BRSRO plant are obtained at angles of 0 °, 15 ° and 25 °. °.

A method for classifying and evaluating a nitrogen content level of a BRSRO foliage according to claim 1, wherein a standard whiteboard and a dark noise are used to perform the spectral correction on the generated hyperspectral images according to the following equation:

LB le = Tg x 100% where I. is a corrected hyperspectral image; [, is an acquired original hyperspectral image; B is a dark flow image; W is a white image taken from the standard whiteboard, and W represents a stable high reflection standard under a halogen lamp.

The method of classifying and evaluating a nitrogen content level of a BRSRO foliage according to claim 1, wherein the SAE neural network comprises an input layer, a concealed layer and a release layer; where the input layer has a 1 x N-dimensional training spectrum, and input S is defined as a set of S = {s {(l), s (2}, s (3), .., s (n)}, where n = 1, 2, 3, .N; wherein the hidden layer comprises an encoding process and a decoding process; wherein a first encoding layer encodes input data s (n) as the next encoding layer, which is e (s), which is calculated as follows:

e (s) = Reluí (ws + De) where ws is an encoding weight matrix, be is an encoding deviation vector used for all encoding processes until the final encoding process produces the attribute extracted from the input, Relu is an activation function; wherein this {function is used for all encryption and decryption processes for activation; and the encoded feature is decoded as follows: d (s) = Relu (wge (s) + bs) where wa is a decoding weight matrix, and ba is a decoding deviation vector.

The method of classifying and evaluating a nitrogen content level of a BRSRO foliage according to claim 1, wherein in step 3) when the hyperspectral data is randomly selected, an average spectrum is selected in a random area of 4 * 4 pixels. is used as a sample, and no less than 4,000 spectral data is generated as a dataset.

A method for classifying and evaluating a nitrogen content level of a BRSRO foliage according to claim 5, wherein in step 3) the sample is used as 1 x 120 vector; where the vector is input for the SAE neural network to be trained 200 times and finally encoded in different characteristic units such as 1 x 100, 1 x 80, 1 x 60, 1 x 40, 1 x 20 and 1 x 5.

A method for classifying and evaluating a nitrogen content level of a BRSRO foliage according to claim 6, wherein in step 3) the number of 1 x 100 characteristic units is selected as an evaluation criterion for the nitrogen level of BRSRO foliage.

—-O-O-Oo-