GB2596886A

GB2596886A - Method and device for performing inversion of crop leaf area index

Info

Publication number: GB2596886A
Application number: GB2101106.9A
Authority: GB
Inventors: Dong Yingying; Li Xueling; Zhu Yining; Hu Tingguang; Huang Wenjiang; Ye Huichun; Wang Kun
Original assignee: Aerospace Information Research Institute of CAS
Current assignee: Aerospace Information Research Institute of CAS
Priority date: 2020-07-14
Filing date: 2021-01-27
Publication date: 2022-01-12
Anticipated expiration: 2041-01-27
Also published as: GB202101106D0; CN111814707A; GB2596886B; CN111814707B

Abstract

The method and device for performing inversion of a crop leaf area index are provided. The method includes acquiring remote sensing image data, performing pre-processing on the remote sensing image data to obtain hyperspectral data and inputting the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model. The target reference inversion model is obtained by optimizing a reference inversion model based on target training samples. The target training samples are obtained by performing data augmentation on original training samples using a Dual-Generative Adversarial network (DualGAN).

Description

METHOD AND DEVICE FOR PERFORMING INVERSION OF CROP LEAF AREA

INDEX

FIELD

100011 The present disclosure relates to the field of remote sensing technology, and in particular to a method and device for performing inversion of a crop leaf area index.

BACKGROUND

[0002] Leaf area index (LAI) refers to a half of a total surface area of leaves in a unit surface area of land. The LAI is used to quantitatively characterize a density degree of crop leaves, to provide an important index indicating growth and development features and physical conditions of a plant community. Quantitatively acquiring spatio-temporal continuous vegetation LAI in a region is of great significance in monitoring crop growth and estimating crop yield. Remote sensing technology has advantages of periodic observation and large area coverage in acquiring ground information so that the remote sensing teclmology plays an important role in agricultural resource monitoring. With the remote sensing technology, spatial and time-series distribution information of key biophysical parameters of terrestrial vegetation can be captured. Therefore, with the remote sensing technology, a feasible method can be provided to observe leaf area index in regional scale.

[0003] In conventional remote sensing inversion methods, a statistical model establishes a relationship between a vegetation index and a leaf area index through an empirical formula such as a linear formula, a quadratic formula, an exponential formula, and a logarithmic formula. In the statistical models, the canopy radiation transmitting process is simplified as simple function relationships, for which the inversion accuracy of leaf area index is limited.

Physical model analyzes and describes the internal mechanisms of various physical effects within crop growth and light transmission processes by canopy radiative transfer models. The physical model is very complex, so a fast inversion for the model is generally realized through iteration by establishing a lookup table. However, in case of performing the inversion of LAI in a large area, the lookup table is required to have enough dimensions in order to achieve desired accuracy. In this case, a sampling interval for variables must be small enough, and thus the applicability is restricted. Inversion accuracy for the LAI may be improved by using deep learning. However, in case of performing quantitative inversion for the LAT in a practical area a small volume of collected data cannot meet the requirement. Therefore, in case that there are only a small volume of hyperspectral data and LAI data collected in a real time manner, there are disadvantages in improving inversion accuracy for the LAI by using deep learning.

SUMMARY

[0004] In order to solve the above problem, a method and device for performing inversion of crop leaf area index are provided according to the present disclosure. With the method and the device, inversion accuracy for the leaf area index is improved based on small samples [0005] In order to realize the above object, the following technical solutions are provided according to the present disclosure.

[0006] A method for performing inversion of a crop leaf area index is provided.

100071 The method includes: acquiring remote sensing image data; performing pre-processing on the remote sensing image data to obtain hyperspectral data; and inputting the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model, where the target reference inversion model is obtained by optimizing a reference inversion model using target training samples, the target training samples are obtained by performing data augmentation on original training samples using a Dual-Generative Adversarial Network (DualGAN).

[0008] In an embodiment, the method further includes: acquiring remote sensing image data and a leaf area index product; performing pre-processing on the remote sensing image data and the leaf area index product to obtain hyperspectral data and corresponding leaf area indexes, and randomly selecting data from the pre-processed data to form the original training samples, wherein the original training samples comprise a hyperspectral data set and a leaf area index data set: performing data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples; and training the reference inversion model provided in advance using the target training samples to obtain the target reference inversion model.

[0009] In an embodiment, performing data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples includes: determining the DualGAN, where the DualGAN comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator is configured to generate a leaf area index based on the hyperspectral data set, the second generator is configured to generate hyperspectral data based on the leaf area index data set, the first discriminator is configured to determine a probability that the generated leaf area index is from a true leaf area index data set, and the second discriminator is configured to determine a probability that the generated hyperspectral data is from a true hyperspectral data set; performing optimization in an iterative manner using the first generator, the first discriminator, the second generator, and the second discriminator, to cause the generated leaf area index and the generated hyperspectral data to meet a predetermined condition, so as to obtain augmented data, where the augmented data comprises the generated leaf area index and the generated hyperspectral data; and mixing the augmented data and the original training samples to obtain the target training samples.

100101 In an embodiment, the method further includes: processing the original training samples using the DualGAN to obtain the augmented data, where processing the original training samples using the DualGAN to obtain the augmented data includes: setting thresholds, wherein the thresholds comprise a threshold of a relative error of a leaf area index and a threshold of a sum of relative square errors for all bands of hyper-spectrwn; acquiring generated data within a range corresponding to the thresholds and determining a data set of a relative error of the leaf area index and a data set of a relative root mean square error of the hyper-spectrum based on the generated data transforming the data set of the relative error of the leaf area index and the data set of the relative root mean square error of the hyper-spectrum to data sets following a standard normal distribution; and selecting the augmented data from the generated data based on the data sets following the standard normal distribution.

[0011] In an embodiment, the first generator is a residual network including several basic residual blocks, in which the main path of the basic residual block is a stack of two convolution layers, wherein the one-dimensional convolution kernel size is 3 and stride is 1, 30 which activated by the ReLU activation function.

100121 In an embodiment, the second generator is configured to implement a mapping from low-dimensional data to high-dimensional data. A first layer of the second generator is a fully connected layer including 16 neurons. The second generator superposes up-sampling data of the second generator on a feature extracted by the first generator through convolution.

[0013] A device for performing inversion of a crop leaf area index is further provided according to an embodiment of the present disclosure. The device includes an acquiring unit, a processing unit and an input unit.

[0014] The acquiring unit is configured to acquire remote sensing image data.

[0015] The processing unit is configured to perform pre-processing on the remote sensing image data to obtain hyperspectral data.

[0016] The input unit is configured to input the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model. The target reference inversion model is obtained by optimizing a reference inversion model based on target training samples. The target training samples are obtained by performing data augmentation on original training samples using a DualGAN.

[0017] In an embodiment, the device further includes: a data acquiring unit, a pre-processing unit, a data augmenting unit and a training unit.

[0018] The data acquiring unit is configured to acquire remote sensing image data and a leaf area index product.

[0019] The pre-processing unit is configured to perform pre-processing on the remote sensing image data and the leaf area index product to obtain hyperspectral data mid corresponding leaf area indexes, and randomly selecting data from the pre-processed data to form the original training samples, wherein the original training samples comprise a hyperspectral data set and a leaf area index data set.

[0020] The data augmenting unit is configured to perform data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples.

[0021] The training unit is configured to train the reference inversion model provided in advance using the target training samples to obtain the target reference inversion model.

[0022] In an embodiment, the data augmentation unit includes a network determining subunit, an optimizing subunit and a mixing subunit.

[0023] The network determining subunit is configured to determine the DualGAN, wherein the DualGAN comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator is configured to generate a leaf area index based on the hyperspectral data set. the second generator is configured to generate hyperspectral data based on the leaf area index data set, the first discriminator is configured to determine a probability that the generated leaf area index is from a true leaf area index data set, and the second discriminator is configured to determine a probability that the generated hyperspectral data is from a true hyperspectral data set.

[0024] The optimizing subunit is configured to perform optimization in an iterative manner using the first generator, the first discriminator, the second generator, and the second discriminator, to cause the generated leaf area index and the generated hyperspectral data to meet a predetermined condition, so as to obtain augmented data, wherein the augmented data comprises the generated leaf area index and the generated hyperspectral data.

100251 The mixing subunit is configured to mix the augmented data and the original training samples to obtain the target training samples.

[0026] in an embodiment, the device further includes an augmentation processing subunit configured to process the original training samples using the DualGAN to obtain the augmented data.

[0027] The augmentation processing subunit, for processing the original training samples using the DualGAN to obtain the augmented data, is configured to: set thresholds, wherein the thresholds comprise a threshold of a relative error of a leaf area index and a threshold of a sum of relative square errors for all bands of hyper-spectrum; acquire generated data within a range corresponding to the thresholds and determine a data set of a relative error of the leaf area index and a data set of a relative root mean square error of the hyper-spectrum based on the generated data; transform the data set of the relative error of the leaf area index and the data set of the relative root mean square error of the hyper-spectrum to data sets following a standard normal distribution; and select the augmented data from the generated data based on the data sets following the standard normal distribution.

[0028] Compared with the conventional technology, a method and device for performing inversion of a crop leaf area index are provided according to the present disclosure. Remote sensing image data is acquired. Pre-processing is performed on the remote sensing image data to obtain hyperspectral data. The hyperspectral data is inputted into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model. The target reference inversion model is obtained by optimizing a reference inversion model using target training samples. The target training samples are obtained by performing data augmentation on original training samples using a DualGAN. In this way, the number of training samples required by a neural network can be met by performing data augmentation on the hyperspectral data and leaf area index that are collected in a real time manner, thereby high precision of the leaf area index based on small samples collected in a real time manner is achieved by estimating the leaf area index based on the hyperspectral data by using the neural network.

BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly describe the technical solutions in the embodiments of the present disclosure or the technical solutions in the conventional technology, drawings to be used in the description of the embodiments of the present disclosure or the convention& technology are briefly described hereinafter. It is apparent that the drawings described below show merely the embodiments of the present disclosure. Those skilled in the art may obtain other drawings according to the provided drawings without any creative effort.

[0030] Figure 1 is a schematic flowchart of a method for performing inversion of a crop leaf area index according to an embodiment of the present disclosure; [0031] Figure 2 is a schematic diagram of a leaf area index generator according to an embodiment of the present disclosure; [0032] Figure 3 is a schematic diagram of a hyper-spectrum generator according to an

embodiment of the present disclosure;

[0033] Figure 4 is a flowchart of a DualGAN network for generating hyperspeetral data and a leaf area index according to an embodiment of the present disclosure; and [0034] Figure 5 is a schematic structural flowchart of a device for performing inversion of a crop leaf area index according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF EMBODIMENTS

[0035] Technical solutions of the embodiments of the present disclosure are clearly and completely described below in conjunction with the drawings of the embodiments of the present disclosure. Apparently, the embodiments described in the following are only some embodiments of the present disclosure, rather than all the embodiments. Any other embodiments obtained by those skilled in the art based on the embodiments in the present disclosure without any creative effort fall within the protection scope of the present disclosure.

[0036] Terms of "the first", "the second" and the like in the specification, the claims and the drawings of the present disclosure are used to distinguish an object from other objects rather than dcscribc a specific order. In addition, terms of "include", "comprisc" or any othcr variants thereof are intended to be non-exclusive. For example, a process, method, system, product or apparatus including a series of steps or units is not limited to including the Fisted steps or units but may include a step or unit not listed.

[0037] In order to describe the embodiments of the present disclosure, terms in the present disclosure are explained below.

[0038] In dual learning, for a given original task model, a dual task model corresponding to the original task model may provide a feedback to the original task model. Similarly, for a given dual task model, an original task model may provide a feedback to the dual task model. In this way, the original task model and the dual task model may provide feedback to each other. learn from each other, and improve together.

[0039] A Generative Adversarial Network (GAN) is a deep learning model including a generator and a discriminator. The generator and the discriminator are optimized by confrontation continuously each other to generate a better output. in an original GAN theory, it is not required that both of the generator and the discriminator are neural networks, as long as they can fit generating functions and discriminant functions. However, in practical applications, both of the generator and the discriminator are deep neural networks. The GAN is required to be trained with a proper training method, otherwise the output may be not ideal due to freedom of the neural network models. A Dual-Generative Adversarial Network (DualGAN) is a kind of GAN.

[0040] A leaf area index (LAT) is defined to be a half of a total surface area of leaves in a unit surface area of land. The LAT is used to quantitatively characterize a density degree of crop leaves, to provide an important index indicating growth and development features and physical conditions of a plant community. Quantitatively acquiring spatio-temporal continuous vegetation LAI in a region is of great significance in monitoring crop growth and estimating crop yield.

[0041] A method for performing inversion of a crop leaf area index is provided according to the present disclosure. In case of performing quantitative inversion for LAT using a neural network, it is generally required a large number of training samples due to complexity of the neural network model. However, in performing quantitative inversion for LAIs in a practical area, the number of samples collected in a real time manner is too small, which is difficult to meet the above requirement. Therefore, in the present disclosure, the number of training samples required by a neural network is provided by performing data augmentation on the hyperspectral data and leaf area indexes that are collected in a real time manner, thereby high precision of the leaf area index based on small samples collected in a real time manner can be achieved by estimating the leaf area index based on the hyperspectral data using the neural network.

[0042] Referring to Figure 1, the method may include the following steps S101 to S103. [0043] In step S101, remote sensing image data is acquired.

[0044] in step S102, pre-processing is performed on the remote sensing image data to obtain hyperspectral data.

[0045] The remote sensing image data may be hyperspectral remote sensing image EnNIAP corrected by atmosphere. The pre-processing may be a normalized process performed on a format or content of the data.

[0046] In step S103, the hyperspearal data is inputted into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model.

[0047] The target reference inversion model is obtained by optimizing a reference inversion model based on target training samples. The target training samples are obtained by performing data augmentation on the origin& training samples using the DualGAN.

[0048] A method for generating a target reference inversion model is provided according to an embodiment of the present disclosure. The method includes steps S201 to S204.

[0049] In step S201, remote sensing image data and a leaf area index product are acquired.

[0050] in step S202, pre-processing is performed on each of the remote sensing image data and the leaf area index product to obtain hyperspectral data and corresponding leaf area indexes. Data is randomly selected from the pre-processed data to form origin& training samples. The original training samples include a hyperspectral data set and a leaf area index data set.

[0051] in step S203, data augmentation is performed on the original training samples using Dua1GAN. The augmented data and the original training samples are mixed to obtain a target training samples.

[0052] In step S204, a reference inversion model provided in advance is trained using the target training samples to obtain the target reference inversion model.

[0053] Performing data augmentation on the original training samples using DualGAN and mixing the augmented data and the original training samples to obtain the target training samples includes: [0054] determining a DualGAN, wherein DualGAN comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator is configured to generate a leaf area index based on the hyperspectral data set, the second generator is configured to generate hyperspectral data based on the leaf area index data set, the first discriminator is configured to determine the probability that the generated leaf area index is from a true leaf area index set, and the second discriminator is configured to determine the probability that the generated hyperspectral data is from a true hyperspectral data set: [0055] performing optimization in an iterative manner using the first generator, the first discriminator, the second generator, and the second discriminator, to cause the generated leaf area index and the generated hyperspectral data to meet a predetermined condition, so as to obtain augmented data, wherein the augmented data comprises the generated leaf area index and the generated hyperspectral data and [0056] mixing the augmented data and the original training samples to obtain the target training samples.

[0057] The original training samples are processed using DualGAN to obtain the augmented data. This step includes: [0058] setting thresholds, wherein the thresholds comprise a threshold of a relative error of a leaf area index and a threshold of a sum of relative square errors for all bands of hyper-spectrum; [0059] acquiring generated data within a range corresponding to the thresholds and determining a data set of the relative error of the leaf area index and a data set of the relative root mean square error of the hyper-spectrum based on the generated data; [0060] transforming the data set of the relative error of the leaf area index and the data set of the relative root mean square error of the hyper-spectrum to data sets following a standard normal distribution; and [0061] selecting the augmented data from the generated data based on the data sets following the standard normal distribution.

[0062] The first generator is a residual network including several basic residual blocks, in which the main path of the residual block is a stack of two convolution layers, wherein the one-dimensional convolution kernel size is 3 and stride is 1, which activated by the ReLU activation function. The second generator is configured to implement a mapping from low-dimensional data to high-dimensional data. A first layer of the second generator is a fully connected layer including 16 neurons. The second generator superposes up-sampling data of the second generator on the features extracted by the first generator through convolution.

[0063] In the present disclosure, data augmentation is performed on the hyperspectral data and the LAIs using the DualGAN, and thus a requirement on the number of training samples in estimating LAI using the neural network can be met. Especially, in case that volumes of the hyperspectral data and the LAIs collected in a real time manner are small, it is important to perform data augmentation on the data collected in a real time manner through the DualGAN.

The DualGAN includes two pairs of generators and discriminators. One of the two pairs of generators and discriminators is configured to generate hyperspectral data based on a LAI and another of the two pairs of generators and discriminators is configured to generate a LAI based on hyperspectral data. After the structure of the DualGAN network is determined, the hyperspectral data and LAIs that are collected in a real time manner are inputted into the DualGAN network. The generators and discriminators are optimized through iterations to -10 -cause the distribution of generated data to close to the distribution of training samples, thereby augmenting remote sensing data. An ultimate object is to effectively improve accuracy of the LAI inversion of the neural network through data augmentation of the hyperspectral data and the Lids, that is, to realize an LAT inversion with high accuracy by a small number of samples. Therefore, it is a key to build the DualGAN model. In addition, a maimer in which the augmented data is selected is also important.

100641 In an embodiment of the present disclosure, the first generator of the DualGAN network of dual learning is to generate a LA!. The first generator is denoted by GA. The first discriminator is denoted by DA. The second generator generating hyperspectral data is denoted by GB. The second discriminator is denoted by Du.

[0065] GA is a residual network including 7 basic residual blocks, as shown in Figure 2. A main path of the basic residual block is a stack of two convolution layers. Each of the two convolution layers has the one-dimensional convolution in which kernel size is 3 and stride is 1, setting padding=same, activated by ReLE activation function. A shortcut of the basic residual block in two cases is described below. In one case that the number of channels of the main path is the same as that of the shortcut of the residual block, the value on the shortcut is equal to an output of the last residual block (in this case, the residual block is as shown in Figure 2(a)). In another case that the number of channels of the main path is different from that of the shortcut, a convolution is set in the shortcut to change the number of channels of the shortcut (in this case, the residual block is as shown in Figure 2(b)). An output of the main path and an output of the shortcut are added to obtain features learned by the residual block. The sum of the outputs from the main path and the shortcut serves as a final output of the residual block after through a ReLU activation function. Then the output of the residual block serves as an input to a max pooling layer to down sampling A max pooling size is set as 2 and a stride is set as 2. In max pooling layers respectively behind a third residual block and a fourth residual block, it is set that padding=1. In other max pooling layers, it is the default setting that is padding=0. After the hyperspectral data is outputted through all residual blocks in GA, the dimension of the hyperspectral data is reduced to 4. Then the outputted hyperspectral data is inputted into a convolution layer with a one-dimensional convolution kernel size of 4 to cause the dimension of the outputted hyperspectral data to be consistent with the dimension of the LA!. Finally, an average of data in all channels is calculated as a final outputted LAI. A structure of the generator GA generating the LAI is as shown in Figure 2(c).

[0066] DA is a four-laver fully connected network. Each of the four fully connected lavers is activated by a LeakyReLU function. A constant A of the LeakyReLU function is equal to 0.2. The number of neurons in the four fully connected layers is set to be 512, 256, 128 and 1 respectively. An input to the DA network is hyperspectral data An output of the DA network is die probability that generated hyperspectral data is from true hyperspectral data.

[0067] GB is a network for generating hyperspectral data based on a LAI. The input of GB is a LAI and the output of GB is 244 bands of hyper-spectrum. GB can realize a mapping from low-dimensional data to high-dimensional data. The first laver of GB network is a fully connected layer including 16 neurons, which indicates that the LAI is mapped to a 16 vector components. Then the hyperspectral data is obtained at the output end of GB by repeating up-sampling and convolution. The GB network imitates a method for integrating decoding information and encoding information in U-Net. The GB network superposes up-sampling features on the features extracted by the GA network by convolution, when the number of channels of the up-sampling features is the same as that of the features extracted by the GA network. In case that the dimension of up-sampling features is different from that of the features extracted by the GA network, center clipping may be performed on the features with high dimension. In practice, the up-sampling features have one dimension more than die features extracted by the GA network. Therefore, the clipping is performed by removing the last dimension of the up-sampling features. GB may obtain accurate hyperspectral data by repeating up-sampling, information superposition and convolution, hi up-sampling, one-dimensional convolution kernel size is 2 and stride is 2. In convolution, one-dimensional convolution kernel size is 3, stride is 1, and padding sets same. For the number of channels of each laver of the network, one may refer to GB, as shown in Figure 3.

[0068] DB is similar to DA and is a three-layer fully connected network. The number of neurons of the three fully connected layers is 16, 16 and 1 respectively. Each of the three fully connected layers is activated by LeakyReLU function. In the LeakyReLU function, A = 0.2.

[0069] Therefore, the combination of the generators GA and GB and the discriminators DA and DB can form a DualGAN network, as shown in Figure 4. in an embodiment of the present 30 disclosure, DualGAN optimizes by RMSProp, a loss function. In the RMSProp for optimizing DA and DB, the hyper-parameter)62 is set to be 0.99 and die weight decay is set to be 0.9. In -12 -the RMSProp for optimizing GA and GB, the hyper-parameter fl2 is set to be 0.95 and the weight decay is set to be 0.9. Initial weights are set to follow a Gaussian distribution with a mean value of 0 and a standard deviation of J7T (n is the number of network weights). All bias are set to be 0. Learning rate n is set to be 2* [O. In order to obtain better generators, it trains the discriminators one steps, then fives steps on generators.

[0070] In the embodiment of the present disclosure, a manner in which the augmented data of the hyperspectral data and the LAI are selected is described below.

[0071] in case of performing data augmentation on the hyperspectral data and the LAI using the DualGAN, more training samples than required are generally obtained. A key of effectively improving LAI inversion accuracy is to select a part of data from augmented data as training samples. Therefore, data selection is also very important. In an embodiment, firstly, data filtering threshold is set. Due to the generation of paired data, a threshold of a relative error of the LAI and a threshold of a sum of relative square errors of the hyper-spectrum are set herein. The two thresholds are set to be 0.03 and 0.25 respectively. Secondly, it is required to simultaneously consider errors of the TAI and the hyperspectral data In the generated data within the thresholds, a data set of the relative error of LAI is denoted by S, and a data set of a relative root-mean-square error of hyperspectral data is denoted by T (the relative root-mean-square error is calculated based on the sum of the relative square errors of the hyper-spectrum herein). S and T are transformed into data sets following standard normal distribution. The transformed data sets are expressed as: and NT v-E.1 * [0072] Then Ns and Ni are combined to obtain a data set E: /3e NT * [0073] Data in the data set E ranked in an ascending order is denoted by Er. .12:' is a set of augmented data finally selected. in this way, the augmented training samples of hyper-spectrum and LAI can be determined.

: (X -X 11" E [0074] in -13 -x is an element in the data set S of the relative error of the LAT. K. is the mean value of data in the set S. as is the standard deviation of the set S. a is an element in a data set following the standard normal distribution and transformed from the data set S. The data set following the standard normal distribution and transformed from the data set S is denoted by N5.

[0075] Similarly, in v -t :{fi I) y is an element in the data set T of the relative root-mean-square error of the hyper-spectrum. T is the mean value of data in the set T. a, is the standard deviation of the set T. @ is an element in a data set following the standard normal distribution and transformed from the data set T. The data set following the standard normal distribution and transformed from the data set T is denoted by N. [0076] A final object is to effectively improve the inversion accuracy of the neural network by the augmented training samples. Firstly LAI reference inversion model is determined fin the present disclosure. SSLLAT-Net and VGG16 are selected as reference inversion models).

Then the original training samples or the augmented training samples are inputted into the reference inversion models respectively to compare inversion accuracy of the test samples.

[0077] Taking a hyperspectral remote sensing image EnMAP as an example, leaf area index quantitative inversion numerical experiment is carried out with cereal, maize and rape seed as research objects. The EnMAP remote sensing image includes 1000 rows x1000 columns x244 spectrum bands. In case that the number of training samples for each of the cereal, maize, mid rape seed is equal to 30 and is augmented to 500. LAI inversion accuracy of the SSLLA1-Net is improved by 9.1%, 25.0% and 20.8% respectively, and LAI inversion accuracy of the VGG16 is improved by 64.1%, 11.1% and 34.8% respectively. In addition, stabilities of results of the LAI inversions are also improved. In case that the number of training samples for the cereal, maize, and rape seed is equal to 200 and is augmented to 1000, LAI inversion accuracy of the SSLLAI-Net is improved by 20.8%, 24.5% and 50.0% respectively, and LAI inversion accuracy of the VGG16 is improved by 20.0%, 11.8% and 6.3% respectively. in view of the above, with the method for augmenting EnMAP data based on the DualGAN provided according to the present disclosure, a requirement on the number of EnMAP training samples required by the neural network can be met, thereby improving inversion accuracy for -14 -the LAI based on small samples. 122 of the reference inversion models obtained based on the technical solution of the present disclosure is shown in Table 1 to Table 3.

Table 1 le comparisons of SSLLAI-Net before and after data augmentation (mean value ± standard deviation)

DG-SSL

SSLLAI

LAI

DG-SSL

SSLLAI

LAI

DG-SSL

SSLLAI SSLLAI

LAI

DG-SSL LAI Types of crops 200

0.945 0.950 0.955 0.971 0.982 0.984 0.992 0.991 Grains +0.020 +0.013 +0.026 +0.007 +0.011 +0.004 +0.002 +0.004 0.988 0.991 0.993 0.994 0.994 0.997 0.996 0.997 Corns +0.008 +0.004 +0.003 +0.002 +0.002 +0.001 +0.003 +0.0008 0.952 0.962 0.960 0.967 0.979 0.986 0.980 0.988 Rapes +0.013 +0.015 +0.021 +0.009 +0.009 +0.003 +0.012 +0.002 Table 2 122 comparisons of VGG16 before and after data augmentation (mean value standard deviation) 50 100 200 Types

DG-

of crops VGG16 VGG16 VGG16 DG- VGG16 DG- VGG16 DG-VGG16 VGG16 VGG16 0.744 0.908 0.910 0.935 0.963 0.962 0.969 0.976 Grains +0.085 +0.031 +0.040 +0.028 +0.014 +0.013 +0.024 +0.003 0.991 0.992 0.995 0.996 0.996 0.998 0.999 0.999 Corns +0.003 +0.002 +0.001 +0.001 +0.002 +0.002 +0.0003 +0.0001 0.931 0.955 0.936 0.963 0.968 0.978 0.978 0.983 Rapes +0.017 +0.010 +0.020 +0.005 +0.013 +0.007 +0.006 +0.003 Table 3 R2 comparisons of reference inversion model before and after data augmentation (mean value ± standard deviation) Ty pes of 300 200 300 crops SSLLA DG-SS SSLLA DG-SS VGG16 DG- DG-I LLAI I LLAI V0016 VGC16 VGG16 0.952 0.962 0.972 0.980 0.935 0.948 0.962 0.976 Grains +0.017 +0.008 +0.015 k0.003 +0.020 +0.016 +0.006 +0.005 0.947 0.960 0.974 0.970 0.966 0.970 0.982 0.986 Corns +0.034 +0.015 +0.008 +0.011 +0.017 +0.012 +0.007 +0.006 0.890 0.945 0.937 0.946 0.889 0.896 0.922 0.951 Rapes +0.030 +0.017 +0.013 k0.006 +0.037 +0.022 +0.017 +0.008 [0078] A method for performing inversion of a crop leaf area index is provided according to the present disclosure. The method includes: acquiring remote sensing image data; performing pre-processing on the remote sensing image data to obtain hyperspectral data; and inputting the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model, where the target reference inversion model is obtained by optimizing a reference inversion model using target training samples, the target training samples are obtained by performing data augmentation on original training samples using the DualGAN In this way; a requirement on the number of training samples required by a neural network can be met by performing data augmentation on the hyperspectral data and leaf area indexes that are collected in a real time manner, thereby high precision of the leaf area index based on small samples collected in a real time manner can be achieved by estimating the leaf area index based on the hyperspectral data using the neural network.

[0079] A device for performing inversion of a crop leaf area index is further provided according to an embodiment of the present disclosure. Referring to Figure 5, the device includes an acquiring unit, a processing unit and an input unit.

[0080] The acquiring unit is configured to acquire remote sensing image data [0081] The processing unit is configured to perform pre-processing on the remote sensing image data to obtain hyperspectral data.

[0082] The input unit is configured to input the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model. The target reference inversion model is obtained by optimizing a reference inversion model based on target training samples. The target training samples are obtained by performing data augmentation on the original training samples using the DualGAN.

-16 - [0083] Based on the above embodiments, the device further includes a data acquiring unit, a pre-processing unit, a data augmenting unit and a training unit.

[0084] The data acquiring unit is configured to acquire remote sensing image data and a leaf area index product.

[0085] The pre-processing unit is configured to perform pre-processing on the remote sensing image data and the leaf area index product to obtain hyperspectral data and corresponding leaf area indexes, and randomly selecting data from the pre-processed data to form the original training samples, wherein the original training samples comprise a hyperspectral data set and a leaf area index data set.

[0086] The data augmenting unit is configured to perform data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples.

[0087] The training unit is configured to train the reference inversion model provided in advance using the target training samples to obtain the target reference inversion model.

[0088] In an embodiment, the data augmenting unit includes a network determining subunit, an optimizing subunit and a mixing subunit.

[0089] The network determining subunit is configured to determine the DualGAN. The DualGAN comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator is configured to generate a leaf area index based on the hyperspectral data set, the second generator is configured to generate hyperspectral data based on the leaf area index data set, the first discriminator is configured to determine the probability that the generated leaf area index is from a true leaf area index data set, and the second discriminator is configured to determine the probability that the generated hyperspectral data is from a true hyperspectral data set.

[0090] The optimizing subunit is configured to perform optimization in an iterative manner using the first generator, the first discriminator, the second generator, and the second discriminator, to cause the generated leaf area index and the generated hyperspectral data to meet a predetermined condition, so as to obtain augmented data, wherein the augmented data comprises the generated leaf area index and the generated hyperspectral data [0091] The mixing subunit is configured to mix the augmented data and the original -17 -training samples to obtain the target training samples.

[0092] In an embodiment, the device further includes an augmentation processing subunit.

[0093] The augmentation processing subunit is configured to process the original training samples using the DualGAN to obtain the augmented data.

[0094] Specifically, the augmentation processing subunit is configured to set thresholds, where the thresholds comprise a threshold of a relative error of a leaf area index and a threshold of a sum of relative square errors for all bands of hyper-spectrum; acquire generated data within a range corresponding to the threshold and determine a data set of the relative error of the leaf area index and a data set of the relative root mean square error of the hyper-spectrum based on the generated data: transform the data set of the relative error of the leaf area index and the data set of the relative root mean square error of the hyper-spectrum to data sets following a standard normal distribution: and select the augmented data from the generated data based on the data sets following the standard normal distribution.

[0095] A device for performing inversion of a crop leaf area index is provided according to die present disclosure. The device includes an acquiring unit, a processing unit and an input unit. The acquiring unit is configured to acquire remote sensing image data. The processing unit is configured to perform pre-processing on the remote sensing image data to obtain hyperspectral data. The input unit is configured to input the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model. The target reference inversion model is obtained by optimizing a reference inversion model based on target training samples. The target training samples are obtained by performing data augmentation on original training samples using a DualGAN. A requirement on die number of training samples required by a neural network can be met by performing data augmentation on the hyperspectral data and leaf area index that are collected in a real time manner, thereby high precision of the leaf area index based on small samples collected in a real time manner is achieved by estimating the leaf area index based on the hyperspectral data by using the neural network.

[0096] The embodiments in this specification are described in a progressive way, each of which emphasizes the differences from others. For the same or similar parts among the embodiments, one may refer to description of other embodiments. Since the device disclosed in the embodiments corresponds to the method disclosed in the embodiments, the description -18 -of the device is relatively simple. For relevant matters, one may refer to the description of the method embodiments.

[0097] With the description of the embodiments disclosed above, those skilled in the art may implement or use technical solutions of the present disclosure. Numerous modifications to the embodiments are apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure may not be limited to the embodiments described herein, but should comply with the widest scope consistent with the principles and novel features disclosed herein.

-19 -

Claims

CLAIMS1. A method for performing inversion of a crop leaf area index comprising: acquiring remote sensing image data; performing pre-processing on the remote sensing image data to obtain hyperspectral data; and inputting the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model wherein the target reference inversion model is obtained by optimizing a reference inversion model using target training samples, the target training samples are obtained by performing data augmentation on original training samples using a Dual-Generative Adversarial Network (DualGAN) 2. The method according to claim I, further comprising: acquiring remote sensing image data and a leaf area index product; performing pre-processing on the remote sensing image data and the leaf area index product to obtain hyperspectral data and corresponding leaf area indexes, and randomly selecting data from the pre-processed data to form the original training samples, wherein the original training samples comprise a hyperspectral data set and a leaf area index data set; performing data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples; and training the reference inversion model provided in advance using the target training samples to obtain the target reference inversion model.3. The method according to claim 2, wherein performing data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples comprises: determining the DualGAN, wherein the DualGAN comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator is configured to generate a leaf area index based on the hyperspectral data set, the second generator is configured to generate hyperspectral data based on the leaf area index data set, the first -20 -discriminator is configured to determine a probability that the generated leaf area index is from a true leaf area index data set, and the second discriminator is configured to determine a probability that the generated hyperspectral data is from a true hyperspectral data set; perforrning optimization in an iterative manner using the first generator, the first discriminator, the second generator, and the second discriminator, to cause the generated leaf area index and the generated hyperspectral data to meet a predetermined condition, so as to obtain augmented data, wherein the augmented data comprises the generated leaf arca index and the generated hyperspectral data; and mixing the augmented data and the original training samples to obtain the target training samples.4. The method according to claim 3, further comprising: processing the original training samples using the DualGAN to obtain the augmented data wherein processing the original training samples using the DualGAN to obtain the augmented data comprises: setting thresholds, wherein the thresholds comprise a threshold of a relative error of a leaf area index and a threshold of a sum of relative square errors for all bands of hyper-spectrum; acquiring generated data within a range corresponding to the thresholds and determining a data set of a relative error of the leaf area index and a data set of a relative root mean square error of the hyper-spectrum based on the generated data; transforming the data set of the relative error of the leaf area index and the data set of the relative root mean square error of the hyper-spectrum to data sets following a standard normal distribution; and selecting the augmented data from the generated data based on the data sets following the standard norrnal distribution.5. The method according to claim 3, wherein the first generator is a residual network comprising several basic residual blocks; a main path of the basic residual block is a stack of two convolution layers, wherein a -21-one-dimensional convolution kernel has a size of 3 and a stride of 1, and is activated by a ReLU activation function.6. The method according to claim 5, wherein the second generator is configured to implement a mapping from low-dimensional data to high-dimension& data; a first layer of the second generator is a fully connected layer comprising 16 neurons; the second generator superposes up-sampling data of the second generator on a feature extracted by the first generator through convolution.7. A device for performing inversion of a crop leaf area index, comprising: an acquiring unit configured to acquire remote sensing image data; a processing unit configured to perform pre-processing on the remote sensing image data to obtain hyperspectral data; and an input unit configured to input the hyperspectral data into a pre-generated target reference inversion model to obtain a leaf area index outputted by the target reference inversion model, wherein the target reference inversion model is obtained by optimizing a reference inversion model based on target training samples, the target training samples are obtained by performing data augmentation on original training samples using a Dual-Generative Adversarial Network (DualGAN).8. The device according to claim 7, further comprising: a data acquiring unit configured to acquire remote sensing image data and a leaf area index product; a pre-processing unit configured to perfonn pre-processing on the remote sensing image data and the leaf area index product to obtain hyperspectral data and corresponding leaf area indexes, and randomly selecting data from the pre-processed data to form the original training samples, wherein the original training samples comprise a hyperspectral data set and a leaf area index data set; a data augmenting unit configured to perform data augmentation on the original training samples using the DualGAN and mixing the augmented data and the original training samples to obtain the target training samples; and -22 -a training unit configured to train the reference inversion model provided in advance using the target training samples to obtain the target reference inversion model.9. The device according to claim 8, wherein the data augmentation unit comprises: a network determining subunit configured to determine the DualGAN, wherein the DualGAN comprises a first generator, a second generator, a first discriminator and a second discriminator, the first generator is configured to generate a leaf area index based on the hyperspectral data set, the second generator is configured to generate hyperspectral data based on the leaf area index data set, the first discriminator is configured to determine a probability that the generated leaf area index is from a true leaf area index data set, and the second discriminator is configured to determine a probability that the generated hyperspectral data is from a true hyperspectral data set; an optimizing subunit configured to perform optimization in an iterative manner using the first generator, the first discriminator, the second generator, and the second discriminator, to cause the generated leaf area index and the generated hyperspectral data to meet a predetermined condition, so as to obtain augmented data, wherein the augmented data comprises the generated leaf area index and the generated hyperspectral data; a mixing subunit configured to mix the augmented data and the original training samples to obtain the target training samples.10. The device according to claim 9, further comprising an augmentation processing subunit configured to process the origin& training samples using the DualGAN to obtain the augmented data, wherein the augmentation processing subunit, for processing the original training samples using the DualGAN to obtain the augmented data, is configured to: set thresholds, wherein the thresholds comprise a threshold of a relative error of a leaf area index and a threshold of a sum of relative square errors for all bands of hyper-spectrum: acquire generated data within a range corresponding to the thresholds and determine a data set of a relative error of the leaf area index and a data set of a relative root mean square error of the hyper-spectrum based on the generated data; transform the data set of the relative error of the leaf area index and the data set of the relative root mean square error of the hyper-spectrum to data sets following a standard normal -23 -distribution; and select the augmented data from the generated data based on the data sets following the standard notmal distribution.-24 -