CN116629492A - Integrated learning optimization evaluation method for soil quality improvement effect - Google Patents

Abstract

The invention relates to the technical field of evaluation methods, in particular to an integrated learning optimization evaluation method for soil quality improvement effect, which comprises an example of a soil quality prediction data set, a mark of the soil quality prediction data set, and construction and verification of a machine learning single model and an integrated model based on the soil quality prediction data set; the construction and verification of the machine learning single model and the integrated model based on the soil quality prediction data set comprises model performance evaluation based on a training set and a verification set, model performance evaluation based on a test set, yield verification of the integrated model and response of a soil quality index to input of different organic materials; the method solves the problems of insufficient biological index information, insufficient comprehensive soil attribute data under large spatial scale, insufficient accuracy of a Minimum Dataset (MDS) verification link and the like in the traditional soil quality quantitative evaluation method.

Description

An integrated learning optimization evaluation method for soil quality improvement effect

technical field

The invention relates to the technical field of evaluation methods, in particular to an integrated learning optimization evaluation method for soil quality improvement effects.

Background technique

As a means to evaluate the impact of human activities such as management measures and land use changes on soil, soil quality assessment is helpful to grasp the status quo and changing dynamics of soil quality in a timely manner, and then realize the sustainable management of land resources. The key step in soil quality evaluation is to establish a set of sensitive and representative evaluation index system. Soil quality is mainly manifested in its internal and external functions, which can be expressed by a series of physical, chemical and biological indicators. Selection principles of soil quality indicators: (1) included in the ecological process and related to the model process; (2) integrated soil physical, chemical and biological properties; (3) accepted by most users and applicable to field conditions; (4) It is easy to measure and has good reproducibility; (5) It is sensitive to changes in climate and management conditions, so that changes in soil properties can be monitored; (6) It should be part of the existing database as much as possible. There are many indicators to choose from in soil quality evaluation. Although selecting more comprehensive indicators in the full data set can reflect the soil quality more truly, it will significantly increase the cost of data acquisition.

Traditional soil quality quantitative evaluation methods have problems such as insufficient biological indicator information, lack of comprehensive soil attribute data at large spatial scales, and insufficient accuracy of the minimum data set (MDS) verification process.

Therefore, in view of the above problems, an integrated learning optimization evaluation method of soil quality improvement effect is proposed.

Contents of the invention

In order to make up for the deficiencies in the prior art, the purpose of the present invention is to propose an integrated learning optimization evaluation method for soil quality improvement effects, including examples of soil quality prediction data sets, marking of soil quality prediction data sets, and based on soil quality prediction data sets The construction and verification of machine learning single model and integrated model;

The construction and verification of the machine learning single model and integrated model based on the soil quality prediction data set include model performance evaluation based on training set and verification set, model performance evaluation based on test set, yield verification of integrated model, soil quality index Response to different organic material inputs.

Preferably, the example of the soil quality prediction data set is established using the smallest data set, and the evaluation indicators finally selected into MDS include six indicators including bulk density, organic matter, available phosphorus, available potassium, microbial biomass carbon and microbial biomass nitrogen.

Preferably, the soil quality prediction data set is marked using the soil quality index and the minimum data set verification, and after scoring and weighting each evaluation index, the soil quality index is calculated based on TDS and MDS respectively.

Preferably, the model performance evaluation based on the training set and the validation set uses Welch method to correct the variance analysis test to study the difference between the soil quality index method and the three machine learning models for the soil quality index.

Preferably, the model performance evaluation based on the training set and verification set uses a 10-fold cross-validation method to further evaluate the performance of each model: the 10-fold cross-validation method randomly divides the training set into 10 mutually exclusive subsets of similar size , and then each time use the union of 9 subsets as the training set, and the remaining subset as the verification set, so as to train and evaluate the model 10 times.

Preferably, the model performance evaluation based on the test set, through further evaluation of the test set model performance, combined with the analysis results of the training set and verification set, RFR-MDS has obtained the highest prediction performance on the training set, and in High accuracy is maintained on the test set, while LightGBMR-MDS achieves similar prediction performance to RFR-MDS on the test set, and has the highest potential to avoid the risk of overfitting.

Preferably, in the yield verification of the integrated model, since there is a strong correlation between soil quality and crop yield, crop yield is usually used to verify the accuracy of soil quality index calculation.

Preferably, the response of the soil quality index to different organic material inputs, for rice, corn and wheat, two integrated models are used to evaluate the impact of different organic material inputs on the overall soil quality index of the three categories of crops; based on the two integrated models Prediction results showed that different organic material types had significant effects on the soil quality index (P<0.001).

The benefits of the present invention are:

1. The present invention utilizes the integrated model of machine learning, in conjunction with the MDS evaluation index system based on soil classification, predicts the TDS soil quality index. The verification results of crop yield showed that the SQI-TDS-classified of each crop under different soil types was significantly positively correlated with the yield (P<0.05), and the ^R2 value of 75.5% of the samples exceeded 0.5, indicating that the calculated Soil quality index is reasonable. The evaluation results of the machine learning model confirmed the high-precision performance and good application prospect of the integrated model in the prediction of soil quality index.

2. The verification results of the linear regression analysis of the training set + test set show that the R ² value of the machine learning integrated model (R ² values of RFR and LightGBMR are 0.976 and 0.974, P<0.001) compared with the MDS of the soil quality index method The R ² value of the verification result (R ² value is 0.771) has been greatly improved.

3. Based on the verification results of the verification set and the test set, LightGBMR is the optimal model choice in terms of model accuracy and improvement of overfitting problems. Therefore, LightGBMR is recommended as a soil quality evaluation that replaces the traditional MDS-based soil quality index method Model. In addition, the two integrated models of RFR and LightGBMR are not limited by the sample size of the soil type for the prediction of soil quality index, and also achieved a fairly high prediction accuracy in the case of relatively small samples.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained according to these drawings on the premise of not paying creative efforts.

Fig. 1 is the common factor variance ratio and the Norm value figure (in (a)-(c), soil physics, chemistry and The biological indicators are concentrated on different color blocks, and the principal component loading value is the correlation coefficient between the indicators and the principal components, which reflects the importance of each indicator to the three principal components. (d) represents the common values obtained by factor analysis based on the full data set Factor variance ratio. The common factor variance ratio indicates the proportion of information that can be explained by the extracted three common factors. In (e), the left y-axis represents the cumulative variance contribution rate, and the right y-axis represents the characteristic Value, the green columns represent the three principal components extracted in the principal component analysis. In (f), the green, purple and orange columns represent the indicators of the Norm values in the three principal components in order from large to small, and the red dotted line is the dividing line of the 10% range of the largest index of the Norm value in each principal component);

Fig. 2 is the linear regression analysis result figure between SQI-TDS of the present invention and SQI-MDS (black dotted line represents 1:1 line);

Fig. 3 is the linear regression analysis result (black dotted line represents 1:1 line of SQI-TDS of the present invention and the prediction result based on MDS single model and integrated model. (a)-(c) is training set; (d)- (f) is a test set) the violin figure of the soil quality index obtained based on different methods of the present invention (the left side of the black dotted line is the soil quality index method, and the right side is a machine learning model. (a) is a training set; (b) is test set);

Fig. 4 is the violin figure of the soil quality index that obtains based on different methods of the present invention (the left side of the black dotted line is the soil quality index method, and the right side is the machine learning model. (a) is a training set; (b) is a test set);

Fig. 5 is a contrasting diagram of yield verification of the three categories of crops according to the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Embodiment one

1. Example of soil quality prediction data set - minimum data set establishment

In order to reduce the number of participating indicators and eliminate the information overlap generated by the interaction between various indicators, this paper conducts principal component analysis on 11 primary indicators and extracts PCs with eigenvalues greater than 1. The KMO (Kaiser-Meyer-Olkin) value exceeds 0.8 (0.834), and the corresponding P value of the Bartlett test is less than 0.05, indicating that the partial correlation between the indicators is strong, and it is suitable for principal component analysis. There are three PCs whose eigenvalues are greater than 1, and the cumulative variance contribution rate reaches 62.5%. First, the indicators are grouped based on the absolute values of the loads of each indicator on the three PCs. In PC1, since the absolute value of microbial biomass carbon, microbial biomass nitrogen, urease, total nitrogen, phosphatase, and sucrase was greater than or equal to 0.5, they were grouped together; PC2 grouped bulk density, available phosphorus, and pH as a group; PC3 includes available potassium and organic matter. For PC1, PC2 and PC3, it is necessary to further analyze the Norm value and correlation among the indicators. The highest Norm value in PC1 is microbial biomass carbon (1.71), and the indicator within 10% of it is only microbial biomass nitrogen. Although the correlation coefficient value between microbial biomass carbon and microbial biomass nitrogen was greater than 0.5 (P<0.001), they were selected into MDS at the same time in this paper to increase the diversity of biological indicators. The highest Norm value in PC2 is available phosphorus (1.18), and no index is within its 10% range. The highest Norm value in PC3 is available potassium (1.08), while organic matter is the only indicator within its 10% range. Since there is no correlation between available potassium and organic matter, they both go into MDS. In addition, in view of the fact that the test weight has a high selection frequency in the MDS, the test weight also enters the MDS. The highest Norm value in PC2 is available phosphorus (1.17), and the index within 10% (1.05) is only available potassium. Since the correlation coefficient value between available potassium and available phosphorus was less than 0.5 (P<0.001), both available phosphorus and available potassium entered the MDS. In summary, the evaluation indicators finally selected into MDS include six indicators including bulk density, organic matter, available phosphorus, available potassium, microbial biomass carbon and microbial biomass nitrogen.

2. Marking of soil quality prediction data set - soil quality index and minimum data set verification

The rationality verification of the minimum data set evaluation index system is an important part of soil quality evaluation, and the accuracy of the evaluation results needs to be verified to ensure the accuracy of the evaluation. After scoring and weighting each evaluation index, the soil quality index was calculated based on TDS and MDS, respectively. The SQI-TDS ranged from 0.138 to 0.992, with an average value of 0.543±0.171 and a coefficient of variation of 31.5%. The SQI-MDS ranged from 0.101 to 1.00, with an average value of 0.587±0.195 and a coefficient of variation of 33.2%. Compared with SQI-TDS, the range of SQI-MDS is larger, the average value is higher, and the fluctuation range is relatively larger. Linear regression analysis was performed on SQI-TDS and SQI-MDS. From the perspective of fitting effect, SQI-TDS and SQI-MDS showed a very significant positive correlation (P<0.001), and the ^R2 value was 0.737, which was consistent with most research results. close to (Guo et al., 2017: Li et al., 2019: Li et al., 2020). The RMSE value is 0.109, and the RPD value is 1.57, which are at a relatively low level. The 1:1 line can reflect the agreement between the two comparison objects. SQI-TDS and SQI-MDS can be more evenly distributed on both sides of the 1:1 line. The results show that: SQI-MDS can achieve the expected verification effect of traditional methods in soil quality evaluation on a large spatial scale. However, there is still a significant gap in the pursuit of high precision.

3. Construction and verification of machine learning single model and integrated model based on soil quality prediction data set

3.1 Model performance evaluation based on training set and validation set

According to the tuning results of the hyperparameters, we first evaluated the model performance on the training set. For the training set, from the violin plots of the predicted values of SQI-TDS, SQI-MDS and machine learning models, the range of DTR-MDS is smaller, while the range of SQI-MDS and the three models is closer to that of SQI- TDS. The soil quality indexes obtained by different methods were most concentrated between the median and the 75% quantile. The density distribution of RFR-MDS and LightGBMR-MDS is closer to that of SQI-TDS. The ANOVA test corrected by Welch method was used to study the difference between the soil quality index method and the three machine learning models for the soil quality index. Different methods showed significance for soil quality index (P<0.001), and the average value of SQI-MDS was significantly higher than that of SQI-TDS, DTR-MDS, RFR-MDS and LightGBMR-MDS (P<0.001). Linear regression analysis was performed on SQI-TDS and DTR-MDS, RFR-MDS and LightGBM-MDS respectively. From the perspective of fitting effect, DTR-MDS, RFR-MDS and LightGBM-MDS were all significantly positively correlated with SQI-TDS Relationship (P<0.001). The ^R2 values of the three models are all above 0.93, achieving quite high accuracy, and the highest among them is RFR-MDS ( ^R2 value is 0.983). For RMSE and RPD values, RFR-MDS. The RPD values of the three models are all over 2.5, which can meet the extremely high requirements for quantitative evaluation of soil quality.

It is worth noting that the high predictive performance of the three machine learning models in the training set may have the risk of overfitting, so this paper uses a 10-fold cross-validation method to further evaluate the performance of each model. The 10-fold cross-validation method randomly divides the training set into 10 mutually exclusive subsets of similar size, and then uses the union of 9 subsets as the training set each time, and the remaining subset as the verification set, so as to test the model. 10 training and evaluation sessions. 10 evaluation results (RMSE values) of the three models. The analysis of variance test results corrected by the Welch method showed that the three models all showed significant RMSE values (P<0.001), and the RMSE value of DTR-MDS was the highest, which was significantly higher than that of RFR-MDS and LightGBM-MDS (P<0.001). 0.001). Similar to the results of the linear regression analysis, the two ensemble models exhibited superior predictive performance.

3.2 Model Performance Evaluation Based on Test Set

Compared with the training set, the evaluation results of the test set are closer to the real application scenarios, so the analysis of the test set is very important. From the violin plots of SQI-TDS, SQI-MDS and the predicted values of the three machine learning models, the data density distribution of RFR-MDS and LightGBMR-MDS is similar to that of SQI-TDS, but the peak value is higher than that of the training set. Except that SQI-MDS is most concentrated around the 75% quantile, SQI-TDS, DTR-MDS, RFR-MDS and LightGBMR-MDS are all most concentrated between the median and 75% quantile. Similar to the ANOVA test results corrected by the Welch method of the training set, the different methods showed significance for the soil quality index (P<0.05), and the average value of SQI-MDS was significantly higher than that of SQI-TDS, DTR-MDS, RFR- MDS and LightGBMR-MDS (P<0.05). Using linear regression analysis to verify, from the point of view of fitting effect, DTR-MDS, RFR-MDS and LightGBM-MDS and SQI-TDS all showed extremely significant positive correlation (P<0.001). For the R ² value, DTR-MDS has a significant drop compared to the training set, and there is an obvious overfitting problem (although the hyperparameters that improve overfitting have been adjusted to the greatest extent). However, the two integrated models of RFR-MDS and LightGBM-MDS have significantly better performance than the single model in improving the problem of overfitting, and the overfitting risk of LightGBM-MDS is the smallest (the R of RFR-MDS and LightGBM-MDS ² values are 0.905 and 0.903, respectively). For RMSE and RPD values, all three models perform worse on the test set than on the training set. LightGBMR-MDS has the lowest RMSE value (0.0549) and the highest RPD value (3.21). The RPD values of the two integrated models reached a high level, both exceeding 3, which is sufficient to meet the requirements of quantitative evaluation of soil quality. Through further evaluation of the model performance of the test set, combined with the analysis results of the training set and the validation set, RFR-MDS achieved the highest prediction performance on the training set and maintained a high accuracy on the test set, while LightGBMR- MDS achieves similar predictive performance to RFR-MDS on the test set and has the highest potential to avoid the risk of overfitting.

3.3 Yield verification of integrated model

Since there is a strong correlation between soil quality and crop yield, crop yield is often used to verify the accuracy of SQI calculations. In this paper, the sample points containing at least 5 sample sizes in the three categories of crops were extracted, and a total of 455 samples were extracted. Both RFR-MDS and LightGBMR-MDS had extremely significant positive correlations with rice, corn and wheat yields (P<0.001). ^R2 values ranged from 0.294–0.397, RFR-MDS had higher ^R2 values in rice and maize, while LightGBMR-MDS showed higher ^R2 values in wheat. In addition, this paper further verifies the yield of the three major crops. The results show that: for rice, both RFR-MDS and LightGBMR-MDS have the highest R ² value in the early rice season in the middle and lower reaches of the Yangtze River; for corn and wheat, RFR-MDS and LightGBMR-MDS both have the highest ^R2 values in the Huanghuaihai region. The R ² values of RFR-MDS in each area of maize were higher than that of LightGBMR-MDS, while the R ² values of RFR-MDS in each area of wheat were lower than that of LightGBMR-MDS. For the rice in the middle and lower reaches of the Yangtze River, the R ² values of RFR-MDS in the early rice season, late rice season, and double-cropping rice production were all higher than those of LightGBMR-MDS, while the R ² values in single-cropping rice and rice-wheat rotation (rice season) were lower than those of LightGBMR-MDS. Lower than LightGBMR-MDS.

3.4 Response of soil quality index to different organic material inputs

For rice, maize, and wheat, we used two ensemble models to assess the impact of different organic material inputs on the overall soil quality index of the three crop categories. Based on the prediction results of the two integrated models, different types of organic materials were significant to the soil quality index (P<0.001). The application of animal-derived organic materials, plant-derived organic materials, and combined application of animal-plant-derived organic materials significantly improved the soil quality index of rice, corn, and wheat planting patterns compared with no fertilization and application of inorganic fertilizers (P<0.01). Under the rice planting model, compared with no fertilization, the soil quality index of the application of animal-derived organic materials, plant-derived organic materials, and animal-plant-derived organic materials based on the RFR model increased by 84.4%, 61.9%, and 80.6%, respectively. The soil quality index of the three based on the LightGBMR model increased by 87.6%, 63.9% and 83.3% respectively; compared with the application of inorganic fertilizers, the application of animal-derived organic materials, plant-derived organic The soil quality index of the application increased by 37.9%, 21.0% and 35.0%, respectively, and the soil quality index based on the LightGBMR model increased by 39.7%, 22.1% and 36.5%, respectively. Under the corn planting model, compared with no fertilization, the soil quality index of the application of animal-derived organic materials, plant-derived organic materials, and animal-plant-derived organic materials based on the RFR model increased by 78.3%, 67.3% and 87.5%, respectively, The soil quality index of the three based on the LightGBMR model increased by 86.1%, 72.8% and 97.4% respectively; compared with the application of inorganic fertilizers, the application of animal-derived organic materials, plant-derived organic The soil quality index of the application increased by 44.0%, 35.1% and 51.5%, respectively, and the soil quality index based on the LightGBMR model increased by 49.1%, 38.4% and 58.1%, respectively. Under the wheat planting mode, compared with no fertilization, the soil quality index of the application of animal-derived organic materials, plant-derived organic materials, and animal-plant-derived organic materials based on the RFR model increased by 80.4%, 61.8%, and 71.2%, respectively. The soil quality index of the three based on the LightGBMR model increased by 83.5%, 64.5% and 72.3% respectively; compared with the application of inorganic fertilizers, the application of animal-derived organic materials, plant-derived organic The soil quality index of the application increased by 31.0%, 17.5% and 24.3%, respectively, and the soil quality index based on the LightGBMR model increased by 31.3%, 17.7% and 23.3%, respectively.

The basic principles, main features and advantages of the present invention have been shown and described above. Those skilled in the industry should understand that the present invention is not limited by the above-mentioned embodiments. What are described in the above-mentioned embodiments and the description only illustrate the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will also have Variations and improvements are possible, which fall within the scope of the claimed invention.

Claims (8)

1. An integrated learning optimization evaluation method of soil quality improvement effect, characterized in that: including the example of soil quality prediction data set, the label of soil quality prediction data set, machine learning single model and integrated model based on soil quality prediction data set construction and verification; The construction and verification of the machine learning single model and integrated model based on the soil quality prediction data set include model performance evaluation based on training set and verification set, model performance evaluation based on test set, yield verification of integrated model, soil quality index Response to different organic material inputs. 2. The integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, it is characterized in that: the example of described soil quality prediction data set adopts minimum data set to establish, and the evaluation index finally selected into MDS comprises bulk density, Organic matter, available phosphorus, available potassium, microbial biomass carbon and microbial biomass nitrogen have a total of 6 indicators. 3. The integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, characterized in that: the mark of the soil quality prediction data set adopts soil quality index and minimum data set verification, and each evaluation index After scoring and weighting, the soil quality index was calculated based on TDS and MDS, respectively. 4. the integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, it is characterized in that: described model performance assessment based on training set and verification set adopts the analysis of variance inspection research soil quality index of Welch method correction The difference between the method and three machine learning models for soil quality index. 5. The integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, characterized in that: the model performance evaluation based on the training set and verification set adopts 10-fold cross-validation method to further evaluate the performance of each model Performance: The 10-fold cross-validation method randomly divides the training set into 10 mutually exclusive subsets of similar size, and then uses the union of 9 subsets each time as the training set, and the remaining subset as the verification set. The model is trained and evaluated 10 times. 6. The integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, characterized in that: the model performance evaluation based on the test set, through further evaluation of the test set model performance, combined with the training set and validation set analysis results, RFR-MDS achieved the highest prediction performance on the training set and maintained high accuracy on the test set, while LightGBMR-MDS achieved similar predictions to RFR-MDS on the test set performance, and has the highest potential to avoid the risk of overfitting. 7. The integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, is characterized in that: the yield verification of described integrated model, because there is stronger correlation between soil quality and crop yield, therefore Crop yields are often used to verify the accuracy of SQI calculations. 8. The integrated learning optimization evaluation method of a kind of soil quality improvement effect according to claim 1, is characterized in that: described soil quality index is inputted to the response of different organic materials, adopts two integrated models for rice, corn and wheat The impact of different organic material inputs on the overall soil quality index of the three types of crops was evaluated; based on the prediction results of the two integrated models, different organic material types showed significant effects on the soil quality index (P<0.001).