CN115375707B - A method and system for accurately segmenting plant leaves under complex backgrounds - Google Patents

Abstract

The invention discloses a method and a system for accurately dividing plant leaves under a complex background, and relates to the technical field of image processing. The method comprises the following steps: acquiring a plant image under a complex background as a target blade image; inputting the target blade image into the blade segmentation model to obtain a predicted image segmentation mask of the target blade image; the predictive image segmentation mask is used for segmenting the target blade image; the blade segmentation model comprises a trained composite backbone network and a trained image segmentation network; the composite backbone network comprises an auxiliary backbone network and a guiding backbone network which are connected by multi-level residual jump; the auxiliary backbone network is constructed based on a convolutional neural network and a transducer network; the guided backbone network is constructed based on a convolutional neural network. The invention can realize plant leaf segmentation under complex background, improve segmentation precision and segmentation speed, and make segmentation result accurately reflect phenotype character of plant leaf.

Description

A method and system for accurately segmenting plant leaves under complex backgrounds

technical field

The invention relates to the technical field of image processing, in particular to a method and system for accurately segmenting plant leaves under complex backgrounds.

Background technique

Plants are subject to biotic and abiotic stresses during their growth. In order to obtain the type of stress, phenotypic traits related to plant leaves must be continuously monitored. Plant in-vivo image detection is a fast monitoring method, but it has become a technical difficulty to accurately segment plant leaves from complex image backgrounds.

Traditional image segmentation algorithms are highly dependent on expert experience and are not universally applicable, and the speed and accuracy of plant leaf segmentation in complex backgrounds are not high, and the segmentation results cannot accurately reflect the phenotypic traits of plant leaves.

With the advancement of algorithm theory and hardware computing power, deep learning has become a better image segmentation method due to its powerful nonlinear learning ability and robust generalization ability. In particular, the deep stacking structure and better feature representation of convolutional neural networks have gradually made it a mainstream architecture in the field of plant phenotypes. Although, convolutional neural networks have been widely used in the field of plant phenotyping. But it also has some obvious disadvantages. For example, it cannot fully learn low-level features of images. Moreover, the absence of global information suggests that it is often ineffective in some complex environments. In recent years, Transformer has become a hot spot in the field of computer vision. Because they have a good understanding of the global context and high accuracy performance on large dataset tasks, they were quickly recognized by plant researchers. But in some respects, Transformer still has flaws that cannot be ignored. For example, they often do not generalize as well as convolutional neural networks due to lack of correct inductive bias. In addition, Transformer cannot be quickly applied to agricultural tasks, because the larger model capacity brings more parameters and higher computing power requirements. Therefore, hybrid neural networks that combine all the advantages of convolutional neural networks and Transformers are gradually becoming a new research direction.

Contents of the invention

The purpose of the present invention is to provide a method and system for accurately segmenting plant leaves under complex backgrounds, so as to realize the segmentation of plant leaves under complex backgrounds, improve the accuracy and speed of leaf segmentation, and provide a data basis for accurately obtaining phenotypic parameters of plant leaves.

To achieve the above object, the present invention provides the following scheme:

A method for accurately segmenting plant leaves under complex backgrounds, the method for accurately segmenting plant leaves under complex backgrounds includes:

Obtaining a target leaf image; the target leaf image is a plant leaf image under a complex background;

The target leaf image is input into the leaf segmentation model to obtain a predicted image segmentation mask of the target leaf image; the predicted image segmentation mask of the target leaf image is used to segment the target leaf image;

The leaf segmentation model is obtained by training a hybrid neural segmentation network; the hybrid neural segmentation network includes a composite backbone network and an image segmentation network; the trained composite backbone network is used to extract features from the target leaf image to obtain a multi-layer fusion feature map of the target leaf image; the trained image segmentation network is used to determine the predicted image segmentation mask of the target leaf image according to the multi-layer fusion feature map of the target leaf image;

The composite backbone network includes an auxiliary backbone network with multi-level residual skip connections and a guided backbone network; the auxiliary backbone network is constructed based on a convolutional neural network and a Transformer network; the guided backbone network is constructed based on a convolutional neural network.

Optionally, the method for determining the blade segmentation model includes:

Obtain a sample data set; the sample data set includes a plurality of sample leaf images and corresponding real image segmentation masks;

Input each sample leaf image into the composite backbone network for feature extraction, and obtain the multi-layer auxiliary feature map and multi-layer fusion feature map of each sample leaf image;

Input the multi-layer auxiliary feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the auxiliary image segmentation mask of each sample leaf image;

Input the multi-layer fusion feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the prediction image segmentation mask of each sample leaf image;

A comprehensive cross-entropy loss is determined according to the real image segmentation mask, the auxiliary image segmentation mask, and the predicted image segmentation mask; the comprehensive cross-entropy loss includes: the cross-entropy loss of the composite backbone network and the cross-entropy loss of the auxiliary backbone network;

Aiming at minimizing the comprehensive cross-entropy loss, the hybrid neural segmentation network is trained to obtain a leaf segmentation model; the leaf segmentation model includes a trained composite backbone network and a trained image segmentation network.

Optionally, said inputting each sample leaf image into the composite backbone network for feature extraction to obtain a multi-layer auxiliary feature map and a multi-layer fusion feature map of each sample leaf image, specifically including:

For any sample leaf image, the sample leaf image is input into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map; the multi-layer auxiliary feature map includes: a multi-layer local feature map and a multi-layer global feature map;

The multi-layer auxiliary feature map and the sample leaf image are input into the guided backbone network for feature extraction to obtain a multi-layer fusion feature map.

Optionally, the auxiliary backbone network includes: three convolutional neural network modules and two self-attention network modules, and there is a residual skip connection between two adjacent network modules;

The described sample leaf image is input into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map, which specifically includes:

The sample leaf image is input to the first convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the first layer of local feature maps;

The first layer of local feature maps are input to the second convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the second layer of local feature maps;

The second layer of local feature maps are input to the third convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the third layer of local feature maps;

Inputting the third layer of local feature maps to the first self-attention network module of the auxiliary backbone network for feature extraction to obtain the first layer of global feature maps;

The first layer of global feature map is input to the second self-attention network module of the auxiliary backbone network for feature extraction to obtain the second layer of global feature map.

Optionally, the guided backbone network includes: five convolutional neural network modules, and residual skip connections exist between two adjacent network modules;

Said inputting said multi-layer auxiliary feature map and said sample leaf image into said guiding backbone network for feature extraction to obtain a multi-layer fusion feature map, specifically comprising:

Inputting the first to third layers of local feature maps, the first to second layers of global feature maps and the sample leaf image to the first convolutional neural network module of the guided backbone network for feature extraction to obtain the first layer of fusion feature maps;

The second to third layers of local feature maps, the first to second layers of global feature maps and the first layer of fusion feature maps are input to the second convolutional neural network module of the guided backbone network for feature extraction to obtain the second layer of fusion feature maps;

The third layer of local feature maps, the first to second layer of global feature maps and the second layer of fusion feature maps are input to the third convolutional neural network module of the guided backbone network for feature extraction, and the third layer of fusion feature maps is obtained;

The first to the second layer of global feature maps and the third layer of fusion feature maps are input to the fourth convolutional neural network module of the guided backbone network for feature extraction to obtain the fourth layer of fusion feature maps;

The second layer global feature map and the fourth layer fusion feature map are input to the fifth convolutional neural network module of the guided backbone network for feature extraction to obtain the fifth layer fusion feature map.

Optionally, each layer of residual skip connection between the auxiliary backbone network and the guided backbone network further includes a channel batch normalization module; before inputting the multi-layer auxiliary feature map and the corresponding intermediate image to the corresponding convolutional neural network module in the guided backbone network, it also includes:

The multi-layer auxiliary feature map is spliced to obtain a corresponding spliced feature map; the spliced feature map is: a spliced feature map of the first to third layer local feature map and the first to second layer global feature map, a spliced feature map of the second to third layer local feature map and the first to second layer global feature map, a spliced feature map of the third layer local feature map and the first to second layer global feature map, a spliced feature map of the first to second layer global feature map or the second layer global feature map;

Superimpose the spliced feature map and the corresponding intermediate image element by element to obtain a corresponding superimposed feature map; the intermediate image is: the sample leaf image, the first layer fusion feature map, the second layer fusion feature map, the third layer fusion feature map or the fourth layer fusion feature map;

Using the corresponding channel batch normalization module to perform channel batch normalization processing on the overlay feature map, and use the processed overlay feature map as an input to the corresponding convolutional neural network module in the guided backbone network.

Optionally, the formula for calculating the comprehensive cross-entropy loss is:

L＝L _Lead +λ·L _Assist ;

Among them, L represents the comprehensive cross-entropy loss; L _Lead represents the cross-entropy loss of the composite backbone network, which is determined by the real image segmentation mask and the predicted image segmentation mask; L _Assist represents the cross-entropy loss of the auxiliary backbone network, which is determined by the real image segmentation mask and the auxiliary image segmentation mask; λ represents the weight of the auxiliary supervision.

Optionally, the calculation formula of the cross-entropy loss of the composite backbone network is:

The calculation formula of the cross-entropy loss of the auxiliary backbone network is:

Among them, L_leadIndicates the cross-entropy loss of the composite backbone network; L_AssistRepresents the cross-entropy loss of the auxiliary backbone network; m represents the total number of pixels of the input sample leaf image; y_iRepresents the real category value of the i-th pixel in the input sample leaf image, and the real category values of all pixels in the input sample leaf image constitute the real image segmentation mask of the input sample leaf image; p_iRepresents the predicted category value of the i-th pixel in the input sample leaf image, and the predicted category values of all pixels in the input sample leaf image constitute the predicted image segmentation mask of the input sample leaf image; q_iRepresents the auxiliary category value of the i-th pixel in the input sample leaf image, and the auxiliary category values of all pixels in the input sample leaf image constitute the auxiliary image segmentation mask of the input sample leaf image; i represents the sequence number of the pixel in the input sample leaf image.

Optionally, the method for obtaining the sample data set includes:

Obtaining a leaf data set; the leaf data set includes multiple plant leaf images under complex backgrounds;

Each plant leaf image in the leaf dataset is denoised, labeled and expanded to obtain multiple sample leaf images and corresponding real image segmentation masks.

A system for precise segmentation of plant leaves under complex backgrounds, which is realized by adopting the above method for precise segmentation of plant leaves under complex backgrounds. The system for precise segmentation of plant leaves under complex backgrounds includes:

A target leaf image acquisition module, configured to acquire a target leaf image; the target leaf image is a plant leaf image under a complex background;

An image segmentation mask prediction module, configured to input the target leaf image into the leaf segmentation model to obtain a predicted image segmentation mask of the target leaf image; the predicted image segmentation mask of the target leaf image is used to segment the target leaf image;

The leaf segmentation model is obtained by training a hybrid neural segmentation network; the hybrid neural segmentation network includes a composite backbone network and an image segmentation network; the trained composite backbone network is used to extract features from the target leaf image to obtain a multi-layer fusion feature map of the target leaf image; the trained image segmentation network is used to determine the predicted image segmentation mask of the target leaf image according to the multi-layer fusion feature map of the target leaf image;

The composite backbone network includes an auxiliary backbone network with multi-level residual skip connections and a guided backbone network; the auxiliary backbone network is constructed based on a convolutional neural network and a Transformer network; the guided backbone network is constructed based on a convolutional neural network.

According to the specific embodiments provided by the invention, the invention discloses the following technical effects:

The present invention provides a method and system for accurately segmenting plant leaves under complex backgrounds. The auxiliary backbone network is used to extract the local features and global features of the target leaf image, and the guided backbone network is used to extract the fusion features of the target leaf image. A multi-level skip connection structure is established between the auxiliary backbone network and the guided backbone network to effectively fuse high and low-level feature maps, so that the composite backbone network can fully learn the edge information and multi-level features of the target leaf image. It not only retains the good generalization ability and convergence ability of the convolutional neural network, but also takes into account the broad global receptive field of the Transformer network. And higher model capacity, so it can realize leaf segmentation in complex background, and has higher segmentation accuracy and faster segmentation speed, so that the segmentation results can more accurately reflect the phenotypic traits of plant leaves.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without paying creative labor.

Fig. 1 is a flowchart of a method for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention;

FIG. 2 is a structural diagram of an auxiliary backbone network provided by an embodiment of the present invention;

FIG. 3 is a structural diagram of a composite backbone network provided by an embodiment of the present invention;

4 is a schematic diagram of the overall network structure of the leaf segmentation model provided by the embodiment of the present invention;

Fig. 5 is a specific flowchart of a method for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention;

6 is a schematic diagram of a calculation process of a weight contribution matrix based on batch normalization provided by an embodiment of the present invention;

Fig. 7 is a schematic diagram of the segmentation result of the first type of leaf image provided by the embodiment of the present invention;

Fig. 8 is a schematic diagram of the segmentation result of the second type of leaf image provided by the embodiment of the present invention;

Fig. 9 is a structural diagram of a system for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention;

Fig. 10 is a specific structural diagram of a system for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

The purpose of the present invention is to provide a method and system for accurately segmenting plant leaves under complex backgrounds, so as to realize the segmentation of plant leaves under complex backgrounds, improve the accuracy and speed of leaf segmentation, and provide a data basis for accurately obtaining phenotypic parameters of plant leaves.

In order to make the above objects, features and advantages of the present invention more comprehensible, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

FIG. 1 is a flowchart of a method for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention. As shown in Figure 1, the method for accurately segmenting plant leaves under complex backgrounds provided by the present invention includes:

Step 101: acquiring a target leaf image; the target leaf image is a plant leaf image under a complex background.

Step 102: Input the target leaf image into the leaf segmentation model to obtain a predicted image segmentation mask of the target leaf image; the predicted image segmentation mask of the target leaf image is used to segment the target leaf image, thereby providing a data basis for obtaining leaf phenotype parameters; the leaf segmentation model is obtained by training a hybrid neural segmentation network; the hybrid neural segmentation network includes a composite backbone network and an image segmentation network; The multi-layer fusion feature map determines the predicted image segmentation mask of the target leaf image; the composite backbone network includes an auxiliary backbone network and a guided backbone network of multi-level residual skip connections; the auxiliary backbone network is constructed based on a convolutional neural network and a Transformer network; the guided backbone network is constructed based on a convolutional neural network. The image segmentation network includes an encoder module and a decoder module connected to each other; the encoder module is constructed based on a spatial convolution pooling pyramid network.

Further, the method for determining the blade segmentation model includes:

Step 1: Obtain a sample data set; the sample data set includes multiple sample leaf images and corresponding real image segmentation masks.

Specifically, the acquisition method of the sample data set includes:

Step 1.1: Obtain a leaf dataset; the leaf dataset includes multiple images of plant leaves under complex backgrounds.

Preferably, the multiple plant leaf images under the complex background include a first type of plant leaf image and a second type of plant leaf image, wherein the first type of plant leaf image includes a healthy plant leaf image, and the second type of plant leaf image includes an image of a plant leaf infected by a disease and insect pest and a stack of unevenly illuminated plant leaf images. The invention improves the robustness and universality of the leaf segmentation model by using two different types of plant leaf images as sample data sets to participate in training, so that accurate segmentation can still be achieved when facing complex samples.

Step 1.2: Denoise, label and expand each plant leaf image in the leaf dataset to obtain multiple sample leaf images and corresponding real image segmentation masks.

Step 2: Input each sample leaf image into the composite backbone network for feature extraction, and obtain the multi-layer auxiliary feature map and multi-layer fusion feature map of each sample leaf image. Wherein, for any sample leaf image, step 2 specifically includes:

Step 2.1: Input the sample leaf image into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map; the multi-layer auxiliary feature map includes: a multi-layer local feature map and a multi-layer global feature map.

Step 2.2: Input the multi-layer auxiliary feature map and the sample leaf image into the guided backbone network for feature extraction to obtain a multi-layer fusion feature map.

FIG. 2 is a structural diagram of an auxiliary backbone network provided by an embodiment of the present invention, and FIG. 3 is a structural diagram of a composite backbone network provided by an embodiment of the present invention. As shown in Figure 2 and Figure 3, the auxiliary backbone network includes: three convolutional neural network modules (ie S0, S1 and S2) and two self-attention network modules (ie S3 and S4), and there is a residual skip connection between two adjacent network modules. The guiding backbone network includes: five convolutional neural network modules, and residual skip connections exist between two adjacent network modules.

Further, the first convolutional neural network module S0 of the auxiliary backbone network includes two two-dimensional convolutional layers connected in sequence. The second convolutional neural network module S1 of the auxiliary backbone network includes three two-dimensional convolutional layers connected in sequence. The third convolutional neural network module S2 of the auxiliary backbone network includes three two-dimensional convolutional layers connected in sequence. The first self-attention network module S3 of the auxiliary backbone network includes a self-attention layer and a fully-connected layer connected in sequence. The second self-attention network module S4 of the auxiliary backbone network includes a self-attention layer and a fully-connected layer connected in sequence. Moreover, there are residual skip connections between the S0 module and the S1 module, between the S1 module and the S2 module, between the S2 module and the S3 module, between the S3 module and the S4 module, between the self-attention layer and the fully connected layer in the S3 module, and between the self-attention layer and the fully connected layer in the S4 module. Each convolutional neural network module of the guided backbone network consists of three two-dimensional convolutional layers.

Therefore, the input of the sample leaf image into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map specifically includes:

Step 2.1.1: Input the sample leaf image to the first convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the first layer of local feature maps.

Step 2.1.2: Input the local feature map of the first layer to the second convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the local feature map of the second layer.

Step 2.1.3: Input the local feature map of the second layer to the third convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the local feature map of the third layer.

Step 2.1.4: Input the third-layer local feature map to the first self-attention network module of the auxiliary backbone network for feature extraction to obtain the first-layer global feature map.

Step 2.1.5: Input the global feature map of the first layer to the second self-attention network module of the auxiliary backbone network for feature extraction, and obtain the global feature map of the second layer.

Said inputting said multi-layer auxiliary feature map and said sample leaf image into said guiding backbone network for feature extraction to obtain a multi-layer fusion feature map, specifically comprising:

Step 2.2.1: Input the local feature maps of the first to third layers, the global feature maps of the first to second layers, and the sample leaf image to the first convolutional neural network module of the guided backbone network for feature extraction, and obtain the first layer of fusion feature map L0.

Step 2.2.2 Input the second to third layer local feature maps, the first to second layer global feature maps and the first layer fusion feature map to the second convolutional neural network module of the guided backbone network for feature extraction, and obtain the second layer fusion feature map L1.

Step 2.2.3 Input the third layer local feature map, the first to second layer global feature map and the second layer fusion feature map to the third convolutional neural network module of the guided backbone network for feature extraction to obtain the third layer fusion feature map L2.

Step 2.2.4 Input the first to second layer global feature maps and the third layer fusion feature map to the fourth convolutional neural network module of the guided backbone network for feature extraction to obtain the fourth layer fusion feature map L3.

Step 2.2.5 Input the second-layer global feature map and the fourth-layer fusion feature map to the fifth convolutional neural network module of the guided backbone network for feature extraction, and obtain the fifth-layer fusion feature map L4.

Further, each layer of residual jump connection between the auxiliary backbone network and the guidance backbone network further includes a channel batch normalization module.

In this embodiment, before inputting the multi-layer auxiliary feature map and the corresponding intermediate image to the corresponding convolutional neural network module in the guided backbone network, it also includes:

(1) The multi-layer auxiliary feature maps are spliced to obtain the corresponding spliced feature maps; the spliced feature maps are: the spliced feature maps of the first to third layers of local feature maps and the first to second layers of global feature maps, the spliced feature maps of the second to third layers of local feature maps and the first to second layers of global feature maps, the spliced feature maps of the third layer of local feature maps and the first to second layers of global feature maps, the spliced feature maps of the first to second layers of global feature maps, or the second layer of global feature maps.

(2) Superposing the spliced feature map and the corresponding intermediate image element by element to obtain a corresponding superimposed feature map; the intermediate image is: the sample leaf image, the first layer fusion feature map, the second layer fusion feature map, the third layer fusion feature map or the fourth layer fusion feature map.

(3) Use the corresponding channel batch normalization module to perform channel batch normalization processing on the overlay feature map, and use the processed overlay feature map as an input to the corresponding convolutional neural network module in the guided backbone network. The present invention performs channel batch normalization processing on superimposed feature maps, which can suppress unimportant channel features in images, thereby realizing the perfect fusion of multi-level feature maps between different backbone networks.

That is to say, the inputting the local feature maps of the first to third layers, the global feature maps of the first to second layers, and the sample leaf image to the first convolutional neural network module of the guided backbone network for feature extraction includes: splicing the local feature maps of the first to third layers with the global feature maps of the first to second layers to obtain a first spliced feature map; superimposing the first spliced feature map and the sample leaf image by elements to obtain a first superimposed feature map; Bootstrap the first convolutional neural network module of the backbone network for feature extraction.

The input of the second to third layer local feature maps, the first to second layer global feature maps and the first layer fusion feature map to the second convolutional neural network module of the guided backbone network for feature extraction specifically includes: splicing the second to third layer local feature maps with the first to second layer global feature maps to obtain a second spliced feature map; superimposing the second spliced feature map and the first layer fusion feature map by element to obtain a second superimposed feature map; using the second channel batch normalization module to perform channel batch normalization processing on the second superimposed feature map and then input it to The second convolutional neural network module of the guided backbone network performs feature extraction.

The input of the third layer local feature map, the first to second layer global feature map and the second layer fusion feature map to the third convolutional neural network module of the guidance backbone network for feature extraction specifically includes: splicing the third layer local feature map and the first to second layer global feature map to obtain a third splicing feature map; superimposing the third splicing feature map and the second layer fusion feature map by element to obtain a third superposition feature map; using the third channel batch normalization module to perform channel batch normalization processing on the third superposition feature map and then input it to the guidance backbone The third convolutional neural network module of the network performs feature extraction.

The input of the first to second layer global feature maps and the third layer fusion feature map to the fourth convolutional neural network module of the guidance backbone network for feature extraction specifically includes: splicing the first to second layer global feature maps to obtain a fourth splicing feature map; superimposing the fourth splicing feature map and the third layer fusion feature map by element to obtain a fourth superposition feature map; using the fourth channel batch normalization module to perform channel batch normalization on the fourth superposition feature map and then inputting it to the fourth convolution neural network module of the guidance backbone network for feature extraction .

The inputting the second-layer global feature map and the fourth-layer fusion feature map to the fifth convolutional neural network module of the guiding backbone network for feature extraction specifically includes: using the second-layer global feature map as the fifth splicing feature map; superimposing the fifth splicing feature map and the fourth layer fusion feature map by elements to obtain the fifth superimposed feature map; using the fifth channel batch normalization module to perform channel batch normalization on the fifth superimposed feature map and then inputting it to the fifth convolutional neural network module of the guiding backbone network for feature extraction.

The global feature map of the second layer extracted by the S4 module is used as the feature map S4', the global feature map of the first layer extracted by the S3 module is used as the feature map S3', the local feature map of the third layer extracted by the S2 module is used as the feature map S2', the local feature map of the second layer extracted by the S1 module is used as the feature map S1', and the local feature map of the first layer extracted by the S0 module is used as the feature map S0'.

Further, taking the generation of the first spliced feature map as an example, the splicing of the multi-layer auxiliary feature maps to obtain the corresponding spliced feature map specifically includes: firstly, using a two-dimensional convolution layer to change the channel dimension of the feature map S4' to be the same as that of the feature map S3', and then perform linear interpolation and upsampling to make the resolution of the feature map S4' the same as that of the feature map S3', so that the channel dimensions and resolutions of the two processed feature maps at this time are exactly the same, and the two are added element by element to obtain the feature map S'. Then the feature map S’ uses a two-dimensional convolutional layer to make the channel dimension the same as the feature map S2’, and then performs linear interpolation and upsampling to make the resolution size the same as the feature map S2’, and then the channel dimension and resolution of the feature map S’ and the feature map S2’ are exactly the same at this time, and the two are added element by element to obtain the feature map S”; and so on, the final splicing to get the feature map S”” (that is, the first stitching feature map).

Correspondingly, taking the generation of the first superimposed feature map as an example, the superposition of the spliced feature map and the corresponding intermediate image by element is obtained to obtain the corresponding superimposed feature map, which specifically includes: using a two-dimensional convolution layer on the feature map S"" to change the channel dimension to be the same as the intermediate image (that is, the sample leaf image), and then perform linear interpolation and upsampling to make the resolution size the same as the intermediate image (that is, the sample leaf image), and then at this time S"" and the intermediate image (that is, the sample leaf image). Fig.

Step 3: Input the multi-layer auxiliary feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the auxiliary image segmentation mask of each sample leaf image; input the multi-layer fusion feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the prediction image segmentation mask of each sample leaf image.

Fig. 4 is a schematic diagram of an overall network structure of a leaf segmentation model provided by an embodiment of the present invention. As shown in Figure 4, when generating the predicted image segmentation mask, first, the feature map L4 extracted by the composite backbone network is passed into the atrous space convolution pooling pyramid module in the encoder of the image segmentation network for learning. Among them, the feature map L4 obtained five multi-scale receptive field feature maps after four different ratio convolution operations and global average pooling operations in Atrous Spatial Pyramid Pooling (ASPP). Then, the five multi-scale receptive field feature maps are concatenated in the channel dimension and then aggregated using two-dimensional convolution, and then the linear interpolation upsampling method is used to expand the spatial size of the feature map to 4 times the original size. Second, the feature map L1 extracted from the composite backbone network is compressed using two-dimensional convolution to compress the channel dimension. Then, the enlarged feature map obtained after upsampling in the encoder of the image segmentation network is residually spliced in the channel dimension. And the spliced feature map is processed by two-dimensional convolution and the weight contribution factor in PixelBatchNormalization (PixelBatchNormalization) to suppress unimportant pixel features. Finally, the spatial size of the feature map is expanded to the size of the input image based on a linear interpolation upsampling method to obtain the predicted image segmentation mask.

Correspondingly, when generating the auxiliary image segmentation mask, the feature map extracted by the S4 module of the auxiliary backbone network is passed to the hollow space convolution pooling pyramid module in the encoder of the image segmentation network, that is, the specific process is similar to the processing of the feature map L4; the feature map extracted by the S1 module of the auxiliary backbone network is compressed using two-dimensional convolution. Then, the enlarged feature map obtained by upsampling in the encoder of the image segmentation network is residually spliced in the channel dimension, that is, the specific process is similar to the processing of the feature map L1.

Step 4: Determine a comprehensive cross-entropy loss according to the real image segmentation mask, the auxiliary image segmentation mask and the predicted image segmentation mask; the comprehensive cross-entropy loss includes: the cross-entropy loss of the composite backbone network and the cross-entropy loss of the auxiliary backbone network.

Step 5: With the goal of minimizing the comprehensive cross-entropy loss, train the hybrid neural segmentation network to obtain the leaf segmentation model. The leaf segmentation model includes a trained composite backbone network and a trained image segmentation network.

Specifically, the formula for calculating the comprehensive cross-entropy loss is:

L＝L _Lead +λ·L _Assist ;

Among them, L represents the comprehensive cross-entropy loss; L _Lead represents the cross-entropy loss of the composite backbone network, which is determined by the real image segmentation mask and the predicted image segmentation mask; L _Assist represents the cross-entropy loss of the auxiliary backbone network, which is determined by the real image segmentation mask and the auxiliary image segmentation mask; λ represents the weight of the auxiliary supervision.

Wherein, the calculation formula of the cross-entropy loss of the composite backbone network is:

The calculation formula of the cross-entropy loss of the auxiliary backbone network is:

Among them, L_leadIndicates the cross-entropy loss of the composite backbone network; L_AssistRepresents the cross-entropy loss of the auxiliary backbone network; m represents the total number of pixels of the input sample leaf image; y_iRepresents the real category value of the i-th pixel in the input sample leaf image, and the real category values of all pixels in the input sample leaf image constitute the real image segmentation mask of the input sample leaf image; p_iRepresents the predicted category value of the i-th pixel in the input sample leaf image, and the predicted category values of all pixels in the input sample leaf image constitute the predicted image segmentation mask of the input sample leaf image; q_iRepresents the auxiliary category value of the i-th pixel in the input sample leaf image, and the auxiliary category values of all pixels in the input sample leaf image constitute the auxiliary image segmentation mask of the input sample leaf image; i represents the sequence number of the pixel in the input sample leaf image.

In this embodiment, by using two kinds of loss functions to supervise and optimize network training, the neural network can achieve faster convergence and improve the efficiency of network training.

Fig. 5 is a specific flowchart of a method for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention. As shown in Figure 5, in practical applications, the specific steps of the method for accurately segmenting plant leaves under complex backgrounds provided by the present invention can also be as follows:

Step 501: Acquire the first type of plant leaf images and the second type of plant leaf images under complex backgrounds; wherein the first type of plant leaf images include healthy plant leaf images, and the second type of plant leaf images include plant leaf images infected by diseases and insect pests and stacked unevenly illuminated plant leaf images.

Step 502: Perform denoising, labeling, and expansion processing on the first-type plant leaf images and the second-type plant leaf images, and divide the processed plant leaf images into a training data set and a verification data set according to a set ratio; wherein, the set ratio is preferably 8:2; the specific denoising process is the use of the existing median filter algorithm; the specific expansion process includes rotating and mirror-flipping the above-mentioned obtained pictures to obtain more training set data.

Step 503: Send the plant leaf images in the training data set to the auxiliary backbone network including the convolution layer and the self-attention layer in the composite backbone network to extract feature maps. As shown in Figure 2, the image first passes through the S0, S1, and S2 convolutional neural network modules in the auxiliary backbone network to extract local and low-level features, and then the extracted feature maps are passed to the S3 and S4 self-attention modules in the auxiliary backbone network to continue extracting global and semantic features of the image.

Step 504: Pass the plant leaf image and the feature maps extracted by the S0, S1, S2, S3, and S4 modules in the auxiliary backbone network to the convolutional neural network-based guided backbone network to extract feature maps. As shown in Figure 3, while extracting local features and semantic features from the bottom up through the convolutional neural network modules in the guided backbone network, the feature maps extracted by the auxiliary backbone network S0, S1, S2, S3, and S4 modules are stitched together at the short-jump residual connection to obtain L0, L1, L2, L3, and L4 feature maps, realizing multi-level fusion of high and low semantic features.

Step 505: Use the weight contribution factors in Channel BatchNormalization to suppress unimportant channel features in the L0, L1, L2, L3, and L4 feature maps obtained in step 504, so as to achieve perfect fusion of multi-level feature maps between different backbone networks. FIG. 6 is a schematic diagram of a calculation process of a weight contribution matrix based on batch normalization provided by an embodiment of the present invention. As shown in Figure 6, the calculation formula for the importance of a channel or pixel is as follows:

Among them, γ _i represents the weight value of the i-th channel (or pixel) calculated in batch normalization, γ _j represents the weight value of the j-th channel (or pixel) calculated in batch normalization, and ω _i represents the importance of the i-th channel (or pixel).

Specifically, after the importance matrix ω is calculated, the batch-normalized feature map is cross-multiplied by the ω matrix, and then the sigmoid nonlinear function is used to process the cross-multiplied result to obtain the final weight contribution matrix. The larger the value in the weight contribution matrix, the more important it is, and the smaller the value, it is relatively unimportant. Then, the weight contribution matrix is cross-multiplied with the original feature map to achieve the effect of suppressing unimportant channel or pixel features.

Step 506: Pass the feature map L4 extracted from the composite backbone network (specifically, the guided backbone network in the composite backbone network) to the atrous space convolution pooling pyramid module in the segmentation network encoder for learning. Among them, the feature map L4 obtained five multi-scale receptive field feature maps after four different ratio convolution operations and global average pooling operations in Atrous Spatial Pyramid Pooling (ASPP). Then the five multi-scale receptive field feature maps are concatenated in the channel dimension and then aggregated using two-dimensional convolution. The feature map spatial size is then expanded to 4 times the original size using a linear interpolation upsampling method. The feature map L1 obtained by extracting the composite backbone network (specifically, the guided backbone network in the composite backbone network) uses two-dimensional convolution to compress the channel dimension. Then, residual stitching is performed in the channel dimension with the feature map obtained after upsampling. And the spliced feature map is processed by two-dimensional convolution and the weight contribution factor in Pixel BatchNormalization (Pixel BatchNormalization) to suppress unimportant pixel features. Finally, based on the linear interpolation upsampling method, the spatial size of the feature map is expanded to the size of the input image to obtain the prediction segmentation mask result. See Figure 4 for the specific process.

Step 507: Supervise and optimize network training using two loss functions.

A hybrid neural segmentation network consisting of a composite backbone network and an image segmentation network is optimized by a cross-entropy loss (cost) function.

In the model, the feature maps extracted by the composite backbone network are used to generate the raw cross-entropy loss L _Lead . In addition, the feature maps extracted by the auxiliary backbone network are used to generate the auxiliary cross-entropy loss L _Assist . Among them, the original cross-entropy loss plays a dominant role in the network optimization process, while the auxiliary cross-entropy loss accelerates the network convergence during the optimization process. Meanwhile, weights are also added to balance auxiliary supervision. Finally, the loss function of the hybrid neural segmentation network looks like this:

L＝L _Lead +λ·L _Assist ;

Among them, L _Lead represents the cross-entropy loss of the composite backbone network (i.e., the original cross-entropy loss); L _Assist represents the cross-entropy loss of the auxiliary backbone network (i.e., the auxiliary cross-entropy loss); λ represents the weight of the auxiliary supervision.

Step 508: Evaluate the segmentation result of the leaf segmentation model by using the boundary intersection ratio.

The calculation formula of Boundary IoU is as follows:

Among them, BIoU represents the boundary intersection ratio, G represents the true category mask of the plant leaf image, P represents the predicted category mask of the plant leaf image, d represents the pixel width of the boundary area, G _d represents the set of all pixels within d pixels from the boundary of the true category mask, and P _d represents the set of all pixels within d pixels from the boundary of the predicted category mask.

To sum up, the present invention proposes a method for accurately segmenting plant leaves under complex backgrounds based on convolutional neural networks and Transformers networks. The convolutional layer and self-attention layer in the composite backbone network are used to extract local and global features of leaf images, and a multi-level connection structure is used to effectively fuse high-level and low-level feature maps, and the edge information and multi-level features of plant leaf images under complex backgrounds are fully learned. It not only retains the good generalization ability and convergence ability of the convolutional layer, but also takes into account the broad global receptive field and higher model capacity of the self-attention layer. This method also has high segmentation accuracy and segmentation speed in the face of various leaf states under complex backgrounds such as different illumination and different background noise interference. The segmentation results of the first type of leaf image and the segmentation results of the second type of leaf image extracted by the method provided by the present invention are shown in FIG. 7 and FIG. 8 .

Fig. 9 is a structural diagram of a system for accurately segmenting plant leaves under a complex background provided by an embodiment of the present invention. As shown in Figure 9, the present invention also provides a system for accurately segmenting plant leaves under complex backgrounds, which specifically includes:

The target leaf image acquisition module 901 is configured to acquire a target leaf image; the target leaf image is an image of a plant leaf under a complex background.

The image segmentation mask prediction module 902 is used to input the target leaf image into the leaf segmentation model to obtain the predicted image segmentation mask of the target leaf image; the predicted image segmentation mask of the target leaf image is used to segment the target leaf image, thereby providing a data basis for obtaining leaf phenotype parameters; the leaf segmentation model is obtained by training a hybrid neural segmentation network; the hybrid neural segmentation network includes a composite backbone network and an image segmentation network; It is used to determine the predicted image segmentation mask of the target leaf image according to the multi-layer fusion feature map of the target leaf image; the composite backbone network includes an auxiliary backbone network and a guided backbone network of multi-level residual skip connections; the auxiliary backbone network is constructed based on a convolutional neural network and a Transformer network; the guided backbone network is constructed based on a convolutional neural network.

In practical applications, the specific structure of the precise plant leaf segmentation system under complex backgrounds provided by the present invention can also be shown in Figure 10, including: an acquisition module 1001, a preprocessing module 1002, a feature map extraction module 1003, an encoder module 1004, a decoder module 1005, a training and optimization module 1006, and a prediction module 1007. Among them, the acquisition module 1001 completes the collection of plant leaf images under complex backgrounds, and then the preprocessing module 1002 realizes labeling and denoising of leaf images, the feature map extraction module 1003, encoder module 1004 and decoder module 1005 realize feature map extraction and learning of leaf images, the training and optimization module 1006 realizes network training and optimization, and finally the prediction module 1007 realizes leaf segmentation mask prediction of plant leaf images under complex backgrounds. Therefore, the precise segmentation system of plant leaves under complex backgrounds provided by the present invention can realize automatic segmentation of plant leaves under complex backgrounds on a large scale and visualize the results.

Specifically, the acquisition module 1001 is configured to: acquire a first type of plant leaf image and a second type of plant leaf image under a complex background, wherein the first type of plant leaf image includes a healthy plant leaf image, and the second type of plant leaf image includes a plant leaf image infected by diseases and insect pests and a stack of unevenly illuminated plant leaf images.

The preprocessing module 1002 is configured to: respectively perform denoising, labeling and expansion processing on the first type of plant leaf image and the second type of plant leaf image, and divide the processed leaf image into a training data set and a verification data set according to a ratio of 8:2; wherein the specific denoising process is to use an existing median filter algorithm. And the extended processing process, which specifically includes rotating and mirror flipping the above obtained pictures to obtain more data of the training set.

The feature map extraction module 1003 is configured to: first transmit the plant leaf image to an auxiliary backbone network including a convolutional layer and a self-attention layer in the composite backbone network. The local and low-level features of the image are extracted through the S0, S1, and S2 convolutional neural network modules in the auxiliary backbone network, and then the extracted feature maps are passed to the S3 and S4 self-attention modules in the auxiliary backbone network to continue to extract global and semantic features of the image. Then the leaf image and the feature maps extracted by the S0, S1, S2, S3, and S4 modules in the auxiliary backbone network are passed into the guided backbone network based on the convolutional neural network. While extracting local features and semantic features from bottom to top through the convolutional neural network modules in the guided backbone network, the leaf image is spliced with the feature maps extracted by the S0, S1, S2, S3, and S4 modules of the auxiliary backbone network at the short-jump residual connection, realizing multi-level fusion of high and low semantic feature maps. Finally, the weight contribution factor in Channel BatchNormalization is used to suppress unimportant channel features.

The encoder module 1004 is configured to: transfer the feature map extracted from the composite backbone network to the encoder module in the image segmentation network for learning. In the encoder, the feature map extracted by the composite backbone network is sent to the Atrous Spatial Pyramid Pooling (ASPP) for learning to obtain a multi-scale receptive field feature map, and then the multi-scale feature map is expanded using the linear interpolation upsampling method after splicing and two-dimensional convolution aggregation processing.

The decoder module 1005 is configured to perform residual splicing of the feature map extended by the encoder module 1004 and the feature map passed into the decoder module in the channel dimension, and then the spliced feature map undergoes two-dimensional convolution processing and pixel batch normalization (Pixel BatchNormalization).

The training and optimization module 1006 is used for: using the training data set obtained by the preprocessing module 1002 and two kinds of loss functions to train and optimize the hybrid neural segmentation network based on the convolutional neural network and the Transformers network to obtain a leaf segmentation model. See step 503 to step 507 for the specific process.

The prediction module 1007 is used to: use the first type of plant leaf image and the second type of plant leaf image under the complex background as the leaf image to be processed, input it into the leaf segmentation model, segment the leaf image to be processed, obtain the predicted leaf segmentation mask of the leaf image to be processed, and evaluate using the boundary intersection ratio (Boundary IoU).

The method and system for accurately segmenting plant leaves under a complex background based on a convolutional neural network and a Transformers network disclosed by the present invention perform leaf segmentation based on a convolutional layer and a self-attention layer. Method Firstly, the plant leaf image is passed into the composite backbone network for feature extraction, and the composite backbone network is composed of an auxiliary backbone network and a guiding backbone network. In the method, the auxiliary backbone network is a hybrid neural network based on convolutional neural network and Transformers, and the guiding backbone network is a convolutional neural network. The plant leaf image is first passed to the auxiliary backbone network for processing, and the auxiliary backbone network includes a convolutional layer and a self-attention layer. Convolutional layers typically have deep non-linear stacking and inductive bias properties, and thus tend to have better generalization and faster convergence. The self-attention layer usually divides the image into several blocks and vectorizes them, and then performs matrix operations on each block vector, so it always has a broad global receptive field and a higher model capacity. At the same time, in the composite backbone network, there is a multi-level cross structure with parallel flow between the auxiliary backbone network and the guiding backbone network, which realizes the effective fusion of feature maps of different levels. In addition, when multi-level feature maps flow between different backbone networks, convolution operations alone cannot be perfectly integrated. Our method uses batch-normalized weight contribution factors to suppress unimportant channels or pixels to simply and efficiently incorporate multi-level features. Finally, the present method introduces auxiliary supervision, which ensures the contribution of the auxiliary backbone network to the output of the segmentation model. Moreover, the joint supervision of the original loss and auxiliary loss enables the segmentation model to learn more discriminative features and converge quickly. Based on the above technology, this method still achieves fast and accurate segmentation of plant leaves in different states when faced with different lighting conditions and occlusion problems in complex environments. Experiments have proved that the method achieves 94.0% Mean Intersectionover Union and 60.8% Boundary IoU when segmenting multi-state plant leaf images under complex backgrounds.

Compared with the prior art, the method and system for accurately segmenting plant leaves under complex backgrounds provided by the present invention have the following characteristics:

1. The leaf segmentation model provided by the present invention is constructed based on convolutional neural network and Transformers network, which can realize accurate segmentation of plant leaves under complex background.

2. The present invention utilizes the Transformer network to process the leaf image to obtain the global context of the image.

3. The present invention uses a hybrid neural network (that is, an auxiliary backbone network) including a convolutional layer and a self-attention layer to process leaf images, which has better generalization ability and faster convergence ability while ensuring a global receptive field.

4. The present invention builds a composite backbone network with a multi-level structure to process leaf images, transfers the feature information of the shallow layer and the deep layer to each other, and integrates feature information of various backbone networks.

5. The present invention uses two types of loss supervision in network optimization and training to achieve faster convergence of the neural network.

6. The neural network built by the present invention can load the existing pre-trained weights without additional cost, so as to realize the rapid transmission of information.

7. The present invention uses the weight contribution factor in the batch normalization (BatchNormalization) to suppress unimportant information in the leaf image.

8. The present invention uses Boundary Intersection over Union (BIoU) to evaluate the segmentation effect, and more carefully distinguishes the integrity of the predicted image segmentation mask.

Each embodiment in this specification is described in a progressive manner, each embodiment focuses on the difference from other embodiments, and the same and similar parts of each embodiment can be referred to each other. As for the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and for the related information, please refer to the description of the method part.

In this paper, specific examples are used to illustrate the principles and implementation methods of the present invention. The descriptions of the above examples are only used to help understand the core ideas of the present invention; meanwhile, for those of ordinary skill in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims (8)

1. a method for precise segmentation of plant leaves under a complex background, characterized in that, the precise segmentation method of plant leaves under the complex background comprises: Obtaining a target leaf image; the target leaf image is a plant leaf image under a complex background; The target leaf image is input into the leaf segmentation model to obtain a predicted image segmentation mask of the target leaf image; the predicted image segmentation mask of the target leaf image is used to segment the target leaf image; The leaf segmentation model is obtained by training a hybrid neural segmentation network; the hybrid neural segmentation network includes a composite backbone network and an image segmentation network; the trained composite backbone network is used to extract features from the target leaf image to obtain a multi-layer fusion feature map of the target leaf image; the trained image segmentation network is used to determine the predicted image segmentation mask of the target leaf image according to the multi-layer fusion feature map of the target leaf image; The composite backbone network includes an auxiliary backbone network and a guided backbone network of multi-level residual skip connections; the auxiliary backbone network is constructed based on a convolutional neural network and a Transformer network; the guided backbone network is constructed based on a convolutional neural network; The method for determining the blade segmentation model includes: Obtain a sample data set; the sample data set includes a plurality of sample leaf images and corresponding real image segmentation masks; Input each sample leaf image into the composite backbone network for feature extraction, and obtain the multi-layer auxiliary feature map and multi-layer fusion feature map of each sample leaf image; Input the multi-layer auxiliary feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the auxiliary image segmentation mask of each sample leaf image; Input the multi-layer fusion feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the prediction image segmentation mask of each sample leaf image; A comprehensive cross-entropy loss is determined according to the real image segmentation mask, the auxiliary image segmentation mask, and the predicted image segmentation mask; the comprehensive cross-entropy loss includes: the cross-entropy loss of the composite backbone network and the cross-entropy loss of the auxiliary backbone network; Taking the minimum comprehensive cross-entropy loss as the goal, the hybrid neural segmentation network is trained to obtain a leaf segmentation model; the leaf segmentation model includes a trained composite backbone network and a trained image segmentation network; Said inputting each sample leaf image into the composite backbone network for feature extraction, obtaining a multi-layer auxiliary feature map and a multi-layer fusion feature map of each sample leaf image, specifically including: For any sample leaf image, the sample leaf image is input into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map; the multi-layer auxiliary feature map includes: a multi-layer local feature map and a multi-layer global feature map; The multi-layer auxiliary feature map and the sample leaf image are input into the guided backbone network for feature extraction to obtain a multi-layer fusion feature map. 2. The method for accurately segmenting plant leaves under complex backgrounds according to claim 1, wherein the auxiliary backbone network comprises: three convolutional neural network modules and two self-attention network modules, and there is a residual jump connection between two adjacent network modules; The described sample leaf image is input into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map, which specifically includes: The sample leaf image is input to the first convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the first layer of local feature maps; The first layer of local feature maps are input to the second convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the second layer of local feature maps; The second layer of local feature maps are input to the third convolutional neural network module of the auxiliary backbone network for feature extraction to obtain the third layer of local feature maps; Inputting the third layer of local feature maps to the first self-attention network module of the auxiliary backbone network for feature extraction to obtain the first layer of global feature maps; The first layer of global feature map is input to the second self-attention network module of the auxiliary backbone network for feature extraction to obtain the second layer of global feature map. 3. The method for accurately segmenting plant leaves under complex backgrounds according to claim 1, wherein the guided backbone network comprises: five convolutional neural network modules, and there is a residual jump connection between two adjacent network modules; Said inputting said multi-layer auxiliary feature map and said sample leaf image into said guiding backbone network for feature extraction to obtain a multi-layer fusion feature map, specifically comprising: Inputting the first to third layers of local feature maps, the first to second layers of global feature maps and the sample leaf image to the first convolutional neural network module of the guided backbone network for feature extraction to obtain the first layer of fusion feature maps; The second to third layers of local feature maps, the first to second layers of global feature maps and the first layer of fusion feature maps are input to the second convolutional neural network module of the guided backbone network for feature extraction to obtain the second layer of fusion feature maps; The third layer of local feature maps, the first to second layer of global feature maps and the second layer of fusion feature maps are input to the third convolutional neural network module of the guided backbone network for feature extraction, and the third layer of fusion feature maps is obtained; The first to the second layer of global feature maps and the third layer of fusion feature maps are input to the fourth convolutional neural network module of the guided backbone network for feature extraction to obtain the fourth layer of fusion feature maps; The second layer global feature map and the fourth layer fusion feature map are input to the fifth convolutional neural network module of the guided backbone network for feature extraction to obtain the fifth layer fusion feature map. 4. The method for accurately segmenting plant leaves under complex backgrounds according to claim 3, wherein each layer of residual jump connection between the auxiliary backbone network and the guide backbone network also includes a channel batch normalization module; before inputting the multi-layer auxiliary feature map and the corresponding intermediate image to the corresponding convolutional neural network module in the guide backbone network, it also includes: The multi-layer auxiliary feature map is spliced to obtain a corresponding spliced feature map; the spliced feature map is: a spliced feature map of the first to third layer local feature map and the first to second layer global feature map, a spliced feature map of the second to third layer local feature map and the first to second layer global feature map, a spliced feature map of the third layer local feature map and the first to second layer global feature map, a spliced feature map of the first to second layer global feature map or the second layer global feature map; Superimpose the spliced feature map and the corresponding intermediate image element by element to obtain a corresponding superimposed feature map; the intermediate image is: the sample leaf image, the first layer fusion feature map, the second layer fusion feature map, the third layer fusion feature map or the fourth layer fusion feature map; Using the corresponding channel batch normalization module to perform channel batch normalization processing on the overlay feature map, and use the processed overlay feature map as an input to the corresponding convolutional neural network module in the guided backbone network. 5. The method for accurately segmenting plant leaves under complex backgrounds according to claim 1, wherein the formula for calculating the comprehensive cross-entropy loss is: L＝L _Lead +λ·L _Assist ; Among them, L represents the comprehensive cross-entropy loss; L _Lead represents the cross-entropy loss of the composite backbone network, which is determined by the real image segmentation mask and the predicted image segmentation mask; L _Assist represents the cross-entropy loss of the auxiliary backbone network, which is determined by the real image segmentation mask and the auxiliary image segmentation mask; λ represents the weight of the auxiliary supervision. 6. The method for accurately segmenting plant leaves under complex backgrounds according to claim 1, wherein the calculation formula of the cross-entropy loss of the composite backbone network is: The calculation formula of the cross-entropy loss of the auxiliary backbone network is: Among them, L_leadIndicates the cross-entropy loss of the composite backbone network; L_AssistRepresents the cross-entropy loss of the auxiliary backbone network; m represents the total number of pixels of the input sample leaf image; y_iRepresents the real category value of the i-th pixel in the input sample leaf image, and the real category values of all pixels in the input sample leaf image constitute the real image segmentation mask of the input sample leaf image; p_iRepresents the predicted category value of the i-th pixel in the input sample leaf image, and the predicted category values of all pixels in the input sample leaf image constitute the predicted image segmentation mask of the input sample leaf image; q_iRepresents the auxiliary category value of the i-th pixel in the input sample leaf image, and the auxiliary category values of all pixels in the input sample leaf image constitute the auxiliary image segmentation mask of the input sample leaf image; i represents the sequence number of the pixel in the input sample leaf image. 7. The method for accurately segmenting plant leaves under complex backgrounds according to claim 1, wherein the method for obtaining the sample data set comprises: Obtaining a leaf data set; the leaf data set includes multiple plant leaf images under complex backgrounds; Each plant leaf image in the leaf dataset is denoised, labeled and expanded to obtain multiple sample leaf images and corresponding real image segmentation masks. 8. A system for precise segmentation of plant leaves under complex backgrounds, characterized in that the system for precise segmentation of plant leaves under complex backgrounds comprises: A target leaf image acquisition module, configured to acquire a target leaf image; the target leaf image is a plant leaf image under a complex background; An image segmentation mask prediction module, configured to input the target leaf image into the leaf segmentation model to obtain a predicted image segmentation mask of the target leaf image; the predicted image segmentation mask of the target leaf image is used to segment the target leaf image; The leaf segmentation model is obtained by training a hybrid neural segmentation network; the hybrid neural segmentation network includes a composite backbone network and an image segmentation network; the trained composite backbone network is used to extract features from the target leaf image to obtain a multi-layer fusion feature map of the target leaf image; the trained image segmentation network is used to determine the predicted image segmentation mask of the target leaf image according to the multi-layer fusion feature map of the target leaf image; The composite backbone network includes an auxiliary backbone network and a guided backbone network of multi-level residual skip connections; the auxiliary backbone network is constructed based on a convolutional neural network and a Transformer network; the guided backbone network is constructed based on a convolutional neural network; The method for determining the blade segmentation model includes: Obtain a sample data set; the sample data set includes a plurality of sample leaf images and corresponding real image segmentation masks; Input each sample leaf image into the composite backbone network for feature extraction, and obtain the multi-layer auxiliary feature map and multi-layer fusion feature map of each sample leaf image; Input the multi-layer auxiliary feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the auxiliary image segmentation mask of each sample leaf image; Input the multi-layer fusion feature map of each sample leaf image into the image segmentation network for prediction segmentation, and obtain the prediction image segmentation mask of each sample leaf image; A comprehensive cross-entropy loss is determined according to the real image segmentation mask, the auxiliary image segmentation mask, and the predicted image segmentation mask; the comprehensive cross-entropy loss includes: the cross-entropy loss of the composite backbone network and the cross-entropy loss of the auxiliary backbone network; Taking the minimum comprehensive cross-entropy loss as the goal, the hybrid neural segmentation network is trained to obtain a leaf segmentation model; the leaf segmentation model includes a trained composite backbone network and a trained image segmentation network; Said inputting each sample leaf image into the composite backbone network for feature extraction, obtaining a multi-layer auxiliary feature map and a multi-layer fusion feature map of each sample leaf image, specifically including: For any sample leaf image, the sample leaf image is input into the auxiliary backbone network for feature extraction to obtain a multi-layer auxiliary feature map; the multi-layer auxiliary feature map includes: a multi-layer local feature map and a multi-layer global feature map; The multi-layer auxiliary feature map and the sample leaf image are input into the guided backbone network for feature extraction to obtain a multi-layer fusion feature map.