CN114493975A - Method and system for target detection of seedling rotating frame - Google Patents

Abstract

The invention relates to a seedling rotating frame target detection method and system, and provides an improved YOLOv5 model for seedling rotating frame target detection based on a decoupling structure based on YOLOv 5. The method comprises the steps of introducing deformable convolution into a backbone network to solve the problem of irregular edges of detected targets (seedlings), improving detection accuracy, then setting a decoupling head network in YOLOv5, decoupling an original detection task into a plurality of subtasks, independently designing each subtask to learn the optimal characteristics of the corresponding task, and finally integrating all detection results to obtain a final rotation detection frame. The method realizes accurate identification of the seedlings based on the improved YOLOv5 model.

Description

A method and system for target detection of seedling rotating frame

technical field

The invention relates to the technical field of seedling cultivation, in particular to a method and system for detecting a target of a seedling rotating frame.

Background technique

With the rapid development of science and technology, seedling tissue culture technology is gradually becoming mature. As a kind of asexual reproduction technology, it has many advantages such as short cultivation cycle, high survival rate, high yield, and can reduce production costs, and has a very good development. prospect. Following the workflow of traditional manual cutting of seedlings, the seedling cutting robot can be divided into a seedling identification system, a seedling pose and coordinate analysis system and a cutting execution system. The target recognition of seedlings is a necessary link in the development of seedling cutting robots.

SUMMARY OF THE INVENTION

In view of this, the present invention provides a seedling rotation frame target detection method and system to achieve accurate identification of seedlings.

For achieving the above object, the present invention provides the following scheme:

A seedling rotation frame target detection method, the method comprises the following steps:

Annotate multiple historical seedling sample images, obtain the label of each historical seedling sample image, and construct a sample set containing the historical seedling image and the label of the historical seedling image;

The improved YOLOv5 model is trained by using the sample set, and the improved YOLOv5 model after training is obtained as a target detection model; the improved YOLOv5 model is to replace the convolution in the CBL structure in the YOLOv5 model with a variable convolution, and Replacing the head network in the YOLOv5 model with the decoupling head network;

Input the image of the seedling to be tested into the target detection model to obtain the seedling detection result.

Optionally, the label of the historical seedling sample image is (c,x,y,l,s,CSL(θ));

Among them, c is the category of the seedlings in the historical seedling sample image; x, y represent the coordinates of the center point of the marked box of the seedlings in the historical seedling sample image; l and s represent the long and short sides of the marked box, respectively ; θ represents the clockwise angle between the long side of the labeled box and the horizontal axis, CSL(θ) represents the angle category when the clockwise angle between the long side of the labeled box and the horizontal axis is θ;

Among them, g(·) is the window function, r is the radius of the window function, and θ′ is the angle category parameter.

Optionally, the improved YOLOv5 model includes a sequentially connected backbone network, a neck network, and a decoupled head network;

The uniform backbone network includes a Focus structure, a CSP1_1 structure, a first CSP1_3 structure, a second CSP1_3 structure and an SPP structure connected in sequence; the CSP structure includes a first convolution layer, a first normalization layer and a first Leaky_relu Activation function, the first CSP1_3 structure and the second CSP1_3 structure both include an improved CBL module, a residual module, a second convolution layer and a Concate layer, and the improved CBL module includes a variable convolution layer, a second normalization layer. Unification layer and second Leaky_relu activation function;

the neck network includes three down-sampling layers and three up-sampling layers;

The decoupling head network includes a third convolution layer, a decoupling branch and a classification branch connected in parallel with the third convolution layer; the decoupling branch includes a fourth convolution layer, a regression box loss function and a bounding box loss, the classification branch includes a fully connected layer, a class loss function, and an angle loss.

Optionally, the function expression of the variable convolution layer is:

Among them, y(p ₀ ) represents the feature value at p ₀ output by the variable convolution layer, p ₀ is any position in the feature map of the input variable convolution layer, and p _n is the grid R centered at p ₀ The nth sampling position in the grid R is the 3×3 convolution kernel, Δp _n is the offset of p _n , and x(p ₀ +p _n +Δp _n ) represents the feature of the input variable convolutional layer Eigenvalues at p ₀ +p _n + _Δpn in the graph.

Optionally, the total loss function used in the process of training the improved YOLOv5 model using the sample set is:

Among them, L represents the total loss function, N represents the number of anchor points, i represents the ith anchor point, obj _i represents the binary value used to distinguish foreground and background, L _reg , L _CSL , L _cls , and L _iou represent regression, respectively Loss function, angle loss function, category loss function and frame loss function, (x, y) represents the coordinates of the center point of the labeled box of the seedlings in the historical seedling sample images, and l and s respectively represent the length change and sum of the labeled boxes. Short side, θ represents the clockwise angle between the long side of the labeled box and the horizontal axis, CSL(θ) represents the angle category when the clockwise angle between the long side of the labeled box and the horizontal axis is θ, λ ₁ , λ ₂ , λ ₃ and λ ₄ represent the weights of regression loss function, angle loss function, class loss function and bounding box loss function, respectively.

Optionally, the regression loss function is:

表示平滑函数，

Among them, a represents the difference between the prediction box and the label box,

represents a smooth function,

The angle loss function is:

Among them, M represents the total number of angle categories, θn is the angle category, y _iθn is the angle sign function of the ith anchor point, 1 if the angle category is true, 0 otherwise, P _iθn is the angle category of the ith anchor point belonging to the probability of the angle class θn;

The class loss function is:

表示第i个锚点的种苗类别属于种苗类别c_n的概率，

表示第i个锚点类别函数；Among them, C represents the number of seedling categories,

represents the probability that the seedling category of the i- _th anchor belongs to the seedling category cn,

Represents the i-th anchor category function;

The bounding box loss function is:

C表示覆盖在预测方框与标注方框之间的最小框，B为预测方框，B^gt为标注方框。Among them, IoU represents the intersection ratio of the prediction box and the label box,

C represents the smallest box covered between the prediction box and the label box, B is the prediction box, and B ^gt is the label box.

A seedling rotation frame target detection system, the system comprises:

The labeling module is used to label multiple historical seedling sample images, obtain the label of each historical seedling sample image, and construct a sample set including the historical seedling image and the label of the historical seedling image;

The model training module is used to train the improved YOLOv5 model using the sample set, and obtain the improved YOLOv5 model after training as a target detection model; the improved YOLOv5 model is to replace the convolution in the CBL structure in the YOLOv5 model with The variable convolution is obtained by replacing the head network in the YOLOv5 model with the decoupled head network;

The detection module is used to input the image of the seedling to be tested into the target detection model to obtain the seedling detection result.

Optionally, the label of the historical seedling sample image is (c,x,y,l,s,CSL(θ));

Among them, c is the category of the seedlings in the historical seedling sample image; x, y represent the coordinates of the center point of the marked box of the seedlings in the historical seedling sample image; l and s represent the long and short sides of the marked box, respectively ; θ represents the clockwise angle between the long side of the labeled box and the horizontal axis, CSL(θ) represents the angle category when the clockwise angle between the long side of the labeled box and the horizontal axis is θ;

Among them, g(·) is the window function, r is the radius of the window function, and θ′ is the angle category parameter.

Optionally, the improved YOLOv5 model includes a sequentially connected backbone network, a neck network, and a decoupled head network;

The uniform backbone network includes a Focus structure, a CSP1_1 structure, a first CSP1_3 structure, a second CSP1_3 structure and an SPP structure connected in sequence; the CSP structure includes a first convolution layer, a first normalization layer and a first Leaky_relu Activation function, the first CSP1_3 structure and the second CSP1_3 structure both include an improved CBL module, a residual module, a second convolution layer and a Concate layer, and the improved CBL module includes a variable convolution layer, a second normalization layer. Unification layer and second Leaky_relu activation function;

the neck network includes three down-sampling layers and three up-sampling layers;

The decoupling head network includes a third convolution layer, a decoupling branch and a classification branch connected in parallel with the third convolution layer; the decoupling branch includes a fourth convolution layer, a regression box loss function and a bounding box loss, the classification branch includes a fully connected layer, a class loss function, and an angle loss.

Optionally, the function expression of the variable convolution layer is:

Among them, y(p ₀ ) represents the feature value at p ₀ output by the variable convolution layer, p ₀ is any position in the feature map of the input variable convolution layer, and p _n is the grid R centered at p ₀ The nth sampling position in the grid R is the 3×3 convolution kernel, Δp _n is the offset of p _n , and x(p ₀ +p _n +Δp _n ) represents the feature of the input variable convolutional layer Eigenvalues at p ₀ +p _n + _Δpn in the graph.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The invention discloses a method and a system for detecting a target of a rotating frame of seedlings. The method of the present invention proposes an improved YOLOv5 model based on a decoupling structure for target detection of a rotating frame of seedlings based on YOLOv5. Introduce deformable convolution into the backbone network to solve the edge irregularity problem of the detected target (seedling), improve the detection accuracy, and then set up a decoupling head network in YOLOv5 to decouple the original detection task into multiple subtasks , each subtask is individually designed to learn the optimal features of the corresponding task, and finally all detection results are integrated to obtain the final rotated detection frame. The method of the invention realizes the accurate identification of seedlings based on the improved YOLOv5 model.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative labor.

Fig. 1 is the flow chart of a kind of seedling rotation frame target detection method provided in the embodiment of the present invention;

2 is a schematic diagram of a method for defining a rotation frame provided by an embodiment of the present invention; FIG. 2(a) is a schematic diagram of a five-parameter method in an angular range of 90°, and FIG. 2(b) is a schematic diagram of a five-parameter method in an angular range of 180°, Figure 2(c) is a schematic diagram of the ordered quadrilateral representation;

3 is a schematic structural diagram of an improved YOLOv5 model provided by an embodiment of the present invention;

4 is a schematic structural diagram of a variable convolution layer provided by an embodiment of the present invention;

5 is a comparison diagram of the receptive field provided by an embodiment of the present invention; wherein, FIG. 5(a) is a schematic diagram of the receptive field of a two-dimensional convolutional layer, and FIG. 5(b) is a schematic diagram of the receptive field of a variable convolutional layer;

FIG. 6 is a schematic structural diagram of a decoupling header network provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The purpose of the present invention is to provide a method and system for detecting the target of a seedling rotating frame, so as to realize the accurate identification of the seedling.

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Taking the identification of Phalaenopsis seedlings as an example, the target recognition of Phalaenopsis seedlings is to identify different parts of the seedlings. The rotating frame detection is usually used for target recognition of aerospace remote sensing images. Zhu Yu et al. The R2-FRCNN rotation frame detection network is proposed, which uses two stages of coarse adjustment and fine adjustment to achieve detection. The coarse adjustment converts the horizontal frame into a rotation frame, and the fine adjustment further optimizes the rotation frame positioning, and carried out on the DOTA remote sensing data set. verify. Xue Yang et al. proposed an end-to-end multi-class rotation frame detection algorithm for complex aerial images, which fused the features of two different layers, and in the second stage of the network, five parameters (x, y, w, h, θ) to define any direction rectangle. Ran Qin et al. proposed an Arbitrary Oriented Area Simple Network (AO-RPN) to generate translation direction proposals for horizontal anchors in the work of aerial image rotation frame target detection, and proposed a multi-head network (MRDet) to decouple the detection task. For multiple sub-tasks, by specially designing each task branch to learn the optimal features of each item, good detection results have been achieved in the DOTA and HRSC2016 datasets. There is no relevant record about using the YOLOv5 model for seedling identification in the prior art.

As shown in Figure 1, the present invention provides a kind of seedling rotation frame target detection method, and described method comprises the following steps:

Step 101: Label a plurality of historical seedling sample images, obtain a label of each historical seedling sample image, and construct a sample set including the historical seedling image and the labels of the historical seedling image.

Using an industrial camera to take pictures, 1505 seedling images were obtained, and the obtained seedling images were divided into training set and test set according to 8:2.

The existing rotation frame definition methods are generally classified into two types, including the five-parameter direction definition method and the eight-parameter quadrilateral definition method. The five-parameter definition method realizes the rotation frame detection in any direction by adding the direction parameter θ, and the definition of the starting and ending points of the angle will lead to different angle ranges. Schematic diagram of the method, in this method, the angle θ represents the acute angle formed by the rotation frame and the x-axis, and this frame is marked as w, and the other frame is marked as h. The θ defined by this method represents the range of [-90,0) , Figure 2(b) is a schematic diagram of the five-parameter method in the range of 180°, and the five-parameter method in the range of 180° in Figure 2(b) is the long-side definition method. In the long-side definition method, the angle θ represents the length of the rotation frame h and The angle formed by the indentation of the x _- axis. At this time, the definition range of the angle is [-90, 90). Figure ₂ (c) is a schematic diagram of the definition method of the ordered quadrilateral. , x ₂ , y ₂ , x ₃ , y ₃ , x ₄ , y ₄ ) are sorted from the leftmost point as the starting point.

In the regression algorithm based on 90° angle, due to the periodicity of angular (PoA) and the exchangeability of edges (EoE), the directional loss will be very large when the angle is in the edge state, which increases the difficulty of regression. ; In the 180° long-side definition method, the sudden increase in loss only comes from PoA; while the quadrilateral definition method pre-sorts the corner points, if the detection frame is inconsistent with the real frame, the boundary overrun problem will occur, resulting in a larger loss. value.

So far, only θ in the long-side definition method based on 180° regression has a boundary problem, and the Circular Smooth Label (CSL) method can deal with the problem caused by the boundary exceeding the limit, so this paper combines the long-side definition method. The rotation frame angle is defined with CSL, and the regression problem of the rotation frame angle is transformed into a classification problem to limit the range of detection results, thereby avoiding the problem of large losses caused by the regression-based angle detection method. The present invention uses formula (1) to classify the entire defined angle range, and 1° is classified into one category. g(x) in formula (1) is a window function, and the window radius is controlled by r. The window function satisfies the four properties of periodicity, symmetry, extreme value and monotonicity. By setting the window function, the model can measure the angular distance between the detection label and the real label, that is, the closer the detection value within a certain range is to the real value, the smaller the loss value.

Specifically, the dataset adopts the PASALVOC labeling format, and uses the rolabelImg labeling tool to manually label the targets in the seedling images, and the labeling files include the matrix coordinate parameters of the real targets. The labeled data is stored in the label file in xml format. First, convert the xml label file into the txt file required by YOLOv5. The converted format is (c, x, y, l, s, θ) where c is the category; x, y represent the coordinates of the center point of the box; l, s represent the long and short sides of the labeled box, respectively; θ represents the clockwise angle between the x-axis and the long side, and θ∈[1,180). The Circular Smooth Label (CSL) method can deal with the problem caused by the boundary exceeding the limit. It combines the long edge definition method and CSL to define the angle of the rotating frame. By converting the regression problem of the angle of the rotating frame into a classification problem, Limit the range of prediction results, thereby avoiding the problem of large losses caused by regression-based angle prediction methods. We categorize the entire defined range of angles, with 1° in one category. The expression of CSL is:

The g(x) in the formula is the window function, and the window radius is controlled by r. The window function satisfies the four properties of periodicity, symmetry, extreme value and monotonicity, and the more typical optional impulse function, Gaussian function, etc. θ′ is the angle category parameter. Input the angle θ in the label into Equation 1, the window function g(x) converts the original label into an angle category, and classifies the label (c, x, y, l, s, CSL(θ)).

Step 102, using the sample set to train the improved YOLOv5 model to obtain the improved YOLOv5 model after training as a target detection model; the improved YOLOv5 model is to replace the convolution in the CBL structure in the YOLOv5 model with a variability volume product, and replace the head network in the YOLOv5 model with the decoupled head network.

As shown in Figure 3, the improved YOLOv5 model includes a backbone network, a neck network, and a decoupled head network.

1. Backbone network

As shown in Figure 3, the backbone network in the improved YOLOv5 model is used to extract the feature representation in the image, which mainly includes the Focus structure, the CSP1_1 structure, the CSP1_3 structure, and the SPP structure. Among them, the Focus structure performs a slicing operation on the input seedling image. Specifically, the image in step 101 is turned into a feature map, and then sent to the CSP1_1 structure. The CSPl_1 structure includes a convolutional layer (Conv), a normalization layer (BN) and a Leaky_relu activation function. CSP1_3 structure includes improved CBL module, residual module and convolution layer, Concate. The SPP structure uses the maximum pooling methods of 1*1, 5*5, 9*9 and 13*13 to process the feature maps respectively, and then performs multi-scale feature fusion through the concat operation. Due to its inherent plant characteristics, phalaenopsis seedlings have irregular structures and geometric diversity. Therefore, the traditional convolution operation is not ideal for rotating frame detection of phalaenopsis seedlings. In view of this problem, a deformable convolution that is more suitable for geometric deformation is introduced, so that the receptive field of the convolution kernel can pay more attention to the target edge features and optimize the detection performance. Based on this, the embodiment of the present invention provides an improved CBL (Conv Layer+BN Layer +LeakyReLU Layer) module, which replaces the convolutional layer in the existing CBL module with a variable convolutional layer.

The traditional two-dimensional convolution is to perform sliding sampling on the feature map through a fixed-structure grid R, and the sampled values w are weighted and summed. The grid R defines the size and weight w of the sampling area. The output y(p ₀ ) for each position p ₀ in the input feature map x is:

where p _n is all sampling positions in grid R centered at p ₀ . As shown in Figure 4, in deformable convolution, an offset p _n is added to each sampling point p ₀ , and the corresponding output y(p ₀ ) becomes:

x is the seedling feature map, p ₀ is each position in the feature map, where p _n is all the sampling positions in the grid R centered on p ₀ , and the grid R is the 3×3 convolution kernel. The convolutions in all CBL structures in the backbone of the present invention are replaced with variable convolutions.

The comparison result of the receptive field of the existing convolution layer and the variable convolution layer proposed in the embodiment of the present invention is shown in Figure 5. According to Figure 5, it can be seen that the geometric shape of the edge of the Phalaenopsis orchid seedling is irregular, and the deformable convolution is used to solve the problem. Its object detection problem is more efficient.

Deformable convolution changes the shape of the traditional sampling area. During the model calculation process, the sampling point may be extended to the part outside the area of interest, and weights are introduced on the basis of the above structure, not only learning the offset value of each position , and also learn the weight of each sampling point to reduce the influence of extreme sampling points on network feature extraction, thereby ensuring the effectiveness of seedling feature extraction. After this step, the preliminary extracted seedling feature information is output.

2. The neck network

As shown in FIG. 3 , the neck network in the embodiment of the present invention exemplarily adopts the general structure of target detection: SPP combined with PANet. The use of PANet increases the number of upsampling and downsampling, and adopts three upsampling and three downsampling, so that the feature information in the low-level information is completely transmitted to the high-level, which reduces the loss in the process of information transmission and improves the low-level information. The utilization of information increases the accuracy of seedling target detection. This step outputs the enhanced seedling feature information.

3. Decoupling the head network

The decoupling head network proposes a decoupling detection structure for the rotating frame target for the coupling problem (see Figure 6 for a schematic diagram of the structure of the decoupling head network): the embodiment of the present invention replaces the original head structure with a decoupling head structure The YOLO head (HEAD) to improve the detection performance is used to optimize the positioning effect of the rotating frame.

The decoupling head network is shown in Figure 6: After the feature information is output from the neck network, it is input into the decoupling head network. After processing by 3*3 convolution layers, different categories of information are classified and processed, and then the decoupling branch Two parallel branches with 3 × 3 convolutional layers are added to perform classification tasks and regression tasks respectively; while the classification branch adopts a fully connected layer structure. The problem of detecting target frame orientation has been transformed from regression to classification through the long-side definition method and CSL method. In the decoupled detection structure, the directed loss detection is classified into the classification branch. Various losses jointly promote the update of network weight parameters. After all kinds of loss calculation iterations, the training model is obtained.

As can be seen in Figure 6, in the regression branch, Cls (category loss) and Angle (angle loss) are obtained through the FC (full connection layer) layer for each layer of features, and Reg (regression box information) IoU (frame loss) is obtained through the Conv layer. ).

The multi-task loss function is:

Among them, N represents the number of anchor points, and obj _n is a binary value, which is used to distinguish the foreground from the background. L _reg , L _CSL , L _cls , and L _iou represent regression loss function, angle loss function, class loss function, and bounding box loss function, respectively.

表示预测的框坐标。where v=(v _x , v _y , v _w , v _h ) represents the frame coordinates of GT,

Represents the predicted box coordinates.

GT is the real box. x is the numerical difference between the predicted box and the ground-truth box.

The angle loss formula is as follows:

Among them, M represents the total number of angle categories, θn is the angle category, y _iθn is the angle sign function, 1 if the category is true, and 0 otherwise. P _iθn is the probability that the observed sample belongs to this angle sample.

The classification loss function is as follows:

Among them, i represents the ith sample, and _Pin represents the probability that the ith sample is predicted to be the nth class.

The frame loss formula is as follows:

in

C represents the smallest box covering between the predicted bounding box and the ground-truth bounding box, B is the predicted bounding box, and Bgt is the ground-truth bounding box.

Cls and Angle in Figure 6 correspond to Cls and Angle of the FC layer in Figure 3,

In the embodiment of the present invention, Cls (category loss), Angle (angle loss), Reg (regression frame loss), and IoU (frame loss) are weighted and accumulated together, so that the model can adaptively adjust the coefficients of the four losses, and save the model's weight parameter. In the part of calculating the loss, it is realized by adding directed loss on the basis of the original confidence Loss, classification Loss and regression Loss. When calculating the directional loss, the regression problem has been transformed into a classification problem by the above-mentioned long-side definition method and CSL method. Therefore, in the embodiment of the present invention, the label of the marked box is processed by CSL before calculating the angle category loss, which is verified by experiments. The directed loss calculation method can improve the detection effect. YOLOv5 regression Loss for horizontal frame detection usually uses different IOUs such as GIOU and CIOU for calculation, but the IOU formula between the two rotation frames is usually not steerable, which makes us have to rotate the IOU loss function to reverse Propagation to adjust its own parameters. In rotation detection, the non-steerable rotation IOU is usually approximated to make the network train normally, so this method is not suitable for the rotation frame target detection task. However, in the embodiment of the present invention, the regression problem is converted into a classification problem by combining the long-side definition method and CSL, and the original IOU loss function of YOLOv5 can be applied, so GIOU is still used for calculation in this part. In the confidence loss part, the rotation frame IOU is selected as the weight coefficient of the confidence branch, so as to avoid a small range of angular deviations when calculating the IOU of the horizontal frame, which will have too much influence on the rotation frame IOU, and then there will be problems when the frame parameters are correctly predicted but the angle parameters are wrong. Excessive score situation.

Calculate the minimum closure area area q1 of the two boxes between the coordinate information of the prediction frame of each part of the seedlings and the coordinate information of the labeling frame, and the area of the intersection q2. Through q1 and q2, calculate the area in the closure area. The proportion of the area belonging to the two boxes in the closure area is q=lq1-q2|/q1, and then calculate the GloU loss between the coordinate information of the prediction frame and the coordinate information of the labeling frame, that is, GloU=p-q, and the GloU loss is used in the training. Backpropagation is performed in the network, and the GloU loss becomes smaller and smaller until the model converges. The above process is repeated once for each iteration of the network, and the model is saved once. After the training iteration is completed, the test accuracy in each model is compared. The model with the highest accuracy is taken as the optimal model.

The classification loss adopts the binary cross entropy loss (BCE loss). The most commonly used calculation indicator in the bounding box regression loss is the intersection-over-union ratio (IoU). A is the predicted frame area, and B is the real frame area. The formula is as follows

Various losses jointly promote the update of network weight parameters. In other words, the update of weight parameters is determined by loss backpropagation. After all kinds of loss calculation iterations, the trained model is obtained.

The parameters of the training process in the embodiment of the present invention are exemplarily set as follows: the number of iterations is 300, the batch size is 8, the polynomial decay learning rate scheduling strategy is adopted, and the initial learning rate is 0.1.

Step 103: Input the image of the seedling to be tested into the target detection model to obtain the seedling detection result.

Input the seedling image to be predicted, and obtain the predicted Cls (category information), Angle (angle information) and Reg (regression frame information) based on the target detection model, and map these information to the input seedling image to obtain the prediction result.

The embodiment of the present invention also provides a seedling rotation frame target detection system, the system includes:

The labeling module is used to label multiple historical seedling sample images, obtain the label of each historical seedling sample image, and construct a sample set including the historical seedling image and the label of the historical seedling image;

The label of the historical seedling sample image is (c,x,y,l,s,CSL(θ));

Among them, c is the category of the seedlings in the historical seedling sample image; x, y represent the coordinates of the center point of the marked box of the seedlings in the historical seedling sample image; l and s represent the long and short sides of the marked box, respectively ; θ represents the clockwise angle between the long side of the labeled box and the horizontal axis, CSL(θ) represents the angle category when the clockwise angle between the long side of the labeled box and the horizontal axis is θ;

Among them, g(·) is the window function, r is the radius of the window function, and θ′ is the angle category parameter.

The model training module is used to train the improved YOLOv5 model using the sample set, and obtain the improved YOLOv5 model after training as a target detection model; the improved YOLOv5 model is to replace the convolution in the CBL structure in the YOLOv5 model with The variable convolution is obtained by replacing the head network in the YOLOv5 model with the decoupling head network.

The improved YOLOv5 model includes a sequentially connected backbone network, a neck network, and a decoupled head network.

The uniform backbone network includes a Focus structure, a CSP1_1 structure, a first CSP1_3 structure, a second CSP1_3 structure and an SPP structure connected in sequence; the CSP structure includes a first convolution layer, a first normalization layer and a first Leaky_relu Activation function, the first CSP1_3 structure and the second CSP1_3 structure both include an improved CBL module, a residual module, a second convolution layer and a Concate layer, and the improved CBL module includes a variable convolution layer, a second normalization layer. Unification layer and second Leaky_relu activation function.

The neck network includes three downsampling layers and three upsampling layers.

The decoupling head network includes a third convolution layer, a decoupling branch and a classification branch connected in parallel with the third convolution layer; the decoupling branch includes a fourth convolution layer, a regression box loss function and a bounding box loss, the classification branch includes a fully connected layer, a class loss function, and an angle loss.

Wherein, the function expression of the variable convolution layer is:

Among them, y(p ₀ ) represents the feature value at p ₀ output by the variable convolution layer, p ₀ is any position in the feature map of the input variable convolution layer, and p _n is the grid R centered at p ₀ The nth sampling position in the grid R is the 3×3 convolution kernel, Δp _n is the offset of p _n , and x(p ₀ +p _n +Δp _n ) represents the feature of the input variable convolutional layer Eigenvalues at p ₀ +p _n + _Δpn in the graph.

The detection module is used to input the image of the seedling to be tested into the target detection model to obtain the seedling detection result.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The invention discloses a method and a system for detecting a target of a rotating frame of seedlings. The method of the present invention proposes an improved YOLOv5 model based on a decoupling structure for target detection of a rotating frame of seedlings based on YOLOv5. The improved YOLOv5 model of the present invention introduces deformable convolution into the backbone network to solve the problem of irregular edges of the detected target, improves the detection accuracy, decouples the original detection task into multiple sub-tasks, and designs each sub-task specially To learn the optimal features of the corresponding task, and then to detect the classification, position, size and direction of the inspected object, compared with the current square frame, the specific angle, position and size of the seedling can be accurately detected, using the advantages of the method of the present invention It can greatly improve the registration accuracy and registration efficiency.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part can be referred to the description of the method.

The principles and implementations of the present invention are described herein using specific examples. The descriptions of the above embodiments are only used to help understand the method and the core idea of the present invention; meanwhile, for those skilled in the art, according to the present invention There will be changes in the specific implementation and application scope. In conclusion, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. A seedling rotating frame target detection method is characterized by comprising the following steps:

labeling the plurality of historical seedling sample images to obtain a label of each historical seedling sample image, and constructing a sample set comprising the historical seedling images and the labels of the historical seedling images;

training an improved YOLOv5 model by using the sample set to obtain a trained improved YOLOv5 model as a target detection model; the improved Yolov5 model is obtained by replacing the convolution in the CBL structure in the Yolov5 model with a variability convolution and replacing the head network in the Yolov5 model with a decoupling head network;

and inputting the seedling image to be detected into the target detection model to obtain a seedling detection result.

2. The seedling rotating frame target detection method of claim 1, wherein the label of the historical seedling sample image is (c, x, y, l, s, CSL (θ));

wherein c is the category of the seedlings in the historical seedling sample image; (x, y) represents the coordinates of the central point of the labeled box of the seedling in the historical seedling sample image; l and s respectively represent the length change and the short side of the marked box; theta represents a clockwise included angle between the long side of the labeling square frame and the horizontal axis, and CSL (theta) represents an angle category when the clockwise included angle between the long side of the labeling square frame and the horizontal axis is theta;

wherein g (-) is a window function, r is a radius of the window function, and θ' is an angle class parameter.

3. The seedling rotating frame target detection method of claim 1, wherein the improved YOLOv5 model comprises a backbone network, a neck network and a decoupling head network connected in sequence;

the homogeneous backbone network comprises a Focus structure, a CSP1_1 structure, a first CSP1_3 structure, a second CSP1_3 structure and an SPP structure which are connected in sequence; the CSP structure comprises a first convolution layer, a first normalization layer and a first Leaky _ relu activation function, the first CSP1_3 structure and the second CSP1_3 structure respectively comprise an improved CBL module, a residual module, a second convolution layer and a Concate layer, and the improved CBL module comprises a variable convolution layer, a second normalization layer and a second Leaky _ relu activation function;

the neck network comprises three down-sampling layers and three up-sampling layers;

the decoupling head network comprises a third convolution layer, a decoupling branch and a classification branch which are connected with the third convolution layer in parallel; the decoupling branch comprises a fourth convolution layer, a regression frame loss function and a frame loss, and the classifying branch comprises a full connection layer, a category loss function and an angle loss.

4. A seedling rotating frame target detection method according to claim 3, wherein the function expression of the variable convolution layer is:

wherein, y (p)₀) P representing variable convolutional layer output₀Characteristic value of (a), p₀Is input at an arbitrary position, p, in the feature map of the variable convolution layer_nIs p₀The nth sample position in a centered grid R of 3 x 3 convolution kernels, Δ p_nIs p_nOffset of (2), x (p)₀+p_n+Δp_n) P in a feature diagram representing an input variable convolution layer₀+p_n+Δp_nCharacteristic value of (a), w (p)_n) P in a feature diagram representing an input variable convolution layer₀+p_n+Δp_nThe weight of the characteristic value of (a).

5. The seedling rotating frame target detection method of claim 1, wherein a total loss function adopted in the process of training the improved YOLOv5 model by using the sample set is as follows:

wherein L represents the total loss function, N represents the number of anchor points, i represents the ith anchor point, obj_iRepresenting a binary value, L, for distinguishing the foreground from the background_reg、L_CSL、L_cls、L_iouRespectively representing a regression loss function, an angle loss function, a category loss function and a frame loss function, (x, y) representing the coordinate of the central point of a labeled square frame of seedlings in a historical seedling sample image, l and s respectively representing the length change and the short side of the labeled square frame, theta represents the clockwise included angle between the long side of the labeled square frame and a horizontal axis, CSL (theta) represents the angle category when the clockwise included angle between the long side of the labeled square frame and the horizontal axis is theta, and lambda (theta) represents the angle category when the clockwise included angle between the long side of the labeled square frame and the horizontal axis is theta₁、λ₂、λ₃And λ₄Weights representing a regression loss function, an angle loss function, a class loss function, and a bounding box loss function, respectively.

6. A seedling rotating frame target detection method according to claim 5, wherein the regression loss function is:

wherein a represents the difference value between the prediction block and the labeling block,

the function of the smoothing is represented by a smooth function,

the angle loss function is:

where M represents the total number of angle categories, θ n is the angle category, y_iθnFor the angle sign function of the ith anchor point, if the angle type is true, 1 is taken, otherwise, 0 and P are taken_iθnThe probability that the angle class of the ith anchor point belongs to the angle class thetan;

the class loss function is:

wherein C represents the number of seedling categories,

the seedling category representing the ith anchor point belongs to the seedling category c_nThe probability of (a) of (b) being,

representing the ith anchor point category function;

the frame loss function is:

where IoU denotes the intersection ratio of the prediction box and the annotation box,

c denotes the minimum box covered between the predicted box and the labeled box, B is the predicted box, B^gtThe boxes are labeled.

7. A seedling rotating frame target detection system, the system comprising:

the labeling module is used for labeling the plurality of historical seedling sample images, obtaining a label of each historical seedling sample image, and constructing a sample set containing the historical seedling images and the labels of the historical seedling images;

the model training module is used for training the improved YOLOv5 model by using the sample set to obtain a trained improved YOLOv5 model as a target detection model; the improved Yolov5 model is obtained by replacing the convolution in the CBL structure in the Yolov5 model with a variability convolution and replacing the head network in the Yolov5 model with a decoupling head network;

and the detection module is used for inputting the seedling image to be detected into the target detection model to obtain a seedling detection result.

8. The seedling rotating frame target detecting system of claim 7, wherein the label of the historical seedling sample image is (c, x, y, l, s, CSL (θ));

wherein c is the category of the seedlings in the historical seedling sample image; x and y represent the coordinates of the central point of the marked box of the seedling in the historical seedling sample image; l and s respectively represent the length change and the short side of the marked box; theta represents a clockwise included angle between the long side of the labeling square frame and the horizontal axis, and CSL (theta) represents an angle category when the clockwise included angle between the long side of the labeling square frame and the horizontal axis is theta;

wherein g (-) is a window function, r is a radius of the window function, and θ' is an angle class parameter.

9. The seedling rotation frame target detection system of claim 7, wherein the improved YOLOv5 model comprises a backbone network, a neck network, and a decoupling head network connected in sequence;

the homogeneous backbone network comprises a Focus structure, a CSP1_1 structure, a first CSP1_3 structure, a second CSP1_3 structure and an SPP structure which are connected in sequence; the CSP structure comprises a first convolution layer, a first normalization layer and a first Leaky _ relu activation function, the first CSP1_3 structure and the second CSP1_3 structure respectively comprise an improved CBL module, a residual module, a second convolution layer and a Concate layer, and the improved CBL module comprises a variable convolution layer, a second normalization layer and a second Leaky _ relu activation function;

the neck network comprises three down-sampling layers and three up-sampling layers;

the decoupling head network comprises a third convolution layer, a decoupling branch and a classification branch which are connected with the third convolution layer in parallel; the decoupling branch comprises a fourth convolution layer, a regression frame loss function and a frame loss, and the classifying branch comprises a full connection layer, a category loss function and an angle loss.

10. A seedling rotation frame target detection system as claimed in claim 9, wherein the function expression of the variable convolution layer is:

wherein, y (p)₀) P representing variable convolutional layer output₀Characteristic value of (a), p₀Is input at an arbitrary position, p, in the feature map of the variable convolution layer_nIs p₀The nth sample position in a centered grid R of 3 x 3 convolution kernels, Δ p_nIs p_nOffset of (2), x (p)₀+p_n+Δp_n) P in a feature diagram representing an input variable convolution layer₀+p_n+Δp_nThe characteristic value of (c).

Images