CN116524351A - Lightweight method and system for rotating target detection based on knowledge distillation - Google Patents

Abstract

The invention relates to a method and a system for detecting a rotation target in light weight based on knowledge distillation, wherein the method comprises the steps of respectively decoding coordinate codes of the detection target output by a student model and a trained teacher model into rotation coordinate forms; converting the rotation coordinates into two-dimensional Gaussian distribution; calculating the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model, and calculating the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the tag real frame; carrying out normalization treatment on KL divergence; calculating to obtain an overall loss function in the process that the teacher model carries out knowledge distillation on the student model according to the normalized KL divergence; prediction was performed using a student model after distillation. The invention provides more accurate positioning information for training of the lightweight network and improves the detection performance of the lightweight network on the optical remote sensing image.

Description

Lightweight method and system for rotating target detection based on knowledge distillation

Technical Field

The present invention relates to the technical field of high-resolution remote sensing image information processing, and in particular to a lightweight method and system for rotating target detection based on knowledge distillation.

Background Art

Object detection in remote sensing images aims to locate objects of interest (such as vehicles, aircraft, and ships) on the surface of the earth and predict their categories. Remote sensing images are mostly viewed from above, and the spatial scenes they contain are larger and more complex. The distribution of objects of interest is uneven, and there are sparse and dense distributions. Remote sensing target detection has the difficulties of small targets, dense distribution, and arbitrary directions. When the length-to-width ratio of the target of interest is large, there is an inclination, and it is closely arranged, only using a horizontal detection frame, the size and length-to-width ratio cannot truly reflect the target object, the target and background pixels cannot be effectively separated, and dense targets are difficult to separate. Each detection frame will contain adjacent targets, and the intersection-union ratio between horizontal detection frames is high, which is easily suppressed in the non-maximum suppression process, resulting in low final detection accuracy. Due to the periodic characteristics of the target orientation, that is, the rotation angle, the design of a suitable loss function is crucial for model training optimization.

Traditional target detection methods often require complex manual feature extraction and fine parameter adjustment, and the generalization of the model is poor and cannot adapt to the ever-changing environment. In recent years, with the improvement of hardware computing power, the development of remote sensing imaging technology and the convenience of remote sensing image resources, deep learning technology has been widely developed, especially the convolutional neural network model, which has strong generalization performance due to its powerful robust feature extraction ability, function fitting ability and end-to-end network model design. Based on the convolutional neural network model, most of the most advanced target detection methods focus on designing advanced network structures or loss functions to improve detection performance, but this method has expensive computational costs. Therefore, some subsequent methods are committed to improving detection performance by using lightweight methods to distill the knowledge in the (teacher) large model. For example, the knowledge distillation method based on the intermediate layer features has been proven to be important in the training and prediction process of the detection network, but if only the intermediate layer features are distilled, the information of the coordinate space and the category space will be lost, which will affect the final detection results; there is also knowledge distillation based on generalized distribution information, which uses classification loss to solve regression problems, including rotating the target position and rotating the target angle, and solving the problem of positioning uncertainty, but it has quantization errors. Converting the rotated coordinates into Gaussian distribution representation has been widely used in the field of rotation detection in recent years, but there is little research on the lightweight method of detectors. Therefore, it is urgent to propose a lightweight method for rotating target detection based on knowledge distillation to overcome the above technical defects of the existing technology.

Summary of the invention

To this end, the technical problem to be solved by the present invention is to overcome the technical defects existing in the prior art, and propose a lightweight method and system for rotation target detection based on knowledge distillation, which uses the coordinate encoding output of the teacher and student models for knowledge distillation, calculates the KL divergence between the Gaussian distribution representations of the rotation coordinates as the added distillation loss to train the student model parameters, provides more accurate positioning information for the training of the lightweight network, and improves the detection performance of the lightweight network on optical remote sensing images.

To solve the above technical problems, the present invention provides a lightweight method for rotating target detection based on knowledge distillation, comprising the following steps:

S1: Decoding the coordinate encodings of the detection target output by the student model and the trained teacher model into rotation coordinate forms respectively to obtain the rotation coordinate representation of the detection target;

S2: converting the rotation coordinate representation into a two-dimensional Gaussian distribution to obtain a Gaussian distribution representation of the detection target;

S3: Calculate the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model to obtain a first KL divergence, and calculate the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame to obtain a second KL divergence;

S4: normalizing the first KL divergence and the second KL divergence;

S5: According to the normalized first KL divergence and the second KL divergence, the overall loss function of the teacher model in the process of knowledge distillation of the student model is calculated;

S6: Prediction using the distilled student model.

In one embodiment of the present invention, in step S2, the method of respectively decoding the coordinate encodings of the detection target output by the student model and the trained teacher model into the form of rotation coordinates includes:

Decode the coordinate encoding Y of the detected target output by the student model and the trained teacher model: (dx, dy, dw, dh, dθ) into a rotated coordinate form (x, y, w, h, θ), where:

x＝ _xa +dx* _wa

y＝ _ya +dy* _ha

w＝wa*e ^dw

h＝ _ha * ^edh

θ＝ _θa +dθ

Where A:( _xa , _ya , _wa , _ha , _θa ) is the anchor box preset by the model.

In one embodiment of the present invention, in step S3, the method of converting the rotated coordinate representation into a two-dimensional Gaussian distribution includes:

Represent the rotated coordinates (x, y, w, h, θ) is converted into a two-dimensional Gaussian distribution The calculation formula of (μ, ∑) is:

μ＝(x，y) ^T

Where (x, y, w, h, θ) is the rotation coordinate representation of the detection target, which respectively represents the horizontal coordinate, vertical coordinate, width, height and rotation angle of the detection target center point, and (μ, ∑) represents the mean and covariance matrix of the Gaussian distribution.

In one embodiment of the present invention, in step S4, the method of calculating the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model includes:

The calculation formula of the KL divergence represented by the Gaussian distribution of the detection target output by the student model and the teacher model is:

In the formula, (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _T , y _T , w _T , h _T , θ _T ), (μ _T , ∑ _T ) represents the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the teacher model, respectively. Δx＝x _S -x _T , Δy＝y _S -y _T , Δθ＝θ _S -θ _T represent the difference in horizontal and vertical coordinates and the difference in rotation angle between the center point of the detection target output by the student model and the label true frame, respectively.

In one embodiment of the present invention, in step S4, a method for calculating the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame includes:

The calculation formula for the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label real box is:

In the formula (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _G , y _G , w _G , h _G , θ _G ), (μ _G , ∑ _G ) represent the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the detection target obtained by the label true frame, respectively. Δx＝x _S -x _G , Δy＝y _S -y _G , Δθ＝θ _S -θ _G represent the difference in the horizontal and vertical coordinates and the difference in rotation angle of the center point of the detection target output by the student model and the detection target obtained by the label true frame, respectively.

In one embodiment of the present invention, in step S7, after prediction is performed using the distilled student model, the prediction box output by the student model is post-processed.

In one embodiment of the present invention, in step S7, the method for post-processing the prediction box output by the student model includes: the specific steps of post-processing are as follows:

Arrange all prediction boxes in each category in ascending or descending order by score;

Select the prediction box with the highest score as the benchmark, and calculate the intersection and union ratio of other prediction boxes with it;

The prediction boxes in each category are screened according to the intersection-and-union ratio, and the prediction boxes that do not meet the screening conditions are deleted, where the screening condition is that the intersection-and-union ratio of the prediction box to be screened and the reference prediction box is less than or equal to a preset threshold.

Among the remaining prediction boxes that meet the screening conditions, select the prediction box with the highest score as the benchmark box, and repeat the above screening steps until all prediction boxes that meet the screening conditions are used as benchmarks for screening.

In addition, the present invention also provides a lightweight system for rotating target detection based on knowledge distillation, comprising the following steps:

A coordinate decoding module, which is used to decode the coordinate encoding of the detection target output by the student model and the trained teacher model into a rotation coordinate form, so as to obtain a rotation coordinate representation of the detection target;

A coordinate conversion module, which is used to convert the rotation coordinate representation into a two-dimensional Gaussian distribution to obtain a Gaussian distribution representation of the detection target;

A KL divergence calculation module is used to calculate the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model to obtain a first KL divergence, and calculate the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame to obtain a second KL divergence;

A normalization processing module, used for performing normalization processing on the first KL divergence and the second KL divergence;

A loss function calculation module, which is used to calculate the overall loss function in the process of knowledge distillation of the teacher model to the student model according to the first KL divergence and the second KL divergence after normalization;

The prediction module is used to make predictions using the distilled student model.

In one embodiment of the present invention, in a coordinate conversion module, a method for converting the rotation coordinate representation into a two-dimensional Gaussian distribution includes:

Represent the rotated coordinates (x, y, w, h, θ) is converted into a two-dimensional Gaussian distribution The calculation formula of (μ, ∑) is:

μ＝(x，y) ^T

Where (x, y, w, h, θ) is the rotation coordinate representation of the detection target, which respectively represents the horizontal coordinate, vertical coordinate, width, height and rotation angle of the detection target center point, and (μ, ∑) represents the mean and covariance matrix of the Gaussian distribution.

In one embodiment of the present invention, in the KL divergence calculation module, a method for calculating the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model includes:

The calculation formula of the KL divergence represented by the Gaussian distribution of the detection target output by the student model and the teacher model is:

In the formula (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _T , y _T , w _T , h _T , θ _T ), (μ _T , ∑ _T ) represents the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the teacher model, respectively. Δx＝x _S -x _T , Δy＝y _S -y _T , Δθ＝θ _S -θ _T represent the difference in horizontal and vertical coordinates and the difference in rotation angle between the center point of the detection target output by the student model and the label true frame, respectively.

The above technical solution of the present invention has the following advantages compared with the prior art:

The present invention discloses a lightweight method and system for rotating target detection based on knowledge distillation, which utilizes the coordinate encoding outputs of teacher and student models for knowledge distillation, calculates the KL divergence between the Gaussian distribution representations of the rotating coordinates as the added distillation loss to train the student model parameters, provides more accurate positioning information for the training of the lightweight network, and improves the detection performance of the lightweight network on optical remote sensing images.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein

FIG1 is a diagram showing the training process of the lightweight method for rotating target detection based on knowledge distillation proposed in the present invention.

FIG2 is a prediction flow chart of the lightweight method for rotating target detection based on knowledge distillation proposed in the present invention.

Figure 3 shows the curve of the detection accuracy of the distillation method proposed in the present invention and other distillation methods based on ResNet18-FPN-RetinaNet on the HRSC2016 dataset as the number of training rounds increases after 20 rounds, where w/o-d is the original network without distillation; FitNets is a distillation method based on all intermediate layer features; DeFeat is a distillation method based on decoupled features; LD is a distillation method based on generalized distribution information between coordinates; KLDD is a distillation method based on the KL divergence between Gaussian distributions of coordinates proposed in the present invention.

Figure 4 shows the curve of the detection accuracy of the distillation method proposed in the present invention and other distillation methods based on MobileNetv2-FPN-RetinaNet on the HRSC2016 dataset as the number of training rounds increases after 20 rounds, where w/o-d is the original network without distillation; FitNets is a distillation method based on all intermediate layer features; DeFeat is a distillation method based on decoupled features; LD is a distillation method based on generalized distribution information between coordinates; KLDD is a distillation method based on the KL divergence between Gaussian distributions of coordinates proposed in the present invention.

Figure 5 shows the comparison of the detection results obtained by different distillation methods on the HRSC2016 dataset, where Figure (5a) is the original annotation of the dataset, Figure (5b) is the detection result of the original ResNet18-FPN-RetinaNet detector without using any distillation method, Figure (5c) is the detection result of the FitNets distillation method based on all intermediate layer features, Figure (5d) is the detection result of the DeFeat distillation method based on the decoupled features of the target area, Figure (5e) is the detection result of the LD distillation method based on the generalized distribution information between the target position coordinates, and Figure (5f) is the detection result of the proposed KLDD distillation method based on the KL divergence between the coordinate Gaussian distributions. The cyan box in the figure indicates the detected true target (TP), the yellow box indicates the detected false target (FP), and the red box indicates the missed detection target (FN).

DETAILED DESCRIPTION

The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

The embodiment of the present invention provides a lightweight method for rotating target detection based on knowledge distillation, comprising the following steps:

S1: Decoding the coordinate codes of the detection target output by the student model and the teacher model into rotation coordinate forms respectively to obtain the rotation coordinate representation of the detection target;

S2: converting the rotation coordinate representation into a two-dimensional Gaussian distribution to obtain a Gaussian distribution representation of the detection target;

S3: Calculate the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model to obtain a first KL divergence, and calculate the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame to obtain a second KL divergence;

S4: normalizing the first KL divergence and the second KL divergence;

S5: According to the normalized first KL divergence and the second KL divergence, the overall loss function of the teacher model in the process of knowledge distillation of the student model is calculated;

S6: Prediction using the distilled student model.

The present invention discloses a lightweight method for rotating target detection based on knowledge distillation, which utilizes the coordinate encoding outputs of teacher and student models for knowledge distillation, calculates the KL divergence between the Gaussian distribution representations of the rotating coordinates as the added distillation loss to train the student model parameters, provides more accurate positioning information for the training of the lightweight network, and improves the detection performance of the lightweight network on optical remote sensing images.

As shown in FIG. 1 and FIG. 2 , the above-mentioned lightweight method for rotating target detection based on knowledge distillation mainly includes the following three parts:

(1) Train a deep convolutional neural network with strong detection accuracy as the teacher model:

This part aims to generate a teacher model that provides additional supervision information for student model training. This experiment chooses ResNet50-FPN as the feature extraction backbone network and RetinaNet as the teacher model of the detection network.

(2) KLDD distillation method based on KL divergence between coordinate Gaussian distributions:

This part aims to generate a two-dimensional Gaussian distribution representation of the rotated coordinates and calculate the KL divergence between Gaussian distributions. The specific steps include: coordinate decoding, Gaussian distribution conversion, and loss calculation of the KL divergence between Gaussian distributions.

Among them, the specific coordinate decoding calculation formula is:

x＝ _xa +dx* _wa

y＝ _ya +dy* _ha

w＝ _wa * ^edw

h＝ _ha * ^edh

θ＝ _θa +dθ

Get the decoded rotation coordinate representation (x, y, w, h, θ), where Y: (dx, dy, dw, dh, dθ) is the coordinate encoding output by the model, and A: ( _xa , _ya , _wa , _ha , _θa ) is the anchor box preset by the model.

The rotation coordinates are expressed as (x, y, w, h, θ) is converted into a two-dimensional Gaussian distribution The specific calculation formula of (μ, ∑) is as follows:

μ＝(x，y) ^T

Where (x, y, w, h, θ) is the rotation coordinate representation of the detection target, which respectively represents the horizontal coordinate, vertical coordinate, width, height and rotation angle of the detection target center point, and (μ, ∑) represents the mean and covariance matrix of the Gaussian distribution.

Among them, the calculation formula of the KL divergence represented by the Gaussian distribution of the detection target output by the student model S and the teacher model T is:

In the formula, (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _T , y _T , w _T , h _T , θ _T ), (μ _T , ∑ _T ) represents the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the teacher model, respectively. Δx＝x _S -x _T , Δy＝y _S -y _T , Δθ＝θ _S -θ _T represent the difference in horizontal and vertical coordinates and the difference in rotation angle between the center point of the detection target output by the student model and the label true frame, respectively.

Among them, the calculation formula of the KL divergence between the Gaussian distribution representation of the detection target output by the student model S and the Gaussian distribution representation of the detection target of the label real box G is:

In the formula (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _G , y _G , w _G , h _G , θ _G ), (μ _G , ∑ _G ) represent the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the detection target obtained by the label true frame, respectively. Δx＝x _S -x _G , Δy＝y _S -y _G , Δθ＝θ _S -θ _G represent the difference in the horizontal and vertical coordinates and the difference in rotation angle of the center point of the detection target output by the student model and the detection target obtained by the label true frame, respectively.

Normalize the KL divergence between Gaussian distributions F(KL):

Get the final KL divergence loss calculation formula between the coordinate parameter Gaussian distribution:

During training, the overall loss function of the model is for:

Where λ ₀ , λ ₁ , and λ ₂ are weight parameters, which are set to 1, 15, and 5.5 respectively; is the positive sample area mask, which is determined by the intersection-over-union ratio of the anchor box and the true target box; _CS , _CT , _CG are the category confidence output by the student and teacher models and the unique hot encoding of the true label category respectively; The rotated coordinates and true label coordinates output by the student and teacher models respectively; is the cross entropy loss; is the KL divergence loss between category distributions; and is the normalized KL divergence between the student-truth and student-teacher Gaussian distributions.

(3) In the prediction stage, only the distilled student model is used for prediction.

This part aims to post-process the prediction box output by the network, that is, non-maximum suppression, to avoid repeated detection of the same target. The specific steps of post-processing are as follows:

Arrange all prediction boxes in each category in ascending or descending order by score;

Select the prediction box with the highest score as the benchmark, and calculate the intersection and union ratio of other prediction boxes with it;

The prediction boxes in each category are screened according to the intersection-and-union ratio, and the prediction boxes that do not meet the screening conditions are deleted, where the screening condition is that the intersection-and-union ratio of the prediction box to be screened and the reference prediction box is less than or equal to a preset threshold.

Among the remaining prediction boxes that meet the screening conditions, select the prediction box with the highest score as the benchmark box, and repeat the above screening steps until all prediction boxes that meet the screening conditions are used as benchmarks for screening.

The beneficial effects of a lightweight method for rotating target detection based on knowledge distillation provided by the present invention are explained below in an experimental verification manner.

Example:

1. Experimental data

In order to verify the effectiveness of the lightweight detection model method proposed in this paper, this experiment used two optical remote sensing rotating target detection datasets: DOTA and HRSC2016.

1) DOTA dataset:

DOTA is one of the largest datasets for rotating object detection in aerial images. It contains 2806 aerial images with image sizes ranging from 800×800 to 4000×4000 pixels. It contains more than 188,000 objects of various scales, orientations, and shapes, marked with arbitrary quadrilaterals, and contains 15 object categories: airplanes, baseball fields, bridges, ground tracks, small vehicles, large vehicles, ships, tennis courts, basketball courts, storage tanks, football fields, roundabouts, ports, swimming pools, and helicopters. The training set, validation set, and test set are divided into 1/2, 1/6, and 1/3 ratios. Due to the large size of the image, it is cropped into 1024×1024 pixel sub-images with an overlap of 200 pixels between sub-images.

2) HRSC2016 dataset:

HRSC2016 is a single-category dataset for ship detection. Its images come from 6 well-known ports and include two scenes: ships at sea and near the coast. The spatial resolution of the images is 0.4-2m. The dataset has a total of 1070 images and 2976 ship targets. The training set, validation set, and test set include 436, 181, and 453 images, respectively. Due to the different sizes of the images, they are scaled to 800×512 pixels. Figure (5a) shows some of the images in the dataset and the target box annotations.

2. Experimental results

This experiment is built based on the MMRotate environment and is conducted on a single NVIDIA RTX3090 GPU. This experiment selects the RetinaNet network with ResNet50-FPN as the backbone as the teacher model, and the student models select the RetinaNet network with ResNet18-FPN and MobileNetV2-FPN as the backbone. In the training phase, random horizontal, vertical, and diagonal flipping are used as data augmentation methods, and the standard momentum optimizer (Momentum) is selected. The weight decay and momentum are 0.0001 and 0.9 respectively. The initial learning rate is set to 0.0025. For the DOTA dataset, a total of 24 epochs are trained, and for the HRSC2016 dataset, a total of 72 epochs are trained, and the batch size is 2. In the test phase, mAP ₅₀ (the average precision when the intersection-over-union ratio threshold for determining the true target is 0.5) is used as an indicator to evaluate the accuracy of the detection model.

1) DOTA dataset:

Tables 1 and 2 show the experimental results obtained using different comparative distillation methods, including the benchmark teacher model ResNet50-FPN, the benchmark student model ResNet18-FPN, and the benchmark student model MobileNetv2-FPN. From the results in the table, it can be seen that the overall accuracy of the proposed method (KLDD) is improved from 65.2% and 56.8% to 68.1% and 61.0% respectively compared with the benchmark student models (ResNet18-FPN and MobileNetv2-FPN), and it is better than the detection results of the current mainstream distillation methods (FitNets, DeFeat, GIImitation, and LD, etc.), with the overall accuracy increased by 4.45% and 7.39% respectively.

Table 3 compares the accuracy (mAP ₅₀ ), floating-point operations (FLOPS), and model parameters (Params) of the two lightweight models ResNet18-FPN-RetinaNet and MobileNetv2-FPN-RetinaNet and the teacher model ResNet50-FPN-RetinaNet used in this experiment. The image size of the input network is set to 3x1024x1024.

Table 1 Experimental results of different distillation methods on the DOTA dataset

Table 2 Experimental results of different distillation methods on the DOTA dataset

Table 3 Experimental results before and after using the KLDD distillation method on the DOTA dataset

2) HRSC2016 dataset:

Tables 4 and 5 show the experimental results obtained using different comparative distillation methods, including the benchmark teacher model ResNet50-FPN, the benchmark student model ResNet18-FPN, and the benchmark student model MobileNetv2-FPN. From the results in the table, it can be seen that the overall accuracy of the proposed method (KLDD) is improved from 86.0% and 79.2% to 89.2% and 80.7% respectively compared with the benchmark student models (ResNet18-FPN and MobileNetv2-FPN), and it is better than the detection results of the current mainstream distillation methods (FitNets, DeFeat, GIImitation, and LD, etc.), with the overall accuracy increased by 3.72% and 1.89% respectively.

Table 6 compares the accuracy (mAP ₅₀ ), floating-point operations (FLOPS), and model parameters (Params) of the two lightweight models ResNet18-FPN-RetinaNet and MobileNetv2-FPN-RetinaNet and the teacher model ResNet50-FPN-RetinaNet used in this experiment. The image size of the input network is set to 3x1024x1024.

Table 4 Experimental results of different distillation methods on the HRSC2016 dataset

Table 5 Experimental results of different distillation methods on the HRSC2016 dataset

Table 6 Experimental results before and after using the KLDD distillation method on the HRSC2016 dataset

It can be seen intuitively from Figure 5 that the proposed method is superior to other distillation methods. For the first image, although the proposed method also has false alarms, it is still better than other distillation methods overall; for the second image, the proposed method correctly detects all ship targets, while the DeFeat distillation method based on decoupled features and the LD distillation method based on generalized distribution information between coordinates have false alarms and missed detections; for the third image, the proposed method correctly detects all ship targets, while other distillation methods generally have false alarms and missed detections.

A lightweight system for rotating target detection based on knowledge distillation disclosed in an embodiment of the present invention is introduced below. The lightweight system for rotating target detection based on knowledge distillation described below and the lightweight method for rotating target detection based on knowledge distillation described above can refer to each other.

The present invention also provides a lightweight system for rotating target detection based on knowledge distillation, comprising the following steps:

A coordinate decoding module, which is used to decode the coordinate encoding of the detection target output by the student model and the trained teacher model into a rotation coordinate form, so as to obtain a rotation coordinate representation of the detection target;

A coordinate conversion module, which is used to convert the rotation coordinate representation into a two-dimensional Gaussian distribution to obtain a Gaussian distribution representation of the detection target;

A KL divergence calculation module is used to calculate the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model to obtain a first KL divergence, and calculate the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame to obtain a second KL divergence;

A normalization processing module, used for performing normalization processing on the first KL divergence and the second KL divergence;

A loss function calculation module, which is used to calculate the overall loss function in the process of knowledge distillation of the teacher model to the student model according to the first KL divergence and the second KL divergence after normalization;

The prediction module is used to make predictions using the distilled student model.

In one embodiment of the present invention, in a coordinate conversion module, a method for converting the rotation coordinate representation into a two-dimensional Gaussian distribution includes:

Represent the rotated coordinates (x, y, w, h, θ) is converted into a two-dimensional Gaussian distribution The calculation formula of (μ, ∑) is:

μ＝(x，y) ^T

Where (x, y, w, h, θ) is the rotation coordinate representation of the detection target, which respectively represents the horizontal coordinate, vertical coordinate, width, height and rotation angle of the detection target center point, and (μ, ∑) represents the mean and covariance matrix of the Gaussian distribution.

In one embodiment of the present invention, in the KL divergence calculation module, a method for calculating the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model includes:

The calculation formula of the KL divergence represented by the Gaussian distribution of the detection target output by the student model and the teacher model is:

In the formula (x _S , y _S , w _S , h _S , θ _S ) and (x _T , y _T , w _T , h _T , θ _T ) (μ _T , ∑ _T ) represents the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the teacher model, respectively. Δx＝x _S -x _T , Δy＝y _S -y _T , Δθ＝θ _S -θ _T represent the difference in horizontal and vertical coordinates and the difference in rotation angle between the center point of the detection target output by the student model and the label true frame, respectively.

The lightweight system for rotating target detection based on knowledge distillation of this embodiment is used to implement part of the embodiment of the aforementioned lightweight method for rotating target detection based on knowledge distillation. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part and will not be introduced in detail here.

In addition, since the lightweight system for rotating target detection based on knowledge distillation of this embodiment is used to implement the aforementioned lightweight method for rotating target detection based on knowledge distillation, its function corresponds to that of the above method and will not be repeated here.

Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

The present application is described with reference to the flowchart and/or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each process and/or box in the flowchart and/or block diagram, and the combination of the process and/or box in the flowchart and/or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one process or multiple processes in the flowchart and/or one box or multiple boxes in the block diagram.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and/or one or more boxes in the block diagram.

Obviously, the above embodiments are merely examples for clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived from these are still within the protection scope of the invention.

Claims (10)

1. A lightweight method for rotating target detection based on knowledge distillation, characterized in that it includes the following steps: S1: Decode the coordinate encoding of the detection target output by the student model and the trained teacher model into the form of rotation coordinates to obtain the rotation coordinate representation of the detection target; S2: converting the rotation coordinate representation into a two-dimensional Gaussian distribution to obtain a Gaussian distribution representation of the detection target; S3: Calculate the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model to obtain a first KL divergence, and calculate the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame to obtain a second KL divergence; S4: normalizing the first KL divergence and the second KL divergence; S5: According to the normalized first KL divergence and the second KL divergence, the overall loss function of the teacher model in the process of knowledge distillation of the student model is calculated; S6: Prediction using the distilled student model. 2. According to the lightweight method for rotating target detection based on knowledge distillation of claim 1, it is characterized in that: in step S2, the method of respectively decoding the coordinate encoding of the detection target output by the student model and the trained teacher model into the form of rotating coordinates includes: Decode the coordinate encoding Y of the detected target output by the student model and the trained teacher model: (dx, dy, dw, dh, dθ) into a rotated coordinate form (x, y, w, h, θ), where: x＝ _xa +dx* _wa y＝ _ya +dy* _ha w＝ _wa * ^edw h＝ _ha * ^edh θ＝ _θa +dθ Where A:( _xa , _ya , _wa , _ha , _θa ) is the anchor box preset by the model. 3. A lightweight method for rotating target detection based on knowledge distillation according to claim 1 or 2, characterized in that: in step S3, the method of converting the rotating coordinate representation into a two-dimensional Gaussian distribution comprises: Represent the rotated coordinates (x, y, w, h, θ) is converted into a two-dimensional Gaussian distribution The calculation formula of (μ, ∑) is: μ＝(x，y) ^T Where (x, y, w, h, θ) is the rotation coordinate representation of the detection target, which respectively represents the horizontal coordinate, vertical coordinate, width, height and rotation angle of the detection target center point, and (μ, ∑) represents the mean and covariance matrix of the Gaussian distribution. 4. A lightweight method for rotating target detection based on knowledge distillation according to claim 3, characterized in that: in step S4, the method for calculating the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model comprises: The calculation formula of the KL divergence represented by the Gaussian distribution of the detection target output by the student model and the teacher model is: In the formula, (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _T , y _T , w _T , h _T , θ _T ), (μ _T , ∑ _T ) represents the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the teacher model, respectively. Δx＝x _S -x _T , Δy＝y _S -y _T , Δθ＝θ _S -θ _T represent the difference in horizontal and vertical coordinates and the difference in rotation angle between the center point of the detection target output by the student model and the label true frame, respectively. 5. A lightweight method for rotating target detection based on knowledge distillation according to claim 3, characterized in that: in step S4, the method of calculating the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame comprises: The calculation formula for the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label real box is: In the formula (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _G , y _G , w _G , h _G , θ _G ), (μ _G , ∑ _G ) represent the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the detection target obtained by the label true frame, respectively. Δx＝x _S -x _G , Δy＝y _S -y _G , Δθ＝θ _S -θ _G represent the difference in the horizontal and vertical coordinates and the difference in rotation angle of the center point of the detection target output by the student model and the detection target obtained by the label true frame, respectively. 6. According to the lightweight method for rotating target detection based on knowledge distillation in claim 1, it is characterized in that: in step S7, after using the distilled student model for prediction, the prediction box output by the student model is post-processed. 7. A lightweight method for rotating target detection based on knowledge distillation according to claim 6, characterized in that: in step S7, the method for post-processing the prediction box output by the student model comprises: the specific steps of post-processing are as follows: Arrange all prediction boxes in each category in ascending or descending order by score; Select the prediction box with the highest score as the benchmark, and calculate the intersection and union ratio of other prediction boxes with it; The prediction boxes in each category are screened according to the intersection-and-union ratio, and the prediction boxes that do not meet the screening conditions are deleted, where the screening condition is that the intersection-and-union ratio of the prediction box to be screened and the reference prediction box is less than or equal to a preset threshold. Among the remaining prediction boxes that meet the screening conditions, select the prediction box with the highest score as the benchmark box, and repeat the above screening steps until all prediction boxes that meet the screening conditions are used as benchmarks for screening. 8. A lightweight system for rotating target detection based on knowledge distillation, characterized in that it includes the following steps: A coordinate decoding module, which is used to decode the coordinate encoding of the detection target output by the student model and the trained teacher model into a rotation coordinate form, so as to obtain a rotation coordinate representation of the detection target; A coordinate conversion module, which is used to convert the rotation coordinate representation into a two-dimensional Gaussian distribution to obtain a Gaussian distribution representation of the detection target; A KL divergence calculation module is used to calculate the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model to obtain a first KL divergence, and calculate the KL divergence between the Gaussian distribution representation of the detection target output by the student model and the Gaussian distribution representation of the detection target obtained by the label true frame to obtain a second KL divergence; A normalization processing module, used for performing normalization processing on the first KL divergence and the second KL divergence; A loss function calculation module, which is used to calculate the overall loss function of the teacher model in the process of knowledge distillation of the student model according to the first KL divergence and the second KL divergence after normalization; The prediction module is used to make predictions using the distilled student model. 9. A lightweight system for rotating target detection based on knowledge distillation according to claim 8, characterized in that: in the coordinate conversion module, the method for converting the rotating coordinate representation into a two-dimensional Gaussian distribution comprises: Represent the rotated coordinates (x, y, w, h, θ) is converted into a two-dimensional Gaussian distribution The calculation formula of (μ, ∑) is: μ＝(x，y) ^T Where (x, y, w, h, θ) is the rotation coordinate representation of the detection target, which respectively represents the horizontal coordinate, vertical coordinate, width, height and rotation angle of the detection target center point, and (μ, ∑) represents the mean and covariance matrix of the Gaussian distribution. 10. A lightweight system for rotating target detection based on knowledge distillation according to claim 8, characterized in that: in the KL divergence calculation module, a method for calculating the KL divergence of the Gaussian distribution representation of the detection target output by the student model and the teacher model comprises: The calculation formula of the KL divergence represented by the Gaussian distribution of the detection target output by the student model and the teacher model is: In the formula (x _S , y _S , w _S , h _S , θ _S ), (μ _S , ∑ _S ) and (x _T , y _T , w _T , h _T , θ _T ), (μ _T , ∑ _T ) represents the rotation coordinate representation and Gaussian distribution representation of the detection target output by the student model and the teacher model, respectively. Δx＝s _S -x _T , Δy＝y _S -y _T , Δθ＝θ _S -θ _T represent the difference in horizontal and vertical coordinates and the difference in rotation angle between the center point of the detection target output by the student model and the label true frame, respectively.