WO2022021029A1

WO2022021029A1 - Detection model training method and device, detection model using method and storage medium

Info

Publication number: WO2022021029A1
Application number: PCT/CN2020/104973
Authority: WO
Inventors: 张雪; 席迎来
Original assignee: 深圳市大疆创新科技有限公司
Priority date: 2020-07-27
Filing date: 2020-07-27
Publication date: 2022-02-03
Also published as: CN113490947A

Abstract

A detection model training method and device, a detection model using method and a storage medium. The method comprises: performing feature extraction on a sample image by means of a first detection model to obtain first feature information, and performing feature extraction on the sample image by means of a trained second detection model to obtain second feature information (S101); on the basis of position information of a target object in the sample image, determining a salient region corresponding to the target object (S102); acquiring first salient region features according to the first feature information and the salient region, and acquiring second salient region features according to the second feature information and the salient region (S103); and adjusting parameters of the first detection model according to the first salient region features and the second salient region features to obtain a trained first detection model (S104). The training reliability and accuracy of the first detection model are improved.

Description

Detection model training method, device, detection model use method and storage medium

technical field

The present application relates to the technical field of computer vision, and in particular, to a detection model training method, an apparatus, a detection model use method, and a storage medium.

Background technique

With the development of science and technology and the rise of deep learning, the technology of identifying objects in images has become one of the very important technologies in computer vision, and the application of deep learning in the field of image object detection has achieved great breakthroughs. For example, the region where a face is located can be identified from a given image.

At present, the technical focus of the target detection algorithm in the existing detection model is on the accuracy of the detection result, so the scale of the existing detection model is large, which makes the existing detection model run slowly and cannot be used in resource allocation. If it is implemented on a small mobile terminal, if the scale of the model is reduced and applied to the mobile terminal, the performance of the detection model cannot be guaranteed, and the scope of use of the model is limited.

SUMMARY OF THE INVENTION

Embodiments of the present application provide a detection model training method, device, detection model use method, and storage medium, which can reduce the scale of the first detection model and improve the reliability and accuracy of the first detection model training.

In a first aspect, an embodiment of the present application provides a detection model training method, including:

Perform feature extraction on the sample image through the first detection model to obtain first feature information, and perform feature extraction on the sample image through the trained second detection model to obtain second feature information;

determining the salient area corresponding to the target based on the position information of the target in the sample image;

obtaining a first salient area feature according to the first feature information and the salient area, and obtaining a second salient area feature according to the second feature information and the salient area;

The parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain a trained first detection model.

In a second aspect, an embodiment of the present application further provides a detection model training apparatus, including a processor and a memory, where a computer program is stored in the memory, and the processor executes the implementation of the present application when calling the computer program in the memory Any of the detection model training methods provided in the example.

In a third aspect, an embodiment of the present application also provides a method for using a detection model, which is applied to computer equipment, where the detection model is a first detection model after training, and the first detection model after training is implemented using the present application. A model obtained by training any of the detection model training methods provided in the example is deployed in the computer equipment; the detection model using method includes:

Get the image to be detected;

The target object in the image is detected by the trained first detection model, and the target position information of the target object in the image is obtained.

In a fourth aspect, an embodiment of the present application further provides a storage medium, where the storage medium is used to store a computer program, and the computer program is loaded by a processor to execute:

This embodiment of the present application may perform feature extraction on the sample image by using the first detection model to obtain the first feature information, and perform feature extraction on the sample image by using the second detection model to obtain the second feature information. Then, the salient region corresponding to the target can be determined based on the position information of the target in the sample image, the first salient region feature can be obtained according to the first feature information and the salient region, and the second salient region can be obtained according to the second feature information and the salient region. salient regional features. At this time, the parameters of the first detection model may be adjusted according to the first salient region feature and the second salient region feature to obtain a trained first detection model. This solution can use the trained second detection model to accurately train the first detection model, so that the trained first detection model can be applied to the mobile terminal to detect the target, which can reduce the number of first detection The scale of the model, and the training of the first detection model based on the salient region corresponding to the determined target and its salient region characteristics can improve the reliability and accuracy of training the first detection model, making the first detection model applicable. wide range.

Description of drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. For those of ordinary skill, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic flowchart of a detection model training method provided by an embodiment of the present application;

2 is a schematic diagram of an image of an area where an extraction target is located provided by an embodiment of the present application;

3 is a schematic flowchart of a process for preprocessing an initial image and key points of a human face provided by an embodiment of the present application;

4 is a schematic diagram of generating multiple candidate regions provided by an embodiment of the present application;

5 is a schematic flowchart of a method for using a detection model provided by an embodiment of the present application;

6 is a schematic flowchart of training a first detection model provided by an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a detection model training apparatus provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

The flowcharts shown in the figures are for illustration only, and do not necessarily include all contents and operations/steps, nor do they have to be performed in the order described. For example, some operations/steps can also be decomposed, combined or partially combined, so the actual execution order may be changed according to the actual situation.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The embodiments described below and features in the embodiments may be combined with each other without conflict.

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a detection model training method provided by an embodiment of the application. The detection model training method can be applied to a detection model training device, and is used to accurately train the smaller-scale first detection model by using the second detection model. The detection model training device may include a mobile phone, a computer, a server, or an unmanned aerial vehicle.

Among them, the UAV can be a rotary-wing UAV, such as a quad-rotor UAV, a hexa-rotor UAV, an octa-rotor UAV, or a fixed-wing UAV, or a rotary-wing and fixed-wing unmanned aerial vehicle. The combination of man and machine is not limited here.

Specifically, as shown in FIG. 1 , the detection model training method may include steps S101 to S104 and so on.

S101. Perform feature extraction on the sample image through a first detection model to obtain first feature information, and perform feature extraction on the sample image through a second detection model to obtain second feature information.

The first detection model and the second detection model can be flexibly set according to actual needs, and the specific types are not limited here. For example, the first detection model and the second detection model can be neural networks.

In some embodiments, the detection model training method can be applied to a distillation algorithm, the first detection model is a student model, and the second detection model is a teacher model.

Among them, the distillation algorithm may be to use one or more trained teacher models (also called Teacher models, which may be larger-scale models) to guide the student model (also called the Student model, the Student model). can be a smaller model) for training. The process of the distillation algorithm can be: Teacher model training, Student model training, and joint training with the Teacher model and the Student model to improve the performance of the Student model. For example, the teacher model and the student model can be trained separately through sample images. After the teacher model and the student model are trained separately, the parameters of the teacher model are fixed, that is, the teacher model only performs feature extraction and no parameter update is performed, and the student model continues. Distillation training.

In the prior art, there are a small number of distillation techniques that can be applied to the detection model, but they are based on the Two-stage (two-stage) target detection technology, which is not applicable to the one-stage (one-stage) target detection. However, this application The embodiment can obtain the saliency area corresponding to the target object, and thereby obtain the salient area features of the first detection model (student model) and the second detection model (teacher model), so as to determine the first detection model based on the salient area features of the two. The detection model is trained not only for the distillation algorithm of two-stage (two-stage) target detection, but also for the distillation algorithm of one-stage (one-stage) target detection, which has wider practicability and improves the efficiency of training.

In some embodiments, the scale of the first detection model is smaller than the scale of the second detection model, which is a trained model. In order to improve the training accuracy of the first detection model, the pre-trained second detection model may be used to guide the training of the first detection model.

Wherein, the sample image may be acquired by a collection device such as a camera or a camera, or the sample image may be acquired from a preset local database or server, or the sample image may be an initial image obtained from the acquisition Generated after preprocessing such as rotation or scaling. The sample image may contain a target, and the type of the target may be flexibly set according to actual needs. For example, the target may include objects such as a human face, a vehicle, a ball, or a dog. It should be noted that a sample image can include multiple images, and the size of each sample image can be the same or different. A sample image can contain one or more objects of the same type, or, a sample image There are many different types of targets that can be included in , which are not specifically limited here.

In some embodiments, before the feature extraction is performed on the sample image by the first detection model, the detection model training method may further include: acquiring an initial image; extracting an image of a region where the target is located from the initial image; extracting the target from the regional image The initial image and the key points are preprocessed to obtain the sample image and the preprocessed key points; the position information of the target object in the sample image is determined according to the preprocessed key points.

In order to enrich the sample images and expand the learning range of the model, the obtained initial images can be preprocessed to obtain abundant sample images, so as to use the abundant sample images to train the first detection model, so as to solve the problem of limited existing data resources. The problem of insufficient training. Specifically, the initial image may be collected by a collection device such as a camera or a camera, or the initial image may be obtained from a preset local database or server, or the like. The initial image may contain objects. For example, the types of objects may include objects such as faces, vehicles, balls, or dogs.

Then, the image of the area where the target is located can be extracted from the initial image. For example, as shown in FIG. 2 , the image of the area where the user's face is located can be extracted from the initial image containing the user; for another example, the image of the area containing the vehicle can be extracted. The image of the area where the vehicle is located is extracted from the initial image. At this time, the key points of the target can be extracted from the area image, and the number, shape, position or size of the key points can be flexibly set according to actual needs, and the specific content is not limited here. For example, key points such as eyes, nose, mouth, and contours of the face can be extracted from the image of the area where the face is located. For another example, the wheels, lights, windows, and body of the vehicle can be extracted from the image of the area where the vehicle is located. etc. key points.

At this point, the initial image can be preprocessed to obtain a sample image, and the keypoints can be preprocessed to obtain preprocessed keypoints. In some embodiments, preprocessing the initial image and the key points to obtain the sample image and the preprocessed key points may include: rotating, translating, zooming and/or adjusting the brightness of the initial image and the key points according to a preset angle , get the sample image and the preprocessed keypoints.

The preprocessing may be flexibly set according to actual needs, for example, the preprocessing may include processing such as rotation, cropping, flipping, translation, scaling, brightness reduction and/or brightness enhancement. The preset angle can be flexibly set according to actual needs. It should be noted that the way of preprocessing the initial image and the way of preprocessing keypoints can be consistent or inconsistent. For example, both the initial image and the key points can be rotated 90 degrees clockwise to obtain the sample image and the preprocessed key points; for another example, the initial image can be rotated 90 degrees clockwise to obtain the sample image, and the The key points are rotated 45 degrees clockwise to obtain the pre-processed key points.

Finally, the position information of the object in the sample image can be determined according to the preprocessed key points. For example, the position of the preprocessed key point in the sample image can be determined, and the position of the preprocessed key point in the sample image can be determined according to the position of the preprocessed key point in the sample image. The area of the target object in the sample image is generated, and the area can be a rectangle or a square, and the position information of the target object in the sample image is determined based on the area of the target object in the sample image. The position information may be the pixel coordinates of the target object, or the pixel coordinates of the vertex of the region of the target object in the sample image, or the like.

As shown in Figure 3, taking the target as a face as an example, the process of preprocessing the initial image and the key points of the face may include:

S11. Acquire an initial image image.

S12. Extract the face area image face_image from the initial image according to the known face frame.

S13, extracting the face key points face_landmarks of the face area image face_image.

S14. Rotate the initial image image and the face key point face_landmarks by any random angle to obtain the rotated image rotate_image and the rotated face key point rotate_landmarks.

S15. Calculate the face frame rotate_box according to the rotated face key points rotate_landmarks, that is, the position information of the face.

S16, save the rotated image rotate_image and the face frame rotate_box.

The automatic preprocessing of the initial image and key points (also known as data enhancement processing) is realized, saving time and effort. It should be noted that the initial image and key points can also be preprocessed manually.

After the location information of the sample image and the target object is obtained, feature extraction can be performed on the sample image through the first detection model to obtain first feature information, and feature extraction can be performed on the sample image through the second detection model to obtain second feature information.

S102. Determine a salient area corresponding to the target based on the position information of the target in the sample image.

The salient area pos-anchors corresponding to the target can be determined based on the position information of the target in the sample image through the first detection model. The salient area can be an area that is convenient for model learning, and can only include positive sample areas or positive samples. region and negative sample region, etc.

In some embodiments, determining the salient area corresponding to the target object based on the position information of the target object in the sample image may include: acquiring a plurality of candidate areas; determining the target area of the target object based on the position information of the target object; From the candidate regions, select the region whose coincidence degree with the target region is greater than the first preset threshold, and obtain a positive sample region; and select from the plurality of candidate regions that the coincidence degree with the target region is within the preset range, and the classification probability value is greater than A region with a preset probability threshold is obtained as a negative sample region, and the preset range can be an interval smaller than the first preset threshold and greater than the second preset threshold; the positive sample region and the negative sample region are set as salient regions corresponding to the target.

In order to improve the reliability of salient regions, so as to improve the accuracy of model training and improve the performance of the model, salient regions containing positive sample regions and negative sample regions can be obtained to train the model. Specifically, first, multiple candidate regions may be acquired. In some embodiments, acquiring multiple candidate regions may include: generating multiple candidate regions based on the second detection model, or acquiring multiple pre-labeled candidate regions.

The shape or size of the candidate region can be flexibly set according to actual needs. For example, as shown in Figure 4, the sample image can be detected by the second detection model to generate multiple candidate regions; The multiple candidate regions marked in advance, and the multiple candidate regions marked in advance can be manually marked or automatically marked.

And, the target area of the target object is determined based on the position information, for example, the target area of the target object may be determined based on the pixel coordinate positions of the four vertex corners of the quadrilateral where the target object is located. Then, the degree of coincidence between each candidate area and the target area can be calculated separately. For example, the intersection over union (IOU) algorithm can be used to calculate the degree of coincidence between each candidate area and the target area: Obtain the candidate area The intersection area with the target area, and the union area between the candidate area and the target area are obtained, and the degree of coincidence between the candidate area and the target area is calculated according to the intersection area and the union area.

Among them, the calculation method of the degree of coincidence between the candidate area and the target area can be as follows: formula (1):

In formula (1), IOU(A, B) represents the degree of coincidence between the candidate area A and the target area B, |A∩B| represents the intersection area between the candidate area A and the target area B, |A∪ B| represents the union area between the candidate area A and the target area B.

For multiple candidate regions, the degree of coincidence with the target region can be calculated by formula (1). When the sample image includes multiple objects, the degree of coincidence of each object can be calculated separately.

Then, a region whose coincidence degree with the target region is greater than the first preset threshold can be selected from the multiple candidate regions to obtain a positive sample region. The specific value of the first preset threshold can be flexibly set according to actual needs. If the coincidence degree between the candidate area and the target area is greater than the first preset threshold, it indicates that the similarity between the candidate area and the target area is high.

And, the classification probability value of each candidate region is calculated, and the value range of the classification probability value may be 0 to 1, for example, the classification probability value of the candidate region being a face region is 0.6 or 0.9. At this time, a region with a degree of coincidence with the target region within a preset range and a classification probability value greater than a preset probability threshold can be selected from the multiple candidate regions to obtain a negative sample region, where the preset range is smaller than the first preset probability threshold. The threshold value is set in an interval greater than the second preset threshold value, and the specific value of the second preset threshold value can be flexibly set according to actual needs. Finally, the positive sample region and the negative sample region can be set as the salient regions corresponding to the target.

In the embodiment of the present invention, not only the positive sample area, but also the information of the negative sample area is obtained to train the first detection model, so that the training is more sufficient, the obtained first detection model is more accurate and reliable, and the shortage of existing training resources is solved. The problem.

S103. Acquire the first salient area feature according to the first feature information and the salient area, and acquire the second salient area feature according to the second feature information and the salient area.

The first salient region feature and the second salient region feature may be flexibly set according to actual needs, and the specific content is not limited here. For example, the first salient region feature may be a feature related to the first feature information in the salient region, and the second salient region feature may be a feature related to the second feature information in the salient region.

In order to improve the accuracy of acquiring the first salient region feature and the second salient region feature, in some embodiments, the first salient region feature is acquired according to the first feature information and the salient region, and according to the second feature information and the salient region, Obtaining the features of the second salient region may include: obtaining first feature information in the positive sample region and the negative sample region, respectively, to obtain the first salient region feature; and obtaining the second feature information of the positive sample region and the negative sample region, respectively, to obtain The second salient regional feature.

S104. Adjust the parameters of the first detection model according to the first salient region feature and the second salient region feature to obtain a trained first detection model.

In this embodiment, the first detection model is used to detect the type and position of the target.

In some embodiments, the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature, and obtaining the trained first detection model may include: acquiring the first salient region feature and the second salient region feature The similarity between the features; the loss value obtained by the first detection model for detecting the sample image; and the parameters of the first detection model are adjusted according to the similarity and the loss value to obtain the trained first detection model.

In order to improve the reliability and accuracy of training the first detection model, the similarity between the first salient region feature and the second salient region feature can be obtained, and the similarity can be calculated by Euclidean distance (ie Euclidean distance). characterization.

In some embodiments, the similarity includes Euclidean distance, and obtaining the similarity between the first salient region feature and the second salient region feature may include: determining the similarity between the first salient region feature and the second salient region feature Euclidean distance to get the similarity between the first salient region feature and the second salient region feature. For example, the Euclidean distance L2-loss (distill-loss) between the first salient region feature and the second salient region feature can be calculated, and the Euclidean distance L2-loss is the first salient region feature and the second salient region feature. The similarity between the two salient region features.

And, the loss value loss obtained by detecting the sample image by the first detection model is obtained, and then the parameters of the first detection model can be adjusted according to the similarity L2-loss and the loss value loss to obtain the trained first detection model.

In some embodiments, adjusting the parameters of the first detection model according to the similarity and the loss value, and obtaining the trained first detection model may include: performing a weighted average operation on the similarity and the loss value to obtain the target loss value; adjust the parameters of the first detection model according to the target loss value to obtain the trained first detection model.

For example, the similarity L2-loss and the loss value loss can be added and averaged to obtain the target loss value=(L2-loss+loss)/2. For another example, the weight value of the similarity L2-loss can be set to A, and the weight value of the loss value loss can be set to B, where A+B=1. In this case, multiply the similarity L2-loss and the loss value respectively. Take the corresponding weight value and sum it up to get the target loss value=L2-loss*A+loss*B.

Then, the parameters of the first detection model may be adjusted according to the target loss value, so that the parameters of the first detection model are adjusted to appropriate values, and the trained first detection model is obtained. Therefore, a high-precision trained first detection model that meets the requirements can be obtained under the dual constraints of limited computing resources, and a large amount of collected data can be saved, saving time and resources, on the premise of achieving the same effect.

In some embodiments, the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature, and after the trained first detection model is obtained, the detection model training method may further include: acquiring the to-be-detected The image; the target object in the image is detected by the trained first detection model, and the target position information of the target object in the image is obtained.

After the trained first detection model is obtained, the trained first detection model can be used to accurately detect the target in the image. For example, the image to be detected may be collected by a collection device such as a camera or a camera, or the image to be detected may be obtained from a preset local database or server, or the like. At this time, the target object in the image can be detected by the trained first detection model, and the target position information of the target object in the image can be obtained. For example, the trained first detection model can be used to detect the face in the image to obtain the target position information of the face in the image, and the target position information can be the vertex position of the polygon (for example, quadrilateral) face frame.

This embodiment of the present application may perform feature extraction on the sample image by using the first detection model to obtain the first feature information, and perform feature extraction on the sample image by using the second detection model to obtain the second feature information. Then, the salient region corresponding to the target can be determined based on the position information of the target in the sample image, the first salient region feature can be obtained according to the first feature information and the salient region, and the second salient region can be obtained according to the second feature information and the salient region. salient regional features. At this time, the parameters of the first detection model may be adjusted according to the first salient region feature and the second salient region feature to obtain a trained first detection model. In this solution, the second detection model can be used to accurately train the smaller-scale first detection model, so that the trained first detection model can be applied to the mobile terminal to detect the target, which reduces the size of the first detection model. The scale of the first detection model, and training the first detection model based on the salient region corresponding to the determined target and its salient region characteristics can improve the reliability and accuracy of the training of the first detection model, so that the scope of application of the first detection model can be improved. wide.

Please refer to FIG. 5 , which is a schematic flowchart of a method for using a detection model provided by an embodiment of the application. The method for using the detection model can be applied to computer equipment for accurately detecting the target in the image based on the trained first detection model. The computer equipment may include mobile terminals, drones, servers, cameras, etc., and the mobile terminals may include mobile phones and tablet computers. The detection model is a trained first detection model, and the trained first detection model is a model obtained by using the above-mentioned detection model training method, and is deployed in a computer device.

For example, as shown in FIG. 6 , the process of training the first detection model may include:

S21. Obtain a sample image.

S22. Train a Teacher model (T-model) based on the sample images.

S23. Train the Student model (S-model) based on the sample images.

S24. Fix the parameters of the Teacher model, and extract the feature feature-T of the sample image through the Teacher model.

S25, extract the feature-S of the sample image through the Student model, and extract the salient region pos_anchors.

S26. Calculate the salient area feature pos_feat_T according to the salient area pos_anchors and the feature feature-T, and calculate the salient area feature pos_feat_S according to the salient area pos_anchors and the feature feature-S.

S27. Calculate the Euclidean distance L2-loss of the salient region feature pos_feat_T and the salient region feature pos_feat_S.

S28. Calculate the original loss value loss of the Student model.

S29, calculate the weighted average of the Euclidean distance L2-loss and the original loss value loss, and retrain the Student model with finetune to obtain the distill-S-model (trained Student model) and save it.

In the above embodiments, the description of each embodiment has its own emphasis. For the part that is not described in detail in a certain embodiment, please refer to the above detailed description of the detection model training method, and will not be repeated here.

Specifically, as shown in FIG. 5 , the method for using the detection model may include steps S201 to S202 and so on.

S201. Acquire an image to be detected.

S202: Detect the target in the image by using the trained first detection model to obtain target position information of the target in the image.

For example, the image to be detected may be collected by a collection device such as a camera or a camera, or the image to be detected may be obtained from a preset local database or server, or the like. At this time, the target object in the image can be detected by the trained first detection model, and the target position information of the target object in the image can be obtained. For example, the trained first detection model can be used to detect the face in the image to obtain the target position information of the face in the image, and the target position information can be the vertex position of the polygon (for example, quadrilateral) face frame. The first detection model after training is used to accurately detect the target in the image.

Please refer to FIG. 7 , which is a schematic block diagram of a detection model training apparatus provided by an embodiment of the present application. The detection model training apparatus 11 may include a processor 111 and a memory 112, and the processor 111 and the memory 112 are connected through a bus, such as an I2C (Inter-integrated Circuit) bus.

Specifically, the processor 111 may be a micro-controller unit (Micro-controller Unit, MCU), a central processing unit (Central Processing Unit, CPU), or a digital signal processor (Digital Signal Processor, DSP) or the like.

Specifically, the memory 112 may be a Flash chip, a read-only memory (ROM, Read-Only Memory) magnetic disk, an optical disk, a U disk, or a mobile hard disk, etc., and may be used to store computer programs.

The processor 111 is configured to call the computer program stored in the memory 112, and implement the detection model training method provided by the embodiment of the present application when executing the computer program, for example, the following steps may be performed:

Perform feature extraction on the sample image through the first detection model to obtain first feature information, and perform feature extraction on the sample image through the second detection model to obtain second feature information; and determine the target object based on the position information of the target object in the sample image Corresponding salient region; according to the first feature information and salient region, obtain the first salient region feature, and according to the second feature information and salient region, obtain the second salient region feature; according to the first salient region feature and the second salient region feature The parameters of the first detection model are adjusted to obtain the trained first detection model.

In some embodiments, when the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain the trained first detection model, the processor 111 is configured to execute: obtain the first salient region feature. The similarity between the regional feature and the second salient region feature; obtain the loss value obtained by the first detection model to detect the sample image; adjust the parameters of the first detection model according to the similarity and the loss value, and obtain the first detection model after training. A detection model.

In some embodiments, when the parameters of the first detection model are adjusted according to the similarity and the loss value to obtain the trained first detection model, the processor 111 is configured to perform: weighted average operation on the similarity and the loss value , obtain the target loss value; adjust the parameters of the first detection model according to the target loss value to obtain the trained first detection model.

In some embodiments, the similarity includes Euclidean distance. When obtaining the similarity between the first salient region feature and the second salient region feature, the processor 111 is configured to perform: determining the first salient region feature and the second salient region feature. The Euclidean distance between the two salient region features, to obtain the similarity between the first salient region feature and the second salient region feature.

In some embodiments, when determining the salient region corresponding to the target object based on the position information of the target object in the sample image, the processor 111 is configured to perform: acquiring a plurality of candidate regions; determining the target region of the target object based on the position information; A positive sample region is obtained by screening out the regions whose coincidence degree with the target region is greater than the first preset threshold from the plurality of candidate regions; screening out from the plurality of candidate regions that the coincidence degree with the target region is within a preset range, and the classification probability is The area whose value is greater than the preset probability threshold value is obtained as a negative sample area, and the preset range is an interval smaller than the first preset threshold value and greater than the second preset threshold value; the positive sample area and the negative sample area are set as the significant area corresponding to the target object .

In some embodiments, when acquiring multiple candidate regions, the processor 111 is configured to perform: generating multiple candidate regions based on the second detection model, or acquiring multiple pre-labeled candidate regions.

In some embodiments, when acquiring the first salient region feature according to the first feature information and the salient region, and acquiring the second salient region feature according to the second feature information and the salient region, the processor 111 is configured to execute: respectively acquiring The first feature information in the positive sample region and the negative sample region is obtained to obtain the first salient region feature; and the second feature information of the positive sample region and the negative sample region are respectively obtained to obtain the second salient region feature.

In some embodiments, after the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain the trained first detection model, the processor 111 is further configured to execute: acquiring the to-be-detected Detect the target object in the image through the trained first detection model, and obtain the target position information of the target object in the image.

In some embodiments, before the feature extraction is performed on the sample image by the first detection model, the processor 111 is configured to perform: acquiring an initial image; extracting an image of the region where the target object is located from the initial image; extracting the target object from the region image The initial image and the key points are preprocessed to obtain the sample image and the preprocessed key points; the position information of the target object in the sample image is determined according to the preprocessed key points.

In some embodiments, when the initial image and the key points are preprocessed to obtain the sample image and the preprocessed key points, the processor 111 is configured to perform: rotate and translate the initial image and the key points according to a preset angle , scaling and/or brightness adjustment to obtain sample images and preprocessed keypoints.

In some embodiments, the target includes a human face.

In some embodiments, the scale of the first detection model is smaller than the scale of the second detection model, which is a trained model.

In some embodiments, the storage medium is applied to the distillation algorithm, the first detection model is a student model, and the second detection model is a teacher model.

Embodiments of the present application further provide a storage medium, where the storage medium is a computer-readable storage medium, where a computer program is stored in the storage medium, and the computer program includes program instructions, and the processor executes the program instructions to realize the provision of the embodiments of the present application. The detection model training method. For example, a processor can execute:

In some embodiments, when the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain the trained first detection model, the processor is configured to execute: acquiring the first salient region feature and the similarity between the second salient region feature; obtain the loss value obtained by the first detection model to detect the sample image; adjust the parameters of the first detection model according to the similarity and the loss value to obtain the first detection model after training Model.

In some embodiments, when the parameters of the first detection model are adjusted according to the similarity and the loss value to obtain the trained first detection model, the processor is configured to perform: performing a weighted average operation on the similarity and the loss value to obtain target loss value; adjust the parameters of the first detection model according to the target loss value to obtain the trained first detection model.

In some embodiments, the similarity includes Euclidean distance, and when acquiring the similarity between the first salient region feature and the second salient region feature, the processor is configured to perform: determining the first salient region feature and the second salient region feature Euclidean distance between regional features to get the similarity between the first salient region feature and the second salient region feature.

In some embodiments, when determining the salient region corresponding to the target object based on the position information of the target object in the sample image, the processor is configured to: obtain a plurality of candidate regions; determine the target region of the target object based on the position information; From the candidate regions, select the regions whose coincidence degree with the target region is greater than the first preset threshold, and obtain a positive sample region; and select from the plurality of candidate regions that the coincidence degree with the target region is within the preset range, and the classification probability value is greater than A region with a preset probability threshold is obtained as a negative sample region, and the preset range is an interval smaller than the first preset threshold and greater than the second preset threshold; the positive sample region and the negative sample region are set as salient regions corresponding to the target.

In some embodiments, when acquiring multiple candidate regions, the processor is configured to perform: generating multiple candidate regions based on the second detection model, or acquiring multiple pre-labeled candidate regions.

In some embodiments, when obtaining the first salient region feature according to the first feature information and the salient region, and obtaining the second salient region feature according to the second feature information and the salient region, the processor is configured to execute: respectively obtaining positive samples obtaining the first salient region feature by obtaining the first feature information in the region and the negative sample region; and obtaining the second feature information of the positive sample region and the negative sample region respectively to obtain the second salient region feature.

In some embodiments, after the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain the trained first detection model, the processor is further configured to perform: acquiring the to-be-detected The image; the target object in the image is detected by the trained first detection model, and the target position information of the target object in the image is obtained.

In some embodiments, before the feature extraction is performed on the sample image by the first detection model, the processor is configured to perform: acquiring an initial image; extracting an image of an area where the target is located from the initial image; extracting the key of the target from the area image point; preprocess the initial image and key points to obtain the sample image and the preprocessed key points; determine the position information of the target in the sample image according to the preprocessed key points.

In some embodiments, when the initial image and the key points are preprocessed to obtain the sample image and the preprocessed key points, the processor is configured to perform: rotate, translate, and zoom the initial image and the key points according to a preset angle and/or brightness adjustment to obtain sample images and preprocessed keypoints.

In some embodiments, the target includes a human face.

The storage medium may be an internal storage unit of the detection model training apparatus described in any of the foregoing embodiments, such as a hard disk or a memory of the detection model training apparatus. The storage medium can also be an external storage device of the detection model training device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a Secure Digital (SD) card, a flash memory card equipped on the detection model training device (Flash Card) etc.

Since the computer program stored in the storage medium can execute any detection model training method provided by the embodiments of the present application, it is possible to implement any of the detection model training methods provided by the embodiments of the present application. For the beneficial effects, refer to the foregoing embodiments for details, which will not be repeated here.

It should be understood that the terms used in the specification of the present application herein are for the purpose of describing particular embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural unless the context clearly dictates otherwise.

It will also be understood that, as used in this specification and the appended claims, the term "and/or" refers to and including any and all possible combinations of one or more of the associated listed items. It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in the present application. Modifications or substitutions shall be covered by the protection scope of this application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

A detection model training method, comprising:

Perform feature extraction on the sample image through the first detection model to obtain first feature information, and perform feature extraction on the sample image through the trained second detection model to obtain second feature information;

determining the salient area corresponding to the target based on the position information of the target in the sample image;

obtaining a first salient area feature according to the first feature information and the salient area, and obtaining a second salient area feature according to the second feature information and the salient area;

The parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain a trained first detection model.
The detection model training method according to claim 1, wherein the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain a trained The first detection model includes:

obtaining the similarity between the first salient region feature and the second salient region feature;

obtaining a loss value obtained by detecting the sample image by the first detection model;

The parameters of the first detection model are adjusted according to the similarity and the loss value to obtain a trained first detection model.
The detection model training method according to claim 2, wherein the adjusting parameters of the first detection model according to the similarity and the loss value, and obtaining the trained first detection model comprises:

Perform a weighted average operation on the similarity and the loss value to obtain a target loss value;

The parameters of the first detection model are adjusted according to the target loss value to obtain a trained first detection model.
The detection model training method according to claim 2, wherein the similarity comprises a Euclidean distance, and the acquiring the similarity between the first salient region feature and the second salient region feature include:

The Euclidean distance between the first salient region feature and the second salient region feature is determined, and the similarity between the first salient region feature and the second salient region feature is obtained.
The detection model training method according to claim 1, wherein the determining the salient region corresponding to the target object based on the position information of the target object in the sample image comprises:

Get multiple candidate regions;

determining a target area of the target based on the location information;

Screening out the regions whose coincidence degree with the target region is greater than the first preset threshold from the plurality of candidate regions to obtain a positive sample region;

From the plurality of candidate regions, select the regions whose coincidence degree with the target region is within the preset range, and the classification probability value is greater than the preset probability threshold, to obtain a negative sample region, and the preset range is smaller than the an interval between the first preset threshold and greater than the second preset threshold;

The positive sample area and the negative sample area are set as salient areas corresponding to the target.
The detection model training method according to claim 5, wherein the acquiring multiple candidate regions comprises:

Generate multiple candidate regions based on the second detection model, or obtain multiple candidate regions marked in advance.
The detection model training method according to claim 5, wherein the first salient region feature is acquired according to the first feature information and the salient region, and the second feature information and the salient region are obtained according to the second feature information and the salient region. region, and obtaining the second salient region features include:

respectively acquiring the first feature information in the positive sample region and the negative sample region to obtain a first salient region feature; and,

The second feature information of the positive sample area and the negative sample area are acquired respectively, to obtain a second salient area feature.
The detection model training method according to claim 1, wherein the parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain a trained After the first detection model, the detection model training method further includes:

Get the image to be detected;

Detect the target in the image by using the trained first detection model to obtain target position information of the target in the image.
The detection model training method according to claim 1, wherein before the feature extraction is performed on the sample image by the first detection model, the detection model training method further comprises:

get the initial image;

extracting an image of the region where the target is located from the initial image;

extracting key points of the target from the area image;

The initial image and the key point are preprocessed to obtain the sample image and the preprocessed key point;

The position information of the target object in the sample image is determined according to the preprocessed key points.
The detection model training method according to claim 9, wherein the preprocessing of the initial image and the key points to obtain the sample image and the preprocessed key points comprises:

Rotate, translate, zoom and/or adjust the brightness of the initial image and the key points according to a preset angle to obtain the sample image and the pre-processed key points.
The detection model training method according to claim 1, wherein the target object comprises a human face.
The detection model training method according to any one of claims 1 to 11, wherein the scale of the first detection model is smaller than the scale of the second detection model, and the second detection model is a trained model .
The detection model training method according to any one of claims 1 to 11, wherein the detection model training method is applied to a distillation algorithm, the first detection model is a student model, and the second detection model is a teacher Model.
A detection model training device, characterized in that it includes a processor and a memory, wherein a computer program is stored in the memory, and the processor executes the method described in any one of claims 1 to 13 when the processor calls the computer program in the memory. The detection model training method described above.
A method for using a detection model, characterized in that, when applied to computer equipment, the detection model is a first detection model after training, and the first detection model after training is a method described in any one of claims 1 to 13. The detection model training method is used to train the obtained model, and is deployed in the computer equipment; the detection model using method includes:

Get the image to be detected;

The target object in the image is detected by the trained first detection model, and the target position information of the target object in the image is obtained.
The method for using a detection model according to claim 15, wherein the computer equipment comprises a mobile terminal, a drone and a camera.
A storage medium, characterized in that the storage medium is used to store a computer program, and the computer program is loaded by a processor to execute:

Perform feature extraction on the sample image through the first detection model to obtain first feature information, and perform feature extraction on the sample image through the trained second detection model to obtain second feature information;

determining the salient area corresponding to the target based on the position information of the target in the sample image;

obtaining a first salient area feature according to the first feature information and the salient area, and obtaining a second salient area feature according to the second feature information and the salient area;

The parameters of the first detection model are adjusted according to the first salient region feature and the second salient region feature to obtain a trained first detection model.
The storage medium according to claim 17, wherein the first detection model after training is obtained by adjusting parameters of the first detection model according to the first salient region feature and the second salient region feature model, the processor is used to execute:

obtaining the similarity between the first salient region feature and the second salient region feature;

obtaining a loss value obtained by detecting the sample image by the first detection model;

The parameters of the first detection model are adjusted according to the similarity and the loss value to obtain a trained first detection model.
The storage medium according to claim 18, wherein when parameters of the first detection model are adjusted according to the similarity and the loss value to obtain a trained first detection model, the processing The device is used to execute:

Perform a weighted average operation on the similarity and the loss value to obtain a target loss value;

The parameters of the first detection model are adjusted according to the target loss value to obtain a trained first detection model.
The storage medium according to claim 18, wherein the similarity comprises a Euclidean distance, and when acquiring the similarity between the first salient region feature and the second salient region feature, the The processor described above is used to execute:

The Euclidean distance between the first salient region feature and the second salient region feature is determined, and the similarity between the first salient region feature and the second salient region feature is obtained.
The storage medium according to claim 17, wherein, when determining the salient region corresponding to the target object based on the position information of the target object in the sample image, the processor is configured to execute:

Get multiple candidate regions;

determining a target area of the target based on the location information;

Screening out the regions whose coincidence degree with the target region is greater than the first preset threshold from the plurality of candidate regions to obtain a positive sample region;

From the plurality of candidate regions, select the regions whose coincidence degree with the target region is within the preset range, and the classification probability value is greater than the preset probability threshold, to obtain a negative sample region, and the preset range is smaller than the an interval between the first preset threshold and greater than the second preset threshold;

The positive sample area and the negative sample area are set as salient areas corresponding to the target.
The storage medium according to claim 21, wherein when acquiring multiple candidate regions, the processor is configured to execute:

Generate multiple candidate regions based on the second detection model, or obtain multiple candidate regions marked in advance.
The storage medium according to claim 21, characterized in that in obtaining the first salient region feature according to the first feature information and the salient region, and obtaining the first salient region feature according to the second feature information and the salient region When the second salient region is characterized, the processor is configured to execute:

respectively acquiring the first feature information in the positive sample region and the negative sample region to obtain a first salient region feature; and,

The second feature information of the positive sample area and the negative sample area are acquired respectively, to obtain a second salient area feature.
The storage medium according to claim 17, wherein the first detection model after training is obtained by adjusting parameters of the first detection model according to the first salient region feature and the second salient region feature After the model, the processor is also used to execute:

Get the image to be detected;

Detect the target in the image by using the trained first detection model to obtain target position information of the target in the image.
The storage medium according to claim 17, wherein before the feature extraction is performed on the sample image by the first detection model, the processor is configured to execute:

get the initial image;

extracting an image of the region where the target is located from the initial image;

extracting key points of the target from the area image;

Preprocessing the initial image and the key points to obtain the sample image and the preprocessed key points;

The position information of the target object in the sample image is determined according to the preprocessed key points.
The storage medium according to claim 25, wherein when the initial image and the key points are preprocessed to obtain the sample image and the preprocessed key points, the processor is configured to execute:

Rotate, translate, zoom and/or adjust the brightness of the initial image and the key points according to a preset angle to obtain the sample image and the pre-processed key points.
The storage medium of claim 17, wherein the target comprises a human face.
The storage medium according to any one of claims 17 to 27, wherein the scale of the first detection model is smaller than the scale of the second detection model, and the second detection model is a trained model.
The storage medium according to any one of claims 17 to 27, wherein the storage medium is applied to a distillation algorithm, the first detection model is a student model, and the second detection model is a teacher model.