CN113780277B - Training method and device of target detection model, electronic equipment and storage medium - Google Patents
Training method and device of target detection model, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113780277B CN113780277B CN202111048969.2A CN202111048969A CN113780277B CN 113780277 B CN113780277 B CN 113780277B CN 202111048969 A CN202111048969 A CN 202111048969A CN 113780277 B CN113780277 B CN 113780277B
- Authority
- CN
- China
- Prior art keywords
- sample
- candidate
- image
- region
- candidate sample
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a training method and device of a target detection model, electronic equipment and a storage medium, wherein the method comprises the following steps: classifying and regressing candidate sample areas in the image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image; determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image; and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model. The learning of the enhancement model on the difficult sample and the attention of the suppression model on the neglected sample are realized.
Description
Technical Field
The embodiment of the invention relates to the technical field of computer vision, in particular to a training method and device of a target detection model, electronic equipment and a storage medium.
Background
Object detection is widely used in many fields as a field of computer vision, and objects such as people and objects in images can be detected by object detection.
Due to the problems of serious image distortion, extreme angles of the target, excessive target shielding, fuzzy target types and the like, the type of the target cannot be determined, and the model learning can be influenced negatively, and all the characteristics can be deleted by a conventional filling scheme, so that the learning effect of the model is reduced. In addition, the model is difficult to learn and is easy to make mistakes to cause the target of missed detection or false detection, and the conventional online selection of the difficult sample forces the difficult sample to be added into training, so that the difficult sample is low in quality, and the training of the model on the difficult sample cannot be emphasized.
Disclosure of Invention
The embodiment of the invention provides a training method, a training device, electronic equipment and a storage medium of a target detection model, which are used for realizing the study of a difficult sample by an enhanced model and the attention of a neglected sample by a suppression model.
In a first aspect, an embodiment of the present invention provides a training method for a target detection model, including:
classifying and regressing candidate sample areas in the image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image;
Determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model.
In a second aspect, an embodiment of the present invention further provides a training device for a target detection model, including:
the classification confidence coefficient and regression position acquisition module is used for classifying and regressing candidate sample areas in the image through forward propagation of the target detection network to obtain classification confidence coefficient and regression positions of candidate samples in the image;
the loss weight information acquisition module is used for determining loss weight information adopted by the candidate sample area in the model training process according to the classification confidence degree and regression position of the candidate sample and the position of the reference sample area; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
The target detection model acquisition module is used for carrying out back propagation according to the loss weight information adopted by the candidate sample area, controlling the parameters of the target detection network to adjust towards the directions of suppressing neglected samples and enhancing difficult samples, and obtaining the trained target detection model.
In a third aspect, an embodiment of the present invention further provides an electronic device, including:
one or more processing devices;
a storage means for storing one or more programs;
when the one or more programs are executed by the one or more processing devices, the one or more processing devices are caused to implement a training method for the object detection model as provided in any embodiment of the present invention.
In a fourth aspect, there is also provided in an embodiment of the present invention a computer readable storage medium having stored thereon a computer program which, when executed by a processing device, implements a training method for the object detection model as provided in any embodiment of the present invention.
The embodiment of the invention provides a training method and device of a target detection model, electronic equipment and a storage medium, wherein the method comprises the following steps: classifying and regressing candidate sample areas in the image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image; determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image; and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model. The learning of the enhancement model on the difficult sample and the attention of the suppression model on the neglected sample are realized.
The foregoing summary is merely an overview of the present invention, and is intended to be implemented in accordance with the teachings of the present invention in order that the same may be more fully understood, and in order that the same or additional objects, features and advantages of the present invention may be more fully understood.
Drawings
Other features, objects and advantages of the present invention will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 is a flowchart of a training method of a target detection model according to an embodiment of the present invention;
FIG. 2 is a flowchart of a training method of a target detection model according to a second embodiment of the present invention;
FIG. 2A is an intersection region legend of candidate sample regions and reference sample regions provided in an embodiment of the present invention;
FIG. 2B is a union region illustration of candidate sample regions and reference sample regions provided in an embodiment of the present invention;
FIG. 3 is a block diagram of a training device for a target detection model according to a third embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
The invention is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention. It should be further noted that, for convenience of description, only some, but not all of the structures related to the present invention are shown in the drawings.
Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart depicts operations (or steps) as a sequential process, many of the operations (or steps) can be performed in parallel, concurrently, or at the same time. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figures. The processes may correspond to methods, functions, procedures, subroutines, and the like.
The following detailed description is made on the training method, device, electronic apparatus and storage medium of the object detection model provided in the present application through the following embodiments and optional solutions thereof.
Example 1
Fig. 1 is a flowchart of a training method of a target detection model according to an embodiment of the present invention. The embodiment of the invention can be suitable for training the target detection model. The method can be performed by a training device of the object detection model, which can be implemented in software and/or hardware and integrated on any electronic device with network communication functions. As shown in fig. 1, the training method of the target detection model provided in the embodiment of the present application may include the following steps:
s110, classifying and regressing candidate sample areas in the image through forward propagation of the target detection network, and obtaining the classification confidence and regression position of the candidate samples in the image.
The target detection may be to find all interested objects in the image, extract features of the objects, and classify and locate the objects at the same time. For example, finding out all the neglected samples and the difficult samples in the image, and classifying, identifying and positioning the two samples according to the characteristics of the neglected samples and the difficult samples.
Forward propagation may refer to a process of propagating forward layer by layer in a neural network, starting from the input layer, through the hidden layer to the output layer. For example, the model may be propagated forward layer by layer from the input layer to the output layer through the hidden layer in the object detection network, so as to obtain the candidate box.
The candidate sample region may refer to a region of the image in which the candidate sample is located, including, but not limited to, an anchor box region in the image and a sample region output via forward propagation.
Classification confidence may refer to the probability that a sample is present in a sample region, for example, the probability that a candidate sample is present in an image, typically with a confidence setting of 95%. And directly classifying and regressing the anchor frame by forward propagation of the target detection network, thereby obtaining the classification confidence coefficient of each sample.
Optionally, in single-stage target detection, the target detection network includes a single-stage convolutional neural network, and the candidate sample region includes an anchor frame region in the image; and, upon two-stage target detection, the target detection network comprises a convolutional neural network of a second stage of the two-stage target detection, and the candidate sample region comprises a sample region that is forward propagated output through the candidate region network of the first stage of the two-stage target detection.
And S120, determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region.
The weight may be determined according to a difference between the candidate frame and the real labeling frame, for example, may be a category difference between the candidate frame and the real labeling frame, and may be an offset of a position of the candidate frame relative to the real labeling frame.
Wherein the reference sample area includes, but is not limited to, a ignored sample area corresponding to a blurred object in the image and a difficult sample area in the image.
And S130, back propagation is carried out according to the loss weight information adopted by the candidate sample area, and parameters of the target detection network are controlled to be adjusted towards the directions of suppressing neglected samples and enhancing difficult samples, so that a trained target detection model is obtained.
The back propagation may be a process of continuously adjusting a gradient of the loss function with respect to each parameter, for example, the gradient descent method may be used to adjust each parameter, and the loss weight is used to update the parameters of the model, so as to reduce errors caused by the weight.
Ignoring the sample may mean that what kind of target cannot be determined due to problems such as serious image distortion, extreme angles of the target, excessive target shielding, and fuzzy target class; the method can also be applied to the field of special images, including but not limited to x-ray security inspection machine images, medical images, underwater images and the like, and the frequency of the occurrence of ambiguous targets in the images is higher due to the specificity of the acquisition technical means and scenes, and the labeling personnel in the scenes cannot know the real types of the targets in advance.
The difficult sample can be the object of missed detection or false detection caused by insufficient data quantity of the articles or difficult characteristic of the articles. For example, in the detection of an object, the background where the object is located is often used as the object to detect, thereby causing false detection.
The embodiment of the invention provides a training method of a target detection model, which classifies and regresses candidate sample areas in an image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image; determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image; and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model. Discarding part of the characteristics of the neglected sample in a softening way by dynamically adjusting the weight of the sample; for the difficult sample, the model is more stable to learn the difficult sample by the more gentle improvement model, so that the learning of the difficult sample by the enhancement model and the attention of the neglected sample by the inhibition model are realized.
Example two
Fig. 2 is a flowchart of a training method of a target detection model according to a second embodiment of the present invention, where the foregoing embodiment is further optimized based on the foregoing embodiment, and the embodiment of the present invention may be combined with each of the alternatives in one or more embodiments. As shown in fig. 2, the training method of the target detection model provided in the embodiment of the present application may include the following steps:
s210, classifying and regressing candidate sample areas in the image through forward propagation of the target detection network, and obtaining the classification confidence and regression position of the candidate samples in the image.
The candidate samples include, but are not limited to, neglected samples and difficult samples, and the neglected samples and the difficult samples need to be obtained before training. Manually labeling neglected samples into a certain class or not labeling is unfavorable for learning the model. If the target is marked, outlier noise may be brought to the learning of the positive sample of the model; if not targeted, the model is treated as a negative sample during learning, but the region itself contains some of the targeted features, which can lead to a rise in the false negative. For difficult samples, the model often fails to detect or misdetects some similar articles in actual detection, and the model learns the articles insufficiently due to insufficient data quantity of the articles or difficult characteristics of the articles.
Optionally, ignoring samples includes ambiguous targets that were previously noted in the image prior to training due to loss of original characteristics due to angle, distortion, occlusion, and/or imaging noise issues, resulting in an inability to make a judgment by human personnel; the difficult sample comprises targets which are marked in the image in advance before training, and the model miss detection probability or false detection probability is larger than a preset probability value when the model is actually used, wherein the targets with the model miss detection probability larger than the preset miss detection probability value are marked as difficult positive samples, and the targets with the model false detection probability larger than the preset false detection probability value are marked as difficult negative samples.
Optionally, the neglected sample is marked manually, and the original characteristic detection target is lost due to the problems of angles, distortion, shielding, imaging noise and the like in the selected image before model training, or the target with increased difficulty in manual judgment can be caused by the ambiguity, so that the position coordinates of the target are marked without distinguishing the categories. And the violent filling is avoided, part of useful features in the region are deleted, and the original distribution of the image features is influenced.
Optionally, the difficult sample is marked manually, the target frequently causing model missing or false detection in the image is selected before model training, the position coordinates of the target are marked, the target frequently missing or false detection among categories is marked as a positive difficult case, and the background target frequently false detection is marked as a negative difficult case. Meanwhile, manual marking can be replaced by a model inspection mode, namely, a trained model is used for reasoning on training data, and false detection and omission detection targets are obtained through calculation according to a reasoning result and real marking information. The method is favorable for improving the quality of difficult sample selection, and the training of the model on difficult cases is emphasized.
The difficult sample comprises a target which is marked in the image in advance before training and has a model missing detection probability or false detection probability larger than a preset probability value when the model is actually used, wherein the target with the model missing detection probability larger than the preset missing detection probability value is marked as the difficult positive sample, and the target with the model false detection probability larger than the preset false detection probability value is marked as the difficult negative sample. The target which is easy to cause missed detection can be screened out through presetting the missed detection probability value to be used as the target which is easy to miss detection; the target which is easy to cause false detection can be screened out through presetting false detection probability values and used as the target which is easy to cause false detection.
Optionally, performing forward propagation calculation on the model to obtain a candidate frame output by the regional generation network, fixing candidate features in the candidate frame to the same scale, and finally performing second-stage classification and regression on the samples to obtain the classification confidence coefficient and the coordinates after regression of each sample. In addition, if the network is a one-stage network such as SSD, YOLO, etc., the forward propagation of the network directly classifies and regresses the anchor frame, so that the classification confidence coefficient and the regressed coordinates of each sample are obtained.
The candidate features in the candidate frames are fixed to the same scale, and the ROI Pooling can be selected and used for target detection, so that the feature images in the neural network can be recycled, the training and detection speed is remarkably increased, and meanwhile, the target detection system is trained in an end-to-end mode.
S220, calculating the area ratio of the intersection area of the candidate sample area and the reference sample area in the image relative to the candidate sample area according to the regression position of the candidate sample and the position of the reference sample area.
The intersection region of the candidate sample region and the reference sample region in the image is shown in fig. 2A, and may refer to a portion where the candidate sample region and the reference sample region overlap; the union region of the candidate sample region and the reference sample region in the image is shown in fig. 2B, and may refer to the entire region obtained by adding the candidate sample region and the reference sample region.
Alternatively, for definition of positive and negative samples in the candidate box, an intersection ratio (iou) is generally used, which may refer to the ratio of the intersection of the candidate sample region and the reference sample region and the union of the candidate sample region and the reference sample region. When the iou of the candidate sample region and the reference sample region is more than or equal to 0.5, the candidate sample region and the reference sample region are considered to be positive samples; a negative sample is considered when the iou of the candidate sample region and the reference sample region is < 0.5.
S230, determining the influence degree value of the reference sample region under the candidate sample region according to the region ratio of the relative candidate sample region and the classification confidence of the candidate sample.
The influence degree value comprises an neglect degree score of neglect samples in the image and a difficulty routine degree score of difficult-to-sample samples in the image.
Optionally, a set of neglecting annotation boxes is defined as I, and the neglecting annotation boxes do not distinguish positive from negative. If the candidate sample region is determined to belong to the positive sample region, adopting a formula
Calculating to obtain an neglect degree score of a neglect sample in the image under the candidate sample area;
if the candidate sample region is determined to belong to the negative sample region, adopting the formula
Calculating to obtain an neglect degree score of a neglect sample in the image under the candidate sample area;
wherein C is i Representing the confidence of the classification of the ith candidate sample in the image, B i Representing the position coordinates of the ith candidate sample in the image, n is the total number of ignored boxes, I j The position coordinates of the j-th ignored sample in the image are denoted iof (B, I) as the area duty ratio of the ignored sample area in the candidate sample area.
Optionally, the set of the refractory sample labeling frames is H, the refractory sample distinguishes a positive refractory labeling frame and a negative refractory labeling frame, the set of the positive refractory labeling frames is HP (hard positive), and the set of the negative refractory labeling frames is HN (hard negative). If the candidate sample region is determined to belong to the positive sample region, adopting a formula
Calculating to obtain a difficulty routine degree score of a difficulty sample in the image under the candidate sample area;
if the candidate sample region is determined to belong to the negative sample region, adopting the formula
Calculating to obtain a difficulty routine degree score of a difficulty sample in the image under the candidate sample area;
wherein C is i Representing the confidence of the classification of the ith candidate sample in the image, B i Representing the position coordinates of the ith candidate sample in the image, I being a difficult sampleTotal number of books, HP k Position coordinates of the kth positive sample, HN k The position coordinates of the kth sample, iof (B, HN), represent the area ratio of the sample area of the sample in the candidate sample area.
S240, determining loss weight information of the candidate sample area according to the influence degree value of the reference sample area under the candidate sample area.
Optionally, according to the ignoring degree score of the ignoring sample in the image under the candidate sample area and the difficulty degree score of the difficult sample in the image, the formula lw=1-S is adopted ignore +S hard Calculating to obtain loss weight information of the candidate sample area;
wherein LW represents loss weight information of the candidate sample region, S ignore A degree of omission score representing the omitted samples in the image under the candidate sample region, S hard A difficulty routine score representing a difficulty sample in the image under the candidate sample region.
S250, back propagation is carried out according to loss weight information adopted by the candidate sample area, parameters of the target detection network are controlled to be adjusted towards the directions of suppressing neglected samples and enhancing difficult samples, and a trained target detection model is obtained.
When calculating the dynamic loss weight of each sample in the set L, the original loss weight is updated by using the weight, and is usually 1 by default, model parameters are updated by using the loss weight when model back propagation calculation, and the model is trained or fine-tuned from the beginning by using the method, so that neglect of samples and enhancement of difficult samples quantitatively are quantitatively inhibited. The model learning of the difficult sample is enhanced by inhibiting the influence of the neglected sample on the model, and two types of problems are well processed through a general paradigm without influencing the time consumption of the model.
The embodiment of the invention provides a training method of a target detection model, which is characterized in that the position coordinates of neglected samples are marked manually, so that the possible useful features of the region are prevented from being deleted during direct violent filling, and the abnormal edges after filling are prevented from influencing the original distribution of image features; marking the position coordinates of the difficult sample manually, marking the frequently missed or inter-category false detection targets as positive difficult samples, marking the frequently false detection background targets as negative difficult samples, and avoiding the low quality of the difficult samples of the online selection of the model; meanwhile, manual marking can be replaced by a model inspection mode, namely, a trained model is used for reasoning on training data, and false detection and omission detection targets are obtained through calculation according to a reasoning result and real marking information. The model is quantitatively improved by quantitatively inhibiting the influence of neglected samples on the model, and the model is quantitatively enhanced to learn difficult samples, so that two types of problems are better processed through a universal paradigm, and the time consumption of the model is not influenced. The method plays a role in softening influence by dynamically adjusting the weight of the sample, and for inhibiting the sample, part of the characteristics are discarded by softening; for a difficult sample, the method can moderately improve the learning of the model on the difficult sample, so that the model training is more stable, and the model training on the difficult sample is emphasized; the learning of the enhancement model on the difficult sample and the attention of the suppression model on the neglected sample are realized.
Example III
Fig. 3 is a block diagram of a training device for a target detection model according to a third embodiment of the present invention. The embodiment of the invention can be applied to the training of the target detection model. The apparatus may be implemented in software and/or hardware and integrated on any electronic device having network communication capabilities. As shown in fig. 3, the training device for the target detection model provided in the embodiment of the present application may include the following: a classification confidence and regression location acquisition module 310, a loss weight information acquisition module 320, and a target detection model acquisition module 330. Wherein:
the classification confidence coefficient and regression position obtaining module 310 is configured to classify and regress the candidate sample region in the image through forward propagation of the target detection network, so as to obtain a classification confidence coefficient and a regression position of the candidate sample in the image;
the loss weight information obtaining module 320 is configured to determine loss weight information adopted by the candidate sample region in the model training process according to the classification confidence level and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
The target detection model obtaining module 330 is configured to perform back propagation according to the loss weight information adopted by the candidate sample region, and control parameters of the target detection network to adjust in a direction in which the neglected sample and the enhanced refractory sample are suppressed, so as to obtain a trained target detection model.
Based on the above embodiment, optionally, the classification confidence and regression location obtaining module 310 includes:
when single-stage target detection is carried out, the target detection network comprises a single-stage convolutional neural network, and the candidate sample region comprises an anchor frame region in an image; and, upon two-stage target detection, the target detection network comprises a convolutional neural network of a second stage of the two-stage target detection, and the candidate sample region comprises a sample region that is forward propagated output through the candidate region network of the first stage of the two-stage target detection.
On the basis of the embodiment, optionally, the neglected sample includes ambiguous targets marked in the image in advance before training, which lose original characteristics due to problems of angles, distortion, shielding and/or imaging noise, so that a person cannot make judgment; the difficult sample comprises targets marked in the image in advance before training, and the model is easy to miss or misdetect when in actual use, wherein the targets easy to miss are marked as difficult positive samples, and the targets easy to misdetect are marked as difficult negative samples. For example, the difficult sample comprises targets which are marked in the image in advance before training, and the model miss detection probability or false detection probability is larger than a preset probability value when the model is actually used, wherein the targets which are marked as difficult positive samples and the targets which are marked as difficult negative samples. The target which is easy to cause missed detection can be screened out through presetting the missed detection probability value to be used as the target which is easy to miss detection; the target which is easy to cause false detection can be screened out through presetting false detection probability values and used as the target which is easy to cause false detection.
Based on the above embodiment, optionally, the loss weight information obtaining module 320 includes:
calculating the area ratio of the intersection area of the candidate sample area and the reference sample area in the image relative to the candidate sample area according to the regression position of the candidate sample and the position of the reference sample area; determining the influence degree value of the reference sample region under the candidate sample region according to the region duty ratio of the relative candidate sample region and the classification confidence of the candidate sample; the influence degree value comprises an neglect degree score of a neglect sample in the image and a difficulty routine degree score of a difficult sample in the image;
and determining loss weight information of the candidate sample region according to the influence degree value of the reference sample region under the candidate sample region.
On the basis of the above embodiment, optionally, when the reference sample area is an neglected sample area, determining the influence degree of the reference sample area under the candidate sample area according to the area ratio of the relative candidate sample area and the classification confidence of the candidate sample includes:
if the candidate sample region is determined to belong to the positive sample region, adopting a formula
Calculating to obtain an neglect degree score of a neglect sample in the image under the candidate sample area;
if the candidate sample region is determined to belong to the negative sample region, adopting the formula
Calculating to obtain an neglect degree score of a neglect sample in the image under the candidate sample area;
wherein C is i Representing the confidence of the classification of the ith candidate sample in the image, B i Representation of the drawingsPosition coordinates of the ith candidate sample in the image, n is the total number of ignored boxes, I j The position coordinates of the j-th ignored sample in the image are denoted iof (B, I) as the area duty ratio of the ignored sample area in the candidate sample area.
On the basis of the above embodiment, optionally, when the reference sample area is a difficult sample area, determining, according to the area ratio of the relative candidate sample area and the classification confidence of the candidate sample, the influence degree of the reference sample area under the candidate sample area includes:
if the candidate sample region is determined to belong to the positive sample region, adopting a formula
Calculating to obtain a difficulty routine degree score of a difficulty sample in the image under the candidate sample area;
if the candidate sample region is determined to belong to the negative sample region, adopting the formula
Calculating to obtain a difficulty routine degree score of a difficulty sample in the image under the candidate sample area;
Wherein C is i Representing the confidence of the classification of the ith candidate sample in the image, B i Representing the position coordinates of the ith candidate sample in the image, I being the total number of difficult samples, HP k Position coordinates of the kth positive sample, HN k The position coordinates of the kth sample, iof (B, HN), represent the area ratio of the sample area of the sample in the candidate sample area.
On the basis of the foregoing embodiment, optionally, determining loss weight information of the candidate sample area according to a value of an influence degree of the reference sample area under the candidate sample area includes:
ignoring in the image under the candidate sample areaThe ignorance degree score of the sample and the difficulty routine degree score of the difficult sample in the image are calculated by adopting a formula LW=1-S ignore +S hard Calculating to obtain loss weight information of the candidate sample area;
wherein S is ignore A degree of omission score representing the omitted samples in the image under the candidate sample region, S hard A difficulty routine score representing a difficulty sample in the image under the candidate sample region.
The training device for the target detection model provided in the embodiment of the invention can execute the training method for the target detection model provided in any embodiment of the invention, has the corresponding functions and beneficial effects of executing the training method for the target detection model, and the detailed process refers to the related operation of the training method for the target detection model in the embodiment.
Example IV
Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention. As shown in fig. 4, an electronic device provided in an embodiment of the present invention includes: one or more processors 410 and a storage 420; the number of processors 410 in the electronic device may be one or more, one processor 410 being illustrated in fig. 4; the storage 420 is used for storing one or more programs; the one or more programs are executed by the one or more processors 410 to cause the one or more processors 410 to implement a method of training a target detection model according to any of the embodiments of the present invention.
The electronic device may further include: an input device 430 and an output device 440.
The processor 410, the memory device 420, the input device 430, and the output device 440 in the electronic device may be connected by a bus or other means, which is illustrated in fig. 4 as being connected by a bus 450.
The storage 420 in the electronic device is used as a computer readable storage medium, and may be used to store one or more programs, which may be software programs, computer executable programs, and modules, such as program instructions/modules corresponding to the training method of the object detection model provided in the embodiments of the present invention. The processor 410 executes various functional applications of the electronic device and data processing by running software programs, instructions and modules stored in the storage 420, i.e. implements the training method of the object detection model in the above-described method embodiment.
The storage 420 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, at least one application program required for a function; the storage data area may store data created according to the use of the electronic device, etc. In addition, the storage 420 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, the storage 420 may further include memory remotely located with respect to the processor 410, which may be connected to the device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 430 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the electronic device. The output 440 may include a display device such as a display screen.
And, when one or more programs included in the above-described electronic device are executed by the one or more processors 410, the programs perform the following operations:
Classifying and regressing candidate sample areas in the image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image;
determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model.
Of course, those skilled in the art will appreciate that the program(s) may also perform the associated operations in the training method of the object detection model provided in any of the embodiments of the present invention when the program(s) included in the electronic device are executed by the processor(s) 410.
Example five
In an embodiment of the present invention, there is provided a computer-readable medium having stored thereon a computer program for executing a training method of an object detection model when executed by a processor, the method comprising:
Classifying and regressing candidate sample areas in the image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image;
determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model.
The computer storage media of embodiments of the invention may take the form of any combination of one or more computer-readable media. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access Memory (Random Access Memory, RAM), a Read-Only Memory (ROM), an erasable programmable Read-Only Memory (Erasable Programmable Read Only Memory, EPROM), a flash Memory, an optical fiber, a portable CD-ROM, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. A computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to: electromagnetic signals, optical signals, or any suitable combination of the preceding. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, radio frequency (RadioFrequency, RF), and the like, or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiments or examples. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.
Claims (10)
1. A method of training a target detection model, the method comprising:
classifying and regressing candidate sample areas in the image through forward propagation of a target detection network to obtain classification confidence and regression positions of candidate samples in the image;
determining loss weight information adopted by the candidate sample region in the model training process according to the classification confidence coefficient and regression position of the candidate sample and the position of the reference sample region; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
and (3) carrying out back propagation according to the loss weight information adopted by the candidate sample area, and controlling the parameters of the target detection network to adjust towards the directions of suppressing the neglected sample and enhancing the difficult sample to obtain a trained target detection model.
2. The method of claim 1, wherein upon single-stage target detection, the target detection network comprises a single-stage convolutional neural network, and the candidate sample region comprises an anchor frame region in the image; and, upon two-stage target detection, the target detection network comprises a convolutional neural network of a second stage of the two-stage target detection, and the candidate sample region comprises a sample region that is forward propagated output through the candidate region network of the first stage of the two-stage target detection.
3. The method according to claim 1, wherein the neglecting of samples comprises ambiguous targets previously noted in the image prior to training that were lost in their original characteristics due to angle, distortion, occlusion and/or imaging noise problems, resulting in an inability of human judgment; the difficult sample comprises targets which are marked in the image in advance before training, and the model miss detection probability or false detection probability is larger than a preset probability value when the model is actually used, wherein the targets with the model miss detection probability larger than the preset miss detection probability value are marked as difficult positive samples, and the targets with the model false detection probability larger than the preset false detection probability value are marked as difficult negative samples.
4. The method of claim 1, wherein determining loss weight information employed by the candidate sample region during model training based on classification confidence and regression locations of the candidate sample and locations of reference sample regions comprises: calculating the area ratio of the intersection area of the candidate sample area and the reference sample area in the image relative to the candidate sample area according to the regression position of the candidate sample and the position of the reference sample area; determining the influence degree value of the reference sample region under the candidate sample region according to the region duty ratio of the relative candidate sample region and the classification confidence of the candidate sample; the influence degree value comprises an neglect degree score of a neglect sample in the image and a difficulty routine degree score of a difficult sample in the image;
And determining loss weight information of the candidate sample region according to the influence degree value of the reference sample region under the candidate sample region.
5. The method of claim 4, wherein when the reference sample region is an ignore sample region, determining the impact level of the reference sample region under the candidate sample region based on the region duty cycle of the relative candidate sample region and the classification confidence of the candidate sample comprises:
if the candidate sample region is determined to belong to the positive sample region, adopting a formula
Calculating to obtain an neglect degree score of a neglect sample in the image under the candidate sample area;
if the candidate sample region is determined to belong to the negative sample region, adopting the formula
Calculating to obtain an neglect degree score of a neglect sample in the image under the candidate sample area;
wherein C is i Representing the confidence of the classification of the ith candidate sample in the image, B i Representing the position coordinates of the ith candidate sample in the image, n is the total number of ignored boxes, I j The position coordinates of the j-th ignored sample in the image are denoted iof (B, I) as the area duty ratio of the ignored sample area in the candidate sample area.
6. The method of claim 4, wherein when the reference sample region is a refractory sample region, determining the impact level of the reference sample region under the candidate sample region based on the region duty cycle of the relative candidate sample region and the classification confidence of the candidate sample comprises:
If the candidate sample region is determined to belong to the positive sample region, adopting a formula
Calculating to obtain a difficulty routine degree score of a difficulty sample in the image under the candidate sample area;
if the candidate sample region is determined to belong to the negative sample region, adopting the formula
Calculating to obtain a difficulty routine degree score of a difficulty sample in the image under the candidate sample area;
wherein C is i Representing the confidence of the classification of the ith candidate sample in the image, B i Representing the position coordinates of the ith candidate sample in the image, I being the total number of difficult samples, HP k For the position coordinates of the kth refractory positive sample, iof (B, HP) is expressed as the area ratio of the refractory positive sample area in the candidate sample area, HN k The position coordinates of the kth negative sample are denoted iof (B, HN) as the area ratio of the negative sample area in the candidate sample area.
7. The method of claim 4, wherein determining loss weight information for the candidate sample region based on the impact level value of the reference sample region under the candidate sample region comprises:
according to the ignoring degree score of the neglected sample in the image under the candidate sample area and the difficulty routine degree score of the difficult sample in the image, adopting a formula LW=1-S ignore +S hard Calculating to obtain loss weight information of the candidate sample area;
wherein S is ignore A degree of omission score representing the omitted samples in the image under the candidate sample region, S hard A difficulty routine score representing a difficulty sample in the image under the candidate sample region.
8. A training apparatus for a target detection model, the apparatus comprising:
the classification confidence coefficient and regression position acquisition module is used for classifying and regressing candidate sample areas in the image through forward propagation of the target detection network to obtain classification confidence coefficient and regression positions of candidate samples in the image;
the loss weight information acquisition module is used for determining loss weight information adopted by the candidate sample area in the model training process according to the classification confidence degree and regression position of the candidate sample and the position of the reference sample area; the reference sample area comprises an neglected sample area corresponding to the fuzzy target in the image and a difficult sample area in the image;
the target detection model acquisition module is used for carrying out back propagation according to the loss weight information adopted by the candidate sample area, controlling the parameters of the target detection network to adjust towards the directions of suppressing neglected samples and enhancing difficult samples, and obtaining the trained target detection model.
9. An electronic device, comprising:
one or more processing devices;
a storage means for storing one or more programs;
when the one or more programs are executed by the one or more processing devices, the one or more processing devices are caused to implement the method of training the object detection model of any of claims 1-7.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processing means, implements a method for training a target detection model according to any one of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111048969.2A CN113780277B (en) | 2021-09-08 | 2021-09-08 | Training method and device of target detection model, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111048969.2A CN113780277B (en) | 2021-09-08 | 2021-09-08 | Training method and device of target detection model, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113780277A CN113780277A (en) | 2021-12-10 |
| CN113780277B true CN113780277B (en) | 2023-06-30 |
Family
ID=78841701
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111048969.2A Active CN113780277B (en) | 2021-09-08 | 2021-09-08 | Training method and device of target detection model, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113780277B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115359308B (en) * | 2022-04-06 | 2024-02-13 | 北京百度网讯科技有限公司 | Model training method, device, equipment, storage medium and program for identifying difficult cases |
| CN115294536B (en) * | 2022-08-10 | 2023-07-28 | 北京百度网讯科技有限公司 | Violation detection method, device, equipment and storage medium based on artificial intelligence |
| CN115131655B (en) * | 2022-09-01 | 2022-11-22 | 浙江啄云智能科技有限公司 | Training method and device of target detection model and target detection method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108416287A (en) * | 2018-03-04 | 2018-08-17 | 南京理工大学 | A kind of pedestrian detection method excavated based on omission negative sample |
| CN109409517A (en) * | 2018-09-30 | 2019-03-01 | 北京字节跳动网络技术有限公司 | The training method and device of object detection network |
| CN109800778A (en) * | 2018-12-03 | 2019-05-24 | 浙江工业大学 | A kind of Faster RCNN object detection method for dividing sample to excavate based on hardly possible |
| CN110942446A (en) * | 2019-10-17 | 2020-03-31 | 付冲 | Pulmonary nodule automatic detection method based on CT image |
| CN111091105A (en) * | 2019-12-23 | 2020-05-01 | 郑州轻工业大学 | Remote sensing image target detection method based on new frame regression loss function |
| CN112507996A (en) * | 2021-02-05 | 2021-03-16 | 成都东方天呈智能科技有限公司 | Face detection method of main sample attention mechanism |
| CN113033719A (en) * | 2021-05-27 | 2021-06-25 | 浙江啄云智能科技有限公司 | Target detection processing method, device, medium and electronic equipment |
| CN113221947A (en) * | 2021-04-04 | 2021-08-06 | 青岛日日顺乐信云科技有限公司 | Industrial quality inspection method and system based on image recognition technology |
-
2021
- 2021-09-08 CN CN202111048969.2A patent/CN113780277B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108416287A (en) * | 2018-03-04 | 2018-08-17 | 南京理工大学 | A kind of pedestrian detection method excavated based on omission negative sample |
| CN109409517A (en) * | 2018-09-30 | 2019-03-01 | 北京字节跳动网络技术有限公司 | The training method and device of object detection network |
| CN109800778A (en) * | 2018-12-03 | 2019-05-24 | 浙江工业大学 | A kind of Faster RCNN object detection method for dividing sample to excavate based on hardly possible |
| CN110942446A (en) * | 2019-10-17 | 2020-03-31 | 付冲 | Pulmonary nodule automatic detection method based on CT image |
| CN111091105A (en) * | 2019-12-23 | 2020-05-01 | 郑州轻工业大学 | Remote sensing image target detection method based on new frame regression loss function |
| CN112507996A (en) * | 2021-02-05 | 2021-03-16 | 成都东方天呈智能科技有限公司 | Face detection method of main sample attention mechanism |
| CN113221947A (en) * | 2021-04-04 | 2021-08-06 | 青岛日日顺乐信云科技有限公司 | Industrial quality inspection method and system based on image recognition technology |
| CN113033719A (en) * | 2021-05-27 | 2021-06-25 | 浙江啄云智能科技有限公司 | Target detection processing method, device, medium and electronic equipment |
Non-Patent Citations (3)
| Title |
|---|
| UnitBox: An Advanced Object Detection Network;Jiahui Yu;《第24届ACM论文集》;516-520 * |
| 一种上下文敏感的多尺度人脸检测方法;陈龙;《激光与光电子学进展》;第56卷(第4期);041003-1-10 * |
| 基于难分样本挖掘的快速区域卷积神经网络目标检测研究;张 烨;《电 子 与 信 息 学 报》;第41卷(第6期);1496-1502 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113780277A (en) | 2021-12-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113780277B (en) | Training method and device of target detection model, electronic equipment and storage medium | |
| US11878433B2 (en) | Method for detecting grasping position of robot in grasping object | |
| CN108229489B (en) | Key point prediction method, network training method, image processing method, device and electronic equipment | |
| CN108122234B (en) | Convolutional neural network training and video processing method and device and electronic equipment | |
| US20180196158A1 (en) | Inspection devices and methods for detecting a firearm | |
| CN108229675B (en) | Neural network training method, object detection method, device and electronic equipment | |
| CN110276293B (en) | Lane line detection method, lane line detection device, electronic device, and storage medium | |
| CN107622501B (en) | Boundary detection method for medical image | |
| CN111899244B (en) | Image segmentation method, network model training method, device and electronic equipment | |
| CN108320306B (en) | Video target tracking method fusing TLD and KCF | |
| CN111079638A (en) | Target detection model training method, device and medium based on convolutional neural network | |
| CN113963148B (en) | Object detection method, object detection model training method and device | |
| CN115631397A (en) | Target detection method and device based on bimodal image | |
| CN115343685A (en) | Multi-dimensional ground penetrating radar detection method, device and equipment applied to disease identification | |
| CN116802683A (en) | Image processing method and system | |
| CN117372928A (en) | Video target detection method and device and related equipment | |
| CN110443812B (en) | Fundus image segmentation method, device, apparatus, and medium | |
| CN113947771B (en) | Image recognition method, apparatus, device, storage medium, and program product | |
| CN114022509B (en) | Target tracking method based on monitoring video of multiple animals and related equipment | |
| CN116052288A (en) | Living body detection model training method, living body detection device and electronic equipment | |
| CN112530554B (en) | Scanning positioning method and device, storage medium and electronic equipment | |
| CN112116608B (en) | Guide wire segmentation method and device, electronic equipment and storage medium | |
| CN115147469A (en) | Registration method, device, equipment and storage medium | |
| CN114120423A (en) | Face image detection method and device, electronic equipment and computer readable medium | |
| CN113936158A (en) | Label matching method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |