CN117475232A - Weak supervision target detection model evaluation method and related device - Google Patents
Weak supervision target detection model evaluation method and related device Download PDFInfo
- Publication number
- CN117475232A CN117475232A CN202311492775.0A CN202311492775A CN117475232A CN 117475232 A CN117475232 A CN 117475232A CN 202311492775 A CN202311492775 A CN 202311492775A CN 117475232 A CN117475232 A CN 117475232A
- Authority
- CN
- China
- Prior art keywords
- detection model
- bpr
- evaluation
- target detection
- average value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 113
- 238000001514 detection method Methods 0.000 title claims abstract description 103
- 238000012360 testing method Methods 0.000 claims abstract description 55
- 238000000034 method Methods 0.000 claims abstract description 45
- 230000008569 process Effects 0.000 claims abstract description 21
- 238000012216 screening Methods 0.000 claims abstract description 19
- 238000012549 training Methods 0.000 claims description 24
- 238000004364 calculation method Methods 0.000 claims description 19
- 230000006870 function Effects 0.000 claims description 10
- 238000012163 sequencing technique Methods 0.000 claims description 7
- 230000000694 effects Effects 0.000 abstract description 5
- 239000013256 coordination polymer Substances 0.000 description 5
- 230000008878 coupling Effects 0.000 description 3
- 238000010168 coupling process Methods 0.000 description 3
- 238000005859 coupling reaction Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000012854 evaluation process Methods 0.000 description 2
- 238000002372 labelling Methods 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
- QYCUDRKEQFHGDK-UHFFFAOYSA-N spiro[1,3-oxazolidine-5,9'-fluorene]-2,4-dione Chemical compound O1C(=O)NC(=O)C21C1=CC=CC=C1C1=CC=CC=C12 QYCUDRKEQFHGDK-UHFFFAOYSA-N 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The application discloses a weak supervision target detection model evaluation method and a related device, wherein the method comprises the following steps: in the process of testing the current weakly supervised target detection model by adopting a preset test data set, analyzing the classification accuracy average value and the classification recall average value corresponding to each object category in a positive case screening mode of a candidate area; calculating the feature similarity of the candidate region and the corresponding background region of each object class according to a preset consistency function to obtain a feature level similarity value; drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, wherein the size of each point on the BPR evaluation curve is the feature level similarity value; and performing performance evaluation on the current weak supervision target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result. The method and the device can solve the technical problem that the actual evaluation effect is poor due to the fact that the existing evaluation index is not fully suitable for performance evaluation of the weak supervision target detection model.
Description
Technical Field
The application relates to the technical field of weak supervision target detection, in particular to a weak supervision target detection model evaluation method and a related device.
Background
The mainstream approach to target detection is a fully supervised target detection algorithm based on deep learning, which requires a large amount of data with object class and position labels to train an efficient target detector. However, acquiring such data is time consuming and labor intensive, and may present a problem of labeling errors or inconsistencies. Thus, how to train a target detector with less or weaker supervisory information is a challenging and practical problem. Weak Supervised Object Detection (WSOD) is a method of training an object detector with only image class labels. Unlike the fully supervised target detection algorithm (FSOD), which requires instance level labeling, the WSOD algorithm can utilize a large amount of inexpensive and readily available data to improve target detection performance, but also faces three challenges: local focus problems, examples distinguish between blur problems and speed problems.
Therefore, the prior art adopts some evaluation indexes to evaluate the performance of the WSOD algorithm, and the problem of the application of the technology in the actual scene is relieved. However, when indexes such as average accuracy mAP are adopted to measure the performance effect of the WSOD algorithm on the public data set, or the average accuracy mAP is not completely suitable for evaluating the performance of the WSOD algorithm.
Disclosure of Invention
The application provides a weak supervision target detection model evaluation method and a related device, which are used for solving the technical problem that the actual evaluation effect is poor because evaluation indexes provided by the prior art are not completely suitable for performance evaluation of a weak supervision target detection model.
In view of this, a first aspect of the present application provides a method for evaluating a weakly supervised object detection model, including:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, analyzing a classification accuracy average value and a classification recall average value corresponding to each object category in a positive case screening mode of a candidate area;
calculating the feature similarity of the candidate region and the corresponding background region of each object class according to a preset consistency function to obtain a feature level similarity value;
drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, wherein the size of each point on the BPR evaluation curve is the feature level similarity value;
and performing performance evaluation on the current weakly supervised target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result.
Preferably, in the process of testing the current weakly supervised target detection model by using the preset test data set, analyzing the classification accuracy average value and the classification recall average value corresponding to each object category by a positive case screening mode of the candidate region, including:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, sequencing candidate areas corresponding to each object category in each image according to the candidate scores to obtain a candidate area sequence;
screening out positive example areas with the candidate scores larger than a score threshold value from the candidate area sequences;
based on the positive example region, calculating the sub-classification accuracy and the sub-classification recall of each image according to the object category set and the candidate region set;
and respectively calculating the average value according to the sub-classification accuracy and the sub-classification recall rate to obtain a classification accuracy average value and a classification recall rate average value of each object class.
Preferably, in the process of testing the current weakly supervised target detection model by using the preset test data set, analyzing the classification accuracy average value and the classification recall average value corresponding to each object category by a positive case screening mode of the candidate region, the method further includes:
acquiring a preset test data set, a preset training data set and an initial weak supervision target detection model;
and carrying out model training on the initial weakly-supervised target detection model by adopting the preset training data set to obtain a current weakly-supervised target detection model.
Preferably, the performance evaluation of the current weakly supervised target detection model is performed according to the total BPR area calculated by the BPR evaluation curve, so as to obtain an evaluation result, including:
dividing a small rectangle corresponding to each object category on the BPR evaluation curve, and calculating the area of the small rectangle;
calculating the weight value of the small rectangle according to the parameter value of each point on the small rectangle;
calculating a class BPR area corresponding to each object class based on the small rectangular area and the weight value;
performing average value calculation according to the class BPR areas corresponding to all the object classes to obtain a total BPR area;
and performing performance evaluation on the current weak supervision target detection model according to the total BPR area to obtain an evaluation result.
A second aspect of the present application provides a weak supervision objective detection model evaluation device, including:
the test calculation unit is used for analyzing the classification accuracy average value and the classification recall average value corresponding to each object category in a positive case screening mode of the candidate area in the process of testing the current weak supervision target detection model by adopting a preset test data set;
the similarity calculation unit is used for calculating the feature similarity of the candidate region and the corresponding background region of each object category according to a preset consistency function to obtain a feature level similarity value;
the curve drawing unit is used for drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, and the size of each point on the BPR evaluation curve is the feature level similarity value;
and the performance evaluation unit is used for performing performance evaluation on the current weak supervision target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result.
Preferably, the test calculation unit is specifically configured to:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, sequencing candidate areas corresponding to each object category in each image according to the candidate scores to obtain a candidate area sequence;
screening out positive example areas with the candidate scores larger than a score threshold value from the candidate area sequences;
based on the positive example region, calculating the sub-classification accuracy and the sub-classification recall of each image according to the object category set and the candidate region set;
and respectively calculating the average value according to the sub-classification accuracy and the sub-classification recall rate to obtain a classification accuracy average value and a classification recall rate average value of each object class.
Preferably, the method further comprises:
the acquisition unit is used for acquiring a preset test data set, a preset training data set and an initial weak supervision target detection model;
and the training unit is used for carrying out model training on the initial weak supervision target detection model by adopting the preset training data set to obtain a current weak supervision target detection model.
Preferably, the performance evaluation unit is specifically configured to:
dividing a small rectangle corresponding to each object category on the BPR evaluation curve, and calculating the area of the small rectangle;
calculating the weight value of the small rectangle according to the parameter value of each point on the small rectangle;
calculating a class BPR area corresponding to each object class based on the small rectangular area and the weight value;
performing average value calculation according to the class BPR areas corresponding to all the object classes to obtain a total BPR area;
and performing performance evaluation on the current weak supervision target detection model according to the total BPR area to obtain an evaluation result.
A third aspect of the present application provides a weakly supervised object detection model evaluation apparatus, the apparatus comprising a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to execute the weak supervision objective detection model assessment method according to the first aspect according to the instructions in the program code.
A fourth aspect of the present application provides a computer readable storage medium for storing program code for performing the weak supervision objective detection model assessment method of the first aspect.
From the above technical solutions, the embodiments of the present application have the following advantages:
in the present application, a method for evaluating a weakly supervised target detection model is provided, including: in the process of testing the current weak supervision target detection model by adopting a preset test data set, analyzing a classification accuracy average value and a classification recall average value corresponding to each object category in a positive case screening mode of a candidate area; calculating the feature similarity of the candidate region and the corresponding background region of each object class according to a preset consistency function to obtain a feature level similarity value; drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, wherein the size of each point on the BPR evaluation curve is the feature level similarity value; and performing performance evaluation on the current weak supervision target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result.
According to the weak supervision target detection model evaluation method, the classification accuracy average value and the classification recall average value in the model test process are calculated and analyzed, the consistency association relation between the candidate area and the background area is accurately reflected through the feature level similarity value, the parameter index for evaluation is ensured to adapt to the characteristic of the weak supervision target detection model, and further the performance evaluation result obtained based on the characteristic index can be ensured to be more accurate and reliable. Therefore, the method and the device can solve the technical problem that the actual evaluation effect is poor because the evaluation index provided by the prior art is not fully suitable for performance evaluation of the weak supervision target detection model.
Drawings
Fig. 1 is a schematic flow chart of a weak supervision objective detection model evaluation method according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of a weak supervision objective detection model evaluation device according to an embodiment of the present disclosure.
Detailed Description
In order to make the present application solution better understood by those skilled in the art, the following description will clearly and completely describe the technical solution in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
For ease of understanding, referring to fig. 1, an embodiment of a method for evaluating a weakly supervised object detection model provided in the present application includes:
and step 101, analyzing the classification accuracy average value and the classification recall average value corresponding to each object category in a positive case screening mode of the candidate region in the process of testing the current weak supervision target detection model by adopting a preset test data set.
Further, step 101 includes:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, sequencing candidate areas corresponding to each object category in each image according to the candidate scores to obtain a candidate area sequence;
screening out positive example areas with candidate scores larger than a score threshold value from the candidate area sequences;
based on the positive example region, calculating the subcategory accuracy and subcategory recall of each image according to the object category set and the candidate region set;
and respectively calculating the average value according to the sub-classification accuracy and the sub-classification recall rate to obtain the classification accuracy average value and the classification recall rate average value of each object class.
Further, step 101, before further includes:
acquiring a preset test data set, a preset training data set and an initial weak supervision target detection model;
and carrying out model training on the initial weakly-supervised target detection model by adopting a preset training data set to obtain the current weakly-supervised target detection model.
It should be noted that, the preset test data set and the preset training data set may be acquired and processed in advance according to actual situations, and the current weak supervision target detection model denoted as M may be obtained after the initial weak supervision target detection model is pre-trained by the preset training data set.
When the current weakly supervised target detection model M is tested by adopting the preset test data set D, the detection result of each object category in each image can be obtained, the average precision AP in the test process can be calculated based on the detection result, the average precision consists of precision and recall rate, and the specific calculation is as follows:
where recovery is recall, precision is accuracy, TP represents the number of correctly predicted samples in all positive samples, FP represents the number of incorrectly predicted samples in all positive samples, FN represents the number of incorrectly predicted samples in all negative samples, and TN represents the number of correctly predicted samples in all negative samples. In the evaluation process of mAP indexes in the prior art, PR curves can be drawn based on accuracy and recall, wherein accuracy is taken as a vertical axis, recall is taken as a horizontal axis, and average value calculation is carried out on accuracy values on the PR curves, so that the PR curves can be obtained:
wherein p (r) is PR curve expression; the mean value can then be calculated:
wherein C ε C is the number of object categories.
In this embodiment, the classification accuracy average CP and the classification recall average CR corresponding to each object class in each image are also calculated first. Before this, some test data need to be clarified, if the preset test data set D includes N images, C object class sets, K object classes, and for each image i e D, it can be assumed that Y i A set of object categories present in image i; for each object class C ε C, it can be assumed that X ic Predicting a candidate region set belonging to the object class c from the model in the image i, S ic A set of candidate scores for candidate regions belonging to the object class c is predicted from the model for image i, and each candidate region corresponds to a candidate score.
Based on the above, for each object class c, a candidate score S can be obtained ic And sequencing the candidate areas of each image i epsilon D to obtain a candidate area sequence, wherein the candidate area in the front of the sequence can be the area with higher candidate score. Then, a score threshold value can be set according to actual conditions, region screening is conducted on the candidate region sequence based on score preset, and candidate regions larger than the score threshold value are reserved as positive example regions. Then can be based on the object class according to the set Y i And candidate region set X ic Calculate each graphSub-classification accuracy CP of image i epsilon D i And sub-category recall CR i All images calculate the corresponding sub-classification accuracy CP i And sub-category recall CR i Then, obtaining the average value of the classification accuracy rate of the current object class c by averaging c And categorized recall mean CR c 。
And 102, calculating the feature similarity of the candidate region of each object category and the corresponding background region according to a preset consistency function to obtain a feature level similarity value.
The preset consistency function may be expressed as:
F(f 1 ,f 2 )＝cos(f 1 ,f 2 )
wherein f 1 、f 2 And respectively extracting the characteristics of the candidate region and the characteristics of the surrounding background region. In particular, it may be determined that the convolutional network in the model is a feature extractor, such as the resnet34 of torch; then dividing the candidate region into 8 regions according to the size of the candidate region, extracting features from the candidate region by using a resnet34, extracting average features from the peripheral region, and calculating cosine similarity between the two types of features to obtain a feature level similarity value F of the candidate region and the corresponding background region.
In this embodiment, for each object class c, a preset consistency function is adopted to calculate feature level similarity between a positive example region and an image background region in a candidate region corresponding to each image i e D, and then the minimum feature set similarity is selected as a feature level similarity value BC to be calculated for the current positive example region i 。
And step 103, drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, wherein the size of each point on the BPR evaluation curve is the feature level similarity value.
Similar to the evaluation process of the mAP index in the prior art, the embodiment also needs to draw a curve based on the parameter value obtained by calculation, takes the classification accuracy CP as the vertical axis and the classification recall CR as the horizontal axis, and each point on the drawn BPR evaluation curve corresponds to one image, and the point is largeThe small feature set similarity value BC. The BPR evaluation curve can intuitively display the performance of the weakly supervised target detection model in terms of classification and consistency of background areas and the influence relation among different factors. It will be appreciated that for each object class c, the CP can be determined c 、CR c And BC (binary code) c Drawing corresponding BPR evaluation curves, and obtaining the number of the BPR evaluation curves by the number of the object categories.
And 104, performing performance evaluation on the current weakly supervised target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result.
Further, step 104 includes:
dividing a small rectangle corresponding to each object category on the BPR evaluation curve, and calculating the area of the small rectangle;
calculating the weight value of the small rectangle according to the parameter values of all points on the small rectangle;
calculating a class BPR area corresponding to each object class based on the small rectangular area and the weight value;
performing average value calculation according to the class BPR areas corresponding to all object classes to obtain a total BPR area;
and performing performance evaluation on the current weak supervision target detection model according to the total BPR area to obtain an evaluation result.
For each object class c, small rectangular segmentation can be performed according to the corresponding BPR evaluation curve, and then the small rectangular area is calculated; parameter values of a plurality of corresponding points, such as an accuracy parameter value LA, a recall parameter value CA, a feature similarity parameter value BC and the like, can be obtained on the basis of the curve on the small rectangle; based on these parameters, a weight value for each small rectangle can be calculated; weighting calculation is carried out according to the small rectangular area and the weight value, so that the class BPR area WSAP of each object class c can be obtained c 。
Class BPR area WSAP for all object classes c The average value calculation is carried out, so that the total BPR area WSAP can be obtained; the performance of the current weakly supervised target detection model is mainly analyzed by adopting the total BPR area, and the larger the total BPR area is, the description is thatThe better the front weak supervision target detection model is, the better the performance is.
According to the weak supervision target detection model evaluation method, the classification accuracy average value and the classification recall average value in the model test process are calculated and analyzed, the consistency association relation between the candidate area and the background area is accurately reflected through the feature level similarity value, the parameter index for evaluation is ensured to adapt to the characteristic of the weak supervision target detection model, and further the performance evaluation result obtained based on the characteristic index can be ensured to be more accurate and reliable. Therefore, the embodiment of the application can solve the technical problem that the evaluation index proposed by the prior art is not completely suitable for performance evaluation of the weak supervision target detection model, so that the actual evaluation effect is poor.
For ease of understanding, referring to fig. 2, the present application provides an embodiment of a weak supervision object detection model evaluation device, including:
the test calculation unit 201 is configured to analyze, in a process of testing the current weakly supervised target detection model by using a preset test data set, a classification accuracy average value and a classification recall average value corresponding to each object category through a positive case screening manner of the candidate region;
a similarity calculation unit 202, configured to calculate feature similarity between the candidate region and the corresponding background region of each object class according to a preset consistency function, so as to obtain a feature level similarity value;
a curve drawing unit 203, configured to draw a BPR evaluation curve based on the classification accuracy average value, the classification recall average value, and the feature level similarity value, where the size of each point on the BPR evaluation curve is the feature level similarity value;
and the performance evaluation unit 204 is configured to perform performance evaluation on the current weakly supervised target detection model according to the total BPR area calculated by the BPR evaluation curve, so as to obtain an evaluation result.
Further, the test calculation unit 201 is specifically configured to:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, sequencing candidate areas corresponding to each object category in each image according to the candidate scores to obtain a candidate area sequence;
screening out positive example areas with candidate scores larger than a score threshold value from the candidate area sequences;
based on the positive example region, calculating the subcategory accuracy and subcategory recall of each image according to the object category set and the candidate region set;
and respectively calculating the average value according to the sub-classification accuracy and the sub-classification recall rate to obtain the classification accuracy average value and the classification recall rate average value of each object class.
Further, the method further comprises the following steps:
an obtaining unit 205, configured to obtain a preset test data set, a preset training data set, and an initial weakly supervised target detection model;
the training unit 206 is configured to perform model training on the initial weakly supervised target detection model by using a preset training data set, so as to obtain a current weakly supervised target detection model.
Further, the performance evaluation unit 204 is specifically configured to:
dividing a small rectangle corresponding to each object category on the BPR evaluation curve, and calculating the area of the small rectangle;
calculating the weight value of the small rectangle according to the parameter values of all points on the small rectangle;
calculating a class BPR area corresponding to each object class based on the small rectangular area and the weight value;
performing average value calculation according to the class BPR areas corresponding to all object classes to obtain a total BPR area;
and performing performance evaluation on the current weak supervision target detection model according to the total BPR area to obtain an evaluation result.
The application also provides a weak supervision target detection model evaluation device, which comprises a processor and a memory;
the memory is used for storing the program codes and transmitting the program codes to the processor;
the processor is configured to execute the weakly supervised object detection model evaluation method of the method embodiment described above based on instructions in the program code.
The application also provides a computer readable storage medium for storing program code for executing the weak supervision objective detection model evaluation method in the above method embodiment.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative, e.g., the division of the units is merely a logical function division, and there may be additional divisions when actually implemented, e.g., multiple units or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or units, which may be in electrical, mechanical or other form.
The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
In addition, each functional unit in each embodiment of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit. The integrated units may be implemented in hardware or in software functional units.
The integrated units, if implemented in the form of software functional units and sold or used as stand-alone products, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be embodied in essence or a part contributing to the prior art or all or part of the technical solution in the form of a software product stored in a storage medium, including several instructions to execute all or part of the steps of the methods described in the embodiments of the present application by a computer device (which may be a personal computer, a server, or a network device, etc.). And the aforementioned storage medium includes: u disk, mobile hard disk, read-Only Memory (ROM), random access Memory (RandomAccess Memory, RAM), magnetic disk or optical disk, etc.
The above embodiments are merely for illustrating the technical solution of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the corresponding technical solutions.
Claims (10)
1. A method for evaluating a weakly supervised object detection model, comprising:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, analyzing a classification accuracy average value and a classification recall average value corresponding to each object category in a positive case screening mode of a candidate area;
calculating the feature similarity of the candidate region and the corresponding background region of each object class according to a preset consistency function to obtain a feature level similarity value;
drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, wherein the size of each point on the BPR evaluation curve is the feature level similarity value;
and performing performance evaluation on the current weakly supervised target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result.
2. The method for evaluating a weakly supervised target detection model according to claim 1, wherein the analyzing the classification accuracy average and the classification recall average corresponding to each object class by the positive case screening method of the candidate region in the process of testing the current weakly supervised target detection model by using the preset test data set comprises:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, sequencing candidate areas corresponding to each object category in each image according to the candidate scores to obtain a candidate area sequence;
screening out positive example areas with the candidate scores larger than a score threshold value from the candidate area sequences;
based on the positive example region, calculating the sub-classification accuracy and the sub-classification recall of each image according to the object category set and the candidate region set;
and respectively calculating the average value according to the sub-classification accuracy and the sub-classification recall rate to obtain a classification accuracy average value and a classification recall rate average value of each object class.
3. The method for evaluating a weakly supervised target detection model according to claim 1, wherein in the process of testing the current weakly supervised target detection model by using the preset test data set, analyzing the classification accuracy average value and the classification recall average value corresponding to each object class by a positive case screening method of the candidate region, the method further comprises:
acquiring a preset test data set, a preset training data set and an initial weak supervision target detection model;
and carrying out model training on the initial weakly-supervised target detection model by adopting the preset training data set to obtain a current weakly-supervised target detection model.
4. The method for evaluating a weakly supervised target detection model as set forth in claim 1, wherein the performing performance evaluation on the current weakly supervised target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result includes:
dividing a small rectangle corresponding to each object category on the BPR evaluation curve, and calculating the area of the small rectangle;
calculating the weight value of the small rectangle according to the parameter value of each point on the small rectangle;
calculating a class BPR area corresponding to each object class based on the small rectangular area and the weight value;
performing average value calculation according to the class BPR areas corresponding to all the object classes to obtain a total BPR area;
and performing performance evaluation on the current weak supervision target detection model according to the total BPR area to obtain an evaluation result.
5. A weakly supervised object detection model evaluation apparatus, comprising:
the test calculation unit is used for analyzing the classification accuracy average value and the classification recall average value corresponding to each object category in a positive case screening mode of the candidate area in the process of testing the current weak supervision target detection model by adopting a preset test data set;
the similarity calculation unit is used for calculating the feature similarity of the candidate region and the corresponding background region of each object category according to a preset consistency function to obtain a feature level similarity value;
the curve drawing unit is used for drawing a BPR evaluation curve based on the classification accuracy average value, the classification recall average value and the feature level similarity value, and the size of each point on the BPR evaluation curve is the feature level similarity value;
and the performance evaluation unit is used for performing performance evaluation on the current weak supervision target detection model according to the total BPR area calculated by the BPR evaluation curve to obtain an evaluation result.
6. The weakly-supervised object detection model evaluation apparatus of claim 5, wherein the test computation unit is specifically configured to:
in the process of testing the current weak supervision target detection model by adopting a preset test data set, sequencing candidate areas corresponding to each object category in each image according to the candidate scores to obtain a candidate area sequence;
screening out positive example areas with the candidate scores larger than a score threshold value from the candidate area sequences;
based on the positive example region, calculating the sub-classification accuracy and the sub-classification recall of each image according to the object category set and the candidate region set;
and respectively calculating the average value according to the sub-classification accuracy and the sub-classification recall rate to obtain a classification accuracy average value and a classification recall rate average value of each object class.
7. The weakly-supervised object detection model evaluation apparatus of claim 5, further comprising:
the acquisition unit is used for acquiring a preset test data set, a preset training data set and an initial weak supervision target detection model;
and the training unit is used for carrying out model training on the initial weak supervision target detection model by adopting the preset training data set to obtain a current weak supervision target detection model.
8. The weakly-supervised object detection model evaluation apparatus of claim 5, wherein the performance evaluation unit is specifically configured to:
dividing a small rectangle corresponding to each object category on the BPR evaluation curve, and calculating the area of the small rectangle;
calculating the weight value of the small rectangle according to the parameter value of each point on the small rectangle;
calculating a class BPR area corresponding to each object class based on the small rectangular area and the weight value;
performing average value calculation according to the class BPR areas corresponding to all the object classes to obtain a total BPR area;
and performing performance evaluation on the current weak supervision target detection model according to the total BPR area to obtain an evaluation result.
9. A weakly supervised object detection model evaluation apparatus, the apparatus comprising a processor and a memory;
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the weakly supervised object detection model evaluation method of any one of claims 1-4 according to instructions in the program code.
10. A computer readable storage medium, characterized in that the computer readable storage medium is for storing a program code for performing the weak supervision objective detection model assessment method according to any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311492775.0A CN117475232A (en) | 2023-11-09 | 2023-11-09 | Weak supervision target detection model evaluation method and related device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311492775.0A CN117475232A (en) | 2023-11-09 | 2023-11-09 | Weak supervision target detection model evaluation method and related device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117475232A true CN117475232A (en) | 2024-01-30 |
Family
ID=89639418
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311492775.0A Pending CN117475232A (en) | 2023-11-09 | 2023-11-09 | Weak supervision target detection model evaluation method and related device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117475232A (en) |
-
2023
- 2023-11-09 CN CN202311492775.0A patent/CN117475232A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111898642B (en) | Key point detection method and device, electronic equipment and storage medium | |
| CN110363220B (en) | Behavior class detection method and device, electronic equipment and computer readable medium | |
| CN110569855B (en) | Long-time target tracking method based on correlation filtering and feature point matching fusion | |
| CN110991397B (en) | Travel direction determining method and related equipment | |
| CN112819821B (en) | Cell nucleus image detection method | |
| CN113763348A (en) | Image quality determination method and device, electronic equipment and storage medium | |
| CN116028870B (en) | Data detection method and device, electronic equipment and storage medium | |
| CN112116592B (en) | Image detection method, training method, device and medium of image detection model | |
| CN109189965A (en) | Pictograph search method and system | |
| CN112395382A (en) | Ship abnormal track data detection method and device based on variational self-encoder | |
| CN113516144A (en) | Target detection method and device and computing equipment | |
| CN113822336A (en) | Cloud hard disk fault prediction method, device and system and readable storage medium | |
| CN115713750B (en) | Lane line detection method and device, electronic equipment and storage medium | |
| CN117475232A (en) | Weak supervision target detection model evaluation method and related device | |
| CN111723719B (en) | Video target detection method, system and device based on category external memory | |
| CN112132239B (en) | Training method, device, equipment and storage medium | |
| CN111209567B (en) | Method and device for judging perceptibility of improving robustness of detection model | |
| CN112861689A (en) | Searching method and device of coordinate recognition model based on NAS technology | |
| CN110837805B (en) | Method, device and equipment for measuring confidence of video tag and storage medium | |
| CN116934747B (en) | Fundus image segmentation model training method, fundus image segmentation model training equipment and glaucoma auxiliary diagnosis system | |
| CN113537407B (en) | Image data evaluation processing method and device based on machine learning | |
| CN113473124B (en) | Information acquisition method, device, electronic equipment and storage medium | |
| CN112749625B (en) | Time sequence behavior detection method, time sequence behavior detection device and terminal equipment | |
| CN117649415B (en) | Cell balance analysis method based on optical flow diagram detection | |
| CN117557841A (en) | Open set target detection method and device applied to industrial detection |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20240315 Address after: Unit 1, Building 1, China Telecom Zhejiang Innovation Park, No. 8 Xiqin Street, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province, 311100 Applicant after: Tianyi Shilian Technology Co.,Ltd. Country or region after: China Address before: Room 1423, No. 1256 and 1258, Wanrong Road, Jing'an District, Shanghai 200072 Applicant before: Tianyi Digital Life Technology Co.,Ltd. Country or region before: China |