CN110969599B - Target detection algorithm performance overall evaluation method and system based on image attributes - Google Patents
Target detection algorithm performance overall evaluation method and system based on image attributes Download PDFInfo
- Publication number
- CN110969599B CN110969599B CN201911081611.2A CN201911081611A CN110969599B CN 110969599 B CN110969599 B CN 110969599B CN 201911081611 A CN201911081611 A CN 201911081611A CN 110969599 B CN110969599 B CN 110969599B
- Authority
- CN
- China
- Prior art keywords
- test set
- target
- image
- score
- quality
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0639—Performance analysis of employees; Performance analysis of enterprise or organisation operations
- G06Q10/06393—Score-carding, benchmarking or key performance indicator [KPI] analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The overall evaluation method and system for the performance of the target detection algorithm based on the image attributes comprise the following steps: selecting an original test set with a tested target type according to a to-be-tested algorithm; carrying out noise adding processing with different intensities on the original test set; dividing the test set subjected to noise addition into a plurality of quality grade data sets, and taking the plurality of quality grade data sets as an image quality test set; dividing a plurality of image resolution test sets, a plurality of target quality test sets and a plurality of target size test sets from an original test set; calculating mAP, and respectively obtaining an image quality score, an image resolution score, a target quality score and a target size score; and carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute. The method and the device realize comprehensive weighted evaluation of multiple indexes of the image attribute at the same time, and can comprehensively and comprehensively reflect the performance of the algorithm to be tested facing a noise map set.
Description
Technical Field
The invention relates to an image processing and target detection technology, in particular to a target detection algorithm performance overall evaluation method and system based on image attributes.
Background
In the evaluation of the traditional target detection algorithm, the accuracy and the recall rate are usually used to describe the quality of an algorithm, and the algorithms such as the mAP (mean Average Precision) and the F-Score can consider the two indexes and are widely applied. However, the general mAP or F-Score calculation method does not make any requirement or limitation on the image of the test set, and does not well combine multiple parameters for comprehensive evaluation, thereby affecting the evaluation accuracy, failing to more objectively and comprehensively compare different detection algorithms, and being necessary to be improved.
Disclosure of Invention
The invention mainly aims at the defects of the related prior art and provides a target detection algorithm performance overall evaluation method and system based on image attributes, wherein the image attributes comprise image quality, image resolution, image target quality and image target size, a test set is processed or divided according to the image attributes, comprehensive weighting evaluation on multiple indexes of the image attributes is realized simultaneously, the performance of an algorithm to be tested facing different attribute image sets can be comprehensively and comprehensively reflected, and the algorithm to be tested can be objectively and comprehensively evaluated conveniently.
In order to achieve the above object, the present invention employs the following techniques:
the overall evaluation method for the performance of the target detection algorithm based on the image attributes is characterized by comprising the following steps of:
selecting an original test set with a tested target type according to a to-be-tested algorithm;
carrying out noise adding and/or smoothing processing with different intensities on an original test set, obtaining a distance index of each image of the test set subjected to noise adding according to an image quality evaluation index, and dividing the test set subjected to noise adding into a plurality of quality grade data sets according to the distance indexes to be used as an image quality test set;
dividing the original test set into image resolution test sets of multiple levels according to the resolution;
dividing the original test set into a plurality of grades of target quality test sets according to the target quality difference;
dividing the original test set into a plurality of grades of target size test sets according to the target size;
testing the algorithm to be tested through an image quality test set, an image resolution test set, a target quality test set and a target size test set respectively, calculating mAP with different grades and different attributes respectively, and obtaining an image quality score, an image resolution score, a target quality score and a target size score respectively after weighting;
and carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute.
The image quality score is obtained by the following steps:
respectively scoring the image quality test sets of different grades to calculate the mAP;
weighting the mAP score by using the normalized weight to obtain an image quality score;
wherein, the normalization weight is obtained by the following steps:
setting the degree of similarity of different grades for the mAP grades of different grades;
obtaining corresponding mAP scores which are respectively needed in the same degree and time sharing according to different grades, and calculating the weight corresponding to each grade;
and carrying out normalization processing on the weight corresponding to each grade to obtain the normalization weight corresponding to each grade.
Wherein, the degree of the same degree of different grades is set for the mAP grades of different grades, which specifically comprises the following steps: the data sets with the highest quality grade to the data sets with the lowest quality grade are equally reduced step by step.
The target quality score is obtained by the following steps:
respectively scoring the multiple target quality test sets to calculate the mAP;
and weighting the mAP score by using the normalized weight to obtain a target quality score.
The image resolution score is obtained by the following steps: and respectively scoring the multiple image resolution test sets to calculate the mAP, and obtaining the image resolution score by using average weighting.
The target size score is obtained by the following steps: and respectively scoring the plurality of target size test sets to calculate the mAP, and obtaining a target size score by utilizing average weighting.
Further, noise adding and/or smoothing processing with different intensities is carried out on the test set, and the method comprises the following steps:
carrying out noise adding processing with different intensities on the test set, wherein the added noise comprises at least one of Gaussian noise, salt and pepper noise, poisson noise and speckle noise;
and/or the presence of a gas in the gas,
smoothing the test set with different intensities, wherein the smoothing comprises at least one of Gaussian blur, average blur and motion blur;
and increasing the number of the pictures in the processed test set by at least 5 times on the basis of the original test set by noise adding and/or smoothing treatment with different intensities.
An overall evaluation system for performance of an object detection algorithm based on image attributes is characterized by comprising the following steps:
the test preparation module is used for selecting an original test set with a tested target type according to a to-be-tested algorithm;
the test set processing module is used for carrying out noise addition and/or smoothing processing with different strengths on the original test set;
the test set grading module is used for obtaining a distance index of each image of the test set after noise addition according to the image quality evaluation index and dividing the test set after noise addition into a plurality of quality grade data sets according to the distance index;
the image quality test set generation module is used for taking the quality grade data sets as an image quality test set;
the image resolution test set generation module is used for dividing the original test set into a plurality of image resolution test sets according to the resolution;
the target quality test set generation module is used for dividing the original test set into a plurality of target quality test sets according to the target quality difference;
the target size test set module is used for dividing the original test set into a plurality of target size test sets according to the target size;
the grading module is used for testing the algorithm to be tested through the image quality test set, the image resolution test set, the target quality test set and the target size test set respectively, calculating mAP respectively and obtaining an image quality grade, an image resolution grade, a target quality grade and a target size grade respectively; and carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute.
The invention has the beneficial effects that:
1. by processing and grading the data set, comprehensive weighted evaluation on multiple indexes of the image attribute is realized, the performance of the algorithm to be tested facing the noise map set can be comprehensively and comprehensively reflected, and the algorithm to be tested can be evaluated more objectively and comprehensively.
2. The method/system combines the image quality, the image resolution, the target quality and the target size to perform comprehensive overall evaluation, wherein a normalization weight processing mode is adopted in the image quality and target quality evaluation, an average weight processing mode is adopted in the target quality and the target size, and an average weight processing mode is adopted in the overall evaluation, so that the method/system considers and combines factors of different image attributes more comprehensively, and is beneficial to more comprehensively, effectively and accurately evaluating the algorithm to be tested.
3. By processing the test set, the method/system is realized, the evaluation of the algorithm to be tested can be integrated on the basis of the multi-image attribute parameters, the performance of the algorithm to be tested in the presence of a noise map set can be distinguished, and for example, the result can be obtained: the algorithm to be tested performs well on a noise-free image, but is sensitive to the detection of a noise-added image target, large resolution, small target and shielding; therefore, the advantages and disadvantages of the algorithm to be tested can be distinguished, and a targeted training suggestion is provided for the algorithm to be tested: such as training of images that should be enhanced in the attributes of the above aspects.
Drawings
Fig. 1 is a flowchart of an overall evaluation method according to an embodiment of the present application.
Fig. 2 is a diagram illustrating image quality scoring according to an embodiment of the present application.
Fig. 3 is a schematic diagram of a normalization weight obtaining step in the embodiment of the present application.
Fig. 4 is a schematic diagram of a target quality scoring step according to an embodiment of the present application.
Fig. 5 is a diagram illustrating an image resolution scoring step according to an embodiment of the present application.
Fig. 6 is a schematic diagram of a target size scoring step according to an embodiment of the present application.
FIG. 7 is a diagram illustrating the processing steps of a test set according to an embodiment of the present application.
Fig. 8 is a block diagram of an overall evaluation system according to an embodiment of the present application.
Detailed description of the preferred embodiments
The following detailed description of embodiments of the method and system of the present application refers to the accompanying drawings.
Specifically, as an example of the present application, an implementation flow of the evaluation method is shown in fig. 1:
(1) And receiving an input algorithm to be tested.
(2) And selecting an original test set with the tested target type according to the algorithm to be tested.
(3) Preparation of each attribute test set:
image quality test set:
the original test set is subjected to noise addition and/or balance processing with different intensities, as shown in fig. 7:
adding one or more of Gaussian noise, salt-and-pepper noise, poisson noise and speckle noise with different intensities into the original test set;
and/or the presence of a gas in the gas,
and performing smoothing operation with different intensities on the original test set, such as one or more of Gaussian blur, average blur and motion blur.
By the noise adding and/or balancing treatment, the number is increased by at least 5 times based on the number of the original test sets.
Specifically, as shown in FIG. 7, the noise is added separately according to the flow of a-b 1-d; when the single smoothing treatment is carried out, the process is finished according to the flow b 2-c-d; the method has the advantages of simultaneously adding noise and smoothing, can be finished according to the flow of a-b-c-d, and can firstly smooth and then add noise during processing.
Obtaining a distance index of each image of the test set after noise addition according to the image quality evaluation index; and evaluating by using various general image quality indexes, such as one or combination of more of common SSIM, MS-SSIM, IW-SSIM, FSIM and MDSI as evaluation indexes to obtain the distance index of each image.
And dividing the test set subjected to noise addition into a plurality of quality grade data sets according to the distance index, wherein the plurality of quality grade data sets are used as image quality test sets.
Image resolution test set: and dividing the original test set into a plurality of image resolution test sets according to the resolution size.
Target quality test set: and dividing the original test set into a plurality of target quality test sets according to the target quality difference.
Target size test set: and dividing the original test set into a plurality of target size test sets according to the target size.
(4) And testing the algorithm to be tested through the image quality test set, the image resolution test set, the target quality test set and the target size test set respectively, calculating the mAP respectively, and obtaining an image quality score, an image resolution score, a target quality score and a target size score respectively.
As shown in fig. 2, the image quality scoring step:
respectively scoring the image quality test sets of different grades to calculate mAP;
and carrying out weighting processing on the mAP score by utilizing the normalized weight to obtain an image quality score.
Setting the degree of similarity of different grades for the mAP grades of different grades, which specifically comprises the following steps: the data sets with the highest quality grade to the data sets with the lowest quality grade are equally reduced step by step.
The normalized weight, as shown in fig. 3, is obtained by the following steps:
setting the degree of similarity of different grades for the mAP grades of different grades;
obtaining corresponding mAP scores which are respectively needed in the same degree and time sharing according to different grades, and calculating the weight corresponding to each grade;
and carrying out normalization processing on the weight corresponding to each grade to obtain the normalization weight corresponding to each grade.
Specifically, the score calculation formula is as follows: wherein n is an integer larger than 2, mAP and w are in one-to-one correspondence according to different grades, mAP is mAP score, and w is normalized weight.
As shown in fig. 4, the target quality scoring step is:
respectively scoring the multiple target quality test sets to calculate the mAP;
and carrying out weighting processing on the mAP score by utilizing the normalized weight to obtain a target quality score.
Specifically, the score calculation formula is as follows: wherein n is an integer greater than 2, mAP and w are in one-to-one correspondence according to different grades, mAP is mAP score, and w is normalized weight.
As shown in fig. 5, for the image resolution scoring step:
and respectively scoring the multiple image resolution test sets to calculate the mAP, and obtaining the image resolution score by using average weighting.
As shown in fig. 6, the scoring step for target size scoring:
and respectively scoring the plurality of target size test sets to calculate the mAP, and obtaining the target size score by using average weighting.
(5) And carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute.
As shown in fig. 8, the overall evaluation system according to the embodiment of the present application includes: the system comprises a test set preparation module, a test set processing module, a test set grading module, an image quality test set generation module, an image resolution test set generation module, a target quality test set generation module, a target size test set module and a grading module.
The test set preparation module is connected with the test set processing module, the image resolution test set generation module, the target quality test set generation module and the target size test set module.
The test set processing module is connected with the test set grading module.
The test set grading module is connected with the image quality test set generating module.
The image quality test set generation module, the image resolution test set generation module, the target quality test set generation module and the target size test set module are respectively connected with the grading module.
As a specific embodiment, as shown in fig. 8:
after the algorithm to be tested is input, the test set preparation module is firstly entered, and the original test set with the tested target type is selected according to the algorithm to be tested.
The method comprises the steps of entering a respective test set processing module, an image resolution test set generating module, a target quality test set generating module and a target size test set module.
And the test set processing module is used for carrying out noise adding processing with different strengths on the original test set.
And the test set grade division module is used for obtaining the distance index of each image of the test set after noise addition according to the image quality evaluation index and dividing the test set after noise addition into a plurality of quality grade data sets according to the distance index.
And the image quality test set generation module takes the plurality of quality grade data sets as an image quality test set.
And the image resolution test set generation module is used for dividing the original test set into a plurality of image resolution test sets according to the resolution.
And the target quality test set generation module is used for dividing the original test set into a plurality of target quality test sets according to the target quality difference.
And the target size test set module is used for dividing the original test set into a plurality of target size test sets according to the target size.
The grading module is used for testing the algorithm to be tested through the image quality test set, the image resolution test set, the target quality test set and the target size test set respectively, calculating mAP respectively and obtaining an image quality grade, an image resolution grade, a target quality grade and a target size grade respectively; and carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute.
Illustrate by way of example
As a specific embodiment of the method/system of the present application, a test set may be divided into 5 quality level data sets T1, T2, T3, T4, and T5 according to a distance index; wherein the quality grade sequence is that T1 is more than T2 and more than T3 and more than T4 and more than T5;
taking T1-T5 as an image quality test set;
dividing an original test set into 3 image resolution test sets according to the resolution;
dividing an original test set into 2 target quality test sets according to the target quality difference;
and dividing the original test set into 3 target size test sets according to the target size.
(1) Calculating an image quality score:
and calculating mAP1 of T1 and P1, calculating mAP2 of T2 and P2, and obtaining mAP3, mAP4 and mAP5 in the same way, so that 5 grades of scores are obtained, and finally weighting is carried out to obtain a quality score S1.
Specifically, the definition of the same degree score is different mAP scores of the algorithm for different noise level images, but the algorithm has the same degree of goodness, a low mAP score is obtained for a test set with high noise and poor quality, a high mAP score is obtained for a test set with low noise and good quality, and the algorithm has the same goodness to a certain degree.
The mAP homonymy degree division condition of each grade is set, the mAP homonymy degree division of grade 1 (with maximum noise) is 0.2, namely the algorithm effect is considered to be good, the mAP homonymy degree division of quality grade 2 is 0.4, the mAP homonymy degree division of quality grade 3 is 0.6, the mAP homonymy degree division of quality grade 4 is 0.8, and the mAP homonymy degree division of quality grade 5 (with minimum noise) is 1. The same degree can be freely set according to different situations.
Therefore, when the mAP scores (degree of similarity) of 1-5 grades are respectively 0.2, 0.4, 0.6, 0.8 and 1, the evaluation algorithm has equivalent effect; therefore, the reciprocal of the similarity score (0.2, 0.4, 0.6, 0.8, 1) was used to calculate the weight (5, 2.5, 1.666667, 1.25, 1).
When the mAP scores of 1-5 are 1, respectively, the total score of quality should be 1, instead of 5. Thus, the weight w = (5, 2.5,1.66666667,1.25, 1)/11.41666667 = (0.4379562, 0.2189781, 0.1459854, 0.10948905, 0.08759124).
And finally, summing mAP w of each grade to obtain a final score S, and calculating a formula, wherein n is 5.
Test 1: when the mAP is (0.2, 0.4, 0.6, 0.8, 1), the respective scores should be 0.08759124 (same degree), and the sum should be 0.4379562.
As a result: substituting into weight normalization formula to obtain total score of 0.4379562, correct. 0.4379562 is a general effect score, and if a higher score is desired, training for a strong noisy image may be enhanced.
Test 2 when the mAP is (0.7, 0.7, 0.7, 0.7, 0.7), the total score should be 0.7.
As a result: the total score obtained by substituting the weight normalization formula is 0.7, which is correct.
(2) Image resolution scoring
And dividing the original test set into 3 types according to the resolution ratio, the medium and the small, respectively grading the 3 types to calculate the mAP, and finally carrying out average weighting to obtain a resolution ratio score S2.
The weighting method comprises the following steps: and (4) average weighting.
(3) Target quality scoring
Dividing the original test set into 2 sets according to the good and poor target quality, respectively scoring the 2 sets to calculate mAP, and finally weighting to obtain a resolution score S3.
The weighting method, normalization weight, is the same as the normalization weight of the image quality score.
Setting the equal degree of the mAP with good target quality to be 1 point, setting the equal degree of the mAP with poor target quality to be 0.5 point, and then carrying out normalization, and dividing: (1/1, 1/0.5)/(1/1 + 1/0.5), i.e. the weights are (0.33333, 0.66666) respectively.
(4) Target size scoring
Dividing the original test set into 3 sets according to the size of the target, the size of the target and the size of the target, respectively scoring the 3 sets to calculate the mAP, and finally carrying out average weighting to obtain a resolution score S4.
The weighting method comprises the following steps: average weighting
(5) Calculating the total score
And scoring the algorithm according to different properties of the images to obtain S1-S4, and finally, carrying out average weighting on the algorithm to obtain a total score Stotal.
The weighting method comprises the following steps: and (4) average weighting.
Comparative summary
Training a kitti data set by adopting yolov3_ tiny edition, and carrying out target detection comparison on a test set as follows:
the conclusion is drawn: the algorithm performs well on a noise-free image, but is sensitive to target detection of a noisy image, large-resolution, small-target and shielding, and the training of the image with the attributes in the aspects above is enhanced.
The method/system realizes comprehensive weighted evaluation of multiple indexes of image attributes through data set processing, classification and the like, can reflect the performance of an algorithm to be tested facing a noise map set comprehensively and comprehensively, and is convenient for evaluating the algorithm to be tested objectively and comprehensively.
Claims (6)
1. The overall evaluation method for the performance of the target detection algorithm based on the image attributes is characterized by comprising the following steps of:
selecting an original test set with a tested target type according to a to-be-tested algorithm;
carrying out noise adding and/or smoothing processing with different intensities on an original test set, obtaining a distance index of each image of the test set subjected to noise adding according to an image quality evaluation index, and dividing the test set into a plurality of quality grade data sets according to the distance index to be used as an image quality test set;
dividing an original test set into a plurality of image resolution test sets according to the resolution;
dividing an original test set into a plurality of target quality test sets according to the target quality difference;
dividing an original test set into a plurality of target size test sets according to target sizes;
testing the algorithm to be tested through an image quality test set, an image resolution test set, a target quality test set and a target size test set respectively, calculating mAP respectively, and obtaining an image quality score, an image resolution score, a target quality score and a target size score respectively;
carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute;
wherein the image quality score is obtained by: respectively scoring the image quality test sets of different grades to calculate mAP; carrying out weighting processing on the mAP score by utilizing the normalized weight to obtain an image quality score; the normalized weights are obtained by: setting the degree of similarity of different grades for the mAP grades of different grades; obtaining corresponding mAP scores which are respectively needed in the same degree and time sharing according to different grades, and calculating the weight corresponding to each grade; carrying out normalization processing on the weight corresponding to each grade to obtain the normalization weight corresponding to each grade;
wherein the target quality score is obtained by: respectively scoring the multiple target quality test sets to calculate the mAP; carrying out weighting processing on the mAP score by utilizing the normalized weight to obtain a target quality score;
wherein the image resolution score is obtained by: respectively scoring the multiple image resolution test sets to calculate mAP, and obtaining image resolution scores by using average weighting;
wherein the target size score is obtained by: and respectively scoring the plurality of target size test sets to calculate the mAP, and obtaining a target size score by utilizing average weighting.
2. The image attribute-based overall evaluation method for target detection algorithm performance as claimed in claim 1, wherein the degree of similarity of different grades is set for the mAP grades of different grades, specifically: the data sets with the highest quality grade to the data sets with the lowest quality grade are equally reduced step by step.
3. The image attribute-based target detection algorithm performance overall evaluation method according to claim 1, wherein the noise adding and/or smoothing processing with different intensities is carried out on the test set, and the method comprises the following steps:
carrying out noise adding processing with different intensities on an original test set, wherein the added noise comprises at least one of Gaussian noise, salt-and-pepper noise, poisson noise and speckle noise;
and/or the presence of a gas in the gas,
smoothing the original test set with different intensities, wherein the smoothing comprises at least one of Gaussian blur, average blur and motion blur;
and increasing the number of the pictures in the processed test set by at least 5 times on the basis of the original test set by noise addition and/or smoothing treatment.
4. The method as claimed in claim 1, wherein the image quality evaluation index is one or more of SSIM, MS-SSIM, IW-SSIM, FSIM, and MDSI.
5. The image attribute-based target detection algorithm performance overall evaluation method according to claim 1, characterized in that:
dividing the test set subjected to noise addition into 5 quality grade data sets T1, T2, T3, T4 and T5 according to the distance index; wherein the quality grade sequence is that T1 is more than T2 and more than T3 and more than T4 and more than T5;
taking T1-T5 as an image quality test set;
dividing an original test set into 3 image resolution test sets according to the resolution;
dividing an original test set into 2 target quality test sets according to the target quality difference;
and dividing the original test set into 3 target size test sets according to the target size.
6. An overall evaluation system for performance of an image attribute-based target detection algorithm is characterized by comprising:
the test preparation module is used for selecting an original test set with a tested target type according to a to-be-tested algorithm;
the test set processing module is used for carrying out noise addition and/or smoothing processing with different intensities on the original test set;
the test set grading module is used for obtaining a distance index of each image of the test set after noise addition according to the image quality evaluation index and dividing the test set after noise addition into a plurality of quality grade data sets according to the distance index;
the image quality test set generation module is used for taking the quality grade data sets as an image quality test set;
the image resolution test set generation module is used for dividing the original test set into a plurality of image resolution test sets according to the resolution;
the target quality test set generation module is used for dividing the original test set into a plurality of target quality test sets according to the target quality difference;
the target size test set module is used for dividing the original test set into a plurality of target size test sets according to the target size;
the grading module is used for testing the algorithm to be tested through the image quality test set, the image resolution test set, the target quality test set and the target size test set respectively, calculating mAP respectively and obtaining an image quality grade, an image resolution grade, a target quality grade and a target size grade respectively; carrying out average weighting processing on the image quality score, the image resolution score, the target quality score and the target size score to obtain an overall evaluation result based on the image attribute;
the scoring module is used for respectively scoring the image quality test sets with different grades to calculate the mAP, and weighting the mAP score by using the normalized weight to obtain an image quality score; wherein the scoring module is configured to obtain the normalized weight by: setting the degree of similarity of different grades for the mAP grades of different grades; obtaining corresponding mAP scores which are respectively needed in the same degree and time sharing according to different grades, and calculating the weight corresponding to each grade; carrying out normalization processing on the weight corresponding to each grade to obtain the normalization weight corresponding to each grade;
the scoring module is used for scoring the multiple target quality test sets respectively to calculate the mAP, and weighting the mAP score by utilizing the normalized weight to obtain a target quality score;
the grading module is used for respectively grading and calculating the mAP of the multiple image resolution test sets and obtaining image resolution grading by utilizing average weighting;
and the scoring module is used for scoring the plurality of target size test sets respectively to calculate the mAP and obtaining the target size score by using average weighting.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911081611.2A CN110969599B (en) | 2019-11-07 | 2019-11-07 | Target detection algorithm performance overall evaluation method and system based on image attributes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911081611.2A CN110969599B (en) | 2019-11-07 | 2019-11-07 | Target detection algorithm performance overall evaluation method and system based on image attributes |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110969599A CN110969599A (en) | 2020-04-07 |
| CN110969599B true CN110969599B (en) | 2023-04-07 |
Family
ID=70030343
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911081611.2A Active CN110969599B (en) | 2019-11-07 | 2019-11-07 | Target detection algorithm performance overall evaluation method and system based on image attributes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110969599B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620729A (en) * | 2009-07-31 | 2010-01-06 | 重庆医科大学 | Method for producing gray image with best quality |
| CN110176005A (en) * | 2019-05-16 | 2019-08-27 | 西安电子科技大学 | Remote sensing image segmentation method based on normalization index and multiple dimensioned model |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3846851B2 (en) * | 2001-02-01 | 2006-11-15 | 松下電器産業株式会社 | Image matching processing method and apparatus |
| US7039222B2 (en) * | 2003-02-28 | 2006-05-02 | Eastman Kodak Company | Method and system for enhancing portrait images that are processed in a batch mode |
| CN101334893B (en) * | 2008-08-01 | 2011-05-04 | 天津大学 | Fused image quality integrated evaluating method based on fuzzy neural network |
| CN102169576B (en) * | 2011-04-02 | 2013-01-16 | 北京理工大学 | Quantified evaluation method of image mosaic algorithms |
| CN104123705B (en) * | 2014-07-22 | 2016-08-24 | 北华大学 | A kind of super-resolution rebuilding picture quality Contourlet territory evaluation methodology |
| EP3166072A1 (en) * | 2015-11-06 | 2017-05-10 | Thomson Licensing | Method for denoising an image and apparatus for denoising an image |
| CN107194898A (en) * | 2017-06-23 | 2017-09-22 | 携程计算机技术（上海）有限公司 | The method for pushing of the methods of exhibiting of hotel's image, storage medium and hotel information |
| CN108416748A (en) * | 2018-02-26 | 2018-08-17 | 阿博茨德（北京）科技有限公司 | The image pre-processing method and device of JPEG compression document |
| CN109490843B (en) * | 2018-11-15 | 2020-08-04 | 成都傅立叶电子科技有限公司 | Normalized radar screen monitoring method and system |
-
2019
- 2019-11-07 CN CN201911081611.2A patent/CN110969599B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101620729A (en) * | 2009-07-31 | 2010-01-06 | 重庆医科大学 | Method for producing gray image with best quality |
| CN110176005A (en) * | 2019-05-16 | 2019-08-27 | 西安电子科技大学 | Remote sensing image segmentation method based on normalization index and multiple dimensioned model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110969599A (en) | 2020-04-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110210560B (en) | Incremental training method, classification method and device, equipment and medium of classification network | |
| Baur et al. | MelanoGANs: high resolution skin lesion synthesis with GANs | |
| CN109766835B (en) | SAR target recognition method for generating countermeasure network based on multi-parameter optimization | |
| CN111199214B (en) | Residual network multispectral image ground object classification method | |
| Po et al. | A novel patch variance biased convolutional neural network for no-reference image quality assessment | |
| CN110956613A (en) | Image quality-based target detection algorithm performance normalization evaluation method and system | |
| CN109934810B (en) | Defect classification method based on improved particle swarm wavelet neural network | |
| US9092697B2 (en) | Image recognition system and method for identifying similarities in different images | |
| TWI775935B (en) | Method of estimating depth in image | |
| CN109344845B (en) | Feature matching method based on triple deep neural network structure | |
| CN105989001B (en) | Image search method and device, image search system | |
| CN111724335A (en) | Detection method and detection system | |
| CN109949200B (en) | Filter subset selection and CNN-based steganalysis framework construction method | |
| CN108369649A (en) | Focus detection | |
| JP4252308B2 (en) | Resolution improving method and apparatus | |
| CN110969599B (en) | Target detection algorithm performance overall evaluation method and system based on image attributes | |
| CN111027589B (en) | Multi-division target detection algorithm evaluation system and method | |
| CN110473169B (en) | Simulation picture confidence evaluation method | |
| CN106846366B (en) | TLD video moving object tracking method using GPU hardware | |
| JP2020181402A (en) | Image processing system, image processing method and program | |
| JP5764094B2 (en) | Image search apparatus, image search method, and image search program | |
| CN112954310A (en) | Image quality detection method, device, computer equipment and readable storage medium | |
| CN104732190A (en) | Synthetic aperture sonar target detection method based on orthogonal texture correlation analysis | |
| Asatryan | Novel Technique for Ro-Reference Image Quality Assessment | |
| Nahari et al. | Artificial Intelligence optimization For Low-Light Image Enhancement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |