CN109714591B - Image quality subjective evaluation method and system based on evaluation label - Google Patents
Image quality subjective evaluation method and system based on evaluation label Download PDFInfo
- Publication number
- CN109714591B CN109714591B CN201910020772.4A CN201910020772A CN109714591B CN 109714591 B CN109714591 B CN 109714591B CN 201910020772 A CN201910020772 A CN 201910020772A CN 109714591 B CN109714591 B CN 109714591B
- Authority
- CN
- China
- Prior art keywords
- image
- evaluation
- quality
- images
- label
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an image quality subjective evaluation method based on an evaluation label, and belongs to the field of image quality evaluation and image analysis. The method comprises the following steps: determining evaluators, scoring standards and viewing conditions, and pre-screening a priority evaluation sequence; and for the selected priority test sequence, forming the pre-screened image data into an evaluation mode of an image pair, and judging the relative quality of two images in the image pair by an organization observer. The invention also discloses an evaluation system, comprising: the system comprises a user management module, an image pre-screening module, a sequence playing module and a data processing module. The invention forms the priority image quality test sequence and forms the image data into the evaluation mode of the image pair, thereby expanding the application range of the subjective evaluation method of the image quality, being suitable for evaluating the images with complex mixed distortion, such as similar underwater images, and the like, and improving the stability and reliability of the evaluation result.
Description
Technical Field
The invention belongs to the field of image quality evaluation and image analysis, and particularly relates to an image quality subjective evaluation method based on an evaluation label for an image with complex mixed distortion, and an evaluation system for evaluating by adopting the method.
Background
In many applications where humans are the ultimate consumers of visual information, it is important how to accurately and efficiently assess the quality of digital images. Vision is a very important basis in marine science research. For years, various underwater monitoring platforms, ROV/AUV, submarine observation systems, underwater watchboards and fishing boats acquire megametric underwater images, videos and high-dimensional hyperspectral images. In water, the underwater images shot by optical attenuation, scattering and light source illumination of a water body have various complex and mixed degradations such as low contrast, blurring, non-uniform illumination, bright spots, color projection and various noises, and marine field investigation and marine observation are very complex and expensive, so that the underwater image quality evaluation method can be used for automatically screening high-quality images, providing objective standards for restoring or enhancing the underwater images and improving the design of an underwater imaging and transmission system, and providing important values for marine science artificial intelligence analysis, a marine task decision system, prediction and the like. The image quality evaluation includes subjective evaluation and objective evaluation. In image quality evaluation studies, since human eyes are the ultimate receptor of images, subjective evaluation is considered as the most direct and accurate method for characterizing visual perception. The subjective quality of the image is the basis for measuring the performance of an objective evaluation algorithm, classifying and screening the image and further analyzing the image. At present, no image subjective quality evaluation method suitable for mixed complex degradation exists.
The conventional subjective image quality evaluation method is designed mainly for an established natural image quality evaluation database. Subjective scoring methods for image quality evaluation can be mainly classified into three categories: the first is a single excitation method, the second is a double excitation method, and the third is a pair comparison method.
The single stimulus method evaluation requires the image to be evaluated to be played on a display without knowing the reference information of the original image. The single excitation method is typically a single excitation continuous quality evaluation method. The single-excitation continuous quality evaluation method plays the materials to be evaluated according to a random sequence, and an observer scores the viewed images by adopting a five-level continuous quality scoring system.
The dual excitation method generally has two embodiments. One is a dual excitation distortion measurement method (DSIS), which plays the reference material and the corresponding test material in turn during the test, prompts the evaluator to score when playing the test material, or plays the reference material and the test material again and again, requires the evaluator to score during the second play, and usually adopts a 5-point scoring mode. The other is the dual stimulus continuous quality scale method (DSCQS), which also plays test material and reference material in sequence, but does not explicitly tell the evaluator in the experiment which of the two materials played is the reference material and which is the test material. Therefore, both playmaterials need to be scored. This method typically plays the test and reference materials twice in a loop, requiring the evaluator to score when playing the second loop. The quality of the test material is measured by calculating the difference in the dispersion between the distorted material and the reference material.
The pairwise comparison method achieves the assessment of performance differences between different products or different algorithms by comparing and scoring two distortion degrees from the same reference image. The method will score into seven grades: very poor, somewhat poor, the same, somewhat good, very good. The pair-wise comparison method is mainly used for evaluating the influence of different systems, different algorithms and different processing parameters on the same content material.
Light is transmitted in water, and absorption and scattering determined by optical properties (IOP) in the water body influence the effect of the whole underwater imaging. Forward scattering causes the point light sources to spread into a circle of confusion, resulting in image blurring; backscatter reduces the contrast of the image, producing a haze that is superimposed on the image. The absorption and scattering are not only generated by the water body, but also influenced by dissolved organic matters and floating particles, and the concentration and target distance of plankton, colored dissolved organic matters and total suspended matters are also main factors influencing the quality of an underwater color image. In addition, the color cast of underwater objects is related to the absorption and attenuation of different wavelengths of light by the body of water. As the underwater depth increases, the colors disappear in sequence according to the wavelength, and the blue light has the shortest wavelength and the longest distance to propagate underwater. Although the visible distance can be increased by adding artificial illumination, the non-uniform illumination condition is often caused, and bright spots are generated in the image and are dark around the bright spots. The artificial light source makes the scattering caused by the suspended matters more serious. The influence of spray, swirl, silt and various marine organisms caused by the movement operation also causes irregular blurring of the image. Besides, the color temperature of the imaging system and the light source can influence the quality of the underwater color image. Therefore, the following problems are mostly present in the captured underwater images: limited viewing distance, low contrast, non-uniform illumination, blur, speckle, color projection, and noise of various complexities.
The underwater image is different from a natural image, and because an original clear underwater image which can be used as a reference cannot be obtained, the traditional double-excitation method is not suitable for the underwater image, the score of the single-excitation method for the underwater image with similar quality cannot show the subtle difference of subjective image quality perception, and for the improvement of the quality of a point obtained in the process of enhancing or restoring the severely degraded image, it is difficult to judge that one method is better than the other method, and the most important method is real-time and automatic processing. The pair-wise comparison method is mainly used for evaluating the influence of different systems, different algorithms and different processing parameters on the same content material.
Disclosure of Invention
The invention aims to solve the technical problem of the prior art and provides an image quality subjective evaluation method based on an evaluation label, which has more excellent performance of distinguishing slight quality difference and wide adaptability and can be used for evaluating images with complex mixed distortion.
Another technical problem to be solved by the present invention is to provide a system for implementing the above subjective evaluation method of image quality based on an evaluation label.
The technical problem to be solved by the present invention is achieved by the following technical means. The invention relates to an image quality subjective evaluation method based on an evaluation label, which adopts a pre-screening method based on color image quality measurement to combine image data to be tested into an evaluation mode of image pairs, organizes an observer to mark relative quality to generate the evaluation label, and processes the label to obtain subjective image quality; the method comprises the following steps:
determining evaluators, scoring standards and observation conditions, and pre-screening a priority evaluation sequence;
(1) the evaluator should have (correct to) normal visual acuity and normal color vision; may not be an expert in graphics imaging; the number of people is determined according to the size of the priority test sequence set, and the evaluation time (including examination and demonstration) of each observer is generally required to be not more than 30 minutes.
(2) Determining a scoring criterion
And (4) marking the quality of two images in the image pair to be tested by an evaluator, and giving a pair evaluation label of the image pair. For image pair (I)1,I2) Such as I1Subjectively better than I2Then the corresponding pair evaluation label l1,2Set to +1, subscripts 1,2 are respectively indicated asImage 1, image 2, simultaneously2,1-1; if I2Subjective quality better than I1Assigned value l1,2＝-1,l2,1In addition, when the image quality is not easily distinguished, the pair evaluation label may not be labeled, and the image pair (I) may be displayed at this time1,I2) Is recorded as l to the evaluation label1,2And l2,1Are all 0.
(3) Determining viewing conditions
The subjective evaluation environment was arranged to obtain the most reliable data. Test environment of subjective experiment: 0.55-0.65 m away from the screen; maximum viewing angle <30 °; the image to be measured on the display screen can not be shielded.
(4) Pre-screening based on color image quality metrics
Assuming a total of N images in the set of images under test, N (N-1)/2 possible pairs of images can be generated. A half-hour testing stage removes pre-test inspection, training, demonstration, etc., and a testing stage typically observes approximately 300 image pairs, thereby deciding to pre-select 44850 image pairs, i.e., 300 images, to generate a preferred test image set, based on the 3 to 5 seconds required to score each image pair.
In order to avoid the over-concentrated quality distribution of the images to be tested in the priority test sequence set, the quality of the images to be tested is ensured to be uniformly sampled in the existing range. The invention can select three indexes of the brightness contrast, the hue variance and the saturation mean value of the color image as the selection standard of the priority test sequence. For N images in the image set to be detected, three index histograms (dividing into ten cells) of each image are calculated, and 10 images are randomly selected in different intervals.
Step two: and for the selected priority test sequence, testing by using a subjective evaluation system, marking the quality of two images in the pair of images to be tested by an evaluator, and giving a pair evaluation label of the pair of images.
Step three: and storing personal information of an evaluator and evaluation result data of the evaluation label of the image, and sequencing all the images according to the evaluation label. And calculating the label score and the percentile score by the following method:
(1) computing image tag scores
For image i, the evaluation label l for all image pairs obtained by comparing image i with other images ji,jAnd i is not equal to j, and the label score S of the image i is calculated through accumulationi：
According to the evaluation labels of all the image pairs, the label scores of the N images can be generated, and the label score of each image falls in the range of [ -N +1, N-1 ]. The current image set is assumed to cover the range of image qualities possible for all tests, from best to worst. The total label sum N-1 corresponds to the best quality value and-N +1 corresponds to the worst quality value.
(2) Calculating image corresponding percentile scores
Calculating a percentile quality score S for an image i based on a linear mappingip：
And finally, obtaining the subjective scores of all the image qualities in the test image data set, wherein the higher the subjective quality score is, the better the image quality is.
The invention relates to an image quality subjective evaluation method based on an evaluation label, which further preferably adopts the technical scheme that: in the second step, the image pair is taken as a unit by utilizing the test of the subjective evaluation system for evaluating the image quality of the label, and the test is carried out according to a randomly generated sequence; after the quality of each pair of image pairs is marked by an evaluator, automatically switching the next group of image pairs to continue testing; the specific method comprises the following steps:
(1) inputting information of an evaluator;
(2) entering a demonstration and introduction interface;
(3) loading a test sequence;
(4) the image pairs are simultaneously displayed on the screen in an equal-size manner;
(5) the evaluator marks the quality of the image pair, and if the relative quality of the two images cannot be judged, the evaluator selects the button which cannot be judged and performs the next group of tests;
(6) all images were scored for evaluation.
The invention relates to an image quality subjective evaluation method based on an evaluation label, which further preferably adopts the technical scheme that: the image quality subjective evaluation method based on the evaluation label is different from the pairing comparison method specified in ITU-R BT.500 in that: the subjective evaluation mode based on the image quality of the evaluation label allows the image pair to be measured to have different image contents without distinguishing distortion types and levels, and can have different sizes.
The invention relates to an image quality subjective evaluation method based on an evaluation label, which further preferably adopts the technical scheme that: the invention is based on the difference between the subjective evaluation mode of image quality of an evaluation label and the pair comparison method specified in ITU-R BT.500, and the difference is that: the image quality subjective evaluation method based on the evaluation labels adopts three-level quality label marking, does not require marking on images with similar quality, and utilizes all obtained evaluation labels to calculate the image quality score.
The invention also discloses a system for subjective evaluation of image quality based on the evaluation label, which is characterized in that the system can realize operations such as image set pre-screening, user information registration, image pair play mode selection, scoring record, evaluation label calculation and the like through man-machine interaction, and the system comprises: the system comprises a user management module, an image pre-screening module, a sequence playing module and a data processing module; wherein:
the user management module comprises two sub-modules of user information addition and user information deletion and is used for managing addition, deletion and the like of user information in an experiment;
the image pre-screening module comprises an image quality measurement calculation submodule such as brightness contrast, hue variance, saturation mean value and the like, and a sampling module, wherein the sampling module is respectively used for calculating the image quality measurement value of each image in the image set to be tested, setting a sampling mode by using the sampling module and generating a priority test sequence;
the sequence playing module is mainly used for playing the image pairs during subjective evaluation, different playing modes can be selected, and random playing is defaulted;
and the data management module comprises three submodules of evaluation scoring, data storage and label calculation and is used for realizing recording and storage of scoring data and statistical analysis of results.
The method is suitable for images with mixed distortion in various complex environments, such as underwater images. The method is also suitable for natural images, and has better performance of distinguishing fine quality difference than other methods. The subjective evaluation method for the image quality of the evaluation label provided by the invention does not limit the image content, is used for different contents, adopts a three-level system to evaluate the evaluation label, and does not require marking on images with similar quality. The image quality is ranked by calculating the evaluation labels of all the images in the image set, so that the applicability is wider.
The invention applies the subjective evaluation method of the image quality of the evaluation label, so that the original image is not needed to be used as reference in the subjective evaluation process, and the method is used for carrying out relative quality judgment on the images with different contents and complicated mixed distortion to obtain the relative subjective image quality among the images.
The method is not limited by image content, can avoid the difficulty in distinguishing the type and degree of image degradation and the sensitivity to the image content and the viewing condition in subjective evaluation, has more excellent distinguishing capability of the obtained image quality score to the fine quality difference, and is more suitable for evaluating the images with complex mixed distortion.
The method of the invention does not need to group the test images according to the type of degradation, thereby ensuring the consistency of the scores among observers. By adopting the subjective evaluation method for the image quality of the evaluation label, the real-time expansion of the subjective evaluation database of the image quality can be realized, and the new version does not need to be iterated continuously to expand the database. With the continuous expansion of the image database, the uniform distribution of the image distortion types and levels in the database in a wide range can be gradually realized, and the problems of uneven distribution of degradation levels and narrow range are avoided. The method has the advantages of simple organization, low requirement on observers, no need of considering and scoring only by making relative judgment, small error and high practical applicability.
The image quality subjective evaluation method based on the evaluation label uses the thinking of progressive learning sequencing to establish the statistical distribution of the reference-free image quality subjective evaluation values with inseparable mixed distortion. The image quality is ranked and scored by applying the subjective evaluation method of the evaluation label, and the problem that no reference image exists in imaging environments such as underwater images and the like is solved. Compared with the traditional subjective quality evaluation method, the method is obviously superior to other subjective image quality evaluation methods in the aspects of the resolution capability of slight quality difference and the like. An image data set with quality scores according with human visual perception is established by utilizing an evaluation label image quality subjective evaluation method, so that the relative score of the image in a current image library can be obtained, and the non-reference evaluation of the image quality is realized.
Compared with the prior art, the beneficial effects of the invention are summarized as follows:
(1) the method can be used in all scenes where the traditional subjective evaluation method can be applied.
The method is more suitable for underwater images and images with mixed distortion in various complex environments.
(2) The invention does not require reference to the original image.
(3) Compared with the existing main subjective image quality evaluation method, the method can more accurately distinguish the subtle differences of the underwater images with similar quality.
(4) The method can well avoid the difficulty in distinguishing the degradation types and degrees of underwater images and the like and the sensitivity to image contents and viewing conditions in subjective evaluation;
(5) according to the method, the test images do not need to be grouped according to the degradation type in the evaluation process, so that the consistency of the evaluation among the observers is ensured;
(6) in the evaluation process, the observer can select the images with similar quality of feeling without scoring, and the method has low requirement on the observer and small error.
Drawings
FIG. 1 is a flow chart of a subjective evaluation method;
FIG. 2 is a histogram (divided into ten cells) with brightness contrast as an indicator;
FIG. 3 is a histogram (divided into ten cells) with hue variance as an indicator;
FIG. 4 is a histogram (divided into ten cells) with the saturation mean as an indicator;
fig. 5 is a histogram (divided into ten cells) with UCIQE as an indicator;
FIG. 6 is a graph of scores for two subjective test methods;
FIGS. 7-10 are graphs corresponding to images with the same score by the single stimulation method, showing the scores by the evaluation label method;
FIG. 11 is a 24 color chart taken from a pond experiment;
FIGS. 12-14 are 24 color chart pictures taken from a water basin experiment;
FIG. 15 is a scatter plot of the distribution of all image scores in group one;
FIG. 16 is a scatter plot of the distribution of all image scores in group two;
FIG. 17 is a scatter plot of the distribution of all image scores in group three;
FIG. 18 is a scatter plot of the distribution of all image scores in group four;
FIG. 19 is two outlier images in a set two image.
Detailed Description
The following further describes the technical solutions of the present invention, so that those skilled in the art can further understand the present invention, and the present invention is not limited by the claims.
The invention adopts a pre-screening method based on color image quality measurement to form image data to be detected into an evaluation mode of image pairs, organizes an observer to judge relative quality, generates an evaluation label, and processes the label to obtain subjective image quality; the method comprises the following steps:
determining evaluators, scoring standards and observation conditions, and pre-screening a priority evaluation sequence.
Assuming a total of N underwater images in a given set of images under test, N (N-1)/2 possible pairs of images can be generated. The assessment process follows the requirements of the subjective test recommendation issued by the international telecommunications union, and a test period is designed to be half an hour in order to avoid fatigue of the observer. A half-hour testing period removes pre-test inspection, training, demonstration, etc., one testing period typically observes approximately 300 image pairs, and 44850 image pairs, i.e., 300 images, may be preselected for testing, based on the 3 to 5 seconds required for each image pair to score.
In the image pre-screening module, CIELAB spatial brightness contrast, hue variance, and saturation mean are used as pre-selected criteria to calculate the correlation indices for all images in the image set, and the histogram and the integrated UCIQE (divided into 10 bins) for each index are shown in fig. 2-5. Approximately 10 images per interval were randomly selected.
45 evaluators are selected, most of the evaluators come from the electronics engineering institute of Huaihai institute of Industrial science, and certain understanding is provided for image processing and image quality evaluation. And logging in an interface of the subjective evaluation system, and counting the name, sex, age, professional background and the like of an evaluator for subsequent research. The steps are carried out in the user management module.
Step two: and for the selected priority test sequence, testing by using an image quality subjective evaluation system of the evaluation label.
Marking the quality of 300 underwater images to construct an image group (I)1,...,In300, N. Each image is paired with an image other than itself to generate a set P of image pairs, where P is N (N-1)/2-44850 sets of image pairs.
(1) And the observer reads an interface program to introduce principles influencing image quality factors, evaluation standards for comparison evaluation, evaluation process time and the like. The software interface will present several sets of typical different-degradation, different-level image pair sets demonstrating the evaluation process. Before the evaluation is formally started, about 5 'simulation demonstrations' are played first to stabilize the evaluation of the observer. The evaluation tags given in these sets of presentations did not participate in the calculation of the test results.
(2) In a laboratory environment that conforms to the subjective testing standards promulgated by the international telecommunications union, a sequence playback module simultaneously displays two images of an image pair on a display. Preprocessing uniform image size may be performed. And playing the image pairs in a random sequence, wherein each group of image pairs requires that the observers perform quality judgment within no more than 3s, and the scoring time of each observer is no more than half an hour.
And (4) marking the quality of two images in the image pair to be tested by an evaluator, and giving a pair evaluation label of the image pair. For image pair (I)1,I2) If subjectively go up picture I1Quality is superior to I2Then, the evaluation label l is checked1,2Set to +1, subscripts 1,2 are denoted image 1, image 2, and l, respectively2,1-1; if subjectively view I2Quality is superior to I1Assigned value l1,2＝-1,l2,1When the image quality is not easily discriminated, the pair evaluation label may not be labeled, and the image pair (I) may be displayed at this time1,I2) Is recorded as l to the evaluation label1,2And l2,1Are all 0.
Step three: and storing personal information of an evaluator and image pair evaluation tag result data, and sequencing all images according to the pair evaluation tags. And calculating the label score and the percentile score by the following method:
for image i, the evaluation label l for all image pairs obtained by comparing image i with other images ji,jAnd i is not equal to j, and the label score S of the image i is calculated through accumulationi：
According to the evaluation labels of all the image pairs, the label scores S of 300 images can be generatediThe label score of each image falls within [ -299, 299]In the interval, the current image set covers the range of image qualities possible for all tests, from best to worst. Total tag and 299 correspond to the best quality value, -299 corresponds toThe worst quality value.
Calculating a percentile quality score S for an image i based on a linear mappingip：
And finally, obtaining the subjective scores of all the image qualities in the underwater image data set, wherein the higher the subjective quality score is, the better the image quality is.
Example 2, a comparison experiment of the subjective evaluation method of image quality of an evaluation label and the subjective evaluation method of image quality of single stimulus.
The underwater image has no original image, so that the dual-excitation method is obviously not applicable, and the comparison method is used for comparing and scoring two distorted objects from the same raw material and is not in accordance with the purpose of the invention. We determined the comparison object as a single excitation sequential image quality evaluation method (SSCQE). The establishment of the laboratory environment and the selection of an observer are kept consistent with the setting of the subjective evaluation method of the image quality of the evaluation label, and a 5-level system scoring test is carried out on 300 underwater images by using a single-excitation subjective image quality evaluation method. Training before the start of the experiment and "simulation demonstration" refer to the steps described in the present invention to ensure that there is no interference from other factors with the comparison of the two methods. The 300 underwater images were divided into 6 groups of 50 pieces and tested in batches to prevent fatigue of the observer. The image playing adopts a random sequence. The viewer is asked to score each image by a 1-5 point score in no more than about 3 seconds. The average image quality subjective performance (MOS) values for the evaluation label method and the single stimulus method are shown in fig. 6.
It can be seen from fig. 6 that the image quality subjective evaluation values of the evaluation label image quality subjective evaluation method and the single excitation method have nearly the same variation curve and the correlation degree reaches 0.95, but the quality scores of the four images are 4 points through single excitation evaluation as in the four images of fig. 7-10, although we can still see the quality difference between them. The scores obtained by the subjective evaluation method of the image quality of the evaluation label embody the capability of distinguishing the nuances in the scores.
Example 5 the accuracy of the image quality score obtained by the subjective evaluation method for evaluating the image quality of a tag was discussed by analyzing the quality of 24 color chart (21.59 × 27.94cm) images taken in a pool.
The length of the pool in which the image is shot is 2.53 meters, the width is 1.02 meters, the height is 1.03 meters, and the shot target is an Aizili standard 24-color card, as shown in figure 11. These images were taken with an OTI-UWC-325/P/E color camera. Underwater images (960 x 576) were obtained under 94.5cm transparency water and natural lighting conditions, and the pictures taken were underwater images with increasing degradation as the camera distance increased.
Imatest is widely applied image evaluation software and comprises modules of SFR, Colorcheck, Stepchart and the like. The subjective test and software test score data of the quality of the 24-color chart images (see fig. 12, 13 and 14) taken at three different distances in the present example are shown in table 1. The image quality is continuously reduced along with the increase of the distance under the same shooting condition and different shooting distances, the label score/percentage score obtained by the subjective evaluation method for evaluating the label image quality is accurately displayed, two images with obvious quality difference in the test result of the single stimulation method obtain the same score, and the fact that the subjective evaluation method for evaluating the label image quality can reflect the slight change of the image quality better than the single stimulation 5-level system score is shown.
CIE1976L a b color space (CIE LAB color space), is the uniform color space recommended by the international society for illumination (CIE) in 1976. The space is a three-dimensional rectangular coordinate system. Is a color measuring system widely adopted at present. The position of the color in the color space is represented by lightness L and chromaticity coordinates a, b. In table 1, Meancamera chroma (saturation) is the average chroma of the camera color divided by the average chroma of the ideal Colorchecker color, expressed as a percentage. Typically between 100% and 120%.
Meancamera chroma(saturation)＝100％×mean((a*2+b*2)1/2)/mean((a*ideal2+b*ideal2)1/2) (4)
The meaning of this value is that the difference between the expressed color of the 24 colors on the test color card image and the standard color of the color card is larger, and the larger the difference between the image expressed color and the original color is, the larger this value is. Δ C × abuncorr, Δ C × abchroma corr, and Δ E × ab are measures of color error in the device-independent CIElab color space, and differences between perceived colors are accounted for by measuring euclidean distances between them. Δ C abuncorr and Δ C abchroma corr calculate only the color, where:
ΔC*abuncorr＝(Δa*)2+(Δb*)2)1/2(5)
chroma corr refers to adjusting the mean chroma of the camera to be the same as the reference value before making the comparison, which indicates the accuracy of the color if the mean chroma is the same as the reference value.
ΔE*ab＝((ΔL*)2+(Δa*)2+(Δb*)2)1/2(6)
By comparing the subjective score of the subjective evaluation method for the image quality of the evaluation label with the software output data, the fact that the real quality of the test image and the MOS value of the subjective evaluation of the image quality of the evaluation label are in a linear relation can be seen. This demonstrates the accuracy of the proposed inventive method.
Table 124 MOS and software output scores for color chip images
Embodiment 3, the accuracy of the quality score of the degraded image sequence obtained by the subjective evaluation method for the image quality of the evaluation tag is checked.
The experimental images are a sequence of degraded images taken at the same region and different angles of turbidity, similar images are taken in this example in four groups in 300 images according to different contents of photographing, and the scores of each group are plotted in fig. 15 to 18, respectively. The larger the image number, the lower the turbidity of the water at the time of image capturing. PLCC, SROCC and KROCC between MOS and image numbers obtained by the subjective evaluation method of image quality for the evaluation tag are listed in table 2. The number of images in each sequence is different due to the automatic sampling selection.
TABLE 2 correlation between MOS values and image numbering
The result shows that the provided result of subjective evaluation on the image quality of the evaluation label and the turbidity of water are in a linear relation, and the real quality level of the image is accurately reflected. Significant outliers were present in the panels shown in fig. 16 and 17, with the lowest KROCC value in panel two, and two outlier images in panel two are highlighted in fig. 19. As shown in fig. 19, the difference between two adjacent images is very blurred. And the selected images in group two are more concentrated in quality distribution. Group four is most nearly linear as shown in fig. 18. And (3) deducing: (1) the images in group four are richer in color than those in group two, and (2) the image quality randomly selected in group four is more obvious in gradient distribution, and no continuous image with similar image quality exists.
Claims (4)
1. An image quality subjective evaluation method based on an evaluation label is characterized in that: forming image data to be detected into an evaluation mode of an image pair by adopting a pre-screening method based on color image quality measurement, organizing an evaluator to mark relative quality, generating an evaluation label, and processing the label to obtain subjective image quality; the method comprises the following specific steps:
determining evaluators, scoring standards and observation conditions, and pre-screening a priority test sequence;
(1) determining evaluators
(2) Determining a scoring criterion
An evaluator marks the quality of two images in the pair of images to be tested and provides a pair evaluation label of the image pair formed by the two images; for image pair (I)1，I2) When I is1Subjectively better than I2Then the corresponding evaluation label l1,2Set to +1, subscripts 1,2 are denoted image 1, image 2, and so on, respectivelyHour l2,1-1; when I is2Subjective quality better than I1Then assign a value of l1,2＝-1，l2,11 ═ 1; in addition, when the image quality is not easily distinguished, the pair evaluation label may not be marked, and the image pair (I) at this time1，I2) Is recorded as l to the evaluation label1,2And l2,1Are all equal to 0;
(3) determining observation conditions
Arranging a subjective evaluation environment to obtain the most credible data; the test environment of the subjective experiment is 0.55-0.65 m away from the screen; maximum viewing angle <30 °; the image to be detected on the display screen is not shielded;
(4) pre-screening based on color image quality measurement, wherein 300 images are pre-selected to generate a priority test sequence in a total of N images in the image set to be tested;
selecting three indexes of brightness contrast, hue variance and saturation mean of the color image as selection standards of a priority test sequence; calculating three index histograms of each image for N images in the image set to be detected, dividing the three index histograms into ten small intervals, and randomly selecting 10 images in different intervals;
step two: for the selected priority test sequence, testing by using a subjective evaluation system of the evaluation label, marking the quality of two images in the pair of images to be tested by an evaluator, and giving the evaluation label of the pair of images;
step three: storing personal information of an evaluator and result data of marking the evaluation label by the image, and sequencing all the images according to the evaluation label; and calculating the label score and the percentile score by the following method:
(1) computing image tag scores
For image i, the evaluation label l for all image pairs obtained by comparing image i with other images ji,jAnd i is not equal to j, and the label score S of the image i is calculated through accumulationi：
Generating respective label scores of the N images according to the evaluation labels of all the image pairs, wherein the label score of each image falls in an interval of [ -N +1, N-1 ]; assuming that the current image set covers the range of image qualities possible for all tests, from best to worst; the total label and N-1 correspond to the best quality value, and-N +1 corresponds to the worst quality value;
(2) calculating image corresponding percentile scores
Calculating a percentile quality score S for an image i based on a linear mappingip：
And finally, obtaining the subjective quality scores of all the images in the test image data set, wherein the higher the subjective quality score is, the better the image quality is.
2. The method for subjective evaluation of image quality based on an evaluation label according to claim 1, wherein: in the second step, the image pair is taken as a unit by utilizing the test of the subjective evaluation system for evaluating the image quality of the label, and the test is carried out according to a randomly generated sequence; after the quality of each pair of image pairs is marked by an evaluator, automatically switching the next group of image pairs to continue testing; the specific method comprises the following steps:
(1) inputting information of an evaluator;
(2) entering a demonstration and introduction interface;
(3) loading a test sequence;
(4) the image pairs are simultaneously displayed on the screen in an equal-size manner;
(5) the evaluator marks the quality of the image pair, and if the relative quality of the two images cannot be judged, the evaluator selects the button which cannot be judged and performs the next group of tests;
(6) all images were scored for evaluation.
3. The method for subjective evaluation of image quality based on evaluation labels according to claim 1 or 2, wherein: the pair of images to be measured is allowed to have different image content without having to distinguish between distortion type and level, and may be of different sizes.
4. The method for subjective evaluation of image quality based on evaluation labels according to claim 1 or 2, wherein: and (4) evaluating by adopting a three-level quality label, not requiring evaluation on images with similar quality, and calculating an image quality score by using all the obtained evaluation labels.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910020772.4A CN109714591B (en) | 2019-01-09 | 2019-01-09 | Image quality subjective evaluation method and system based on evaluation label |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910020772.4A CN109714591B (en) | 2019-01-09 | 2019-01-09 | Image quality subjective evaluation method and system based on evaluation label |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109714591A CN109714591A (en) | 2019-05-03 |
| CN109714591B true CN109714591B (en) | 2020-08-25 |
Family
ID=66260981
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910020772.4A Active CN109714591B (en) | 2019-01-09 | 2019-01-09 | Image quality subjective evaluation method and system based on evaluation label |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109714591B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110473182A (en) * | 2019-10-08 | 2019-11-19 | 中国人民解放军61646部队 | A kind of subjective assessment processing method, device, electronic equipment and medium towards the full link simulation image of visible light |
| CN111179238B (en) * | 2019-12-24 | 2022-12-20 | 东华大学 | Subset confidence ratio dynamic selection method for underwater image set-oriented guidance consistency enhancement evaluation |
| CN111556192B (en) * | 2020-03-24 | 2021-05-14 | 上海交通大学 | Mobile phone photo dragging comparison sorting method and system |
| CN112950567A (en) * | 2021-02-25 | 2021-06-11 | 北京金山云网络技术有限公司 | Quality evaluation method, quality evaluation device, electronic device, and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR20080029371A (en) * | 2006-09-29 | 2008-04-03 | 광운대학교 산학협력단 | Method of image quality evaluation, and system thereof |
| CN103458267B (en) * | 2013-09-04 | 2016-07-06 | 中国传媒大学 | A kind of video picture quality subjective evaluation and system |
| CN103632369B (en) * | 2013-12-05 | 2016-02-03 | 淮海工学院 | The combination of block average definition is without reference underwater picture universal property quantity measuring method |
-
2019
- 2019-01-09 CN CN201910020772.4A patent/CN109714591B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109714591A (en) | 2019-05-03 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109714591B (en) | Image quality subjective evaluation method and system based on evaluation label | |
| Yang et al. | An underwater color image quality evaluation metric | |
| Chadwick et al. | The perception of gloss: A review | |
| Yee et al. | A perceptual metric for production testing | |
| CN101207832A (en) | Method for checking digital camera color reduction | |
| JP6473401B2 (en) | Skin gloss evaluation apparatus, gloss evaluation method, and gloss evaluation program | |
| Corney et al. | The brightness of colour | |
| Nagai et al. | Temporal properties of material categorization and material rating: visual vs non-visual material features | |
| CN109741285A (en) | A kind of construction method and system of underwater picture data set | |
| Barkowsky et al. | On the perceptual similarity of realistic looking tone mapped high dynamic range images | |
| Te Pas et al. | Perception of object illumination depends on highlights and shadows, not shading | |
| Witzel et al. | Why bananas look yellow: The dominant hue of object colours | |
| Bringier et al. | No-reference perceptual quality assessment of colour image | |
| Kim et al. | Perceptual crosstalk prediction on autostereoscopic 3D display | |
| Tsuda et al. | Constancy of visual working memory of glossiness under real-world illuminations | |
| Vlasuyk et al. | Synthesis of the Perceptionally Linear Color Space Using Machine Learning Methods | |
| Khandual et al. | Colorimetric processing of digital colour image! | |
| Bernardo et al. | A study on the user perception to color variations | |
| Tong et al. | Testing color dif ference evaluation methods for color digital images | |
| Dolhasz et al. | Measuring observer response to object-scene disparity in composites | |
| Funt et al. | Metamer mismatching underlies color difference sensitivity | |
| Cook et al. | 85‐4: a subpixel‐based objective image quality metric with application to visually lossless image compression evaluation | |
| Li et al. | On the limits of perceptual quality measures for enhanced underwater images | |
| Morimoto et al. | Color and gloss constancy under diverse lighting environments | |
| WO2022034741A1 (en) | Measurement device, measurement system, determination method, and determination program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |