CN112330648B - Non-reference image quality evaluation method and device based on semi-supervised learning - Google Patents
Non-reference image quality evaluation method and device based on semi-supervised learning Download PDFInfo
- Publication number
- CN112330648B CN112330648B CN202011261570.8A CN202011261570A CN112330648B CN 112330648 B CN112330648 B CN 112330648B CN 202011261570 A CN202011261570 A CN 202011261570A CN 112330648 B CN112330648 B CN 112330648B
- Authority
- CN
- China
- Prior art keywords
- image
- quality evaluation
- image quality
- brightness
- semi
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000013441 quality evaluation Methods 0.000 title claims abstract description 67
- 238000012549 training Methods 0.000 claims abstract description 40
- 230000004927 fusion Effects 0.000 claims abstract description 13
- 239000003086 colorant Substances 0.000 claims abstract description 9
- 238000012706 support-vector machine Methods 0.000 claims description 17
- 238000005070 sampling Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 14
- 230000000007 visual effect Effects 0.000 claims description 14
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 12
- 230000035945 sensitivity Effects 0.000 claims description 8
- 238000012360 testing method Methods 0.000 claims description 7
- 238000006243 chemical reaction Methods 0.000 claims description 4
- 238000001303 quality assessment method Methods 0.000 abstract description 18
- 238000010586 diagram Methods 0.000 description 5
- 238000011156 evaluation Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000004456 color vision Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000005094 computer simulation Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000007429 general method Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30168—Image quality inspection
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The application relates to a non-reference image quality evaluation method, a non-reference image quality evaluation device, a non-reference image quality evaluation computer device and a non-reference image quality evaluation storage medium based on semi-supervised learning, wherein the non-reference image quality evaluation method comprises the following steps: acquiring an input image to be evaluated; dividing the brightness component of the image into an overexposure region, a low exposure region and a normal exposure region by a clustering method, and extracting the brightness characteristic of the image according to a clustering result; converting the image into a gray image, and extracting texture features of multi-scale fusion of the gray image; extracting chromaticity characteristics of corresponding channels of the image based on three RGB color channels, converting RGB into two opposite color spaces and obtaining chromaticity characteristics based on opposite colors; based on the brightness characteristics, texture characteristics and chromaticity characteristics of the image, training an image quality evaluation model by a semi-supervised learning method, and outputting the trained model for image quality evaluation. The generalization capability and the precision of the non-reference quality assessment model are effectively improved.
Description
Technical Field
The present invention relates to the field of image quality evaluation technologies, and in particular, to a non-reference image quality evaluation method and apparatus based on semi-supervised learning, a computer device, and a storage medium.
Background
The goal of image quality assessment is to measure the visual quality of an image using a computational model to be consistent with human subjective judgment. In many application scenarios, such as image compression, image reconstruction, image enhancement, etc., accurate assessment of visual quality of an image is an important task. Since subjective image quality assessment techniques require a lot of manpower, time, etc., and cannot be used in many scenes (e.g., real-time scenes, etc.), objective image quality assessment techniques need to be developed to automatically and robustly measure image quality.
At present, the objective quality evaluation of the picture quality can be divided into: full reference image quality assessment, half reference image quality assessment, and no reference image quality assessment. The full reference image quality evaluation is to measure the quality of a distorted image based on the difference between the reference image and the distorted image in the case where the full information of the reference image can be acquired. The semi-reference image quality assessment is to measure the quality of the distorted image taking into account only part of the reference image information. The no reference image quality assessment is to measure the quality of the distorted image without any reference image information. Although the three objective evaluation methods have research significance and value according to different application conditions, the information of the reference image cannot be obtained in most practical application scenes. Therefore, the non-reference image quality evaluation has higher practical application value and research value.
The existing reference-free image quality evaluation method can be roughly divided into a method based on natural scene statistics, a method based on a human eye vision model and a method based on learning. Considering that there are now many emerging image types that are becoming more and more important, many conventional general methods based on natural scene statistics are no longer applicable in many application scenarios. In various complex application scenarios, the learning-based method is more effective, meanwhile, the visual characteristics of human eyes are required to be considered in various aspects, and the quality evaluation model is designed to follow the visual characteristics of human eyes, so that the quality evaluation model is necessarily closer to subjective judgment. In addition, since the data amount of the image quality evaluation database is small, the learning-based method easily enters over fitting, so that generalization performance is poor.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a non-reference image quality evaluation method, apparatus, computer device, and storage medium based on semi-supervised learning.
A non-reference image quality evaluation method based on semi-supervised learning, the method comprising:
acquiring an input image to be evaluated;
dividing the brightness component of the image into an overexposure region, a low exposure region and a normal exposure region by a clustering method, and extracting the brightness characteristic of the image according to a clustering result;
converting the image into a gray image, and extracting texture features of multi-scale fusion of the gray image;
extracting chromaticity characteristics of corresponding channels of the image based on three RGB color channels, converting RGB into two opposite color spaces and obtaining chromaticity characteristics based on opposite colors;
based on the brightness characteristics, texture characteristics and chromaticity characteristics of the image, training an image quality evaluation model by a semi-supervised learning method, and outputting the trained model for image quality evaluation.
In one embodiment, the step of training the image quality assessment model by a semi-supervised learning method based on the luminance, texture, and chrominance features of the image includes:
extracting brightness features, texture features and chromaticity features of the image by using a small number of marked data sets and a large number of unmarked data sets;
establishing a semi-supervised regression model for collaborative training through a support vector machine;
and reading the label of the marked data set to train the image quality evaluation model.
In one embodiment, the step of reading in the tag of the marked dataset to train the image quality assessment model further comprises:
reading in a marked sample feature set L and an unmarked sample feature set U, and initializing and training two support vector machine regression models m1 and m2;
the regression model m1 carries out regression prediction on the unmarked sample feature set U1, a sample with the highest marking confidence degree is selected from the prediction results, and the prediction results are added into the training set L2 of m2;
the regression model m2 carries out regression prediction on the unmarked sample feature set U2, a sample with the highest marking confidence degree is selected from the prediction results, and the prediction results are added into the training set L1 of m 1;
updating L1, U1, L2 and U2, and retraining regression models m1 and m2;
judging whether the iteration times reach the maximum iteration times, and repeatedly executing the regression prediction step if the iteration times reach the maximum iteration times;
and if the maximum iteration times are reached, testing the test sample set by using the final regression models m1 and m2 respectively, and taking the average value of the two model prediction results as the final image quality evaluation score.
In one embodiment, the step of dividing the brightness component of the image into an overexposed region, a underexposed region and a normally exposed region by a clustering method, and extracting the brightness feature of the image according to the clustering result includes:
dividing the brightness level of the image by combining a membership fuzzy C-means clustering method;
extracting brightness characteristics of the image according to the clustering result, wherein a calculation formula of the brightness characteristics is as follows:
wherein N is total Representing the total number of pixels of the image, N over Representing the number of pixels of the overexposed region in the image, N normal Representing the number of pixels of the normally exposed area in the image, N under Representing the number of pixels in the low exposure area in the image; w (w) 1 ，w 2 ，w 3 Three weights, namely w, respectively set for the brightness sensitivity curve of human eyes 1 ＝0.774，w 2 ＝0.871，w 3 =0.355, lf is the extracted luminance feature.
In one embodiment, the step of converting the image into a gray scale image and extracting texture features of the gray scale image multi-scale fusion comprises:
calculating radius R 1 The LBP characteristic representation with sampling point k=8 is shown in=1, and the calculation formula is:
wherein T is set to 0 as a threshold value, (g) 0 ,g 1 ,...,g (K-1) ) Represents the gray value of K sampling points g c The gray value of the central pixel point of the local area is phi (·) which is used for calculating the number of bit-wise conversion;
and calculates a frequency histogram of the characteristic representation:
wherein N is b The number of pixel points with the pixel value of b in the LBP feature map is the number of pixel points, and N is the total number of pixels of the LBP feature map;
taking the radii of different sizes as R respectively 2 ＝2，R 3 Repeating the above calculation steps with sampling point k=8 to obtain 3 frequency histograms with the value of=3Is->
Calculating texture features fused with different scales:
wherein HF is a texture feature of the image.
In one embodiment, the step of extracting the chromaticity characteristics of the corresponding channels of the image based on the three RGB color channels includes:
computing visual sensitivity weighted chroma features CF of the image 1 ：
CF 1 ＝0.299log I R +0.587log I G +0.114log I B
Wherein N is the image of the imageTotal number of elements, m represents each color channel of the color image, I m Representing the pixel mean for each color channel.
In one embodiment, the step of converting RGB into two opposing color spaces and obtaining opposing color-based chromaticity characteristics includes:
computing opponent-color-based chroma features CF of the image 2 ：
α＝G-R，β＝B-0.5(R+G)
Where N is the total number of pixels and α, β represent 2 opposing color spaces.
A non-reference image quality evaluation device based on semi-supervised learning, the device comprising:
the image acquisition module is used for acquiring an input image to be evaluated;
the brightness characteristic extraction module is used for dividing brightness components of the image into an overexposure region, a low exposure region and a normal exposure region through a clustering method and extracting brightness characteristics of the image according to a clustering result;
the gray level feature extraction module is used for converting the image into a gray level image and extracting texture features of multi-scale fusion of the gray level image;
the system comprises a chromaticity feature extraction module, a color matching module and a color matching module, wherein the chromaticity feature extraction module is used for extracting chromaticity features of corresponding channels of the image based on three RGB color channels, converting RGB into two opposite color spaces and obtaining chromaticity features based on opposite colors;
the model training module is used for training an image quality evaluation model based on brightness characteristics, texture characteristics and chromaticity characteristics of the image through a semi-supervised learning method and outputting the trained model for image quality evaluation.
A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any one of the methods described above when the computer program is executed.
A computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of any of the methods described above.
The non-reference image quality evaluation method, the non-reference image quality evaluation device, the computer equipment and the storage medium based on semi-supervised learning utilize brightness characteristics obtained by clustering results of image brightness components; converting the image into a gray image and extracting texture features of multi-scale fusion of the image; extracting chromaticity characteristics of corresponding channels of the image based on the three RGB color channels, converting the RGB into two opposite color spaces and obtaining chromaticity characteristics based on opposite colors; and finally, establishing a semi-supervised regression model of cooperative training by using a support vector machine according to three types of characteristics including brightness, texture and chromaticity and limited subjective evaluation scores, and finally obtaining an image quality evaluation model. According to the invention, three types of characteristics related to image quality, including brightness characteristics, texture characteristics and chromaticity characteristics, which accord with the visual characteristics of human eyes are designed, a support vector machine is utilized to establish a cooperative training semi-supervised regression model, and the generalization capability and precision of a non-reference quality assessment model are effectively improved.
Drawings
FIG. 1 is a flow diagram of a non-reference image quality assessment method based on semi-supervised learning in one embodiment;
FIG. 2 is a conceptual diagram of a non-reference image quality assessment method based on semi-supervised learning in one embodiment;
FIG. 3 is a flow chart of a non-reference image quality evaluation method based on semi-supervised learning in another embodiment;
FIG. 4 is a block diagram of a non-reference image quality evaluation device based on semi-supervised learning in one embodiment;
fig. 5 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.
In one embodiment, as shown in fig. 1, there is provided a non-reference image quality evaluation method based on semi-supervised learning, the method comprising:
102, acquiring an input image to be evaluated;
step 104, dividing the brightness component of the image into an overexposure region, a underexposure region and a normal exposure region by a clustering method, and extracting the brightness characteristic of the image according to a clustering result;
step 106, converting the image into a gray image, and extracting texture features of multi-scale fusion of the gray image;
step 108, extracting chromaticity characteristics of corresponding channels of the image based on the three RGB color channels, converting the RGB into two opposite color spaces and obtaining chromaticity characteristics based on the opposite colors;
step 110, training an image quality evaluation model by a semi-supervised learning method based on the brightness features, texture features and chromaticity features of the image, and outputting the trained model for image quality evaluation.
At present, in the existing reference-free image quality evaluation method, the data volume of the image quality evaluation database is small, and the learning-based method is easy to enter over fitting, so that the generalization performance is poor, so that the reference-free image quality evaluation method based on semi-supervised learning is provided in the embodiment to overcome the defect of a small sample.
The overall conception flow of the method is shown in fig. 2, which comprises the following steps: the method provides brightness characteristics obtained by utilizing the clustering result of the brightness components of the image, multi-scale fused image texture characteristics, chromaticity characteristics conforming to the visual characteristics of human eyes and color richness capable of being used for expressing the image. And then, establishing a semi-supervised regression model of collaborative training by using a support vector machine through three types of characteristics of brightness, texture and chromaticity and limited subjective evaluation scores, and finally obtaining an image quality evaluation model. The method comprises the following specific implementation steps:
firstly, an image to be evaluated input by a user is obtained, and the brightness component of the image is divided into an overexposed area, a low exposed area and a normal exposed area by using a clustering method. Considering that uncertainty exists always when human eyes make judgment, in the embodiment, the brightness level of the image is divided by adopting a fuzzy C-means clustering method combined with membership, so that the clustering result is more in line with human eyes.
Specifically, for the clustering result, the brightness features of the input image are extracted. Because the sensitivity of human eyes to different exposure degrees is different, the specific gravity of the areas with different exposure degrees in the whole image is calculated in the embodiment, different weights are added to accord with the visual characteristics of human eyes, and finally the brightness characteristics of the image are obtained. The specific calculation mode is as follows:
wherein N is total Representing the total number of pixels of the image, N over Representing the number of pixels in the overexposed region in the image, N normal Representing the number of pixels in the normally exposed area of the image, N under Representing the number of pixels in the low exposure area of the image; w (w) 1 ，w 2 ，w 3 Three weights, namely w, respectively set for the brightness sensitivity curve of human eyes 1 ＝0.774，w 2 ＝0.871，w 3 =0.355, lf is the extracted luminance feature.
Then, converting the image to be detected into a gray image and obtaining the texture features of the image multi-scale fusion.
In a specific embodiment, the texture features of the image are expressed using a rotation-invariant equivalent LBP operator. In order to consider the influence of different visual fields of human eyes at the same time, the embodiment fuses LBP representations under a plurality of radiuses to obtain the texture characteristics of image multi-scale fusion. The specific calculation process is as follows:
calculating radius R 1 The LBP characteristic representation with sampling point k=8 is shown in=1, and the calculation formula is:
wherein T is set to 0 as a threshold value, (g) 0 ,g 1 ,...,g (K-1) ) Represents the gray value of K sampling points g c The gray value of the central pixel point of the local area is phi (·) which is used for calculating the number of bit-wise conversion;
and calculates a frequency histogram of the feature representation:
wherein N is b The number of pixel points with the pixel value of b in the LBP feature map is the number of pixel points, and N is the total number of pixels of the LBP feature map;
taking the radii of different sizes as R respectively 2 ＝2，R 3 Repeating the above calculation steps with sampling point k=8 to obtain 3 frequency histograms with the value of=3Is->
Calculating texture features fused with different scales:
wherein HF is a texture feature of the image.
Next, the human eye visual system (HVS) characteristics indicate that the human eye has different sensitivities to the RGB three primary color channels. To simulate this, the color intensity of the visual sensitivity weighting is calculated in this embodiment to measure different degrees of color distortion.
CF 1 ＝0.299log I R +0.587log I G +0.114log I B
Wherein N is the total number of pixels of the image, m represents each color channel of the color image, I m Representing the pixel mean for each color channel.
Furthermore, according to studies on human vision, subjective perception is proportional to the logarithm of stimulus, and color perception is mainly performed in the opposing color space. Therefore, the color richness of the image is measured by the combination of the color channels in the logarithmic domain in the present embodiment.
Computing opponent-color-based chrominance features CF of an image 2 ：
α＝G-R，β＝B-0.5(R+G)
Where N is the total number of pixels and α, β represent 2 opposing color spaces.
Finally, by extracting the three types of features described above, including luminance features, texture features, and chrominance features, for a small number of labeled datasets, such as the conventional datasets LIVE, TID2013, CSIQ, etc., and a large number of unlabeled datasets, and training the quality prediction model by reading the labels (MOS or DMOS) of the labeled datasets, a co-trained semi-supervised regression model is built using support vector machines in this embodiment, taking into account the lower computational complexity and higher prediction accuracy desired by the quality assessment task. The model has the advantages of semi-supervised learning, is suitable for solving the condition of lacking a large number of marked samples, combines the advantages of a support vector machine in solving the problems of small samples and nonlinear regression, adopts two regression models trained cooperatively, relieves the defect of easy accumulation and amplification of errors in a semi-supervised self-training algorithm using only a single regression model, and improves the generalization capability of the regression model.
In the above embodiment, the brightness characteristics obtained by using the clustering result of the image brightness components; converting the image into a gray image and extracting texture features of multi-scale fusion of the image; extracting chromaticity characteristics of corresponding channels of the image based on the three RGB color channels, converting the RGB into two opposite color spaces and obtaining chromaticity characteristics based on opposite colors; and finally, establishing a semi-supervised regression model of cooperative training by using a support vector machine according to three types of characteristics including brightness, texture and chromaticity and limited subjective evaluation scores, and finally obtaining an image quality evaluation model. According to the scheme, three types of characteristics related to image quality, including brightness characteristics, texture characteristics and chromaticity characteristics, which accord with the visual characteristics of human eyes are designed, a support vector machine is utilized to establish a cooperative training semi-supervised regression model, and generalization capability and precision of a non-reference quality assessment model are effectively improved.
In one embodiment, as shown in fig. 3, there is provided a non-reference image quality evaluation method based on semi-supervised learning, in which the step of training an image quality evaluation model by the semi-supervised learning method based on luminance features, texture features, and chrominance features of an image includes:
step 302, extracting brightness features, texture features and chromaticity features of the image by using a small number of marked data sets and a large number of unmarked data sets;
step 304, establishing a semi-supervised regression model for collaborative training through a support vector machine;
and 306, reading the label of the marked data set to train an image quality evaluation model.
Specifically, the specific procedure of the training algorithm for reading the label of the marked data set to perform the image quality evaluation model in this embodiment is as follows:
step 3.1: reading in a marked sample feature set L and an unmarked sample feature set U, and initializing and training two Support Vector Machine (SVM) regression models m1 and m2.
Step 3.2: and (3) carrying out regression prediction on the unmarked sample feature set U1 by the regression model m1, selecting a sample with the highest marking confidence degree from the prediction results and adding the prediction result thereof into the training set L2 of m2.
Step 3.3: and (3) carrying out regression prediction on the unmarked sample feature set U2 by the regression model m2, selecting a sample with the highest marking confidence degree from the prediction results and adding the prediction result thereof into the training set L1 of m 1.
Step 3.4: updating L1, U1, L2 and U2, and retraining the regression models m1 and m2.
Step 3.5: and (3) judging whether the iteration times reach the maximum iteration times, and returning to the step (3.2) if the iteration times do not reach the maximum iteration times.
Step 3.6: after the iteration times are reached, the test sample sets are respectively tested by using the final regression models m1 and m2, and the average value of the prediction results of the two models is used as the final objective quality evaluation score.
In this embodiment, three types of image quality-related features, including luminance features, texture features, and chromaticity features, are designed to conform to the visual characteristics of the human eye. In addition, in the embodiment, a support vector machine is utilized to establish a semi-supervised regression model of collaborative training. The semi-supervised learning mode effectively utilizes abundant unlabeled samples, and the collaborative training method utilizes two mutually-influenced regression models, so that generalization capability and precision of the non-reference quality assessment model are effectively improved.
In one embodiment, as shown in fig. 4, there is provided a non-reference image quality evaluation apparatus 400 based on semi-supervised learning, the apparatus comprising:
an image acquisition module 401, configured to acquire an input image to be evaluated;
the brightness feature extraction module 402 is configured to divide a brightness component of the image into an overexposed region, a underexposed region and a normal exposed region by using a clustering method, and extract brightness features of the image according to a clustering result;
a gray feature extraction module 403, configured to convert the image into a gray image, and extract texture features of multi-scale fusion of the gray image;
the chromaticity feature extraction module 404 is configured to extract chromaticity features of corresponding channels of the image based on three color channels of RGB, convert RGB into two opposite color spaces, and obtain chromaticity features based on opposite colors;
the model training module 405 is configured to train an image quality evaluation model by a semi-supervised learning method based on the brightness features, texture features, and chromaticity features of the image, and output the trained model for performing image quality evaluation.
In one embodiment, the model training module 405 is also used to extract luminance, texture, and chrominance features of an image by extracting a small number of labeled datasets and a large number of unlabeled datasets; establishing a semi-supervised regression model for collaborative training through a support vector machine; and reading the label of the marked data set to train the image quality evaluation model.
In one embodiment, the model training module 405 is further configured to read in the marked sample feature set L, the unmarked sample feature set U, and initialize and train two support vector machine regression models m1 and m2; the regression model m1 carries out regression prediction on the unmarked sample feature set U1, a sample with the highest marking confidence degree is selected from the prediction results, and the prediction results are added into the training set L2 of m2; the regression model m2 carries out regression prediction on the unmarked sample feature set U2, a sample with the highest marking confidence degree is selected from the prediction results, and the prediction results are added into the training set L1 of m 1; updating L1, U1, L2 and U2, and retraining regression models m1 and m2; judging whether the iteration times reach the maximum iteration times, and repeatedly executing the regression prediction step if the iteration times reach the maximum iteration times;
and if the maximum iteration times are reached, testing the test sample set by using the final regression models m1 and m2 respectively, and taking the average value of the two model prediction results as the final image quality evaluation score.
The detailed definition of the non-reference image quality evaluation device based on semi-supervised learning can be found in the above definition of the non-reference image quality evaluation method based on semi-supervised learning, and will not be repeated here.
In one embodiment, a computer device is provided, the internal structure of which may be as shown in FIG. 5. The computer device includes a processor, a memory, and a network interface connected by a device bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The nonvolatile storage medium stores an operating device, a computer program, and a database. The internal memory provides an environment for the operation of the operating device and the computer program in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program, when executed by a processor, implements a non-reference image quality assessment method based on semi-supervised learning.
It will be appreciated by those skilled in the art that the structure shown in fig. 5 is merely a block diagram of some of the structures associated with the present application and is not limiting of the computer device to which the present application may be applied, and that a particular computer device may include more or fewer components than shown, or may combine certain components, or have a different arrangement of components.
In one embodiment, a computer device is provided that includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps in the method embodiments above when executing the computer program.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, carries out the steps of the above method embodiments.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the various embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples merely represent a few embodiments of the present application, which are described in more detail and are not to be construed as limiting the scope of the invention. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application is to be determined by the claims appended hereto.
Claims (9)
1. A non-reference image quality evaluation method based on semi-supervised learning, the method comprising:
acquiring an input image to be evaluated;
dividing the brightness component of the image into an overexposure region, a low exposure region and a normal exposure region by a clustering method, and extracting the brightness characteristic of the image according to a clustering result;
converting the image into a gray image, and extracting texture features of multi-scale fusion of the gray image;
extracting chromaticity characteristics of corresponding channels of the image based on three RGB color channels, converting RGB into two opposite color spaces and obtaining chromaticity characteristics based on opposite colors;
training an image quality evaluation model by a semi-supervised learning method based on the brightness features, the texture features and the chromaticity features of the image, and outputting the trained model for image quality evaluation;
the step of converting the image into a gray level image and extracting texture features of the multi-scale fusion of the gray level image comprises the following steps:
calculating radius R 1 The LBP characteristic representation with sampling point k=8 is shown in=1, and the calculation formula is:
wherein T and T' are set to 0, g as thresholds 0 ,g 1 ,...,g (K-1) Represents the gray value of K sampling points g c Is the center of the local areaThe gray value of the pixel points, psi (·) is used for calculating the number of bitwise conversions; LBP K,R For the LBP characteristic value g with the radius R of the sampling number K i Is the gray value of the sampling points around the center pixel point, s () is the gray value used to calculate LBP K,R Intermediate parameters of (2);
and calculates a frequency histogram of the characteristic representation:
wherein N is b The number of pixel points with the pixel value of b in the LBP feature map is the number of pixel points, and N is the total number of pixels of the LBP feature map;
taking the radii of different sizes as R respectively 2 ＝2，R 3 Repeating the above calculation steps with sampling point k=8 to obtain 3 frequency histograms with the value of=3Is->
Calculating texture features fused with different scales:
wherein HF is a texture feature of the image.
2. The non-reference image quality evaluation method based on semi-supervised learning as set forth in claim 1, wherein the training of the image quality evaluation model by the semi-supervised learning method based on the luminance feature, the texture feature, and the chrominance feature of the image includes:
extracting brightness features, texture features and chromaticity features of the image by using a small number of marked data sets and a large number of unmarked data sets;
establishing a semi-supervised regression model for collaborative training through a support vector machine;
and reading the label of the marked data set to train the image quality evaluation model.
3. The non-reference image quality evaluation method based on semi-supervised learning as recited in claim 2, wherein the step of reading in the labels of the labeled dataset for training of the image quality evaluation model further comprises:
reading in a marked sample feature set L and an unmarked sample feature set U, and initializing and training two support vector machine regression models m1 and m2;
the regression model m1 carries out regression prediction on the unmarked sample feature set U1, a sample with the highest marking confidence degree is selected from the prediction results, and the prediction results are added into the training set L2 of m2;
the regression model m2 carries out regression prediction on the unmarked sample feature set U2, a sample with the highest marking confidence degree is selected from the prediction results, and the prediction results are added into the training set L1 of m 1;
updating L1, U1, L2 and U2, and retraining regression models m1 and m2;
judging whether the iteration times reach the maximum iteration times, and repeatedly executing the regression prediction step if the iteration times reach the maximum iteration times;
and if the maximum iteration times are reached, testing the test sample set by using the final regression models m1 and m2 respectively, and taking the average value of the two model prediction results as the final image quality evaluation score.
4. The non-reference image quality evaluation method based on semi-supervised learning as set forth in claim 1, wherein the step of dividing the brightness component of the image into an overexposed region, a underexposed region, and a normally exposed region by a clustering method, and extracting the brightness feature of the image according to the clustering result comprises:
dividing the brightness level of the image by combining a membership fuzzy C-means clustering method;
extracting brightness characteristics of the image according to the clustering result, wherein a calculation formula of the brightness characteristics is as follows:
wherein N is total Representing the total number of pixels of the image, N over Representing the number of pixels of the overexposed region in the image, N normal Representing the number of pixels of the normally exposed area in the image, N under Representing the number of pixels in the low exposure area in the image; w (w) 1 ，w 2 ，w 3 Three weights, namely w, respectively set for the brightness sensitivity curve of human eyes 1 ＝0.774，w 2 ＝0.871，w 3 =0.355, lf is the extracted luminance feature.
5. The non-reference image quality evaluation method based on semi-supervised learning as set forth in claim 1, wherein the step of extracting chromaticity characteristics of the image corresponding channels based on three RGB color channels includes:
computing visual sensitivity weighted chroma features CF of the image 1 ：
CF 1 ＝0.299logI R +0.587logI G +0.114logI B
Wherein N is the total number of pixels of the image, m represents each color channel of the color image, I m The pixel mean value of each color channel is represented, and i and j are pixel point locations.
6. The non-reference image quality evaluation method based on semi-supervised learning as set forth in claim 5, wherein the step of converting RGB into two opponent color spaces and obtaining opponent color-based chromaticity characteristics includes:
computing opponent-color-based chroma features CF of the image 2 ：
α＝G-R，β＝B-0.5(R+G)
Where N is the total number of pixels, α, β represents 2 opposing color spaces, and i and j are pixel point locations.
7. A non-reference image quality evaluation device based on semi-supervised learning, the device comprising:
the image acquisition module is used for acquiring an input image to be evaluated;
the brightness characteristic extraction module is used for dividing brightness components of the image into an overexposure region, a low exposure region and a normal exposure region through a clustering method and extracting brightness characteristics of the image according to a clustering result;
the gray level feature extraction module is used for converting the image into a gray level image and extracting texture features of multi-scale fusion of the gray level image;
the system comprises a chromaticity feature extraction module, a color matching module and a color matching module, wherein the chromaticity feature extraction module is used for extracting chromaticity features of corresponding channels of the image based on three RGB color channels, converting RGB into two opposite color spaces and obtaining chromaticity features based on opposite colors;
the model training module is used for training an image quality evaluation model based on brightness characteristics, texture characteristics and chromaticity characteristics of the image through a semi-supervised learning method and outputting the trained model for image quality evaluation;
the gray feature extraction module is further used for:
calculating radius R 1 The LBP characteristic representation with sampling point k=8 is shown in=1, and the calculation formula is:
wherein T and T' are set to 0, g as thresholds 0 ,g 1 ,...,g (K-1) Represents the gray value of K sampling points g c The gray value of the central pixel point of the local area is phi (·) which is used for calculating the number of bit-wise conversion; LBP K,R For the LBP characteristic value g with the radius R of the sampling number K i Is the gray value of the sampling points around the center pixel point, s () is the gray value used to calculate LBP K,R Intermediate parameters of (2);
and calculates a frequency histogram of the characteristic representation:
wherein N is b The number of pixel points with the pixel value of b in the LBP feature map is the number of pixel points, and N is the total number of pixels of the LBP feature map;
taking the radii of different sizes as R respectively 2 ＝2，R 3 Repeating the above calculation steps with sampling point k=8 to obtain 3 frequency histograms with the value of=3Is->
Calculating texture features fused with different scales:
wherein HF is a texture feature of the image.
8. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the method according to any one of claims 1 to 6 when the computer program is executed by the processor.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011261570.8A CN112330648B (en) | 2020-11-12 | 2020-11-12 | Non-reference image quality evaluation method and device based on semi-supervised learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011261570.8A CN112330648B (en) | 2020-11-12 | 2020-11-12 | Non-reference image quality evaluation method and device based on semi-supervised learning |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112330648A CN112330648A (en) | 2021-02-05 |
| CN112330648B true CN112330648B (en) | 2024-01-05 |
Family
ID=74317355
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011261570.8A Active CN112330648B (en) | 2020-11-12 | 2020-11-12 | Non-reference image quality evaluation method and device based on semi-supervised learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112330648B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113362354B (en) * | 2021-05-07 | 2024-04-30 | 安徽国际商务职业学院 | Quality evaluation method, system, terminal and storage medium for tone mapping image |
| CN115457614B (en) * | 2021-05-20 | 2023-08-04 | 马上消费金融股份有限公司 | Image quality evaluation method, model training method and device |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110046673A (en) * | 2019-04-25 | 2019-07-23 | 上海大学 | No reference tone mapping graph image quality evaluation method based on multi-feature fusion |
| CN110706196A (en) * | 2018-11-12 | 2020-01-17 | 浙江工商职业技术学院 | Clustering perception-based no-reference tone mapping image quality evaluation algorithm |
| WO2020077866A1 (en) * | 2018-10-17 | 2020-04-23 | 平安科技（深圳）有限公司 | Moire-based image recognition method and apparatus, and device and storage medium |
-
2020
- 2020-11-12 CN CN202011261570.8A patent/CN112330648B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020077866A1 (en) * | 2018-10-17 | 2020-04-23 | 平安科技（深圳）有限公司 | Moire-based image recognition method and apparatus, and device and storage medium |
| CN110706196A (en) * | 2018-11-12 | 2020-01-17 | 浙江工商职业技术学院 | Clustering perception-based no-reference tone mapping image quality evaluation algorithm |
| CN110046673A (en) * | 2019-04-25 | 2019-07-23 | 上海大学 | No reference tone mapping graph image quality evaluation method based on multi-feature fusion |
Non-Patent Citations (1)
| Title |
|---|
| 基于人眼感知的无参考色调映射图像质量评价;马华林;张立燕;;计算机应用与软件(07);第164-170页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112330648A (en) | 2021-02-05 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Gu et al. | Learning a no-reference quality assessment model of enhanced images with big data | |
| JP6330385B2 (en) | Image processing apparatus, image processing method, and program | |
| CN110930353A (en) | Method and device for detecting state of hole site protection door, computer equipment and storage medium | |
| CN108010024B (en) | Blind reference tone mapping image quality evaluation method | |
| CN111680746B (en) | Vehicle damage detection model training, vehicle damage detection method, device, equipment and medium | |
| CN112330648B (en) | Non-reference image quality evaluation method and device based on semi-supervised learning | |
| CN110059700B (en) | Image moire recognition method and device, computer equipment and storage medium | |
| CN112419270B (en) | No-reference image quality evaluation method and device under meta-learning and computer equipment | |
| CN114241505B (en) | Method and device for extracting chemical structure image, storage medium and electronic equipment | |
| CN107146220B (en) | A kind of universal non-reference picture quality appraisement method | |
| CN111160114B (en) | Gesture recognition method, gesture recognition device, gesture recognition equipment and computer-readable storage medium | |
| CN111260645B (en) | Tampered image detection method and system based on block classification deep learning | |
| WO2021147866A1 (en) | Ecg signal acquisition method and device, storage medium, and electronic device | |
| CN113205017A (en) | Cross-age face recognition method and device | |
| Morzelona | Human visual system quality assessment in the images using the IQA model integrated with automated machine learning model | |
| CN113743378B (en) | Fire monitoring method and device based on video | |
| CN111179245B (en) | Image quality detection method, device, electronic equipment and storage medium | |
| CN116543001B (en) | Color image edge detection method and device, equipment and storage medium | |
| CN111612732A (en) | Image quality evaluation method, image quality evaluation device, computer equipment and storage medium | |
| US20230386023A1 (en) | Method for detecting medical images, electronic device, and storage medium | |
| CN116959712A (en) | Lung adenocarcinoma prognosis method, system, equipment and storage medium based on pathological image | |
| Hong et al. | Saliency-based feature learning for no-reference image quality assessment | |
| CN114519729A (en) | Image registration quality evaluation model training method and device and computer equipment | |
| CN112699809A (en) | Acne category identification method and device, computer equipment and storage medium | |
| Yáñez et al. | Image quality assessment on identity documents |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |