CN108765366A - It is a kind of based on autonomous learning without with reference to color image quality evaluation method - Google Patents

Abstract

The invention relates to a self-learning-based non-reference color image quality evaluation method, which belongs to the field of image processing. The method includes: firstly, using the quaternion theory to represent the color images in the training set with a quaternion matrix; secondly, dividing the training image into blocks and using human visual perception to obtain the local features of the image block; then, using the autonomous The learning strategy constructs the image dictionary, and the most representative image blocks are used as the atoms of the dictionary; finally, after the same preprocessing is performed on the color image to be evaluated, the maximum similarity between the image to be evaluated and the image dictionary is calculated, and the support vector Regression analysis gets the final quality evaluation score and updates the image dictionary in real time. The invention fully considers the representativeness of the dictionary atoms in the image dictionary and the self-learning ability, and can evaluate images of different distortion types at the same time, and the evaluation result tends to be consistent with the subjective evaluation.

Description

A no-reference color image quality assessment method based on autonomous learning

technical field

The invention belongs to the technical field of image processing, in particular to the field of image quality evaluation methods, and relates to a no-reference image quality evaluation method based on autonomous learning.

Background technique

Image quality evaluation technology has always been a key technology in the field of image processing, which can be used to evaluate the effect of image processing methods, or to select appropriate image processing methods based on image quality. Therefore, image quality evaluation technology plays a very important role in image processing.

According to whether reference images are needed, image quality assessment algorithms can be divided into three categories: full-reference quality assessment, part-reference image quality assessment and no-reference quality assessment. Full-reference quality assessment needs to rely on complete reference image information, partial-reference quality assessment requires partial reference image information, and no-reference quality assessment method does not require reference images. Because the no-reference quality evaluation method is more in line with the practical application value, it has received more attention and research.

The current no-reference evaluation methods basically start from the grayscale image, but in addition to the contrast distortion of the grayscale image, the image will also have color distortion such as hue shift and saturation reduction, which makes the no-reference color image Quality evaluation is more in line with reality. Existing no-reference color image quality assessment methods usually convert it to a grayscale image or measure the quality of each color component individually and then combine the measured values with different weights to apply grayscale measurements to the color image. The former will be lost in the grayscale conversion process and ignore the color information of the image. The latter is difficult to determine the weight to find the best color model. Therefore, the present invention directly starts from the three primary colors of the color image to satisfy the no-reference effective evaluation of the color image.

Contents of the invention

In view of this, the purpose of the present invention is to provide a method for evaluating the quality of color images without reference based on autonomous learning, which has strong representation of atoms in the dictionary and can effectively evaluate images of different distortion types; The increase of the number of times can independently learn and update the image dictionary, and has wide applicability.

To achieve the above object, the present invention provides the following technical solutions:

A non-reference color image quality evaluation method based on autonomous learning. Firstly, an autonomous learning strategy is used to select representative samples to construct an image dictionary, and then the constructed image dictionary is used to perform a mapping relationship with the image to be evaluated to obtain a quality evaluation score. Update image dictionaries in real time using autonomous learning strategies;

The method specifically includes the following steps:

S1: For color images, the pixels in the three color channels of red (R), green (G), and blue (B) are represented by a hypercomplex number through quaternion theory, and the quaternion matrix of the color image is obtained;

S2: Divide the image into blocks, extract the local features of the image block through the visual characteristics of the human eye, and eliminate the correlation between the image blocks;

S3: Use the self-learning strategy to independently select the image block with the least similarity among the image blocks, and judge the difference between the image block and all atoms in the dictionary. If the difference is small, put it into the dictionary, and cycle in turn until reaching the dictionary dimension when output dictionary;

S4: Obtain the final quality evaluation score through the mapping relationship between the image to be evaluated and the dictionary, and through the support vector regression SVR method;

S5: Update the dictionary in real time with an autonomous learning strategy according to the image to be evaluated and the obtained quality evaluation score.

Further, the step S1 specifically includes:

A group of color images with known subjective evaluation score DMOS values are used as training samples, and the pixels in the three color channels of red (R), green (G) and blue (B) in each color image are used with quaternion 3 imaginary parts, the real part is 0, so that each pixel of the color image is represented as a pure quaternion:

f(x,y)=f _R (x,y) i+f _G (x,y) j+f _B (x,y) k

Among them, x and y respectively represent the coordinates of the pixel in the image, f _R (x, y), f _G (x, y), f _B (x, y) are the coordinates in the corresponding color channel (x, y ), i, j, k are the three imaginary units of the quaternion.

Further, the step S2 specifically includes the following steps:

S21: Decompose each image into non-overlapping image blocks with a scale of d×d, let x _c be the center point of the image block, then the other pixels in the block are x ₁ , x ₂ ,...x _n , then the The pixel difference value y' of the image block can be obtained by subtracting other pixels in the image block from x _c respectively, and its mathematical expression is:

y'＝(x ₁ -x _c ,x ₂ -x _c ,…,x _n -x _c )

S22: Since the response of the human eye to the image has a logarithmic nonlinear characteristic, the pixel difference value of the image block can be represented by a local feature vector through the nonlinear perception characteristic of the human eye, and its mathematical expression is:

z=sign(y') log(|y'|+1)

S23: Use the difference between image blocks to eliminate similar image blocks, where the difference can be obtained through the angle between image blocks, that is

Among them, D(z _i ) represents the difference between the image block z _i and other image blocks in the training set U, z _i ·z _j represents the inner product between image blocks, and ||·|| represents the modulus value of the vector; if D(z _i )=0 means that the two image blocks are the same, and the latter image block can be deleted to eliminate the similarity between the image blocks;

S24: Using principal component analysis (PCA) to whiten the image block, the redundant information of the image block can be eliminated, and the mathematical expression is:

Among them, x _i is the original image feature, x _PCAwhite,i is the image feature after whitening, λ _i is the PCA transformation moment, C is a small constant to avoid the denominator being 0, and m is the total number of image blocks.

Further, the step S3 specifically includes the following steps:

S31: Initialization: set the training set as U, the number of atoms to be constructed as K, the dictionary S=Φ, and Φ is an empty set;

S32: Estimate the similarity between the image blocks in the training set U, that is, calculate the Euclidean distance and angle between the image blocks:

Among them, R( _zi ) _represents the minimum similarity between the image block z in the training set U and other image blocks, is the Euclidean distance between image patches z _i and z _j , is the angle between image blocks z _i and z _j .

S33: Sort the image blocks in the training set U from small to large in similarity, put the first K image blocks into the dictionary S in order, and construct an initial dictionary;

S34: Calculate the minimum difference value d between the K+1th image block z _K+1 and all atoms in the dictionary S, where the difference is the angle between the image block z _K+1 and the atoms in the dictionary S, mathematics The expression is:

Among them, s _j represents the atom in the dictionary S, z _K+1 · s _j represents the inner product between the image block and the atom s _j in the dictionary, and ||·|| represents the modulus of the vector.

Similarly, use the same method to calculate the minimum difference value D(s _i ) among the atoms in the dictionary S, where s _i is the atom with the smallest difference between the atoms in the dictionary and other atoms.

S35: If the minimum difference value d>D, then update the dictionary, that is, replace the atom x _j in the dictionary S with the image block z _t , then make K=K+1 and return to S34; otherwise, go to S36

S36: Output a dictionary.

Further, the step S4 specifically includes:

S41: Preprocess the color image to be evaluated according to steps S1 and S2 to obtain a set of local feature vectors of image blocks after the image to be evaluated is divided into blocks in Represents the local feature vector of a certain image block in the image to be evaluated;

S42: Calculate each image block by using the Euclidean distance and the related formula of the included angle between the image blocks Maximum similarity with atoms in dictionary S:

in, Indicates the image block in the image to be evaluated The maximum similarity value with all atoms in the dictionary S, is the image block and the Euclidean distance between atoms s _j in the dictionary, is the image block and the angle between the atoms s _j in the dictionary.

S43: Summarize the maximum similarity between all the image blocks of the image to be evaluated and the atoms in the dictionary S in the form of a vector matrix, put them into the support vector regression SVR method, combine the DMOS value prediction of the corresponding atoms to obtain the image quality score . Where the vector matrix can be expressed as:

Further, the step S5 specifically includes:

S51: Calculate the image block with the least similarity among the image blocks of the image to be evaluated whose image quality score is known as the most representative image block, and calculate the difference between the image block and all atoms in the dictionary, and determine the minimum difference value d ;;

S52: Calculate the difference between all atoms in the dictionary, and determine the atom with the smallest difference and the corresponding difference value t;

S53: Determine whether the difference value d is greater than the difference value t, if it is greater, replace the image block with the atom with the smallest difference value in the dictionary, and return to S51 to continue execution, otherwise, the dictionary is not updated;

S54: Output the updated dictionary.

The beneficial effects of the present invention are: the method of the present invention designs an image dictionary that can be learned independently to realize strong representation of atoms in the image dictionary, and can update the dictionary in real time, so that the color image quality evaluation effect under different situations can achieve good evaluation results.

Description of drawings

In order to make the purpose, technical scheme and beneficial effect of the present invention clearer, the present invention provides the following drawings for illustration:

Fig. 1 is the flow chart of the color image quality evaluation method of the present invention;

FIG. 2 is a flow chart of the autonomous learning of the image dictionary according to the present invention.

Detailed ways

The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

The present invention autonomously selects the atoms in the dictionary according to the similarity between the image blocks in the color image training set after the image is divided into blocks and the difference between the image blocks and the atoms in the dictionary. Each atom is very representative, even The lower dimension also has a significant evaluation effect; at the same time, as the number of training and testing samples increases, it can learn to update the image dictionary autonomously, and has wide applicability.

As shown in Figure 1, the non-reference color image quality evaluation method based on autonomous learning of the present invention specifically includes the following steps:

1. Image preprocessing

1.1. A group of color images with known subjective evaluation score DMOS values are used as a training data set, and the pixels in the three color channels of red (R), green (G) and blue (B) in each color image are used four The three imaginary parts of the number represent, and the real part is 0, so that the color image is represented by a pure quaternion matrix. Compared with the traditional sub-channel processing or the method of processing after transforming into a grayscale image, the quaternion method can better reflect the integrity of the color image.

The present invention can use common LIVE, CSIQ, TID and other database images as the training database, of course, it can also select the image library of the equipment to be tested according to the needs, and organize subjective evaluation, so as to achieve the consistency between the used data and subjective experience.

1.2. Each image is decomposed into non-overlapping image blocks with a scale of d×d. Since the correlation between pixels can well describe the distortion of the image, the present invention calculates the distance between a pixel and adjacent pixels in a loop. Grayscale difference features.

According to the logarithmic nonlinearity of the human visual response, the image is more in line with the human visual perception after logarithmic transformation, so the local feature vector of each image block is obtained based on the nonlinear perception characteristics of the human eye, and then in the form of a set represent the features of each image.

1.3. Use the difference between image blocks to eliminate similar image blocks. When the difference value is 0, it means that the two image blocks are the same, and the latter image block can be deleted to eliminate the similarity between image blocks. This is mainly because image blocks often appear Repeated structure, this operation can ensure the independence of each training sample, and can also reduce the amount of calculation and increase timeliness;

1.4, in order to remove the correlation between image features, reduce the redundant information of image block, the present invention utilizes principal component analysis (PCA) to carry out whitening to image block:

Among them, x _i is the original image feature, x _PCAwhite,i is the image feature after whitening, λ _i is the PCA transformation moment, C is a small constant to avoid the denominator being 0, and m is the total number of image blocks.

2. Image dictionary construction

2.1. Initialization: Let the training set be U, the number of atoms to be constructed is K, the dictionary S=Φ, and Φ is an empty set;

2.2. Estimate the similarity between the image blocks in the training set U, that is, calculate the Euclidean distance and angle between the image blocks:

Among them, R( _zi ) _represents the minimum similarity between the image block z in the training set U and other image blocks, is the Euclidean distance between image patches z _i and z _j , is the angle between image blocks z _i and z _j .

2.3. Sort the image blocks in the training set U from small to large in similarity, put the first K image blocks into the dictionary S in order, and construct the initial dictionary;

2.4. Calculate the minimum difference value d between the K+1th image block z _K+1 and all atoms in the dictionary S, where the difference is the angle between the image block z _K+1 and the atoms in the dictionary S, mathematics The expression is:

Among them, s _j represents the atom in the dictionary S, z _K+1 · s _j represents the inner product between the image block and the atom s _j in the dictionary, and ||·|| represents the modulus of the vector.

Similarly, use the same method to calculate the minimum difference value D(s _i ) among the atoms in the dictionary S, where s _i is the atom with the smallest difference between the atoms in the dictionary and other atoms.

2.5. If the minimum difference value d>D, then update the dictionary, that is, replace the atom x _j in the dictionary S with the image block z _t , then make K=K+1 and return 2.4; otherwise, go to 2.6;

2.6. Output dictionary.

3. Image quality evaluation

3.1. Preprocess the color image to be evaluated according to the methods of step 1 and step S2, and obtain the local feature vector set of the image block after the image to be evaluated is divided into blocks in Represents the local feature vector of a certain image block in the image to be evaluated;

3.2. Calculate each image block using the Euclidean distance and angle-related formulas between image blocks Maximum similarity with atoms in dictionary S:

in, Indicates the image block in the image to be evaluated The maximum similarity value with all atoms in the dictionary S, is the image block and the Euclidean distance between atoms s _j in the dictionary, is the image block and the angle between the atoms s _j in the dictionary.

3.3. Summarize the maximum similarity between all image blocks of the image to be evaluated and the atoms in the dictionary S in the form of a vector matrix, and put them into the support vector regression SVR method, and combine the DMOS value prediction of the corresponding atoms to obtain the image quality score . Where the vector matrix can be expressed as:

4. The image dictionary is updated, as shown in Figure 2:

4.1. Calculate the image block with the least similarity between the image blocks of the image to be evaluated with the known image quality score, as the most representative image block, and calculate the difference between the image block and all atoms in the dictionary, and determine the minimum difference value d;

4.2. Calculate the difference between all atoms in the dictionary, and determine the atom with the smallest difference and the corresponding difference value t;

4.3. Determine whether the difference value d is greater than the difference value t. If it is greater, replace the image block with the atom with the minimum difference value in the dictionary, and return to 4.1 to continue execution. Otherwise, the dictionary will not be updated;

4.4. Output the updated dictionary.

Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that it can be described in terms of form and Various changes may be made in the details without departing from the scope of the invention defined by the claims.

Claims (6)

1. A no-reference color image quality evaluation method based on autonomous learning is characterized in that firstly, a sample with strong representativeness is selected by using an autonomous learning strategy to construct an image dictionary, then a mapping relation is carried out by using the constructed image dictionary and an image to be evaluated to obtain a quality evaluation score, and finally, the image dictionary is updated in real time by using the autonomous learning strategy;

the method specifically comprises the following steps:

s1: aiming at a color image, expressing pixels in three color channels of red R, green G and blue B by a hypercomplex number through a quaternion theory to obtain a quaternion matrix of the color image;

s2: the image is processed in a blocking mode, the local features of the image blocks are extracted through the visual characteristics of human eyes, and the correlation among the image blocks is eliminated;

s3: utilizing an autonomous learning strategy to autonomously select an image block with the minimum similarity in the image blocks, judging the differences between the image block and all atoms in the dictionary, if the differences are small, putting the image block into the dictionary, and sequentially circulating until the dimension of the dictionary is reached and outputting the dictionary;

s4: obtaining a final quality evaluation score through a mapping relation between the image to be evaluated and the dictionary and a Support Vector Regression (SVR) method;

s5: and updating the dictionary in real time by using an autonomous learning strategy according to the image to be evaluated and the obtained quality evaluation score.

2. The method for evaluating the quality of a color image without reference based on autonomous learning according to claim 1, wherein the step S1 specifically comprises:

a group of color images with known subjective evaluation score DMOS values are used as training samples, pixels in three color channels of red R, green G and blue B in each color image are represented by 3 imaginary parts of quaternions, the real part is 0, and therefore each pixel of the color images is represented by a pure quaternion:

f(x,y)＝f_R(x,y)·i+f_G(x,y)·j+f_B(x,y)·k

wherein x and y respectively represent the coordinates of the pixel points in the image, and f_R(x,y)、f_G(x,y)、f_B(x, y) are pixel values corresponding to coordinates (x, y) in the color channel, respectively, and i, j, k are 3 imaginary units of quaternion.

3. The method for evaluating the quality of a color image without reference based on autonomous learning according to claim 1, wherein the step S2 specifically comprises the following steps:

s21: decomposing each image into non-overlapping image blocks with dimension dxd, and setting x_cIf it is the central point of the image block, the other pixel points in the block are x₁,x₂,…x_nThen, the other pixels in the image block are respectively compared with x_cSubtracting to obtain a pixel difference value y' of the image block, wherein the mathematical expression is as follows:

y'＝(x₁-x_c,x₂-x_c,…,x_n-x_c)

s22: since the response of human eyes to images has a logarithmic nonlinearity characteristic, the pixel difference value of an image block can be represented by a local feature vector through the nonlinear perception characteristic of human eyes, and the mathematical expression is as follows:

z＝sign(y')·log(|y'|+1)

wherein z represents a local feature of the image block;

s23: eliminating similar image blocks by using differences between image blocks, wherein the differences are obtained by the included angles between the image blocks, i.e.

Wherein D (z)_i) Representing image blocks z in a training set U_iDifference from other image blocks, z_i·z_jRepresenting the inner product between image blocks, | | | |, represents the modulus of the vector; if D (z)_i) If it is 0, it indicates that the two image blocks are the same, and the latter image block can be deleted to eliminate the similarity between the image blocks;

s24: whitening an image block by using a Principal Component Analysis (PCA) method, and eliminating redundant information of the image block, wherein a mathematical expression is as follows:

wherein x is_iIs the original image feature, x_PCAwhite,iIs the whitened image feature, λ_iIs the PCA transformation moment, C is a small constant to avoid denominator of 0, and m is the total number of image blocks.

4. The method for evaluating the quality of a color image without reference based on autonomous learning according to claim 1, wherein the step S3 specifically comprises the following steps:

s31: initialization: setting a training set as U, setting the atom number to be constructed as K, setting the dictionary S as phi, and setting phi as an empty set;

s32: and (3) estimating the similarity among the image blocks in the training set U, namely calculating the Euclidean distance and the included angle among the image blocks:

wherein R (z)_i) Representing image blocks z in a training set U_iThe minimum similarity with other image blocks,is an image block z_iAnd z_jThe euclidean distance between them,is an image block z_iAnd z_jThe included angle between them;

s33: sequencing image blocks in a training set U from small to large in similarity, and sequentially placing the first K image blocks into a dictionary S to construct an initial dictionary;

s34: computing the K +1 th image block z_K+1And the minimum difference value d between all atoms in the dictionary S, wherein the difference is the image block z_K+1And the included angle between the atom in the dictionary S is shown as the mathematical expression:

wherein s is_jRepresenting atoms in a dictionary S, z_K+1·s_jRepresenting image blocks and atoms in dictionaries s_jInner product between, | | | |, represents the module value of the vector;

similarly, the dictionary S primitive is calculated by the same methodMinimum value of intersubular dissimilarity D(s)_i) Wherein s is_iThe atom with the smallest difference between the atoms in the dictionary and other atoms;

s35: if the minimum difference value D > D, the dictionary is updated, i.e. the image block z is used_tReplacing atoms x in a dictionary S_jThen K ═ K +1 and return to S34; otherwise, go to S36;

s36: and outputting the dictionary.

5. The method according to claim 3, wherein the step S4 specifically comprises:

s41: preprocessing the color image to be evaluated according to the steps S1 and S2 to obtain a local feature vector set of the image block of the image to be evaluated after the image is blockedWhereinRepresenting a local feature vector of a certain image block in an image to be evaluated;

s42: calculating each image block by using Euclidean distance and included angle correlation formula between image blocksMaximum similarity to atoms in dictionary S:

wherein,representing image blocks in an image to be evaluatedMaximum of all atoms in dictionary SThe value of the similarity is set to be,is an image blockAnd an atom in the dictionary s_jThe euclidean distance between them,is an image blockAnd an atom in the dictionary s_jThe included angle between them;

s43: summarizing the maximum similarity between all image blocks of the image to be evaluated and atoms in a dictionary S according to a vector matrix form, putting the image blocks into a Support Vector Regression (SVR) method, and predicting by combining DMOS values of corresponding atoms to obtain an image quality score; wherein the vector matrix is represented as:

6. the method for evaluating the quality of a color image without reference based on autonomous learning according to claim 1, wherein the step S5 specifically comprises:

s51: calculating the similarity between image blocks of an image to be evaluated with known image quality scores, taking the smallest image block as the most representative image block, calculating the difference between the image block and all atoms in a dictionary, and determining the minimum difference value d; (ii) a

S52: calculating the difference between all atoms in the dictionary, and determining the atom with the minimum difference and the corresponding difference value t;

s53: judging whether the difference value d is larger than the difference value t, if so, replacing the atom with the minimum difference value in the dictionary with the image block, returning to S51 for continuous execution, otherwise, not updating the dictionary;

s54: and outputting the updated dictionary.