CN105913413B - An Objective Evaluation Method for Color Image Quality Based on Online Manifold Learning - Google Patents

Abstract

The invention discloses an objective evaluation method for color image quality based on online manifold learning, which takes into account the relationship between saliency and objective evaluation of image quality, uses a visual saliency detection algorithm, and obtains the respective saliency maps of a reference image and a distorted image by using a visual saliency detection algorithm. to obtain the maximum fused saliency map, and based on the maximum saliency of the image blocks in the maximum fused saliency map, the absolute difference value is used to measure the significant difference between the reference image block and the corresponding distorted image block, and the reference image block is extracted from the filter. Visually important image blocks and distorted visually important image blocks, and then use the manifold feature vectors of the reference visually important image blocks and the distorted visually important image blocks to calculate the objective quality evaluation value of the distorted image, the evaluation effect is significantly improved, and the objective evaluation results are consistent with subjective perception high correlation between them.

Description

An Objective Evaluation Method for Color Image Quality Based on Online Manifold Learning

technical field

The invention relates to an image quality evaluation method, in particular to an objective evaluation method of color image quality based on online manifold learning.

Background technique

Limited by the performance of the image processing system, various types of distortion will be introduced in the process of image acquisition, transmission and encoding. The introduction of distortion will reduce the quality of the image and also hinder people from obtaining information from the image. Image quality is an important index to compare the performance of various image processing algorithms and the parameters of image processing systems. Therefore, it is of great value to construct effective image quality evaluation methods in the fields of image transmission, multimedia network communication, and video analysis. Generally, image quality evaluation methods are divided into two categories: subjective evaluation and objective evaluation. Since the final destination of the image is human, the subjective evaluation method is the most reliable evaluation method, but it is time-consuming and labor-intensive, and it is not easy to embed in the image processing system, so limited in practical applications. In contrast, objective evaluation methods have the advantages of simple operation, convenience and practicality, and are currently the focus of research in academia and industry.

At present, the simplest and most widely used objective evaluation methods are peak signal tonoise ratio (PSNR) and mean square error (MSE). Considering the visual characteristics of the human eye, the evaluation results often do not match the subjective perception of the human eye. In fact, the processing of image signals by the human eye is not carried out point by point. In view of this, the researchers introduced the visual characteristics of the human eye to make the objective evaluation result more consistent with the visual perception of the human eye. For example, based on structural similarity (Structural Similarity, SSIM), the structural information of the image is characterized from three aspects of image brightness, contrast and structure, and then the image quality is evaluated. In its follow-up work, based on SSIM, a multi-scale SSIM evaluation method, a complex wavelet SSIM evaluation method and an SSIM evaluation method based on information content weighting are proposed to improve the performance of SSIM. In addition to the evaluation method based on structural similarity, Sheikh et al. regard the full reference image quality evaluation as an information fidelity problem. Information Fidelity, VIF) image quality assessment method. Starting from the critical threshold and super-threshold characteristics of visual perception of images, combined with wavelet transform, Chandler et al. proposed an image quality evaluation method based on wavelet visual signal-to-noise ratio (VSNR). Better adapt to different visual conditions. Although researchers have deeply explored the human visual system, due to the complexity of the human eye system, the cognition of the human visual system is still relatively superficial, so it is still impossible to propose an objective evaluation method of image quality that is completely consistent with the subjective perception of the human eye.

In order to better reflect the characteristics of the human visual system, objective evaluation methods of image quality based on sparse representation and visual attention have attracted more and more attention. Many studies have shown that sparse representations can well describe the activity of neurons in the primary visual cortex of the human brain. For example, Guha et al. disclose an image quality evaluation method based on sparse representation. The method is divided into two stages. The first stage is the dictionary learning stage: image patches randomly selected from the reference image are used as training samples, using The KSVD algorithm trains an over-complete dictionary; the second stage is the evaluation stage: use the orthogonal matching pursuit algorithm (OMP) to sparsely encode the image blocks in the reference image and the corresponding image blocks in the distorted image, and obtain the reference image sparse coefficients And the sparse coefficient of the distorted image, and then the objective evaluation value of the image is obtained. However, these sparse representation-based image quality objective evaluation methods all need to use the orthogonal matching pursuit algorithm for sparse coding, which requires a lot of motion overhead. Moreover, the acquisition of over-complete dictionaries in this type of method is done through offline operations, which requires a large number of effective The natural images are used as training samples, and there are limitations for image processing with real-time requirements.

For such high-dimensional data of digital images, there is essentially a large amount of information redundancy, which needs to be processed by dimensionality reduction technology, and it is expected that its essential structure can be maintained while reducing dimensionality. Manifold Learning has become a research hotspot in the field of information science since it was first proposed in the famous scientific journal "Science" in 2000. Assuming that the data is a low-dimensional manifold uniformly sampled in a high-dimensional Euclidean space, manifold learning is to recover the low-dimensional manifold structure from the high-dimensional sampling data, that is, to find the low-dimensional manifold in the high-dimensional space, and find the low-dimensional manifold in the high-dimensional space. The corresponding embedding map is generated to achieve dimensionality reduction. Studies have shown that manifolds are the basis of perception, and things are perceived in a manifold manner in the brain. In recent years, manifold learning has been widely used in image denoising, face recognition, human behavior detection, etc., and has achieved good results. Aiming at the problem that the column vectors in the Locality Preserving Projection (LPP) algorithm are not orthogonal, Deng et al. improved it to obtain the Orthogonal Locality Preserving Projection (OLPP) algorithm, which can find the flow of data. It has a linear structure and has a better local retention ability and discriminative ability. Manifold learning can simulate the description of image signals in primary visual cortex cells, so that the visual perception features of images can be accurately extracted. The low-dimensional manifold feature of the image can describe the nonlinear change relationship between the distorted images well, and the distorted images will be arranged in the manifold space according to the type of change and the magnitude of the intensity. Therefore, it is necessary to study an objective evaluation method of image quality based on manifold learning, which has a high degree of agreement between the objective evaluation results and human visual perception.

SUMMARY OF THE INVENTION

The technical problem to be solved by the present invention is to provide an objective evaluation method for color image quality based on online manifold learning, which can effectively improve the correlation between objective evaluation results and subjective perception.

The technical scheme adopted by the present invention to solve the above-mentioned technical problems is: an objective evaluation method for color image quality based on online manifold learning, which is characterized by comprising the following steps:

① Let IR denote an ^undistorted reference image with a width of W and a height of ^H , and let ID denote the distorted image to be ^evaluated corresponding to IR;

(2) Using the visual saliency detection algorithm, obtain the ^saliency maps of IR and ID respectively, and denote them as ^MR and ^{MD respectively} ; then according to ^MR and ^MD , calculate the maximum fusion saliency map, denoted as ^MF , and denote ^MF The pixel value of the pixel with the middle coordinate position (x, y) is recorded as M ^F (x, y), M ^F (x, y) = max(MR (x, y), ^{M D} (x, y) ), where ^1≤x≤W , ^1≤y≤H , max() is the function of taking the maximum value, and MR (x, y) represents the pixel value of the pixel whose coordinate position is (x, y) in MR , ^MD (x, y) represents the pixel value of the pixel ^whose coordinate position is (x, y) in MD;

③ ^Divide IR , ID , ^MR , ^MD and ^MF into 8×8 sliding ^windows respectively. non-overlapping image blocks of the same size;

Then, the color values of the ^R , G, and ^B channels of all pixels in each image block in each of IR and ID are vectorized, and the ^R , G, and B channels of all pixels in the jth image block in IR are vectorized. The color vector formed by the vectorization of the color values of the G and B channels is recorded as The color vector formed by quantizing the color values of the R, G, and ^B channels of all pixels in the jth image block in ID is denoted as Among them, the initial value of j is 1, and The dimensions of are 192×1, The values of the 1st element to the 64th element in IR are in a one-to-one correspondence with the color value of the ^R channel of each pixel point in the jth image block in the IR scanned in a progressive scan manner, The values of the 65th element to the 128th element in the one-to-one correspondence are the color values of the G channel of each pixel point in the ^jth image block in the IR scanned in a progressive scan manner, The values of the 129th element to the 192th element in the one-to-one correspondence are the color values of the ^B channel of each pixel point in the jth image block in the IR scanned in a progressive scan manner, The values of the 1st element to the 64th element in the one-to-one correspondence are the color values of the R channel of each pixel in the ^jth image block in the ID scanned in a progressive scan manner, The values of the 65th element to the 128th element in the one-to-one correspondence are the color values of the ^G channel of each pixel in the jth image block in the ID scanned in a progressive scan manner, The value of the 129th element to the 192nd element in the one-to-one correspondence is the color value of the ^B channel of each pixel point in the jth image block in the ID scanned in a progressive scanning manner;

^Quantize the pixel values of all pixels in each image block in ^MR , ^MD and ^MF , and quantize the pixel values of all pixels in the jth image block in MR to form The pixel value vector of is denoted as The pixel value vector formed by ^quantizing the pixel value vector of all pixels in the jth image block in MD is denoted as The pixel value vector formed by ^quantizing the pixel value vector of all pixels in the jth image block in MF is denoted as in, and The dimensions of are 64×1, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the jth image block in the ^MR scanned in a progressive scanning manner, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the ^jth image block in the MD scanned in a progressive scan manner, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the ^jth image block in the progressive scanning mode scanning MF;

④ Calculate the ^saliency of each image block in MF, and record the ^saliency of the jth image block in MF as d _j , Among them, 1≤i≤64, express the value of the i-th element in ;

Then arrange the ^saliency of all image blocks in MF in descending order, and then determine the sequence number of the image blocks corresponding to the first t ₁ saliency, where, λ ₁ represents the scale coefficient of image block selection, λ ₁ ∈(0,1];

Then find out the image blocks corresponding to the determined ^t1 serial numbers in IR, and define them as reference image blocks _; find out the image ^blocks corresponding to the determined _t1 serial numbers in ID, and define them as distorted image blocks Find out the image block corresponding to the determined ^t ₁ serial numbers in ^MR , and define it as a reference significant image block; Find out the image block corresponding to the determined t ₁ serial numbers in MD, and define it as a significant distortion image block;

^⑤Use the absolute difference to measure the significant difference between each reference image block in IR and the corresponding distorted image block in ID, and compare the ^t'th reference image block in ^IR with the ^t'th in ID The significant difference value of the distorted image block is denoted as e _t ', Among them, the initial value of t' is 1, 1≤t'≤t1, the symbol "||" is the absolute value symbol, Represents the pixel value vector corresponding to the ^t'th reference salient image block in MR the value of the ith element in , Represents the pixel value vector corresponding to the ^t'th distorted image block in MD the value of the i-th element in ;

Then arrange the measured t ₁ significant difference values in descending order, and then determine the reference image blocks and distorted image blocks corresponding to the first t ₂ significant difference values after sorting, and assign the determined t ₂ reference image blocks to the It is defined as a reference visual important image block, and the matrix formed by the color vectors corresponding to all reference visual important image blocks is used as a reference visual important image block matrix, denoted as Y ^R ; The determined t ₂ distorted image blocks are defined as distorted vision important image blocks, and the matrix formed by the color vectors corresponding to all distorted visual important image blocks is taken as the distorted visual important image block matrix, denoted as Y ^D , where t ₂ =λ ₂ ×t ₁ , λ ₂ represents the reference image block and Distorted image block selects the scale coefficient, λ ₂ ∈(0,1], the dimensions of Y ^R and Y ^D are both 192×t ₂ , and the t”th column vector in Y ^R is the determined t”th reference. The color vector corresponding to the image block, the t"-th column vector in Y ^D is the color vector corresponding to the determined t"-th distorted image block, and the initial value of t" is 1, 1≤t"≤t ₂ ;

⑥ Center the value of each element in each column vector in Y ^R minus the mean value of the values of all elements in the column vector, and denote the matrix obtained after centering as Y, where the dimension of Y is The number is 192×t ₂ ;

Then use principal component analysis to perform dimension reduction and whitening operations on Y to obtain a matrix after dimension reduction and whitening operations, which is denoted as Y ^w , Y ^w =W×Y, where the dimension of Y ^w is M×t ₂ , W Indicates the whitening matrix, the dimension of W is M×192, 1<M<<192, and the symbol "<<" is much smaller than the symbol;

^⑦Using the orthogonal local preserving projection algorithm to train ^Yw online to obtain the feature base matrix of Yw, which is denoted as D, where the dimension of D is M×192;

⑧ Calculate the manifold feature vector of each reference visually important image block according to Y ^R and D, and denote the manifold feature vector of the t”th reference visually important image block as u _t” , Among them, the dimension of u _t” is M×1, is the t"th column vector in Y ^R ; and according to Y ^D and D, calculate the manifold feature vector of each distorted visual important image block, and record the manifold feature vector of the t" distorted visual important image block as v _t” , Among them, the dimension of v _t" is M × 1, is the t"th column vector in Y ^D ;

⑨ According to the manifold feature vectors of all reference visually important image blocks and the manifold feature vectors of all distorted ^visually important image blocks, calculate the objective quality evaluation value of ID, denoted as Score, Among them, 1≤m≤M, u _t” (m) represents the value of the mth element in u _t” , v _t” (m) represents the value of the mth element in v _t” , and C is a very Small constant, used to guarantee the stability of the result.

The acquisition process of Y ^w in the described step ⑥ is: ⑥_1, let C represent the covariance matrix of Y, Among them, the dimension of C is 192×192, and Y ^T is the transpose of Y; ⑥_2, decompose the eigenvalues of C to obtain all the largest eigenvalues and corresponding eigenvectors, among which, the dimension of the eigenvectors is 192×1 ; ⑥_3, take M maximum eigenvalues and corresponding M eigenvectors; ⑥_4, calculate whitening matrix W according to the taken M maximum eigenvalues and corresponding M eigenvectors, W=Ψ- ^1/2 ×E ^T , where the dimension of Ψ is M×M, Ψ=diag(ψ ₁ ,...,ψ _M ), The dimension of E is 192×M, E=[e ₁ ,...,e _M ], diag() is the main diagonal matrix representation, ψ ₁ ,...,ψ _M corresponds to the selected ₁ , _. dimension and the matrix Y ^w after the whitening operation, Y ^w =W×Y.

In the step (4), λ ₁ =0.7 is taken.

In the step ⑤, λ ₂ =0.6 is taken.

In the described step ⑨, take C=0.04.

Compared with the prior art, the advantages of the present invention are:

1) The method of the present invention takes into account the relationship between saliency and objective evaluation of image quality, and uses a visual saliency detection algorithm to obtain the maximum fusion saliency map by obtaining the respective saliency maps of the reference image and the distorted image, and in the maximum fusion saliency map. On the basis of the maximum saliency of the image block, the absolute difference value is used to measure the significant difference between the reference image block and the corresponding distorted image block, and then the reference visual important image block and the distorted visual important image block are filtered and extracted, and then the reference visual image block is extracted. The objective quality evaluation value of the distorted image is calculated by using the manifold feature vector of the important image block and the important image block of distorted vision, the evaluation effect is obviously improved, and the correlation between the objective evaluation result and the subjective perception is high.

2) The method of the present invention starts from the image data itself and searches for the intrinsic geometric structure of the data by means of manifold learning, obtains the feature base matrix through training, and uses the feature base matrix to reduce the dimension of the reference visual important image block and the distortion important image block to obtain the manifold The feature vector, the manifold feature vector after dimensionality reduction still maintains the geometric characteristics of high-dimensional image data, reduces a lot of redundant information, and is simpler and more accurate when calculating the objective quality evaluation value of the distorted image.

3) The method of the present invention is aimed at the problem that offline learning and acquisition of over-complete dictionaries in the existing sparse representation-based image quality objective evaluation methods require a large number of effective training samples and have limitations in image processing with real-time requirements. The feature basis matrix is obtained by online learning and training of the orthogonal local preserving projection algorithm with reference to the visually important image blocks, and the feature basis matrix can be obtained in real time, so the robustness is higher and the evaluation effect is more stable.

Description of drawings

Fig. 1 is the overall realization block diagram of the method of the present invention;

Fig. 2a is the scatter fitting curve diagram of the inventive method in the LIVE image database;

Fig. 2b is the scatter fitting curve diagram of the method of the present invention in the CSIQ image database;

Fig. 2c is a scatter fitting curve diagram of the method of the present invention in the TID2008 image database.

Detailed ways

The present invention will be further described in detail below with reference to the embodiments of the accompanying drawings.

A method for objective evaluation of color image quality based on online manifold learning proposed by the present invention, its overall implementation block diagram is shown in Figure 1, which includes the following steps:

① Let IR denote the ^undistorted reference image with width W and height ^H , and let ID denote the distorted image to be evaluated corresponding to IR ^.

2) Adopt the existing visual saliency detection algorithm (Saliency Detection based on ^SimplePriors , ^SDSP ) to obtain the respective saliency maps of IR and ID respectively, which are correspondingly recorded as ^MR and ^MD ; then according to ^MR and ^MD , calculate the maximum The fused saliency map is denoted as MF , and the pixel value of the pixel whose coordinate position is (x, y) in ^MF is denoted as ^MF (x, y), ^MF (x, y)=max( ^{MR (} x, y), MD (x, y)), where ^1≤x≤W , ^1≤y≤H , max() is the function of taking the maximum value, and MR (x, y) ^represents the coordinate position in MR is the pixel value of the pixel at (x, y), and ^MD (x, y) represents the pixel value of the ^pixel at the coordinate position (x, y) in MD.

③ ^Divide IR , ID , ^MR , ^MD and ^MF into 8×8 sliding ^windows respectively. There are non-overlapping image blocks of the same size. If the size of the image cannot be divisible by 8×8, the extra pixels will not be processed.

Then, the color values of the ^R , G, and ^B channels of all pixels in each image block in each of IR and ID are vectorized, and the ^R , G, and B channels of all pixels in the jth image block in IR are vectorized. The color vector formed by the vectorization of the color values of the G and B channels is recorded as The color vector formed by quantizing the color values of the R, G, and ^B channels of all pixels in the jth image block in ID is denoted as Among them, the initial value of j is 1, and The dimensions of are 192×1, The values of the 1st element to the 64th element in IR correspond one-to-one to the color value of the ^R channel of each pixel point in the jth image block in the IR scanned in a progressive scan manner, that is, The value of the 1st element in IR is the color value of the ^R channel of the pixel in the 1st row and 1st column in the jth image block in IR, The value of the second element in IR is the color value of the ^R channel of the pixel in the 1st row and 2nd column in the jth image block in IR, and so on; The values of the 65th element to the 128th element in the one-to-one correspondence are the color values of the G channel of each pixel point in the ^jth image block in the IR scanned in a progressive scan manner, that is, The value of the 65th element in IR is the color value of the G channel of the pixel in the 1st row and 1st column in the ^jth image block in IR, The value of the ^66th element in IR is the color value of the G channel of the pixel in the 1st row and 2nd column in the jth image block in IR, and so on; The values of the 129th element to the 192th element in the one-to-one correspondence are the color values of the ^B channel of each pixel point in the jth image block in the IR scanned in a progressive scanning manner, that is, The value of the 129th element in IR is the color value of the ^B channel of the pixel in the 1st row and 1st column in the jth image block in IR, The value of the 130th element in IR is the color value of the ^B channel of the pixel in the 1st row and 2nd column in the jth image block in IR, and so on; The values of the 1st element to the 64th element in the one-to-one correspondence are the color values of the R channel of each pixel point in the ^jth image block in the ID scanned in a progressive scan manner, that is The value of the 1st element in ID is the color value of the R channel of the pixel in the 1st row and 1st column in the ^jth image block in ID, The value of the second element in the ID is the color value of the R channel of the pixel in the 1st row and 2nd column in the ^jth image block in ID, and so on; The values of the 65th element to the 128th element in the one-to-one correspondence are the color values of the ^G channel of each pixel in the jth image block in the ID scanned in a progressive scan manner, that is, The value of the 65th element in ID is the color value of the ^G channel of the pixel in the 1st row and 1st column in the jth image block in ID, The value of the 66th element in the ID is the color value of the ^G channel of the pixel in the 1st row and 2nd column in the jth image block in ID, and so on; The values of the 129th element to the 192th element in the one-to-one correspondence are the color values of the ^B channel of each pixel point in the jth image block in the ID scanned in a progressive scan manner, that is, The value of the 129th element in ID is the color value of the ^B channel of the pixel in the 1st row and 1st column in the jth image block in ID, The value of the 130th element in ID is the color value of the ^B channel of the pixel in the 1st row and 2nd column in the jth image block in ID, and so on.

^Quantize the pixel values of all pixels in each image block in ^MR , ^MD and ^MF , and quantize the pixel values of all pixels in the jth image block in MR to form The pixel value vector of is denoted as The pixel value vector formed by ^quantizing the pixel value vector of all pixels in the jth image block in MD is denoted as The pixel value vector formed by ^quantizing the pixel value vector of all pixels in the jth image block in MF is denoted as in, and The dimensions of are 64×1, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the jth image block in the ^MR scanned in the progressive scanning manner, that is The value of the first element in ^MR is the pixel value of the pixel in the 1st row and 1st column in the jth image block in MR, The value of the second element in ^MR is the pixel value of the pixel in the 1st row and 2nd column in the jth image block in MR, and so on; The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the ^jth image block in the MD scanned in a progressive scan manner, that is The value of the first element in MD is the pixel value of the pixel in the 1st row and 1st column in the ^jth image block in MD, The value of the second element in MD is the pixel value of the pixel in the 1st row and 2nd column in the ^jth image block in MD, and so on; The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel point in the ^jth image block in the MF scanned in the progressive scanning manner, that is The value of the first element in MF is the pixel value of the pixel in the 1st row and 1st column in the ^jth image block in MF, The value of the second element in MF is the pixel value of the pixel in the first row and second column in the ^jth image block in MF, and so on.

④ Calculate the ^saliency of each image block in MF, and record the ^saliency of the jth image block in MF as d _j , Among them, 1≤i≤64, express The value of the i-th element in , that is, the pixel value of the i-th ^pixel in the j-th image block in MF.

Then arrange the ^saliency of all image blocks in MF in descending order, and then determine the sequence numbers of the image blocks corresponding to the first t ₁ saliency (that is, the largest t ₁ saliency), where, λ ₁ represents a scale coefficient for image block selection, λ ₁ ∈(0,1], and λ ₁ =0.7 in this embodiment.

Then find out the image blocks corresponding to the determined ^t1 serial numbers in IR, and define them as reference image blocks _; find out the image ^blocks corresponding to the determined _t1 serial numbers in ID, and define them as distorted image blocks Find out the image block corresponding to the determined ^t ₁ serial numbers in ^MR , and define it as a reference significant image block; Find out the image block corresponding to the determined t ₁ serial numbers in MD, and define it as a significant distortion image block.

^⑤Use the absolute difference to measure the significant difference between each reference image block in IR and the corresponding distorted image block in ID, and compare the ^t'th reference image block in ^IR with the ^t'th in ID The significant difference value of the distorted image block is denoted as e _t' , Among them, the initial value of t' is 1, 1≤t'≤t ₁ , the symbol "|·|" is the absolute value symbol, Represents the pixel value vector corresponding to the ^t'th reference salient image block in MR The value of the i-th element in , that is, the pixel value of the i-th pixel in the ^t' -th reference salient image block in MR, Represents the pixel value vector corresponding to the ^t'th distorted image block in MD The value of the i-th element in , that is, the pixel value of the i-th pixel point in the ^t' -th distorted image block in MD.

Then arrange the measured t ₁ significant difference values in descending order, and then determine the reference image blocks and distorted image blocks corresponding to the first t ₂ significant difference values (that is, the largest t ₂ significant differences) , the determined t ₂ reference image blocks are defined as reference visually important image blocks, and the matrix formed by the color vectors corresponding to all reference visually important image blocks is used as the reference visually important image block matrix, denoted as Y ^R ; the determined The t ₂ distorted image blocks are defined as distorted visual important image blocks, and the matrix formed by the color vectors corresponding to all distorted visual important image blocks is taken as the distorted visual important image block matrix, denoted as Y ^D , where t ₂ =λ ₂ ×t ₁ , λ ₂ represents the scale coefficient selected for the reference image block and the distorted image block, λ ₂ ∈(0,1], in this embodiment, λ ₂ =0.6, and the dimensions of Y ^R and Y ^D are both 192× t ₂ , the t"-th column vector in Y ^R is the color vector corresponding to the determined t"-th reference image block, and the t"-th column vector in Y ^D is the determined t"-th distorted image block The corresponding color vector, the initial value of t" is 1, 1≤t"≤t ₂ .

⑥ Center the value of each element in each column vector in Y ^R minus the mean value of the values of all elements in the column vector, and denote the matrix obtained after centering as Y, where the dimension of Y is The number is 192×t ₂ .

Then use the existing principal component analysis (Principal Components Analysis, PCA) to perform dimension reduction and whitening operations on Y obtained after the centralization process, and obtain the matrix after dimension reduction and whitening operations, denoted as Y ^w , Y ^w =W× Y, wherein the dimension of Y ^w is M×t ₂ , W represents the whitening matrix, the dimension of W is M×192, 1<M<<192, and the symbol “<<” is much smaller than the symbol.

In this embodiment, the principal component analysis process is realized by eigenvalue decomposition of the sample data covariance matrix, that is, the acquisition process of Y ^w in step ⑥ is: ⑥_1, let C represent the covariance matrix of Y, Among them, the dimension of C is 192×192, and Y ^T is the transpose of Y; ⑥_2, decompose the eigenvalues of C to obtain all the largest eigenvalues and corresponding eigenvectors, among which, the dimension of the eigenvectors is 192×1 6-3, get M maximum eigenvalues and corresponding M eigenvectors, to realize the dimensionality reduction operation to Y, get M=8 in the present embodiment, namely only got the first 8 principal components for training, also That is to say, the dimension is reduced from 192 dimensions to M=8 dimensions; ⑥_4. Calculate the whitening matrix W according to the M largest eigenvalues and the corresponding M eigenvectors, W=Ψ ^-1/2 ×E ^T , where, The dimension of Ψ is M×M, Ψ=diag(ψ ₁ ,...,ψ _M ), The dimension of E is 192×M, E=[e ₁ ,...,e _M ], diag() is the main diagonal matrix representation, ψ ₁ ,...,ψ _M corresponds to the selected ₁ , _. dimension and the matrix Y ^w after the whitening operation, Y ^w =W×Y.

^⑦Using the existing Orthogonal Locality Preserving Projection (OLPP) algorithm to train Yw online to obtain the feature basis matrix of ^Yw , denoted as D, where the dimension of D is M×192.

⑧ Calculate the manifold feature vector of each reference visually important image block according to Y ^R and D, and denote the manifold feature vector of the t”th reference visually important image block as u _t” , Among them, the dimension of u _t” is M×1, is the t"th column vector in Y ^R ; and according to Y ^D and D, calculate the manifold feature vector of each distorted visual important image block, and record the manifold feature vector of the t" distorted visual important image block as v _t” , Among them, the dimension of v _t" is M × 1, is the t"th column vector in Y ^D.

⑨ According to the manifold feature vectors of all reference visually important image blocks and the manifold feature vectors of all distorted ^visually important image blocks, calculate the objective quality evaluation value of ID, denoted as Score, Among them, 1≤m≤M, u _t” (m) represents the value of the mth element in u _t” , v _t” (m) represents the value of the mth element in v _t” , and C is a very A small constant is used to ensure the stability of the result, and in this embodiment, C=0.04.

In order to further illustrate the feasibility and effectiveness of the method of the present invention, experiments were carried out.

In this embodiment, three public authoritative image databases are selected, namely, the LIVE image database, the CSIQ image database, and the TID2008 image database for experiments. Each index of each image database is detailed in Table 1, including the number of reference images, the number of distorted images, and the number of distortion types. Among them, each image database provides the average subjective score difference for each distorted image.

Table 1 Indicators of authoritative image database

image database number of reference images Number of distorted images Number of Distortion Types LIVE 29 779 5 CSIQ 30 866 6 TID2008 25 1700 17

Next, the correlation between the objective quality evaluation value of each distorted image obtained by the method of the present invention and the average subjective score difference is analyzed. Here, three commonly used objective parameters for evaluating image quality evaluation methods are used as evaluation indicators, namely Pearson Linear Correlation Coefficients (PLCC) to reflect the accuracy of prediction, Spearman Rank Order Correlation coefficient (SROCC) ) reflects the monotonicity of the forecast, and the root mean squared error (RMSE) reflects the consistency of the forecast. Among them, the value range of PLCC and SROCC is [0, 1]. The closer the value is to 1, the better the objective evaluation method of image quality is, and vice versa, the worse it is; the smaller the RMSE value, the more accurate the prediction of the objective image quality evaluation method. , the better the performance, and vice versa, the worse.

For all the distorted images in the above-mentioned LIVE image database, CSIQ image database and TID2008 image database, according to the process of step 1 to step 9 of the method of the present invention, the objective quality evaluation value of each distorted image is obtained by calculating in the same way. The correlation between the objective quality evaluation value of the distorted image obtained by the experiment and the average subjective score difference was analyzed. First, the objective quality evaluation value is obtained, and then the objective quality evaluation value is nonlinearly fitted with a five-parameter Logistic function, and finally the performance index value between the objective evaluation result and the average subjective score difference is obtained. In order to verify the effectiveness of the present invention, the method of the present invention is compared and analyzed on the three image databases listed in Table 1 with the existing six kinds of objective evaluation methods of full reference image quality with relatively advanced performance, indicating that the three image databases The PLCC, SROCC and RMSE coefficients of the evaluation performance are listed in Table 2. The six methods involved in the comparison in Table 2 are: the classic PSNR method, the structural similarity-based evaluation method (SSIM) proposed by Z. Wang, N .Damera Venkata's method based on degradation model (IFC), H.R.Sheikh's method based on information fidelity criterion (VIF), D.M.Chandler's method based on visual signal-to-noise ratio (VSNR) in wavelet domain, T. The sparse representation-based image quality assessment method (SPARQ) proposed by Guha. It can be seen from the data listed in Table 2 that the performance of the method of the present invention is second only to the VIF method on the LIVE image database, and the best performance on both the CSIQ image database and the TID image database. There is a good correlation between the objective quality evaluation value of the distorted image calculated by the method of the present invention and the average subjective score difference. In addition, the PLCC and SROCC values of the LIVE image database and the CSIQ image database both exceed 0.94, and the PLCC and SROCC values of the TID2008 image database with more complex distortion types also reach 0.82, and the performance of the method of the present invention after the weighted average is better than the existing ones. The six methods have different degrees of improvement. It is shown that the objective evaluation results of the method of the present invention are consistent with the subjective perception results of human eyes, and the evaluation effect is stable, which fully demonstrates the effectiveness of the method of the present invention.

Table 2 The performance comparison between the method of the present invention and the existing objective evaluation method of image quality

Fig. 2a shows the scatter fitting curve diagram of the method of the present invention in the LIVE image database, Fig. 2b shows the scatter point fitting curve diagram of the method of the present invention in the CSIQ image database, and Fig. 2c shows the method of the present invention Scatter-fit curve plot in the TID2008 image database. It can be clearly seen from Fig. 2a, Fig. 2b and Fig. 2c that the scatter points are distributed near the fitting line and exhibit good monotonicity and continuity.

Claims (4)

1. a color image quality objective evaluation method based on online manifold learning is characterized in that comprising the following steps: ① Let IR denote an ^undistorted reference image with a width of W and a height of ^H , and let ID denote the distorted image to be ^evaluated corresponding to IR; (2) Using the visual saliency detection algorithm, obtain the ^saliency maps of IR and ID respectively, and denote them as ^MR and ^{MD respectively} ; then according to ^MR and ^MD , calculate the maximum fusion saliency map, denoted as ^MF , and denote ^MF The pixel value of the pixel with the middle coordinate position (x, y) is recorded as M ^F (x, y), M ^F (x, y) = max(MR (x, y), ^{M D} (x, y) ), where ^1≤x≤W , ^1≤y≤H , max() is the function of taking the maximum value, and MR (x, y) represents the pixel value of the pixel whose coordinate position is (x, y) in MR , ^MD (x, y) represents the pixel value of the pixel ^whose coordinate position is (x, y) in MD; ③ ^Divide IR , ID , ^MR , ^MD and ^MF into 8×8 sliding ^windows respectively. non-overlapping image blocks of the same size; Then, the color values of the ^R , G, and ^B channels of all pixels in each image block in each of IR and ID are vectorized, and the ^R , G, and B channels of all pixels in the jth image block in IR are vectorized. The color vector formed by the vectorization of the color values of the G and B channels is recorded as The color vector formed by quantizing the color values of the R, G, and ^B channels of all pixels in the jth image block in ID is denoted as Among them, the initial value of j is 1, and The dimensions of are 192×1, The values of the 1st element to the 64th element in IR are in a one-to-one correspondence with the color value of the ^R channel of each pixel point in the jth image block in the IR scanned in a progressive scan manner, The values of the 65th element to the 128th element in the one-to-one correspondence are the color values of the G channel of each pixel point in the ^jth image block in the IR scanned in a progressive scan manner, The values of the 129th element to the 192th element in the one-to-one correspondence are the color values of the ^B channel of each pixel point in the jth image block in the IR scanned in a progressive scan manner, The values of the 1st element to the 64th element in the one-to-one correspondence are the color values of the R channel of each pixel in the ^jth image block in the ID scanned in a progressive scan manner, The values of the 65th element to the 128th element in the one-to-one correspondence are the color values of the ^G channel of each pixel in the jth image block in the ID scanned in a progressive scan manner, The value of the 129th element to the 192nd element in the one-to-one correspondence is the color value of the ^B channel of each pixel point in the jth image block in the ID scanned in a progressive scanning manner; ^Quantize the pixel values of all pixels in each image block in ^MR , ^MD and ^MF , and quantize the pixel values of all pixels in the jth image block in MR to form The pixel value vector of is denoted as The pixel value vector formed by ^quantizing the pixel value vector of all pixels in the jth image block in MD is denoted as The pixel value vector formed by ^quantizing the pixel value vector of all pixels in the jth image block in MF is denoted as in, and The dimensions of are 64×1, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the jth image block in the ^MR scanned in a progressive scanning manner, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the ^jth image block in the MD scanned in a progressive scan manner, The values of the 1st element to the 64th element in the one-to-one correspondence are the pixel values of each pixel in the ^jth image block in the progressive scanning mode scanning MF; ④ Calculate the ^saliency of each image block in MF, and record the ^saliency of the jth image block in MF as d _j , Among them, 1≤i≤64, express the value of the i-th element in ; Then arrange the ^saliency of all image blocks in MF in descending order, and then determine the sequence number of the image blocks corresponding to the first t ₁ saliency, where, λ ₁ represents the scale coefficient of image block selection, λ ₁ ∈(0,1]; Then find out the image blocks corresponding to the determined ^t1 serial numbers in IR, and define them as reference image blocks _; find out the image ^blocks corresponding to the determined _t1 serial numbers in ID, and define them as distorted image blocks Find out the image block corresponding to the determined ^t ₁ serial numbers in ^MR , and define it as a reference significant image block; Find out the image block corresponding to the determined t ₁ serial numbers in MD, and define it as a significant distortion image block; ^⑤Use the absolute difference to measure the significant difference between each reference image block in IR and the corresponding distorted image block in ID, and compare the ^t'th reference image block in ^IR with the ^t'th in ID The significant difference value of the distorted image block is denoted as e _t ', Among them, the initial value of t' is 1, 1≤t'≤t ₁ , the symbol "||" is the symbol for taking the absolute value, Represents the pixel value vector corresponding to the ^t'th reference salient image block in MR the value of the ith element in , Represents the pixel value vector corresponding to the ^t'th distorted image block in MD the value of the i-th element in ; Then arrange the measured t ₁ significant difference values in descending order, and then determine the reference image blocks and distorted image blocks corresponding to the first t ₂ significant difference values after sorting, and assign the determined t ₂ reference image blocks to the It is defined as a reference visual important image block, and the matrix formed by the color vectors corresponding to all reference visual important image blocks is used as a reference visual important image block matrix, denoted as Y ^R ; The determined t ₂ distorted image blocks are defined as distorted vision important image blocks, and the matrix formed by the color vectors corresponding to all distorted visual important image blocks is taken as the distorted visual important image block matrix, denoted as Y ^D , where t ₂ =λ ₂ ×t ₁ , λ ₂ represents the reference image block and Distorted image block selects the scale coefficient, λ ₂ ∈(0,1], the dimensions of Y ^R and Y ^D are both 192×t ₂ , and the t”th column vector in Y ^R is the determined t”th reference. The color vector corresponding to the image block, the t"-th column vector in Y ^D is the color vector corresponding to the determined t"-th distorted image block, and the initial value of t" is 1, 1≤t"≤t ₂ ; ⑥ Center the value of each element in each column vector in Y ^R minus the mean value of the values of all elements in the column vector, and denote the matrix obtained after centering as Y, where the dimension of Y is The number is 192×t ₂ ; Then use principal component analysis to perform dimension reduction and whitening operations on Y to obtain a matrix after dimension reduction and whitening operations, which is denoted as Y ^w , Y ^w =W×Y, where the dimension of Y ^w is M×t ₂ , W Indicates the whitening matrix, the dimension of W is M×192, 1<M<<192, and the symbol "<<" is much smaller than the symbol; ^⑦Using the orthogonal local preserving projection algorithm to train ^Yw online to obtain the feature base matrix of Yw, which is denoted as D, where the dimension of D is M×192; ⑧ Calculate the manifold feature vector of each reference visually important image block according to Y ^R and D, and denote the manifold feature vector of the t”th reference visually important image block as u _t” , Among them, the dimension of u _t” is M×1, is the t"th column vector in Y ^R ; and according to Y ^D and D, calculate the manifold feature vector of each distorted visual important image block, and record the manifold feature vector of the t" distorted visual important image block as v _t” , Among them, the dimension of v _t" is M × 1, is the t"th column vector in Y ^D ; ⑨ According to the manifold feature vectors of all reference visually important image blocks and the manifold feature vectors of all distorted ^visually important image blocks, calculate the objective quality evaluation value of ID, denoted as Score, Among them, 1≤m≤M, u _t” (m) represents the value of the mth element in u _t” , v _t” (m) represents the value of the mth element in v _t” , and C is a very Small constant, used to ensure the stability of the result; The acquisition process of Y ^w in the described step ⑥ is: ⑥_1, let C represent the covariance matrix of Y, Among them, the dimension of C is 192×192, and Y ^T is the transpose of Y; ⑥_2, decompose the eigenvalues of C to obtain all the largest eigenvalues and corresponding eigenvectors, among which, the dimension of the eigenvectors is 192×1 ; ⑥_3, take M maximum eigenvalues and corresponding M eigenvectors; ⑥_4, calculate whitening matrix W according to the taken M maximum eigenvalues and corresponding M eigenvectors, W=Ψ- ^1/2 ×E ^T , where the dimension of Ψ is M×M, Ψ=diag(ψ ₁ ,...,ψ _M ), The dimension of E is 192×M, E=[e ₁ ,...,e _M ], diag() is the main diagonal matrix representation, ψ ₁ ,...,ψ _M corresponds to the selected ₁ , _. dimension and the matrix Y ^w after the whitening operation, Y ^w =W×Y. 2 . The method for objective evaluation of color image quality based on online manifold learning according to claim 1 , wherein λ ₁ =0.7 is taken in the step ④. 3 . 3 . The method for objectively evaluating the quality of color images based on online manifold learning according to claim 1 , characterized in that in step ⑤ , λ ₂ =0.6 is taken. 4 . 4 . The method for objectively evaluating the quality of color images based on online manifold learning according to claim 1 , wherein C=0.04 is taken in the step ⑨. 5 .