CN107590796B - Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual - Google Patents

Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual Download PDF

Info

Publication number
CN107590796B
CN107590796B CN201710597144.3A CN201710597144A CN107590796B CN 107590796 B CN107590796 B CN 107590796B CN 201710597144 A CN201710597144 A CN 201710597144A CN 107590796 B CN107590796 B CN 107590796B
Authority
CN
China
Prior art keywords
residual
image
quality
map
global
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201710597144.3A
Other languages
Chinese (zh)
Other versions
CN107590796A (en
Inventor
冯丹丹
侯春萍
岳广辉
马彤彤
刘月
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201710597144.3A priority Critical patent/CN107590796B/en
Publication of CN107590796A publication Critical patent/CN107590796A/en
Application granted granted Critical
Publication of CN107590796B publication Critical patent/CN107590796B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Landscapes

  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

The invention relates to a full-reference mixed distortion image quality evaluation method based on sparse decomposition residual errors, which comprises the following steps: selecting natural pictures, and selecting image blocks from the natural pictures as training image block training dictionaries D; obtaining a sparse decomposition residual energy map; evaluating the image distortion level based on the sparse decomposition residual energy diagram, wherein the method comprises the following steps: calculating local residual energy feature similarity; obtaining local residual quality fraction Q by weighting a variance map obtained by a reference imagerl(ii) a Solving global residuals Gres from local residual energy mapsr(ii) a Calculating the quality fraction of the global residual error characteristics; the evaluation quality score of the final residual error feature is divided by a local residual error quality score QrlAnd global residual quality score QrgThe two parts are synthesized to obtain the final evaluation quality score Qr。

Description

Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual
Technical Field
The invention belongs to the field of image processing, in particular relates to an objective evaluation system of a plane image, and relates to a full-reference mixed distortion image quality evaluation method.
Background
With the development of digital image processing technology, image quality evaluation technology has become a research hotspot in the field of image processing. Image quality evaluation methods can be divided into two categories, subjective evaluation and objective evaluation. The former evaluates the quality of an object by the subjective feeling of people, and the latter gives quantitative indexes by mathematical modeling to simulate the visual perception mechanism of human beings to measure the quality of images. Subjective evaluation has high reliability, but the cost is high, the time consumption is long, and the operation is difficult, so the objective evaluation method is more concerned by students. The objective evaluation can be classified into full-reference, half-reference, and no-reference objective evaluation methods according to the degree of dependence on the reference image.
For the full-reference image quality evaluation method, the evaluation performance depends on the correlation degree of the subjective quality score and the objective evaluation score. The simplest full-reference evaluation methods are Mean Square Error (MSE) and peak signal-to-noise ratio (PSNR), which are simple in calculation and clear in meaning, but cannot well reflect subjective feeling of people. Sparse representation has been widely studied in recent years because it can well reflect the visual characteristics of the human eye. But the existing sparse representation-based method is only effective for a single distortion type, and only sparse coefficients are utilized to evaluate the image distortion level. The processes of image acquisition, compression, transmission and the like often cause various distortions to the image, so that the evaluation of the mixed distortion type image has important practical significance.
Disclosure of Invention
The invention provides a full-reference image quality evaluation method suitable for evaluating mixed distorted images, aiming at the problem that an objective quality evaluation method based on sparse representation is effective only for a single distortion type and only considers sparse coefficients for quality evaluation. The method evaluates the distortion level of a mixed-distortion image by considering the effect of distortion on local and global residuals. Even the scheme is as follows:
a full-reference mixed distortion image quality evaluation method based on sparse decomposition residual errors comprises the following steps:
(1) selecting natural pictures, selecting image blocks from the natural pictures as training image blocks, and training the dictionary by adopting a K-SVD algorithm to obtain a training dictionary D;
(2) obtaining a sparse decomposition residual energy map: performing sparse decomposition on the reference image and the distorted image thereof by an orthogonal matching pursuit algorithm to respectively obtain sparse decomposition residual errors which are recorded as resrAnd resdAnd obtaining residual energy characteristic maps of the reference image and the distorted image respectively by utilizing the inner products of the reference image and the distorted image, and recording the residual energy characteristic maps as ERrAnd ERd；
(3) Evaluating the image distortion level based on the sparse decomposition residual energy diagram, wherein the method comprises the following steps:
1) calculating local residual energy feature similarity: ER is measured by computing the phase information similarity of the residual energy gradient map of a reference image and its distorted image blocksrAnd ERdThe obtained similarity result is recorded as PDres；
2) By calculating PDresAnd obtaining local residual quality fraction by weighting the variance map obtained by the reference image and recording the local residual quality fraction as Qrl；
3) Solving the global residual from the local residual energy map: firstly, the reference image variance map is used as weight to process the local residual energy map, and the residual energy map of the processed reference and distorted image is recorded as wresrAnd wresdCalculating the global residual value of the distorted image and the reference image as Gres by the local residual in a mean-reducing and square-summing modedAnd Gresr；
4) Calculating the quality fraction of the global residual features: dividing the global residual value of the distorted image by the global residual value of the reference image to obtain a global residual quality score Qrg；
5) The evaluation quality score of the final residual error feature is divided by a local residual error quality score QrlAnd global residual quality score QrgThe two parts are synthesized to obtain the final evaluation quality score Qr。
The method processes the sparse decomposition residual, and evaluates the image distortion level through the gradient phase similarity of the image local residual energy and the image global residual value processed by the reference image global residual value. Experiments show that the evaluation result on the MDID2013 and MLIVE data has better correlation and higher accuracy with the subjective evaluation result, so that the quality of the mixed distorted image can be well evaluated.
Drawings
FIG. 1 training dictionary
FIG. 2 residual versus sparse coefficient energy plot
Fig. 3 shows residual horizontal maps before different pictures with similar DMOS values, (a) DMOS 0.5032res 49.6451, (b) DMOS 0.5035res 72.4497, (c) DMOS 0.5054res 56.6210, and (d) DMOS 0.4923res 89.2474
Fig. 4 shows residual horizontal maps after different processed pictures with similar DMOS values, (a) DMOS 0.5032res 0.1215, (b) DMOS 0.5035res 0.1144, (c) DMOS 0.5054res 0.1202, and (d) DMOS 0.4923res 0.1075
Detailed Description
The invention is further elucidated with reference to the drawing.
The method firstly carries out dictionary training, and then carries out image quality evaluation based on sparse decomposition residual error, and the specific method is as follows:
in the first step, natural images for training are selected, and 10 pictures are selected for dictionary training. Before dictionary training and quality evaluation, in order to eliminate the influence of image content, the image is normalized. In the dictionary training part, 10000 image blocks of 8 × 8 size are randomly selected from the training image as training image blocks.
And secondly, training the dictionary. Combining each image block in the training image block into a training sample set Y ═ Y in the form of a column vector1,y2,...,yp]∈Rn×PWherein each image block yp∈Rn×1P is 1,2, P contains n pixels, where n is 64 and P is 10000. Dictionary training is performed with the sample set as input. The invention trains a dictionary D ═ D with size of 64 × 2561,d2,...,dm]∈Rn×mWhere m is 256. The maximum value L of the number of non-zero terms of the sparse representation coefficient for each block is set to 8. Under the constraint of L, seeking to reconstruct a sparse representation coefficient matrix with minimum error, c ═ c1,c2,...,cp]∈Rm×PThe objective function is as follows:
Figure BDA0001356284050000021
here | · | non-conducting phosphor2Represents a two-norm (| · |) non-conducting phosphor0Representing a zero norm. At this stage, DAnd c is unknown, and in order to solve the problem of high computational complexity, the method adopts a K-SVD algorithm to train the dictionary. The algorithm continuously updates the dictionary D and the sparse coefficient c in an iteration mode, and each iteration is divided into two stages: the first stage, under the condition that the dictionary is fixed, calculating a sparse coefficient by an orthogonal Matching pursuit algorithm OMP (orthogonal Matching pursuit); and in the second stage, updating the dictionary under the condition that the sparse coefficient is fixed. The dictionary D is obtained by this method, as shown in fig. 1.
And thirdly, obtaining a sparse decomposition residual energy map. In fig. 2, the original image, the sparse coefficient energy map and the residual energy map are sequentially arranged from left to right, and it can be seen that the dark and bright areas of the residual energy map are basically opposite to those of the sparse coefficient energy map, so that the distortion level of the distorted image can be reflected by processing the residual energy map by a certain method. In view of the above situation, the present invention analyzes the local residual and the global residual, so as to achieve the purpose of evaluating the image by using the decomposed residual.
Firstly, the invention carries out sparse decomposition on a reference image block and a distorted image block thereof by an OMP algorithm, and simultaneously obtains sparse decomposition residual errors of the reference image block and the distorted image block thereof
Figure BDA0001356284050000031
And
Figure BDA0001356284050000032
respectively as follows:
Figure BDA0001356284050000033
Figure BDA0001356284050000034
then, the residual energy characteristics of the reference and the distorted image blocks are obtained by utilizing the inner product
Figure BDA0001356284050000035
And
Figure BDA0001356284050000036
respectively as follows:
Figure BDA0001356284050000037
Figure BDA0001356284050000038
wherein the content of the first and second substances,
Figure BDA0001356284050000039
is the number of image blocks.
And fourthly, evaluating the image distortion level based on the sparse decomposition residual energy diagram. The invention processes the residual error from two aspects of local and global, and the processing process is as follows:
1) local residual energy feature similarity
The invention evaluates the distortion level, ER, of an image block by calculating the similarity of the residual energy characteristics of a reference and its distorted image blockrAnd ERdThe similarity between them is measured by the phase information of the residual energy gradient map, as follows:
Figure BDA00013562840500000310
wherein, i is 1, 1., M, j is 1, N. C3Is a constant. PD (photo diode)resI.e. the phase information similarity result of the obtained residual energy gradient map, the energy gradient GrAnd GdRespectively as follows:
Figure BDA00013562840500000311
Figure BDA00013562840500000312
2) quality score of local residual features
In order to process the situation that the phase is small or the phase is close to pi, the quality map is obtained after the phase similarity result of the residual energy gradient map obtained in the step 1) is processed as follows:
Figure BDA0001356284050000041
wherein the content of the first and second substances,
Figure BDA0001356284050000042
rho is the obtained quality graph, and then the variance is used as weight to solve the quality fraction Q of the local residual error characteristicsrlComprises the following steps:
Figure BDA0001356284050000043
3) solving global residuals from local residuals
In order to enable the decomposition residual to better evaluate the image quality, the invention also analyzes the influence of the image distortion level on the global residual besides considering the local residual.
Because the area with stronger contrast has more important influence on the cognitive quality score, before the global residual is solved, the method takes the variance value of the reference image block as the weight to respectively process the residual energy values of the reference image block and the distorted image block thereof, and comprises the following steps:
Figure BDA0001356284050000044
Figure BDA0001356284050000045
Figure BDA0001356284050000046
and
Figure BDA0001356284050000047
is the result after processing. Wherein the content of the first and second substances,
Figure BDA0001356284050000048
obtaining a global residual from the local residual by:
Figure BDA0001356284050000049
Figure BDA00013562840500000410
wherein, GresdAnd GresrFor the resulting global residual values of the distorted image and the reference image,
Figure BDA00013562840500000411
and
Figure BDA00013562840500000412
respectively the mean of the residual energies of the distorted image block and the reference image block,
Figure BDA00013562840500000413
is the number of image blocks in each image.
4) Quality score of global residual features
As can be seen from fig. 3, due to the influence of the image content, the residual values of different images with similar DMOS values have different levels, and the reference image residual value is used to process the distorted image residual value in the present invention, so as to achieve the purpose of eliminating the influence of the image content, which is specifically as follows:
Qrg＝|Gresd/Gresr-1| (15)
Qrgis the quality score of the global residual features. As shown in fig. 4, the residual levels of different images are very close to each other after the above-described processing, and the residual levels are well consistent with the DMOS values.
5) Quality integration
The final residual feature evaluation score is represented by QrlAnd QrgThe two parts are synthesized to obtain:
Figure BDA0001356284050000051
Qrnamely the final quality fraction, wherein the parameters a, b are epsilon to [0,1 ∈]。
And fifthly, analyzing the experimental result. The method evaluates the performance of the algorithm on two mixed distortion databases, namely the MDID2013 database and the MLIVE database. The number of reference images, the number of distorted images, and the distortion types contained in each database are shown in table 1. Unlike the MDID2013 database, the MLIVE database contains two sub-databases, each of which has the same reference image.
TABLE 1 database information
Figure BDA0001356284050000052
In order to measure the consistency between the algorithm evaluation result and the subjective evaluation result, the invention uses three evaluation indexes of Pearson Linear Correlation Coefficient (PLCC), Spearman rank order correlation coefficient (SRCC) and Root Mean Square Error (RMSE). PLCC reflects the accuracy of the prediction, SRCC reflects the monotonicity of the prediction, and RMSE correlates with the consistency of the prediction. The smaller the value of RMSE, the closer the values of PLCC and SRCC are to 1, representing better algorithm performance. The invention adopts a nonlinear regression function with 5 parameters when calculating PLCC and RMSE values:
Figure BDA0001356284050000053
wherein: beta is aiAnd i is 1,2,3,4 and 5 as a fitting parameter.
The resulting performance of the invention on the two mixed distortion databases MDID2013 and MLIVE is shown in table 2.
TABLE 2 sparse decomposition residual Performance
Figure BDA0001356284050000054
As can be seen from table 2, the evaluation result has better correlation and higher accuracy with the subjective evaluation result, and thus the quality of the mixed distortion image can be well evaluated by processing the local residual energy and the global residual energy.

Claims (1)

1. A full-reference mixed distortion image quality evaluation method based on sparse decomposition residual errors comprises the following steps:
(1) selecting natural pictures, selecting image blocks from the natural pictures as training image blocks, and training the dictionary by adopting a K-SVD algorithm to obtain a training dictionary D;
(2) obtaining a sparse decomposition residual energy map: performing sparse decomposition on the reference image and the distorted image thereof by an orthogonal matching pursuit algorithm to respectively obtain sparse decomposition residual errors which are recorded as resrAnd resdAnd obtaining residual energy characteristic maps of the reference image and the distorted image respectively by utilizing the inner products of the reference image and the distorted image, and recording the residual energy characteristic maps as ERrAnd ERd；
(3) Evaluating the image distortion level based on the sparse decomposition residual energy diagram, wherein the method comprises the following steps:
1) calculating local residual energy feature similarity: ER is measured by computing the phase information similarity of the residual energy gradient map of a reference image and its distorted image blocksrAnd ERdThe obtained similarity result is recorded as PDres：
Figure FDA0002697040970000011
Wherein, i is 1, 1., M, j is 1, 1., N; c3Is a constant; PD (photo diode)resI.e. the phase information similarity result of the obtained residual energy gradient map, the energy gradient GrAnd GdAre respectively as：
Figure FDA0002697040970000012
Figure FDA0002697040970000013
2) By calculating PDresAnd obtaining local residual quality fraction by weighting the variance map obtained by the reference image and recording the local residual quality fraction as QrlThe method comprises the following steps:
in order to process the condition that the phase is smaller or the phase is close to pi, the quality map is obtained after the phase similarity result of the residual energy gradient map is processed as follows:
Figure FDA0002697040970000014
wherein the content of the first and second substances,
Figure FDA0002697040970000015
rho is the obtained quality map, and then the variance V (i, j) is used as weight to solve the quality fraction Q of the local residual error characteristicsrlComprises the following steps:
Figure FDA0002697040970000016
3) solving the global residual from the local residual energy map: firstly, the reference image variance map is used as weight to process the local residual energy map, and the residual energy map of the processed reference and distorted image is recorded as wresrAnd wresdCalculating the global residual value of the distorted image and the reference image as Gres by the local residual in a mean-reducing and square-summing modedAnd Gresr；
4) Calculating the quality fraction of the global residual features: removing distorted image global by reference image global residual valueResidual value to obtain a global residual quality fraction Qrg；
5) The evaluation quality score of the final residual features is represented by a local residual quality score QrlAnd global residual quality score QrgThe two parts are synthesized and marked as Qr。
CN201710597144.3A 2017-07-20 2017-07-20 Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual Active CN107590796B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710597144.3A CN107590796B (en) 2017-07-20 2017-07-20 Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710597144.3A CN107590796B (en) 2017-07-20 2017-07-20 Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual

Publications (2)

Publication Number Publication Date
CN107590796A CN107590796A (en) 2018-01-16
CN107590796B true CN107590796B (en) 2021-01-26

Family

ID=61041662

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710597144.3A Active CN107590796B (en) 2017-07-20 2017-07-20 Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual

Country Status (1)

Country Link
CN (1) CN107590796B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108428226B (en) * 2018-02-02 2020-06-16 中国科学院国家空间科学中心 Distortion image quality evaluation method based on ICA sparse representation and SOM
CN111583656A (en) * 2020-04-03 2020-08-25 北京博研中能科技有限公司 Fill electric pile parking stall intelligence prevention and control system suitable for new forms of energy car

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036501A (en) * 2014-06-03 2014-09-10 宁波大学 Three-dimensional image quality objective evaluation method based on sparse representation
CN104134204A (en) * 2014-07-09 2014-11-05 中国矿业大学 Image definition evaluation method and image definition evaluation device based on sparse representation
CN105894522A (en) * 2016-04-28 2016-08-24 宁波大学 Multi-distortion stereo image quality objective evaluation method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104036501A (en) * 2014-06-03 2014-09-10 宁波大学 Three-dimensional image quality objective evaluation method based on sparse representation
CN104134204A (en) * 2014-07-09 2014-11-05 中国矿业大学 Image definition evaluation method and image definition evaluation device based on sparse representation
CN105894522A (en) * 2016-04-28 2016-08-24 宁波大学 Multi-distortion stereo image quality objective evaluation method

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
Full-reference quality assessment of stereoscopic images by learning sparse monocular and binocular features;Li, Kemeng 等;《Optoelectronic Imaging and Multimedia Technology III》;20141111;第9273卷;第1-10页 *
Perceptual Image Quality Assessment Combining Free-energy Principle and Sparse Representation;Yutao Liu 等;《2016 IEEE International Symposium on Circuits and Systems (ISCAS)》;20161231;第1586-1589页 *
基于显著性的图像质量评价方法研究;赵丹;《信息技术》;20131231(第5期);第192-193页 *
基于稀疏表示的图像快速卡通+纹理分解;曲兰鹏;《中国优秀硕士学位论文全文数据库 信息科技辑》;20120915(第9期);第I138-747页 *

Also Published As

Publication number Publication date
CN107590796A (en) 2018-01-16

Similar Documents

Publication Publication Date Title
Shao et al. Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties
CN108428227B (en) No-reference image quality evaluation method based on full convolution neural network
Chang et al. Perceptual image quality assessment by independent feature detector
Chang et al. Sparse feature fidelity for perceptual image quality assessment
CN112085102B (en) No-reference video quality evaluation method based on three-dimensional space-time characteristic decomposition
Hong et al. Image quality assessment based on matching pursuit
Deng et al. Blind noisy image quality assessment using sub-band kurtosis
CN109978854B (en) Screen content image quality evaluation method based on edge and structural features
CN109523513B (en) Stereoscopic image quality evaluation method based on sparse reconstruction color fusion image
CN106920232A (en) Gradient similarity graph image quality evaluation method and system based on conspicuousness detection
CN104202594B (en) A kind of method for evaluating video quality based on 3 D wavelet transformation
CN104134204A (en) Image definition evaluation method and image definition evaluation device based on sparse representation
CN105260998A (en) MCMC sampling and threshold low-rank approximation-based image de-noising method
CN108399620B (en) Image quality evaluation method based on low-rank sparse matrix decomposition
CN109788275A (en) Naturality, structure and binocular asymmetry are without reference stereo image quality evaluation method
CN102547368A (en) Objective evaluation method for quality of stereo images
Ma et al. Reduced-reference stereoscopic image quality assessment using natural scene statistics and structural degradation
Geng et al. A stereoscopic image quality assessment model based on independent component analysis and binocular fusion property
CN109754390B (en) No-reference image quality evaluation method based on mixed visual features
CN107590796B (en) Full-reference mixed distortion image quality evaluation method based on sparse decomposition residual
Okarma Extended hybrid image similarity–combined full-reference image quality metric linearly correlated with subjective scores
CN109257592B (en) Stereoscopic video quality objective evaluation method based on deep learning
Xu et al. Remote-sensing image usability assessment based on ResNet by combining edge and texture maps
CN107481221B (en) Full-reference mixed distortion image quality evaluation method based on texture and cartoon sparse representation
CN108010023A (en) High dynamic range images quality evaluating method based on tensor domain curvature analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant