CN104143188A

CN104143188A - Image quality evaluation method based on multi-scale edge expression

Info

Publication number: CN104143188A
Application number: CN201410320983.7A
Authority: CN
Inventors: 翟广涛; 闵雄阔; 杨小康; 李铎
Original assignee: Shanghai Jiaotong University
Current assignee: Shanghai Jiaotong University
Priority date: 2014-07-04
Filing date: 2014-07-04
Publication date: 2014-11-12

Abstract

The invention provides an image quality evaluation method based on multi-scale edge expression. According to the image quality evaluation method based on multi-scale edge expression, image quality evaluation is conducted by means of multi-scale edge expression in a wavelet domain according to the serious independence, existing when a natural scene is understood by a human vision system, on the edges and the outline. The image quality evaluation method based on multi-scale edge expression comprises the steps that firstly, the wavelet transform of an image is calculated on multiple scales, so that the module and the argument of a wavelet factor are obtained; secondly, the structural similarity between the module of the wavelet factor of an original image and the module of the wavelet factor of a distorted image is calculated, and namely image quality evaluation is conducted according to the multi-scale module similarity algorithm; or the structural similarity of the multi-scale edges is calculated according to the multi-scale edge expression obtained based on the local maximum value of the module of the wavelet factor, and namely image quality evaluation is conducted according to the multi-scale edge similarity algorithm. According to the image quality evaluation method based on multi-scale edge expression, the performance is high, and the performance of the M2S method and the performance of the M3S method are higher than the performance of the widely-applied methods based on the peak signal to noise ratio and the performance of the widely-applied method based on the simple structure similarity.

Description

Picture quality evaluation method based on multi-scale edge expression

Technical Field

The invention relates to the technical field of image processing, in particular to a picture quality evaluation method based on multi-scale edge expression.

Background

Image quality evaluation techniques have a very important role in many image processing applications, such as system performance evaluation, image enhancement, denoising, and encoding parameter optimization. The image quality evaluation method can be divided into three categories according to whether the original image can be obtained: whole ginseng, half ginseng and non-ginseng methods. The full-parameter method generally uses the whole original image, the half-parameter method selectively uses partial features of the original image, and the non-parameter method does not need to use the original image. Many researchers have been investing in the study of image quality evaluation schemes using computers in recent years, but image quality evaluation is nevertheless a very challenging task due to a lack of understanding of the human visual system.

Mean Square Error (MSE) and peak to noise ratio (PSNR), although sometimes not particularly consistent with the subjective scores of testers, are still the most common quality assessment criteria at present due to their simplicity. Researchers have recently proposed many full-reference image quality evaluation methods, but they are still not widely adopted like MSE and PSNR; due to the complexity of feature extraction, an effective semi-parameter image quality evaluation method is provided; the non-parameter image quality evaluation method is more difficult to develop.

Earlier work in many areas of nervous system science, such as the paper "receptor Fields and Functional Architecture" published on pages 106 to 154 of Journal of Neuroscience 160, volume D.Hubel et al and the paper "receptor Fields and Functional Architecture of Monkey structure Architecture" published on pages 215 to 243 of volume 195; and in the field of computer vision, for example, the paper "Scale-Space for discovery Signals" published by t.lindeberg in IEEE trans.pattern Analysis and Machine Intelligence (IEEE schema Analysis and Machine Intelligence collection) at page 234 to page 254 of volume 3, volume 12, and the "Local Scale control for edge detection and blank evaluation" published by j.h. elder et al in IEEE trans.pattern Analysis and Machine Intelligence collection, volume 7, page 699 to page 716 of volume 20, volume 7, all show that the multi-Scale decomposition technique better simulates the psychophysiological mechanisms of the human visual system than the single-Scale technique, so that the multi-Scale decomposition technique is expected to achieve more satisfactory results in natural images. In addition, some Research works, such as the "evaluation for boundary-specific grouping in human Vision" published by J.H. Elder et al in Vision Research, volume 38, pages 143 to 152, show that the human visual system relies heavily on edges and contours in perceiving surface properties and understanding natural scenes.

Disclosure of Invention

Aiming at the defects in the prior art, the invention aims to provide a picture quality evaluation method based on multi-scale edge expression, which has good performance and better performance than the widely used peak signal-to-noise ratio and simple structure similarity method.

In order to achieve the above object, the present invention provides a picture quality evaluation method based on multi-scale edge expression, which performs image quality evaluation in a wavelet domain by using the multi-scale edge expression according to the severe dependence on edges and contours when a human visual system understands a natural scene: firstly, calculating image wavelet transformation on multiple scales to obtain a mode and an argument of a wavelet factor, and then calculating the structural similarity of an original image and a distortion image wavelet factor mode, namely performing image quality evaluation by using a multiple-scale mode similarity algorithm; or obtaining the multi-scale edge expression of the image by the local maximum value of the wavelet factor model, and calculating the structural similarity of the multi-scale edge, namely performing the quality evaluation of the image by using a multi-scale edge similarity algorithm.

The method specifically comprises the following steps:

the first step, carrying out second-order wavelet transform on an image: firstly, a first-order partial derivative is solved for a second-order differentiable two-dimensional smooth function theta (x, y) meeting the condition to obtain a wavelet function psi¹(x,y),ψ²(x, y) wavelet function at scale 2^jIs expanded to the second order ofThe two-dimensional function f (x, y) is in the scale 2^jThe up-second order wavelet transform can be expressed in a vector form as

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

Wherein theta (x, y) is a two-dimensional smooth function which can be microminiaturized in the second order and satisfies a certain condition, a two-dimensional function f (x, y) is an image to be evaluated, and (x, y) is the coordinate of the image,respectively, image f (x, y) at scale 2^jThe wavelet factors of the horizontal direction and the vertical direction of the upper second-order wavelet transform are changed, j is 1, and 2 and 3 … are positive integers;

second, multi-scale edge extraction, for scale 2^jThe wavelet transformation vector of the upper image is obtained by modulusIn the direction ofGiving out that the multi-scale edge of the image is the local maximum of the second-order wavelet transform mode of the image, whereinI.e. wavelet transform vector

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

The die of (a) is used,the direction of the wavelet transformation vector of the image is obtained;

thirdly, calculating the structural similarity of the original image and the distortion image in the wavelet transform mode on multiple scales, and adopting a maximum mode similarity method (M)³S), namely, the image quality of the distortion map is evaluated by quantifying the similarity of the original image and the multi-scale edge of the distortion map on multiple scales and finally combining the similarity on the multiple scales; or using a maximum modulus similarity method (M)³S), namely, the similarity of the original image and the multi-scale edge of the distorted image on the multi-scale is quantified, and finally the similarity on the multi-scale is combined to evaluate the image quality of the distorted image.

Preferably, in the first step, the second differentiable two-dimensional smoothing function θ (x, y) should satisfy the following condition:

<math> <mrow> <mi>θ</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>&GreaterEqual;</mo> <mn>0</mn> <mo>,</mo> <munder> <mrow> <mo>&Integral;</mo> <mo>&Integral;</mo> </mrow> <msup> <mi>R</mi> <mn>2</mn> </msup> </munder> <mi>θ</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mi>dxdy</mi> <mo>=</mo> <mn>1</mn> </mrow> </math>

the first order partial derivatives are:

<math> <mrow> <msup> <mi>ψ</mi> <mn>1</mn> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <mo>&PartialD;</mo> <mi>θ</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <msup> <mi>ψ</mi> <mn>2</mn> </msup> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <mo>&PartialD;</mo> <mi>θ</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> </mrow> </math>

ψ¹(x,y),ψ²the (x, y) mean is 0, so it can be used as a wavelet function whose second-order expansion is expressed as:

the function f (x, y) is e.L²(R²) In the dimension 2^jThe second order wavelet transform of (j ═ 1,2,3 …, a positive integer) is represented as:

wherein,wavelet factors in the horizontal direction and the vertical direction respectively;

smoothing the image at multiple scales using a function θ (x, y) yields the following gradient vector representation:

<math> <mrow> <mfenced open='[' close=']'> <mtable> <mtr> <mtd> <msubsup> <mi>W</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> <mn>1</mn> </msubsup> <mi>f</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <msubsup> <mi>W</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> <mn>2</mn> </msubsup> <mi>f</mi> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mtd> </mtr> </mtable> </mfenced> <mo>=</mo> <msup> <mn>2</mn> <mi>j</mi> </msup> <mfenced open='[' close=']'> <mtable> <mtr> <mtd> <mfrac> <mo>&PartialD;</mo> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mrow> <mo>(</mo> <mi>f</mi> <mo>*</mo> <msub> <mi>θ</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mo>)</mo> </mrow> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mtd> </mtr> <mtr> <mtd> <mfrac> <mo>&PartialD;</mo> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mrow> <mo>(</mo> <mi>f</mi> <mo>*</mo> <msub> <mi>θ</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mo>)</mo> </mrow> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mtd> </mtr> </mtable> </mfenced> <msup> <mn>2</mn> <mi>j</mi> </msup> <mover> <mo>&dtri;</mo> <mo>&RightArrow;</mo> </mover> <mrow> <mo>(</mo> <mi>f</mi> <mo>*</mo> <msub> <mi>θ</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mo>)</mo> </mrow> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> </mrow> </math>

wherein,

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

i.e. the vector representation form of the image wavelet transformation, theta (x, y) is a smoothing function, f (x, y) is an image,is shown in dimension 2^jF (x, y) is smoothed by θ (x, y), which represents the convolution operation,andrespectively representing the derivation of the partial derivatives for x and y,andnamely to representThe values at (x, y) for the partial derivatives of x and y,i.e. the gradient description of the image wavelet transform.

Preferably, in the second step, in dimension 2^jUpper, lower imagef (x, y) wavelet transform vector

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

The mold (A) is as follows:

M_{2^{j}} f (x, y) = \sqrt{{| W_{2^{j}}^{1} f (x, y) |}^{2} + {| W_{2^{j}}^{2} f (x, y) |}^{2}}

wherein,i.e. the modulus of the wavelet transform vector,two components of the wavelet transform vector are respectively;

the direction of the wavelet transform of the image f (x, y) is given by:

A_{2^{j}} f (x, y) = \arctan | \frac{W_{2^{j}}^{1} f (x, y)}{W_{2^{j}}^{2} f (x, y)} |

wherein,i.e. the direction of the wavelet transform vector,two components of the wavelet transform vector are respectively;

the variable point isIn the direction ofAnd obtaining the local maximum point, wherein the multi-scale edge of the image is the local maximum of the second-order wavelet transform mode of the image.

Preferably, in the third step, the multi-scale modulo similarity method (M)²S), wherein the similarity of the patterns is measured by Structural Similarity (SSIM);

by usingThe modes of the wavelet transform representing the original image and the distortion map respectively, where N represents the number of layers of decomposition, are then in the scale 2^jThe Mean Structural Similarity (MSSIM) of the upper original graph and the distortion graph model can be expressed asCalculating the multi-scale mode similarity when the decomposition series is N:

<math> <mrow> <msup> <mi>M</mi> <mn>2</mn> </msup> <mi>S</mi> <mrow> <mo>(</mo> <mi>N</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>W</mi> <mi>i</mi> </msub> <mo>·</mo> <mi>MSSIM</mi> <mo>[</mo> <msub> <mi>Mo</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>,</mo> <mi>M</mi> <msub> <mi>d</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>,</mo> </mrow> </math>

wherein, W_iThe relative weights for different layers can be set asN is the number of layers to be decomposed,the modes of wavelet transformation, MSSIM, of the original image and of the distorted image]Is an average structural similarity algorithm, M²S (N) is the multi-scale mode similarity when the decomposition series is N.

Preferably, in a third step, the maximum modulus similarity method (M)³S), wherein the similarity of the multi-scale edges is measured by Structural Similarity (SSIM);

by using

M {Mo}_{2^{j}} (x, y), M {Md}_{2^{j}} (x, y), (j = 1,2, . . ., N)

The local maximum values of the mode of wavelet transformation of the original image and the distortion image are respectively shown, and the similarity of the multi-scale maximum mode when the calculation decomposition level is N is as follows:

<math> <mrow> <msup> <mi>M</mi> <mn>3</mn> </msup> <mi>S</mi> <mrow> <mo>(</mo> <mi>N</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>W</mi> <mi>i</mi> </msub> <mo>·</mo> <mi>MSSIM</mi> <mo>[</mo> <msub> <mi>MMo</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>,</mo> <mi>MM</mi> <msub> <mi>d</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>,</mo> </mrow> </math>

wherein, W_iThe relative weights for different layers can be set asN is the number of layers to be decomposed,local maxima, MSSIM, of the original and distorted wavelet transform modes, respectively]Is an average structural similarity algorithm, M³S (N) is the multi-scale maximum mode similarity when the decomposition series is N.

The principle of the invention is as follows:

according to the severe dependence of the human visual system on edges and contours when understanding a natural scene and the good simulation of the psychophysiological mechanism of the human visual system by the multi-scale decomposition technology, the structural similarity of the mode of wavelet transform or the local maximum value (multi-scale edge) of the mode of wavelet transform on the multi-scale of an original image and a distortion image is calculated, and finally the Structural Similarity (SSIM) of the mode of wavelet transform or the multi-scale edge on the multi-scale is combined for image quality evaluation.

Compared with the prior art, the invention has the following beneficial effects:

the image quality evaluation method adopts a multi-scale mode similarity algorithm and a multi-scale maximum mode similarity algorithm, and under most conditions, the multi-scale mode similarity algorithm and the multi-scale maximum mode similarity algorithm have better performance than a widely used peak signal-to-noise ratio algorithm and a simple structural similarity algorithm, so that the method has good performance.

Drawings

Other features, objects and advantages of the invention will become more apparent upon reading of the detailed description of non-limiting embodiments with reference to the following drawings:

FIG. 1 is a flow chart of a method according to an embodiment of the present invention;

FIGS. 2(a) and 2(b) show the weighted variance regression correlation and the spearman rank correlation of 19 model prediction values with mean opinion scores over a JPEG picture set, respectively;

FIGS. 2(c) and 2(d) show the weighted variance regression correlation and the spearman rank correlation of 19 model predictors with mean opinion score, respectively, over a JPEG2000 picture set;

FIGS. 2(e) and 2(f) show the weighted variance regression correlation and the spearman rank correlation of 19 model predictors with mean opinion scores over a JPEG and JPEG2000 combined picture set, respectively;

FIGS. 3(a) -3 (d) are the PSNR, SSIM, M of all pictures in the implementation of the present invention²S(3),M³And S (7) a scatter diagram of the predicted values and Mean Opinion Scores (MOS) of the four models.

Detailed Description

The present invention will be described in detail with reference to specific examples. The following examples will assist those skilled in the art in further understanding the invention, but are not intended to limit the invention in any way. It should be noted that variations and modifications can be made by persons skilled in the art without departing from the spirit of the invention. All falling within the scope of the present invention.

As shown in fig. 1, the present embodiment provides a picture quality evaluation method based on multi-scale edge expression, which performs image quality evaluation in a wavelet domain by using multi-scale edge expression according to the severe dependence on edges and contours when a human visual system understands a natural scene; firstly, calculating image wavelet transformation on multiple scales to obtain a mode and an argument of a wavelet factor, and then calculating the structural similarity of an original image and a distortion image wavelet factor mode, namely performing image quality evaluation by using a multiple-scale mode similarity algorithm; or obtaining the multi-scale edge expression of the image by the local maximum value of the wavelet factor model, and calculating the structural similarity of the multi-scale edge, namely performing the quality evaluation of the image by using a multi-scale edge similarity algorithm; wherein, M in FIG. 1²S represents a multi-scale mode similarity method, M³S represents a multi-scale maximum mode similarity method.

The method specifically comprises the following steps:

firstly, performing second-order wavelet transformation on an image, and firstly solving a first-order partial derivative of a second-order differentiable two-dimensional smooth function theta (x, y) meeting a certain condition to obtain a wavelet function psi¹(x,y),ψ²(x, y) wavelet function at scale 2^jIs expanded to the second order ofThe two-dimensional function f (x, y) is in the scale 2^jThe up-second order wavelet transform can be expressed in a vector form as

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}];

The two-dimensional smoothing function θ (x, y) that can be microminiaturized in the second order should satisfy the following condition:

the first order partial derivatives are:

let the function f (x, y) be L²(R²) A two-dimensional image, it is in the dimension 2^jThe second order wavelet transform of (a) is expressed as:

wherein,

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

Second step, imageMultiscale edge extraction, for scale 2^jThe wavelet transformation vector of the image is obtained by modulus extractionIn the direction ofGiving out that the multi-scale edge of the image is the local maximum value of the second-order wavelet transform mode of the image;

in the dimension 2^jWavelet transform vector of the upper image f (x, y)

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

The mold (A) is as follows:

M_{2^{j}} f (x, y) = \sqrt{{| W_{2^{j}}^{1} f (x, y) |}^{2} + {| W_{2^{j}}^{2} f (x, y) |}^{2}}

the direction is given by:

A_{2^{j}} f (x, y) = \arctan | \frac{W_{2^{j}}^{1} f (x, y)}{W_{2^{j}}^{2} f (x, y)} |

Thirdly, calculating the structural similarity of the wavelet transform model of the original image and the distortion image on multiple scales, combining the structural similarity on the multiple scales, and evaluating the image quality of the distortion image;

multi-scale mode similarity algorithm (M) for calculating similarity of original image and distortion image wavelet transform mode²S), namely, the similarity of an original image and a distortion image in a multi-scale wavelet transform mode is quantized, and finally the similarity in the multi-scale is combined to evaluate the image quality of the distortion image, wherein the similarity of the mode is measured by Structural Similarity (SSIM); the SSIM may be an index for measuring the similarity between two images, which is published by Z.Wang et al, IEEE Transactions on Image Processing, volume 13, page 4, page 600 to page 612, and which is proposed in "Image quality assessment from information to structural precision";

wherein W_iThe relative weights of the different layers can be set to

Or calculating the similarity of the original image and the multi-scale edge of the distorted image-maximum mode similarity algorithm (M)³S), namely, the image quality of the distortion map is evaluated by quantifying the similarity of multi-scale edges of the original image and the distortion map on multiple scales and finally combining the similarity on the multiple scales, wherein the similarity of the multi-scale edges is measured by Structural Similarity (SSIM);

by using

M {Mo}_{2^{j}} (x, y), M {Md}_{2^{j}} (x, y), (j = 1,2, . . ., N)

The local maximum values of the mode of wavelet transformation of the original image and the distortion image are respectively shown, and the multi-scale mode similarity when the calculation decomposition level is N is as follows:

<math> <mrow> <msup> <mi>M</mi> <mn>3</mn> </msup> <mi>S</mi> <mrow> <mo>(</mo> <mi>N</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>W</mi> <mi>i</mi> </msub> <mo>·</mo> <mi>MSSIM</mi> <mo>[</mo> <msub> <mi>MMo</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>,</mo> <mi>MM</mi> <msub> <mi>d</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>]</mo> <mo>.</mo> </mrow> </math>

according to the steps, the performance of the proposed method is compared with that of MSE, PSNR and SSIM, and compared models in experiments are listed as follows:

Model1:PSNR

Model2:MSE

Model3:SSIM

Model4:M²S(1)

Model5:M²S(2)

...

Model11:M²S(8)

Model12:M³S(1)

Model13:M³S(2)

...

Model19:M³S(8)

the experiment was performed on an image quality evaluation library LIVE provided by HR Sheikh et al, which contains 233 JPEG compressed pictures and 227 JPEG2000 compressed pictures at different bit rates. Quantitative evaluation is carried out by using a method provided by a Video Quality Expert Group (VQEG) in Video Quality Experts Group (VQEG) Phase I Full Reference-TV test, namely, the correlation between objective scores after weighted variance regression and subjective scores, the Spanish-scale correlation of the objective scores and the subjective scores and the abnormal value proportion of predicted values after nonlinear mapping.

As shown in fig. 2, the present invention has the effect that the weighted variance regression correlation and the spearman rank correlation of 19 model predictors and mean opinion score are performed on different sets of pictures. FIGS. 2(a) and 2(b) show the weighted variance regression correlation and the spearman rank correlation of 19 Model predictors with mean opinion score on a JPEG photo set, respectively, wherein the horizontal axes p1-p19 show Model1-Model19 models, respectively, and the vertical axis shows correlation values; FIGS. 2(c) and 2(d) are graphs showing the weighted variance regression correlation and the spearman rank correlation of 19 Model predictors with mean opinion score on a JPEG2000 picture set, respectively, wherein the horizontal axes p1-p19 represent Model1-Model19 models, respectively, and the vertical axis represents correlation values; fig. 2(e) and 2(f) show the weighted variance regression correlation and spearman rank correlation of 19 Model predictors and mean opinion scores in JPEG and JPEG2000 combined sets, respectively, where the horizontal axes p1-p19 show Model1-Model19, respectively, and the vertical axis shows the correlation values. The error bars in fig. 2(a) -2(f) represent 95% confidence intervals for each predictor. From FIGS. 2(a) -2(f), it can be seen that models4-9 and models13-19 have the same excellent performance; models1-2 performed the worst; other models have centered performance.

As shown in table 1, the performance of 19 quality evaluation models evaluated on the LIVE library JPEG picture set, JPEG2000 picture set, and all pictures by using 3 indexes is shown, where Variance weighted regression represents a weighted Variance regression Correlation value, Spearman Correlation represents a Spearman rank Correlation value, and Outlier Ratio represents an abnormal value Ratio of a predicted value after nonlinear mapping. From this table, it can be seen that models6 and models18, M²S (3) and M³S (7) has optimal performance.

Table 1 quality evaluation performance parameter values of different quality evaluation models on different picture sets

As shown in FIGS. 3(a) to 3(d), PSNR, SSIM, M²S(3),M³And S (7) a scatter diagram of the predicted values and Mean Opinion Score (MOS) of the four models, wherein + represents data points of the MOS values relative to the predicted values of the models, in the three curves, the middle solid line is a logic function fitting curve containing four parameters, and the dotted lines on the two sides of the solid line are 95% confidence intervals of the fitting curve.

It was also found in the experiments that M is the number of pictures in the evaluation of JPEG and JPEG2000 pictures²S generally has good performance on a lower scale; and M³S generally has good performance on a higher scale. If M is used²S, evaluating the quality of the image, namely only performing image processing on the imageDecomposing a low level; and for M³S, only the features contained in the multi-scale edges are needed, but obtaining the same accurate predicted value usually requires calculating wavelet decomposition of higher levels.

The foregoing description of specific embodiments of the present invention has been presented. It is to be understood that the present invention is not limited to the specific embodiments described above, and that various changes and modifications may be made by one skilled in the art within the scope of the appended claims without departing from the spirit of the invention.

Claims

1. A picture quality evaluation method based on multi-scale edge expression is characterized by comprising the following steps:

the first step, carrying out second-order wavelet transform on an image: firstly, a first-order partial derivative is solved for a second-order differentiable two-dimensional smooth function theta (x, y) meeting the condition to obtain a wavelet function psi¹(x,y),ψ²(x, y) wavelet function at scale 2^jIs expanded to the second order ofThe two-dimensional function f (x, y) is thenDimension 2^jThe up-second order wavelet transform can be expressed in a vector form as

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}],

step two, multi-scale edge extraction: for dimension 2^jThe wavelet transformation vector of the upper image is obtained by modulusIn the direction ofGiving out that the multi-scale edge of the image is the second-order wavelet transform mode of the imageA local maximum of, whereinI.e. wavelet transform vector of image

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}],

thirdly, calculating the structural similarity of the original image and the distortion image on the multi-scale wavelet transform mode, and adopting a multi-scale mode similarity method M²S, evaluating the image quality of the distortion map by quantifying the similarity of the original image and the wavelet transform model of the distortion map on multiple scales and finally combining the similarity on the multiple scales; or using maximum modulus similarity method M³And S, evaluating the image quality of the distortion map by quantifying the similarity of the original image and the multi-scale edge of the distortion map on multiple scales and finally combining the similarity on the multiple scales.

2. The method according to claim 1, wherein in the first step, the second order differentiable two-dimensional smoothing function θ (x, y) should satisfy the following condition:

the first order partial derivatives are:

the function f (x, y) is e.L²(R²) In the dimension 2^jThe second order wavelet transform of (a) is expressed as:

smoothing the image at multiple scales with a function θ (x, y) yields the following gradient vector description:

wherein,

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

3. The picture quality evaluation method based on multi-scale edge expression according to claim 1, wherein in the second step, in the scale 2^jWavelet transform vector of the upper image f (x, y)

[\begin{matrix} W_{2^{j}}^{1} f (x, y) \\ W_{2^{j}}^{2} f (x, y) \end{matrix}]

The mold (A) is as follows:

M_{2^{j}} f (x, y) = \sqrt{{| W_{2^{j}}^{1} f (x, y) |}^{2} + {| W_{2^{j}}^{2} f (x, y) |}^{2}}

wherein,i.e. the modulus of the wavelet transform vector of the image f (x, y),the components in the horizontal direction and the vertical direction of the wavelet transformation vector are respectively;

the direction of the wavelet transform of the image f (x, y) is given by:

A_{2^{j}} f (x, y) = \arctan | \frac{W_{2^{j}}^{1} f (x, y)}{W_{2^{j}}^{2} f (x, y)} |

wherein,i.e. the direction of the wavelet transform vector,the components in the horizontal direction and the vertical direction of the wavelet transformation vector are respectively;

4. The picture quality evaluation method based on multi-scale edge expression according to any one of claims 1 to 3, wherein in the third step, the multi-scale mode similarity method M²S, wherein the similarity of the modules is measured through the structural similarity;

by usingThe modes of the wavelet transform representing the original image and the distortion map respectively, where N represents the number of layers of decomposition, are then in the scale 2^jThe average structural similarity of the original graph and the distortion graph model is expressed asCalculating the multi-scale mode similarity when the decomposition series is N:

wherein, W_iRelative weights for different layers are set asN is the number of layers to be decomposed,the modes of wavelet transformation, MSSIM, of the original image and of the distorted image]In order to be an average structural similarity algorithm,M²s (N) is the multi-scale mode similarity when the decomposition series is N.

5. The picture quality evaluation method based on multi-scale edge expression according to any one of claims 1 to 3, wherein in the third step, the maximum mode similarity method M³S, wherein the similarity of the multi-scale edges is measured through structural similarity;

by using

M {Mo}_{2^{j}} (x, y), M {Md}_{2^{j}} (x, y), (j = 1,2, . . ., N)

<math> <mrow> <msup> <mi>M</mi> <mn>3</mn> </msup> <mi>S</mi> <mrow> <mo>(</mo> <mi>N</mi> <mo>)</mo> </mrow> <mo>=</mo> <munderover> <mi>Σ</mi> <mrow> <mi>i</mi> <mo>=</mo> <mn>1</mn> </mrow> <mi>N</mi> </munderover> <msub> <mi>W</mi> <mi>i</mi> </msub> <mo>·</mo> <mi>MSSIM</mi> <mo>[</mo> <mi>M</mi> <msub> <mi>Mo</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>,</mo> <msub> <mi>MMd</mi> <msup> <mn>2</mn> <mi>j</mi> </msup> </msub> <mrow> <mo>(</mo> <mi>x</mi> <mo>,</mo> <mi>y</mi> <mo>)</mo> </mrow> <mo>]</mo> </mrow> </math>

wherein, W_iRelative weights for different layers are set asN is the number of layers to be decomposed,local maxima, MSSIM, of the original and distorted wavelet transform modes, respectively]Is an average structural similarity algorithm, M³S (N) is the multi-scale maximum mode similarity when the decomposition series is N.