CN105049835A - Perceived stereoscopic image quality objective evaluation method - Google Patents

Abstract

The invention relates to a perceived stereoscopic image quality objective evaluation method. The method comprises the following steps of acquiring the perceived brightness maps of a reference image and a distorted image respectively; making the perceived brightness maps of the reference image and the distorted image respectively decomposed into five bandpass images by using a logarithmic cosine filter set; screening the distorted image pixel points whose differences with pixel points of the reference image cannot be perceived; using a contrast sensitivity function to create each channel threshold and judging the covered pixel points in the bandpass images of the reference image and the distorted image; respectively calculating the evaluating indexes of left and right viewpoint qualities and further using a mean value method to acquire a stereoscopic image quality evaluating index. By means of the method, accurate evaluation on the stereoscopic image quality can be performed.

Description

The method for objectively evaluating of perception stereo image quality

Technical field

The invention belongs to image processing field, relate to a kind of image quality evaluating method, especially relate to a kind of method for objectively evaluating of perception stereo image quality.

Background technology

In the last few years, image coding technique and stereo display technique develop rapidly, stereo-picture treatment technology received increasing concern, became the study hotspot of domestic and international association area gradually.Bring the visual experience of people relative to plane picture, stereo-picture is even better, brings the telepresenc that people experience more really and shake.But relative to plane picture, stereo-picture shared data space in collection, transmission and storing process but improves significantly, under the prerequisite reducing data space occupancy volume, how to ensure the quality of stereo-picture, will directly affect the development of stereo-picture treatment technology.

Stereo image quality evaluation method is divided into two kinds: subjective assessment and objective evaluation.In objective evaluation method for quality of stereo images, the final stay of two nights of image is the mankind, and its result reliability is higher.But because subjective method is wasted time and energy, and be vulnerable to the impact of the factors such as tested mood, fatigue strength and test environment, therefore be subject to serious restriction in actual applications.Objective evaluation method for quality of stereo images is by simulating human vision system (HumanVisualSystem, HVS) build Mathematical Modeling or provide the method for the mathematical formulae meeting human-eye visual characteristic, calculate the numerical value determined as objective evaluation result, in order to the quality and the third dimension that replace the mankind to evaluate stereo-picture.Compare subjective evaluation method, objective evaluation method for quality of stereo images is time saving and energy saving, practical, may be used for quality detecting system, also can embedded images treatment system.Therefore, set up a set of evaluation of stereo image quality accurately and effectively mechanism and there is far-reaching significance.

Summary of the invention

The present invention is intended to the defect solving conventional images assessment technique, using the screening of contrast sensitivity function to reject cannot by the pixel discovered, and whether each pixel that each band leads in image is covered to utilize threshold value to judge, realize the modeling of contrast shielding effect, the consistency between effective raising objective evaluation result and human subject experience.Stereoscopic image quality makes accurate evaluation.The present invention is achieved through the following technical solutions:

A method for objectively evaluating for perception stereo image quality, comprises the following steps:

The first step: carry out luminance non-linearity conversion respectively to reference picture and distorted image, becomes the law of logarithm linear relationship, obtains perceived brightness figure with image intrinsic brilliance L according to the perceived brightness S of human eye;

Second step: use logarithmic cosine bank of filters to be decomposed into five logical image b of band with reference to the perceived brightness figure of image _i(x, y), i is logarithmic cosine filter ID, i=1 ~ 5, and a low pass subband l ₀(x, y); Image is led to for each band, obtains local luminance average figure according to following formula $l_i(x,y)=\left\{\begin{array}{ccc}{l}_0(x,y)& \mathrm{if}& i=1\\ l_0(x,y)+\underset{j=1}{\overset{i-1}{\mathrm{\Σ}}}{b}_j(x,y)& \mathrm{if}& i>1\end{array}\right.,$ And calculate by local mean value figure the contrast figure that each band leads to image wherein (x, y) is the coordinate of image time domain pixel; According to the method described above to the perceived brightness figure process of distorted image;

3rd step: sift out in distorted image and cannot discover with reference picture the pixel difference:

(1) the logical other thresholding of picture contrast minimum discernable of each band is calculated in formula, A (f) is contrast sensitivity function, f is image space frequency, herein f=i=1,2,3,4,5;

(2) if the contrast figure c of the logical image of each band of reference picture _ieach of the pixel of (x, y) and distorted image is with the contrast figure of logical image the absolute difference of corresponding pixel points be less than the T of this pixel _i(x, y), then show that human eye can not discover the difference of the two; Distortion map image-tape is led to image these pixel assignment be that corresponding reference tape leads to image b _ithe pixel value of (x, y); Finally, the band of distorted image is kept to lead to image residual pixel value constant:

4th step: use contrast sensitivity construction of function each passage thresholding, judges that reference picture and distortion map image-tape to lead in image by the pixel covered:

(1) the logical image thresholding of band is calculated in formula, 2 ⁱfor the centre frequency of logarithmic cosine bank of filters, α is viewing angle, A (2 ⁱ/ α) be with 2 ⁱ/ α is the contrast sensitivity function of spatial frequency;

(2) if the contrast image element value of the logical image of each band of reference picture and distorted image is less than the logical image gate limit value t of its band _ipixel covered, be then 0 by the pixel value assignment accordingly with logical image, residual pixel value remains unchanged:

5th step: the evaluation index Q calculating left and right viewpoint quality respectively _l, Q _r, use mean value method to left and right viewpoint quality evaluation index Q _l, Q _rsue for peace, obtain stereo image quality evaluation index Q _p, method is as follows:

(1) for right viewpoint, the reference picture first integrated through the 4th step process is respectively with logical image, calculates the image o (x, y) of reconstruct: b _i' (x, y) is respectively with logical image through the reference picture of the 4th step process; Then each band of computing reference and distorted image leads to image error and e (x, y), $e(x,y)={\left(\underset{i}{\mathrm{\Σ}}{|{b_i}^{\′}(x,y)-{{\tilde{b}}_i}^{\′\′}(x,y)|}^{\mathrm{\β}}\right)}^{1/\mathrm{\β}},$ Wherein, for being respectively with logical image through the distorted image of the 4th step process, β be value 1 ~ 4 summation index, according to following formulae discovery signal to noise ratio, as the evaluation index Q of the left viewpoint quality of distorted image _l: $Q_l=10{\log }_{10}\left(\frac{\underset{x}{\mathrm{\Σ}}\underset{y}{\mathrm{\Σ}}{o}^2(x,y)}{\underset{x}{\mathrm{\Σ}}\underset{y}{\mathrm{\Σ}}{e}^2(x,y)}\right);$

(2) right view-point image quality evaluation index Q is in like manner calculated _r;

(3) finally use mean value method left and right view-point image quality evaluation index to be merged, calculate stereo image quality evaluation index evaluation index Q _phigher, distorted image quality is better.

Preferably, the method for objectively evaluating of described perception stereo image quality, adopts transfer function to be in second step cosine filter group image is decomposed, wherein r is pole spatial frequency coordinate, F _ir () is band pass filter, their center is respectively 2 ⁱ tester described in step passes through viewing distance L at the viewing angle α of viewing point H ₁with display device height L ₂calculate, namely

This method effectively improve traditional statistics method ignore human visual system, can not the defect of comprehensive token image quality, simulate the psychophysics characteristic of multiple human visual system, can evaluate the stereo-picture of different distortion accurately and effectively, the validity evaluated and reliability all obtain good compromise.

Accompanying drawing explanation

Fig. 1 standard stereo material " boy "

Fig. 2 standard stereo material " family "

Fig. 3 standard stereo material " flower "

Fig. 4 standard stereo material " girl "

Fig. 5 standard stereo material " river "

Fig. 6 standard stereo material " tree "

Fig. 7 stereoscopic display device

Fig. 8 this method stereo image quality objective evaluation model

Fig. 9 " boy " leads to image with reference to the gray-scale map of left viewpoint figure and each band, and (a) is the gray-scale map of the left viewpoint figure of " boy " reference picture, and Fig. 9 (b) ~ (f) is divided into reference picture to be respectively with logical image b _i(x, y) (i=1,2,3,4,5)

The left viewpoint band of " boy " distortion before and after Figure 10 optimizes leads to image, and (a), for before optimization, (b) is for optimizing

Figure 11 testee is at the viewing angle of viewing point H

The logical figure of band before and after Figure 12 shielding effect, (a), for before shielding effect, (b) is for after shielding effect

The original image of Figure 13 reconstruct.

Embodiment

Below in conjunction with technical scheme process in detail.

The reference experiment material that the design uses all takes from the steric information storehouse that broadband wireless communications and three-dimensional imaging research institute provide, choose uncompressed in image library, do not add original image totally 6 width of making an uproar, be respectively standard stereo material " boy ", " family ", " flower ", " girl ", " river ", " tree ", resolution is 1280*1024, as shown in figures 1 to 6.Utilize MATLAB that 6 width original images are done to compression in various degree and add to make an uproar, obtain 270 width distorted images altogether, subjective assessment is done to all distorted images, record average suggestion value (MeanOpinionScore, MOS), to verify the correlation of the subjective and objective mark of stereo-picture.The stereoscopic imaging apparatus that this method subjective experiment uses is Tianjin dimension display technologies Co., Ltd " 3DWINDOWS-19A0 ", as shown in Figure 7.In this method subjective experiment, tested comprise specialty tested and amateur tested, all there is normal parallax third dimension, totally 20 tested, be respectively school postgraduate and undergraduate, the male sex 11, women 9, be engaged in tested totally 16 people of steric information treatment research, be engaged in tested totally 4 people of other directions research.

Below in conjunction with Fig. 8, stereoscopic image quality evaluating method is described in detail.

The first step, according to weber-Fei Henieer law, i.e. luminance non-linearity characteristic, the perceived brightness S of human eye becomes logarithm linear relationship with image intrinsic brilliance L.Shown in (1)

S＝KlnL+K ₀＝K'lgL+K ₀(1)

Wherein, S is perceived brightness, and L is absolute brightness, and K is constant, relevant to the mean flow rate of entire image, when mean picture brightness darker or brighter time, less K value should be selected.Usually according to human eye normal brightness scope, K=1 is got.K'=Kln10, K ₀for constant.

For gray level image, the speed that perceived brightness increases tends towards stability along with the increase of brightness value, and namely human eye is all insensitive to very black or very bright region.Therefore, utilize the brightness value of formula (1) to each pixel of gray level image to carry out nonlinear transformation, obtain perceived brightness figure, thus simulation human eye is for the actual impression of pixel.

Second step, copies the structure of human eye visual perception passage, and this method uses logarithmic cosine bank of filters simulating human vision multichannel characteristic.The perceived brightness figure of the reference picture first step obtained decomposes five logical subbands of band and a low pass subband, obtains five logical images of band, and constructs the contrast figure that each band leads to image respectively.The frequency domain transfer function of this bank of filters is expressed as

$F_i\left(r\right)=\frac{1}{2}[1+\cos (\mathrm{\π}{\log }_2\left(r\right)-\mathrm{\πi})]---\left(2\right)$

Wherein, r is pole spatial frequency coordinate, F ₁(r) ~ F ₅r () is band pass filter, their center is respectively 2 ⁱ(i=1,2,3,4,5) week/image.Kernel frequency filter F ₀r () is a low pass filter, its form is F ₁r the distortion of (), is expressed as

$F_0\left(r\right)=\frac{1}{2}[1+\cos (\mathrm{\π}{\log }_2(r+2)-\mathrm{\π})]---\left(3\right)$

Therefore, on frequency domain, by wherein, ω ₁, ω ₂be respectively level, vertical spatial frequency coordinate, r, θ are respectively pole spatial frequency coordinate, and meet relation θ=arctan (ω ₁/ ω ₂), P (r, θ) is the Fourier transform of image, L ₀the low-frequency component that (r, θ) is image, B _ithe band that (r, θ) is image leads to composition, the radio-frequency component that H (r, θ) is image.Radio-frequency component due to image contains less information, and in great majority application, image does not exist change perceived, and therefore this method ignores radio-frequency component H (r, θ), in time domain, is expressed as by the reference picture of logarithmic cosine bank of filters

$p(x,y)=l_0(x,y)+\underset{i}{\mathrm{\Σ}}{b}_i(x,y)---\left(4\right)$

Wherein, (x, y) is the coordinate of reference picture time domain pixel, l ₀(x, y) is the low-pass pictures of reference picture, b _i(x, y) is that the band of reference picture leads to image, is expressed as

l ₀(x,y)＝IFFT[P(r,θ)F ₀(r,θ)](5)

b _i(x,y)＝IFFT[P(r,θ)F _i(r,θ)](6)

Each band leads to image as shown in Figure 9, and wherein Fig. 9 (a) is the gray-scale map of the left viewpoint figure of " boy " reference picture, and Fig. 9 (b) ~ (f) is respectively with logical image b for reference picture _i(x, y) (i=1,2,3,4,5).

For the logical image of each band of reference picture, calculate corresponding local luminance average figure

$l_i(x,y)=\left\{\begin{array}{ccc}{l}_0(x,y)& \mathrm{if}& i=1\\ l_0(x,y)+\underset{j=1}{\overset{i-1}{\mathrm{\Σ}}}{b}_j(x,y)& \mathrm{if}& i>1\end{array}\right.---\left(7\right)$

And the contrast figure of image is led to by each band of local luminance average figure computing reference image

$c_i(x,y)=\frac{b_i(x,y)}{l_i(x,y)}---\left(8\right)$

According to the method described above to the perceived brightness figure process of distorted image.

3rd step, utilizes the other thresholding of contrast minimum discernable to sift out in distorted image and cannot discover with reference picture the pixel difference.Concrete grammar is as follows: first, according to the other thresholding of the contrast minimum discernable of formulae discovery reference picture respectively with logical subband below

$T_i(x,y)=\frac{0.86(A\left(f\right)c_i\left(\mathrm{xy}\right)-1)+0.3}{A\left(f\right)}$

Wherein c _ithe contrast figure of the logical image of each band that (x, y) is reference picture, A (f) is contrast sensitivity function f is image space frequency, herein f=i=1,2,3,4,5.

The concrete optimization method of the logical image of each band of distorted image is: if reference picture contrast figure is c _ithe pixel of (x, y) and distorted image contrast figure the absolute difference of corresponding pixel points be less than the T of this pixel _i(x, y), then show that human eye can not discover the difference of the two.First, this kind of pixel S set is found out by this method _i, shown in (10); Then, distortion band is led to image these pixel assignment be that corresponding reference tape leads to image b _ithe pixel value of (x, y); Finally, distortion band is kept to lead to image residual pixel value constant because human eye can discover with reference to and these residual pixels of distorted image between difference, computing formula is such as formula shown in (11).Figure 10 is that the left viewpoint band of distortion of boy leads to image schematic diagram before and after optimizing.

$S_i=\{(x,y):|c_i(x,y)-{\tilde{c}}_i(x,y)|<T_i(x,y)\}---\left(10\right)$

${{\tilde{b}}_i}^{\&prime;}(x,y)=\left\{\begin{array}{cc}{b}_i(x,y)& \mathrm{if}(x,y)\&Element;S_i\\ {\tilde{b}}_i(x,y)& \mathrm{otherwise}\end{array}\right.---\left(11\right)$

4th step, uses the thresholding of each passage of contrast sensitivity construction of function, judges in the logical image of band by the pixel covered.

To be the appreciable excitation of a kind of script because of the existence of another perception excitation become shielding effect is not easy perceived phenomenon, so can construct the thresholding that each band leads to subband, usage threshold value judges whether each pixel in contrast figure is covered, simulate shielding effect, find out in the logical image of band by the pixel covered.Because contrast sensitivity function (CSF) tests by threshold value the function obtained, the physical significance of its inverse is exactly a threshold value, therefore the thresholding that this method uses CSF to construct each band leads to subband.

According to calculating, subjective experiment testee is α=9.55 ° at the viewing angle of viewing point H, as shown in figure 11.By the business 2 of the center of logarithmic cosine bank of filters and α ⁱ/ α spends the input of sensitivity function A (f) as a comparison, the inverse of A (f) is led to the threshold value t of subband as each band _i, namely

$t_i=\frac{1}{A(2^i/\mathrm{\&alpha;})}---\left(11\right)$

Then, the pixel value finding out reference picture and distorted image contrast figure is less than the logical subband threshold value t of its band _ipixel, these pixels are in the logical image of corresponding band by the pixel covered;

Finally, by logical for band image, these are 0 by the pixel assignment covered, and the value of residual pixel point remains unchanged.Figure 12 is that the left viewpoint band of reference picture of boy leads to image b ₅(x, y) optimizes the schematic diagram of front and back.

5th step, is integrated the original image of being simulated by visually-perceptible and is respectively with logical image, obtains the original image reconstructed.Because human eye is low to high and low frequency subband susceptibility, so ignore their effect.So, integrate the original image of being simulated by visually-perceptible and be respectively with logical image, obtain the reference picture o (x, y) reconstructed, as shown in Equation 12, wherein b _i' (x, y) is respectively with logical image through the reference picture of third and fourth step process.Figure 13 is the reference picture after boy reconstruct.

$o(x,y)=\underset{i}{\mathrm{\&Sigma;}}{b_i}^{\&prime;}(x,y)---\left(12\right)$

Then Minkowski formula is adopted to integrate error respectively with logical image of reference picture and distorted image and e (x, y), as shown in Equation 13, wherein, for being respectively with logical image, b through the distorted image of third and fourth step process _i' (x, y), for the original image after shielding effect process and distorted image, β be usual value in the summation index of 1 ~ 4, get β=2 herein.

$e(x,y)={\left(\underset{i}{\mathrm{\&Sigma;}}{|{b_i}^{\&prime;}(x,y)-{{\tilde{b}}_i}^{\&prime;\&prime;}(x,y)|}^{\mathrm{\&beta;}}\right)}^{1/\mathrm{\&beta;}}---\left(13\right)$

Then, left and right viewpoint signal to noise ratio (signal-to-noiseratio, SNR) is calculated respectively according to following formula, as the evaluation index Q of distorted image left and right viewpoint quality _lq _r

$Q_l=10{\log }_{10}\left(\frac{\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}{o}^2(x,y)}{\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}{e}^2(x,y)}\right)---\left(14\right)$

Last this method uses mean value method to left and right viewpoint quality evaluation index Q _lq _rsue for peace, obtain stereo image quality evaluation index Q _p

$Q_p=\frac{Q_l+Q_r}{2}---\left(15\right)$

Table 1 is that method for objectively evaluating compares with five kinds of correlations of subjective assessment average mark (MOS) with the stereo image quality evaluation score of other 11 kinds of method for objectively evaluating herein.From data in table, all there is stronger correlation with subjective evaluation result in the evaluation result of method for objectively evaluating stereoscopic image quality in this paper, can reflect stereo image quality preferably, meet the subjective feeling of human eye.

Evaluation method	PLCC	SRCC	KRCC	MAE	RMS
						SNR	0.6868	0.7272	0.5451	0.1767	0.2085
PSNR	0.6648	0.7345	0.5704	0.2071	0.2435
						VIF	0.7603	0.7929	0.6052	0.2056	0.2520
PVIF	0.7888	0.8699	0.6861	0.1994	0.2337
						VSNR	0.6793	0.6998	0.5132	0.2403	0.2837
UQI	0.5976	0.6124	0.4610	0.1752	0.2388
						SSIM	0.5246	0.5767	0.4163	0.2258	0.2877
MS-SSIM	0.7447	0.7797	0.5970	0.2365	0.2861
						IW-PSNR	0.8377	0.9168	0.7430	0.1707	0.1992
IW-SSIM	0.7956	0.8315	0.6558	0.2056	0.2416
						HVSNR	0.6206	0.6623	0.5760	0.2169	0.2494
Q p	0.9042	0.9387	0.7874	0.1461	0.1681

Table 112 kind of a method for objectively evaluating correlation compares

Claims (3)

1. an objective evaluation algorithm for perception stereo image quality, comprises the following steps:

The first step: carry out luminance non-linearity conversion respectively to reference picture and distorted image, becomes the law of logarithm linear relationship, obtains perceived brightness figure with image intrinsic brilliance L according to the perceived brightness S of human eye.

Second step: use logarithmic cosine bank of filters to be decomposed into five logical image b of band with reference to the perceived brightness figure of image _i(x, y), i is logarithmic cosine filter ID, i=1 ~ 5, and a low pass subband l ₀(x, y); Image is led to for each band, obtains local luminance average figure according to following formula $l_i(x,y)=\left\{\begin{array}{cc}{l}_0(x,y)& \mathrm{if\; i}=1\\ l_0(x,y)+\underset{j=1}{\overset{i-1}{\mathrm{\&Sigma;}}}{b}_j(x,y)& \mathrm{if\; i}>1\end{array}\right.,$ And calculate by local mean value figure the contrast figure that each band leads to image wherein (x, y) is the coordinate of image time domain pixel; According to the method described above to the perceived brightness figure process of distorted image;

3rd step: sift out in distorted image and cannot discover with reference picture the pixel difference:

(1) the logical other thresholding of picture contrast minimum discernable of each band is calculated in formula, A (f) is contrast sensitivity function, f is image space frequency, herein f=i=1,2,3,4,5;

(2) if the contrast figure c of the logical image of each band of reference picture _ieach of the pixel of (x, y) and distorted image is with the contrast figure of logical image the absolute difference of corresponding pixel points be less than the T of this pixel _i(x, y), then show that human eye can not discover the difference of the two; Distortion map image-tape is led to image these pixel assignment be that corresponding reference tape leads to image b _ithe pixel value of (x, y); Finally, the band of distorted image is kept to lead to image residual pixel value constant;

4th step: use contrast sensitivity construction of function each passage thresholding, judges that reference picture and distortion map image-tape to lead in image by the pixel covered:

(1) the logical image thresholding of band is calculated in formula, 2 ⁱfor the centre frequency of logarithmic cosine bank of filters, α is viewing angle, A (2 ⁱ/ α) be with 2 ⁱ/ α is the contrast sensitivity function of spatial frequency;

(2) if the contrast image element value of the logical image of each band of reference picture and distorted image is less than the logical image gate limit value t of its band _ipixel covered, be then 0 by the pixel value assignment accordingly with logical image, residual pixel value remains unchanged;

5th step: the evaluation index Q calculating left and right viewpoint quality respectively _l, Q _r, use mean value method to left and right viewpoint quality evaluation index Q _l, Q _rsue for peace, obtain stereo image quality evaluation index Q _p, method is as follows:

(1) for right viewpoint, the reference picture first integrated through the 4th step process is respectively with logical image, calculates the image o (x, y) of reconstruct: b ' _i(x, y) is respectively with logical image through the reference picture of the 4th step process; Then each band of computing reference and distorted image leads to image error and e (x, y), $e(x,y)={\left(\underset{i}{\mathrm{\&Sigma;}}{|b_i^{\&prime;}(x,y)-{\tilde{b}}_i^{\&prime;\&prime;}(x,y)|}^{\mathrm{\&beta;}}\right)}^{1/\mathrm{\&beta;}},$ Wherein, for being respectively with logical image through the distorted image of the 4th step process, β be value 1 ~ 4 summation index, according to following formulae discovery signal to noise ratio, as the evaluation index Q of the left viewpoint quality of distorted image _l: $Q_l=10{\log }_{10}\left(\frac{\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}{o}^2(x,y)}{\underset{x}{\mathrm{\&Sigma;}}\underset{y}{\mathrm{\&Sigma;}}{e}^2(x,y)}\right);$

(2) right view-point image quality evaluation index Q is in like manner calculated _r;

(3) finally use mean value method left and right view-point image quality evaluation index to be merged, calculate stereo image quality evaluation index evaluation index Q _phigher, distorted image quality is better.

2. the objective evaluation algorithm of perception stereo image quality according to claim 1, is characterized in that, adopts transfer function to be in second step cosine filter group image is decomposed, wherein r is pole spatial frequency coordinate, F _ir () is band pass filter, their center is respectively 2 ⁱweek/image.

3. the objective evaluation algorithm of perception stereo image quality according to claim 1, is characterized in that, the tester described in the 4th step passes through viewing distance L at the viewing angle α of viewing point H ₁with display device height L ₂calculate, namely