CN104954778A - Objective stereo image quality assessment method based on perception feature set - Google Patents

Abstract

The invention discloses an objective stereo image quality assessment method based on a perception feature set. According to the method, the distortion degree of a stereo image is measured according to the distortion degree of a visual perception feature map, a saliency map, a gradient map and an airspace just noticeable distortion map which are related to viewpoint perception quality are extracted, a disparity map related to third dimension and quality is extracted, distortion degrees of four perception feature maps are taken as features of the stereo image to create the perception feature set, a complicated human vision system is simulated according to a random forest machine learning algorithm, and feature parameters are fused. The objective stereo image quality assessment method has the advantages that change conditions of visual quality under the condition that the stereo image is influenced by various image processing and compressing methods can be reflected objectively, assessment performance of the method is not influenced by the content of the stereo image and the type of distortion, and conformity with subjective perception of human eyes is achieved.

Description

A kind of objective evaluation method for quality of stereo images based on perception feature set

Technical field

The present invention relates to a kind of image quality evaluating method, especially relate to a kind of objective evaluation method for quality of stereo images based on perception feature set.

Background technology

Stereo image quality evaluates the important component part that (Stereo Image Quality Assessment, SIQA) is three-dimensional video-frequency technology.Stereoscopic image/video is in collection, storage, process and inevitably can introduce noise or distortion in transmitting, and causes the decline of stereoscopic image/video quality, and therefore, the evaluation of stereoscopic image/video quality is the major issue of a needs research and solution.Stereo image quality evaluation method is generally divided into subjectivity and objectivity two class evaluation method.There is time-consuming, effort, shortcoming that cost is high because of it in subjective evaluation method, not easily realizes and apply; And method for objectively evaluating is integrated algorithm, model, can obtain the objective quality of stereo-picture quickly and easily, without the need to manual intervention, but its evaluation method is not yet ripe, and await further investigation, therefore method for objectively evaluating has become the emphasis of research.

In recent years, the research of stereo image quality objective evaluation achieves a series of achievement, two classes can be divided into generally: the first kind: the quality evaluation evaluation method of plane picture being directly applied to stereo-picture, if the people such as You are by the plane picture quality evaluating method of classics, as PSNR, MS-SSIM, VIF etc. are directly used in the evaluation of left visual point image and right visual point image, correspondence obtains the mass value of left visual point image and the mass value of right visual point image, get the quality of mean value as stereo-picture of two mass values again, but the quality evaluation of stereo-picture and plane picture makes a big difference, depth perception distortion level is also the key factor of the perceived quality affecting stereo-picture.Equations of The Second Kind: add the evaluation model that parallax information improves stereo-picture on the basis of plane picture quality evaluating method, as the people such as Yang add the antipode figure of stereo-picture to carry out stereo image quality evaluation; And for example the people such as Benoit is by depth information and plane picture quality evaluating method combining assessment stereo image quality; The people such as Hachicha propose the stereo image quality evaluation method based on the proper discernable distortion of binocular, and weigh relief change with binocular fusion competition, but, the third dimension of stereo-picture is the result of binocular fusion, binocular competition, binocular suppression, its mechanism is very complicated, therefore relief measurement is very difficult, and this also causes the consistency of these methods and subjective perception lower.These stereo image quality evaluation methods are all obtain global quality by local quality above, or the objective quality of stereo-picture is obtained with linear or nonlinear Feature fusion, but human visual system is extremely complicated system, these evaluation methods all cannot the human visual system of simulate complexity, causes evaluating accuracy lower.

Summary of the invention

Technical problem to be solved by this invention is to provide a kind of objective evaluation method for quality of stereo images based on perception feature set, and it can improve the correlation between objective evaluation result and subjective perception effectively.

The present invention solves the problems of the technologies described above adopted technical scheme: a kind of objective evaluation method for quality of stereo images based on perception feature set, it is characterized in that comprising the following steps:

1. I is made _orgrepresent original undistorted stereo-picture, make I _disrepresent the stereo-picture of distortion to be evaluated, by I _orgleft visual point image be designated as L _org, by I _orgright visual point image be designated as R _org, by I _disleft visual point image be designated as L _dis, by I _disright visual point image be designated as R _dis;

2. adopt frequency modulation conspicuousness detection algorithm, obtain L _org, R _org, L _disand R _disrespective remarkable figure, correspondence is designated as with then calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{sal}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^L(i,j)-S_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{sal}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^R(i,j)-S_{\mathrm{dis}}^R(i,j))}^2;$

Wherein, M represents I _organd I _diswidth, N represents I _organd I _disheight, 1≤i≤M, 1≤j≤N, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

3. adopt horizontal Sobel operator, obtain L _org, R _org, L _disand R _disrespective horizontal gradient figure, correspondence is designated as with and adopt vertical Sobel operator, obtain L _org, R _org, L _disand R _disrespective vertical gradient map, correspondence is designated as with then basis with obtain L _orggradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{org}}^L(i,j)=\sqrt{G_h^{L,\mathrm{org}}{(i,j)}^2+G_v^{L,\mathrm{org}}{(i,j)}^2};$ And according to with obtain R _orggradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{org}}^R(i,j)=\sqrt{G_h^{R,\mathrm{org}}{(i,j)}^2+G_v^{R,\mathrm{org}}{(i,j)}^2};$ According to with obtain L _disgradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{dis}}^L(i,j)=\sqrt{G_h^{L,\mathrm{dis}}{(i,j)}^2+G_v^{L,\mathrm{dis}}{(i,j)}^2};$ According to with obtain R _disgradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{dis}}^R(i,j)=\sqrt{G_h^{R,\mathrm{dis}}{(i,j)}^2+G_v^{R,\mathrm{dis}}{(i,j)}^2};$ Calculate again with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{gra}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^L(i,j)-S_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{gra}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(G_{\mathrm{org}}^R(i,j)-G_{\mathrm{dis}}^R(i,j))}^2;$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

4. adopt the proper discernable distortion model in spatial domain, obtain L _org, R _org, L _disand R _disthe proper discernable distortion map in respective spatial domain, correspondence is designated as with then calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{JND}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(J_{\mathrm{org}}^L(i,j)-J_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{JND}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(J_{\mathrm{org}}^R(i,j)-J_{\mathrm{dis}}^R(i,j))}^2;$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

5. adopt light stream matching method, obtain I _orghorizontal parallax amplitude figure and I _orgvertical parallax amplitude figure, correspondence is designated as with then basis with obtain I _orgdisparity map, be designated as D _org, by D _orgmiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as D _org(i, j), equally, adopt light stream matching method, obtain I _dishorizontal parallax amplitude figure and I _disvertical parallax amplitude figure, correspondence is designated as with then basis with obtain I _disdisparity map, be designated as D _dis, by D _dismiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as D _dis(i, j), calculate D afterwards _organd D _dismean square error, be designated as MSE _dsp, ${\mathrm{MSE}}_{\mathrm{dsp}}=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(D_{\mathrm{org}}(i,j)-D_{\mathrm{dis}}(i,j))}^2;$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

6. will with the sets definition that arranged in sequence is formed is I _disperception feature set, be designated as P, $P=\{{\mathrm{MSE}}_{\mathrm{sal}}^L,{\mathrm{MSE}}_{\mathrm{sal}}^R,{\mathrm{MSE}}_{\mathrm{gra}}^L,{\mathrm{MSE}}_{\mathrm{gra}}^R,{\mathrm{MSE}}_{\mathrm{JND}}^L,{\mathrm{MSE}}_{\mathrm{JND}}^R,{\mathrm{MSE}}_{\mathrm{dsp}}\};$

7. adopt n original undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion level of different type of distortion, using this distortion stereo-picture set as training set, training set comprises several distortion stereo-pictures; The mean subjective suggestion of the every width distortion stereo-picture then utilizing subjective quality assessment method evaluation to go out in training set is divided, and the mean subjective suggestion of the jth width distortion stereo-picture in training set is divided and is designated as MOS _j; Again according to step 1. to step process 6., obtain the perception feature set of the every width distortion stereo-picture in training set in an identical manner, the perception feature set of the jth width distortion stereo-picture in training set be designated as P _j;

Wherein, n>=1,1≤j≤S, S represents total width number of the distortion stereo-picture comprised in training set, MOS _j∈ [0,5];

8. random forest machine learning algorithm is adopted, the perception feature set of all distortion stereo-pictures in training set is trained, make through training the regression function value that obtains to divide with corresponding mean subjective suggestion between error minimum, structure obtains random forest training pattern;

9. according to constructing the random forest training pattern obtained, to I _disperception feature set P test, prediction obtain I _disevaluating objective quality predicted value, be designated as Q _dis, Q _dis=MOD (P), wherein, the function representation form that MOD () is random forest training pattern.

Compared with prior art, the invention has the advantages that:

1) the inventive method weighs the distortion level of stereo-picture with the distortion level of visually-perceptible characteristic pattern, extract the remarkable figure relevant to viewpoint perceived quality, the proper discernable distortion map of gradient map and spatial domain, extract the disparity map relevant to three-dimensional perceived quality, using the structural feature perception feature set of the distortion level of four kinds of Perception Features figure as stereo-picture, use the human visual system of random forest machine learning algorithm Simulation of Complex again, carry out the fusion of characteristic parameter, the inventive method can reflect that stereo-picture is subject to the situation of change of various image procossing and the lower visual quality of compression method impact objectively, and the assess performance of the inventive method is not subject to the impact of stereoscopic image content and type of distortion, consistent with the subjective perception of human eye.

2) the inventive method is trained by the perception feature set of random forest machine learning algorithm to distortion stereo-picture, structure obtains random forest training pattern, again according to constructing the random forest training pattern obtained, the perception feature set of distortion stereo-picture to be evaluated is tested, prediction obtains the evaluating objective quality predicted value of distortion stereo-picture to be evaluated, this characteristic parameter that makes is with the evaluating objective quality predicted value of the amalgamation mode predicted distortion stereo-picture of the best, avoid the complicated simulation process of correlation properties to human visual system and mechanism, and because the perception feature set of training and the perception feature set of test are separate, therefore test result depending on unduly training data can be avoided, thus the correlation that can effectively improve between objective evaluation result and subjective perception.

Accompanying drawing explanation

Fig. 1 be the inventive method totally realize block diagram;

Fig. 2 a is the left visual point image of original undistorted horse stereo-picture;

The left visual point image of distortion that Fig. 2 b obtains after JPEG compression for the image shown in Fig. 2 a;

Fig. 2 c is the remarkable figure of the image shown in Fig. 2 a;

Fig. 2 d is the remarkable figure of the image shown in Fig. 2 b;

Fig. 2 e is the gradient map of the image shown in Fig. 2 a;

Fig. 2 f is the gradient map of the image shown in Fig. 2 b;

The proper discernable distortion map in spatial domain that Fig. 2 g is the image shown in Fig. 2 a;

The proper discernable distortion map in spatial domain that Fig. 2 h is the image shown in Fig. 2 b;

Fig. 2 i is the disparity map of stereo-picture corresponding to the image shown in Fig. 2 a;

Fig. 2 j is the disparity map of stereo-picture corresponding to the image shown in Fig. 2 b;

Fig. 3 a is the left visual point image of Akko (being of a size of 640 × 480) stereo-picture;

Fig. 3 b is the left visual point image of Altmoabit (being of a size of 1024 × 768) stereo-picture;

Fig. 3 c is the left visual point image of Balloons (being of a size of 1024 × 768) stereo-picture;

Fig. 3 d is the left visual point image of Doorflower (being of a size of 1024 × 768) stereo-picture;

Fig. 3 e is the left visual point image of Kendo (being of a size of 1024 × 768) stereo-picture;

Fig. 3 f is the left visual point image of LeaveLaptop (being of a size of 1024 × 768) stereo-picture;

Fig. 3 g is the left visual point image of Lovebierd1 (being of a size of 1024 × 768) stereo-picture;

Fig. 3 h is the left visual point image of Newspaper (being of a size of 1024 × 768) stereo-picture;

Fig. 3 i is the left visual point image of Puppy (being of a size of 720 × 480) stereo-picture;

Fig. 3 j is the left visual point image of Soccer2 (being of a size of 720 × 480) stereo-picture;

Fig. 3 k is the left visual point image of Horse (being of a size of 480 × 270) stereo-picture;

Fig. 3 l is the left visual point image of Xmas (being of a size of 640 × 480) stereo-picture.

Embodiment

Below in conjunction with accompanying drawing embodiment, the present invention is described in further detail.

The sensitivity of distortion in human eye stereoscopic image is different, therefore the consistency of the evaluation method of global quality and subjective perception is obtained by local quality lower, and Perception Features figure is the shallow-layer reflection of stereo-picture in human nervous system, that stereo-picture is experienced the most intuitively in human eye, when the distortion of stereo-picture causes Perception Features figure distortion, human eye also can discover these distortions sensitively, therefore, the objective evaluation method for quality of stereo images that the present invention proposes extracts four kinds of Perception Features figure, comprise remarkable figure, gradient map, the proper discernable distortion map in spatial domain and disparity map, and formed perception feature set using its distortion level as characteristic parameter, weigh the distortion level of stereo-picture.The method of the objective evaluation method for quality of stereo images machine learning that the present invention proposes carries out Fusion Features, obtains the highest feature high dimensional nonlinear Fusion Model of accuracy, thus reach higher forecasting accuracy by training random forest machine learning algorithm.

A kind of objective evaluation method for quality of stereo images based on perception feature set that the present invention proposes, it totally realizes block diagram as shown in Figure 1, and it comprises the following steps:

1. I is made _orgrepresent original undistorted stereo-picture, make I _disrepresent the stereo-picture of distortion to be evaluated, by I _orgleft visual point image be designated as L _org, by I _orgright visual point image be designated as R _org, by I _disleft visual point image be designated as L _dis, by I _disright visual point image be designated as R _dis.

Fig. 2 a gives the left visual point image of original undistorted horse stereo-picture, and Fig. 2 b gives the left visual point image of distortion that the image shown in Fig. 2 a obtains after JPEG compression.

2. adopt existing frequency modulation conspicuousness detection algorithm, obtain L _org, R _org, L _disand R _disrespective remarkable figure, correspondence is designated as with then calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{sal}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^L(i,j)-S_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{sal}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^R(i,j)-S_{\mathrm{dis}}^R(i,j))}^2.$

Wherein, M represents I _organd I _diswidth, N represents I _organd I _disheight, 1≤i≤M, 1≤j≤N, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j).

Remarkable figure, Fig. 2 d that Fig. 2 c gives the image shown in Fig. 2 a gives the remarkable figure of the image shown in Fig. 2 b.

3. adopt existing horizontal Sobel operator, obtain L _org, R _org, L _disand R _disrespective horizontal gradient figure, correspondence is designated as with and adopt existing vertical Sobel operator, obtain L _org, R _org, L _disand R _disrespective vertical gradient map, correspondence is designated as with then basis with obtain L _orggradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{org}}^L(i,j)=\sqrt{G_h^{L,\mathrm{org}}{(i,j)}^2+G_v^{L,\mathrm{org}}{(i,j)}^2};$ And according to with obtain R _orggradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{org}}^R(i,j)=\sqrt{G_h^{R,\mathrm{org}}{(i,j)}^2+G_v^{R,\mathrm{org}}{(i,j)}^2};$ According to with obtain L _disgradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{dis}}^L(i,j)=\sqrt{G_h^{L,\mathrm{dis}}{(i,j)}^2+G_v^{L,\mathrm{dis}}{(i,j)}^2};$ According to with obtain R _disgradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{dis}}^R(i,j)=\sqrt{G_h^{R,\mathrm{dis}}{(i,j)}^2+G_v^{R,\mathrm{dis}}{(i,j)}^2};$ Calculate again with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{gra}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^L(i,j)-S_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{gra}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(G_{\mathrm{org}}^R(i,j)-G_{\mathrm{dis}}^R(i,j))}^2.$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j).

Fig. 2 e gives the gradient map of the image shown in Fig. 2 a, and Fig. 2 f gives the gradient map of the image shown in Fig. 2 b.

4. adopt proper discernable distortion (Just-Noticeable-Distortion, the JND) model in existing spatial domain, obtain L _org, R _org, L _disand R _disthe proper discernable distortion map in respective spatial domain, correspondence is designated as with then calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{JND}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(J_{\mathrm{org}}^L(i,j)-J_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{JND}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(J_{\mathrm{org}}^R(i,j)-J_{\mathrm{dis}}^R(i,j))}^2.$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j).

At this, by L _org, R _org, L _disand R _disrespectively as pending image, then the detailed process obtaining the proper discernable distortion map in spatial domain of pending image is:

4.-1, pending image is designated as I;

-2 4., obtain the proper discernable distortion map of brightness of I, be designated as JND _l, by JND _lmiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as JND _l(i, j), wherein, represent that in I, coordinate position is the background value of the pixel of (i, j);

-3 4., obtain the proper discernable distortion map of texture of I, be designated as JND _t, by JND _tmiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as JND _t(i, j), JND _t(i, j)=η G (i, j) W _e(i, j), wherein, η is regulatory factor, gets η=0.01 at this, and G (i, j) represents that in I, coordinate position is the maximum gradient mean value of pixel under the high-pass filtering operator of different directions of (i, j), grad _k(i, j) represents that in I, coordinate position is the gradient mean value of pixel under the high-pass filtering operator in a kth direction of (i, j), and four direction is horizontal direction, vertical direction and two diagonals respectively, W _e(i, j) represents that in I, coordinate position is the Weighted Edges factor of the pixel of (i, j), $W_e(i,j)=0.0001\stackrel{\‾}{I}(i,j)+0.115;$

4.-4, according to JND _land JND _t, obtain the proper discernable distortion map in spatial domain of I, be designated as JND _s, by JND _smiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as JND _s(i, j), JND _s(i, j)=JND _l(i, j)+JND _t(i, j)-0.3 × min{JND _l(i, j), JND _t(i, j) }, wherein, min () is for getting minimum value function.

Fig. 2 g gives the proper discernable distortion map in spatial domain of the image shown in Fig. 2 a, and Fig. 2 h gives the proper discernable distortion map in spatial domain of the image shown in Fig. 2 b.

5. adopt existing light stream matching method, obtain I _orghorizontal parallax amplitude figure and I _orgvertical parallax amplitude figure, correspondence is designated as with then basis with obtain I _orgdisparity map, be designated as D _org, by D _orgmiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as D _org(i, j), equally, adopt existing light stream matching method, obtain I _dishorizontal parallax amplitude figure and I _disvertical parallax amplitude figure, correspondence is designated as with then basis with obtain I _disdisparity map, be designated as D _dis, by D _dismiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as D _dis(i, j), calculate D afterwards _organd D _dismean square error, be designated as MSE _dsp, ${\mathrm{MSE}}_{\mathrm{dsp}}=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(D_{\mathrm{org}}(i,j)-D_{\mathrm{dis}}(i,j))}^2.$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j).

Fig. 2 i gives the disparity map of stereo-picture corresponding to image shown in Fig. 2 a, and Fig. 2 j gives the disparity map of stereo-picture corresponding to image shown in Fig. 2 b.

6. will with the sets definition that arranged in sequence is formed is I _disperception feature set, be designated as P, $P=\{{\mathrm{MSE}}_{\mathrm{sal}}^L,{\mathrm{MSE}}_{\mathrm{sal}}^R,{\mathrm{MSE}}_{\mathrm{gra}}^L,{\mathrm{MSE}}_{\mathrm{gra}}^R,{\mathrm{MSE}}_{\mathrm{JND}}^L,{\mathrm{MSE}}_{\mathrm{JND}}^R,{\mathrm{MSE}}_{\mathrm{dsp}}\}.$

7. adopt n original undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion level of different type of distortion, using this distortion stereo-picture set as training set, training set comprises several distortion stereo-pictures; The mean subjective suggestion of the every width distortion stereo-picture then utilizing existing subjective quality assessment method evaluation to go out in training set is divided, and the mean subjective suggestion of the jth width distortion stereo-picture in training set is divided and is designated as MOS _j; Again according to step 1. to step process 6., obtain the perception feature set of the every width distortion stereo-picture in training set in an identical manner, the perception feature set of the jth width distortion stereo-picture in training set be designated as P _j.

Wherein, n>=1, as got n=1000,1≤j≤S, S represents total width number of the distortion stereo-picture comprised in training set, MOS _j∈ [0,5].

8. existing random forest machine learning algorithm is adopted, the perception feature set of all distortion stereo-pictures in training set is trained, make through training the regression function value that obtains to divide with corresponding mean subjective suggestion between error minimum, structure obtains random forest training pattern.

9. according to constructing the random forest training pattern obtained, to I _disperception feature set P test, prediction obtain I _disevaluating objective quality predicted value, be designated as Q _dis, Q _dis=MOD (P), wherein, the function representation form that MOD () is random forest training pattern.

For further illustrating feasibility and the validity of the inventive method, the inventive method is tested.

Random Forest model is made up of 20000 decision trees, and the characteristic of structure decision tree is 2.

Adopt 12 undistorted stereo-pictures shown in Fig. 3 a to Fig. 3 l, set up its distortion stereo-picture set under the different distortion level of 5 kinds of different type of distortion, comprise the stereo-picture of the distortion stereo-picture of 60 width JPEG compressed encodings distortion (JPEG), the distortion stereo-picture of 60 width JP2000 compressed encodings distortion (JP2K), the distortion stereo-picture of 60 width white Gaussian noise distortions (WN), the distortion stereo-picture of 60 width Gaussian Blur distortions (GB) and 72 width 312 width distortions H.264 in coding distortion (H.264) situation.The correlation that the respective evaluating objective quality predicted value of stereo-picture of the distortion that analysis and utilization the inventive method obtains and mean subjective are marked between difference.By randomly draw in the stereo-picture of above-mentioned 312 width distortions 80% the stereo-picture composing training collection of distortion, the stereo-picture of the distortion of residue 20% forms test set; Then the mean subjective suggestion utilizing subjective quality assessment method evaluation to go out the stereo-picture of the every width distortion in training set is divided; Again according to step 1. to step process 6., obtain the perception feature set of the stereo-picture of the every width distortion in training set in an identical manner; Then random forest machine learning algorithm is adopted, the perception feature set of all distortion stereo-pictures in training set is trained, make through training the regression function value that obtains to divide with corresponding mean subjective suggestion between error minimum, structure obtains random forest training pattern; Afterwards according to constructing the random forest training pattern obtained, test the perception feature set of the stereo-picture of the every width distortion in test set, prediction obtains the evaluating objective quality predicted value of the stereo-picture of the every width distortion in test set.

Here, utilize 3 of evaluate image quality evaluating method conventional objective parameters as evaluation index, i.e. Pearson linear correlation property coefficient (Pearson Linear Correlation Coefficients, PLCC), Spearman rank correlation coefficient (Spearman Rank Order Correlation coefficient, and root-mean-square error (Rooted Mean Squared Error, RMSE) SROCC).The span of PLCC and SROCC is [0,1], and its value, more close to 1, shows that evaluation method is better, otherwise, poorer; RMSE value is less, and represent that the prediction of evaluation method is more accurate, performance is better, otherwise, then poorer.Represent PLCC, SROCC and RMSE coefficient of assess performance as listed in table 1.From the data listed by table 1, PLCC and SROCC value is all more than 0.94, RMSE is lower than 5.8, that is, evaluating objective quality predicted value and the mean subjective correlation of marking between difference of the stereo-picture of the distortion utilizing the inventive method to obtain are very high, show that the result of objective evaluation result and human eye subjective perception is more consistent, be enough to the validity that the inventive method is described.

The correlation that the evaluating objective quality predicted value of the stereo-picture of the distortion that table 1 calculates by the inventive method and mean subjective are marked between difference

Claims (1)

1., based on an objective evaluation method for quality of stereo images for perception feature set, it is characterized in that comprising the following steps:

1. I is made _orgrepresent original undistorted stereo-picture, make I _disrepresent the stereo-picture of distortion to be evaluated, by I _orgleft visual point image be designated as L _org, by I _orgright visual point image be designated as R _org, by I _disleft visual point image be designated as L _dis, by I _disright visual point image be designated as R _dis;

2. adopt frequency modulation conspicuousness detection algorithm, obtain L _org, R _org, L _disand R _disrespective remarkable figure, correspondence is designated as with then calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{sal}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^L(i,j)-S_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{sal}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(S_{\mathrm{org}}^R(i,j)-S_{\mathrm{dis}}^R(i,j))}^2;$

Wherein, M represents I _organd I _diswidth, N represents I _organd I _disheight, 1≤i≤M, 1≤j≤N, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

3. adopt horizontal Sobel operator, obtain L _org, R _org, L _disand R _disrespective horizontal gradient figure, correspondence is designated as with and adopt vertical Sobel operator, obtain L _org, R _org, L _disand R _disrespective vertical gradient map, correspondence is designated as with then basis with obtain L _orggradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{org}}^L(i,j)=\sqrt{G_h^{L,\mathrm{org}}{(i,j)}^2+G_v^{L,\mathrm{org}}{(i,j)}^2};$ And according to with obtain R _orggradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{org}}^R(i,j)=\sqrt{G_h^{R,\mathrm{org}}{(i,j)}^2+G_v^{R,\mathrm{org}}{(i,j)}^2};$ According to with obtain L _disgradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{dis}}^L(i,j)=\sqrt{G_h^{L,\mathrm{dis}}{(i,j)}^2+G_v^{L,\mathrm{dis}}{(i,j)}^2};$ According to with obtain R _disgradient map, be designated as will middle coordinate position is that the pixel value of the pixel of (i, j) is designated as $G_{\mathrm{dis}}^R(i,j)=\sqrt{G_h^{R,\mathrm{dis}}{(i,j)}^2+G_v^{R,\mathrm{dis}}{(i,j)}^2};$ Calculate again with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{gra}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(G_{\mathrm{org}}^L(i,j)-G_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{gra}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(G_{\mathrm{org}}^R(i,j)-G_{\mathrm{dis}}^R(i,j))}^2;$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

4. adopt the proper discernable distortion model in spatial domain, obtain L _org, R _org, L _disand R _disthe proper discernable distortion map in respective spatial domain, correspondence is designated as with then calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{JND}}^L=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(J_{\mathrm{org}}^L(i,j)-J_{\mathrm{dis}}^L(i,j))}^2;$ Equally, calculate with mean square error, be designated as ${\mathrm{MSE}}_{\mathrm{JND}}^R=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(J_{\mathrm{org}}^R(i,j)-J_{\mathrm{dis}}^R(i,j))}^2;$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

5. adopt light stream matching method, obtain I _orghorizontal parallax amplitude figure and I _orgvertical parallax amplitude figure, correspondence is designated as with then basis with obtain I _orgdisparity map, be designated as D _org, by D _orgmiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as D _org(i, j), equally, adopt light stream matching method, obtain I _dishorizontal parallax amplitude figure and I _disvertical parallax amplitude figure, correspondence is designated as with then basis with obtain I _disdisparity map, be designated as D _dis, by D _dismiddle coordinate position is that the pixel value of the pixel of (i, j) is designated as D _dis(i, j), $D_{\mathrm{dis}}(i,j)=\sqrt{{\left(D_h^{\mathrm{dis}}(i,j)\right)}^2+{\left(D_v^{\mathrm{dis}}(i,j)\right)}^2};$ Calculate D afterwards _organd D _dismean square error, be designated as MSE _dsp, ${\mathrm{MSE}}_{\mathrm{dsp}}=\frac{1}{M\×N}{\mathrm{\Σ}}_{i=1}^M{\mathrm{\Σ}}_{j=1}^N{(D_{\mathrm{org}}(i,j)-D_{\mathrm{dis}}(i,j))}^2;$

Wherein, represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j), represent middle coordinate position is the pixel value of the pixel of (i, j);

6. will and MSE _dspthe sets definition that arranged in sequence is formed is I _disperception feature set, be designated as P, $P=\{{\mathrm{MSE}}_{\mathrm{sal}}^L,{\mathrm{MSE}}_{\mathrm{sal}}^R,{\mathrm{MSE}}_{\mathrm{gra}}^L,{\mathrm{MSE}}_{\mathrm{gra}}^R,{\mathrm{MSE}}_{\mathrm{JND}}^L,{\mathrm{MSE}}_{\mathrm{JND}}^R,{\mathrm{MSE}}_{\mathrm{dsp}}\};$

7. adopt n original undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion level of different type of distortion, using this distortion stereo-picture set as training set, training set comprises several distortion stereo-pictures; The mean subjective suggestion of the every width distortion stereo-picture then utilizing subjective quality assessment method evaluation to go out in training set is divided, and the mean subjective suggestion of the jth width distortion stereo-picture in training set is divided and is designated as MOS _j; Again according to step 1. to step process 6., obtain the perception feature set of the every width distortion stereo-picture in training set in an identical manner, the perception feature set of the jth width distortion stereo-picture in training set be designated as P _j;

Wherein, n>=1,1≤j≤S, S represents total width number of the distortion stereo-picture comprised in training set, MOS _j∈ [0,5];

8. random forest machine learning algorithm is adopted, the perception feature set of all distortion stereo-pictures in training set is trained, make through training the regression function value that obtains to divide with corresponding mean subjective suggestion between error minimum, structure obtains random forest training pattern;

9. according to constructing the random forest training pattern obtained, to I _disperception feature set P test, prediction obtain I _disevaluating objective quality predicted value, be designated as Q _dis, Q _dis=MOD (P), wherein, the function representation form that MOD () is random forest training pattern.