CN105338343A - No-reference stereo image quality evaluation method based on binocular perception - Google Patents

Abstract

The invention relates to an image quality evaluation method, and specifically relates to a no-reference stereo image quality evaluation method based on binocular perception, and belongs to the image analysis field. The method utilizes the left view and the right view of a distorted stereo image to calculate the parallax and the information entropy, and then the left view and the right view are respectively synthesized into a monocular image and a dot product graph. Natural scene statistical characteristics are extracted from four input images (the left view, the right view, the monocular image and the dot product graph) respectively. Quality forecasting and evaluation can be performed by using a machine learning method (such as SVM (Support Vector Machine)). The no-reference stereo image quality evaluation method is high in the subjective consistency and can be embedded into an application system related with stereo image/video processing, thus having high application value.

Description

A kind of nothing based on binocular perception is with reference to stereo image quality evaluation method

Technical field

The present invention relates to a kind of image quality evaluating method, particularly a kind of nothing based on binocular perception is with reference to stereo image quality evaluation method, belongs to art of image analysis.

Background technology

In the past in a decade or so in, the quantity sharp increase of the stereoscopic image/video resource that people can contact.Can say popular, popular, the trend that stereoscopic image/video resource has become.Along with progressively popularizing of three-dimensional resource, thing followed side effect is uneven visual quality.Each stage that stereoscopic image/video stores in scene collection, coding, Internet Transmission, decoding, post-processed, compression and shows all inevitably can introduce distortion.Such as, in scene gatherer process due to fuzzy distortion that the factors such as apparatus parameter setting, camera lens rock cause; The compression artefacts that compress of stereo image storage causes, the binocular competition that especially asymmetric compressed encoding is introduced and depression effect; Three-dimensional film is in end processing sequences, and because the understanding for the three-dimensional perception theory of human eye is not enough, the stereoeffect after process can cause the fatigue of human eye and the dispirited of spirit, and then affects the physical and mental health of the mankind.

Therefore, the visual quality of how to evaluate stereoscopic image/video resource has become a urgent problem.The application system of reality can the ability of the three-dimensional media quality of automatic Evaluation in the urgent need to obtaining, and then can promote the visual quality of media.Specifically, this research has following using value:

(1) can embed in actual application system (projection system, network transmission system etc. of such as three-dimensional film), the quality of real-time monitoring stereoscopic image/video;

(2) may be used for the quality evaluating various image/video Processing Algorithm, instrument (the asymmetric compressed encoding, stereoscopic image/video sampling instrument etc. of such as stereo-picture);

(3) may be used for the quality audit of stereoscopic image/video works, prevent stereo product inferior from endangering the physical and mental health of spectators.

In sum, for objectively without the research with reference to stereo image quality evaluation model, there is important theory value and realistic meaning.The present invention proposes a kind of nothing based on binocular perception with reference to stereo image quality evaluation method, the prior art of its reference is two step frameworks of the image quality evaluation that the people such as Moorthy propose in document " Atwo-stepframeworkforconstructingblindimagequalityindice s ", the basic background technology Major Natural scene statistics feature related to.

(1) two step frameworks of image quality evaluation

The people such as Moorthy propose two step frameworks of non-reference picture quality appraisement, namely distorted image are carried out successively to the quality evaluation of distortion classification and certain distortion type.

(1) distortion classification: the training set of Given Graph picture and corresponding type of distortion, trains a grader with the characteristic vector of image and correct classification as input.After obtaining grader, input piece image, just can carry out probability Estimation to the type of distortion in image, this estimate show in image contain each type of distortion number.Like this for an image feature vector, grader can export the vectorial p of a n dimension.

(2) quality evaluation of certain distortion type: for n kind type of distortion, closes at training set respectively and trains respective regression model that image feature vector is mapped on mass fraction.After obtaining n matching device, input piece image, just can utilize n matching device to carry out quality evaluation to image respectively, obtains image and estimates that q, q are also n dimensions about the quality of certain distortion.

(3) quality gathers: according to two vectorial p and q obtained, each probability weight occurred based on distortion in the mark image of certain distortion quality, can obtain objective prediction mark

Wherein, p _irepresent the i-th dimension component of vectorial p, q _irepresent the i-th dimension component of vectorial q, n represents the kind number of distortion.

(2) natural scene statistical nature

Natural scene statistical modeling refers to that the coefficient value distribution situation of certain or some coefficient domain of stereoscopic image is analyzed, according to the rule of its distribution, utilize parameter distribution function to carry out matching to it, the parameter of matching as the characteristic value of image, for quality evaluation.Utilize natural scene statistical property to carry out quality evaluation be one the most effectively, the idea in forward position the most, the whether suitable quality directly determining quality evaluation algorithm performance of natural scene statistical model.The natural scene statistical nature used herein comprises the asymmetrical generalized Gaussian distribution feature of symmetrical generalized Gaussian distribution characteristic sum.

The symmetrical generalized Gaussian distribution of zero-mean can be described as:

Wherein,

Γ () is gamma function, and α is form parameter feature, and β is scale parameter feature.

The asymmetric generalized Gaussian distribution of zero-mean can be described as:

Wherein,

α is form parameter characteristic sum β _land β _rbe respectively the scale parameter feature of the left and right sides.

Summary of the invention

The object of the invention is to solve without the performance with reference to stereo image quality assessment technique low, subjective consistency is poor, and time complexity and the large problem of space complexity, provide a kind of based on binocular perception without reference stereo natural image quality evaluating method.

The inventive method is achieved through the following technical solutions.

Based on a nothing reference stereo image quality evaluation method for binocular perception, step is as follows:

Step one, left and right view for testing image, calculate its disparity map d, then calculates left and right view respectively in the amount of information of locus i, utilize amount of information composite calulation single eye images c={c _i: i ∈ I};

Computational methods are as follows:

Step 1.1, utilizes gauss hybrid models, and left and right view is decomposed into two random fields.

v＝s·u，(6)

Wherein, represent left and right view, i is spatial index.S={s _i: i ∈ I} is non-negative yardstick random field, for average is 0, covariance is C _ugaussian vectors.

Step 1.2, calculates the amount of information of left and right view at locus i

Step 1.3, based on binocular apperceive characteristic, utilizes the comentropy of left and right view, synthesis single eye images c={c _i: i ∈ I}.

Wherein, for the comentropy of left view on the i of locus, G _l,ifor the gray value of left view on the i of locus, d _ifor the horizontal parallax at i place, locus, e is a very little positive number is an empirical parameter to guarantee that coefficient is greater than 0, α.

Step 2, according to disparity map and left and right view, calculate dot product figure P={P _i: i ∈ I};

Computational methods are as follows:

Wherein, G _l,ifor the gray value of left view on the i of locus, d _ifor the horizontal parallax at i place, locus.

Step 3, feature extraction.

Natural scene statistical nature is extracted respectively on left view, right view, single eye images and dot product figure.

The method of step 4, employing step one and step 2 processes each the width stereo-picture in database, calculates the quality characteristic vector that each width stereo-picture is corresponding.Then utilize the machine learning method based on study, training set is trained, test set is tested, quality characteristic vector is mapped as corresponding mass fraction.And then utilize the quality of existing algorithm performance index (SROCC, PCC etc.) to algorithm to assess.

Natural scene statistical nature in step 3 can also be replaced by the feature such as various global characteristics, local feature, color characteristic, textural characteristics of image.

Machine learning method in step 4 can adopt SVMs (SVM), the machine learning methods such as neural net.

Beneficial effect

The nothing based on binocular perception that the present invention proposes, with reference to stereo image quality evaluation method, compared with the prior art has subjective consistency high; Can be embedded in the relevant application system of stereoscopic image/video process, there is very strong using value.

Accompanying drawing explanation

Fig. 1 is the flow chart without reference stereo image quality evaluation method based on binocular apperceive characteristic of the present invention;

Fig. 2 is the left view of testing image in the specific embodiment of the invention 1;

Fig. 3 is the right view of testing image in the specific embodiment of the invention 1;

Fig. 4 is the single eye images of testing image in the specific embodiment of the invention 1;

Fig. 5 is the dot product figure of testing image in the specific embodiment of the invention 1.

Embodiment

Elaborate below in conjunction with the execution mode of the drawings and specific embodiments to the inventive method.

Embodiment

As shown in Figure 1, specific implementation process is the flow process of this method:

Step one, left and right view for testing image, calculate its disparity map d, then calculates the amount of information of left and right view at locus i respectively, utilizes amount of information to synthesize single eye images c={c _i: i ∈ I}.Circular is as follows:

Step 1.1, utilizes gauss hybrid models, and left and right view is decomposed into two random fields.

v＝s·u，(11)

Wherein, represent left and right view, i is spatial index.S={s _i: i ∈ I} is non-negative yardstick random field, for average is 0, covariance is C _ugaussian vectors.

Step 1.2, calculates the amount of information of left and right view at locus i

Step 1.3, based on binocular apperceive characteristic, utilizes the comentropy of left and right view, synthesis single eye images c={c _i: i ∈ I}.

Wherein, for the comentropy of left view on the i of locus, G _l,ifor the gray value of left view on the i of locus, d _ifor the horizontal parallax at i place, locus, e is a very little positive number is an empirical parameter to guarantee that coefficient is greater than 0, α.

Step 2, according to disparity map and left and right view, calculate dot product figure P={P _i: i ∈ I}.Circular is as follows:

$P_i=G_{L,i}*G_{R,i+d_i},---\left(15\right)$

Wherein, G _l,ifor the gray value of left view on the i of locus, d _ifor the horizontal parallax at i place, locus.

Step 3, feature extraction.Natural scene statistical nature is extracted respectively on left view, right view, single eye images and dot product figure.Circular is as follows:

Step 3.1, four kinds of input pictures extract symmetrical generalized Gaussian distribution feature, and characteristic value is form parameter characteristic sum scale parameter feature, and computational methods are shown in formula (1), (2).In addition as a supplement, the degree of bias value of distribution and kurtosis value have also been added into characteristic vector.

Step 3.2, for four kinds of input pictures, i.e. left view, right view, single eye images and dot product figure, calculate the dot product between its neighbor on four direction (level, vertically, leading diagonal and counter-diagonal) respectively, computational methods are as follows:

H(i,j)＝Im(i,j)Im(i,j+1)，(16)

V(i,j)＝Im(i,j)Im(i+1,j)，(17)

D1(i,j)＝Im(i,j)Im(i+1,j+1)，(18)

D2(i,j)＝Im(i,j)Im(i+1,j-1)，(19)

Wherein, Im represents the one in four kinds of input pictures, and H represents the dot product figure of horizontal direction neighbor, and V represents vertical direction, and D1 represents leading diagonal direction, D2 vice diagonal.

Step 3.3, respectively on the neighbor dot product figure of the four direction of four kinds of input pictures, extract asymmetric symmetrical generalized Gaussian distribution feature, characteristic value is scale parameter feature on the right side of scale parameter characteristic sum on the left of form parameter feature, and computational methods are shown in formula (3), (4), (5).In addition as a supplement, the degree of bias value of distribution and kurtosis value have also been added into characteristic vector.

Step 4, adopt the method for step one and step 2 to process each the width stereo-picture in database, calculate each width stereo-picture corresponding quality characteristic vector.Then the method for SVMs (SVM) is utilized, train training set extracting the quality characteristic vector obtained, obtain distortion disaggregated model and certain distortion Environmental Evaluation Model, then two step frameworks of image quality evaluation are utilized, test set is tested, the quality characteristic vector that the disaggregated model, certain distortion Environmental Evaluation Model and the test set that utilize training to obtain are corresponding, carries out prediction of quality, obtains the mass fraction that each quality characteristic vector is corresponding.And then utilize the quality of existing algorithm performance index (SROCC, PCC etc.) to algorithm to assess.

We implement our algorithm on LIVE3DII image quality evaluation database.This database is made up of 8 width natural images and 360 width distorted images, comprises five kinds of type of distortion: JP2K compressions, JPEG compression, white noise (WN), rapid decay (FF) and fuzzy (Blur).Symmetrical distortion and expense symmetrical distortion is contained in this database.We have selected the 2D/3D image quality evaluation algorithm algorithm as a comparison of several current exhibits excellent.In order to eliminate the impact of training data, we have carried out the duplicate test of 1000 times 80% training-20% test on the database, and namely the data of 80% are used for training, and the data of remaining 20% are used for test, and training data and test data do not exist the overlap of content.Existing algorithm performance index (intermediate values of 1000 duplicate test SROCC) is finally utilized to assess (see table 1) the quality of algorithm.

In table 1LIVE3DII storehouse, each algorithm performance index (intermediate values of 1000 duplicate test SROCC) compares

	WN	JP2K	JPEG	Blur	FF	All
							2D PSNR	0.932	0.799	0.121	0.902	0.588	0.834
2D SSIM	0.938	0.858	0.436	0.879	0.586	0.877
							2D MS-SSIM	0.942	0.893	0.611	0.926	0.734	0.924
2D BRISQUE	0.940	0.812	0.569	0.860	0.784	0.901
							Benoit	0.930	0.910	0.603	0.931	0.699	0.899
You	0.940	0.860	0.439	0.882	0.588	0.878
							Chen FR	0.948	0.888	0.53	0.925	0.707	0.916
Hewage	0.940	0.856	0.500	0.690	0.545	0.814
							Akhter	0.914	0.866	0.675	0.555	0.640	0.383
Chen NR	0.919	0.863	0.617	0.878	0.652	0.891
							The method proposed	0.956	0.853	0.721	0.896	0.806	0.940

We can find out that the method that in the present invention, we propose all shows good subjective consistency for all kinds of distorted image, and versatility is good; And compared with the existing methods comparatively, in performance, tool has great advantage.The subjective consistency of the method proposed from overall performance the present invention is better than other 2D/3D image quality evaluation algorithms.

Claims (4)

1., based on a nothing reference stereo image quality evaluation method for binocular perception, it is characterized in that: its concrete steps are as follows:

Step one, left and right view for testing image, calculate its disparity map d, then calculates left and right view respectively in the amount of information of locus i, utilize amount of information composite calulation single eye images c={c _i: i ∈ I};

Step 2, according to disparity map and left and right view, calculate dot product figure P={P _i: i ∈ I};

Computational methods are as follows:

$P_i=G_{L,i}*{G_{R,i+d}}_i,---\left(1\right)$

Wherein, G _l,ifor the gray value of left view on the i of locus, d _ifor the horizontal parallax at i place, locus;

Step 3, feature extraction;

Natural scene statistical nature is extracted respectively on left view, right view, single eye images and dot product figure;

Circular is as follows:

The method of step 4, employing step one and step 2 processes each the width stereo-picture in database, calculates the quality characteristic vector that each width stereo-picture is corresponding; Then utilize the machine learning method based on study, training set is trained, test set is tested, quality characteristic vector is mapped as corresponding mass fraction; And then utilize the quality of existing algorithm performance index (SROCC, PCC etc.) to algorithm to assess.

2. a kind of nothing based on binocular perception according to claim 1 is with reference to stereo image quality evaluation method, it is characterized in that: the computational methods calculating its single eye images in described step 1 are as follows:

Step 1.1, utilizes gauss hybrid models, and left and right view is decomposed into two random fields;

v＝s·u，(2)

Wherein, represent left and right view, i is spatial index; S={s _i: i ∈ I} is non-negative yardstick random field, for average is 0, covariance is C _ugaussian vectors;

Step 1.2, calculates the amount of information of left and right view at locus i

$\begin{array}{c}In\left(\stackrel{\→}{v_i}\right)=h\left(\stackrel{\→}{v_i}\right|s_i),---\left(3\right)\\ =\frac{1}{2}{\log }_2\left({s_i}^2{C}_u+1\right),---\left(4\right)\end{array}$

Step 1.3, based on binocular apperceive characteristic, utilizes the comentropy of left and right view, synthesis single eye images c={c _i: i ∈ I};

Wherein, for the comentropy of left view on the i of locus, G _l,ifor the gray value of left view on the i of locus, d _ifor the horizontal parallax at i place, locus, e is a very little positive number is an empirical parameter to guarantee that coefficient is greater than 0, α.

3. a kind of nothing based on binocular perception according to claim 1 is with reference to stereo image quality evaluation method, it is characterized in that: the natural scene statistical nature in step 3 also comprises to be replaced by the feature such as various global characteristics, local feature, color characteristic, textural characteristics of image.

4. a kind of nothing based on binocular perception according to claim 1 is with reference to stereo image quality evaluation method, it is characterized in that: the machine learning method in step 4 comprises employing SVMs (SVM), the machine learning methods such as neural net.