Summary of the invention
Technical problem to be solved by this invention provides a kind of stereo image quality method for objectively evaluating that can effectively improve the correlation of objective evaluation result and subjective perception.
The present invention solves the problems of the technologies described above the technical scheme that is adopted: a kind of stereo image quality method for objectively evaluating is characterized in that may further comprise the steps:
1. make S
OrgUndistorted stereo-picture for original makes S
DisFor the stereo-picture of distortion to be evaluated, with S
OrgLeft visual point image be designated as L
Org, with S
OrgRight visual point image be designated as R
Org, with S
DisLeft visual point image be designated as L
Dis, with S
DisRight visual point image be designated as R
Dis
2. to L
Org, R
Org, L
DisAnd R
Dis4 width of cloth images are implemented singular value decomposition respectively, obtain L respectively
Org, R
Org, L
DisAnd R
DisEach self-corresponding singular value vector of 4 width of cloth images is with L
OrgThe singular value vector be designated as
With R
OrgThe singular value vector be designated as
With L
DisThe singular value vector be designated as
With R
DisThe singular value vector be designated as
Wherein, the dimension of each singular value vector is m, and m=min (M, N), min () is for getting minimum value function, the horizontal size size of M presentation video, the vertical dimension size of N presentation video;
3. calculate L
OrgThe singular value vector
With L
DisThe singular value vector
The absolute difference vector, be designated as X
L,
With X
LAs L
DisCharacteristic vector, calculate R
OrgThe singular value vector
With R
DisThe singular value vector
The absolute difference vector, be designated as X
R,
With X
RAs R
DisCharacteristic vector, wherein, " || " is the symbol that takes absolute value;
4. to L
DisCharacteristic vector X
LAnd R
DisCharacteristic vector X
RCarry out linear weighted function, obtain S
DisCharacteristic vector, be designated as X, X=w
LX
L+ w
RX
R, wherein, w
LExpression L
DisWeights proportion, w
RExpression R
DisWeights proportion, w
L+ w
R=1;
5. adopt n undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion levels of different type of distortion, this distortion stereo-picture set comprises the stereo-picture of several distortions, utilizes the subjective quality evaluation method to obtain the average subjective scoring difference of the stereo-picture of every width of cloth distortion in the set of distortion stereo-picture respectively, is designated as DMOS, DMOS=100-MOS, wherein, MOS represents the subjective scoring average, DMOS ∈ [0,100], n 〉=1;
6. adopt and calculating S
DisThe identical method of characteristic vector X, the characteristic vector of the stereo-picture of every width of cloth distortion in the set of calculated distortion stereo-picture respectively, the characteristic vector for the stereo-picture of i width of cloth distortion in the set of distortion stereo-picture is designated as X with it
i, wherein, 1≤i≤n ', the width of cloth number of the stereo-picture of the distortion that comprises in the set of n ' expression distortion stereo-picture;
7. adopt support vector regression that the characteristic vector of the stereo-picture of all distortions of identical type of distortion in the set of distortion stereo-picture is trained, and the support vector regression training pattern of utilizing training to obtain is tested the stereo-picture of every width of cloth distortion of same type of distortion, calculate the evaluating objective quality predicted value of the stereo-picture of every width of cloth distortion of identical type of distortion in the set of distortion stereo-picture, evaluating objective quality predicted value for the stereo-picture of i width of cloth distortion in the set of distortion stereo-picture is designated as Q with it
i, Q
i=f (X
i), f () is the function representation form, Q
i=f (X
i) expression Q
iBe X
iFunction.
Described step detailed process 2. is:
2.-1, with size be the L of M * N
OrgBe expressed as the two-dimensional matrix of M * N dimension, be designated as
By the two-dimensional matrix of singular value decomposition with M * N dimension
Be expressed as
Wherein,
The orthogonal matrix of expression M * M dimension,
The orthogonal matrix of expression N * N dimension,
Expression
Transposed matrix,
The diagonal matrix of expression M * N dimension;
2.-2, the diagonal matrix that M * N is tieed up
Diagonal element as the two-dimensional matrix of M * N dimension
Singular value, from the two-dimensional matrix of M * N dimension
Singular value in take out the singular value formation L of m non-zero
OrgThe singular value vector, be designated as
Wherein, and m=min (M, N), min () is for getting minimum value function;
2.-3, to R
Org, L
DisAnd R
DisAdopt with step 2.-1 to 2.-2 identical operations, obtain R
Org, L
DisAnd R
DisThe singular value vector, be designated as respectively
With
Described step detailed process 7. is:
7.-1, the stereo-picture of all distortions of same type of distortion in the distortion stereo-picture set is divided into mutually disjoint 5 groups of subclass, select 4 groups of subclass composing training sample datas set wherein arbitrarily, be designated as Ω
q, { X
k, DMOS
k∈ Ω
q, wherein, q represents training sample data set omega
qIn the width of cloth number of stereo-picture of the distortion that comprises, X
kExpression training sample data set omega
qIn the characteristic vector of stereo-picture of k width of cloth distortion, DMOS
kExpression training sample data set omega
qIn the average subjective scoring difference of stereo-picture of k width of cloth distortion, 1≤k≤q;
7.-2, structure X
kRegression function f (X
k),
Wherein, f () is the function representation form, and w is a weight vector, w
TBe the transposed matrix of w, b is a bias term,
Expression training sample data set omega
qIn the linear function of characteristic vector Xk of stereo-picture of k width of cloth distortion,
D (X
k, X
l) be the kernel function in the support vector regression,
X
lBe training sample data set omega
qIn the characteristic vector of stereo-picture of l width of cloth distortion, γ is a nuclear parameter, is used to reflect the scope of importing sample value, the scope of sample value is big more, and the γ value is also just big more, and exp () expression is the exponential function at the end with e, e=2.71828183, " || || " for asking the Euclidean distance symbol;
7.-3, adopt support vector regression to training sample data set omega
qIn the characteristic vector of stereo-picture of all distortion train, make error minimum between the regression function value that obtains through training and the average subjective scoring difference, match obtains the weight vector w of optimum
OptBias term b with optimum
Opt, with the weight vector w of optimum
OptBias term b with optimum
OptCombination be designated as
The weight vector w of the optimum that utilization obtains
OptBias term b with optimum
OptStructure support vector regression training pattern is designated as
Wherein, Ψ represents training sample data set omega
qIn the set of combination of the characteristic vector of stereo-picture of all distortion all weight vector of training and bias term,
Expression minimizes probability density function, X
InpExpress support for the input vector of vector regression training pattern, (w
Opt)
TBe w
OptTransposed matrix,
Express support for the input vector X of vector regression training pattern
InpLinear function;
7.-4, according to the support vector regression training pattern, the stereo-picture that remains the every width of cloth distortion in 1 group of subclass is tested, prediction obtains the evaluating objective quality predicted value of the stereo-picture of every width of cloth distortion in this group subclass, evaluating objective quality predicted value for the stereo-picture of j width of cloth distortion in this group subclass is designated as Q with it
j, Q
j=f (X
j),
Wherein, X
jThe characteristic vector of representing the stereo-picture of j width of cloth distortion in this group subclass,
The linear function of representing the stereo-picture of j width of cloth distortion in this group subclass;
7.-5, according to step 7.-1 to 7.-4 process, respectively the stereo-picture of all distortions of different type of distortion in the set of distortion stereo-picture is trained, obtain the evaluating objective quality predicted value of the stereo-picture of every width of cloth distortion in the set of distortion stereo-picture.
Described step 6. in the characteristic vector process of the stereo-picture that calculates the JPEG compression artefacts, get w
L=0.50, w
R=0.50; In the characteristic vector process of the stereo-picture that calculates the JPEG2000 compression artefacts, get w
L=0.15, w
R=0.85; In the characteristic vector process of the stereo-picture that calculates the Gaussian Blur distortion, get w
L=0.10, w
R=0.90; In the characteristic vector process of the stereo-picture that calculates the white noise distortion, get w
L=0.20, w
R=0.80; In calculating the characteristic vector process of the stereo-picture of coding distortion H.264, get w
L=0.10, w
R=0.90.
Compared with prior art, the invention has the advantages that:
1) the inventive method is mapped in the high-dimensional feature space by the characteristic vector of support vector regression with stereo-picture, in high-dimensional feature space, carry out Linear Estimation again, the characteristic vector of structure optimum regression function stereoscopic image is tested, avoided human visual system's the correlation properties and the complicated simulation process of mechanism, and because training sample and test sample book are separate, can avoid test result to the depending on unduly of training data, thereby can effectively improve the correlation of objective evaluation result and subjective perception.
2) the inventive method adopts singular value decomposition method to obtain the left visual point image of stereo-picture and the characteristic vector of right visual point image, again according to the different type of distortion situations of stereo-picture, adopt different weights proportion that the characteristic vector of its left visual point image and right visual point image is carried out linear weighted function, obtain the characteristic vector information of stereo-picture, the characteristic vector information of the stereo-picture that obtains has stronger stability and can reflect the mass change situation of stereo-picture preferably, can reflect the stereoscopic vision masking effect of human eye well.
Description of drawings
Fig. 1 is the overall realization block diagram of the inventive method;
Fig. 2 a is the left visual point image of Akko (being of a size of 640 * 480) stereo-picture;
Fig. 2 b is the right visual point image of Akko (being of a size of 640 * 480) stereo-picture;
Fig. 3 a is the left visual point image of Altmoabit (being of a size of 1024 * 768) stereo-picture;
Fig. 3 b is the right visual point image of Altmoabit (being of a size of 1024 * 768) stereo-picture;
Fig. 4 a is the left visual point image of Balloons (being of a size of 1024 * 768) stereo-picture;
Fig. 4 b is the right visual point image of Balloons (being of a size of 1024 * 768) stereo-picture;
Fig. 5 a is the left visual point image of Doorflower (being of a size of 1024 * 768) stereo-picture;
Fig. 5 b is the right visual point image of Doorflower (being of a size of 1024 * 768) stereo-picture;
Fig. 6 a is the left visual point image of Kendo (being of a size of 1024 * 768) stereo-picture;
Fig. 6 b is the right visual point image of Kendo (being of a size of 1024 * 768) stereo-picture;
Fig. 7 a is the left visual point image of LeaveLaptop (being of a size of 1024 * 768) stereo-picture;
Fig. 7 b is the right visual point image of LeaveLaptop (being of a size of 1024 * 768) stereo-picture;
Fig. 8 a is the left visual point image of Lovebierd1 (being of a size of 024 * 768) stereo-picture;
Fig. 8 b is the right visual point image of Lovebier1 (being of a size of 1024 * 768) stereo-picture;
Fig. 9 a is the left visual point image of Newspaper (being of a size of 1024 * 768) stereo-picture;
Fig. 9 b is the right visual point image of Newspaper (being of a size of 1024 * 768) stereo-picture;
Figure 10 a is the left visual point image of Puppy (being of a size of 720 * 480) stereo-picture;
Figure 10 b is the right visual point image of Puppy (being of a size of 720 * 480) stereo-picture;
Figure 11 a is the left visual point image of Soccer2 (being of a size of 720 * 480) stereo-picture;
Figure 11 b is the right visual point image of Soccer2 (being of a size of 720 * 480) stereo-picture;
Figure 12 a is the left visual point image of Horse (being of a size of 720 * 480) stereo-picture;
Figure 12 b is the right visual point image of Horse (being of a size of 720 * 480) stereo-picture;
Figure 13 a is the left visual point image of Xmas (being of a size of 640 * 480) stereo-picture;
Figure 13 b is the right visual point image of Xmas (being of a size of 640 * 480) stereo-picture;
Figure 14 is the scatter diagram of objective image quality evaluation predicted value with the average subjective scoring difference of the stereo-picture of each distortion in the set of distortion stereo-picture.
Embodiment
Embodiment describes in further detail the present invention below in conjunction with accompanying drawing.
A kind of stereo image quality method for objectively evaluating that the present invention proposes, it totally realizes block diagram as shown in Figure 1, it mainly may further comprise the steps:
1. make S
OrgUndistorted stereo-picture for original makes S
DisFor the stereo-picture of distortion to be evaluated, with S
OrgLeft visual point image be designated as L
Org, with S
OrgRight visual point image be designated as R
Org, with S
DisLeft visual point image be designated as L
Dis, with S
DisRight visual point image be designated as R
Dis
2. to L
Org, R
Org, L
DisAnd R
Dis4 width of cloth images are implemented singular value decomposition respectively, obtain L respectively
Org, R
Org, L
DisAnd R
DisEach self-corresponding singular value vector of 4 width of cloth images is with L
OrgThe singular value vector be designated as
With R
OrgThe singular value vector be designated as
With L
DisThe singular value vector be designated as
With R
DisThe singular value vector be designated as
Wherein, the dimension of each singular value vector is m, and (M, N), min () is for getting minimum value function, the horizontal size size of M presentation video, the vertical dimension size of N presentation video for m=min.
In the present embodiment, step detailed process 2. is:
2.-1, with size be the L of M * N
OrgBe expressed as the two-dimensional matrix of M * N dimension, be designated as
By the two-dimensional matrix of singular value decomposition with M * N dimension
Be expressed as
Wherein,
The orthogonal matrix of expression M * M dimension,
The orthogonal matrix of expression N * N dimension,
Expression
Transposed matrix,
The diagonal matrix of expression M * N dimension;
2.-2, the diagonal matrix that M * N is tieed up
Diagonal element as the two-dimensional matrix of M * N dimension
Singular value, from the two-dimensional matrix of M * N dimension
Singular value in take out the singular value formation L of m non-zero
OrgThe singular value vector, be designated as
Wherein, and m=min (M, N), min () is for getting minimum value function;
2.-3, to R
Org, L
DisAnd R
DisAdopt with step 2.-1 to 2.-2 identical operations, obtain R
Org, L
DisAnd R
DisThe singular value vector, be designated as respectively
With
3. calculate L
OrgThe singular value vector
With L
DisThe singular value vector
The absolute difference vector, be designated as X
L,
With X
LAs L
DisCharacteristic vector, calculate R
OrgThe singular value vector
With R
DisThe singular value vector
The absolute difference vector, be designated as X
R,
With X
RAs R
DisCharacteristic vector, wherein, " || " is the symbol that takes absolute value.
4. to L
DisCharacteristic vector X
LAnd R
DisCharacteristic vector X
RCarry out linear weighted function, obtain S
DisCharacteristic vector, be designated as X, X=w
LX
L+ w
RX
R, wherein, w
LExpression L
DisWeights proportion, w
RExpression R
DisWeights proportion, w
L+ w
R=1.
5. adopt n undistorted stereo-picture, set up its distortion stereo-picture set under the different distortion levels of different type of distortion, this distortion stereo-picture set comprises the stereo-picture of several distortions, utilizes existing subjective quality evaluation method to obtain the average subjective scoring difference of the stereo-picture of every width of cloth distortion in the set of distortion stereo-picture respectively, is designated as DMOS, DMOS=100-MOS, wherein, MOS represents the subjective scoring average, DMOS ∈ [0,100], n 〉=1.
In the present embodiment, utilize stereo-picture as Fig. 2 a and Fig. 2 b formation, the stereo-picture that Fig. 3 a and Fig. 3 b constitute, the stereo-picture that Fig. 4 a and Fig. 4 b constitute, the stereo-picture that Fig. 5 a and Fig. 5 b constitute, the stereo-picture that Fig. 6 a and Fig. 6 b constitute, the stereo-picture that Fig. 7 a and Fig. 7 b constitute, the stereo-picture that Fig. 8 a and Fig. 8 b constitute, the stereo-picture that Fig. 9 a and Fig. 9 b constitute, the stereo-picture that Figure 10 a and Figure 10 b constitute, the stereo-picture that Figure 11 a and Figure 11 b constitute, the stereo-picture that Figure 12 a and Figure 12 b constitute, the stereo-picture of Figure 13 a and Figure 13 b formation undistorted stereo-picture of totally 12 width of cloth (n=12) has been set up its distortion stereo-picture set under the different distortion levels of different type of distortion, this distortion stereo-picture set comprises the stereo-picture of 312 width of cloth distortions of 5 kinds of type of distortion altogether, the stereo-picture of the distortion of JPEG compression totally 60 width of cloth wherein, the stereo-picture of the distortion of JPEG2000 compression is totally 60 width of cloth, the stereo-picture of the distortion of Gaussian Blur (Gaussian Blur) is totally 60 width of cloth, the stereo-picture of the distortion of white noise (White Noise) is totally 60 width of cloth, H.264 the stereo-picture of Bian Ma distortion totally 72 width of cloth.
6. adopt and calculating S
DisThe identical method of characteristic vector X, the characteristic vector of the stereo-picture of every width of cloth distortion in the set of calculated distortion stereo-picture respectively, the characteristic vector for the stereo-picture of i width of cloth distortion in the set of distortion stereo-picture is designated as X with it
i, wherein, 1≤i≤n ', the width of cloth number of the stereo-picture of the distortion that comprises in the set of n ' expression distortion stereo-picture.
In this specific embodiment, according to the stereoscopic vision masking effect inconsistent characteristic of human eye to different type of distortion, left visual point image to the stereo-picture of different type of distortion is provided with different weights proportion with right visual point image, in the characteristic vector process of the stereo-picture that calculates the JPEG compression artefacts, get w
L=0.50, w
R=0.50; In the characteristic vector process of the stereo-picture that calculates the JPEG2000 compression artefacts, get w
L=0.15, w
R=0.85; In the characteristic vector process of the stereo-picture that calculates the Gaussian Blur distortion, get w
L=0.10, w
R=0.90; In the characteristic vector process of the stereo-picture that calculates the white noise distortion, get w
L=0.20, w
R=0.80; In calculating the characteristic vector process of the stereo-picture of coding distortion H.264, get w
L=0.10, w
R=0.90.
7. because the characteristic vector of the stereo-picture of distortion is the higher dimensional space vector, need in higher dimensional space, construct linear decision function and realize non-linear decision function in the former space, (Support Vector Regression SVR) is the method for the non-linear higher dimensional space conversion of a kind of reasonable realization to support vector regression.Adopt support vector regression that the characteristic vector of the stereo-picture of all distortions of identical type of distortion in the set of distortion stereo-picture is trained, and the support vector regression training pattern of utilizing training to obtain is tested the stereo-picture of every width of cloth distortion of same type of distortion, calculate the evaluating objective quality predicted value of the stereo-picture of every width of cloth distortion of identical type of distortion in the set of distortion stereo-picture, evaluating objective quality predicted value for the stereo-picture of i width of cloth distortion in the set of distortion stereo-picture is designated as Q with it
i, Q
i=f (X
i), f () is the function representation form, Q
i=f (X
i) expression Q
iBe X
iFunction.
In this specific embodiment, step detailed process 7. is:
7.-1, the stereo-picture of all distortions of same type of distortion in the distortion stereo-picture set is divided into mutually disjoint 5 groups of subclass, select 4 groups of subclass composing training sample datas set wherein arbitrarily, be designated as Ω
q, { X
k, DMOS
k∈ Ω
q, wherein, q represents training sample data set omega
qIn the width of cloth number of stereo-picture of the distortion that comprises, X
kExpression training sample data set omega
qIn the characteristic vector of stereo-picture of k width of cloth distortion, DMOS
kExpression training sample data set omega
qIn the average subjective scoring difference of stereo-picture of k width of cloth distortion, 1≤k≤q;
7.-2, structure X
kRegression function f (X
k),
Wherein, f () is the function representation form, and w is a weight vector, w
TBe the transposed matrix of w, b is a bias term,
Expression training sample data set omega
qIn the characteristic vector X of stereo-picture of k width of cloth distortion
kLinear function,
D (X
k, X
l) be the kernel function in the support vector regression,
X
lBe training sample data set omega
qIn the characteristic vector of stereo-picture of l width of cloth distortion, γ is a nuclear parameter, is used to reflect the scope of importing sample value, the scope of sample value is big more, and the γ value is also just big more, and exp () expression is the exponential function at the end with e, e=2.71828183, " || || " for asking the Euclidean distance symbol;
In the present embodiment, JPEG compression artefacts, JPEG 2000 compression artefacts, Gaussian Blur distortion, white noise distortion and H.264 the γ value of coding distortion get 42,52,54,130 and 116 respectively.
7.-3, adopt support vector regression to training sample data set omega
qIn the characteristic vector of stereo-picture of all distortion train, make error minimum between the regression function value that obtains through training and the average subjective scoring difference, match obtains the weight vector w of optimum
OptBias term b with optimum
Opt, with the weight vector w of optimum
OptBias term b with optimum
OptCombination be designated as (w
Opt, b
Opt),
The weight vector w of the optimum that utilization obtains
OptBias term b with optimum
OptStructure support vector regression training pattern is designated as
Wherein, Ψ represents training sample data set omega
qIn the set of combination of the characteristic vector of stereo-picture of all distortion all weight vector of training and bias term,
Expression minimizes probability density function, X
InpExpress support for the input vector of vector regression training pattern, (w
Opt)
TBe w
OptTransposed matrix,
Express support for the input vector X of vector regression training pattern
InpLinear function;
7.-4, according to the support vector regression training pattern, the stereo-picture that remains the every width of cloth distortion in 1 group of subclass is tested, prediction obtains the evaluating objective quality predicted value of the stereo-picture of every width of cloth distortion in this group subclass, evaluating objective quality predicted value for the stereo-picture of j width of cloth distortion in this group subclass is designated as Q with it
j, Q
j=f (X
j),
Wherein, X
jThe characteristic vector of representing the stereo-picture of j width of cloth distortion in this group subclass,
The linear function of representing the stereo-picture of j width of cloth distortion in this group subclass;
7.-5, according to step 7.-1 to 7.-4 process, respectively the stereo-picture of all distortions of different type of distortion in the set of distortion stereo-picture is trained, obtain the evaluating objective quality predicted value of the stereo-picture of every width of cloth distortion in the set of distortion stereo-picture.
Adopt 12 undistorted stereo-pictures shown in Fig. 2 a to Figure 13 b to analyze the objective image quality evaluation predicted value and the average correlation between the subjective scoring difference of the stereo-picture of the distortion that present embodiment obtains at the stereo-picture of in various degree JPEG compression, JPEG2000 compression, Gaussian Blur, white noise and H.264 312 width of cloth distortions under the coding distortion situation.Here, 2 objective parameters commonly used that utilize the evaluate image quality evaluating method are as evaluation index, be Pearson correlation coefficient (the Correlation Coefficient under the nonlinear regression condition, CC), Spearman coefficient correlation (Rank-Order Correlation Coefficient, ROCC), the stereo-picture of CC reflection distortion is estimated the accuracy of objective models, and ROCC reflects its monotonicity.The objective image evaluation quality predicted value of the stereo-picture of the distortion that will calculate by present embodiment is done four parameter L ogistic function nonlinear fittings, and the high more explanation method for objectively evaluating of CC and ROCC value is good more with average subjective scoring difference correlation.CC, the ROCC coefficient of reflection three-dimensional image objective evaluation model performance are as shown in table 1, from the listed data of table 1 as can be known, correlation between the final objective image quality evaluation predicted value of the stereo-picture of the distortion that obtains by present embodiment and the average subjective scoring difference is very high, the result who shows objective evaluation result and human eye subjective perception is more consistent, is enough to illustrate the validity of the inventive method.
Figure 14 has provided the scatter diagram of objective image quality evaluation predicted value with the average subjective scoring difference of the stereo-picture of each distortion in the set of distortion stereo-picture, curve is obtained by four parameter L ogistic function nonlinear fittings, diffusing point is concentrated more, illustrates that the consistency of objective models and subjective perception is good more.As can be seen from Figure 14, the scatter diagram that adopts the inventive method to obtain is more concentrated, and the goodness of fit between the subjective assessment data is higher.
The image quality of stereoscopic images of the distortion that table 1 present embodiment obtains is estimated the correlation between predicted value and the subjective scoring