CN103903259A

CN103903259A - Objective three-dimensional image quality evaluation method based on structure and texture separation

Info

Publication number: CN103903259A
Application number: CN201410105777.4A
Authority: CN
Inventors: 邵枫; 李柯蒙; 李福翠
Original assignee: Ningbo University
Current assignee: Ningbo University
Priority date: 2014-03-20
Filing date: 2014-03-20
Publication date: 2014-07-02

Abstract

The invention discloses an objective three-dimensional image quality evaluation method based on structure and texture separation. The method includes the steps that firstly, structure and texture separation is conducted on a left viewpoint image body and a right viewpoint image body of an original undistorted three-dimensional image and a left viewpoint image body and a right viewpoint image body of a distorted three-dimensional image to be evaluated to obtain structure images and texture images of the left viewpoint image bodies and structure images and texture images of the right viewpoint image bodies; gradient similarity is used for evaluating the structure images of the left viewpoint image bodies and the structure images of the right viewpoint image bodies respectively, structural similarity is used for evaluating the texture images of the left viewpoint image bodies and the texture images of the right viewpoint image bodies, and an objective image quality evaluation prediction value of the distorted three-dimensional image to be evaluated is obtained through fusion. The objective three-dimensional image quality evaluation method based on structure and texture separation has the advantages that the structure images obtained through decomposition and the texture images obtained through decomposition can well represent influences on image quality of image structure and texture information, so that it seems that an evaluation result better conforms to a human visual system, and therefore the relevance of the objective evaluation result and subjective perception is effectively improved.

Description

Three-dimensional image quality objective evaluation method based on structure texture separation

Technical Field

The invention relates to an image quality evaluation method, in particular to a three-dimensional image quality objective evaluation method based on structure texture separation.

Background

With the rapid development of image coding technology and stereoscopic display technology, the stereoscopic image technology has received more and more extensive attention and application, and has become a current research hotspot. The stereoscopic image technology utilizes the binocular parallax principle of human eyes, the left viewpoint image and the right viewpoint image from the same scene are respectively and independently received by the two eyes, and the binocular parallax is formed through brain fusion, so that the stereoscopic image with depth perception and reality perception is appreciated. Because of the influence of the acquisition system and the storage compression and transmission equipment, a series of distortions are inevitably introduced into the stereo image, and compared with a single-channel image, the stereo image needs to ensure the image quality of two channels simultaneously, so that the quality evaluation of the stereo image is of great significance. However, currently, there is no effective objective evaluation method for evaluating the quality of stereoscopic images. Therefore, establishing an effective objective evaluation model of the quality of the stereo image has very important significance.

The current objective evaluation method for the quality of the stereo image is to directly apply a plane image quality evaluation method to evaluate the quality of the stereo image or evaluate the depth perception of the stereo image by evaluating the quality of a disparity map, however, the process of fusing the stereo image to generate the stereo effect is not an extension of the simple plane image quality evaluation method, human eyes do not directly watch the disparity map, and the evaluation of the depth perception of the stereo image by the quality of the disparity map is not very accurate. Therefore, how to effectively simulate the binocular stereo perception process in the stereo image quality evaluation process and how to analyze the influence mechanism of different distortion types on the stereo perception quality so that the evaluation result can more objectively reflect the human visual system are problems to be researched and solved in the process of objectively evaluating the stereo image quality.

Disclosure of Invention

The invention aims to provide a three-dimensional image quality objective evaluation method based on structure texture separation, which can effectively improve the correlation between objective evaluation results and subjective perception.

The technical scheme adopted by the invention for solving the technical problems is as follows: a three-dimensional image quality objective evaluation method based on structure texture separation is characterized in that the processing process is as follows:

firstly, respectively implementing structure texture separation on a left viewpoint image and a right viewpoint image of an original undistorted stereo image and a left viewpoint image and a right viewpoint image of a distorted stereo image to be evaluated to obtain respective structure images and texture images;

secondly, obtaining an objective evaluation prediction value of the image quality of the structural image of the left viewpoint image of the distorted stereo image to be evaluated by calculating the gradient similarity between each pixel point in the structural image of the left viewpoint image of the original undistorted stereo image and the corresponding pixel point in the structural image of the left viewpoint image of the distorted stereo image to be evaluated; similarly, obtaining an objective evaluation prediction value of the image quality of the structural image of the right viewpoint image of the distorted stereo image to be evaluated by calculating the gradient similarity between each pixel point in the structural image of the right viewpoint image of the original undistorted stereo image and the corresponding pixel point in the structural image of the right viewpoint image of the distorted stereo image to be evaluated;

secondly, obtaining an objective image quality evaluation prediction value of the texture image of the left viewpoint image of the distorted stereo image to be evaluated by calculating the structural similarity between each subblock with the size of 8 multiplied by 8 in the texture image of the left viewpoint image of the original undistorted stereo image and the subblock with the corresponding size of 8 multiplied by 8 in the texture image of the left viewpoint image of the distorted stereo image to be evaluated; similarly, obtaining an objective evaluation prediction value of the image quality of the texture image of the right viewpoint image of the distorted stereo image to be evaluated by calculating the structural similarity between each subblock with the size of 8 × 8 in the texture image of the right viewpoint image of the original undistorted stereo image and the subblock with the corresponding size of 8 × 8 in the texture image of the right viewpoint image of the distorted stereo image to be evaluated;

thirdly, fusing the image quality objective evaluation predicted values of the structural images of the left viewpoint image and the right viewpoint image of the distorted three-dimensional image to be evaluated to obtain the image quality objective evaluation predicted value of the structural image of the distorted three-dimensional image to be evaluated; similarly, fusing the image quality objective evaluation predicted values of the texture images of the left viewpoint image and the right viewpoint image of the distorted three-dimensional image to be evaluated to obtain the image quality objective evaluation predicted value of the texture image of the distorted three-dimensional image to be evaluated;

and finally, fusing the image quality objective evaluation predicted value of the structural image and the texture image of the distorted three-dimensional image to be evaluated to obtain the image quality objective evaluation predicted value of the distorted three-dimensional image to be evaluated.

The objective evaluation method for the quality of the stereo image based on the structure texture separation specifically comprises the following steps:

making S_orgRepresenting the original undistorted stereo image, let S_disA stereoscopic image representing distortion to be evaluated, S_orgIs noted as { L_org(x, y) }, adding S_orgIs noted as { R_org(x, y) }, adding S_disIs noted as { L_dis(x, y) }, adding S_disIs noted as { R_dis(x, y) }, wherein (x, y) denotes a coordinate position of a pixel point in the left viewpoint image and the right viewpoint image, x is 1. ltoreq. x.ltoreq.W, y is 1. ltoreq. y.ltoreq.H, W denotes a width of the left viewpoint image and the right viewpoint image, H denotes a height of the left viewpoint image and the right viewpoint image, L is L_org(x, y) represents { L }_orgThe coordinate position in (x, y) } is the pixel value of the pixel point with (x, y), R_org(x, y) represents { R_orgThe pixel value L of the pixel point with the coordinate position (x, y) in (x, y) } is_dis(x, y) represents { L }_disThe coordinate position in (x, y) } is the pixel value of the pixel point with (x, y), R_dis(x, y) represents { R_disThe coordinate position in (x, y) is the pixel value of the pixel point of (x, y);

② are respectively paired with { L_org(x,y)}、{R_org(x,y)}、{L_dis(x, y) } and { R }_dis(x, y) } structural texture separation to obtain { L }_org(x,y)}、{R_org(x,y)}、{L_dis(x, y) } and { R }_dis(x, y) } respective structural and texture images, will { L_org(x, y) } structural and texture image correspondences are notedAnd

will { R_org(x, y) } structural and texture image correspondences are noted

And

will { L_dis(x, y) } structural and texture image correspondences are noted

And

let the structural image and texture image of { Rdis (x, y) } correspond as

And

wherein,

to represent

The pixel value of the pixel point with the middle coordinate position of (x, y)，

To represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to representThe middle coordinate position is the pixel value of the pixel point of (x, y);

calculating

Each pixel point in (1)

The gradient similarity between the corresponding pixel points will beThe pixel point with the (x, y) coordinate position and

the gradient similarity between pixel points with (x, y) as the middle coordinate position is recorded as

Wherein,

m_{L, org}^{str} (x, y) = \sqrt{{({gx}_{L, org}^{str} (x, y))}^{2} + {({gy}_{L, org}^{str} (x, y))}^{2}},

m_{L, dis}^{str} (x, y) = \sqrt{{({gx}_{L, dis}^{str} (x, y))}^{2} + {({gy}_{L, dis}^{str} (x, y))}^{2}},

to represent

The horizontal direction gradient of the pixel point with the middle coordinate position (x, y),to represent

The vertical direction gradient of the pixel point with the middle coordinate position (x, y),

to represent

The horizontal direction gradient of the pixel point with the middle coordinate position (x, y),

to represent

Gradient of pixel points with (x, y) as middle coordinate position in vertical direction, C₁Is a control parameter; then according to

Each pixel point in (1)

Calculating the gradient similarity between corresponding pixel points

The predicted value of objective evaluation of image quality is recorded as

Also, calculate

Each pixel point in (1)

Property of will

The pixel point with the (x, y) coordinate position and

Wherein,

m_{R, org}^{str} (x, y) = \sqrt{{({gx}_{R, org}^{str} (x, y))}^{2} + {({gy}_{R, org}^{str} (x, y))}^{2}},

m_{R, dis}^{str} (x, y) = \sqrt{{({gx}_{R, dis}^{str} (x, y))}^{2} + {({gy}_{R, dis}^{str} (x, y))}^{2}},

to represent

to represent

to represent

Each pixel point in (1)

Calculating the gradient similarity between corresponding pixel points

The predicted value of objective evaluation of image quality is recorded as

Fourthly, passing throughObtaining

Each of the sub-blocks having a size of 8 x 8 and

calculating the structural similarity between the sub-blocks with the corresponding size of 8 multiplied by 8

The predicted value of objective evaluation of image quality is recorded as

Also, by obtaining

Each of the sub-blocks having a size of 8 x 8 and

calculating the structural similarity between the sub-blocks with the corresponding size of 8 multiplied by 8The predicted value of objective evaluation of image quality is recorded as

Fifthly, to

And

carrying out fusion to obtain S_disThe predicted value of the objective evaluation of the image quality of the structural image is marked as Q_str，

Wherein, w_sTo represent

And

the weight proportion of (2);

also, for

And

carrying out fusion to obtain S_disThe predicted value of the texture image is recorded as Q_tex，

Wherein, w_tTo represent

And

the weight proportion of (2);

sixthly to Q_strAnd Q_texCarrying out fusion to obtain S_disThe predicted value of the objective evaluation of image quality is expressed as Q, Q = w × Q_str+(1-w)×Q_texWherein w represents Q_strAnd S_disThe weight ratio of (2).

In the step II, { L_org(x, y) } structural image

And texture images

The acquisition process comprises the following steps:

② 1a, will { L_orgDefining the current pixel point to be processed in (x, y) } as the current pixel pointA front pixel point;

2a, setting the current pixel point at { L_orgThe coordinate position in (x, y) is recorded as p, each pixel point except the current pixel point in a 21 × 21 neighborhood window with the current pixel point as the center is defined as a neighborhood pixel point, a block formed by a 9 × 9 neighborhood window with the current pixel point as the center is defined as a current sub-block, and the current sub-block is recorded as

Defining blocks formed by 9 × 9 neighborhood windows with each neighborhood pixel point in 21 × 21 neighborhood window with the current pixel point as the center as neighborhood sub-blocks, and defining the blocks in the 21 × 21 neighborhood window with the current pixel point as the center and in { L } neighborhood sub-blocks_org(x, y) in the (x, y) } neighborhood sub-block formed by 9 multiplied by 9 neighborhood window with neighborhood pixel point with coordinate position q as centerWherein p ∈ Ω, q ∈ Ω, where Ω denotes { L ∈ Ω_org(x, y) } set of coordinate positions of all pixel points, (x)₂,y₂) Representing a current sub-block

The pixel point in (1) is in the current sub-block

Coordinate position of (1) x₂≤9,1≤y₂≤9，

Representing a current sub-block

The coordinate position is (x)₂,y₂) (x) pixel value of the pixel point of (c)₃,y₃) To represent

Is at

Coordinate position of (1) x₃≤9,1≤y₃≤9，

To represent

The middle coordinate position is (x)₃,y₃) The pixel value of the pixel point of (1);

in the step 2a, for any neighborhood pixel point and any pixel point in the current sub-block, the pixel point is assumed to be in the { L [ ]_orgThe coordinate position in (x, y) } is (x, y), if x is<1 and y is more than or equal to 1 and less than or equal to H, then { L ≦ H_orgAssigning the pixel value of the pixel point with the coordinate position (1, y) in the (x, y) } to the pixel point; if x>W is 1. ltoreq. y.ltoreq.H, then { L_orgAssigning the pixel value of the pixel point with the coordinate position (W, y) in the (x, y) } to the pixel point; if x is 1. ltoreq. W and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (x,1) in the (x, y) } to the pixel point; if x is 1. ltoreq. W and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (x, H) in the (x, y) } to the pixel point; if x<1 and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (1,1) in the (x, y) } to the pixel point; if x>W and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (W,1) in the (x, y) } to the pixel point; if x<1 and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (1, H) in the (x, y) } to the pixel point; if x>W and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (W, H) in the (x, y) } to the pixel point;

② 3a, obtaining the current sub-block

The feature vector of each pixel point in (1) is used for converting the current sub-block into the current sub-block

The feature vector of the pixel point with the middle coordinate position (x2, y2) is recorded as

<math> <mrow> <msubsup> <mi>X</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mo>[</mo> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>&PartialD;</mo> <mi>I</mi> </mrow> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>&PartialD;</mo> <mi>I</mi> </mrow> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <msup> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <msup> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>|</mo> <mo>,</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>]</mo> </mrow> </math>

Wherein

has a dimension of 7, the symbol "[ 2 ]]"is a vector representation symbol, the symbol" | "is an absolute value symbol,

representing a current sub-block

The middle coordinate position is (x)₂,y₂) The density value of the pixel point of (a),

is composed of

The first partial derivative in the horizontal direction,

is composed of

The first partial derivative in the vertical direction,

is composed of

The second partial derivative in the horizontal direction,

is composed of

Second partial derivatives in the vertical direction;

② 4a, according to the current sub-block

Calculating the current sub-block according to the feature vector of each pixel point

Covariance matrix of (2), as

Wherein,has a dimension of 7 x 7,representing a current sub-blockThe mean vector of the feature vectors of all the pixel points in (1),

is composed of

The transposed vector of (1);

② 5a, for the current sub-block

Covariance matrix of

The Cholesky decomposition is carried out and,obtaining the current sub-block

Sigma feature set of (D), noted

Wherein L is^TIs a transposed matrix of the L and,

has a dimension of 7X 15, symbol "[ 2 ]]"is a vector representing a symbol where 1. ltoreq. i'. ltoreq.7, L⁽¹⁾1 st column vector representing L, L^(i')I' th column vector representing L, L⁽⁷⁾A 7 th column vector representing L;

secondly, 6a, adopting the same operation as the operation from the step II to obtain the Sigma characteristic set of the neighborhood sub-block formed by a 9 multiplied by 9 neighborhood window taking each neighborhood pixel point as the center, and leading the Sigma characteristic set to be matched with the neighborhood sub-blockSigma feature set of

Has a dimension of 7 × 15;

7a, according to the current sub-blockSigma feature set ofAnd a Sigma characteristic set of a neighborhood sub-block consisting of a 9 multiplied by 9 neighborhood window with each neighborhood pixel point as a center, and acquiring the structural information of the current pixel point and recording the structural information as the structural information

<math> <mrow> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>,</mo> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> <mo>×</mo> <msub> <mi>L</mi> <mi>org</mi> </msub> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math>

Wherein, N' (p) indicates that all neighborhood pixels in a 21 × 21 neighborhood window centered on the current pixel in { Lorg (x, y) } are in { L }_orgA set of coordinate positions in (x, y) }, exp () represents an exponential function with e as the base, e =2.71828183, σ represents the standard deviation of a gaussian function, the symbol "| | |" is the euclidean distance calculation symbol, L_org(q) represents { L }_org(x, y) } the pixel value of a pixel point with the coordinate position of q;

② 8a, according to the structure information of current pixel point

Obtaining the texture information of the current pixel point and recording the texture information as

I_{L, org}^{tex} (p), I_{L, org}^{tex} (p) = L_{org} (p) - I_{L, org}^{str} (p),

Wherein L is_org(p) representing a pixel value of a current pixel point;

② 9a, will { L_orgTaking the next pixel point to be processed in (x, y) as the current pixel point, and then returning to the step 2a to continue executing until the pixel point is L_orgAll pixel points in (x, y) are processed to obtain { L }_orgThe structural information and the texture information of each pixel point in (x, y) } are expressed by { L }_orgThe structural information of all the pixel points in (x, y) } constitutes { L }_org(x, y) } structural image, noted

From { L_orgTexture information of all pixel points in (x, y) } constitutes { L }_org(x, y) } texture image, noted

Acquiring { L by adopting steps from 1a to 9a_org(x, y) } structural image

And texture imagesSame operation, get { R_org(x, y) } structural image

And texture images

{L_dis(x, y) } structural imageAnd texture images

{R_dis(x, y) } structural image

And texture images

In the step (iv)

Objectively evaluating the predicted value of image quality

The acquisition process comprises the following steps:

fourthly-1 a, respectively

And

is divided into

Sub-blocks of size 8 × 8, which do not overlap with each other, are formed

Defining the current kth sub-block to be processed as the current first sub-blockThe current, to be processed, k sub-block is defined as the current, second sub-block, wherein,

k has an initial value of 1;

fourthly-2 a, recording the current first sub-block as

Record the current second sub-block as

Wherein (x)₄,y₄) To represent

And

x is more than or equal to 1₄≤8,1≤y₄≤8，

To represent

The middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (a),to represent

The middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (1);

fourthly-3 a, calculating the current first sub-blockMean and standard deviation of (D), corresponding notation

And

likewise, the current second sub-block is calculated

Mean and standard deviation of (D), corresponding notation

And

fourthly-4 a, calculating the current first sub-blockWith the current second sub-block

Structural similarity between them, is recorded as

Wherein, C₂Is a control parameter;

(iv) 5a, let k = k +1, will

Wait for next oneThe sub-block processed is taken as the current first sub-block and will be

Taking the next sub-block to be processed as the current second sub-block, and then returning to the step (2 a) to continue execution until the next sub-block to be processed is reached

And

all the sub-blocks in the Chinese herbal medicine are processed to obtain the Chinese herbal medicine

Each sub-block of (1) and

wherein "=" in k = k +1 is an assignment symbol;

fourthly-6 a, according to

Each sub-block of (1) and

structural similarity between corresponding sub-blocks in the sequence, calculating

The predicted value of objective evaluation of image quality is recorded as

In the step (iv)

Objectively evaluating the predicted value of image quality

The acquisition process comprises the following steps:

fourthly-1 b, respectively mixing

Andis divided into

Sub-blocks of size 8 × 8, which do not overlap with each other, are formed

Defining the current kth sub-block to be processed as the current first sub-block

The current, to be processed, k sub-block is defined as the current, second sub-block, wherein,

k has an initial value of 1;

fourthly-2 b, recording the current first sub-block as

Record the current second sub-block as

Wherein (x)₄,y₄) To representAnd

x is more than or equal to 1₄≤8,1≤y₄≤8，

To represent

The middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (a),

to representThe middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (1);

fourthly-3 b, calculating the current first sub-block

Mean and standard deviation of (D), corresponding notation

And

likewise, the current second sub-block is calculated

Mean and standard deviation of (D), corresponding notation

And

fourthly-4 b, calculating the current first sub-block

With the current second sub-block

Structural similarity between them, is recorded as

Wherein, C₂Is a control parameter;

(iv) 5b, let k = k +1, will

Taking the next sub-block to be processed as the current first sub-block

Taking the next sub-block to be processed as the current second sub-block, and then returning to the step (2 b) to continue execution until the next sub-block to be processed is reached

And

Each sub-block of (1) and

wherein "=" in k = k +1 is an assignment symbol;

fourthly-6 b, according to

Each sub-block of (1) and

The predicted value of objective evaluation of image quality is recorded as

Compared with the prior art, the invention has the advantages that:

1) the method of the invention considers that the distortion can cause the loss of image structure or texture information, so that the distorted stereo image is separated into the structure image and the texture image, and different parameters are adopted to respectively fuse the image quality objective evaluation predicted values of the structure image and the texture image of the left viewpoint image and the right viewpoint image, thus better reflecting the quality change condition of the stereo image and leading the evaluation result to be more in line with the human visual system.

2) The method adopts the gradient similarity to evaluate the structural image and the structural similarity to evaluate the texture image, so that the influence of the loss of structural and texture information on the image quality can be well represented, and the correlation between objective evaluation results and subjective perception can be effectively improved.

Drawings

FIG. 1 is a block diagram of an overall implementation of the method of the present invention;

FIG. 2 is a scatter diagram of the difference between the objective evaluation predicted value of image quality and the mean subjective score of each distorted stereo image in Ningbo university stereo image library obtained by the method of the present invention;

fig. 3 is a scatter diagram of the difference between the objective evaluation prediction value of image quality and the average subjective score of each distorted stereo image in the LIVE stereo image library obtained by the method of the present invention.

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

The invention provides a three-dimensional image quality objective evaluation method based on structure texture separation, the overall implementation block diagram of which is shown in figure 1, and the processing process of the method is as follows:

The invention relates to a three-dimensional image quality objective evaluation method based on structure texture separation, which specifically comprises the following steps:

making S_orgRepresenting the original undistorted stereo image, let S_disA stereoscopic image representing distortion to be evaluated, S_orgIs noted as { L_org(x, y) }, adding S_orgIs noted as { R_org(x, y) }, adding S_disIs noted as { L_dis(x, y) }, adding S_disIs noted as { R_dis(x, y) }, wherein (x, y) denotes a coordinate position of a pixel point in the left viewpoint image and the right viewpoint image, x is 1. ltoreq. x.ltoreq.W, y is 1. ltoreq. y.ltoreq.H, W denotes a width of the left viewpoint image and the right viewpoint image, H denotes a height of the left viewpoint image and the right viewpoint image, L is L_org(x, y) represents { L }_orgThe coordinate position in (x, y) } is the pixel value of the pixel point with (x, y), R_org(x, y) represents { R_org(x, y) } position of coordinate in (x, y) } spacePixel value of pixel point of (x, y), L_dis(x, y) represents { L }_disThe coordinate position in (x, y) } is the pixel value of the pixel point with (x, y), R_dis(x, y) represents { R_disAnd the coordinate position in the (x, y) is the pixel value of the pixel point of (x, y).

Here, the Ningbo university stereo image library and the LIVE stereo image library are used to analyze the correlation between the image quality objective evaluation prediction value of the distorted stereo image obtained in the present embodiment and the average subjective score difference value. The Ningbo university stereo image library is composed of 12 undistorted stereo images, 60 distorted stereo images under JPEG compression with different distortion degrees, 60 distorted stereo images under JPEG2000 compression, 60 distorted stereo images under Gaussian blur, 60 distorted stereo images under Gaussian white noise, and 72 distorted stereo images under H.264 coding distortion. The LIVE stereo image library is composed of 20 undistorted stereo images, 80 distorted stereo images under JPEG compression with different distortion degrees, 80 distorted stereo images under JPEG2000 compression, 45 distorted stereo images under gaussian blur, 80 distorted stereo images under gaussian white noise, and 80 distorted stereo images under Fast Fading distortion.

② are respectively paired with { L_org(x,y)}、{R_org(x,y)}、{L_dis(x, y) } and { R }_dis(x, y) } structural texture separation to obtain { L }_org(x,y)}、{R_org(x,y)}、{L_dis(x, y) } and { R }_dis(x, y) } respective structural and texture images, will { L_org(x, y) } structural and texture image correspondences are noted

And

will { R_org(x, y) } structural and texture image correspondences are noted

And

will { L_dis(x, y) } structural and texture image correspondences are noted

And

will { R_dis(x, y) } structural and texture image correspondences are noted

And

wherein,

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),to represent

The middle coordinate position is the pixel value of the pixel point of (x, y).

In this embodiment, step two is { L_org(x, y) } structural imageAnd texture images

The acquisition process comprises the following steps:

② 1a, will { L_orgAnd (x, y) defining the current pixel point to be processed as the current pixel point.

2a, setting the current pixel point at { L_org(x,y) is recorded as p, each pixel point except the current pixel point in a 21 × 21 neighborhood window with the current pixel point as the center is defined as a neighborhood pixel point, a block formed by a 9 × 9 neighborhood window with the current pixel point as the center is defined as a current sub-block, and the current sub-block is recorded as

Defining blocks formed by 9 × 9 neighborhood windows with each neighborhood pixel point in 21 × 21 neighborhood window with the current pixel point as the center as neighborhood sub-blocks, and defining the blocks in the 21 × 21 neighborhood window with the current pixel point as the center and in { L } neighborhood sub-blocks_org(x, y) in the (x, y) } neighborhood sub-block formed by 9 multiplied by 9 neighborhood window with neighborhood pixel point with coordinate position q as center

Wherein p ∈ Ω, q ∈ Ω, where Ω denotes { L ∈ Ω_org(x, y) } set of coordinate positions of all pixel points, (x)₂,y₂) Representing a current sub-blockThe pixel point in (1) is in the current sub-block

Coordinate position of (1) x₂≤9,1≤y₂≤9，

Representing a current sub-block

The middle coordinate position is (x)₂,y₂) (x) pixel value of the pixel point of (c)₃,y₃) To representIs atCoordinates of (5)Position, 1. ltoreq. x₃≤9,1≤y₃≤9，To represent

The middle coordinate position is (x)₃,y₃) The pixel value of the pixel point of (1).

In the step 2a, for any pixel point in the current sub-block, the pixel point is assumed to be in { L [ ]_orgThe coordinate position in (x, y) } is (x, y), if x is<1 and y is more than or equal to 1 and less than or equal to H, then { L ≦ H_orgAssigning the pixel value of the pixel point with the coordinate position (1, y) in the (x, y) } to the pixel point; if x>W is 1. ltoreq. y.ltoreq.H, then { L_orgAssigning the pixel value of the pixel point with the coordinate position (W, y) in the (x, y) } to the pixel point; if x is 1. ltoreq. W and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (x,1) in the (x, y) } to the pixel point; if x is 1. ltoreq. W and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (x, H) in the (x, y) } to the pixel point; if x<1 and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (1,1) in the (x, y) } to the pixel point; if x>W and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (W,1) in the (x, y) } to the pixel point; if x<1 and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (1, H) in the (x, y) } to the pixel point; if x>W and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (W, H) in the (x, y) } to the pixel point; similarly, for any neighborhood pixel point, the same operation is performed as for any pixel point in the current sub-block, so that the pixel value of the neighborhood pixel point beyond the image boundary is replaced by the pixel value of the nearest boundary pixel point. That is, in the above step 2a, if the coordinate position of a certain pixel point in the block formed by the 9 × 9 neighborhood window with the current pixel point as the center exceeds { L }_org(x, y) } the pixel value of the pixel is the image of the nearest boundary pixelReplacing the prime value; if the coordinate position of the neighborhood pixel point in the 21 multiplied by 21 neighborhood window taking the current pixel point as the center exceeds { L }_org(x, y) } the pixel value of the neighborhood pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of a pixel point in a block formed by a 9 x 9 neighborhood window with each neighborhood pixel point in the 21 x 21 neighborhood window with the current pixel point as the center exceeds { L }_org(x, y) }, the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel.

② 3a, obtaining the current sub-block

Wherein,

has a dimension of 7, the symbol "[ 2 ]]"is a vector representation symbol, the symbol" | "is an absolute value symbol,representing a current sub-block

The middle coordinate position is (x)₂,y₂) The density value of the pixel point of (a),is composed of

The first partial derivative in the horizontal direction,is composed of

In the vertical directionThe first-order partial derivative of (a) is,

is composed ofThe second partial derivative in the horizontal direction,

is composed ofSecond partial derivative in vertical direction.

② 4a, according to the current sub-block

Covariance matrix of (2), as

Wherein,

has a dimension of 7 x 7,

representing a current sub-block

The mean vector of the feature vectors of all the pixel points in (1),

is composed of

The transposed vector of (1).

② 5a, for the current sub-block

Covariance matrix of

The Cholesky decomposition is carried out and,

obtaining the current sub-block

Sigma feature set of (D), noted

Wherein L is^TIs a transposed matrix of the L and,

has a dimension of 7X 15, symbol "[ 2 ]]"is a vector representing a symbol where 1. ltoreq. i'. ltoreq.7, L⁽¹⁾1 st column vector representing L, L^(i')I' th column vector representing L, L⁽⁷⁾The 7 th column vector of L is represented.

Secondly, 6a, adopting the same operation as the operation from the step II to obtain the Sigma characteristic set of the neighborhood sub-block formed by a 9 multiplied by 9 neighborhood window taking each neighborhood pixel point as the center, and leading the Sigma characteristic set to be matched with the neighborhood sub-block

Sigma feature set of

Has a dimension of 7 × 15.

7a, according to the current sub-blockSigma feature set of

And a Sigma characteristic set of a neighborhood sub-block consisting of a 9 multiplied by 9 neighborhood window with each neighborhood pixel point as a center, and acquiring the structural information of the current pixel point and recording the structural information as the structural information

Wherein N' (p) represents { L_org21 × 21 neighbors centered on the current pixel in (x, y) } methodAll neighborhood pixels in the domain window are in { L_orgSet of coordinate positions in (x, y) }, exp () represents an exponential function based on e, e =2.71828183, σ represents the standard deviation of the gaussian function, in this example, σ =0.06, the symbol "| is the euclidean distance calculation symbol, L_org(q) represents { L }_orgAnd (x, y) } the pixel value of the pixel point with the coordinate position of q.

② 8a, according to the structure information of current pixel point

Wherein L is_org(p) represents the pixel value of the current pixel point.

Acquiring { L by adopting steps from 1a to 9a_org(x, y) } structural image

And texture images

Same operation, get { R_org(x, y) } structural image

And texture images

{L_dis(x, y) } structural image

And texture images

{R_dis(x, y) } structural image

And texture images

Namely: step two, { R_org(x, y) } structural imageAnd texture images

The acquisition process comprises the following steps:

2- (1 b) will be { R_orgAnd (x, y) defining the current pixel point to be processed as the current pixel point.

2b, setting the current pixel point at { R_orgThe coordinate position in (x, y) is recorded as p, each pixel point except the current pixel point in a 21 × 21 neighborhood window with the current pixel point as the center is defined as a neighborhood pixel point, a block formed by a 9 × 9 neighborhood window with the current pixel point as the center is defined as a current sub-block, and the current sub-block is recorded as

Each of 21 x 21 neighborhood windows centered on the current pixel pointThe blocks formed by the 9 multiplied by 9 neighborhood windows with the neighborhood pixel point as the center are all defined as neighborhood sub-blocks, and are in the 21 multiplied by 21 neighborhood window with the current pixel point as the center and are in the { R_org(x, y) in the (x, y) } neighborhood sub-block formed by 9 multiplied by 9 neighborhood window with neighborhood pixel point with coordinate position q as center

Wherein p ∈ Ω, q ∈ Ω, where Ω denotes { R ∈ Ω_org(x, y) } set of coordinate positions of all pixel points, (x)₂,y₂) Representing a current sub-block

The pixel point in (1) is in the current sub-block

Coordinate position of (1) x₂≤9,1≤y₂≤9，

Representing a current sub-block

The middle coordinate position is (x)₂,y₂) (x) pixel value of the pixel point of (c)₃,y₃) To represent

Is atCoordinate position of (1) x₃≤9,1≤y₃≤9，

To represent

In the second step-2 b, if the coordinate position of a certain pixel point in the block formed by the 9 multiplied by 9 neighborhood window taking the current pixel point as the center exceeds { R }_org(x, y) } the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of the neighborhood pixel point in the 21 multiplied by 21 neighborhood window taking the current pixel point as the center exceeds { R }_org(x, y) } the pixel value of the neighborhood pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of a pixel point in a block formed by a 9 x 9 neighborhood window with each neighborhood pixel point in the 21 x 21 neighborhood window with the current pixel point as the center exceeds { R }_org(x, y) }, the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel.

② 3b, obtaining the current sub-block

Wherein,

The density value of the pixel point with the middle coordinate position of (x2, y2),

is composed of

The first partial derivative in the horizontal direction,

is composed of

The first partial derivative in the vertical direction,

is composed of

The second partial derivative in the horizontal direction,is composed of

Second partial derivative in vertical direction.

2-4 b, according to the current sub-block

Covariance matrix of (2), as

Wherein,

has a dimension of 7 x 7,representing a current sub-block

The mean vector of the feature vectors of all the pixel points in (1),

is composed of

The transposed vector of (1).

② 5b, for the current sub-block

Covariance matrix of

The Cholesky decomposition is carried out and,

obtaining the current sub-blockSigma feature set of (D), noted

Wherein L is^TIs a transposed matrix of the L and,

6b, adopting the same operation as the steps from 3b to 5b to obtain the Sigma characteristic set of the neighborhood sub-block formed by a 9 multiplied by 9 neighborhood window with each neighborhood pixel point as the center, and leading the Sigma characteristic set to be used for solving the problem that the Sigma characteristic set of the neighborhood sub-block is not suitable for the next step

Sigma feature set of

Has a dimension of 7 × 15.

7b, according to the current sub-block

Sigma feature set of

<math> <mrow> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>,</mo> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> <mo>×</mo> <msub> <mi>L</mi> <mi>org</mi> </msub> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>org</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math>

Wherein N' (p) represents { R_orgAll neighborhood pixels in the 21 x 21 neighborhood window centered on the current pixel in (x, y) are in { R }_orgSet of coordinate positions in (x, y) }, exp () represents an exponential function with e as the base, e =2.71828183, σ represents the standard deviation of the gaussian function, in this embodiment, σ =0.06, the symbol "| | |" is the euclidean distance calculation symbol, R_org(q) represents { R_orgAnd (x, y) } the pixel value of the pixel point with the coordinate position of q.

② 8b, according to the structure information of current pixel point

Wherein R is_org(p) represents the pixel value of the current pixel point.

② 9b, will { R_orgTaking the next pixel point to be processed in (x, y) as the current pixel point, and then returning to the step 2b to continue executing until the pixel point is { R }_orgAll pixel points in (x, y) are processed to obtain { R }_orgThe structural information and the texture information of each pixel point in (x, y) } are represented by { R }_orgThe structural information of all pixel points in (x, y) constitutes R_org(x, y) } structural image, noted

From { R_orgTexture information of all pixel points in (x, y) } constitutes { R }_org(x, y) } texture image, noted

Step two, { L_dis(x, y) } structural imageAnd texture imagesThe acquisition process comprises the following steps:

② 1c, will { L_disAnd (x, y) defining the current pixel point to be processed as the current pixel point.

2c, setting the current pixel point at { L_disThe coordinate position in (x, y) is recorded as p, each pixel point except the current pixel point in a 21 × 21 neighborhood window with the current pixel point as the center is defined as a neighborhood pixel point, a block formed by a 9 × 9 neighborhood window with the current pixel point as the center is defined as a current sub-block, and the current sub-block is recorded as

Defining blocks formed by 9 × 9 neighborhood windows with each neighborhood pixel point in 21 × 21 neighborhood window with the current pixel point as the center as neighborhood sub-blocks, and defining the blocks in the 21 × 21 neighborhood window with the current pixel point as the center and in { L } neighborhood sub-blocks_dis(x, y) in the (x, y) } neighborhood sub-block formed by 9 multiplied by 9 neighborhood window with neighborhood pixel point with coordinate position q as center

Wherein p ∈ Ω, q ∈ Ω, where Ω denotes { L ∈ Ω_dis(x, y) } set of coordinate positions of all pixel points, (x)₂,y₂) Representing a current sub-block

The pixel point in (1) is in the current sub-block

Coordinate position of (1) x₂≤9,1≤y₂≤9，

Representing a current sub-block

Is at

Coordinate position of (1) x₃≤9,1≤y₃≤9，To represent

In the second step 2c, if the coordinate position of a certain pixel point in the block formed by the 9 multiplied by 9 neighborhood window taking the current pixel point as the center exceeds { L }_dis(x, y) } the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of the neighborhood pixel point in the 21 multiplied by 21 neighborhood window taking the current pixel point as the center exceeds { L }_dis(x, y) } the pixel value of the neighborhood pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of a pixel point in a block formed by a 9 x 9 neighborhood window with each neighborhood pixel point in the 21 x 21 neighborhood window with the current pixel point as the center exceeds { L }_dis(x, y) }, the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel.

② 3c, obtaining the current sub-block

The middle coordinate position is (x)₂,y₂) The feature vector of the pixel point is recorded as

<math> <mrow> <msubsup> <mi>X</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mo>[</mo> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>&PartialD;</mo> <mi>I</mi> </mrow> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>&PartialD;</mo> <mi>I</mi> </mrow> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <msup> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <msup> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>|</mo> <mo>,</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>]</mo> </mrow> </math>

Wherein

representing a current sub-block

is composed ofThe first partial derivative in the horizontal direction,

is composed ofThe first partial derivative in the vertical direction,

is composed of

The second partial derivative in the horizontal direction,

is composed of

Second partial derivative in vertical direction.

2-4 c, according to the current sub-block

Covariance matrix of (2), as

Wherein,

has a dimension of 7 x 7,representing a current sub-block

The mean vector of the feature vectors of all the pixel points in (1),

is composed ofThe transposed vector of (1).

② 5c, for the current sub-block

Covariance matrix ofThe Cholesky decomposition is carried out and,

obtaining the current sub-block

Sigma ofCollect a collection, mark as

Wherein L is^TIs a transposed matrix of the L and,

6c, adopting the same operation as the steps from 3c to 5c to obtain the Sigma characteristic set of the neighborhood sub-block formed by a 9 multiplied by 9 neighborhood window with each neighborhood pixel point as the center, and leading the Sigma characteristic set to be used for solving the problem that the Sigma characteristic set of the neighborhood sub-block is not suitable for the next step

Sigma feature set of

Has a dimension of 7 × 15.

7c, according to the current sub-block

Sigma feature set of

<math> <mrow> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>,</mo> <msubsup> <mi>I</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> <mo>×</mo> <msub> <mi>L</mi> <mi>dis</mi> </msub> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>L</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math>

Wherein N' (p) represents { L_disAll neighborhood pixels in the 21 x 21 neighborhood window centered on the current pixel in (x, y) } are in { L }_disSet of coordinate positions in (x, y) }, exp () represents an exponential function with e as the base, e =2.71828183, σ represents the standard deviation of the gaussian function, in this embodiment, σ =0.06, the symbol "| | |" is the euclidean distance calculation symbol, L_dis(q) represents { L }_disAnd (x, y) } the pixel value of the pixel point with the coordinate position of q.

② 8c, according to the structure information of current pixel point

I_{L, dis}^{tex} (p), I_{L, dis}^{tex} (p) = L_{dis} (p) - I_{L, dis}^{str} (p),

Wherein L is_dis(p) represents the pixel value of the current pixel point.

② 9c, will { L_disTaking the next pixel point to be processed in (x, y) as the current pixel point, and then returning to the step 2c to continue executing until the pixel point is L_disAll pixel points in (x, y) are processed to obtain { L }_disThe structural information and the texture information of each pixel point in (x, y) } are expressed by { L }_disThe structural information of all the pixel points in (x, y) } constitutes { L }_dis(x, y) } structural image, noted

From { L_disTexture information of all pixel points in (x, y) } constitutes { L }_dis(x, y) } texture image, noted

Step two, { R_dis(x, y) } structural image

And texture images

The acquisition process comprises the following steps:

② 1d, mixing { R_disAnd (x, y) defining the current pixel point to be processed as the current pixel point.

2d, setting the current pixel point at { R_disThe coordinate position in (x, y) is recorded as p, each pixel point except the current pixel point in a 21 × 21 neighborhood window with the current pixel point as the center is defined as a neighborhood pixel point, a block formed by a 9 × 9 neighborhood window with the current pixel point as the center is defined as a current sub-block, and the current sub-block is recorded as

Defining blocks formed by 9 × 9 neighborhood windows with each neighborhood pixel point in 21 × 21 neighborhood window with the current pixel point as the center as neighborhood sub-blocks, and defining the blocks in the 21 × 21 neighborhood window with the current pixel point as the center and in the { R } neighborhood sub-blocks_dis(x, y) in the (x, y) } neighborhood sub-block formed by 9 multiplied by 9 neighborhood window with neighborhood pixel point with coordinate position q as center

Wherein p ∈ Ω, q ∈ Ω, where Ω denotes { R ∈ Ω_dis(x, y) } set of coordinate positions of all pixel points, (x)₂,y₂) Representing a current sub-block

The pixel point in (1) is in the current sub-blockCoordinate position of (1) x₂≤9,1≤y₂≤9，Representing a current sub-block

Is at

Coordinate position of (1) x₃≤9,1≤y₃≤9，To represent

In the second step 2d, if the coordinate position of a certain pixel point in the block formed by the 9 multiplied by 9 neighborhood window taking the current pixel point as the center exceeds { R }_dis(x, y) } the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of the neighborhood pixel point in the 21 multiplied by 21 neighborhood window taking the current pixel point as the center exceeds { R }_dis(x, y) } the pixel value of the neighborhood pixel is replaced by the pixel value of the nearest boundary pixel; if the coordinate position of a pixel point in a block formed by a 9 x 9 neighborhood window with each neighborhood pixel point in the 21 x 21 neighborhood window with the current pixel point as the center exceeds { R }_dis(x, y) }, the pixel value of the pixel is replaced by the pixel value of the nearest boundary pixel.

② 3d, obtaining the current sub-block

<math> <mrow> <msubsup> <mi>X</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>=</mo> <mo>[</mo> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>&PartialD;</mo> <mi>I</mi> </mrow> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msubsup> <mrow> <mo>&PartialD;</mo> <mi>I</mi> </mrow> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <msup> <mrow> <mo>&PartialD;</mo> <mi>x</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>|</mo> <mo>,</mo> <mo>|</mo> <mfrac> <mrow> <msup> <mo>&PartialD;</mo> <mn>2</mn> </msup> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mrow> <mo>(</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>)</mo> </mrow> </mrow> <msup> <mrow> <mo>&PartialD;</mo> <mi>y</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>|</mo> <mo>,</mo> <msub> <mi>x</mi> <mn>2</mn> </msub> <mo>,</mo> <msub> <mi>y</mi> <mn>2</mn> </msub> <mo>]</mo> </mrow> </math>

Wherein

has a dimension of 7, the symbol "[ 2 ]]"is a vector representation symbol, and the symbol" | "is an absolute value symbolThe number of the mobile station is,representing a current sub-block

is composed of

The first partial derivative in the horizontal direction,

is composed of

The first partial derivative in the vertical direction,is composed of

The second partial derivative in the horizontal direction,

is composed of

Second partial derivative in vertical direction.

4d, according to the current sub-block

Covariance matrix of (2), as

Wherein,

has a dimension of 7 x 7,

representing a current sub-blockThe mean vector of the feature vectors of all the pixel points in (1),

is composed ofThe transposed vector of (1).

② 5d, for the current sub-block

Covariance matrix ofThe Cholesky decomposition is carried out and,

obtaining the current sub-block

Sigma feature set of (D), noted

Wherein L is^TIs a transposed matrix of the L and,

6d, adopting the same operation as the steps from 3d to 5d to obtain the Sigma feature set of the neighborhood sub-block formed by a 9 multiplied by 9 neighborhood window with each neighborhood pixel point as the center, and leading the Sigma feature set to be used for solving the problem that the Sigma feature set of the neighborhood sub-block is not suitable for the next step

Sigma feature set of

Has a dimension of 7 × 15.

7d, according to the current sub-block

Sigma feature set ofAnd a Sigma characteristic set of a neighborhood sub-block consisting of a 9 multiplied by 9 neighborhood window with each neighborhood pixel point as a center, and acquiring the structural information of the current pixel point and recording the structural information as the structural information

<math> <mrow> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>,</mo> <msubsup> <mi>I</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>str</mi> </msubsup> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <mfrac> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> <mo>×</mo> <msub> <mi>L</mi> <mi>dis</mi> </msub> <mrow> <mo>(</mo> <mi>q</mi> <mo>)</mo> </mrow> </mrow> <mrow> <munder> <mi>Σ</mi> <mrow> <mi>q</mi> <mo>&Element;</mo> <mi>N</mi> <mo>′</mo> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> </mrow> </munder> <mi>exp</mi> <mrow> <mo>(</mo> <mo>-</mo> <mfrac> <msup> <mrow> <mo>|</mo> <mo>|</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>p</mi> </msubsup> <mo>-</mo> <msubsup> <mi>S</mi> <mrow> <mi>R</mi> <mo>,</mo> <mi>dis</mi> </mrow> <mi>q</mi> </msubsup> <mo>|</mo> <mo>|</mo> </mrow> <mn>2</mn> </msup> <msup> <mrow> <mn>2</mn> <mi>σ</mi> </mrow> <mn>2</mn> </msup> </mfrac> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>,</mo> </mrow> </math>

Wherein N' (p) represents { R_disAll neighborhood pixels in the 21 x 21 neighborhood window centered on the current pixel in (x, y) are in { R }_disSet of coordinate positions in (x, y) }, exp () represents an exponential function with e as the base, e =2.71828183, σ represents the standard deviation of the gaussian function, in this embodiment, σ =0.06, the symbol "| | |" is the euclidean distance calculation symbol, R_dis(q) represents { R_disAnd (x, y) } the pixel value of the pixel point with the coordinate position of q.

② 8d, according to the structure information of current pixel point

I_{R, dis}^{tex} (p), I_{R, dis}^{tex} (p) = R_{dis} (p) - I_{R, dis}^{str} (p),

Wherein R is_dis(p) represents the pixel value of the current pixel point.

② 9d, will { R_disTaking the next pixel point to be processed in (x, y) as the current pixel point, and then returning to the step 2d to continue executing until the pixel point is { R }_disAll pixel points in (x, y) are processed to obtain { R }_disThe structural information and the texture information of each pixel point in (x, y) } are represented by { R }_disThe structural information of all pixel points in (x, y) constitutes R_dis(x, y) } structural image, notedFrom { R_disTexture information of all pixel points in (x, y) } constitutes { R }_dis(x, y) } texture image, noted

Comparing with original image, the structure image separates out detail information such as texture from original image to make structure information more stable, so the invention method calculates

Each pixel point in (1)

Wherein,

m_{L, org}^{str} (x, y) = \sqrt{{({gx}_{L, org}^{str} (x, y))}^{2} + {({gy}_{L, org}^{str} (x, y))}^{2}},

m_{L, dis}^{str} (x, y) = \sqrt{{({gx}_{L, dis}^{str} (x, y))}^{2} + {({gy}_{L, dis}^{str} (x, y))}^{2}},

to represent

to represent

to represent

to represent

Gradient of pixel points with (x, y) as middle coordinate position in vertical direction, C₁For controlling the parameters, in this example C is taken₁= 0.0026; then according to

Each pixel point in (1)

Calculating the gradient similarity between corresponding pixel points

The predicted value of objective evaluation of image quality is recorded as

Also, calculateEach pixel point in (1)

Wherein,

m_{R, org}^{str} (x, y) = \sqrt{{({gx}_{R, org}^{str} (x, y))}^{2} + {({gy}_{R, org}^{str} (x, y))}^{2}},

m_{R, dis}^{str} (x, y) = \sqrt{{({gx}_{R, dis}^{str} (x, y))}^{2} + {({gy}_{R, dis}^{str} (x, y))}^{2}},

to represent

to represent

The vertical direction gradient of the pixel point with the middle coordinate position (x, y),to represent

to represent

Each pixel point in (1)

Calculating the gradient similarity between corresponding pixel points

Image of (2)The predicted value of objective quality evaluation is recorded as

Fourthly, because the information of the mean value and the standard deviation can well evaluate the change of the detail information of the image, the method of the invention obtains the information of the mean value and the standard deviationEach of the sub-blocks having a size of 8 x 8 andcalculating the structural similarity between the sub-blocks with the corresponding size of 8 multiplied by 8

The predicted value of objective evaluation of image quality is recorded as

In this embodiment, step (iv)

Objectively evaluating the predicted value of image quality

The acquisition process comprises the following steps:

fourthly-1 a, respectively

And

is divided into

Sub-blocks of size 8 × 8, which do not overlap with each other, are formedDefining the current kth sub-block to be processed as the current first sub-block

the initial value of k is 1.

Fourthly-2 a, recording the current first sub-block asRecord the current second sub-block as

Wherein (x)₄,y₄) To represent

Andx is more than or equal to 1₄≤8,1≤y₄≤8，

To represent

to representThe middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (1).

Fourthly-3 a, calculating the current first sub-block

Mean and standard deviation of (D), corresponding notation

And

likewise, the current second sub-block is calculated

Mean and standard deviation of (D), corresponding notation

And

fourthly-4 a, calculating the current first sub-block

With the current second sub-block

Structural similarity between them, is recorded as

Wherein, C₂For controlling the parameters, in this example C is taken₂=0.85。

(iv) 5a, let k = k +1, will

Taking the next sub-block to be processed as the current first sub-block

And

Each sub-block of (1) and

wherein "=" in k = k +1 is an assignerNumber (n).

Fourthly-6 a, according to

Each sub-block of (1) and

The predicted value of objective evaluation of image quality is recorded as

Also, by obtaining

Each of the sub-blocks having a size of 8 x 8 and

The predicted value of objective evaluation of image quality is recorded as

In this embodiment, the step (iv)

Objectively evaluating the predicted value of image quality

Has obtainedThe process is as follows:

fourthly-1 b, respectively mixing

And

is divided into

Sub-blocks of size 8 × 8, which do not overlap with each other, are formed

the initial value of k is 1.

Fourthly-2 b, recording the current first sub-block as

Record the current second sub-block as

Wherein (x)₄,y₄) To representAnd

x is more than or equal to 1₄≤8,1≤y₄≤8，

To represent

to represent

The middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (1).

Fourthly-3 b, calculating the current first sub-block

Mean and standard deviation of (D), corresponding notation

And

likewise, the current second sub-block is calculated

Mean and standard deviation of (D), corresponding notation

And

fourthly-4 b, calculating the current first sub-blockWith the current second sub-block

Structural similarity between them, is recorded as

(iv) 5b, let k = k +1, will

Taking the next sub-block to be processed as the current first sub-blockTaking the next sub-block to be processed as the current second sub-block, and then returning to the step (2 b) to continue execution until the next sub-block to be processed is reached

And

Each sub-block of (1) and

wherein "=" in k = k +1 is an assigned symbol.

Fourthly-6 b, according to

Each sub-block of (1) and

The predicted value of objective evaluation of image quality is recorded as

Fifthly, to

And

Wherein, w_sTo represent

And

in this embodiment, for the Ningbo university stereo image library, take w_s= 0.980; for LIVE stereo image library, take w_s=0.629。

Also, forAnd

Wherein, w_tTo represent

And

in this embodiment, for the Ningbo university stereo image library, take w_t= 0.888; for LIVE stereo image library, take w_t=0.503。

Sixthly to Q_strAnd Q_texCarrying out fusion to obtain S_disThe predicted value of the objective evaluation of image quality is expressed as Q, Q = w × Q_str+(1-w)×Q_texWherein w represents Q_strAnd S_disIn this embodiment, for the stereo image library of ningbo university, w =0.882 is taken; for LIVE stereo image library, take w = 0.838.

Here, 4 common objective parameters of the evaluation method for evaluating image quality are used as evaluation indexes, that is, Pearson correlation coefficient (PLCC), Spearman correlation coefficient (SROCC), Kendall correlation coefficient (KROCC), mean square error (RMSE), accuracy of the objective evaluation result of the stereo image in which PLCC and RMSE reflect distortion, and monotonicity of SROCC and KROCC reflects monotonicity thereof under nonlinear regression conditions.

The method is used for calculating the image quality objective evaluation predicted value of each distorted three-dimensional image in the Ningbo university three-dimensional image library and the image quality objective evaluation predicted value of each distorted three-dimensional image in the LIVE three-dimensional image library, and then the average subjective score difference value of each distorted three-dimensional image in the Ningbo university three-dimensional image library and the average subjective score difference value of each distorted three-dimensional image in the LIVE three-dimensional image library are obtained by using the existing subjective evaluation method. The image quality objective evaluation predicted value of the distorted stereo image calculated according to the method is subjected to five-parameter Logistic function nonlinear fitting, and the higher the PLCC, SROCC and KROCC values are, the lower the RMSE value is, the better the correlation between the objective evaluation method and the average subjective score difference is. Tables 1, 2, 3 and 4 show Pearson correlation coefficient, Spearman correlation coefficient, Kendall correlation coefficient and mean square error between the image quality objective evaluation predicted value and the average subjective score difference value of the distorted stereo image obtained by the method of the present invention. As can be seen from tables 1, 2, 3 and 4, the correlation between the final objective evaluation prediction value of image quality of the distorted stereoscopic image obtained by the method of the present invention and the average subjective score difference is very high, which indicates that the objective evaluation result is more consistent with the result of human eye subjective perception, and is sufficient to explain the effectiveness of the method of the present invention.

Fig. 2 shows a scatter diagram of the difference between the objective evaluation predicted value of the image quality of each distorted stereoscopic image in the Ningbo university stereoscopic image library and the average subjective score obtained by the method of the present invention, and fig. 3 shows a scatter diagram of the difference between the objective evaluation predicted value of the image quality of each distorted stereoscopic image in the LIVE stereoscopic image library and the average subjective score obtained by the method of the present invention, wherein the more concentrated the scatter diagram, the better the consistency between the objective evaluation result and the subjective perception is. As can be seen from fig. 2 and 3, the scatter diagram obtained by the method of the present invention is more concentrated, and has a higher degree of matching with the subjective evaluation data.

TABLE 1 Objective evaluation prediction value and average subjective evaluation of image quality of distorted stereoscopic images obtained by the method of the present invention

Pearson correlation coefficient comparison between variance values

TABLE 2 comparison of Spearman correlation coefficients between objective evaluation prediction values of image quality and mean subjective score differences for distorted stereo images obtained by the method of the invention

TABLE 3 Kendall correlation coefficient comparison between the image quality objective evaluation prediction value and the average subjective score difference of the distorted stereo image obtained by the method of the present invention

TABLE 4 mean square error comparison between the predicted value of objective evaluation of image quality and the difference of mean subjective score of distorted stereoscopic images obtained by the method of the present invention

Claims

1. A three-dimensional image quality objective evaluation method based on structure texture separation is characterized in that the processing process is as follows:

2. The objective evaluation method for stereo image quality based on structure texture separation according to claim 1, characterized in that it comprises the following steps:

will { R_org(x, y) } structural and texture image correspondences are noted

Andwill { L_dis(x, y) } structural and texture image correspondences are noted

And

will { R_dis(x, y) } structural image and texture image correspondenceAnd

wherein,to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),to representThe middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y),

to represent

The middle coordinate position is the pixel value of the pixel point of (x, y);

calculating

Each pixel point in (1)

The gradient similarity between the corresponding pixel points will be

The pixel point with the (x, y) coordinate position andthe gradient similarity between pixel points with (x, y) as the middle coordinate position is recorded as

Wherein,

m_{L, org}^{str} (x, y) = \sqrt{{({gx}_{L, org}^{str} (x, y))}^{2} + {({gy}_{L, org}^{str} (x, y))}^{2}}

，

m_{L, dis}^{str} (x, y) = \sqrt{{({gx}_{L, dis}^{str} (x, y))}^{2} + {({gy}_{L, dis}^{str} (x, y))}^{2}},

to represent

to represent

to represent

Each pixel point in (1)

Calculating the gradient similarity between corresponding pixel points

The predicted value of objective evaluation of image quality is recorded as

Also, calculate

Each pixel point in (1)The gradient similarity between the corresponding pixel points will beThe pixel point with the (x, y) coordinate position and

Wherein,

m_{R, org}^{str} (x, y) = \sqrt{{({gx}_{R, org}^{str} (x, y))}^{2} + {({gy}_{R, org}^{str} (x, y))}^{2}}

m_{R, dis}^{str} (x, y) = \sqrt{{({gx}_{R, dis}^{str} (x, y))}^{2} + {({gy}_{R, dis}^{str} (x, y))}^{2}}

to representThe horizontal direction gradient of the pixel point with the middle coordinate position (x, y),

to represent

to represent

to represent

Gradient of pixel points with (x, y) as middle coordinate position in vertical direction, C₁Is a control parameter; then according toEach pixel point in (1)

Calculating the gradient similarity between corresponding pixel pointsThe predicted value of objective evaluation of image quality is recorded as

ObtainingEach of the sub-blocks having a size of 8 x 8 and

The predicted value of objective evaluation of image quality is recorded as

Also, by obtaining

Each of the sub-blocks having a size of 8 x 8 and

The predicted value of objective evaluation of image quality is recorded as

Fifthly, to

And

to carry outFusing to obtain S_disThe predicted value of the objective evaluation of the image quality of the structural image is marked as Q_str，

Wherein, w_sTo representAnd

the weight proportion of (2);

also, for

And

carrying out fusion to obtain S_disThe predicted value of the texture image is recorded as Q_str，

Wherein, w_tTo represent

Andthe weight proportion of (2);

3. The objective evaluation method for stereo image quality based on structure texture separation as claimed in claim 2, wherein in step (ii) { L }, the objective evaluation method for stereo image quality based on structure texture separation is adopted_org(x, y) } structural image

And texture images

The acquisition process comprises the following steps:

② 1a, will { L_orgDefining the current pixel point to be processed in (x, y) as the current pixel point;

Wherein p ∈ Ω, q ∈ Ω, where Ω denotes { L ∈ Ω_org(x, y) } set of coordinate positions of all pixel points, (x)₂,y₂) Representing a current sub-block

The pixel point in (1) is in the current sub-blockCoordinate position of (1) x₂≤9,1≤y₂≤9，

Representing a current sub-block

The middle coordinate position is (x)₂,y₂) (x) pixel value of the pixel point of (c)₃,y₃) To representIs atCoordinate position of (1) x₃≤9,1≤y₃≤9，

To represent

in the step 2a, for any neighborhood pixel point and any pixel point in the current sub-block, the pixel point is assumed to be in the { L [ ]_orgThe coordinate position in (x, y) } is (x, y), if x is<1 and y is more than or equal to 1 and less than or equal to H, then { L ≦ H_orgAssigning the pixel value of the pixel point with the coordinate position (1, y) in the (x, y) } to the pixel point; if x>W is 1. ltoreq. y.ltoreq.H, then { L_orgAssigning the pixel value of the pixel point with the coordinate position (W, y) in the (x, y) } to the pixel point; if x is 1. ltoreq. W and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (x,1) in the (x, y) } to the pixel point; if x is 1. ltoreq. W and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (x, H) in the (x, y) } to the pixel point; if x<1 and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (1,1) in the (x, y) } to the pixel point; if x>W and y<1, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (W,1) in the (x, y) } to the pixel point;if x<1 and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (1, H) in the (x, y) } to the pixel point; if x>W and y>H, then { L }_orgAssigning the pixel value of the pixel point with the coordinate position (W, H) in the (x, y) } to the pixel point;

② 3a, obtaining the current sub-block

Wherein

representing a current sub-block

is composed ofThe first partial derivative in the horizontal direction,

is composed of

The first partial derivative in the vertical direction,is composed ofThe second partial derivative in the horizontal direction,

is composed ofSecond partial derivatives in the vertical direction;

② 4a, according to the current sub-block

Covariance matrix of (2), as

Wherein,

has a dimension of 7 x 7,

is composed of

The transposed vector of (1);

② 5a, for the current sub-block

Covariance matrix ofThe Cholesky decomposition is carried out and,

obtaining the current sub-block

Sigma feature set of (D), noted

Wherein L is^TIs a transposed matrix of the L and,has a dimension of 7X 15, symbol "[ 2 ]]"is a vector representing a symbol where 1. ltoreq. i'. ltoreq.7, L⁽¹⁾1 st column vector representing L, L^(i')I' th column vector representing L, L⁽⁷⁾A 7 th column vector representing L;

Has a dimension of 7 × 15;

7a, according to the current sub-block

Sigma feature set of

Wherein N' (p) represents { L_orgAll neighborhood pixels in the 21 x 21 neighborhood window centered on the current pixel in (x, y) } are in { L }_orgA set of coordinate positions in (x, y) }, exp () represents an exponential function with e as the base, e =2.71828183, σ represents the standard deviation of a gaussian function, the symbol "|" is the euclidean distance calculation symbol, L_org(q) represents { L }_org(x, y) } the pixel value of a pixel point with the coordinate position of q;

② 8a, according to the structure information of current pixel point

I_{L, org}^{tex} (p), I_{L, org}^{tex} (p) = L_{org} (p) - I_{L, org}^{str} (p),

Wherein L is_org(p) representing a pixel value of a current pixel point;

By

Texture information composition of all pixels in (1) { L }_org(x, y) } texture image, noted

Acquiring { L by adopting steps from 1a to 9a_org(x, y) } structural image

And texture imagesSame operation, get { R_org(x, y) } structural imageAnd texture images{L_dis(x, y) } structural image

And texture images

{R_dis(x, y) } structural image

And texture images

4. The objective evaluation method for stereo image quality based on structure texture separation according to claim 2 or 3, characterized in that the step (c) is

Objectively evaluating the predicted value of image quality

The acquisition process comprises the following steps:

fourthly-1 a, respectively

Andis divided into

Sub-blocks of size 8 × 8, which do not overlap with each other, are formed

k has an initial value of 1;

fourthly-2 a, recording the current first sub-block as

Record the current second sub-block as

Wherein (x)₄,y₄) To represent

And

x is more than or equal to 1₄≤8,1≤y₄≤8，

To represent

The middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (a),to representThe middle coordinate position is (x)₄,y₄) The pixel value of the pixel point of (1);