CN104994375A - Three-dimensional image quality objective evaluation method based on three-dimensional visual saliency - Google Patents

Abstract

The invention discloses a stereoscopic image quality objective evaluation method based on stereoscopic saliency. The stereoscopic image quality objective evaluation method includes the following steps: optimizing the stereoscopic saliency map by simulating the central offset and the central fovea characteristics, extracting The optimized stereoscopic saliency map; obtain the stereoscopic image comprehensive quality map according to the quality map of the distorted stereoscopic image quality map and the absolute difference map; carry out weighted summation to the stereoscopic image comprehensive quality map by the stereoscopic saliency map, and obtain Objective evaluation value of distorted stereoscopic image quality. The overall performance of the method in objective evaluation of stereoscopic image quality is superior to the algorithm in the prior art, has practical feasibility, obtains a relatively high objective evaluation index of stereoscopic image quality, and satisfies various needs in practical applications.

Description

An Objective Evaluation Method of Stereoscopic Image Quality Based on Stereoscopic Saliency

technical field

The invention relates to the field of stereoscopic images, in particular to an objective evaluation method of stereoscopic image quality based on stereoscopic salience.

Background technique

In recent years, stereoscopic image processing technology has been widely used in the entertainment industry and scientific fields. Different from the visual experience brought by flat images, stereoscopic images provide people with a more realistic stereoscopic visual experience. However, various distortions will occur in the process of collecting, compressing, storing, transporting and displaying stereoscopic images, which will affect people's stereoscopic vision experience. Stereoscopic image quality, as an inherent attribute of stereoscopic images, is an important index for judging the distortion of stereoscopic images. Although the accuracy of the subjective quality evaluation method is high, it is time-consuming, laborious and costly. Therefore, it is of great significance to establish an objective evaluation algorithm for stereoscopic image quality that can accurately and efficiently simulate the subjective evaluation results of human eyes.

After years of research, researchers at home and abroad have proposed many objective evaluation algorithms for stereoscopic image quality. Initially, some classic planar image quality evaluation algorithms such as mean square error (Mean Squared Error, MSE), peak signal to noise ratio (Peak Signal Noise Ratio, PSNR), structural similarity (Structural Similarity, SSIM) [1] are directly applied Based on the left and right viewpoints of the stereo image, the objective evaluation value of the stereo image quality is obtained by weighting the quality of the left and right images [2]. Literature [3] and Literature [4] considered the impact of depth perception on stereoscopic image quality evaluation, and combined the 2D evaluation method with the difference information or depth information of stereoscopic images for stereoscopic image quality evaluation.

At present, researchers have used some easy-to-model human vision characteristics, such as: nonlinear characteristics of brightness amplitude, multi-channel characteristics, contrast sensitivity characteristics and masking effects, etc., into the objective evaluation algorithm of stereoscopic image quality. With the gradual deepening of the understanding of the human visual system in various research fields, the integration of more complex and advanced human visual characteristics into the objective evaluation model has become an inevitable development direction. As an advanced feature of the human visual system, visual saliency refers to the different intensity of attention allocated by the human eye to different regions of the image. The salient area is an area that is easier for the human eye to pay attention to, so when watching a distorted stereoscopic image, the distortion in the salient area has a greater impact on the visual experience of the human eye. The visual saliency map provides an important basis for the objective evaluation of stereoscopic image quality. Literature [5-8] uses the influence of visual saliency in the objective evaluation algorithm of stereoscopic image quality. Literature [6] combines stereo saliency and SSIM algorithm [1] to evaluate the quality of rendered images. Literature [7] regards the phase consistency map and the saliency map as the features of the distorted stereo image, uses the local matching function to weight the difference between the two feature maps to calculate the correlation between the left and right viewpoints, and conducts the empirical value evaluation of the stereo image quality. Literature [8] used the SSIM algorithm to obtain the structural similarity map of the composite image of the original and distorted stereo images, and fused it with the stereo saliency map to obtain the stereo image quality evaluation index. Literature [9] combines the left and right views of the original and distorted stereo images with their saliency maps to determine the selective saliency maps of the left and right views of the original and distorted stereo images; then the SSIM algorithm is used for the selective saliency maps to obtain the salient structure similarity map, through Different weights are assigned to the edge area, smooth area and texture area to obtain the objective evaluation value of single view; finally, the objective evaluation value of stereoscopic image quality is obtained by weighting the objective evaluation value of the left and right views. Literature [5][7] used the disparity map of the original stereo image when extracting the stereo saliency map, ignoring the influence of the distortion of the depth information of the distorted stereo image on the detection of the stereo saliency map of the distorted stereo image.

However, the above method does not consider the influence of other visual characteristics of the human eye and depth information on the quality evaluation of the stereoscopic image, resulting in that the objective evaluation value cannot effectively reflect the subjective evaluation result.

Contents of the invention

The present invention provides an objective evaluation method of stereoscopic image quality based on stereoscopic saliency. According to the fusion of stereoscopic saliency map and stereoscopic image comprehensive quality map, the present invention accurately and effectively establishes the objective evaluation of stereoscopic image quality reflecting subjective evaluation results model, see the description below:

A stereoscopic image quality objective evaluation method based on stereoscopic salience, the stereoscopic image quality objective evaluation method comprises the following steps:

By simulating the characteristics of central offset and fovea, the stereoscopic saliency map is optimized, and the optimized stereoscopic saliency map is extracted;

Obtaining the comprehensive quality map of the stereo image according to the quality map of the distorted stereo image and the quality map of the absolute difference map;

The stereoscopic image comprehensive quality map is weighted and summed through the stereoscopic saliency map to obtain an objective evaluation value of the distorted stereoscopic image quality.

Wherein, before the step of optimizing the stereoscopic saliency map by simulating the central offset and the fovea characteristics, and extracting the optimized stereoscopic saliency map, the stereoscopic image quality objective evaluation method also includes the following steps:

Stereoscopic saliency maps of distorted stereo images are obtained by simulating visual saliency.

Wherein, the step of obtaining the stereoscopic saliency map of the distorted stereo image by simulating the visual saliency is specifically:

Convert the right view of the distorted stereoscopic image from the RGB color space to the YCbCr color space, divide it into non-overlapping image blocks, and perform discrete cosine transform on the image blocks;

The luminance contrast feature saliency map, the chroma contrast feature saliency map, the texture contrast feature saliency map and the absolute difference map of the distorted stereo image are normalized, and the stereo of the distorted stereo image is obtained by linearly combining these normalized maps. Visual saliency map.

Wherein, the step of optimizing the stereoscopic saliency map by simulating the central offset and the fovea characteristics, and extracting the optimized stereoscopic saliency map is specifically:

Preliminary optimization of stereoscopic saliency maps by simulating the central offset factor;

Simulate the characteristics of the fovea, and finally optimize the stereoscopic saliency map after the initial optimization.

Wherein, the step of obtaining the comprehensive quality map of the stereoscopic image according to the quality map of the distorted stereoscopic image quality map and the absolute difference map is specifically:

Extract the phase similarity of the right view of the distorted stereo image;

Extract the gradient similarity of the right view of the distorted stereo image;

Respectively extract the similarity of the chroma of the right view of the distorted stereo image;

Calculate the similarity between the original stereo image right view and the distorted stereo image right view;

Calculating the similarity of the left view of the distorted stereo image, using the average value method to obtain the quality map of the distorted stereo image with the similarity of the right view of the distorted stereo image and the similarity map of the left view;

Using a structural similarity algorithm for the absolute difference map of the original stereo image and the absolute difference map of the distorted stereo image, extracting the quality map of the absolute difference map of the distorted stereo image;

The quality map of the distorted stereo image is fused with the quality map of the absolute difference map to obtain the comprehensive quality map of the stereo image.

The beneficial effect of the technical solution provided by the present invention is that: on the basis of the initial stereo saliency map, the method adopts the characteristics of central offset and fovea to obtain the optimal stereo vision saliency map, which better simulates the process of extracting salient regions by the human visual system visual process. Then the stereoscopic saliency map and the stereoscopic image quality map are weighted and multiplied and normalized to obtain the objective evaluation value of the stereoscopic image. Compared with the other three algorithms, this algorithm avoids the defect of using the same weight for different regions of the whole image. The experimental verification shows that the overall performance of this method is better than the 3SIQA[8] algorithm, DSSIM[3] algorithm and SSIM[1] algorithm in terms of objective evaluation of stereoscopic image quality, and it is practical.

Description of drawings

Figure 1 is a schematic diagram of the source image "tree2";

Figure 2 is a schematic diagram of the source image "girl";

Figure 3 is a schematic diagram of the source image "tree1";

Figure 4 is a schematic diagram of the source image "tju";

Figure 5 is a schematic diagram of the source image "family";

Figure 6 is a schematic diagram of the degraded image "river";

Fig. 7 is a schematic diagram of the source image "ox";

Fig. 8 is a schematic diagram of the source image "woman";

Fig. 9 is a block diagram of a stereoscopic image quality evaluation method based on stereoscopic saliency;

Figure 10 is a comparison chart of the original saliency map and the saliency map combined with two influencing factors;

(a) is the right view of the distorted stereo image; (b) is the stereo saliency map SM1 before optimization; (c) is the stereo saliency map SM' optimized by central offset; (d) is SM' optimized by the fovea The stereo saliency map SM.

FIG. 11 is a flow chart of an objective evaluation method for stereoscopic image quality based on stereoscopic salience.

Detailed ways

In order to make the purpose, technical solution and advantages of the present invention clearer, the implementation manners of the present invention will be further described in detail below.

In order to solve the problems existing in the background technology, the present invention aims to firstly determine the stereoscopic saliency map of the distorted stereo image by simulating the visual saliency, and then carry out the preliminary determination of the stereoscopic saliency map by simulating the central offset and the fovea characteristics. optimization to extract a more accurate stereoscopic saliency map; secondly, the quality map of the stereoscopic image is obtained by fusing the quality maps of the left and right views and the quality map of the absolute difference map; finally, the stereoscopic image comprehensive quality map is obtained by using the stereoscopic saliency map The weighted summation is used to obtain the objective evaluation value of the quality of the distorted stereoscopic image, as described below for details.

Example 1

A stereoscopic image quality objective evaluation method based on stereoscopic salience, see Figure 11, the stereoscopic image quality objective evaluation method includes the following steps:

101: By simulating central offset and fovea characteristics, optimize the stereoscopic saliency map, and extract the optimized stereoscopic saliency map;

102: Obtain a stereoscopic image comprehensive quality map according to the quality map of the distorted stereoscopic image and the quality map of the absolute difference image;

103: Perform weighted summation on the comprehensive quality map of the stereo image through the stereo vision saliency map to obtain an objective evaluation value of the quality of the distorted stereo image.

Wherein, before step 101, the stereoscopic image quality objective evaluation method further includes: obtaining a stereoscopic saliency map of the distorted stereoscopic image by simulating visual saliency, and this step is specifically:

Convert the right view of the distorted stereoscopic image from the RGB color space to the YCbCr color space, divide it into non-overlapping image blocks, and carry out discrete cosine transform (Discrete Cosine Transform, discrete cosine) transformation to the image blocks;

The saliency map of the luminance contrast feature, the saliency map of the chroma contrast feature, the saliency map of the texture contrast feature and the absolute difference map of the distorted stereo image are normalized to obtain the stereo vision saliency map of the distorted stereo image.

Wherein, in step 101, by simulating the central offset and the fovea characteristics, the stereoscopic saliency map is optimized, and the steps of extracting the optimized stereoscopic saliency map are specifically:

Preliminary optimization of stereoscopic saliency maps by simulating the central offset factor;

Simulate the characteristics of the fovea, and finally optimize the stereoscopic saliency map after the initial optimization.

Wherein, the step of obtaining the comprehensive quality map of the stereo image according to the quality map of the distorted stereo image quality map and the absolute difference map in step 102 is specifically:

Extract the phase similarity of the right view of the distorted stereo image;

Extract the gradient similarity of the right view of the original stereo image and the right view of the distorted stereo image;

Extract the similarity of the chroma of the distorted right view respectively;

Calculate the similarity between the original stereo image right view and the distorted stereo image right view;

Calculate the similarity of the left view of the distorted stereo image, and use the average value method to obtain the quality map of the distorted stereo image by using the gradient similarity of the right view of the distorted stereo image and the similarity of the left view;

Using a structural similarity algorithm for the absolute difference map of the original stereo image and the absolute difference map of the distorted stereo image, extracting the quality map of the absolute difference map of the distorted stereo image;

The quality map of the distorted stereo image is fused with the quality map of the absolute difference map to obtain the comprehensive quality map of the stereo image.

To sum up, this method realizes the objective evaluation of stereoscopic image quality through steps 101-103, accurately and effectively establishes an objective evaluation model of stereoscopic image quality reflecting subjective evaluation results, and satisfies various needs in practical applications.

Example 2

201: Obtain a stereoscopic saliency map of a distorted stereo image by simulating visual saliency;

According to the literature [9], this method extracts the brightness contrast feature, chroma contrast feature and texture contrast feature of the right view of the distorted stereo image.

(1) Convert the right view of the distorted stereoscopic image from RGB to YCbCr color space, divide it into non-overlapping image blocks, and carry out discrete cosine transform to the image blocks;

Convert the right view of the distorted stereo image from the RGB color space to the YCbCr color space, the Y channel represents brightness information, and the Cb and Cr channels represent chrominance information; the right view of the distorted stereo image is divided into 8*8 non-overlapping image blocks, and the image blocks are Carry out discrete cosine (DCT) transform; Express the luminance feature with the direct current (DC) coefficient L=Y _DC of Y channel, the direct current (DC) coefficient Cb _DC of Cb channel, the direct current (DC) coefficient Cr _DC of Cr channel represent chromaticity respectively Feature C ₁ and chrominance feature C ₂ , namely C ₁ =Cb _DC and C ₂ =Cr _DC ; use the first 9 low-frequency AC (AC) coefficients of the Y channel {Y _AC1 , Y _AC2, ..., Y _AC9 } Indicates the texture feature T, that is, T={Y _AC1 , Y _AC2, ..., Y _AC9 }.

Y＝0.299R+0.578G+0.114B

Cr＝0.5R-0.4178G-0.0813B+128 (1)

Cb=-0.1687R-0.3313G+0.5B+128

The detailed operation steps of performing discrete cosine transform on an image block are well known to those skilled in the art. For example, reference may be made to the transform method in document [10], which will not be repeated in this embodiment of the present invention.

(2) Calculate the saliency value of the image block i, and obtain the luminance contrast feature saliency map F ^L and the chroma contrast feature saliency map and Texture contrast feature saliency map T;

The calculation process is well known to those skilled in the art. For example, each feature saliency map can be obtained by referring to the calculation method in document [9], which will not be described in this embodiment of the present invention.

(3) The feature saliency map F ^L , and T and the absolute difference map Δ of the distorted stereo image are normalized, and the stereo vision saliency map SM1 is preliminarily obtained by using the formula (2):

$\mathrm{<span\; class="notranslate">\; SM</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; L</span>}}{<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; f</span>}}^{{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

Wherein, Δ=|L-R|, L and R are the left view and the right view of the distorted stereo image, respectively.

202: Optimizing the stereoscopic saliency map by simulating central offset and fovea characteristics, and extracting the optimized stereoscopic saliency map;

(1) Simulate the central offset factor to perform preliminary optimization on the stereoscopic saliency map;

$\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; B</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; \{</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\frac{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; i</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; +</span>\frac{{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; j</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, CB(i,j) represents the offset information of the distorted stereo image pixel point (i,j) to its center point (i ₀ ,j ₀ ); (i ₀ ,j ₀ ) represents the center point coordinates of the distorted right view , (i, j) is the x-coordinate of the pixel point, σ _h and σ _v represent the standard deviation of the distorted stereoscopic image in the horizontal direction and vertical direction respectively, defined as σ _h =W/3, σ _v =H/3, W and H Indicates the number of horizontal pixels and vertical pixels of the distorted stereo image.

Preliminary optimization of the stereo vision saliency map SM1 using formula (4):

SM'＝αSM1+(1-α)CB (4)

Among them, CB is the central offset map obtained by formula (3), and α is the control parameter, which is defined as α=0.7.

(2) Simulate the characteristics of the fovea, and finally optimize the stereoscopic saliency map after the initial optimization.

Use formula (5) to simulate the central fovea characteristics:

$\mathrm{<span\; class="notranslate">\; C</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}\mathrm{<span\; class="notranslate">\; exp</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; \&delta;</span>}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; e</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; e</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; )</span>$

Among them, the unit of e is degree (degree); f is the spatial frequency, and its unit is cycle/degree; C ₀ is the contrast threshold; δ is the spatial frequency attenuation parameter; e ₂ is the half-resolution eccentricity constant . According to the experimental fitting results [10], δ=0.106, e ₂ =2.3, C ₀ =1/64.

The threshold is determined by the maximum between-class variance method, and the preliminary optimized stereo vision saliency map is divided into salient regions and non-salient regions. Suppose any pixel coordinate is (i ₁ , j ₁ ), and the coordinate of the nearest significant pixel point is (i ₂ , j ₂ ), then the retinal eccentricity e:

$\mathrm{<span\; class="notranslate">\; e</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; the\; tan</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\frac{\mathrm{<span\; class="notranslate">\; d</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 6</span>}<span\; class="notranslate">\; )</span>$

Among them, v is the viewing distance, the value of this method is 5 (the unit is the image width), and d is the Euclidean distance between the pixel point (i ₁ , j ₁ ) and the pixel point (i ₂ , j ₂ ): $d=\sqrt{{(i_1-i_2)}^2+{(j_1-j_2)}^2}.$

Use formula (7) to optimize the preliminary optimized stereoscopic saliency map SM' to obtain the final stereoscopic saliency map SM, and extend SM to the same size as the right view of the distorted stereo image.

SM＝SM'×C(f,e) (7)

203: Obtain a stereoscopic image comprehensive quality map according to the quality map of the distorted stereoscopic image and the quality map of the absolute difference image;

The present invention combines phase consistency features, image gradient features and chromaticity features to independently process the luminance information and chromaticity information of the image in the YIQ color space, and uses the feature similarity algorithm [11] to obtain the quality of a single view of a distorted stereoscopic image picture. Wherein, Y represents the brightness information of the image, and I and Q represent the chrominance information of the image, taking the right view as an example.

(1) Extract the phase similarity S _PC (x) of the right view of the distorted stereo image;

That is, use the Reter Kovesi [12] algorithm to extract the phase consistency feature PC _d (x) of the right view of the distorted stereo image at the pixel point x, as shown in formula (8). The same method is used to obtain the phase consistency feature PC _o (x) of the right view of the original stereo image at the pixel point x.

${\mathrm{<span\; class="notranslate">\; PC</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; l</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; E.</span>}}_{{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; l</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}\underset{\mathrm{<span\; class="notranslate">\; no</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; \&theta;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&epsiv;</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 8</span>}<span\; class="notranslate">\; )</span>$

Among them, θ _l =lπ/L, l={0,1,...,L-1} is the orientation angle of the filter,

L is the number of directions and is defined as L=4, is the amplitude of the response vector in direction θ _l and scale n, is the local energy function on the direction θ _l , ε is a very small constant.

Use formula (9) to determine the phase similarity of the right view of the distorted stereo image:

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; P</span>}\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; PC</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CenterDot;</span>{\mathrm{<span\; class="notranslate">\; PC</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}{{\mathrm{<span\; class="notranslate">\; PC</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; PC</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 9</span>}<span\; class="notranslate">\; )</span>$

Among them, PC _o (x) and PC _d (x) are the phase consistency values at the pixel point _x of the right view of the original stereo image and the right view of the distorted stereo image, respectively. T ₁ is a constant that prevents the denominator from being zero.

(2) Extract the gradient similarity of the right view of the distorted stereo image;

Use the Scharr operator to filter the brightness information Y of the right view of the original stereo image and the right view of the distorted stereo image respectively, and the filter values of pixel x in the horizontal and vertical directions of the right view of the distorted stereo image are G ₁ (x) and G ₂ (x), use the formula (10) to calculate the gradient magnitude GM _d (x) of the right view of the distorted stereo image at the pixel point _x , and use the same method to obtain the gradient magnitude GM _o (x) of the right view of the original stereo image at the pixel point x .

${\mathrm{<span\; class="notranslate">\; GM</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\sqrt{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; G</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; )</span>$

Use the formula to calculate the gradient similarity of the right view of the distorted stereo image:

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; G</span>}\mathrm{<span\; class="notranslate">\; m</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; GM</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; GM</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}{{\mathrm{<span\; class="notranslate">\; GM</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; GM</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 11</span>}<span\; class="notranslate">\; )</span>$

Among them, GM _o (x) and GM _d (x) are the gradient amplitude values at the pixel point x of the right view of the original stereo image and the right view of the distorted stereo image respectively, and T ₂ is a constant, which has the same meaning as T ₁ .

(3) Extract the similarity of the chroma I and Q of the distorted right view respectively;

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; I</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}{{{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 12</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; Q</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}{{{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; d</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 4</span>}}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 13</span>}<span\; class="notranslate">\; )</span>$

Wherein, T ₃ and T ₄ are constants, which have the same meaning as T ₁ . I _o , Q _o are the chromaticity information of the right view of the original stereo image, and I _d , Q _d are the chromaticity information of the right view of the distorted stereo image, respectively.

(4) Calculate the similarity between the original stereoscopic image right view and the distorted stereoscopic image right view;

Combine S _pc (x), S _GM (x), S _I (x) and S _Q (x) according to the following formula to calculate the similarity between the right view of the original stereo image and the right view of the distorted stereo image:

S _R (x) = [S _pc (x)] ^τ [S _GM (x)] ^β [S _I (x)S _Q (x)] ^λ (14)

Among them, τ, β, λ are adjustment parameters, and τ, β, λ are all defined as 1.

(5) According to the feature similarity method of calculating the right view of the distorted stereo image established in steps (1) to (4), calculate the similarity S _L (x) of the left view of the distorted stereo image, and combine S _R (x) and S _L (x) The distorted stereoscopic image quality map S ₀ (x) is obtained by using the mean value method.

(6) Structural similarity [1] (SSIM) algorithm is used for the absolute difference map of the original stereo image and the absolute difference map of the distorted stereo image to extract the quality map D(x) of the absolute difference map of the distorted stereo image. Then, using the algorithm proposed by AlexandreBenoit[4], the quality map S ₀ (x) of the distorted stereo image is fused with the quality map D(x) of the absolute difference map to obtain the comprehensive quality map of the stereo image:

Q ₀ (x)＝S ₀ (x)·(1+D(x)) (15)

204: Perform weighted summation on the comprehensive quality map of the stereo image through the stereo vision saliency map to obtain an objective evaluation value of the quality of the distorted stereo image.

The stereoscopic saliency map SM of the distorted stereo image and the stereo image comprehensive quality map Q ₀ (x) are weighted and summed and normalized to obtain the objective evaluation value Q of the stereo image quality.

$\mathrm{<span\; class="notranslate">\; Q</span>}<span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; \&Omega;</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; S</span>}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}{\underset{\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; \&Omega;</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}\mathrm{<span\; class="notranslate">\; S</span>}\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 16</span>}<span\; class="notranslate">\; )</span>$

Among them, Ω represents the entire spatial domain of the image, and SM(x) is the value of the stereoscopic saliency map SM at the pixel point x.

To sum up, this method realizes the objective evaluation of stereoscopic image quality through steps 201-204, accurately and effectively establishes an objective evaluation model of stereoscopic image quality reflecting subjective evaluation results, and meets various needs in practical applications.

Example 3

The method is described in detail below in conjunction with the technical solution: the present invention specifically includes three parts: extracting a stereoscopic salience map, extracting a stereoscopic image comprehensive quality map and merging the two, and the specific steps are as follows:

301: Initially extract a stereoscopic saliency map;

(1) Convert the right view of the distorted stereoscopic image from the RGB color space to the YCbCr (4:2:0) color space according to formula (1), the Y channel represents the brightness information, and the Cb and Cr channels represent the chroma information; the right view is divided into 8*8 non-overlapping image blocks, carry out discrete cosine transform to image blocks according to formula (17);

Y＝0.299R+0.578G+0.114B

Cr＝0.5R-0.4178G-0.0813B+128

Cb=-0.1687R-0.3313G+0.5B+128

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>\underset{\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; cos</span>}\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; N</span>}}\mathrm{<span\; class="notranslate">\; cos</span>}\frac{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; )</span>\mathrm{<span\; class="notranslate">\; no</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}}{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; Y</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; C</span>}\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 17</span>}<span\; class="notranslate">\; )</span>$

$\begin{array}{cc}\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{c}\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\\ \sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.& \mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\left\{\begin{array}{c}\sqrt{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\\ \sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; N</span>}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.\end{array}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 18</span>}<span\; class="notranslate">\; )</span>$

Wherein, both m and n are [0, N-1], N=8. F(m,n) represents the coefficient after the discrete cosine transform. In the YCbCr color space, Y(m,n), Cb(m,n), and Cr(m,n) respectively represent the coefficients of the brightness information Y, chrominance information Cb and Cr after the discrete cosine transform. When m=n=0, calculate the direct current (DC) coefficient Then get the brightness feature L=Y _DC and the chrominance feature. The remaining 63 coefficients are alternating current (AC) coefficients. Use the zig-zag scanning method to obtain the first 9 low-frequency AC (AC) coefficients of the channel {Y(0,1), Y(1,0), Y(2,0), Y(1,1), Y(0 ,2), Y(0,3), Y(1,2), Y(2,1), Y(3,0)}, and then get the texture feature T={Y _AC1, Y _AC2, ..., Y _AC9 }.

(2) Use formula (19) to calculate the contrast between image block i and other image blocks to estimate the saliency value of image block i, and obtain the luminance contrast feature saliency map ^FL and chroma contrast feature saliency map and Texture contrast feature saliency map T.

${\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; \&NotEqual;</span>\mathrm{<span\; class="notranslate">\; j</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}\sqrt{\mathrm{<span\; class="notranslate">\; 2</span>}\mathrm{<span\; class="notranslate">\; \&pi;</span>}}}{\mathrm{<span\; class="notranslate">\; e</span>}}^{<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; /</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}{\mathrm{<span\; class="notranslate">\; \&sigma;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 19</span>}<span\; class="notranslate">\; )</span>$

where k represents a feature and k∈{L,C ₁ ,C ₂ ,T}, σ is a parameter and defines σ=20, l _ij is the Euclidean distance between image blocks i and j, Indicates the feature contrast of image patches i and j on feature k.

Use formula (20) to calculate the contrast ratio of brightness features and chroma features, and use formula (21) to calculate the contrast ratio of texture features

${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; |</span>}{{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; d</span>}}}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; \&Element;</span><span\; class="notranslate">\; \{</span>\mathrm{<span\; class="notranslate">\; L</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 20</span>}<span\; class="notranslate">\; )</span>$

${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; =</span>\frac{\sqrt{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}{{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; j</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; twenty\; one</span>}<span\; class="notranslate">\; )</span>$

in, and The eigenvalues of image block i and image block j on feature d respectively, t is the AC coefficient and

t ∈ {1,2,2,...,9}, Indicates the coefficient t of the image block i on the texture feature T.

(3) The feature saliency map F ^L obtained by formula (19), T and the absolute difference map Δ of the distorted stereo image are subjected to the Max-Min normalization operation, and the stereo saliency map SM1 is preliminarily obtained according to the formula (22):

$\mathrm{<span\; class="notranslate">\; S</span>}\mathrm{<span\; class="notranslate">\; m</span>}\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; f</span>}}^{{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; \&Delta;</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; /</span>\mathrm{<span\; class="notranslate">\; 5</span>}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; twenty\; two</span>}<span\; class="notranslate">\; )</span>$

Among them, Δ=|L-R|, L and R are the grayscale images of the left view and right view of the distorted stereo image, respectively

302: Optimizing the stereo saliency map;

(1) Simulate the central offset factor [13] and optimize the saliency map SM1.

(2) The density of retinal photoreceptors decreases rapidly from the fovea to the periphery [14][15]. Therefore, when the image is mapped onto the retina, the spatial frequency resolution of the human eye to the fovea area is relatively high, and this part of the area is the area that the human eye focuses on, that is, the salient area; with the increase of the eccentricity e, the human eye's Spatial resolution will be reduced. This characteristic is simulated by using the contrast sensitivity function (Contrast Sensitivity Function, CSF) [15], and according to the experimental fitting result [10], δ=0.106, e ₂ =2.3, C ₀ =1/64.

The threshold is determined by the method of maximum between-class variance, and the stereo saliency map is divided into salient regions and non-salient regions. Multiply SM' and C(f,e) to optimize the stereoscopic saliency map SM' to obtain the final stereoscopic saliency map, and expand it to the same size as the right view of the distorted stereo image.

For the specific operation of this step, refer to Embodiment 2, which will not be repeated in this embodiment of the present invention.

303: Calculate the quality map of the distorted stereo image.

The present invention combines phase consistency features, image gradient features and chromaticity features to independently process the luminance information and chromaticity information of the image in the YIQ color space, and uses the feature similarity algorithm [11] to obtain the quality of a single view of a distorted stereoscopic image Figure, take the right view as an example.

(1) Convert the image from the RGB color space to the YIQ color space, and use the Reter Kovesi[12] algorithm to extract the phase consistency information at the pixel point x of the original and distorted right views. Determines the phase similarity of the distorted right view.

(2) Extract the gradient similarity of the original and distorted right views.

(3) Extract the chroma similarity of the distorted right view.

(4) Calculate the similarity between the original stereo image right view and the distorted right view.

(5) Calculate the similarity S _L (x) of the distorted left view according to the method for calculating the feature similarity of the distorted right view established in steps (1) to (4).

(6) Structural similarity [1] (SSIM) algorithm is used for the absolute difference map of the original stereo image and the absolute difference map of the distorted stereo image to extract the quality map D(x) of the absolute difference map of the distorted stereo image. Using the Alexandre Benoit[4] algorithm, the image quality map S ₀ (x) is fused with the quality map D(x) of the absolute difference map to obtain the comprehensive quality map Q ₀ (x) of the stereo image.

For the specific operation of this step, refer to Embodiment 2, which will not be repeated in this embodiment of the present invention.

304: Obtain an objective evaluation value of stereoscopic image quality.

The weighted sum of the stereoscopic saliency map SM and the stereoscopic image quality map of the distorted stereoscopic image is summed and normalized to obtain the objective evaluation value of the stereoscopic image quality.

To sum up, this method realizes the objective evaluation of stereoscopic image quality through steps 301 to 304, accurately and effectively establishes an objective evaluation model of stereoscopic image quality reflecting subjective evaluation results, and satisfies various needs in practical applications.

Example 4

The reference experimental materials used in this design are all taken from the stereo information database provided by the Institute of Broadband Wireless Communication and Stereo Imaging. A total of 8 uncompressed, non-noised and unblurred original images in the image database are selected, namely "tree2", "family", "girl", "river", "tree1", "ox", "tju", "woman", as shown in Figures 1 to 8. Perform JPEG compression distortion and Gaussian blur distortion on 8 standard stereoscopic image pairs And Gaussian plus noise distortion processing, a total of 260 distorted stereo image pairs are obtained. Each image resolution is 2560 × 1024. Subjective evaluation is made on all distorted image pairs, and the average opinion value (MeanOpinion Score, MOS) is recorded. In 406 Among the stereoscopic image samples, 150 images are selected as training samples and 256 images are selected as testing samples.

Figure 10 is a schematic diagram of the comparison of the stereo saliency map before and after optimization, Figure 10 (a) is the right view of the distorted stereo image, (b) is the stereo saliency map SM1 before optimization, (c) is the center partial Stereo saliency map SM' after shift optimization, (d) is (c) stereo saliency map SM after foveo optimization. Due to the existence of the central offset factor, the pixels closer to the middle area of the image are more likely to attract the attention of the viewer, so comparing (b) and (c), it is found that the pixels in the central area of the image are notable after the introduction of the central offset factor Increased; under the influence of the central fovea characteristics, the resolution of the human eye to the area farther away from the salient area is low, so the saliency of the pixels in the non-salient area of (d) after (c) optimization is reduced. Therefore, the stereoscopic saliency map optimized by using the central offset factor and the central foveal characteristics is more in line with the human visual characteristics.

Table 1 lists the performance indicators between this algorithm and the subjective and objective evaluation values of SSIM[1], DSSIM[3] and 3SIQA[8] algorithms.

Table 1 Performance indicators of different evaluation algorithms

It can be seen from the data in Table 1 that for different distortion types, the PLCC index and RMSE index of this algorithm have been improved to varying degrees, the PLCC value is above 0.95, and the RMSE value is below 0.45. Although the KROCC value of this algorithm for Gaussian noise distortion and Gaussian blur distortion is slightly lower than that of the literature [8] algorithm, the overall performance index of this algorithm is much higher than the other three algorithms, which is due to the literature [8] and [1] The algorithm does not consider depth information. The literature [3] and SSIM [1] algorithms do not consider the influence of stereoscopic saliency. This algorithm considers the influence of stereoscopic saliency and depth information. Therefore, for different distortion types and distortion strengths, this The objective evaluation value of the algorithm is in good agreement with the subjective evaluation results.

references

[1] Z.Wang, A.C.Bovik A, H.R.Sheikh, et al. Image Quality Assessment: From Error Visibility to Structural Similarity [J]. IEEE Transactions on Image Processing, Vol.13, No.4, April 2004.

[2]Patrizio Campisi,Patrick Le Callet,Enrico Marini.STEREOSCOPIC IMAGES QUALITYASSESSMENT[C].15th European Signal Processing Conference(EUSIPCO 2007),Poznan,Poland,September 3-7,2007.

[3] de Miranda Cardoso, J.V., Danilo Miranda Regis, C., Sampaio de Alencar, M. On the preference of disparity-based weighting technique applied to 3D image quality assessment [C]. Broadband Multimedia Systems and Broadcasting (BMSB), 2014 IEEE international Symposium on, Beijing, 25-27 June 2014.

[4] Alexandre Benoit, Patrick Le Callet, Patrizio Campisi, et al. Using disparity for quality assessment of stereoscopic images [C]. 15th IEEE International Conference on Image Processing (ICIP). San Diego, CA, 2008: 389-392.

[5]Dongdong Zhang, Jiahe Huang, Di Zang, et al.An quality metric for 3D rendered image based on stereo saliency and structural similarity[C].Global High Tech Congress on Electronics(GHTCE),2013IEEE,Shenzhen,17-19Nov. 2013:170-174.

[6] Qi F, Jiang T T, Ma S W, et al. Quality of experience assessment for stereoscopic images [C]. 2012 IEEE International Symposium on Cirruits and Systems (ISCAS), Seoul, Korea, 2012:1712–1715. [7] Shao Feng, Duan Fen-fang, et al. 3D Visual attention for stereoscopic image quality assessment [J]. Journal of software, 2014, Vol 9, No 7:1841-1847.

[8]Xiu-qin Chu, Yuan-Yuan Wu, Qian Li.Saliency structure stereoscopic image quality assessment method[J].Optik-International Journal for Light and Electron Optics,2014,125(2):704-709.

[9]Yuming Fang, Junle Wang, Manish Narwaria, et al.Saliency Detection for Stereoscopic Images[J].IEEE TRANSACTIONS ON IMAGE PROCESSING,VOL.23,NO.6,JUNE 2014.

[10] W.S.Geisler and J.S.Perry, A real-time foveated multiresolution system for low-bandwidth video communication, Proc. SPIE, vol.3299, pp.294–305, July 1998.

[11]Lin Zhang, Lei Zhang, Xuanqin Mou, et al.FSIM: A Feature Similarity Index for ImageQuality Assessment[J].IEEE TRANSACTIONS ON IMAGE PROCESSING,VOL.20,NO.8,AUGUST 2011.

[12]Kovesi P.Image features from phase congruency[J].Videre:Journal of Computer VisionResearch,1999,1(3):1-30.

[13]Le Meur O.,Le Callet,P.,Barba,et al.A coherent computational approach to model bottom-up visual attention[J].Pattern Analysis and Machine Intelligence,IEEE Transactions on,2006,28(5): 802-817.

[14]P.Tseng, R.Carmi, I.G.M.Camerson, et al.Quantifying center bias of observers in freeviewing of dynamic natural scenes[J].Journal of Vision,2009,vol.9,no.7.

[15] Zhou Wang, Ligang Lu, Alan Conrad Bovik. Foveation Scalable Video Coding With Automatic Fixation Selection [J]. IEEE TRANSACTIONS ON IMAGE PROCESSING, FEBRUARY 2003, VOL.12, NO.2:243-254.

In the embodiments of the present invention, unless otherwise specified, the models of the devices are not limited, as long as they can complete the above functions.

Those skilled in the art can understand that the accompanying drawing is only a schematic diagram of a preferred embodiment, and the serial numbers of the above-mentioned embodiments of the present invention are for description only, and do not represent the advantages and disadvantages of the embodiments.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (5)

1. A stereoscopic image quality objective evaluation method based on stereoscopic visual saliency is characterized by comprising the following steps:

optimizing the stereoscopic vision saliency map by simulating central deviation and fovea characteristics, and extracting the optimized stereoscopic vision saliency map;

obtaining a comprehensive quality image of the stereo image according to the quality images of the distorted stereo image and the absolute difference image;

and carrying out weighted summation on the stereo image comprehensive quality image through the stereo salient image to obtain an objective evaluation value of the quality of the distorted stereo image.

2. The objective evaluation method for stereoscopic image quality based on stereoscopic saliency according to claim 1, characterized in that before the step of extracting the optimized stereoscopic saliency map by optimizing the stereoscopic saliency map through simulating the center shift and fovea characteristics, the objective evaluation method for stereoscopic image quality further comprises the following steps:

and acquiring a stereoscopic vision saliency map of the distorted stereoscopic image by simulating the visual saliency.

3. The method for objectively evaluating the quality of a stereoscopic image based on stereoscopic saliency according to claim 1, wherein the step of obtaining the stereoscopic saliency map of the distorted stereoscopic image by simulating the visual saliency is specifically as follows:

converting the right view of the distorted stereo image from an RGB color space to a YCbCr color space, dividing the distorted stereo image into non-overlapping image blocks, and performing discrete cosine transform on the image blocks;

and normalizing the brightness contrast characteristic saliency map, the chroma contrast characteristic saliency map, the texture contrast characteristic saliency map and the absolute difference map of the distorted stereo image to obtain the stereo visual saliency map of the distorted stereo image.

4. The objective evaluation method for stereoscopic image quality based on stereoscopic saliency according to claim 1, characterized in that the stereoscopic saliency map is optimized by simulating the center shift and fovea characteristics, and the step of extracting the optimized stereoscopic saliency map comprises:

simulating a central offset factor, and performing primary optimization on the stereoscopic vision saliency map;

and simulating the foveal characteristic, and finally optimizing the preliminarily optimized stereoscopic vision saliency map.

5. The objective evaluation method for stereoscopic image quality based on stereoscopic visual saliency according to claim 1, wherein the step of obtaining the stereoscopic image comprehensive quality map from the quality maps of the distorted stereoscopic image quality map and the absolute difference map specifically comprises:

extracting the phase similarity of the right view of the distorted stereo image;

extracting gradient similarity of a right view of the distorted stereo image;

respectively extracting the similarity of the chromaticity of the distorted right view;

calculating the similarity between the original stereo image right view and the distorted stereo image right view;

calculating the similarity of a left view of the distorted stereo image, and obtaining a quality image of the distorted stereo image by using the gradient similarity of a right view of the distorted stereo image and the similarity of the left view by using an average value method;

extracting a quality diagram of the absolute difference diagram of the distorted stereo image by adopting a structural similarity algorithm for the absolute difference diagram of the original stereo image and the absolute difference diagram of the distorted stereo image;

and fusing the quality image of the distorted stereo image with the quality image of the absolute difference image to obtain a comprehensive quality image of the stereo image.