CN110944165A

CN110944165A - Stereoscopic image visual comfort level improving method combining perceived depth quality

Info

Publication number: CN110944165A
Application number: CN201911105732.6A
Authority: CN
Inventors: 应宏微; 郁梅; 池碧蔚; 张君君; 蒋刚毅
Original assignee: Ningbo University
Current assignee: Ningbo University
Priority date: 2019-11-13
Filing date: 2019-11-13
Publication date: 2020-03-31
Anticipated expiration: 2039-11-13
Also published as: CN110944165B

Abstract

The invention discloses a method for improving the visual comfort of a three-dimensional image by combining perceived depth quality, which comprises the steps of firstly establishing a visual satisfaction objective evaluation model for guiding the improvement process, wherein the model integrates the visual comfort and the perceived depth quality objective evaluation consisting of a perceived absolute distance and a perceived relative distance; redrawing an original stereo image needing to improve visual comfort by adopting a viewing distance nonlinear translation method, wherein the redrawn stereo image has the characteristic of unchanged virtual scene geometric proportion; and then respectively calculating the visual satisfaction and the objective evaluation value of the visual comfort of the redrawn three-dimensional image, wherein when the visual satisfaction reaches the maximum value in the redrawing for many times and the visual comfort value is greater than that of the original three-dimensional image, the redrawn three-dimensional image is the optimal three-dimensional image, and the improved three-dimensional image improves the visual comfort and simultaneously keeps better perception depth quality and is more consistent with the preference of a human visual system.

Description

Stereoscopic image visual comfort level improving method combining perceived depth quality

Technical Field

The invention relates to a method for improving the visual comfort of a stereoscopic image, in particular to a method for improving the visual comfort of a stereoscopic image by combining perceived depth quality.

Background

With the rapid development of stereoscopic video display technology and high-quality stereoscopic video content acquisition technology, the visual satisfaction of stereoscopic images/videos has become an important issue in the design of stereoscopic systems. Visual Comfort (VC) of a stereoscopic image is one of important factors affecting the visual satisfaction of the stereoscopic image. The research on the visual comfort of the stereo image is physiological fatigue and discomfort caused on the premise that the content of the stereo image is not distorted; the research on the Perceived Depth Quality (PDQ) of a stereo image is that the stereo image brings the stereoscopic impression, immersion feeling and presence feeling to a viewer, and the visual comfort and the perceived depth quality are important factors influencing the visual satisfaction of the stereo image. The method for improving the visual comfort of the research stereo image plays an important role in improving the visual satisfaction of a viewer and guiding the production and post-processing of 3D content.

The visual comfort of the stereoscopic image is closely related to the binocular parallax, the binocular parallax is a cause for generating stereoscopic sensation, and the overlarge binocular parallax is an important factor for generating visual discomfort. The existing objective evaluation method for the visual comfort of the stereo image mainly extracts the parallax features of the stereo image, such as parallax amplitude, parallax gradient, relative parallax, parallax range and the like, then establishes an objective evaluation model for the visual comfort by using a statistical method or a machine learning algorithm, and when the model is used for guiding the manufacturing and post-processing of 3D content, binocular parallax can be excessively reduced so as to enable the visual comfort to be optimal, but the stereoscopic impression, immersion impression, presence impression and reality impression are weaker, namely, the perceived depth quality is weakened, so that the visual satisfaction is reduced. Therefore, a method for improving the comfort level of the stereoscopic image with the perceived depth quality fused is required to be researched, so that the visual comfort level of the stereoscopic image is improved, and the overall visual satisfaction degree of the stereoscopic image is also improved.

Disclosure of Invention

The technical problem to be solved by the invention is to provide a method for improving the visual comfort of a stereoscopic image by combining with the perceived depth quality, which effectively improves the visual comfort of the stereoscopic image and greatly improves the overall visual satisfaction of the stereoscopic image.

The technical scheme adopted by the invention for solving the technical problems is as follows: a stereoscopic image visual comfort level improving method combining perceived depth quality is characterized by comprising four processes of stereoscopic image redrawing based on viewing distance nonlinear translation, modeling of a stereoscopic image visual satisfaction degree objective evaluation model, objective evaluation of the visual satisfaction degree and the visual comfort level of a stereoscopic image and optimal stereoscopic image discrimination;

the method comprises the following specific steps of:

step 1_ 1: will have a width of W_imageAnd has a height H_imageThe stereo image to be improved in visual comfort is recorded as S₀Will S₀The left view image and the right view image are correspondingly denoted as SL₀And SR₀(ii) a Then obtain the SR₀Is recorded as DR₀(ii) a Then calculating DR₀The screen parallax value of each pixel point in (1) is calculated as DR₀The screen parallax value of the pixel point with the middle coordinate position (x, y) is recorded as DRS₀(x, y); then according to DR₀Calculating S as the screen parallax value of each pixel point₀The distance between each point in the formed virtual scene and the human eye, i.e. calculating S₀Viewing distance, S, of each point in the formed virtual scene₀The viewing distance of a point with a coordinate position (x, y) in the formed virtual scene is recorded as VD₀(x,y)，

Finally, the S is₀The matrix formed by the viewing distances of all the points in the virtual scene is denoted as VD₀(ii) a Wherein x represents the column coordinate of the pixel point, y represents the row coordinate of the pixel point, x is more than or equal to 0 and is more than W_image，0≤y＜H_imageP represents a binocular pupillary distance of human eyes, and T represents a distance from the human eyes to a 3D display for displaying a stereoscopic image;

step 1_ 2: let j denote the number of translation steps, and the value range of j is more than or equal to 0 and less than or equal to N; s when j is 0₀All points in the formed virtual scene do not translate in the depth direction; when j is more than or equal to 1 and less than or equal to N, enabling S to be₀The point with the minimum viewing distance in the formed virtual scene is translated by a distance of j multiplied by delta Z in the depth direction; then let S₀All the points except the point with the minimum viewing distance in the formed virtual scene are subjected to nonlinear translation in the depth direction to obtain a new virtual scene, and then a matrix formed by the viewing distances of all the points in the new virtual scene is obtained and recorded as VD_jWill VD_jThe value of the element with the middle subscript position (x, y), that is, the viewing distance of the point with the coordinate position (x, y) in the new virtual scene is recorded as VD_j(x,y)，

Wherein j is a positive integer, N is a positive integer, and N belongs to [10,20 ]]And deltaz represents the step size of the translation,

represents VD₀Minimum value of (1);

step 1_ 3: j is more than or equal to 1 and less than or equal to N according to VD_jCalculating the screen parallax value of each pixel point in the parallax image of the right viewpoint image of the new three-dimensional image corresponding to the new virtual scene, and recording the screen parallax value of the pixel point with the coordinate position (x, y) in the parallax image of the right viewpoint image of the new three-dimensional image as DRS_j(x,y)，

Then calculating the parallax image of the right viewpoint image of the new stereo image and recording the parallax image as DR_jWill DR_jThe pixel value of the pixel point with the middle coordinate position (x, y) is recorded as DR_j(x,y)，

DR_j(x, y) is also a new perspective viewThe parallax value of a pixel point with a coordinate position of (x, y) in the right viewpoint image of the image, and the screen width of a 3D display for displaying a stereoscopic image is W_3DAnd a screen height of the 3D display for displaying the stereoscopic image is H_3D；

Step 1_ 4: calculating each pixel point and SL in the left viewpoint image of the new stereoscopic image when j is more than or equal to 1 and less than or equal to N₀The relation between the corresponding pixel points on the column coordinates, the pixel point with the coordinate position of (x, y) in the left viewpoint image of the new stereo image and the SL₀The relation between the pixel points with the (x, y) middle coordinate position on the column coordinate is recorded as Δ L (x, y), and Δ L (x, y) is (DR)₀(x,y)-DR_j(x, y))/2; also, each pixel point and SR in the right view image of the new stereoscopic image are calculated₀The relation between the corresponding pixel points on the column coordinates, the pixel point with the coordinate position of (x, y) in the right viewpoint image of the new three-dimensional image and the SR₀The relation between the pixel points with the (x, y) middle coordinate position on the column coordinate is recorded as Δ R (x, y), and Δ R (x, y) — (DR)₀(x,y)-DR_j(x, y))/2; wherein, DR₀(x, y) denotes DR₀The pixel value DR of the pixel point with the middle coordinate position (x, y)₀(x, y) is also SR₀The middle coordinate position is the parallax value of the pixel point of (x, y);

step 1_ 5: obtaining SL when j is more than or equal to 1 and less than or equal to N₀The left viewpoint image obtained after redrawing is marked as SL_j(ii) a And obtaining the SR₀The right viewpoint image obtained after redrawing is marked as SR_j(ii) a Then from SL_jAnd SR_jForm a redrawn three-dimensional image, denoted as S_j(ii) a Wherein SL_jAnd SR_jThe acquisition process comprises the following steps:

step 1_5 a: SR'_jRepresents SR₀If x + delta R (x, y) < W is more than or equal to 0, the right viewpoint image obtained after the first redrawing_imageThen SR 'is calculated'_jThe pixel value of the pixel point with the middle coordinate position of (x, y) is recorded as SR'_j(x,y)，SR'_j(x,y)＝SR₀(x + Δ R (x, y), y); if x + Δ R (x, y) < 0 or x + Δ R (x, y) ≧ W_imageThen SR'_jDetermining the pixel point with the middle coordinate position (x, y) as a cavity pixel point; wherein, SR₀(x + Δ R (x, y), y) represents SR₀The middle coordinate position is the pixel value of the pixel point of (x + delta R (x, y), y);

step 1_5 b: SR 'judgment'_jIf the hollow pixel point exists in the buffer, SR 'is carried out'_jAnew noted SR'_jThen, step 1_5c is performed; if the empty pixel exists, each empty pixel is processed, and SR 'is set'_jIf the pixel point with the middle coordinate position (x, y) is a cavity pixel point, then x + DR is more than or equal to 0₀(x,y)+ΔR(x,y)＜W_imageCalculating the redrawn pixel value of the hole pixel point to be equal to SL₀(x+DR₀(x, y) + Δ R (x, y), y), when x + DR₀(x, y) + Δ R (x, y) < 0 or x + DR₀(x,y)+ΔR(x,y)≥W_imageThe hollow pixel points are not redrawn, and an SR is obtained after all the hollow pixel points are processed₀The right viewpoint image obtained after the second redrawing is marked as SR "_jThen, step 1_5c is performed; wherein SL₀(x+DR₀(x, y) + Δ R (x, y), y) represents SL₀The middle coordinate position is (x + DR)₀(x, y) + Δ R (x, y), y);

step 1_5 c: line of SL'_jRepresents SL₀If x-DR is not less than 0, the left viewpoint image obtained after the first redrawing_j(x,y)＜W_imageThen calculate SL'_jThe pixel value of the pixel point with the middle coordinate position of (x, y) is recorded as SL'_j(x,y)，SL'_j(x,y)＝SR”_j(x-DR_j(x, y), y); if x-DR_j(x, y) < 0 or x-DR_j(x,y)≥W_imageThen SL 'is prepared'_jDetermining the pixel point with the middle coordinate position (x, y) as a cavity pixel point; wherein, SR "_j(x-DR_j(x, y), y) represents SR "_jThe middle coordinate position is (x-DR)_j(x, y), y) pixel values of the pixels;

step 1_5 d: judging SL'_jIf there is a hole pixel point, if there is no hole pixel point, SL'_jAnew as SL "_jThen, step 1_5e is performed; if the signal exists, processing each hollow pixel point, and setting SL'_jIf the pixel point with the middle coordinate position (x, y) is a cavity pixel point, then when x + delta L (x, y) < W is more than or equal to 0_imageCalculating the redrawn pixel value of the hole pixel point to be equal to SL₀(x + Δ L (x, y), y) when x + Δ L (x, y) < 0 or x + Δ L (x, y) ≧ W_imageThe hollow pixel points are not redrawn, and SL is obtained after all the hollow pixel points are processed₀The left viewpoint image obtained after the second redrawing is marked as SL "_jThen, step 1_5e is performed; wherein SL₀(x + Δ L (x, y), y) represents SL₀The middle coordinate position is the pixel value of the pixel point of (x + delta L (x, y), y);

step 1_5 e: if SL "_jIf no hollow pixel point exists in the interface, SL is used "_jAs SL_j(ii) a If SR'_jIf no hollow pixel exists in the image, then SR is carried out "_jAs SR_j(ii) a If SL "_jAnd SR'_jIf there are still void pixel points, then use image repairing technique to repair each void pixel point, and use SL "_jThe left viewpoint image obtained after the patching is used as SL_jIs prepared from SR "_jThe right viewpoint image obtained after patching is used as SR_j；

The modeling part of the three-dimensional image visual satisfaction objective evaluation model comprises the following specific steps:

step 2_ 1: selecting M pieces of width W_imageAnd has a height H_imageThe stereo images of (a) form a stereo image set; wherein M is a positive integer, and M is more than or equal to 20;

step 2_ 2: defining the ith stereo image to be processed currently in the stereo image set as a current stereo image; wherein i is a positive integer, i is more than or equal to 1 and less than or equal to M;

step 2_ 3: recording the current stereo image as phi_i,0(ii) a Then changing the value of j from 1 to N, repeatedly executing the j values for 1 time in the same way according to the processes from step 1_1 to step 1_5, and repeatedly executing the j values from 1 to N for N times to obtain phi_i,0Corresponding N redrawing three-dimensional images, and obtaining phi when the number of translation steps is j_i,₀The corresponding redrawn stereo image is recorded as phi_i,jWhile obtaining phi_i,0A matrix VD of viewing distances of all points in the formed virtual scene_i,0And each redrawn stereoscopic image pairThe viewing distance of all points in the new virtual scene is used to form a matrix, and phi_i,jThe matrix formed by the viewing distances of all the points in the corresponding new virtual scene is recorded as VD_i,j；

Step 2_ 4: taking the next to-be-processed stereo image in the stereo image set as a current stereo image by setting i to i + 1; then, returning to the step 2_3 to continue executing until N redrawn three-dimensional images corresponding to each three-dimensional image in the three-dimensional image set are obtained, and forming a training set by all three-dimensional images in the three-dimensional image set and the obtained M multiplied by N redrawn three-dimensional images; wherein, the' in the i-i +1 is an assignment symbol;

step 2_ 5: obtaining an average subjective evaluation mean value of 3D visual satisfaction of each three-dimensional image in a training set; then, the set formed by the mean subjective evaluation average of the 3D visual satisfaction of all the stereo images in the training set is recorded as { VSMOS_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; VSMOS when j is 0_i,0Represents phi_i,0The average subjective evaluation mean value of 3D visual satisfaction is that when j is more than or equal to 1 and less than or equal to N, VSMOS_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe average subjective evaluation mean value of the 3D visual satisfaction degree comprises the comprehensive effect of the visual comfort degree and the perception depth;

step 2_ 6: calculating the objective evaluation value of the visual comfort degree of each three-dimensional image in the training set according to the objective evaluation model of the visual comfort degree; then, the set of objective evaluation values of visual comfort of all the stereo images in the training set is recorded as { VCA }_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; where, when j is 0, VCA_i,0Represents phi_i,0The objective evaluation value of visual comfort is VCA when j is more than or equal to 1 and less than or equal to N_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe objective evaluation value of the visual comfort of the user;

step 2_ 7: according to the set { VD_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N, and calculating the objective evaluation value of the perceived absolute distance of each three-dimensional image in the training set; then all stereo images in the training set are combinedIs given as { APDA (adaptive distance estimation and estimation) in the set of objective evaluation values of perceived absolute distance_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; wherein, VD is when j is more than or equal to 1 and less than or equal to N_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jAPDA when j is 0_i,0Represents phi_i,0The perceived absolute distance objective evaluation value of (1) is more than or equal to j and less than or equal to N, and APDA_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe perceived absolute distance objective evaluation value of (1);

step 2_ 8: according to the set { VD_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N, and calculating the objective evaluation value of the perception relative distance of each three-dimensional image in the training set; then, a set of objective evaluation values of perceptual relative distances of all stereo images in the training set is recorded as { RPDA_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; wherein, when j is 0, RPDA_i,0Represents phi_i,0The perceived relative distance objective evaluation value of (1) is more than or equal to j and less than or equal to N, and the RPDA is used for evaluating the distance_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe perceived relative distance objective evaluation value of (1);

step 2_ 9: using a training model of ε -SVR in VSMOS_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N and { VCA_i,j|1≤i≤M,0≤j≤N}、{APDA_i,j|1≤i≤M,0≤j≤N}、{RPDA_i,jFitting is carried out between i is more than or equal to 1 and less than or equal to M and j is more than or equal to 0 and less than or equal to N to obtain the objective evaluation model of the visual satisfaction degree of the stereo image, and the description is as follows: STFA (VCA, APDA, RPDA); the kernel function of the training model epsilon-SVR uses a histogram interaction type, a width coefficient gamma is 1/256, a penalty coefficient C is 4, an insensitive loss coefficient epsilon is 0.1, STFA () is a function representation form of a stereoscopic image visual satisfaction objective evaluation model, VCA, APDA and RPDA are all input of the stereoscopic image visual satisfaction objective evaluation model, VCA is used for representing a visual comfort objective evaluation value of a stereoscopic image, APDA is used for representing a perception absolute distance objective evaluation value of the stereoscopic image, and RPDA is used for representing a perception relative distance objective evaluation value of the stereoscopic image;

the objective evaluation part of the visual satisfaction and the visual comfort of the stereo image comprises the following specific steps:

step 3_ 1: s is obtained in the same manner as the procedure of step 2_6_jObjective evaluation value of visual comfort of (1), noted as VCA_j(ii) a And obtains S in the same manner as the procedure of step 2_7_jIs recorded as APDA_j(ii) a S is obtained in the same manner as the procedure of step 2_8_jThe objective evaluation value of the perceived relative distance is marked as RPDA_j(ii) a Then VCA_j、APDA_j、RPDA_jInputting the three-dimensional image visual satisfaction objective evaluation model obtained in the step 2-9 to obtain S_jIs recorded as STFA_j，STFA_j＝STFA(VCA_j,APDA_j,RPDA_j) (ii) a Wherein j is more than or equal to 0 and less than or equal to N;

the specific steps of the optimal stereo image discrimination part are as follows:

step 4_ 1: directly performing step 4_2 when j is 0 or j is 1; when j is more than or equal to 2 and less than or equal to N, the pointer points to S_j-1Judging STFA_j-1＞STFA_j-2、STFA_j-1＞STFA_j、VCA_j-1＞VCA₀Whether or not to satisfy simultaneously, if so, then S_j-1Is determined as S₀The optimal stereo image with improved visual comfort is obtained; otherwise, directly executing the step 4_ 2; wherein, STFA_j-1S obtained when the number of steps representing translation is j-1₀Corresponding redrawn three-dimensional image S_j-1Is the visual satisfaction value of (1), STFA when j is 2_j-2Is STFA₀And when j is more than 2 and less than or equal to N, STFA_j-2S obtained when the number of steps representing translation is j-2₀Corresponding redrawn three-dimensional image S_j-2A visual satisfaction value of;

step 4_ 2: let j equal j +1, then repeat step 1_2 to step 1_5 and step 3_2 and step 4_ 1; wherein, j is the assigned symbol in j + 1.

In the step 1_1, the first step,

wherein，DR₀(x, y) denotes DR₀The pixel value DR of the pixel point with the middle coordinate position (x, y)₀(x, y) is also SR₀The parallax value of the pixel point with the middle coordinate position (x, y) and the screen width W of the 3D display for displaying the three-dimensional image_3DAnd a screen height of the 3D display for displaying the stereoscopic image is H_3D。

In step 2_5, the specific process of obtaining the average subjective evaluation mean value of the 3D visual satisfaction of each stereo image in the training set comprises:

step 2_5 a: selecting more than 10 testees containing gender of male and female to participate in subjective experiments, wherein the visual system of each tester meets the condition that the stereoscopic vision is less than 60 arcsec and passes the color test;

step 2_5 b: making a distance between each subject and a 3D display for displaying a stereoscopic image 3 times a screen height of the 3D display, and enabling each subject to view the stereoscopic image displayed on the 3D display in head-up; informing each subject of a scoring standard, wherein the scoring standard is two subjective visual perceptions of visual comfort and perception depth of the comprehensive stereo image, and scoring on the whole;

step 2_5 c: enabling the 3D display to sequentially display each three-dimensional image in the training set, and displaying for 2-3 times in a co-circulation mode, enabling the display time of each three-dimensional image on the 3D display to be 5-10 seconds, enabling the 3D display to display the next three-dimensional image after the testee has a rest for 3-10 seconds, and enabling the 3D display to display a middle gray image during the rest of the testee; each subject looks up to view the stereoscopic images displayed on the 3D display, and when each stereoscopic image in the training set is displayed in turn in the last pass, each subject scores the 3D visual satisfaction level of the stereoscopic image just displayed on the 3D display during the rest period according to the absolute category rating in the international standards ITU-T p.910 and ITU-T p.911, 5 points represent "very satisfactory", 4 points represent "satisfactory", 3 points represent "normal", 2 points represent "unsatisfactory", 1 point represents "very unsatisfactory";

step 2_5 d: after each subject scores the 3D visual satisfaction level of each stereo image in the training set, removing abnormal scores and keeping normal scores according to a screening method disclosed in the international standard ITU-R BT.500-11;

step 2_5 e: and taking the average value of all normal scores of the stereo images as the average subjective score average value of the 3D visual satisfaction of the stereo images aiming at each stereo image in the training set.

In the step 2_7, APDA_i,jThe acquisition process comprises the following steps:

step 2_7 a: for VD_i,jThe values of all elements in the sequence are sorted from big to small or from small to big; then, the smallest top 1% value is removed from the values of all the sorted elements; finding out the minimum value from the values of all the elements left, and recording the minimum value as

Step 2_7 b: computing

In the step 2_8, RPDA_i,jThe acquisition process comprises the following steps:

step 2_8 a: for VD_i,jThe values of all elements in the sequence are sorted from big to small or from small to big; then, the smallest top 1% value and the largest top 1% value are removed from the values of all the sorted elements; finding out the minimum and maximum values from the values of all the elements left, and recording the minimum and maximum values as corresponding values

And

step 2_8 b: computing

Wherein e represents a natural base number, G_i,jRepresenting the perceptual relative enhancement coefficient(s),

to represent

The perceived distance obtained by the conversion is converted,

to represent

The perceived distance obtained by the conversion is converted,

when j is 0

The perceived distance obtained by the conversion is converted,

when j is 0

The perceived distance obtained by the conversion is converted,

compared with the prior art, the invention has the advantages that:

the method adopts the visual satisfaction objective evaluation model to guide redrawing of the three-dimensional image, so that the redrawn three-dimensional image is optimal, wherein the created visual satisfaction objective evaluation model integrates the visual comfort and perception depth quality evaluation of the three-dimensional image, and the satisfaction of the overall visual experience of the finally obtained redrawn three-dimensional image is optimal while the visual comfort is improved; in addition, the established viewing distance nonlinear translation method has the advantage of keeping the consistency of the geometric proportion of the virtual scene before and after the redrawing of the stereo image, namely, the sense of reality on the geometric proportion is kept, and the perceived depth quality is improved.

Drawings

FIG. 1 is a block diagram of an overall implementation of the method of the present invention;

FIG. 2 is a three-dimensional image S to be viewed with enhanced comfort₀Viewing distance VD of point with coordinate position (x, y) in formed virtual scene₀(x, y) and S₀Right view image SR₀Is a parallax image DR₀Screen parallax value DRS of pixel point with (x, y) as middle coordinate position₀(x, y), a geometric schematic diagram of the relationship between the distance T from the human eye to the 3D display and the binocular pupillary distance p of the human eye;

fig. 3 is a block diagram of an implementation of an objective evaluation model of the visual satisfaction degree of a stereoscopic image in the method of the present invention.

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

The invention provides a method for improving the visual comfort of a three-dimensional image by combining perceived depth quality, which has the overall realization block diagram as shown in figure 1 and is characterized by comprising four processes of three-dimensional image redrawing based on viewing distance nonlinear translation, modeling of a three-dimensional image visual satisfaction objective evaluation model, objective evaluation of the visual satisfaction and the visual comfort of the three-dimensional image and optimal three-dimensional image discrimination.

The method comprises the following specific steps of:

step 1_ 1: will have a width of W_imageAnd has a height H_imageThe stereo image to be improved in visual comfort is recorded as S₀Will S₀Left view image and right view image ofIs correspondingly denoted as SL₀And SR₀(ii) a Then obtain the SR₀Is recorded as DR₀(ii) a Then calculating DR₀The screen parallax value of each pixel point in (1) is calculated as DR₀The screen parallax value of the pixel point with the middle coordinate position (x, y) is recorded as DRS₀(x, y); then according to DR₀Calculating S as the screen parallax value of each pixel point₀The distance between each point in the formed virtual scene and the human eye, i.e. calculating S₀Viewing distance, S, of each point in the formed virtual scene₀The viewing distance of a point with a coordinate position (x, y) in the formed virtual scene is recorded as VD₀(x,y)，

Finally, the S is₀The matrix formed by the viewing distances of all the points in the virtual scene is denoted as VD₀(ii) a Wherein x represents the column coordinate of the pixel point, y represents the row coordinate of the pixel point, x is more than or equal to 0 and is more than W_image，0≤y＜H_imageP denotes a binocular pupillary distance of human eyes, and T denotes a distance from the human eyes looking down to a 3D display for displaying a stereoscopic image.

In this embodiment, in step 1_1,

wherein, DR₀(x, y) denotes DR₀The pixel value DR of the pixel point with the middle coordinate position (x, y)₀(x, y) is also SR₀The parallax value of the pixel point with the middle coordinate position (x, y) and the screen width W of the 3D display for displaying the three-dimensional image_3DAnd a screen height of the 3D display for displaying the stereoscopic image is H_3D。

Here, DR₀Obtaining by the prior art; fig. 2 shows a stereo image S to be visually enhanced in comfort₀Viewing distance VD of point with coordinate position (x, y) in formed virtual scene₀(x, y) and S₀Right view image SR₀Is a parallax image DR₀Screen parallax value DRS of pixel point with (x, y) as middle coordinate position₀(x, y) head-up to the human eyeA geometrical schematic diagram of the relationship between the distance T down to the 3D display, the binocular pupillary distance p of the human eye.

represents VD₀Minimum value of (1).

DR_j(x, y) is also the parallax value of the pixel point with the coordinate position (x, y) in the right viewpoint image of the new stereo image, and the screen width of the 3D display for displaying the stereo image is W_3DAnd a screen height of the 3D display for displaying the stereoscopic image is H_3D。

Step 1_ 4: calculating each pixel point and SL in the left viewpoint image of the new stereoscopic image when j is more than or equal to 1 and less than or equal to N₀The relation between the corresponding pixel points on the column coordinates, the pixel point with the coordinate position of (x, y) in the left viewpoint image of the new stereo image and the SL₀The relation between the pixel points with the (x, y) middle coordinate position on the column coordinate is recorded as Δ L (x, y), and Δ L (x, y) is (DR)₀(x,y)-DR_j(x, y))/2; also, each pixel point and SR in the right view image of the new stereoscopic image are calculated₀The relation between the corresponding pixel points on the column coordinates, the pixel point with the coordinate position of (x, y) in the right viewpoint image of the new three-dimensional image and the SR₀The relation between the pixel points with the (x, y) middle coordinate position on the column coordinate is recorded as Δ R (x, y), and Δ R (x, y) — (DR)₀(x,y)-DR_j(x, y))/2; wherein, DR₀(x, y) denotes DR₀The pixel value DR of the pixel point with the middle coordinate position (x, y)₀(x, y) is also SR₀And the middle coordinate position is the parallax value of the pixel point of (x, y).

step 1_5 a: SR'_jRepresents SR₀If x + delta R (x, y) < W is more than or equal to 0, the right viewpoint image obtained after the first redrawing_imageThen SR 'is calculated'_jThe pixel value of the pixel point with the middle coordinate position of (x, y) is recorded as SR'_j(x,y)，SR'_j(x,y)＝SR₀(x + Δ R (x, y), y); if x + Δ R (x, y) < 0 or x + Δ R (x, y) ≧ W_imageThen SR'_jDetermining the pixel point with the middle coordinate position (x, y) as a cavity pixel point; wherein, SR₀(x + Δ R (x, y), y) represents SR₀The middle coordinate position is the pixel value of the pixel point of (x + Δ R (x, y), y).

Step 1_5 b: SR 'judgment'_jIf the hollow pixel point exists in the buffer, SR 'is carried out'_jAnew noted SR'_jThen, step 1_5c is performed; if the empty pixel exists, each empty pixel is processed, and SR 'is set'_jIf the pixel point with the middle coordinate position (x, y) is a cavity pixel point, then x + DR is more than or equal to 0₀(x,y)+ΔR(x,y)＜W_imageCalculating the redrawn pixel value of the hole pixel point to be equal to SL₀(x+DR₀(x, y) + Δ R (x, y), y), when x + DR₀(x, y) + Δ R (x, y) < 0 or x + DR₀(x,y)+ΔR(x,y)≥W_imageThe hollow pixel points are not redrawn, and an SR is obtained after all the hollow pixel points are processed₀The right viewpoint image obtained after the second redrawing is marked as SR "_jThen, step 1_5c is performed; wherein SL₀(x+DR₀(x, y) + Δ R (x, y), y) represents SL₀The middle coordinate position is (x + DR)₀(x, y) + Δ R (x, y), y).

Step 1_5 c: line of SL'_jRepresents SL₀If x-DR is not less than 0, the left viewpoint image obtained after the first redrawing_j(x,y)＜W_imageThen calculate SL'_jThe pixel value of the pixel point with the middle coordinate position of (x, y) is recorded as SL'_j(x,y)，SL'_j(x,y)＝SR”_j(x-DR_j(x, y), y); if x-DR_j(x, y) < 0 or x-DR_j(x,y)≥W_imageThen SL 'is prepared'_jDetermining the pixel point with the middle coordinate position (x, y) as a cavity pixel point; wherein, SR "_j(x-DR_j(x, y), y) represents SR "_jThe middle coordinate position is (x-DR)_j(x, y), y) pixel values of the pixels.

Step 1_5 d: judging SL'_jIf there is a hole pixel point, if there is no hole pixel point, SL'_jAnew as SL "_jThen, step 1_5e is performed; if the signal exists, processing each hollow pixel point, and setting SL'_jIf the pixel point with the middle coordinate position (x, y) is a cavity pixel point, then when x + delta L (x, y) < W is more than or equal to 0_imageCalculating the redrawn pixel value of the hole pixel point to be equal to SL₀(x + Δ L (x, y), y) when x + Δ L (x, y) < 0 or x + Δ L (x, y) ≧ W_imageThe hollow pixel points are not redrawn, and SL is obtained after all the hollow pixel points are processed₀The left viewpoint image obtained after the second redrawing is marked as SL "_jThen, step 1_5e is performed; wherein SL₀(x + Δ L (x, y), y) represents SL₀The middle coordinate position is the pixel value of the pixel point of (x + Δ L (x, y), y).

Step 1_5 e: if SL "_jIf no hollow pixel point exists in the interface, SL is used "_jAs SL_j(ii) a If SR'_jIf no hollow pixel exists in the image, then SR is carried out "_jAs SR_j(ii) a If SL "_jAnd SR'_jIf there are still hollow pixel points, then use OpenCV library or other existing image repairing technique to repair each hollow pixel point, and use SL "_jThe left viewpoint image obtained after the patching is used as SL_jIs prepared from SR "_jThe right viewpoint image obtained after patching is used as SR_j。

As shown in fig. 3, the specific steps of the modeling part of the objective evaluation model for the visual satisfaction of the stereoscopic image are as follows:

step 2_ 1: selecting M pieces of width W_imageAnd has a height H_imageThe stereo images of (a) form a stereo image set; wherein M is a positive integer, and M is more than or equal to 20.

Step 2_ 2: defining the ith stereo image to be processed currently in the stereo image set as a current stereo image; wherein i is a positive integer, and i is more than or equal to 1 and less than or equal to M.

Step 2_ 3: recording the current stereo image as phi_i,0(ii) a Then changing the value of j from 1 to N, repeatedly executing the j values for 1 time in the same way according to the processes from step 1_1 to step 1_5, and repeatedly executing the j values from 1 to N for N times to obtain phi_i,0Corresponding N redrawing three-dimensional images, and obtaining phi when the number of translation steps is j_i,0The corresponding redrawn stereo image is recorded as phi_i,jAt the same time obtainTo phi_i,0A matrix VD of viewing distances of all points in the formed virtual scene_i,0Viewing distances of all points in the new virtual scene corresponding to each redrawn stereo image are formed into a matrix, and phi is calculated_i,jThe matrix formed by the viewing distances of all the points in the corresponding new virtual scene is recorded as VD_i,j。

Step 2_ 4: taking the next to-be-processed stereo image in the stereo image set as a current stereo image by setting i to i + 1; then, returning to the step 2_3 to continue executing until N redrawn three-dimensional images corresponding to each three-dimensional image in the three-dimensional image set are obtained, and forming a training set by all three-dimensional images in the three-dimensional image set and the obtained M multiplied by N redrawn three-dimensional images; wherein, in i +1, "═ is an assigned symbol.

Step 2_ 5: obtaining an average subjective evaluation mean value of 3D visual satisfaction of each three-dimensional image in a training set; then, the set formed by the mean subjective evaluation average of the 3D visual satisfaction of all the stereo images in the training set is recorded as { VSMOS_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; VSMOS when j is 0_i,0Represents phi_i,0The average subjective evaluation mean value of 3D visual satisfaction is that when j is more than or equal to 1 and less than or equal to N, VSMOS_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe average subjective evaluation mean value of the 3D visual satisfaction degree comprises the comprehensive effect of the visual comfort degree and the perception depth.

In this embodiment, in step 2_5, the specific process of obtaining the average subjective evaluation mean of the 3D visual satisfaction of each stereo image in the training set includes:

step 2_5 a: more than 10 subjects including gender were selected to participate in the subjective experiment, each subject's visual system satisfied stereoscopic vision less than 60 arcsec and passed the color test.

Step 2_5 b: making a distance between each subject and a 3D display for displaying a stereoscopic image 3 times a screen height of the 3D display, and enabling each subject to view the stereoscopic image displayed on the 3D display in head-up; and informing each subject of a scoring standard, wherein the scoring standard is two subjective visual perceptions of visual comfort and perception depth of the comprehensive stereo image, and scoring on the whole.

Step 2_5 c: enabling the 3D display to sequentially display each three-dimensional image in the training set, and displaying for 2-3 times in a co-circulation mode, enabling the display time of each three-dimensional image on the 3D display to be 5-10 seconds, enabling the 3D display to display the next three-dimensional image after the testee has a rest for 3-10 seconds, and enabling the 3D display to display a middle gray image during the rest of the testee; each subject looks up to view the stereoscopic images displayed on the 3D display, and when each stereoscopic image in the training set is displayed in turn on the last pass, each subject scores the 3D visual satisfaction level of the stereoscopic image just displayed on the 3D display during the rest period according to the absolute category rating in the international standards ITU-T p.910 and ITU-T p.911, 5 scores represent "very satisfactory", 4 scores represent "satisfactory", 3 scores represent "normal", 2 scores represent "unsatisfactory", and 1 score represents "very unsatisfactory".

Here, it is sufficient that (M +1) × N stereoscopic images are circularly displayed for 2-3 times, and if the number of times of circular display is too many, visual fatigue of a subject is easily caused, so that the problem of inaccuracy exists in the last time of scoring; the display time of each stereoscopic image on the 3D display may be determined according to the size of the stereoscopic image, and the display time of the large-sized stereoscopic image may be generally set to be longer, for example, to be 10 seconds.

Step 2_5 d: after each subject scored the 3D visual satisfaction rating of each stereoscopic image in the training set, the abnormal scores were removed and the normal scores were retained according to the screening method disclosed in international standard ITU-R bt.500-11.

Step 2_ 6: calculating the objective evaluation value of the visual comfort degree of each three-dimensional image in the training set according to the objective evaluation model of the visual comfort degree; then look at all stereo images in the training setThe set of objective evaluation values of perceptual comfort is denoted as { VCA }_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; where, when j is 0, VCA_i,0Represents phi_i,0The objective evaluation value of visual comfort is VCA when j is more than or equal to 1 and less than or equal to N_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe visual comfort of the subject is evaluated objectively.

Here, the objective evaluation model of visual comfort was derived from the thesis: jung, h.sohn, s.lee, h.w.park, and y.m.ro. "compressing Visual display comfort of Stereoscopic Images using human attention Model", IEEE Transactions on Circuits & Systems for Video Technology, vol.23, No.12, pp.2077-2082,2013 (published in journal of circuit and system of Video Technology, volume 23, phase 12, page 2077 and 2082,2013, by the institute of electrical and electronics engineers, using the attention Model to predict the Visual comfort of Stereoscopic Images).

Step 2_ 7: according to the set { VD_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N, and calculating the objective evaluation value of the perceived absolute distance of each three-dimensional image in the training set; then, the set of objective evaluation values of perceived absolute distance of all stereo images in the training set is recorded as { APDA_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; wherein, VD is when j is more than or equal to 1 and less than or equal to N_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jAPDA when j is 0_i,0Represents phi_i,0The perceived absolute distance objective evaluation value of (1) is more than or equal to j and less than or equal to N, and APDA_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe perceived absolute distance objective evaluation value of (1).

In this embodiment, in step 2_7, APDA_i,jThe acquisition process comprises the following steps:

Step 2_7 b: computing

Step 2_ 8: according to the set { VD_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N, and calculating the objective evaluation value of the perception relative distance of each three-dimensional image in the training set; then, a set of objective evaluation values of perceptual relative distances of all stereo images in the training set is recorded as { RPDA_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; wherein, when j is 0, RPDA_i,0Represents phi_i,0The perceived relative distance objective evaluation value of (1) is more than or equal to j and less than or equal to N, and the RPDA is used for evaluating the distance_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe perceived relative distance of (1) is objectively evaluated.

In this embodiment, in step 2_8, RPDA_i,jThe acquisition process comprises the following steps:

And

step 2_8 b: computing

to represent

The perceived distance obtained by the conversion is converted,

to represent

The perceived distance obtained by the conversion is converted,

when j is 0

The perceived distance obtained by the conversion is converted,

when j is 0

The perceived distance obtained by the conversion is converted,

step 2_ 9: adopts a classical training model of epsilon-SVR in VSMOS_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N and { VCA_i,j|1≤i≤M,0≤j≤N}、{APDA_i,j|1≤i≤M,0≤j≤N}、{RPDA_i,jFitting is carried out between i is more than or equal to 1 and less than or equal to M and j is more than or equal to 0 and less than or equal to N to obtain the objective evaluation model of the visual satisfaction degree of the stereo image, and the description is as follows: STFA (VCA, APDA, RPDA); wherein, trainingThe kernel function of the model epsilon-SVR is a histogram interaction type, the width coefficient gamma is 1/256, the penalty coefficient C is 4, the insensitive loss coefficient epsilon is 0.1, STFA () is a function representation form of a stereoscopic image visual satisfaction objective evaluation model, VCA, APDA and RPDA are all input of the stereoscopic image visual satisfaction objective evaluation model, VCA is used for representing a visual comfort objective evaluation value of a stereoscopic image, APDA is used for representing a perception absolute distance objective evaluation value of the stereoscopic image, and RPDA is used for representing a perception relative distance objective evaluation value of the stereoscopic image.

step 3_ 1: s is obtained in the same manner as the procedure of step 2_6_jObjective evaluation value of visual comfort of (1), noted as VCA_j(ii) a And obtains S in the same manner as the procedure of step 2_7_jIs recorded as APDA_j(ii) a S is obtained in the same manner as the procedure of step 2_8_jThe objective evaluation value of the perceived relative distance is marked as RPDA_j(ii) a Then VCA_j、APDA_j、RPDA_jInputting the three-dimensional image visual satisfaction objective evaluation model obtained in the step 2-9 to obtain S_jIs recorded as STFA_j，STFA_j＝STFA(VCA_j,APDA_j,RPDA_j) (ii) a Wherein j is more than or equal to 0 and less than or equal to N.

step 4_ 1: directly performing step 4_2 when j is 0 or j is 1; when j is more than or equal to 2 and less than or equal to N, the pointer points to S_j-1Judging STFA_j-1＞STFA_j-2、STFA_j-1＞STFA_j、VCA_j-1＞VCA₀Whether or not to satisfy simultaneously, if so, then S_j-1Is determined as S₀The optimal stereo image with improved visual comfort is obtained; otherwise, directly executing the step 4_ 2; wherein, STFA_j-1S obtained when the number of steps representing translation is j-1₀Corresponding redrawn three-dimensional image S_j-1Is the visual satisfaction value of (1), STFA when j is 2_j-2Is thatSTFA₀And when j is more than 2 and less than or equal to N, STFA_j-2S obtained when the number of steps representing translation is j-2₀Corresponding redrawn three-dimensional image S_j-2A visual satisfaction value of.

For S₀Setting the appropriate step length for redrawing, automatically calculating and storing the three-dimensional image S after each redrawing_jObjective visual comfort evaluation value VCA_jAnd visual satisfaction value STFA_j(ii) a If STFA_jIs a maximum point and VCA_jGreater than VCA₀Then the step number is not increased, and S is the same_jIs the optimal stereo image, i.e. S₀The optimal result after the visual comfort is improved; if STFA_jIf the maximum value point is not reached, the step number continues to increase.

To further illustrate the feasibility and effectiveness of the method of the present invention, experiments were conducted on the method of the present invention.

Here, taking a stereo image database provided by korea scientific technical institute image and video systems laboratory (IVY LAB) as an example, the stereo image database includes 120 stereo images and parallax images of right viewpoint images of each stereo image, includes indoor and outdoor stereo images of various scene depths, and gives an average subjective evaluation mean of 3D visual satisfaction of each stereo image.

In this experiment, 22 stereo images in which the mean subjective evaluation of 3D visual satisfaction in the stereo image database was lower than 3 points were selected, and the numbers were: 2. 28, 29, 30, 32, 33, 35, 39, 46, 47, 49, 50, 51, 52, 53, 55, 70, 73, 74, 101, 102, 103. And respectively redrawing the 22 original stereo images based on the stereo images subjected to nonlinear translation based on the viewing distance, wherein the step length is set to be 100mm, the step number is 12 steps, obtaining 22 multiplied by 12 to 264 redrawn stereo images in total, and forming a stereo image set by 286 stereo images of the 22 original stereo images and the 264 redrawn stereo images in total.

In order to verify the performance of the stereoscopic image visual satisfaction objective evaluation model in the method, 18 original stereoscopic images and 234 redrawn stereoscopic images corresponding to the original stereoscopic images are randomly selected from a stereoscopic image set to form a training set, the remaining 4 original images in the stereoscopic image set and 52 redrawn stereoscopic images corresponding to the original stereoscopic images form a test set, and 7315 times of cross verification are needed. 4 common objective parameters of the evaluation method for evaluating the image quality are used as evaluation indexes, namely Pearson correlation coefficient (PLCC), Spearman correlation coefficient (SROCC), Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) under the condition of nonlinear regression. PLCC reflects the correlation between objective evaluation results and subjective evaluation values; SROCC reflects monotonicity and consistency of objective evaluation results and subjective evaluation values; MAE and RMSE reflect the accuracy of the objective evaluation results. The method of the invention is utilized to obtain the visual satisfaction value of each three-dimensional image in the three-dimensional image set, and the five-parameter Logistic function nonlinear fitting is carried out on the visual satisfaction values of the 286 three-dimensional images, and the higher the PLCC and SROCC values are, and the smaller the MAE and RMSE values are, the better the correlation between the objective evaluation result of the three-dimensional image visual satisfaction objective evaluation model and the average subjective evaluation mean value is. Table 1 shows performance evaluation index values of the stereoscopic image visual satisfaction objective evaluation model.

TABLE 1 evaluation of performance index values

Performance evaluation index	PLCC	SROCC	MAE	RMSE
					Index value	0.9439	0.9349	0.2643	0.3308

As can be seen from table 1, the objective evaluation result obtained by using the stereoscopic image visual satisfaction objective evaluation model is consistent with the result of subjective perception of human eyes, and the effectiveness of the stereoscopic image visual satisfaction objective evaluation model is well illustrated.

Claims

1. A stereoscopic image visual comfort level improving method combining perceived depth quality is characterized by comprising four processes of stereoscopic image redrawing based on viewing distance nonlinear translation, modeling of a stereoscopic image visual satisfaction degree objective evaluation model, objective evaluation of the visual satisfaction degree and the visual comfort level of a stereoscopic image and optimal stereoscopic image discrimination;

the method comprises the following specific steps of:

step 1_ 1: will have a width of W_imageAnd has a height H_imageThe stereo image to be improved in visual comfort is recorded as S₀Will S₀The left view image and the right view image are correspondingly denoted as SL₀And SR₀(ii) a Then obtain the SR₀Is recorded as DR₀(ii) a Then calculating DR₀The screen parallax value of each pixel point in (1) is calculated as DR₀The screen parallax value of the pixel point with the middle coordinate position (x, y) is recorded as DRS₀(x, y); then according to DR₀Calculating S as the screen parallax value of each pixel point₀The distance between each point in the formed virtual scene and the human eye, i.e. calculating S₀Viewing distance, S, of each point in the formed virtual scene₀Viewing of a point in the virtual scene formed with a coordinate position of (x, y)The distance is recorded as VD₀(x,y)，

represents VD₀Minimum value of (1);

step 1_ 3: j is more than or equal to 1 and less than or equal to N according to VD_jCalculating the screen parallax value of each pixel point in the parallax image of the right viewpoint image of the new three-dimensional image corresponding to the new virtual scene, and converting the right viewpoint image of the new three-dimensional image into the screen parallax valueThe screen parallax value of the pixel point with the coordinate position (x, y) in the parallax image is recorded as DRS_j(x,y)，

DR_j(x, y) is also the parallax value of the pixel point with the coordinate position (x, y) in the right viewpoint image of the new stereo image, and the screen width of the 3D display for displaying the stereo image is W_3DAnd a screen height of the 3D display for displaying the stereoscopic image is H_3D；

step 2_ 3: recording the current stereo image as phi_i,0(ii) a Then changing the value of j from 1 to N, repeatedly executing the j values for 1 time in the same way according to the processes from step 1_1 to step 1_5, and repeatedly executing the j values from 1 to N for N times to obtain phi_i,0Corresponding N redrawing three-dimensional images, and obtaining phi when the number of translation steps is j_i,0The corresponding redrawn stereo image is recorded as phi_i,jWhile obtaining phi_i,0A matrix VD of viewing distances of all points in the formed virtual scene_i,0Viewing distances of all points in the new virtual scene corresponding to each redrawn stereo image are formed into a matrix, and phi is calculated_i,jThe matrix formed by the viewing distances of all the points in the corresponding new virtual scene is recorded as VD_i,j；

step 2_ 6: calculating the objective evaluation value of the visual comfort degree of each three-dimensional image in the training set according to the objective evaluation model of the visual comfort degree; then look at all stereo images in the training setThe set of objective evaluation values of perceptual comfort is denoted as { VCA }_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; where, when j is 0, VCA_i,0Represents phi_i,0The objective evaluation value of visual comfort is VCA when j is more than or equal to 1 and less than or equal to N_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe objective evaluation value of the visual comfort of the user;

step 2_ 7: according to the set { VD_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N, and calculating the objective evaluation value of the perceived absolute distance of each three-dimensional image in the training set; then, the set of objective evaluation values of perceived absolute distance of all stereo images in the training set is recorded as { APDA_i,jI is more than or equal to 1 and less than or equal to M, and j is more than or equal to 0 and less than or equal to N }; wherein, VD is when j is more than or equal to 1 and less than or equal to N_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jAPDA when j is 0_i,0Represents phi_i,0The perceived absolute distance objective evaluation value of (1) is more than or equal to j and less than or equal to N, and APDA_i,jPhi obtained when the number of steps representing translation is j_i,0Corresponding redrawn stereo image phi_i,jThe perceived absolute distance objective evaluation value of (1);

step 2_ 9: using a training model of ε -SVR in VSMOS_i,jI is more than or equal to 1 and less than or equal to M, j is more than or equal to 0 and less than or equal to N and { VCA_i,j|1≤i≤M,0≤j≤N}、{APDA_i,j|1≤i≤M,0≤j≤N}、{RPDA_i,jFitting is carried out between i is more than or equal to 1 and less than or equal to M and j is more than or equal to 0 and less than or equal to N to obtain the objective evaluation model of the visual satisfaction degree of the stereo image, and the description is as follows: STFA (VCA, APDA, RPDA);the kernel function of the training model epsilon-SVR uses a histogram interaction type, a width coefficient gamma is 1/256, a penalty coefficient C is 4, an insensitive loss coefficient epsilon is 0.1, STFA () is a function representation form of a stereoscopic image visual satisfaction objective evaluation model, VCA, APDA and RPDA are all input of the stereoscopic image visual satisfaction objective evaluation model, VCA is used for representing a visual comfort objective evaluation value of a stereoscopic image, APDA is used for representing a perception absolute distance objective evaluation value of the stereoscopic image, and RPDA is used for representing a perception relative distance objective evaluation value of the stereoscopic image;

2. The method for improving the visual comfort of a stereoscopic image with combination of perceived depth quality as claimed in claim 1, wherein in step 1_1,

3. The method according to claim 1 or 2, wherein in step 2_5, the specific process of obtaining the mean subjective evaluation of the 3D visual satisfaction of each stereoscopic image in the training set comprises:

4. The method according to claim 3, wherein in step 2_7, APDA is used for improving the visual comfort of the stereoscopic image with the combination of perceived depth quality_i,jThe acquisition process comprises the following steps:

Step 2_7 b: computing

5. The method according to claim 3, wherein in step 2_8, RPDA is used for improving the visual comfort of the stereoscopic image with the combination of perceived depth quality_i,jThe acquisition process comprises the following steps: