CN104822058B - A kind of stereo-picture notable figure extracting method - Google Patents

Abstract

The invention discloses a kind of stereo-picture notable figure extracting method, it is in the training stage, extract the contrast metric vector in each region in the right visual point image of several training stereo-pictures, generic features vector background priori features vector, and merge the characteristic vector for reflecting vision significance in each region obtained in the right visual point image of every width stereo-picture, set up characteristic vector and random forest regression training model that average eye is moved between value；At test phase, calculate the characteristic vector for reflecting vision significance in each region in the right visual point image of test stereo-picture, and according to training the random forest regression training model obtained, it was predicted that obtain testing the three-dimensional notable figure of stereo-picture；Advantage is the notable situation of change that extracted feature can preferably reflect various factors, thus is effectively improved the forecasting accuracy of vision saliency value.

Description

Stereo image saliency map extraction method

Technical Field

The present invention relates to a method for processing an image signal, and in particular, to a method for extracting a saliency map of a stereoscopic image.

Background

In human visual reception and information processing, due to limited brain resources and difference in importance of external environment information, human brain does not have the same sense as external environment information but shows selective characteristics in the processing process. People are not evenly focused on every area of an image when watching the image or video clip, but are more focused on some salient areas. How to detect and extract the salient regions with high visual attention in the video is an important research content in the field of computer vision and content-based video retrieval.

However, the process of generating stereoscopic vision by sensing a stereoscopic image by human eyes is not a simple process of superimposing a left viewpoint image and a right viewpoint image, so that stereoscopic vision features (such as three-dimensional visual attention) are not simple expansion of plane visual characteristics, and how to effectively extract the stereoscopic vision features from the stereoscopic image and how to make the extracted stereoscopic vision features conform to three-dimensional viewing behaviors of human eyes is a problem which needs to be researched and solved in the process of extracting a visual saliency map from the stereoscopic image.

Disclosure of Invention

The invention aims to solve the technical problem of providing a method for extracting a three-dimensional image saliency map, which accords with the saliency semantic features and has stronger extraction stability and higher extraction accuracy.

The technical scheme adopted by the invention for solving the technical problems is as follows: a method for extracting a three-dimensional image saliency map is characterized by comprising a training stage and a testing stage, wherein the training stage comprises the following specific steps:

① -1, the selected N sets of different stereo images and the right parallax image of each stereo image form a set, which is marked as { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, wherein N is more than or equal to 1, and L_iRepresents { L_i,R_i,d_iL 1. ltoreq. i. ltoreq.N } left viewpoint image of ith stereoscopic image, R_iRepresents { L_i,R_i,d_iI is not less than 1 and not more than N, d_iRepresents { L_i,R_i,d_iI is not less than 1 and not more than N, and right parallax images of the ith stereo image;

① -2, using superpixel splitting technique to divide { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, the right viewpoint image of each stereo image is divided into M non-overlapping areas, R is_iThe h-th area in (1) is denoted as SP_i,hWherein M is more than or equal to 1, h is more than or equal to 1 and less than or equal to M;

① -3, calculation of { L_i,R_i,d_iI is not less than 1 and not more than N, R is the contrast characteristic vector of each region in the right viewpoint image of each stereoscopic image_iH-th area SP of (1)_i,hIs recorded as a contrast feature vector $X_{i,h}^c=[d_{i,h}{,d}_{i,h}^1,d_{i,h}^2,d_{i,h}^3,d_{i,h}^4,d_{i,h}^5,d_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol, d_i,hRepresents SP_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the neighboring area,f_i,hhas a dimension of 20, f_i,hRepresents SP_i,hC mean value of the frequency response feature vectors of all the pixels in (1)_i,hHas a dimension of 9, c_i,hRepresents SP_i,hThe mean value of the color feature vectors of all the pixel points in (d)_i,hRepresents SP_i,hIs measured in the mean value of the parallax amplitude of (c),represents SP_i,hColor histograms of R, G and B components of all pixel points in RGB color space and SP_i,hThe distances of all pixel points in the adjacent regions of the color histogram of the R component, the G component and the B component of the RGB color space,represents SP_i,hThe color histograms of L component, a component and b component of all pixel points in CIELAB color space and SP_i,hThe distances of the color histograms of the L component, the a component and the b component of the CIELAB color space of all the pixel points in the neighboring region of (a),represents SP_i,hColor histogram and SP of H component of all pixel points in HVS color space_i,hAll pixel points in the neighboring region of (a) are at the distance of the color histogram of the H component of the HVS color space,represents SP_i,hColor histogram of S component of all pixel points in HVS color space and SP_i,hThe distance of all pixel points in the neighboring region of (a) in the color histogram of the S component of the HVS color space,represents SP_i,hLBP feature statistical histogram and SP of all pixel points in_i,hThe distance of the statistical histogram of the LBP features of all pixel points in the neighboring region,represents SP_i,hThe parallax statistical histogram and SP of all the pixel points in_i,hThe distance of the parallax statistical histogram of all the pixel points in the adjacent region, where the adjacent region is R_iNeutral SP_i,hAn adjacent region;

① -4, calculation of { L_i,R_i,d_iI is not less than 1 and not more than N, R is the general feature vector of each region in the right viewpoint image of each stereoscopic image_iH-th area SP of (1)_i,hIs given as Wherein,has a dimension of 33, here the symbol "[ 2 ]]"is a vector representing a symbol and,has a dimension of 20 a and has a high degree of,represents SP_i,hThe frequency response feature vector of all the pixel points inThe difference is that the number of the first and second,has a dimension of 9 a and has a high degree of,represents SP_i,hThe variance of the color feature vectors of all the pixel points in (a),represents SP_i,hOf the parallax amplitude, x_i,hHas a dimension of 2, x_i,hRepresents SP_i,hS of the center pixel point_i,hRepresents SP_i,hThe area of (d);

① -5, calculating { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, and R is equal to R_iH-th area SP of (1)_i,hIs recorded as $X_{i,h}^b=[e_{i,h},e_{i,h}^1,e_{i,h}^2,e_{i,h}^3,e_{i,h}^4,e_{i,h}^5,e_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol, e_i,hRepresents SP_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the background region,f_i,hhas a dimension of 20, f_i,hRepresents SP_i,hC mean value of the frequency response feature vectors of all the pixels in (1)_i,hHas a dimension of 9, c_i,hRepresents SP_i,hThe mean of the color feature vectors of all the pixel points in (a),represents SP_i,hIs measured in the mean value of the parallax amplitude of (c),represents SP_i,hThe color histogram of R component, G component and B component of all pixel points in RGB color space and R_iThe distances of all the pixel points in the background region in the color histogram of the R component, G component and B component of the RGB color space,represents SP_i,hThe color histograms of L component, a component and b component and R of all pixel points in CIELAB color space_iThe distances of the color histograms of the L component, the a component and the b component of the CIELAB color space of all the pixel points in the background area in (1),represents SP_i,hColor histogram and R of H component of all pixel points in HVS color space_iThe distance of all the pixel points in the background region in the color histogram of the H component of the HVS color space,represents SP_i,hColor histogram of S component of all pixel points in HVS color space and R_iThe distance of all pixel points in the background region in the color histogram of the S component of the HVS color space,represents SP_i,hLBP feature statistical histogram and R of all pixel points in_iThe distance of the statistical histogram of the LBP features of all the pixels in the background region in (1),represents SP_i,hThe parallax statistical histogram and R of all the pixel points in_iThe distance of the parallax statistical histogram of all the pixel points in the background region is R_iThe areas positioned at the leftmost side, the rightmost side, the uppermost side and the lowermost side;

① -6, will { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N), the contrast characteristic vector, the general characteristic vector and the background prior characteristic vector of each region in the right viewpoint image of each stereo image are arranged in sequence to form { L ≦_i,R_i,d_iI 1 ≦ i ≦ N } for each region in the right view image of each stereoscopic image, R is a feature vector for reflecting visual saliency_iH-th area SP of (1)_i,hThe feature vector for reflecting the visual saliency is marked as X_i,h，Wherein, X_i,hHas a dimension of 105, here the symbol "[ 2 ]]"is a vector representation symbol;

① -7, using random forest regression, on { L }_i,R_i,d_iI is not less than 1 and not more than N, training the feature vectors for reflecting the visual saliency of all the regions in the right viewpoint images of all the stereoscopic images, and minimizing the error between the regression function value obtained through training and the average eye movement value,obtaining an optimal random forest regression training model, and recording the optimal random forest regression training model as f (D)_inp) Wherein f () is a functional representation form, D_inpRepresenting an input vector of a random forest regression training model;

the specific steps of the test stage are as follows:

② -1, for any one test stereo image S_testWill S_testThe left viewpoint image, the right viewpoint image, and the right parallax image are expressed as L_test、R_test、d_test(ii) a Then adopting super pixel segmentation technique to divide R_testDividing into M non-overlapping regions, and dividing R_testThe h-th area in (1) is denoted as SP_h'; wherein M is more than or equal to 1, h is more than or equal to 1 and less than or equal to M;

② -2, following the procedure of step ① -3 through step ① -6, R is obtained in the same manner of operation_testFor each region of (a) to reflect visual saliency, R_testH-th area SP of (1)_h' the feature vector for reflecting the visual saliency is denoted as F_test,h(ii) a Then, training a model f (D) according to the optimal random forest regression obtained in the training stage_inp) Will F_test,hObtaining R as an input vector of an optimal random forest regression training model_testOf each region of (a), R_testH-th area SP of (1)_h' the three-dimensional visual saliency value is denoted as S_3D,h，S_3D,h＝f(F_test,h) (ii) a Then R is put_testThe three-dimensional visual saliency value of each region in the R-image is taken as the saliency value of all pixel points in the corresponding region, thereby obtaining R_testIs marked as { S_3D(x, y) }, wherein (x, y) here denotes S_testX is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent S_testWidth and height of (S)_3D(x, y) represents S_3DAnd the coordinate position in the (x, y) is the pixel value of the pixel point of (x, y).

R in the step ① -3_iH-th area SP of (1)_i,hContrast characteristic ofVectorThe acquisition process comprises the following steps:

a1, calculating R_iH-th area SP of (1)_i,hThe mean value of the frequency response characteristic vectors of all the pixel points is recorded as f_i,h，f_i,hTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsMean of the frequency response amplitudes of the elements, where f_i,hHas a dimension of 20 a and has a high degree of,

a2, calculating R_iH-th area SP of (1)_i,hThe mean value of the color feature vectors of all the pixel points in (1) is marked as c_i,h， $c_{i,h}=[{\stackrel{\‾}{R}}_{i,h},{\stackrel{\‾}{G}}_{i,h},{\stackrel{\‾}{B}}_{i,h},{\stackrel{\‾}{L}}_{i,h},{\stackrel{\‾}{a}}_{i.h},{\stackrel{\‾}{b}}_{i,h},{\stackrel{\‾}{H}}_{i,h},{\stackrel{\‾}{V}}_{i,h},{\stackrel{\‾}{S}}_{i,h}],$ Wherein, c_i,hHas a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the R components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the G components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the B components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average value of the color values of the L components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the a component of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hB component of all pixel points in CIELAB color spaceIs determined by the mean of the color values of (c),represents R_iH-th area SP of (1)_i,hThe average of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the V components of all the pixel points in the HVS color space,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the S components of all the pixel points in the HVS color space;

a3, calculating R_iH-th area SP of (1)_i,hIs the mean of the parallax amplitude ofIs equal to d_iNeutral SP_i,hThe mean value of the pixel values of all the pixel points in the corresponding region;

a4, mixing f_i,h、c_i,hAndarranged in sequence to form R_iH-th area SP of (1)_i,hFirst feature vector of (1), denoted as u_i,h，Wherein u is_i,hHas a dimension of 30, here the symbol "[ 2 ]]"is a vector representation symbol;

a5, calculating R_iH-th area SP of (1)_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the neighboring region, denoted d_i,h，Wherein d is_i,hHas a dimension of 30, p is more than or equal to 1 and less than or equal to M,represents R_iH-th area SP of (1)_i,hU of adjacent areas of the sequence number_i,pRepresents R_iP-th area SP in (1)_i,pThe symbol "|" is an absolute value symbol, and P represents R_iH-th area SP of (1)_i,hIs referred to as R_iNeutral SP_i,hAn adjacent region;

a6, calculating R_iH-th area SP of (1)_i,hThe color histograms of R, G and B components of all the pixel points in RGB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of the H component of all the pixel points in the HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of S component of all pixel points in HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe statistical histogram of the LBP characteristics of all the pixel points in (1) is recorded asCalculation of R_iH of (1)Region SP_i,hThe parallax statistical histogram of all the pixel points in (1) is recorded asWherein,has dimension of 16³，Has dimension of 16³，Has a dimension of 16 a and has a high degree of,has a dimension of 16 a and has a high degree of,has a dimension of 256, and has a high thermal conductivity,has a dimension of 16;

a7, calculatingAnd R_iH-th area SP of (1)_i,hThe distances of the color histograms of the R component, the G component and the B component of the RGB color space of all the pixel points in the adjacent areas are recorded as

ComputingAnd R_iH-th area SP of (1)_i,hAll pixel points in adjacent areas of (1) are in CIThe distance of the color histogram of the L, a and b components of the ELAB color space is noted

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the color histogram of the H component of the HVS color space of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the color histogram of the S component of the HVS color space of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the LBP feature statistical histogram of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the parallax statistical histogram of all the pixel points in the adjacent region is recorded as

Wherein p is more than or equal to 1 and less than or equal to M,represents R_iH-th area SP of (1)_i,hP represents R_iH-th area SP of (1)_i,hIs the chi () function of the chi-squared distance,represents R_iP-th area SP in (1)_i,pThe color histograms of the R, G and B components of all the pixel points in the RGB color space,represents R_iP-th area SP in (1)_i,pThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space,represents R_iP-th area SP in (1)_i,pThe color histogram of the H component of the HVS color space for all the pixel points in (a),represents R_iP-th area SP in (1)_i,pThe color histogram of the S component of the HVS color space for all the pixel points in (a),represents R_iP-th area SP in (1)_i,pThe statistical histogram of LBP features of all pixel points in (1),represents R_iP-th area SP in (1)_i,pThe parallax statistics histogram of all the pixel points in the image;

a8, d_i,h、Andarranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as the contrast feature vector of $X_{i,h}^b=[e_{i,h},e_{i,h}^1,e_{i,h}^2,e_{i,h}^3,e_{i,h}^4,e_{i,h}^5,e_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol.

R in the step ① -4_iH-th area SP of (1)_i,hIs a universal feature vectorThe acquisition process comprises the following steps:

b1, calculating R_iH-th area SP of (1)_i,hThe variance of the frequency response feature vectors of all the pixel points is recorded asTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsThe variance of the frequency response amplitude of the individual elements, wherein,has a dimension of 20 a and has a high degree of,

b2, calculating R_iH-th area SP of (1)_i,hThe variance of the color feature vectors of all the pixel points is recorded as ${\hat{c}}_{i,h}=[{\hat{R}}_{i,h},{\hat{G}}_{i,h},{\hat{B}}_{i,h},{\hat{L}}_{i,h},{\hat{a}}_{i,h},{\hat{b}}_{i,h},{\hat{H}}_{i,h},{\hat{V}}_{i,h},{\hat{S}}_{i,h}],$ Wherein,has a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe variance of the color values of the R component of the RGB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the G component of the RGB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the B component of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the L component of the CIELAB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the a component of the CIELAB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the b-component in CIELAB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the V component of the HVS color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the S component of all the pixel points in the HVS color space;

b3, calculating R_iH-th area SP of (1)_i，hThe variance of the parallax amplitude of (1), is recorded asIs equal to d_iNeutral SP_i,hVariance of pixel values of all pixel points in the corresponding region;

b4, obtaining R_iH-th area SP of (1)_i,hOf the central pixelCoordinate position of point, noted as x_i,hWherein x is_i,hHas a dimension of 2;

b5, calculating R_iH-th area SP of (1)_i,hArea of (d), denoted as s_i,h；

b6, willx_i,hAnd s_i,hArranged in sequence to form R_iH-th area SP of (1)_i,hIs a universal feature vector of $X_{i,h}^a=[{\hat{f}}_{i,h},{\hat{c}}_{i,h},{\hat{d}}_{i,h},x_{i,h},s_{i,h}],$ Wherein,has a dimension of 33, here the symbol "[ 2 ]]"is a vector representing a symbol.

R in the step ① -5_iH-th area SP of (1)_i,hBackground prior feature vector ofThe acquisition process comprises the following steps:

c1, calculating R_iH-th area SP of (1)_i,hThe Chinese herbal medicineThe mean value of the frequency response feature vectors with pixel points is recorded as f_i,h，f_i,hTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsMean of the frequency response amplitudes of the elements, where f_i,hHas a dimension of 20 a and has a high degree of,

c2, calculating R_iH-th area SP of (1)_i,hThe mean value of the color feature vectors of all the pixel points in (1) is marked as c_i,h， $c_{i,h}=[{\stackrel{\‾}{R}}_{i,h},{\stackrel{\‾}{G}}_{i,h},{\stackrel{\‾}{B}}_{i,h},{\stackrel{\‾}{L}}_{i,h},{\stackrel{\‾}{a}}_{i,h},{\stackrel{\‾}{b}}_{i,h},{\stackrel{\‾}{H}}_{i,h},{\stackrel{\‾}{V}}_{i,h},{\stackrel{\‾}{S}}_{i,h}],$ Wherein, c_i,hHas a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the R components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the G components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the B components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average value of the color values of the L components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the a component of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the b components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the V components of all the pixel points in the HVS color space,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the S components of all the pixel points in the HVS color space;

c3, calculating R_iH-th area SP of (1)_i,hIs the mean of the parallax amplitude ofIs equal to d_iNeutral SP_i,hThe mean value of the pixel values of all the pixel points in the corresponding region;

c4, mixing f_i,h、c_i,hAndarranged in sequence to form R_iH-th area SP of (1)_i,hFirst feature vector of (1), denoted as u_i,h，Wherein u is_i,hHas a dimension of 30, here the symbol "[ 2 ]]"is a vector representation symbol;

c5, calculating R_iH-th area SP of (1)_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the background region, denoted as e_i,h，Wherein e is_i,hHas a dimension of 30, q is more than or equal to 1 and less than or equal to M,represents R_iAll backgrounds in (1)Set of sequence numbers of regions, u_i,qRepresents R_iThe q-th region SP in (1)_i,qThe symbol "|" is an absolute value symbol, and Q represents R_iThe total number of background regions in (1), where the background region is R_iThe areas positioned at the leftmost side, the rightmost side, the uppermost side and the lowermost side;

c6, calculating R_iH-th area SP of (1)_i,hThe color histograms of R, G and B components of all the pixel points in RGB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of the H component of all the pixel points in the HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of S component of all pixel points in HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe statistical histogram of the LBP characteristics of all the pixel points in (1) is recorded asCalculation of R_iH-th area SP of (1)_i,hThe parallax statistical histogram of all the pixel points in (1) is recorded asWherein,has dimension of 16³，Has dimension of 16³，Has a dimension of 16 a and has a high degree of,has a dimension of 16 a and has a high degree of,has a dimension of 256, and has a high thermal conductivity,has a dimension of 16;

c7, calculationAnd R_iThe distances of the color histograms of the R, G and B components in the RGB color space of all the pixel points in the background region are recorded as

ComputingAnd R_iThe distances of the color histograms of the L component, the a component and the b component of all the pixel points in the background area in the CIELAB color space are recorded as the distances

ComputingAnd R_iThe distance of the color histogram of the H component of the HVS color space of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the color histogram of the S component of the HVS color space of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the LBP feature statistical histogram of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the parallax statistical histogram of all the pixel points in the background region is recorded as

Wherein q is more than or equal to 1 and less than or equal to M,represents R_iQ represents R_iThe total number of background regions, χ () is a chi-squared distance function,represents R_iThe q-th region SP in (1)_i,qThe color histograms of the R, G and B components of all the pixel points in the RGB color space,represents R_iThe q-th region SP in (1)_i,qThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space,represents R_iThe q-th region SP in (1)_i,qThe color histogram of the H component of the HVS color space for all the pixel points in (a),represents R_iThe q-th region SP in (1)_i,qThe color histogram of the S component of the HVS color space for all the pixel points in (a),represents R_iThe q-th region SP in (1)_i,qThe statistical histogram of LBP features of all pixel points in (1),represents R_iThe q-th region SP in (1)_i,qAll the pixels in (1)A disparity statistical histogram of points;

c8, mixing_i,h、 Arranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as the background prior feature vector of $X_{i,h}^b=[e_{i,h},e_{i,h}^1,e_{i,h}^2,e_{i,h}^3,e_{i,h}^4,e_{i,h}^5,e_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol.

Said R_iThe obtaining process of the frequency response characteristic vector of each pixel point comprises the following steps:

1) -1, using a Gabor filter pair R_iFiltering to obtain R_iEach pixel point in (1) is at a different center frequencyFrequency and amplitude of frequency response at different directional factors, R_iThe frequency response amplitude of the pixel point with the middle coordinate position (x, y) under the condition that the center frequency is omega and the direction factor is theta is marked as G (x, y; omega, theta), wherein (x, y) represents { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, x is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent { L ≦ N }_i,R_i,d_iI 1 ≦ i ≦ N }, ω represents the center frequency of the Gabor filter, ω ∈ Φ_ωθ represents the directional factor of the Gabor filter, θ ∈ Φ_θ，Φ_ωRepresenting the set of all centre frequencies of the Gabor filter, phi_θRepresents the set of all directional factors of the Gabor filter;

1) -2, reacting R_iThe frequency response amplitude of each pixel point in the R-array is arranged in sequence under different central frequencies and different direction factors to form R_iR, the frequency response feature vector of each pixel point in_iThe frequency response characteristic vector of the pixel point with the middle coordinate position (x, y) is marked as f_i(x, y) wherein f_iThe dimension of (x, y) is 20.

Said R_iThe obtaining process of the color feature vector of each pixel point comprises the following steps:

2) -1, calculating R_iThe color value of each pixel point in different color spaces is R_iThe color values of the R component, the G component and the B component of the pixel point with the (x, y) middle coordinate position in the RGB color space are respectively recorded as R (x, y), G (x, y) and B (x, y), and R (x, y) is recorded_iThe color values of the L component, the a component and the b component of the pixel point with the middle coordinate position (x, y) in the CIELAB color space are respectively recorded as L (x, y), a (x, y) and b (x, y), and R is used for indicating the color values of the L component, the a component and the b component in the CIELAB color space as L (x, y), a (x, y) and b (_iThe color values of the H component, the V component and the S component of the pixel point with the (x, y) middle coordinate position in the HVS color space are respectively recorded as H (x, y), V (x, y) and S (x, y), wherein (x, y) here represents { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, x is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent { L ≦ N }_i,R_i,d_iThe width and height of the stereoscopic image in the condition that i is more than or equal to 1 and less than or equal to N;

2) -2, reacting R_iThe color values of each pixel point in the different color spaces are arranged in sequence to form R_iThe color feature vector of each pixel point in (1), R_iThe color feature vector of the pixel point with the middle coordinate position (x, y) is marked as c_i(x,y)，c_i(x,y)＝[R(x,y),G(x,y),B(x,y),L(x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)]Wherein c is_iThe dimension of (x, y) is 9, where the symbol "[ 2 ],"]"is a vector representing a symbol.

Compared with the prior art, the invention has the advantages that:

1) the method simultaneously considers the contrast characteristic vector, the general characteristic vector and the background prior characteristic vector of each region in the right viewpoint image of the stereo image, and fuses to obtain the characteristic vector for reflecting the visual saliency of each region in the right viewpoint image of the stereo image, so the method has higher extraction accuracy and stronger stability, can better reflect the remarkable change condition of various factors, and conforms to the remarkable semantic characteristics.

2) The method of the invention constructs a random forest regression training model between the feature vector for reflecting the visual saliency and the average eye movement value through training, and then predicts the three-dimensional visual saliency value of each region in the right viewpoint image of the tested stereo image by using the random forest regression training model, thereby obtaining the three-dimensional saliency map of the tested stereo image and effectively improving the prediction accuracy of the visual saliency value.

Drawings

FIG. 1 is a block diagram of an overall implementation of the method of the present invention;

FIG. 2a is a right viewpoint Image of "Image 1";

FIG. 2b is a real eye diagram of the right viewpoint Image of "Image 1";

FIG. 2c is a three-dimensional saliency map of "Image 1";

FIG. 3a is a right viewpoint Image of "Image 2";

FIG. 3b is a real eye diagram of the right viewpoint Image of "Image 2";

FIG. 3c is a three-dimensional saliency map of "Image 2";

FIG. 4a is a right viewpoint Image of "Image 3";

FIG. 4b is a real eye diagram of the right viewpoint Image of "Image 3";

FIG. 4c is a three-dimensional saliency map of "Image 3";

FIG. 5a is a right viewpoint Image of "Image 4";

FIG. 5b is a real eye diagram of the right viewpoint Image of "Image 4";

FIG. 5c is a three-dimensional saliency map of "Image 4";

FIG. 6a is a right viewpoint Image of "Image 5";

FIG. 6b is a real eye diagram of the right viewpoint Image of "Image 5";

FIG. 6c is a three-dimensional saliency map of "Image 5";

FIG. 7a is a right viewpoint Image of "Image 6";

FIG. 7b is a real eye diagram of the right viewpoint Image of "Image 6";

FIG. 7c is a three-dimensional saliency map of "Image 6";

FIG. 8a is a right viewpoint Image of "Image 7";

FIG. 8b is a real eye diagram of the right viewpoint Image of "Image 7";

fig. 8c is a three-dimensional saliency map of "Image 7".

Detailed Description

The invention is described in further detail below with reference to the accompanying examples.

The overall implementation block diagram of the method for extracting the saliency map of the stereo image provided by the invention is shown in fig. 1, and the method comprises two processes of a training stage and a testing stage, wherein the specific steps of the training stage are as follows:

① -1, the selected N sets of different stereo images and the right parallax image of each stereo image form a set, which is marked as { L_i,R_i,d_iI is not less than 1 and not more than N, where N is not less than 1, and in this embodiment, N is 600, and L is_iRepresents { L_i,R_i,d_iL 1. ltoreq. i. ltoreq.N } left viewpoint image of ith stereoscopic image, R_iRepresents { L_i,R_i,d_iI is not less than 1 and not more than N, d_iRepresents { L_i,R_i,d_iAnd i is not less than 1 and not more than N.

In this embodiment, a training stereo image set is constructed using a three-dimensional human eye tracking database (NUS 3D-sales database) provided by singapore national university, which contains 600 pairs of stereo images and corresponding right parallax images, and gives a real eye-diagram for each pair of stereo images.

① -2, using existing superpixel splitting techniques_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, the right viewpoint image of each stereo image is divided into M non-overlapping areas, R is_iThe h-th area in (1) is denoted as SP_i,hCan be changed into { L_i,R_i,d_iI is not less than 1 and not more than N), the right viewpoint image of each stereo image is represented as a set of M regions, R is represented as a set of M regions_iThe set of M regions re-represented is denoted as SP_i,h}; wherein M is not less than 1, in this embodiment, M is 400, and h is not less than 1 and not more than M.

①-3Calculating { L }_i,R_i,d_iI is not less than 1 and not more than N, R is the contrast characteristic vector of each region in the right viewpoint image of each stereoscopic image_iH-th area SP of (1)_i,hIs recorded as a contrast feature vector $X_{i,h}^c=[d_{i,h}{,d}_{i,h}^1,d_{i,h}^2,d_{i,h}^3,d_{i,h}^4,d_{i,h}^5,d_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol, d_i,hRepresents SP_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the neighboring area,f_i,hhas a dimension of 20, f_i,hRepresents SP_i,hC mean value of the frequency response feature vectors of all the pixels in (1)_i,hHas a dimension of 9, c_i,hRepresents SP_i,hThe mean of the color feature vectors of all the pixel points in (a),represents SP_i,hIs measured in the mean value of the parallax amplitude of (c),represents SP_i,hColor histograms of R, G and B components of all pixel points in RGB color space and SP_i,hThe distances of all pixel points in the adjacent regions of the color histogram of the R component, the G component and the B component of the RGB color space,represents SP_i,hThe color histograms of L component, a component and b component of all pixel points in CIELAB color space and SP_i,hThe distances of the color histograms of the L component, the a component and the b component of the CIELAB color space of all the pixel points in the neighboring region of (a),represents SP_i,hColor histogram and SP of H component of all pixel points in HVS color space_i,hAll pixel points in the neighboring region of (a) are at the distance of the color histogram of the H component of the HVS color space,represents SP_i,hColor histogram of S component of all pixel points in HVS color space and SP_i,hThe distance of all pixel points in the neighboring region of (a) in the color histogram of the S component of the HVS color space,represents SP_i,hLBP feature statistical histogram and SP of all pixel points in_i,hThe distance of the statistical histogram of the LBP features of all pixel points in the neighboring region,represents SP_i,hThe parallax statistical histogram and SP of all the pixel points in_i,hIn adjacent areas ofThe distance of the parallax statistical histogram of all the pixel points, wherein the adjacent region is R_iNeutral SP_i,hAdjacent regions.

In this embodiment, R in step ① -3_iH-th area SP of (1)_i,hContrast feature vector ofThe acquisition process comprises the following steps:

a1, calculating R_iH-th area SP of (1)_i,hThe mean value of the frequency response characteristic vectors of all the pixel points is recorded as f_i,h，f_i,hTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsMean of the frequency response amplitudes of the elements, where f_i,hHas a dimension of 20 a and has a high degree of,

a2, calculating R_iH-th area SP of (1)_i,hThe mean value of the color feature vectors of all the pixel points in (1) is marked as c_i,h， $c_{i,h}=[{\stackrel{\‾}{R}}_{i,h},{\stackrel{\‾}{G}}_{i,h},{\stackrel{\‾}{B}}_{i,h},{\stackrel{\‾}{L}}_{i,h},{\stackrel{\‾}{a}}_{i,h},{\stackrel{\‾}{b}}_{i,h},{\stackrel{\‾}{H}}_{i,h},{\stackrel{\‾}{V}}_{i,h},{\stackrel{\‾}{S}}_{i,h}],$ Wherein, c_i,hHas a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the R components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the G components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the B components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average value of the color values of the L components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hAll ofThe average value of the color values of the a component of the pixel point in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the b components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the V components of all the pixel points in the HVS color space,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the S component of all the pixel points in the HVS color space.

a3, calculating R_iH-th area SP of (1)_i,hIs the mean of the parallax amplitude ofIs equal to d_iNeutral SP_i,hAnd the mean value of the pixel values of all the pixel points in the corresponding region.

a4, mixing f_i,h、c_i,hAndarranged in sequence to form R_iH-th area SP of (1)_i,hFirst feature vector of (1), denoted as u_i,h，Wherein u is_i,hHas a dimension of 30, here the symbol "[ 2 ]]"is a vector representing a symbol.

a5, calculating R_iH-th area SP of (1)_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the neighboring region, denoted d_i,h，Wherein d is_i,hHas a dimension of 30, p is more than or equal to 1 and less than or equal to M,represents R_iH-th area SP of (1)_i,hU of adjacent areas of the sequence number_i,pRepresents R_iP-th area SP in (1)_i,p(SP_i,pIs SP_i,hAdjacent region of (b) with the symbol "|" being an absolute value symbol, P represents R_iH-th area SP of (1)_i,hIn this embodiment, P is 20, and the adjacent area is R_iNeutral SP_i,hAdjacent regions.

a6, calculating R_iH-th area SP of (1)_i,hThe color histograms of R, G and B components of all the pixel points in RGB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of the H component of all the pixel points in the HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of S component of all pixel points in HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe statistical histogram of the LBP characteristics of all the pixel points in (1) is recorded asCalculation of R_iH-th area SP of (1)_i,hThe parallax statistical histogram of all the pixel points in (1) is recorded asWherein,has dimension of 16³，Has dimension of 16³，Has a dimension of 16 a and has a high degree of,has a dimension of 16 a and has a high degree of,has a dimension of 256, and has a high thermal conductivity,has dimension of 16.

a7, calculatingAnd R_iH-th area SP of (1)_i,hThe distances of the color histograms of the R component, the G component and the B component of the RGB color space of all the pixel points in the adjacent areas are recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distances of the color histograms of the L component, the a component and the b component of all the pixel points in the adjacent areas in the CIELAB color space are recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the color histogram of the H component of the HVS color space of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the color histogram of the S component of the HVS color space of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the LBP feature statistical histogram of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the parallax statistical histogram of all the pixel points in the adjacent region is recorded as

Wherein p is more than or equal to 1 and less than or equal to M,represents R_iH-th area SP of (1)_i,hP represents R_iH-th area SP of (1)_i,hP is 20, χ () is a Chi-distance measure function,represents R_iP-th area SP in (1)_i,pThe color histograms of the R, G and B components of all the pixel points in the RGB color space,represents R_iP-th area SP in (1)_i,pL of all pixel points in CIELAB color spaceColor histograms of the quantity, a-component and b-component,represents R_iP-th area SP in (1)_i,pThe color histogram of the H component of the HVS color space for all the pixel points in (a),represents R_iP-th area SP in (1)_i,pThe color histogram of the S component of the HVS color space for all the pixel points in (a),represents R_iP-th area SP in (1)_i,pThe statistical histogram of LBP features of all pixel points in (1),represents R_iP-th area SP in (1)_i,pThe disparity statistical histogram of all the pixel points in (1).

a8, d_i,h、Andarranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as the contrast feature vector of $X_{i,h}^b=[e_{i,h},e_{i,h}^1,e_{i,h}^2,e_{i,h}^3,e_{i,h}^4,e_{i,h}^5,e_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol.

① -4, calculation of { L_i,R_i,d_iI is not less than 1 and not more than N, R is the general feature vector of each region in the right viewpoint image of each stereoscopic image_iH-th area SP of (1)_i,hIs given as Wherein,has a dimension of 33, here the symbol "[ 2 ]]"is a vector representing a symbol and,has a dimension of 20 a and has a high degree of,represents SP_i,hThe variance of the frequency response feature vectors of all the pixel points in (a),has a dimension of 9 a and has a high degree of,represents SP_i,hThe variance of the color feature vectors of all the pixel points in (a),represents SP_i,hOf the parallax amplitude, x_i,hHas a dimension of 2, x_i,hRepresents SP_i,hS of the center pixel point_i,hRepresents SP_i,hThe area of (a).

In this embodiment, R in step ① -4_iH-th area SP of (1)_i,hIs a universal feature vectorThe acquisition process comprises the following steps:

b1, calculating R_iH-th area SP of (1)_i,hThe variance of the frequency response feature vectors of all the pixel points is recorded asTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsThe variance of the frequency response amplitude of the individual elements, wherein,has a dimension of 20 a and has a high degree of,

b2, calculating R_iH-th area SP of (1)_i,hThe variance of the color feature vectors of all the pixel points is recorded as ${\hat{c}}_{i,h}=[{\hat{R}}_{i,h},{\hat{G}}_{i,h},{\hat{B}}_{i,h},{\hat{L}}_{i,h},{\hat{a}}_{i,h},{\hat{b}}_{i,h},{\hat{H}}_{i,h},{\hat{V}}_{i,h},{\hat{S}}_{i,h}],$ Wherein,has a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe variance of the color values of the R component of the RGB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the G component of the RGB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the B component of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the L component of the CIELAB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the a component of the CIELAB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the b-component in CIELAB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the V component of the HVS color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the S component of the HVS color space for all the pixel points in (1).

b3, calculating R_iH-th area SP of (1)_i,hThe variance of the parallax amplitude of (1), is recorded asIs equal to d_iNeutralization ofSP_i,hThe variance of the pixel values of all the pixel points in the corresponding region.

b4, obtaining R_iH-th area SP of (1)_i,hThe coordinate position of the central pixel point is marked as x_i,hWherein x is_i,hHas a dimension of 2.

b5, calculating R_iH-th area SP of (1)_i,hArea of (d), denoted as s_i,h。

b6, willx_i,hAnd s_i,hArranged in sequence to form R_iH-th area SP of (1)_i,hIs a universal feature vector of $X_{i,h}^a=[{\hat{f}}_{i,h},{\hat{c}}_{i,h},{\hat{d}}_{i,h},x_{i,h},s_{i,h}],$ Wherein,has a dimension of 33, here the symbol "[ 2 ]]"is a vector representing a symbol.

① -5, calculating { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, and R is equal to R_iH-th area SP of (1)_i,hIs recorded as $X_{i,h}^b=[e_{i,h},e_{i,h}^1,e_{i,h}^2,e_{i,h}^3,e_{i,h}^4,e_{i,h}^5,e_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol, e_i,hRepresents SP_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the background region,f_i,hhas a dimension of 20, f_i,hRepresents SP_i,hC mean value of the frequency response feature vectors of all the pixels in (1)_i,hHas a dimension of 9, c_i,hRepresents SP_i,hThe mean of the color feature vectors of all the pixel points in (a),represents SP_i,hIs measured in the mean value of the parallax amplitude of (c),represents SP_i,hThe color histogram of R component, G component and B component of all pixel points in RGB color space and R_iThe distances of all the pixel points in the background region in the color histogram of the R component, G component and B component of the RGB color space,represents SP_i,hThe color histograms of L component, a component and b component and R of all pixel points in CIELAB color space_iThe distances of the color histograms of the L component, the a component and the b component of the CIELAB color space of all the pixel points in the background area in (1),represents SP_i,hColor histogram and R of H component of all pixel points in HVS color space_iThe distance of all the pixel points in the background region in the color histogram of the H component of the HVS color space,represents SP_i,hColor histogram of S component of all pixel points in HVS color space and R_iThe distance of all pixel points in the background region in the color histogram of the S component of the HVS color space,represents SP_i,hLBP feature statistical histogram and R of all pixel points in_iThe distance of the statistical histogram of the LBP features of all the pixels in the background region in (1),represents SP_i,hThe parallax statistical histogram and R of all the pixel points in_iThe distance of the parallax statistical histogram of all the pixel points in the background region is R_iThe leftmost, rightmost, uppermost and lowermost regions of the group.

In this embodiment, R in step ① -5_iH-th area SP of (1)_i,hBackground prior feature vector ofThe acquisition process comprises the following steps:

c1, calculating R_iH-th area SP of (1)_i,hThe mean value of the frequency response characteristic vectors of all the pixel points is recorded as f_i,h，f_i,hTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsMean of the frequency response amplitudes of the elements, where f_i,hHas a dimension of 20 a and has a high degree of,

c2, calculating R_iH-th area SP of (1)_i,hThe mean value of the color feature vectors of all the pixel points in (1) is marked as c_i,h， $c_{i,h}=[{\stackrel{\‾}{R}}_{i,h},{\stackrel{\‾}{G}}_{i,h},{\stackrel{\‾}{B}}_{i,h},{\stackrel{\‾}{L}}_{i,h},{\stackrel{\‾}{a}}_{i,h},{\stackrel{\‾}{b}}_{i,h},{\stackrel{\‾}{H}}_{i,h},{\stackrel{\‾}{V}}_{i,h},{\stackrel{\‾}{S}}_{i,h}],$ Wherein, c_i,hHas a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the R components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the G components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the B components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average value of the color values of the L components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the a component of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the b components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the V components of all the pixel points in the HVS color space,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the S component of all the pixel points in the HVS color space.

c3, calculating R_iH-th area SP of (1)_i,hIs the mean of the parallax amplitude ofIs equal to d_iNeutral SP_i,hAnd the mean value of the pixel values of all the pixel points in the corresponding region.

c4, mixing f_i,h、c_i,hAndarranged in sequence to form R_iH-th area SP of (1)_i,hFirst feature vector of (1), denoted as u_i,h，Wherein u is_i,hHas a dimension of 30, here the symbol "[ 2 ]]"is a vector representation symbol;

c5, calculating R_iH-th area SP of (1)_i,hFirst feature of (1)Eigenvector u_i,hThe distance from the first feature vector of the background region, denoted as e_i,h，Wherein e is_i,hHas a dimension of 30, q is more than or equal to 1 and less than or equal to M,represents R_iSet of sequence numbers of all background regions in u_i,qRepresents R_iThe q-th region SP in (1)_i,q(SP_i,qIs R_iBackground region in), the symbol "|" is an absolute value symbol, Q represents R_iThe total number of background regions in (1), where the background region is R_iThe areas positioned at the leftmost side, the rightmost side, the uppermost side and the lowermost side in the middle are about to fall on R_iThe leftmost, rightmost, uppermost and lowermost regions of (a) are taken as background regions.

c6, calculating R_iH-th area SP of (1)_i,hThe color histograms of R, G and B components of all the pixel points in RGB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of the H component of all the pixel points in the HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of S component of all pixel points in HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe statistical histogram of the LBP characteristics of all the pixel points in (1) is recorded asCalculation of R_iH-th area SP of (1)_i,hThe parallax statistical histogram of all the pixel points in (1) is recorded asWherein,has dimension of 16³，Has dimension of 16³，Has a dimension of 16 a and has a high degree of,has a dimension of 16 a and has a high degree of,has a dimension of 256, and has a high thermal conductivity,has dimension of 16.

c7, calculationAnd R_iThe distances of the color histograms of the R, G and B components in the RGB color space of all the pixel points in the background region are recorded as

ComputingAnd R_iThe distances of the color histograms of the L component, the a component and the b component of all the pixel points in the background area in the CIELAB color space are recorded as the distances

ComputingAnd R_iThe distance of the color histogram of the H component of the HVS color space of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the color histogram of the S component of the HVS color space of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the LBP feature statistical histogram of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the parallax statistical histogram of all the pixel points in the background region is recorded as

Wherein q is more than or equal to 1 and less than or equal to M,represents R_iQ represents R_iχ () is a Chi-distance measure function,represents R_iThe q-th region SP in (1)_i,qThe color histograms of the R, G and B components of all the pixel points in the RGB color space,represents R_iThe q-th region SP in (1)_i,qThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space,represents R_iThe q-th region SP in (1)_i,qThe color histogram of the H component of the HVS color space for all the pixel points in (a),represents R_iThe q-th region SP in (1)_i,qThe color histogram of the S component of the HVS color space for all the pixel points in (a),represents R_iThe q-th region SP in (1)_i,qThe statistical histogram of LBP features of all pixel points in (1),represents R_iThe q-th region SP in (1)_i,qThe disparity statistical histogram of all the pixel points in (1).

c8, mixing_i,h、Andarranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as the background prior feature vector of $X_{i,h}^b=[e_{i,h},e_{i,h}^1,e_{i,h}^2,e_{i,h}^3,e_{i,h}^4,e_{i,h}^5,e_{i,h}^6],$ Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol.

① -6, will { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N), the contrast characteristic vector, the general characteristic vector and the background prior characteristic vector of each region in the right viewpoint image of each stereo image are arranged in sequence to form { L ≦_i,R_i,d_iI 1 ≦ i ≦ N } for each region in the right view image of each stereoscopic image, R is a feature vector for reflecting visual saliency_iH-th area SP of (1)_i,hThe feature vector for reflecting the visual saliency is marked as X_i,h，Wherein, X_i,hHas a dimension of 105, here the symbol "[ 2 ]]"is a vector representing a symbol.

① -7, using existing random forest regression, on { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N), training the feature vectors for reflecting the visual saliency in all the regions of the right viewpoint images of all the stereo images, minimizing the error between the regression function value obtained through training and the average eye movement value, obtaining the optimal random forest regression training model, and marking the optimal random forest regression training model as f (D)_inp) Wherein f () is a functional representation form, D_inpAnd representing the input vector of the random forest regression training model.

The specific steps of the test phase are as follows:

② -1, for any one test stereo image S_testWill S_testThe left viewpoint image, the right viewpoint image, and the right parallax image are expressed as L_test、R_test、d_test(ii) a Then adopting the existing super-pixel segmentation technology to divide R_testIs divided intoM non-overlapping regions, R_testThe h-th area in (1) is denoted as SP_h', R may be_testRe-expressed as a set of M regions, denoted as { SP_h' }; wherein M is not less than 1, in this embodiment, M is 400, and h is not less than 1 and not more than M.

② -2, following the procedure of step ① -3 through step ① -6, R is obtained in the same manner of operation_testFor each region of (a) to reflect visual saliency, R_testH-th area SP of (1)_h' the feature vector for reflecting the visual saliency is denoted as F_test,h，F_test,hThe acquisition process comprises the following steps: calculation of R_testH-th area SP of (1)_h' the contrast feature vector is denoted ${X_h^c}^{\&prime;}=[{d_h}^{\&prime;},{d_h^1}^{\&prime;},{d_h^2}^{\&prime;},{d_h^3}^{\&prime;},{d_h^4}^{\&prime;},{d_h^5}^{\&prime;},{d_h^6}^{\&prime;}],$ Wherein,has a dimension of 36, d_h' means SP_h' first feature vector u_h' distance from the first feature vector of the neighboring area,f_hdimension of' 20, f_h' means SP_h' average of frequency response feature vectors, c_hDimension of' 9, c_h' means SP_hThe mean value of the color feature vector of',represents SP_hThe average value of the magnitude of the parallax of',represents SP_h' color histograms of R, G, and B components of all pixels in RGB color space and SP_h' distances of color histograms of R, G and B components of RGB color space for all pixel points in neighboring regions of,represents SP_h' color histograms of L, a and b components of all pixels in CIELAB color space and SP_h' distances of color histograms of L component, a component and b component of CIELAB color space for all pixel points in neighboring regions,represents SP_h' color histogram of H component of HVS color space and SP of all pixel points in_h'the distance of all pixel points in the neighborhood of' in the color histogram of the H component of the HVS color space,represents SP_h' the color histogram of S component of all pixel points in HVS color space and SP_h'distance of all pixel points in the neighborhood of' in the color histogram of the S component of the HVS color space,represents SP_hLBP of all the pixels in `Feature statistical histogram and SP<sub>h</sub>' the distance of the statistical histogram of the LBP features of all pixel points in the neighborhood of,represents SP<sub>h</sub>' the statistical histogram of parallax and SP of all the pixels in<sub>h</sub>' the distance between the disparity statistical histogram of all the pixels in the neighboring region of SP<sub>h</sub>' adjacent regions; calculation of R<sub>test</sub>H-th area SP of (1)<sub>h</sub>' general feature vector is denoted as ${X_h^a}^{\&prime;}=[{{\hat{f}}_h}^{\&prime;},{{\hat{c}}_h}^{\&prime;},{{\hat{d}}_h}^{\&prime;},{x_h}^{\&prime;},{s_h}^{\&prime;}],$ Wherein,has a dimension of 33 a and has a high degree of,has a dimension of 20 a and has a high degree of,represents SP<sub>h</sub>The variance of the frequency response feature vector of',has a dimension of 9 a and has a high degree of,represents SP<sub>h</sub>The variance of the color feature vector of',represents SP<sub>h</sub>Variance of parallax amplitude, x of `_hThe dimension of' is 2, x_h' means SP_h' coordinate position of center pixel, s_h' means SP_hThe area of `; calculation of R<sub>test</sub>H-th area SP of (1)<sub>h</sub>' the background prior feature vector is denoted as ${X_h^b}^{\&prime;}=[{e_h}^{\&prime;},{e_h^1}^{\&prime;},{e_h^2}^{\&prime;},{e_h^3}^{\&prime;},{e_h^4}^{\&prime;},{e_h^5}^{\&prime;},{e_h^6}^{\&prime;}],$ Wherein,has a dimension of 36, e<sub>h</sub>' means SP<sub>h</sub>' first feature vector u<sub>h</sub>' distance from the first feature vector of the background region,represents SP<sub>h</sub>All the pixels inDistances of color histograms of R, G, and B components of the RGB color space from color histograms of R, G, and B components of the RGB color space of all pixel points in the background region,represents SP<sub>h</sub>' distances of the color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space from the color histograms of the L component, the a component and the b component of all the pixel points in the background area in the CIELAB color space,represents SP<sub>h</sub>' distance between the color histogram of the H component of the HVS color space of all the pixel points in the background region and the color histogram of the H component of the HVS color space of all the pixel points in the background region,represents SP<sub>h</sub>' distance between the color histogram of S component of HVS color space of all pixel points in the background region and the color histogram of S component of HVS color space of all pixel points in the background region,represents SP<sub>h</sub>' distance between the statistical histogram of LBP features of all pixels in the background region and the statistical histogram of LBP features of all pixels in the background region,represents SP<sub>h</sub>The distance between the parallax statistical histograms of all the pixel points in the background region and the parallax statistical histograms of all the pixel points in the background region, where the background region refers to the regions located at the leftmost side, the rightmost side, the uppermost side, and the lowermost side; r is to be<sub>test</sub>H-th area SP of (1)<sub>h</sub>' the contrast feature vector, the generic feature vector and the background prior feature vector are arranged in order to form R<sub>test</sub>H-th area SP of (1)<sub>h</sub>' the feature vector for reflecting visual saliency,is marked as F<sub>test,h</sub>，Wherein, F<sub>test,h</sub>Has a dimension of 105; then, training a model f (D) according to the optimal random forest regression obtained in the training stage<sub>inp</sub>) Will F<sub>test,h</sub>Obtaining R as an input vector of an optimal random forest regression training model<sub>test</sub>Of each region of (a), R<sub>test</sub>H-th area SP of (1)<sub>h</sub>' the three-dimensional visual saliency value is denoted as S<sub>3D,h</sub>，S<sub>3D,h</sub>＝f(F<sub>test,h</sub>) (ii) a Then R is put<sub>test</sub>As the saliency value of all pixel points in the corresponding region, e.g. for R<sub>test</sub>H-th area SP of (1)<sub>h</sub>', will S<sub>3D,h</sub>As SP<sub>h</sub>Significant values of all pixels in `, thus obtaining R_testIs marked as { S_3D(x, y) }, wherein (x, y) here denotes S_testX is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent S_testWidth and height of (S)_testIs the same as the width of the stereo image selected in the training stage, S_testIs the same as the height of the stereo image selected in the training stage, S_3D(x, y) represents S_3DAnd the coordinate position in the (x, y) is the pixel value of the pixel point of (x, y).

In this embodiment, R_iThe obtaining process of the frequency response characteristic vector of each pixel point comprises the following steps:

1) -1, using a Gabor filter pair R_iFiltering to obtain R_iUnder different central frequencies and different direction factors, each pixel point in the signal is subjected to frequency response amplitude R_iThe frequency response amplitude of the pixel point with the middle coordinate position (x, y) under the condition that the center frequency is omega and the direction factor is theta is marked as G (x, y; omega, theta), wherein (x, y) represents { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, x is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent { L ≦ N }_i,R_i,d_iI 1 ≦ i ≦ N }, ω represents the center frequency of the Gabor filter, ω ∈ Φ_ωθ represents the directional factor of the Gabor filter, θ ∈ Φ_θ，Φ_ωRepresents the set of all center frequencies of the Gabor filter, phi in this embodiment_ω＝{1.74,2.47,3.49,4.93,6.98}，Φ_θRepresents the set of all the directional factors of the Gabor filter, phi in this embodiment_θ＝{0°,90°,180°,270°}。

1) -2, reacting R_iThe frequency response amplitude of each pixel point in the R-array is arranged in sequence under different central frequencies and different direction factors to form R_iR, the frequency response feature vector of each pixel point in_iThe frequency response characteristic vector of the pixel point with the middle coordinate position (x, y) is marked as f_i(x,y)，f_i(x, y) is a vector consisting of G (x, y; 1.74,0 °), G (x, y; 2.47,0 °), G (x, y; 3.49,0 °), G (x, y; 4.93,0 °), G (x, y; 6.98,0 °), G (x, y; 1.74,90 °), G (x, y; 2.47,90 °), G (x, y; 3.49,90 °), G (x, y; 4.93,90 °), G (x, y; 6.98,90 °), G (x, y; 1.74,180 °), G (x, y; 2.47,180 °), G (x, y; 3.49,180 °), G (x, y; 4.93,180 °), G (x, y; 6.98,180 °), G (x, y °, 1.74,270 °), G (x, y; 2.47,270, G (x, y; 3.49,270; 4.93,270, y; 6.98,270, 26 °, 6.98,270:, wherein the vector is arranged in that order_iThe dimension of (x, y) is 20.

In this embodiment, R_iThe obtaining process of the color feature vector of each pixel point comprises the following steps:

2) -1, calculating R_iThe color value of each pixel point in different color spaces is R_iThe color values of the R component, the G component and the B component of the pixel point with the (x, y) middle coordinate position in the RGB color space are respectively recorded as R (x, y), G (x, y) and B (x, y), and R (x, y) is recorded_iThe color values of the L component, the a component and the b component of the pixel point with the middle coordinate position (x, y) in the CIELAB color space are respectively recorded as L (x, y), a (x, y) and b (x, y), and R is used for indicating the color values of the L component, the a component and the b component in the CIELAB color space as L (x, y), a (x, y) and b (_iH component and V component of pixel point with (x, y) as middle coordinate position in HVS color spaceThe color values of the quantity and S component are denoted as H (x, y), V (x, y), and S (x, y), respectively, where (x, y) here denotes { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, x is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent { L ≦ N }_i,R_i,d_iI is not less than 1 and not more than N.

2) -2, reacting R_iThe color values of each pixel point in the different color spaces are arranged in sequence to form R_iThe color feature vector of each pixel point in (1), R_iThe color feature vector of the pixel point with the middle coordinate position (x, y) is marked as c_i(x,y)，c_i(x,y)＝[R(x,y),G(x,y),B(x,y),L(x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)]Wherein c is_iThe dimension of (x, y) is 9, where the symbol "[ 2 ],"]"is a vector representing a symbol.

The method of the invention is used for extracting three-dimensional saliency maps of five stereo images, namely Image1, Image2, Image3, Image4 and Image5 in a three-dimensional human eye tracking database (3Deye-tracking database) provided by southern French university. FIG. 2a shows a right viewpoint Image of "Image 1", FIG. 2b shows a real eye diagram of a right viewpoint Image of "Image 1", and FIG. 2c shows a three-dimensional saliency map of "Image 1"; FIG. 3a shows a right viewpoint Image of "Image 2", FIG. 3b shows a real eye diagram of a right viewpoint Image of "Image 2", and FIG. 3c shows a three-dimensional saliency map of "Image 2"; FIG. 4a shows a right viewpoint Image of "Image 3", FIG. 4b shows a real eye diagram of a right viewpoint Image of "Image 3", and FIG. 4c shows a three-dimensional saliency map of "Image 3"; FIG. 5a shows a right viewpoint Image of "Image 4", FIG. 5b shows a real eye diagram of a right viewpoint Image of "Image 4", and FIG. 5c shows a three-dimensional saliency map of "Image 4"; FIG. 6a shows a right viewpoint Image of "Image 5", FIG. 6b shows a real eye diagram of a right viewpoint Image of "Image 5", and FIG. 6c shows a three-dimensional saliency map of "Image 5"; FIG. 7a shows a right viewpoint Image of "Image 6", FIG. 7b shows a real eye diagram of a right viewpoint Image of "Image 6", and FIG. 7c shows a three-dimensional saliency map of "Image 6"; fig. 8a shows a right viewpoint Image of "Image 7", fig. 8b shows a real eye diagram of a right viewpoint Image of "Image 7", and fig. 8c shows a three-dimensional saliency map of "Image 7". As can be seen from fig. 2a to 8c, the three-dimensional saliency map obtained by the method of the present invention can well conform to the features of the saliency semantics due to the consideration of the contrast feature, the general feature, and the background prior feature.

Claims (6)

1. A method for extracting a three-dimensional image saliency map is characterized by comprising a training stage and a testing stage, wherein the training stage comprises the following specific steps:

① -1, the selected N sets of different stereo images and the right parallax image of each stereo image form a set, which is marked as { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, wherein N is more than or equal to 1, and L_iRepresents { L_i,R_i,d_iL 1. ltoreq. i. ltoreq.N } left viewpoint image of ith stereoscopic image, R_iRepresents { L_i,R_i,d_iI is not less than 1 and not more than N, d_iRepresents { L_i,R_i,d_iI is not less than 1 and not more than N, and right parallax images of the ith stereo image;

① -2, using superpixel splitting technique to divide { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, the right viewpoint image of each stereo image is divided into M non-overlapping areas, R is_iThe h-th area in (1) is denoted as SP_i,hWherein M is more than or equal to 1, h is more than or equal to 1 and less than or equal to M;

① -3, calculation of { L_i,R_i,d_iI is not less than 1 and not more than N, R is the contrast characteristic vector of each region in the right viewpoint image of each stereoscopic image_iH-th area SP of (1)_i,hIs recorded as a contrast feature vector Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol, d_i,hRepresents SP_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the neighboring area,f_i,hhas a dimension of 20, f_i,hRepresents SP_i,hC mean value of the frequency response feature vectors of all the pixels in (1)_i,hHas a dimension of 9, c_i,hRepresents SP_i,hThe mean of the color feature vectors of all the pixel points in (a),represents SP_i,hIs measured in the mean value of the parallax amplitude of (c),represents SP_i,hColor histograms of R, G and B components of all pixel points in RGB color space and SP_i,hThe distances of all pixel points in the adjacent regions of the color histogram of the R component, the G component and the B component of the RGB color space,represents SP_i,hThe color histograms of L component, a component and b component of all pixel points in CIELAB color space and SP_i,hThe distances of the color histograms of the L component, the a component and the b component of the CIELAB color space of all the pixel points in the neighboring region of (a),represents SP_i,hColor histogram and SP of H component of all pixel points in HVS color space_i,hAll pixel points in the neighboring region of (a) are at the distance of the color histogram of the H component of the HVS color space,represents SP_i,hColor histogram of S component of all pixel points in HVS color space and SP_i,hThe distance of all pixel points in the neighboring region of (a) in the color histogram of the S component of the HVS color space,represents SP_i,hLBP feature statistical histogram and SP of all pixel points in_i,hThe distance of the statistical histogram of the LBP features of all pixel points in the neighboring region,represents SP_i,hThe parallax statistical histogram and SP of all the pixel points in_i,hThe distance of the parallax statistical histogram of all the pixel points in the adjacent region, where the adjacent region is R_iNeutral SP_i,hAn adjacent region;

① -4, calculation of { L_i,R_i,d_iI is not less than 1 and not more than N, R is the general feature vector of each region in the right viewpoint image of each stereoscopic image_iH-th area SP of (1)_i,hIs given as Wherein,has a dimension of 33, here the symbol "[ 2 ]]"is a vector representing a symbol and,has a dimension of 20 a and has a high degree of,represents SP_i,hThe variance of the frequency response feature vectors of all the pixel points in (a),has a dimension of 9 a and has a high degree of,represents SP_i,hThe variance of the color feature vectors of all the pixel points in (a),represents SP_i,hOf the parallax amplitude, x_i,hHas a dimension of 2, x_i,hRepresents SP_i,hS of the center pixel point_i,hRepresents SP_i,hThe area of (d);

① -5, calculating { L_i,R_i,d_iI 1 ≦ i ≦ N } for each region in the right view image for each stereoscopic imageA background prior feature vector of (2), R_iH-th area SP of (1)_i,hIs recorded as Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol, e_i,hRepresents SP_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the background region,f_i,hhas a dimension of 20, f_i,hRepresents SP_i,hC mean value of the frequency response feature vectors of all the pixels in (1)_i,hHas a dimension of 9, c_i,hRepresents SP_i,hThe mean of the color feature vectors of all the pixel points in (a),represents SP_i,hIs measured in the mean value of the parallax amplitude of (c),represents SP_i,hThe color histogram of R component, G component and B component of all pixel points in RGB color space and R_iThe distances of all the pixel points in the background region in the color histogram of the R component, G component and B component of the RGB color space,represents SP_i,hThe color histograms of L component, a component and b component and R of all pixel points in CIELAB color space_iOf all pixels in the background region in CIELAB color spaceThe distances of the color histograms of the L component, the a component and the b component,represents SP_i,hColor histogram and R of H component of all pixel points in HVS color space_iThe distance of all the pixel points in the background region in the color histogram of the H component of the HVS color space,represents SP_i,hColor histogram of S component of all pixel points in HVS color space and R_iThe distance of all pixel points in the background region in the color histogram of the S component of the HVS color space,represents SP_i,hLBP feature statistical histogram and R of all pixel points in_iThe distance of the statistical histogram of the LBP features of all the pixels in the background region in (1),represents SP_i,hThe parallax statistical histogram and R of all the pixel points in_iThe distance of the parallax statistical histogram of all the pixel points in the background region is R_iThe areas positioned at the leftmost side, the rightmost side, the uppermost side and the lowermost side;

① -6, will { L_i,R_i,d_iI is more than or equal to 1 and less than or equal to N), the contrast characteristic vector, the general characteristic vector and the background prior characteristic vector of each region in the right viewpoint image of each stereo image are arranged in sequence to form { L ≦_i,R_i,d_iI 1 ≦ i ≦ N } for each region in the right view image of each stereoscopic image, R is a feature vector for reflecting visual saliency_iH-th area SP of (1)_i,hThe feature vector for reflecting the visual saliency is marked as X_i,h，Wherein, X_i,hHas a dimension of 105, here the symbol "[ 2 ]]"is a vector representation symbol;

① -7, using random forest regression, on { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N), training the feature vectors for reflecting the visual saliency in all the regions of the right viewpoint images of all the stereo images, minimizing the error between the regression function value obtained through training and the average eye movement value, obtaining the optimal random forest regression training model, and marking the optimal random forest regression training model as f (D)_inp) Wherein f () is a functional representation form, D_inpRepresenting an input vector of a random forest regression training model;

the specific steps of the test stage are as follows:

② -1, for any one test stereo image S_testWill S_testThe left viewpoint image, the right viewpoint image, and the right parallax image are expressed as L_test、R_test、d_test(ii) a Then adopting super pixel segmentation technique to divide R_testDividing into M non-overlapping regions, and dividing R_testThe h-th area in (1) is denoted as SP_h'; wherein M is more than or equal to 1, h is more than or equal to 1 and less than or equal to M;

② -2, following the procedure of step ① -3 through step ① -6, R is obtained in the same manner of operation_testFor each region of (a) to reflect visual saliency, R_testH-th area SP of (1)_h' the feature vector for reflecting the visual saliency is denoted as F_test,h(ii) a Then, training a model f (D) according to the optimal random forest regression obtained in the training stage_inp) Will F_test,hObtaining R as an input vector of an optimal random forest regression training model_testOf each region of (a), R_testH-th area SP of (1)_h' the three-dimensional visual saliency value is denoted as S_3D,h，S_3D,h＝f(F_test,h) (ii) a Then R is put_testThe three-dimensional visual saliency value of each region in the R-image is taken as the saliency value of all pixel points in the corresponding region, thereby obtaining R_testIII of (2)Dimension saliency map, denoted as { S_3D(x, y) }, wherein (x, y) here denotes S_testX is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent S_testWidth and height of (S)_3D(x, y) represents S_3DAnd the coordinate position in the (x, y) is the pixel value of the pixel point of (x, y).

2. The method for extracting saliency map of stereoscopic images as claimed in claim 1, wherein said step ① -3R_iH-th area SP of (1)_i,hContrast feature vector ofThe acquisition process comprises the following steps:

a1, calculating R_iH-th area SP of (1)_i,hThe mean value of the frequency response characteristic vectors of all the pixel points is recorded as f_i,h，f_i,hTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsMean of the frequency response amplitudes of the elements, where f_i,hHas a dimension of 20 a and has a high degree of,

a2, calculating R_iH-th area SP of (1)_i,hThe mean value of the color feature vectors of all the pixel points in (1) is marked as c_i,h，Wherein, c_i,hHas a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the R components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the G components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the B components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average value of the color values of the L components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the a component of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the b components of all the pixel points in the CIELAB color space,represents R_iThe h region SP of_i,hThe average of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hAll the pixel points inThe mean of the color values of the V component in the HVS color space,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the S components of all the pixel points in the HVS color space;

a3, calculating R_iH-th area SP of (1)_i,hIs the mean of the parallax amplitude of Is equal to d_iNeutral SP_i,hThe mean value of the pixel values of all the pixel points in the corresponding region;

a4, mixing f_i,h、c_i,hAndarranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as a first feature vector ofWherein u is_i,hHas a dimension of 30, here the symbol "[ 2 ]]"is a vector representation symbol;

a5, calculating R_iH-th area SP of (1)_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the neighboring region, denoted d_i,h，Wherein d is_i,hHas a dimension of 30, p is more than or equal to 1 and less than or equal to M,represents R_iH-th area SP of (1)_i,hU of adjacent areas of the sequence number_i,pRepresents R_iP-th area SP in (1)_i,pThe symbol "|" is an absolute value symbol, and P represents R_iH-th area SP of (1)_i,hIs referred to as R_iNeutral SP_i,hAn adjacent region;

a6, calculating R_iH-th area SP of (1)_i,hThe color histograms of R, G and B components of all the pixel points in RGB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of the H component of all the pixel points in the HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of S component of all pixel points in HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe statistical histogram of the LBP characteristics of all the pixel points in (1) is recorded asCalculation of R_iH-th area SP of (1)_i,hThe parallax statistical histogram of all the pixel points in (1) is recorded asWherein,has dimension of 16³，Has dimension of 16³，Has a dimension of 16 a and has a high degree of,has a dimension of 16 a and has a high degree of,has a dimension of 256, and has a high thermal conductivity,has a dimension of 16;

a7, calculatingAnd R_iH-th area SP of (1)_i,hThe distances of the color histograms of the R component, the G component and the B component of the RGB color space of all the pixel points in the adjacent areas are recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distances of the color histograms of the L component, the a component and the b component of all the pixel points in the adjacent areas in the CIELAB color space are recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the color histogram of the H component of the HVS color space of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the color histogram of the S component of the HVS color space of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the LBP feature statistical histogram of all the pixel points in the adjacent region is recorded as

ComputingAnd R_iH-th area SP of (1)_i,hThe distance of the parallax statistical histogram of all the pixel points in the adjacent region is recorded as

Wherein p is more than or equal to 1 and less than or equal to M,represents R_iH-th area SP of (1)_i,hP represents R_iH-th area SP of (1)_i,hIs the chi () function of the chi-squared distance,represents R_iP-th area SP in (1)_i,pThe color histograms of the R, G and B components of all the pixel points in the RGB color space,represents R_iP-th area SP in (1)_i,pThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space,represents R_iP-th area SP in (1)_i,pThe color histogram of the H component of the HVS color space for all the pixel points in (a),represents R_iP-th area SP in (1)_i,pColor of S component of all pixel points in HVS color spaceThe color histogram is a histogram of the color,represents R_iP-th area SP in (1)_i,pThe statistical histogram of LBP features of all pixel points in (1),represents R_iP-th area SP in (1)_i,pThe parallax statistics histogram of all the pixel points in the image;

a8, d_i,h、Andarranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as the contrast feature vector of Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol.

3. The method for extracting saliency map of stereoscopic images as claimed in claim 2, wherein said step ① -4R_iH-th area SP of (1)_i,hIs a universal feature vectorThe acquisition process comprises the following steps:

b1, calculating R_iH-th area SP of (1)_i,hFrequency response characteristics of all pixel points inVariance of vector, noted To (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsThe variance of the frequency response amplitude of the individual elements, wherein,has a dimension of 20 a and has a high degree of,

b2, calculating R_iH-th area SP of (1)_i,hThe variance of the color feature vectors of all the pixel points is recorded as Wherein,has a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i，hThe variance of the color values of the R component of the RGB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the G component of the RGB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the B component of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the L component of the CIELAB color space for all pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the a component of the CIELAB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the b-component in CIELAB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe variance of the color values of the V component of the HVS color space for all pixel points in (a),represents R_iH of (1)Region SP_i,hThe variance of the color values of the S component of all the pixel points in the HVS color space;

b3, calculating R_iH-th area SP of (1)_i,hThe variance of the parallax amplitude of (1), is recorded as Is equal to d_iNeutral SP_i,hVariance of pixel values of all pixel points in the corresponding region;

b4, obtaining R_iH-th area SP of (1)_i,hThe coordinate position of the central pixel point is marked as x_i,hWherein x is_i,hHas a dimension of 2;

b5, calculating R_iH-th area SP of (1)_i,hArea of (d), denoted as s_i,h；

b6, willx_i,hAnd s_i,hArranged in sequence to form R_iH-th area SP of (1)_i,hIs a universal feature vector of Wherein,has a dimension of 33, here the symbol "[ 2 ]]"is a vector representing a symbol.

4. The method for extracting saliency map of stereoscopic images as claimed in claim 3, wherein said step ① -5R_iH-th area SP of (1)_i,hBackground prior feature vector ofThe acquisition process comprises the following steps:

c1, calculating R_iH-th area SP of (1)_i,hThe mean value of the frequency response characteristic vectors of all the pixel points is recorded as f_i,h，f_i,hTo (1)The value of each element is equal to R_iH-th area SP of (1)_i,hIn the frequency response feature vector of all the pixel pointsMean of the frequency response amplitudes of the elements, where f_i,hHas a dimension of 20 a and has a high degree of,

c2, calculating R_iH-th area SP of (1)_i,hThe mean value of the color feature vectors of all the pixel points in (1) is marked as c_i,h，Wherein, c_i,hHas a dimension of 9, here the symbol "[ 2 ]]"is a vector representing a symbol and,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the R components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the G components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the B components of the RGB color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i，hThe average value of the color values of the L components of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the a component of all the pixel points in the CIELAB color space,represents R_iH-th area SP of (1)_i,hThe average value of the color values of the b components of all the pixel points in the CIELAB color space,represents R_iThe h region SP of_i,hThe average of the color values of the H component of the HVS color space for all the pixel points in (a),represents R_iH-th area SP of (1)_i,hThe average of the color values of the V components of all the pixel points in the HVS color space,represents R_iH-th area SP of (1)_i,hThe mean value of the color values of the S components of all the pixel points in the HVS color space;

c3, calculating R_iH-th area SP of (1)_i,hIs the mean of the parallax amplitude of Is equal to d_iNeutral SP_i,hThe mean value of the pixel values of all the pixel points in the corresponding region;

c4, mixing f_i,h、c_i,hAndarranged in sequence to form R_iH-th area SP of (1)_i,hFirst feature vector of (1), denoted as u_i,h，Wherein u is_i,hHas a dimension of 30, here the symbol "[ 2 ]]"is a vector representation symbol;

c5, calculating R_iH-th area SP of (1)_i,hFirst feature vector u of_i,hThe distance from the first feature vector of the background region, denoted as e_i,h，Wherein e is_i,hHas a dimension of 30, q is more than or equal to 1 and less than or equal to M,represents R_iSet of sequence numbers of all background regions in u_i,qRepresents R_iThe q-th region SP in (1)_i,qThe symbol "|" is an absolute value symbol, and Q represents R_iThe total number of background regions in (1), where the background region is R_iThe areas positioned at the leftmost side, the rightmost side, the uppermost side and the lowermost side;

c6, calculating R_iH-th area SP of (1)_i,hThe color histograms of R, G and B components of all the pixel points in RGB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space are recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of the H component of all the pixel points in the HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe color histogram of S component of all pixel points in HVS color space is recorded asCalculation of R_iH-th area SP of (1)_i,hThe statistical histogram of the LBP characteristics of all the pixel points in (1) is recorded asCalculation of R_iH-th area SP of (1)_i,hThe parallax statistical histogram of all the pixel points in (1) is recorded asWherein,has dimension of 16³，Has dimension of 16³，Has a dimension of 16 a and has a high degree of,has a dimension of 16 a and has a high degree of,has a dimension of 256, and has a high thermal conductivity,has a dimension of 16;

c7, calculationAnd R_iThe distances of the color histograms of the R component, the G component and the B component of the RGB color space of all the pixel points in the background region are recorded as

ComputingAnd R_iThe distances of the color histograms of the L component, the a component and the b component of all the pixel points in the background area in the CIELAB color space are recorded as the distances

ComputingAnd R_iThe distance of the color histogram of the H component of the HVS color space of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the color histogram of the S component of the HVS color space of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the LBP feature statistical histogram of all the pixel points in the background area is recorded as

ComputingAnd R_iThe distance of the parallax statistical histogram of all the pixel points in the background region is recorded as

Wherein q is more than or equal to 1 and less than or equal to M,represents R_iAll ofSet of sequence numbers of background regions, Q denotes R_iThe total number of background regions, χ () is a chi-squared distance function,represents R_iThe q-th region SP in (1)_i,qThe color histograms of the R, G and B components of all the pixel points in the RGB color space,represents R_iThe q-th region SP in (1)_i,qThe color histograms of the L component, the a component and the b component of all the pixel points in the CIELAB color space,represents R_iThe q-th region SP in (1)_i,qThe color histogram of the H component of the HVS color space for all the pixel points in (a),represents R_iThe q-th region SP in (1)_i,qThe color histogram of the S component of the HVS color space for all the pixel points in (a),represents R_iThe q-th region SP in (1)_i,qThe statistical histogram of LBP features of all pixel points in (1),represents R_iThe q-th region SP in (1)_i,qThe parallax statistics histogram of all the pixel points in the image;

c8, mixing_i,h、Andarranged in sequence to form R_iH-th area SP of (1)_i,hIs recorded as the background prior feature vector of Wherein,has a dimension of 36, here the symbol "[ 2 ]]"is a vector representing a symbol.

5. The method according to any one of claims 2 to 4, wherein R is the number of pixels in the R-map_iThe obtaining process of the frequency response characteristic vector of each pixel point comprises the following steps:

1) -1, using a Gabor filter pair R_iFiltering to obtain R_iUnder different central frequencies and different direction factors, each pixel point in the signal is subjected to frequency response amplitude R_iThe frequency response amplitude of the pixel point with the middle coordinate position (x, y) under the condition that the center frequency is omega and the direction factor is theta is marked as G (x, y; omega, theta), wherein (x, y) represents { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, x is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent { L ≦ N }_i,R_i,d_iI 1 ≦ i ≦ N }, ω represents the center frequency of the Gabor filter, ω ∈ Φ_ωθ represents the directional factor of the Gabor filter, θ ∈ Φ_θ，Φ_ωRepresenting the set of all centre frequencies of the Gabor filter, phi_θRepresents the set of all directional factors of the Gabor filter;

1) -2, reacting R_iThe frequency response amplitude of each pixel point in the R-array is arranged in sequence under different central frequencies and different direction factors to form R_iThe frequency response feature vector of each pixel point in (a),r is to be_iThe frequency response characteristic vector of the pixel point with the middle coordinate position (x, y) is marked as f_i(x, y) wherein f_iThe dimension of (x, y) is 20.

6. The method as claimed in claim 5, wherein R is a number of terms_iThe obtaining process of the color feature vector of each pixel point comprises the following steps:

2) -1, calculating R_iThe color value of each pixel point in different color spaces is R_iThe color values of the R component, the G component and the B component of the pixel point with the (x, y) middle coordinate position in the RGB color space are respectively recorded as R (x, y), G (x, y) and B (x, y), and R (x, y) is recorded_iThe color values of the L component, the a component and the b component of the pixel point with the middle coordinate position (x, y) in the CIELAB color space are respectively recorded as L (x, y), a (x, y) and b (x, y), and R is used for indicating the color values of the L component, the a component and the b component in the CIELAB color space as L (x, y), a (x, y) and b (_iThe color values of the H component, the V component and the S component of the pixel point with the (x, y) middle coordinate position in the HVS color space are respectively recorded as H (x, y), V (x, y) and S (x, y), wherein (x, y) here represents { L }_i,R_i,d_iI is more than or equal to 1 and less than or equal to N, x is more than or equal to 1 and less than or equal to W, y is more than or equal to 1 and less than or equal to H, and W and H correspondingly represent { L ≦ N }_i,R_i,d_iThe width and height of the stereoscopic image in the condition that i is more than or equal to 1 and less than or equal to N;

2) -2, reacting R_iThe color values of each pixel point in the different color spaces are arranged in sequence to form R_iThe color feature vector of each pixel point in (1), R_iThe color feature vector of the pixel point with the middle coordinate position (x, y) is marked as c_i(x,y)，c_i(x,y)＝[R(x,y),G(x,y),B(x,y),L(x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)]Wherein c is_iThe dimension of (x, y) is 9, where the symbol "[ 2 ],"]"is a vector representing a symbol.