CN104822058B - A kind of stereo-picture notable figure extracting method - Google Patents
A kind of stereo-picture notable figure extracting method Download PDFInfo
- Publication number
- CN104822058B CN104822058B CN201510173098.5A CN201510173098A CN104822058B CN 104822058 B CN104822058 B CN 104822058B CN 201510173098 A CN201510173098 A CN 201510173098A CN 104822058 B CN104822058 B CN 104822058B
- Authority
- CN
- China
- Prior art keywords
- component
- area
- color
- pixel points
- color space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 47
- 239000013598 vector Substances 0.000 claims abstract description 219
- 238000012360 testing method Methods 0.000 claims abstract description 80
- 238000012549 training Methods 0.000 claims abstract description 37
- 230000000007 visual effect Effects 0.000 claims abstract description 33
- 230000004424 eye movement Effects 0.000 claims abstract description 20
- 238000007637 random forest analysis Methods 0.000 claims abstract description 19
- 230000011218 segmentation Effects 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims 13
- 238000001914 filtration Methods 0.000 claims 1
- 238000010586 diagram Methods 0.000 description 11
- 230000006870 function Effects 0.000 description 8
- 238000000605 extraction Methods 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 230000007613 environmental effect Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000010365 information processing Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 238000011160 research Methods 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
本发明公开了一种立体图像显著图提取方法,其在训练阶段,提取多幅训练立体图像的右视点图像中的每个区域的对比度特征矢量、通用特征矢量和背景先验特征矢量,并融合得到每幅立体图像的右视点图像中的每个区域的用于反映视觉显著性的特征矢量,建立特征矢量与平均眼动值之间的随机森林回归训练模型;在测试阶段,计算测试立体图像的右视点图像中的每个区域的用于反映视觉显著性的特征矢量,并根据已训练得到的随机森林回归训练模型,预测得到测试立体图像的三维显著图;优点是所提取的特征能够较好地反映各种因素的显著变化情况,从而有效地提高了视觉显著值的预测准确性。
The invention discloses a method for extracting a saliency map of a stereoscopic image. In the training stage, the contrast feature vector, the general feature vector and the background prior feature vector of each area in the right viewpoint image of multiple training stereo images are extracted and fused. Obtain the feature vector used to reflect the visual salience of each region in the right view point image of each stereo image, and establish a random forest regression training model between the feature vector and the average eye movement value; in the test phase, calculate the test stereo image The feature vector used to reflect the visual saliency of each region in the right viewpoint image, and according to the trained random forest regression training model, predict the 3D saliency map of the test stereo image; the advantage is that the extracted features can be compared with It can well reflect the significant changes of various factors, thus effectively improving the prediction accuracy of visually significant values.
Description
技术领域technical field
本发明涉及一种图像信号的处理方法,尤其是涉及一种立体图像显著图提取方法。The invention relates to a method for processing image signals, in particular to a method for extracting saliency maps of stereoscopic images.
背景技术Background technique
在人类视觉接收与信息处理中,由于大脑资源有限以及外界环境信息重要性区别,因此在处理过程中人脑对外界环境信息并不是一视同仁的,而是表现出选择特征。人们在观看图像或者视频片段时注意力并非均匀分布到图像的每个区域,而是对某些显著区域关注度更高。如何将视频中视觉注意度高的显著区域检测并提取出来是计算机视觉以及基于内容的视频检索领域的一个重要的研究内容。In human visual reception and information processing, due to limited brain resources and differences in the importance of external environmental information, the human brain does not treat external environmental information equally in the processing process, but shows selective characteristics. When people watch images or video clips, their attention is not evenly distributed to every area of the image, but they pay more attention to certain salient areas. How to detect and extract salient regions with high visual attention in videos is an important research content in the field of computer vision and content-based video retrieval.
然而,人眼感知立体图像产生立体视觉的过程并不是简单的左视点图像和右视点图像叠加的过程,因此,立体视觉特征(例如:三维视觉注意力)并不是平面视觉特性的简单拓展,如何从立体图像中有效地提取出立体视觉特征、如何使得提取出的立体视觉特征符合人眼三维观看行为,都是在对立体图像进行视觉显著图提取过程中需要研究解决的问题。However, the process of human perception of stereoscopic images to produce stereoscopic vision is not a simple process of superimposing left-viewpoint images and right-viewpoint images. Therefore, stereoscopic vision features (such as: three-dimensional visual attention) are not simply an extension of planar vision characteristics. How to effectively extract stereoscopic features from stereoscopic images and how to make the extracted stereoscopic features conform to the three-dimensional viewing behavior of human eyes are all problems that need to be studied and solved in the process of extracting visual saliency maps from stereoscopic images.
发明内容Contents of the invention
本发明所要解决的技术问题是提供一种立体图像显著图提取方法,其符合显著语义特征,且具有较强的提取稳定性和较高的提取准确性。The technical problem to be solved by the present invention is to provide a method for extracting saliency maps of stereoscopic images, which conforms to saliency semantic features, and has strong extraction stability and high extraction accuracy.
本发明解决上述技术问题所采用的技术方案为:一种立体图像显著图提取方法,其特征在于包括训练阶段和测试阶段两个过程,所述的训练阶段的具体步骤如下:The technical solution adopted by the present invention to solve the above-mentioned technical problems is: a method for extracting a saliency map of a stereoscopic image, which is characterized in that it includes two processes of a training phase and a testing phase, and the specific steps of the training phase are as follows:
①-1、将选取的N副各不相同的立体图像以及每幅立体图像的右视差图像构成一个集合,记为{Li,Ri,di|1≤i≤N},其中,N≥1,Li表示{Li,Ri,di|1≤i≤N}中的第i幅立体图像的左视点图像,Ri表示{Li,Ri,di|1≤i≤N}中的第i幅立体图像的右视点图像,di表示{Li,Ri,di|1≤i≤N}中的第i幅立体图像的右视差图像;①-1. The selected N sets of different stereoscopic images and the right disparity images of each stereoscopic image form a set, which is recorded as {L i , R i , d i |1≤i≤N}, where N ≥1, L i represents the left viewpoint image of the i-th stereo image in {L i , R i , d i |1≤i≤N}, and R i represents {L i , R i , d i |1≤i ≤N} in the right viewpoint image of the i-th stereo image, d i represents the right parallax image of the i-th stereo image in {L i , R i , d i |1≤i≤N};
①-2、采用超像素分割技术将{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像分割成M个互不重叠的区域,将Ri中的第h个区域记为SPi,h,其中,M≥1,1≤h≤M;①-2. Use superpixel segmentation technology to divide the right viewpoint image of each stereoscopic image in {L i , R i , d i |1≤i≤N} into M non-overlapping regions, and divide R i The hth region of is denoted as SP i,h , where M≥1, 1≤h≤M;
①-3、计算{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的对比度特征矢量,将Ri中的第h个区域SPi,h的对比度特征矢量记为
①-4、计算{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的通用特征矢量,将Ri中的第h个区域SPi,h的通用特征矢量记为 其中,的维数为33,此处符号“[]”为矢量表示符号,的维数为20,表示SPi,h中的所有像素点的频率响应特征矢量的方差,的维数为9,表示SPi,h中的所有像素点的颜色特征矢量的方差,表示SPi,h的视差幅值的方差,xi,h的维数为2,xi,h表示SPi,h的中心像素点的坐标位置,si,h表示SPi,h的面积;①-4. Calculate the general feature vector of each region in the right view point image of each stereoscopic image in {L i , R i , d i |1≤i≤N}, and divide the hth region in R i The general eigenvector of SP i,h is denoted as in, The dimension of is 33, where the symbol “[]” is a vector representation symbol, The dimension of is 20, Represents the variance of the frequency response feature vector of all pixels in SP i,h , The dimension of is 9, Represents the variance of the color feature vectors of all pixels in SP i,h , Represents the variance of the parallax magnitude of SP i, h , the dimension of xi, h is 2, xi, h represents the coordinate position of the center pixel of SP i, h , s i, h represents the area of SP i, h ;
①-5、计算{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的背景先验特征矢量,将Ri中的第h个区域SPi,h的背景先验特征矢量记为
①-6、将{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的对比度特征矢量、通用特征矢量和背景先验特征矢量按顺序进行排列,构成{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的用于反映视觉显著性的特征矢量,将Ri中的第h个区域SPi,h的用于反映视觉显著性的特征矢量记为Xi,h,其中,Xi,h的维数为105,此处符号“[]”为矢量表示符号;①-6. The contrast feature vector, general feature vector and background prior feature vector of each region in the right view image of each stereo image in {L i , R i , d i |1≤i≤N} Arranged in order to form the feature vector used to reflect the visual salience of each region in the right view image of each stereoscopic image in {L i , R i , d i |1≤i≤N}, R The feature vector used to reflect the visual salience of the hth region SP i ,h in i is denoted as Xi ,h , Among them, the dimensions of X i, h are 105, and the symbol “[]” here is a vector representation symbol;
①-7、采用随机森林回归,对{Li,Ri,di|1≤i≤N}中的所有立体图像的右视点图像中的所有区域的用于反映视觉显著性的特征矢量进行训练,并使得经过训练得到的回归函数值与平均眼动值之间的误差最小,得到最优的随机森林回归训练模型,记为f(Dinp),其中,f()为函数表示形式,Dinp表示随机森林回归训练模型的输入矢量;①-7. Use random forest regression to perform feature vectors for reflecting visual salience of all areas in the right viewpoint images of all stereoscopic images in {L i , R i , d i |1≤i≤N} training, and make the error between the regression function value obtained through training and the average eye movement value the smallest, and obtain the optimal random forest regression training model, which is denoted as f(D inp ), where f() is the function representation, D inp represents the input vector of the random forest regression training model;
所述的测试阶段的具体步骤如下:The specific steps of the testing phase are as follows:
②-1、对于任意一副测试立体图像Stest,将Stest的左视点图像、右视点图像、右视差图像对应记为Ltest、Rtest、dtest;然后采用超像素分割技术将Rtest分割成M个互不重叠的区域,将Rtest中的第h个区域记为SPh';其中,M≥1,1≤h≤M;②-1. For any pair of test stereo images S test , record the left viewpoint image, right viewpoint image, and right disparity image of S test as L test , R test , and d test respectively ; then use superpixel segmentation technology to divide R test Divide into M non-overlapping areas, and record the hth area in R test as SP h '; where, M≥1, 1≤h≤M;
②-2、按照步骤①-3至步骤①-6的过程,以相同的操作方式获取Rtest中的每个区域的用于反映视觉显著性的特征矢量,将Rtest中的第h个区域SPh'的用于反映视觉显著性的特征矢量记为Ftest,h;然后根据训练阶段得到的最优的随机森林回归训练模型f(Dinp),将Ftest,h作为最优的随机森林回归训练模型的输入矢量,获取Rtest中的每个区域的三维视觉显著值,将Rtest中的第h个区域SPh'的三维视觉显著值记为S3D,h,S3D,h=f(Ftest,h);再将Rtest中的每个区域的三维视觉显著值作为对应区域中的所有像素点的显著值,从而得到Rtest的三维显著图,记为{S3D(x,y)},其中,此处(x,y)表示Stest中的像素点的坐标位置,1≤x≤W,1≤y≤H,W和H对应表示Stest的宽度和高度,S3D(x,y)表示{S3D(x,y)}中坐标位置为(x,y)的像素点的像素值。②-2. According to the process from step ①-3 to step ①-6, obtain the feature vector used to reflect the visual salience of each region in the R test in the same operation mode, and convert the hth region in the R test to The feature vector used to reflect the visual salience of SP h ' is recorded as F test,h ; then according to the optimal random forest regression training model f(D inp ) obtained in the training stage, F test,h is used as the optimal random The input vector of the forest regression training model, obtain the 3D visual saliency value of each region in the R test , and record the 3D visual saliency value of the hth region SP h ' in the R test as S 3D,h , S 3D,h = f(F test, h ); then take the 3D visual saliency value of each region in the R test as the saliency value of all pixels in the corresponding region, thereby obtaining the 3D saliency map of the R test , denoted as {S 3D ( x, y)}, where (x, y) represents the coordinate position of the pixel in the S test , 1≤x≤W, 1≤y≤H, W and H correspond to the width and height of the S test , S 3D (x, y) represents the pixel value of the pixel at the coordinate position (x, y) in {S 3D (x, y)}.
所述的步骤①-3中Ri中的第h个区域SPi,h的对比度特征矢量的获取过程为:The contrast feature vector of the hth region SP i,h in R i in the step ①-3 The acquisition process is:
a1、计算Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量的均值,记为fi,h,fi,h中的第个元素的值等于Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量中的第个元素的频率响应振幅的均值,其中,fi,h的维数为20, a1. Calculate the mean value of the frequency response feature vectors of all the pixel points in the hth region SP i,h in R i , denoted as f i,h , the first in f i,h The value of the element is equal to the h-th area SP i,h in the frequency response feature vector of all pixels in R i The mean value of the frequency response amplitude of elements, where the dimensions of f i,h are 20,
a2、计算Ri中的第h个区域SPi,h中的所有像素点的颜色特征矢量的均值,记为ci,h,
a3、计算Ri中的第h个区域SPi,h的视差幅值的均值,记为等于di中与SPi,h对应的区域中的所有像素点的像素值的均值;a3. Calculate the mean value of the parallax magnitude of the hth region SP i, h in R i , denoted as Equal to the mean value of the pixel values of all pixels in the area corresponding to SP i, h in d i ;
a4、将fi,h、ci,h和按顺序进行排列,构成Ri中的第h个区域SPi,h的第一特征矢量,记为ui,h,其中,ui,h的维数为30,此处符号“[]”为矢量表示符号;a4. Put f i, h , c i, h and Arranged in order to form the first feature vector of the hth region SP i,h in R i , denoted as u i,h , Among them, the dimensions of u i and h are 30, and the symbol “[]” here is a vector representation symbol;
a5、计算Ri中的第h个区域SPi,h的第一特征矢量ui,h与相邻区域的第一特征矢量的距离,记为di,h,其中,di,h的维数为30,1≤p≤M,表示Ri中的第h个区域SPi,h的相邻区域的序号的集合,ui,p表示Ri中的第p个区域SPi,p的第一特征矢量,符号“||”为取绝对值符号,P表示Ri中的第h个区域SPi,h的相邻区域的总个数,此处的相邻区域是指Ri中与SPi,h相邻的区域;a5. Calculate the distance between the first feature vector u i,h of the hth area SP i,h in R i and the first feature vector of the adjacent area, denoted as d i,h , Among them, the dimensions of d i, h are 30, 1≤p≤M, Represents the set of serial numbers of the adjacent regions of SP i, h in the hth region in R i , u i, p represents the first feature vector of the pth region SP i, p in R i , symbol "||" In order to take the absolute value sign, P represents the total number of adjacent areas of the hth area SP i, h in R i , where the adjacent area refers to the area adjacent to SP i, h in R i ;
a6、计算Ri中的第h个区域SPi,h中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的H分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的S分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的LBP特征统计直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的视差统计直方图,记为其中,的维数为163,的维数为163,的维数为16,的维数为16,的维数为256,的维数为16;a6. Calculate the color histogram of the R component, G component and B component of all pixels in the hth region SP i,h in R i in the RGB color space, denoted as Calculate the color histogram of the L component, a component and b component of all pixels in the hth area SP i, h in R i in the CIELAB color space, denoted as Calculate the color histogram of the H component of all pixels in the hth region SP i,h in R i in the HVS color space, denoted as Calculate the color histogram of the S component of the HVS color space for all pixels in the hth region SP i,h in R i , denoted as Calculate the LBP feature statistical histogram of all pixels in the hth region SP i,h in R i , denoted as Calculate the disparity statistical histogram of all pixels in the hth area SP i,h in R i , denoted as in, The dimension of is 16 3 , The dimension of is 16 3 , The dimension of is 16, The dimension of is 16, The dimension of is 256, The dimension of is 16;
a7、计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图的距离,记为 a7. Calculate The distance from the color histogram of the R component, G component and B component of all pixels in the adjacent area of the hth area SP i, h in R i in the RGB color space is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the L component, a component and b component of the CIELAB color space to all pixels in the adjacent area of the hth area SP i, h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在HVS颜色空间的H分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the H component of the HVS color space to all pixels in the adjacent area of the h-th area SP i, h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在HVS颜色空间的S分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the S component of the HVS color space to all pixels in the adjacent area of the hth area SP i,h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点的LBP特征统计直方图的距离,记为 calculate The distance from the LBP feature statistical histogram of all pixels in the adjacent area of the hth area SP i,h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点的视差统计直方图的距离,记为 calculate The distance from the disparity statistical histogram of all pixels in the adjacent area of SP i, h in the hth area SP i, h in R i is denoted as
其中,1≤p≤M,表示Ri中的第h个区域SPi,h的相邻区域的序号的集合,P表示Ri中的第h个区域SPi,h的相邻区域的总个数,χ()为求卡方距离函数,表示Ri中的第p个区域SPi,p中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点在HVS颜色空间的H分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点在HVS颜色空间的S分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点的LBP特征统计直方图,表示Ri中的第p个区域SPi,p中的所有像素点的视差统计直方图;Among them, 1≤p≤M, Indicates the set of serial numbers of the adjacent areas of the hth area SP i, h in R i , P indicates the total number of adjacent areas of the hth area SP i, h in R i , χ() is chi-square distance function, Indicates the color histogram of the R component, G component and B component of all pixels in the pth area SP i,p in R i in the RGB color space, Represents the color histogram of the L component, a component and b component of all pixels in the pth area SP i,p in R i in the CIELAB color space, Represents the color histogram of the H component of the HVS color space for all pixels in the p-th region SP i,p in R i , Represents the color histogram of the S component of the HVS color space for all pixels in the p-th region SP i,p in R i , Represents the LBP feature statistical histogram of all pixels in the p-th region SP i,p in R i , Represents the disparity statistical histogram of all pixels in the p-th region SP i, p in R i ;
a8、将di,h、和按顺序进行排列,构成Ri中的第h个区域SPi,h的对比度特征矢量,记为
所述的步骤①-4中Ri中的第h个区域SPi,h的通用特征矢量的获取过程为:The general feature vector of the h-th area SP i,h in R i in the step ①-4 The acquisition process is:
b1、计算Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量的方差,记为中的第个元素的值等于Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量中的第个元素的频率响应振幅的方差,其中,的维数为20, b1. Calculate the variance of the frequency response feature vectors of all pixels in the hth region SP i,h in R i , denoted as in the first The value of the element is equal to the h-th area SP i,h in the frequency response feature vector of all pixels in R i The variance of the frequency response amplitude of elements, where, The dimension of is 20,
b2、计算Ri中的第h个区域SPi,h中的所有像素点的颜色特征矢量的方差,记为
b3、计算Ri中的第h个区域SPi,h的视差幅值的方差,记为等于di中与SPi,h对应的区域中的所有像素点的像素值的方差;b3. Calculate the variance of the parallax magnitude of the hth region SP i in R i , h , denoted as Equal to the variance of the pixel values of all pixels in the area corresponding to SP i, h in d i ;
b4、获取Ri中的第h个区域SPi,h的中心像素点的坐标位置,记为xi,h,其中,xi,h的维数为2;b4. Obtain the coordinate position of the central pixel point of the hth region SP i,h in R i , denoted as x i,h , where the dimension of x i,h is 2;
b5、计算Ri中的第h个区域SPi,h的面积,记为si,h;b5. Calculate the area of the hth region SP i,h in R i , and denote it as s i,h ;
b6、将xi,h和si,h按顺序进行排列,构成Ri中的第h个区域SPi,h的通用特征矢量,记为
所述的步骤①-5中Ri中的第h个区域SPi,h的背景先验特征矢量的获取过程为:The background priori feature vector of the hth region SP i,h in R i in the step ①-5 The acquisition process is:
c1、计算Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量的均值,记为fi,h,fi,h中的第个元素的值等于Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量中的第个元素的频率响应振幅的均值,其中,fi,h的维数为20, c1. Calculate the mean value of the frequency response feature vectors of all pixels in the hth area SP i,h in R i , denoted as f i,h , the first in f i,h The value of the element is equal to the h-th area SP i,h in the frequency response feature vector of all pixels in R i The mean value of the frequency response amplitude of elements, where the dimensions of f i,h are 20,
c2、计算Ri中的第h个区域SPi,h中的所有像素点的颜色特征矢量的均值,记为ci,h,
c3、计算Ri中的第h个区域SPi,h的视差幅值的均值,记为等于di中与SPi,h对应的区域中的所有像素点的像素值的均值;c3. Calculate the mean value of the parallax magnitude of the hth region SP i, h in R i , denoted as Equal to the mean value of the pixel values of all pixels in the area corresponding to SP i, h in d i ;
c4、将fi,h、ci,h和按顺序进行排列,构成Ri中的第h个区域SPi,h的第一特征矢量,记为ui,h,其中,ui,h的维数为30,此处符号“[]”为矢量表示符号;c4, put f i, h , c i, h and Arranged in order to form the first feature vector of the hth region SP i,h in R i , denoted as u i,h , Among them, the dimensions of u i and h are 30, and the symbol “[]” here is a vector representation symbol;
c5、计算Ri中的第h个区域SPi,h的第一特征矢量ui,h与背景区域的第一特征矢量的距离,记为ei,h,其中,ei,h的维数为30,1≤q≤M,表示Ri中的所有背景区域的序号的集合,ui,q表示Ri中的第q个区域SPi,q的第一特征矢量,符号“||”为取绝对值符号,Q表示Ri中的背景区域的总个数,此处的背景区域是指Ri中位于最左边、最右边、最上边、最下边的区域;c5. Calculate the distance between the first feature vector u i,h of the hth area SP i,h in R i and the first feature vector of the background area, denoted as e i,h , Among them, the dimensions of e i, h are 30, 1≤q≤M, Represents the set of serial numbers of all background regions in R i , u i, q represents the first feature vector of the qth region SP i, q in R i , the symbol "||" is an absolute value symbol, Q represents R The total number of background regions in i , where the background region refers to the regions located in the leftmost, rightmost, uppermost, and lowermost in R i ;
c6、计算Ri中的第h个区域SPi,h中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的H分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的S分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的LBP特征统计直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的视差统计直方图,记为其中,的维数为163,的维数为163,的维数为16,的维数为16,的维数为256,的维数为16;c6. Calculate the color histogram of the R component, G component and B component of all pixels in the hth region SP i,h in R i in the RGB color space, denoted as Calculate the color histogram of the L component, a component and b component of all pixels in the hth area SP i, h in R i in the CIELAB color space, denoted as Calculate the color histogram of the H component of all pixels in the hth region SP i,h in R i in the HVS color space, denoted as Calculate the color histogram of the S component of the HVS color space for all pixels in the hth region SP i,h in R i , denoted as Calculate the LBP feature statistical histogram of all pixels in the hth area SP i,h in R i , denoted as Calculate the disparity statistical histogram of all pixels in the hth area SP i,h in R i , denoted as in, The dimension of is 16 3 , The dimension of is 16 3 , The dimension of is 16, The dimension of is 16, The dimension of is 256, The dimension of is 16;
c7、计算与Ri中的背景区域中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图的距离,记为 c7, calculate The distance from the color histogram of the R component, G component and B component of the RGB color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the L component, a component and b component of the CIELAB color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点在HVS颜色空间的H分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the H component of the HVS color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点在HVS颜色空间的S分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the S component of the HVS color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点的LBP特征统计直方图的距离,记为 calculate The distance from the LBP feature statistical histogram of all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点的视差统计直方图的距离,记为 calculate The distance from the disparity statistical histogram of all pixels in the background area in R i is denoted as
其中,1≤q≤M,表示Ri中的所有背景区域的序号的集合,Q表示Ri中的背景区域的总个数,χ()为求卡方距离函数,表示Ri中的第q个区域SPi,q中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点在HVS颜色空间的H分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点在HVS颜色空间的S分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点的LBP特征统计直方图,表示Ri中的第q个区域SPi,q中的所有像素点的视差统计直方图;Among them, 1≤q≤M, Represent the collection of the sequence numbers of all background regions in R i , Q represents the total number of background regions in R i , χ () is the chi-square distance function, Represents the color histogram of the R component, G component and B component of all pixels in the qth area SP i,q in R i in the RGB color space, Represents the color histogram of the L component, a component and b component of all pixels in the qth region SP i, q in R i in the CIELAB color space, Represents the color histogram of the H component of all pixels in the qth region SP i,q in R i in the HVS color space, Represents the color histogram of the S component of the HVS color space for all pixels in the qth region SP i,q in R i , Represents the LBP feature statistical histogram of all pixels in the qth region SP i,q in R i , Represents the disparity statistical histogram of all pixels in the qth region SP i, q in R i ;
c8、将ei,h、 按顺序进行排列,构成Ri中的第h个区域SPi,h的背景先验特征矢量,记为
所述的Ri中的每个像素点的频率响应特征矢量的获取过程为:The acquisition process of the frequency response feature vector of each pixel in the R i is:
1)-1、采用Gabor滤波器对Ri进行滤波处理,得到Ri中的每个像素点在不同中心频率和不同方向因子下的频率响应振幅,将Ri中坐标位置为(x,y)的像素点在中心频率为ω和方向因子为θ下的频率响应振幅记为G(x,y;ω,θ),其中,此处(x,y)表示{Li,Ri,di|1≤i≤N}中的立体图像中的像素点的坐标位置,1≤x≤W,1≤y≤H,W和H对应表示{Li,Ri,di|1≤i≤N}中的立体图像的宽度和高度,ω表示Gabor滤波器的中心频率,ω∈Φω,θ表示Gabor滤波器的方向因子,θ∈Φθ,Φω表示Gabor滤波器的所有中心频率的集合,Φθ表示Gabor滤波器的所有方向因子的集合;1)-1. Use the Gabor filter to filter R i to obtain the frequency response amplitude of each pixel in R i at different center frequencies and different direction factors, and set the coordinate position in R i as (x, y ) pixel at center frequency ω and direction factor θ is denoted as G(x,y; ω,θ), where (x,y) means {L i ,R i ,d The coordinate position of the pixel in the stereoscopic image in i |1≤i≤N}, 1≤x≤W, 1≤y≤H, W and H correspond to {L i , R i , d i |1≤i ≤ N} in the width and height of the stereo image, ω represents the center frequency of the Gabor filter, ω∈Φ ω , θ represents the direction factor of the Gabor filter, θ∈Φ θ , Φ ω represents all the center frequencies of the Gabor filter The set of , Φ θ represents the set of all direction factors of the Gabor filter;
1)-2、将Ri中的每个像素点在不同中心频率和不同方向因子下的频率响应振幅按顺序进行排列,构成Ri中的每个像素点的频率响应特征矢量,将Ri中坐标位置为(x,y)的像素点的频率响应特征矢量记为fi(x,y),其中,fi(x,y)的维数为20。1)-2. Arrange the frequency response amplitudes of each pixel point in R i in different center frequencies and different direction factors in order to form the frequency response feature vector of each pixel point in R i , and set R i The frequency response feature vector of the pixel at the middle coordinate position (x, y) is recorded as f i (x, y), where the dimension of f i (x, y) is 20.
所述的Ri中的每个像素点的颜色特征矢量的获取过程为:The acquisition process of the color feature vector of each pixel in the R i is:
2)-1、计算Ri中的每个像素点在不同颜色空间的颜色值,将Ri中坐标位置为(x,y)的像素点在RGB颜色空间的R分量、G分量和B分量的颜色值分别记为R(x,y)、G(x,y)和B(x,y),将Ri中坐标位置为(x,y)的像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色值分别记为L(x,y)、a(x,y)和b(x,y),将Ri中坐标位置为(x,y)的像素点在HVS颜色空间的H分量、V分量和S分量的颜色值分别记为H(x,y)、V(x,y)和S(x,y),其中,此处(x,y)表示{Li,Ri,di|1≤i≤N}中的立体图像中的像素点的坐标位置,1≤x≤W,1≤y≤H,W和H对应表示{Li,Ri,di|1≤i≤N}中的立体图像的宽度和高度;2)-1. Calculate the color value of each pixel point in R i in different color spaces, and use the R component, G component and B component of the pixel point whose coordinate position is (x, y) in R i in the RGB color space The color values of are recorded as R( x , y), G(x, y) and B(x, y) respectively, and the L component, The color values of a component and b component are recorded as L(x, y), a(x, y) and b(x, y) respectively, and the pixel point whose coordinate position in R i is (x, y) is in HVS color The color values of the H component, V component and S component of the space are respectively recorded as H(x, y), V(x, y) and S(x, y), where (x, y) here represents {L i , R i , d i |1≤i≤N} in the coordinate position of the pixel in the stereo image, 1≤x≤W, 1≤y≤H, W and H correspond to {L i ,R i ,d Width and height of stereo images in i |1≤i≤N};
2)-2、将Ri中的每个像素点在不同颜色空间的颜色值按顺序进行排列,构成Ri中的每个像素点的颜色特征矢量,将Ri中坐标位置为(x,y)的像素点的颜色特征矢量记为ci(x,y),ci(x,y)=[R(x,y),G(x,y),B(x,y),L(x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)],其中,ci(x,y)的维数为9,此处符号“[]”为矢量表示符号。2)-2. Arrange the color values of each pixel in R i in different color spaces in order to form the color feature vector of each pixel in R i , and set the coordinate position in R i as (x, y) is recorded as c i (x, y), c i (x, y) = [R(x, y), G(x, y), B(x, y), L (x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)], where c i (x,y ) has a dimension of 9, where the symbol “[]” is a vector representation symbol.
与现有技术相比,本发明的优点在于:Compared with the prior art, the present invention has the advantages of:
1)本发明方法同时考虑了立体图像的右视点图像中的每个区域的对比度特征矢量、通用特征矢量和背景先验特征矢量,并融合得到立体图像的右视点图像中的每个区域的用于反映视觉显著性的特征矢量,因此本发明方法具有较高的提取准确性和较强的稳定性,并能够较好地反映各种因素的显著变化情况,符合显著语义特征。1) The method of the present invention simultaneously considers the contrast feature vector, general feature vector and background prior feature vector of each region in the right viewpoint image of the stereoscopic image, and fuses to obtain the use of each region in the right viewpoint image of the stereoscopic image. The method of the present invention has high extraction accuracy and strong stability, and can better reflect the significant changes of various factors, and conforms to the significant semantic features.
2)本发明方法通过训练构建用于反映视觉显著性的特征矢量与平均眼动值之间的随机森林回归训练模型,然后利用该随机森林回归训练模型来预测测试立体图像的右视点图像中的每个区域的三维视觉显著值,从而得到测试立体图像的三维显著图,有效地提高了视觉显著值的预测准确性。2) The method of the present invention constructs the random forest regression training model between the feature vector and the average eye movement value for reflecting visual salience by training, then utilizes this random forest regression training model to predict the The 3D visual saliency value of each region is obtained to obtain the 3D saliency map of the test stereo image, which effectively improves the prediction accuracy of the visual saliency value.
附图说明Description of drawings
图1为本发明方法的总体实现框图;Fig. 1 is the overall realization block diagram of the inventive method;
图2a为“Image1”的右视点图像;Figure 2a is the right view image of "Image1";
图2b为“Image1”的右视点图像的真实眼动图;Figure 2b is the real eye movement diagram of the right viewpoint image of "Image1";
图2c为“Image1”的三维显著图;Figure 2c is the 3D saliency map of "Image1";
图3a为“Image2”的右视点图像;Figure 3a is the right view image of "Image2";
图3b为“Image2”的右视点图像的真实眼动图;Figure 3b is the real eye movement diagram of the right viewpoint image of "Image2";
图3c为“Image2”的三维显著图;Figure 3c is the 3D saliency map of "Image2";
图4a为“Image3”的右视点图像;Figure 4a is the right view image of "Image3";
图4b为“Image3”的右视点图像的真实眼动图;Figure 4b is the real eye movement diagram of the right viewpoint image of "Image3";
图4c为“Image3”的三维显著图;Figure 4c is the 3D saliency map of "Image3";
图5a为“Image4”的右视点图像;Figure 5a is the right view image of "Image4";
图5b为“Image4”的右视点图像的真实眼动图;Figure 5b is the real eye movement diagram of the right viewpoint image of "Image4";
图5c为“Image4”的三维显著图;Figure 5c is the 3D saliency map of "Image4";
图6a为“Image5”的右视点图像;Figure 6a is the right view image of "Image5";
图6b为“Image5”的右视点图像的真实眼动图;Figure 6b is the real eye movement diagram of the right viewpoint image of "Image5";
图6c为“Image5”的三维显著图;Figure 6c is the 3D saliency map of "Image5";
图7a为“Image6”的右视点图像;Figure 7a is the right view image of "Image6";
图7b为“Image6”的右视点图像的真实眼动图;Figure 7b is the real eye movement diagram of the right viewpoint image of "Image6";
图7c为“Image6”的三维显著图;Figure 7c is the 3D saliency map of "Image6";
图8a为“Image7”的右视点图像;Figure 8a is the right view image of "Image7";
图8b为“Image7”的右视点图像的真实眼动图;Figure 8b is the real eye movement diagram of the right viewpoint image of "Image7";
图8c为“Image7”的三维显著图。Figure 8c is the 3D saliency map of "Image7".
具体实施方式detailed description
以下结合附图实施例对本发明作进一步详细描述。The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments.
本发明提出的一种立体图像显著图提取方法,其总体实现框图如图1所示,其包括训练阶段和测试阶段两个过程,训练阶段的具体步骤如下:A kind of three-dimensional image saliency map extracting method that the present invention proposes, its overall realization block diagram as shown in Figure 1, it comprises two processes of training phase and testing phase, and the concrete steps of training phase are as follows:
①-1、将选取的N副各不相同的立体图像以及每幅立体图像的右视差图像构成一个集合,记为{Li,Ri,di|1≤i≤N},其中,N≥1,在本实施例中取N=600,Li表示{Li,Ri,di|1≤i≤N}中的第i幅立体图像的左视点图像,Ri表示{Li,Ri,di|1≤i≤N}中的第i幅立体图像的右视点图像,di表示{Li,Ri,di|1≤i≤N}中的第i幅立体图像的右视差图像。①-1. The selected N sets of different stereoscopic images and the right disparity images of each stereoscopic image form a set, which is recorded as {L i , R i , d i |1≤i≤N}, where N ≥1, N=600 in this embodiment, L i represents the left viewpoint image of the i-th stereoscopic image in {L i , R i , d i |1≤i≤N}, and R i represents {L i ,R i ,d i |1≤i≤N} in the right view image of the i-th stereo image, d i represents the i-th stereo in {L i ,R i ,d i |1≤i≤N} Right disparity image of the image.
在本实施例中,采用新加坡国立大学提供的三维人眼跟踪数据库(NUS 3D-Saliency database)构造训练立体图像集,该立体图像数据库包含600副立体图像以及对应的右视差图像,并给出了每副立体图像的真实眼动图。In this embodiment, the three-dimensional human eye tracking database (NUS 3D-Saliency database) provided by the National University of Singapore (NUS 3D-Saliency database) is used to construct a training stereoscopic image set. The stereoscopic image database contains 600 pairs of stereoscopic images and corresponding right parallax images. Realistic eye-movement maps for each stereoscopic image.
①-2、采用现有的超像素分割技术将{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像分割成M个互不重叠的区域,将Ri中的第h个区域记为SPi,h,可将{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像重新表示为M个区域的集合,将Ri重新表示的M个区域的集合记为{SPi,h};其中,M≥1,在本实施例中取M=400,1≤h≤M。①-2. Use the existing superpixel segmentation technology to segment the right viewpoint image of each stereoscopic image in {L i , R i , d i |1≤i≤N} into M non-overlapping regions, and The h-th region in R i is denoted as SP i,h , and the right viewpoint image of each stereo image in {L i ,R i ,d i |1≤i≤N} can be re-expressed as M regions A set, the set of M regions re-expressed by R i is denoted as {SP i,h }; where M≥1, M=400 in this embodiment, 1≤h≤M.
①-3、计算{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的对比度特征矢量,将Ri中的第h个区域SPi,h的对比度特征矢量记为
在此具体实施例中,步骤①-3中Ri中的第h个区域SPi,h的对比度特征矢量的获取过程为:In this specific embodiment, the contrast feature vector of the h-th region SP i,h in R i in step ①-3 The acquisition process is:
a1、计算Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量的均值,记为fi,h,fi,h中的第个元素的值等于Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量中的第个元素的频率响应振幅的均值,其中,fi,h的维数为20, a1. Calculate the mean value of the frequency response feature vectors of all pixels in the hth area SP i,h in R i , denoted as f i,h , the first in f i,h The value of the element is equal to the h-th area SP i,h in the frequency response feature vector of all pixels in R i The mean value of the frequency response amplitude of elements, where the dimension of f i,h is 20,
a2、计算Ri中的第h个区域SPi,h中的所有像素点的颜色特征矢量的均值,记为ci,h,
a3、计算Ri中的第h个区域SPi,h的视差幅值的均值,记为等于di中与SPi,h对应的区域中的所有像素点的像素值的均值。a3. Calculate the mean value of the parallax magnitude of the hth region SP i, h in R i , denoted as It is equal to the mean value of the pixel values of all the pixel points in the area corresponding to SP i, h in d i .
a4、将fi,h、ci,h和按顺序进行排列,构成Ri中的第h个区域SPi,h的第一特征矢量,记为ui,h,其中,ui,h的维数为30,此处符号“[]”为矢量表示符号。a4. Put f i, h , c i, h and Arranged in order to form the first feature vector of the hth area SP i,h in R i , denoted as u i,h , Wherein, the dimensions of u i, h are 30, and the symbol “[]” here is a vector representation symbol.
a5、计算Ri中的第h个区域SPi,h的第一特征矢量ui,h与相邻区域的第一特征矢量的距离,记为di,h,其中,di,h的维数为30,1≤p≤M,表示Ri中的第h个区域SPi,h的相邻区域的序号的集合,ui,p表示Ri中的第p个区域SPi,p(SPi,p为SPi,h的相邻区域)的第一特征矢量,符号“||”为取绝对值符号,P表示Ri中的第h个区域SPi,h的相邻区域的总个数,在本实施例中取P=20,此处的相邻区域是指Ri中与SPi,h相邻的区域。a5. Calculate the distance between the first feature vector u i,h of the hth area SP i,h in R i and the first feature vector of the adjacent area, denoted as d i,h , Among them, the dimensions of d i, h are 30, 1≤p≤M, Indicates the set of serial numbers of the adjacent areas of the hth area SP i,h in R i , u i,p indicates the pth area SP i,p in R i (SP i,p is the value of SP i,h Adjacent region), the symbol "||" is an absolute value symbol, and P represents the total number of adjacent regions of the hth region SP i, h in R i , which is taken in this embodiment P=20, the adjacent region here refers to the region adjacent to SP i,h in R i .
a6、计算Ri中的第h个区域SPi,h中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的H分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的S分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的LBP特征统计直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的视差统计直方图,记为其中,的维数为163,的维数为163,的维数为16,的维数为16,的维数为256,的维数为16。a6. Calculate the color histogram of the R component, G component and B component of all pixels in the hth region SP i,h in R i in the RGB color space, denoted as Calculate the color histogram of the L component, a component and b component of all pixels in the hth area SP i, h in R i in the CIELAB color space, denoted as Calculate the color histogram of the H component of all pixels in the hth region SP i,h in R i in the HVS color space, denoted as Calculate the color histogram of the S component of the HVS color space for all pixels in the hth region SP i,h in R i , denoted as Calculate the LBP feature statistical histogram of all pixels in the hth region SP i,h in R i , denoted as Calculate the disparity statistical histogram of all pixels in the hth area SP i,h in R i , denoted as in, The dimension of is 16 3 , The dimension of is 16 3 , The dimension of is 16, The dimension of is 16, The dimension of is 256, The dimension of is 16.
a7、计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图的距离,记为 a7. Calculate The distance from the color histogram of the R component, G component and B component of all pixels in the adjacent area of the hth area SP i, h in R i in the RGB color space is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the L component, a component and b component of the CIELAB color space to all pixels in the adjacent area of the hth area SP i, h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在HVS颜色空间的H分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the H component of the HVS color space to all pixels in the adjacent area of the hth area SP i,h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点在HVS颜色空间的S分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the S component of the HVS color space to all pixels in the adjacent area of the hth area SP i,h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点的LBP特征统计直方图的距离,记为 calculate The distance from the LBP feature statistical histogram of all pixels in the adjacent area of the hth area SP i,h in R i is denoted as
计算与Ri中的第h个区域SPi,h的相邻区域中的所有像素点的视差统计直方图的距离,记为 calculate The distance from the disparity statistical histogram of all pixels in the adjacent area of SP i, h in the hth area SP i, h in R i is denoted as
其中,1≤p≤M,表示Ri中的第h个区域SPi,h的相邻区域的序号的集合,P表示Ri中的第h个区域SPi,h的相邻区域的总个数,在本实施例中取P=20,χ()为求卡方距离(Chi-distance measure)函数,表示Ri中的第p个区域SPi,p中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点在HVS颜色空间的H分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点在HVS颜色空间的S分量的颜色直方图,表示Ri中的第p个区域SPi,p中的所有像素点的LBP特征统计直方图,表示Ri中的第p个区域SPi,p中的所有像素点的视差统计直方图。Among them, 1≤p≤M, Represents the set of the sequence numbers of the adjacent regions of the hth region SP i, h in R i , and P represents the total number of adjacent regions of the hth region SP i, h in R i , in this embodiment Take P=20, χ() is the function of seeking chi-square distance (Chi-distance measure), Represents the color histogram of the R component, G component and B component of all pixels in the pth area SP i,p in R i in the RGB color space, Represents the color histogram of the L component, a component and b component of all pixels in the pth area SP i,p in R i in the CIELAB color space, Represents the color histogram of the H component of the HVS color space for all pixels in the p-th region SP i,p in R i , Represents the color histogram of the S component of the HVS color space for all pixels in the p-th region SP i,p in R i , Represents the LBP feature statistical histogram of all pixels in the p-th region SP i,p in R i , Represents the disparity statistical histogram of all pixels in the p-th region SP i,p in R i .
a8、将di,h、和按顺序进行排列,构成Ri中的第h个区域SPi,h的对比度特征矢量,记为
①-4、计算{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的通用特征矢量,将Ri中的第h个区域SPi,h的通用特征矢量记为 其中,的维数为33,此处符号“[]”为矢量表示符号,的维数为20,表示SPi,h中的所有像素点的频率响应特征矢量的方差,的维数为9,表示SPi,h中的所有像素点的颜色特征矢量的方差,表示SPi,h的视差幅值的方差,xi,h的维数为2,xi,h表示SPi,h的中心像素点的坐标位置,si,h表示SPi,h的面积。①-4. Calculate the general feature vector of each region in the right view point image of each stereoscopic image in {L i , R i , d i |1≤i≤N}, and divide the hth region in R i The general eigenvector of SP i,h is denoted as in, The dimension of is 33, where the symbol “[]” is a vector representation symbol, The dimension of is 20, Represents the variance of the frequency response feature vector of all pixels in SP i,h , The dimension of is 9, Represents the variance of the color feature vectors of all pixels in SP i,h , Represents the variance of the parallax magnitude of SP i, h , the dimension of xi, h is 2, xi, h represents the coordinate position of the center pixel of SP i, h , s i, h represents the area of SP i, h .
在此具体实施例中,步骤①-4中Ri中的第h个区域SPi,h的通用特征矢量的获取过程为:In this specific embodiment, the universal feature vector of the hth region SP i,h in R i in step ①-4 The acquisition process is:
b1、计算Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量的方差,记为中的第个元素的值等于Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量中的第个元素的频率响应振幅的方差,其中,的维数为20, b1. Calculate the variance of the frequency response feature vectors of all pixels in the hth region SP i, h in R i , denoted as in the first The value of the element is equal to the h-th area SP i,h in the frequency response feature vector of all pixels in R i The variance of the frequency response amplitude of elements, where, The dimension of is 20,
b2、计算Ri中的第h个区域SPi,h中的所有像素点的颜色特征矢量的方差,记为
b3、计算Ri中的第h个区域SPi,h的视差幅值的方差,记为等于di中与SPi,h对应的区域中的所有像素点的像素值的方差。b3. Calculate the variance of the parallax magnitude of the hth region SP i, h in R i , denoted as It is equal to the variance of the pixel values of all pixels in the area corresponding to SP i, h in d i .
b4、获取Ri中的第h个区域SPi,h的中心像素点的坐标位置,记为xi,h,其中,xi,h的维数为2。b4. Obtain the coordinate position of the center pixel of the hth area SP i,h in R i , denoted as x i,h , where the dimension of x i,h is 2.
b5、计算Ri中的第h个区域SPi,h的面积,记为si,h。b5. Calculate the area of the hth region SP i,h in R i , which is denoted as s i,h .
b6、将xi,h和si,h按顺序进行排列,构成Ri中的第h个区域SPi,h的通用特征矢量,记为
①-5、计算{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的背景先验特征矢量,将Ri中的第h个区域SPi,h的背景先验特征矢量记为
在此具体实施例中,步骤①-5中Ri中的第h个区域SPi,h的背景先验特征矢量的获取过程为:In this specific embodiment, the background prior feature vector of the hth region SP i,h in R i in step ①-5 The acquisition process is:
c1、计算Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量的均值,记为fi,h,fi,h中的第个元素的值等于Ri中的第h个区域SPi,h中的所有像素点的频率响应特征矢量中的第个元素的频率响应振幅的均值,其中,fi,h的维数为20, c1. Calculate the mean value of the frequency response feature vectors of all the pixel points in the hth area SP i,h in R i , denoted as f i,h , the first in f i,h The value of the element is equal to the h-th area SP i,h in the frequency response feature vector of all pixels in R i The mean value of the frequency response amplitude of elements, where the dimension of f i,h is 20,
c2、计算Ri中的第h个区域SPi,h中的所有像素点的颜色特征矢量的均值,记为ci,h,
c3、计算Ri中的第h个区域SPi,h的视差幅值的均值,记为等于di中与SPi,h对应的区域中的所有像素点的像素值的均值。c3. Calculate the mean value of the parallax magnitude of the hth region SP i, h in R i , denoted as It is equal to the mean value of the pixel values of all the pixel points in the area corresponding to SP i, h in d i .
c4、将fi,h、ci,h和按顺序进行排列,构成Ri中的第h个区域SPi,h的第一特征矢量,记为ui,h,其中,ui,h的维数为30,此处符号“[]”为矢量表示符号;c4, put f i, h , c i, h and Arranged in order to form the first feature vector of the hth region SP i,h in R i , denoted as u i,h , Among them, the dimensions of u i and h are 30, and the symbol “[]” here is a vector representation symbol;
c5、计算Ri中的第h个区域SPi,h的第一特征矢量ui,h与背景区域的第一特征矢量的距离,记为ei,h,其中,ei,h的维数为30,1≤q≤M,表示Ri中的所有背景区域的序号的集合,ui,q表示Ri中的第q个区域SPi,q(SPi,q为Ri中的背景区域)的第一特征矢量,符号“||”为取绝对值符号,Q表示Ri中的背景区域的总个数,此处的背景区域是指Ri中位于最左边、最右边、最上边、最下边的区域,即将落在Ri中的最左边、最右边、最上边和最下边的区域作为背景区域。c5. Calculate the distance between the first feature vector u i,h of the hth area SP i,h in R i and the first feature vector of the background area, denoted as e i,h , Among them, the dimension of e i, h is 30, 1≤q≤M, Represents the set of serial numbers of all background regions in R i , u i,q represents the first feature vector of the qth region SP i, q in R i (SP i,q is the background region in R i ), symbol "||" is the absolute value symbol, Q represents the total number of background areas in R i , where the background area refers to the leftmost, rightmost, uppermost, and lowermost areas in The leftmost, rightmost, uppermost and lowermost regions in R i are used as background regions.
c6、计算Ri中的第h个区域SPi,h中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的H分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点在HVS颜色空间的S分量的颜色直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的LBP特征统计直方图,记为计算Ri中的第h个区域SPi,h中的所有像素点的视差统计直方图,记为其中,的维数为163,的维数为163,的维数为16,的维数为16,的维数为256,的维数为16。c6. Calculate the color histogram of the R component, G component and B component of all pixels in the hth region SP i,h in R i in the RGB color space, denoted as Calculate the color histogram of the L component, a component and b component of all pixels in the hth area SP i, h in R i in the CIELAB color space, denoted as Calculate the color histogram of the H component of all pixels in the hth region SP i,h in R i in the HVS color space, denoted as Calculate the color histogram of the S component of the HVS color space for all pixels in the hth region SP i,h in R i , denoted as Calculate the LBP feature statistical histogram of all pixels in the hth region SP i,h in R i , denoted as Calculate the disparity statistical histogram of all pixels in the hth area SP i,h in R i , denoted as in, The dimension of is 16 3 , The dimension of is 16 3 , The dimension of is 16, The dimension of is 16, The dimension of is 256, The dimension of is 16.
c7、计算与Ri中的背景区域中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图的距离,记为 c7, calculate The distance from the color histogram of the R component, G component and B component of the RGB color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the L component, a component and b component of the CIELAB color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点在HVS颜色空间的H分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the H component of the HVS color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点在HVS颜色空间的S分量的颜色直方图的距离,记为 calculate The distance from the color histogram of the S component of the HVS color space to all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点的LBP特征统计直方图的距离,记为 calculate The distance from the LBP feature statistical histogram of all pixels in the background area in R i is denoted as
计算与Ri中的背景区域中的所有像素点的视差统计直方图的距离,记为 calculate The distance from the disparity statistical histogram of all pixels in the background area in R i is denoted as
其中,1≤q≤M,表示Ri中的所有背景区域的序号的集合,Q表示Ri中的背景区域的总个数,χ()为求卡方距离(Chi-distance measure)函数,表示Ri中的第q个区域SPi,q中的所有像素点在RGB颜色空间的R分量、G分量和B分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点在HVS颜色空间的H分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点在HVS颜色空间的S分量的颜色直方图,表示Ri中的第q个区域SPi,q中的所有像素点的LBP特征统计直方图,表示Ri中的第q个区域SPi,q中的所有像素点的视差统计直方图。Among them, 1≤q≤M, Indicates the collection of the serial numbers of all background regions in R i , Q represents the total number of background regions in R i , χ() is the function of seeking chi-square distance (Chi-distance measure), Represents the color histogram of the R component, G component and B component of all pixels in the qth region SP i,q in R i in the RGB color space, Represents the color histogram of the L component, a component and b component of all pixels in the qth region SP i, q in R i in the CIELAB color space, Represents the color histogram of the H component of all pixels in the qth region SP i,q in R i in the HVS color space, Represents the color histogram of the S component of the HVS color space for all pixels in the qth region SP i,q in R i , Represents the LBP feature statistical histogram of all pixels in the qth region SP i,q in R i , Represents the disparity statistical histogram of all pixels in the qth region SP i,q in R i .
c8、将ei,h、和按顺序进行排列,构成Ri中的第h个区域SPi,h的背景先验特征矢量,记为
①-6、将{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的对比度特征矢量、通用特征矢量和背景先验特征矢量按顺序进行排列,构成{Li,Ri,di|1≤i≤N}中的每幅立体图像的右视点图像中的每个区域的用于反映视觉显著性的特征矢量,将Ri中的第h个区域SPi,h的用于反映视觉显著性的特征矢量记为Xi,h,其中,Xi,h的维数为105,此处符号“[]”为矢量表示符号。①-6. The contrast feature vector, general feature vector and background prior feature vector of each region in the right view image of each stereo image in {L i , R i , d i |1≤i≤N} Arranged in order to form the feature vector used to reflect the visual salience of each region in the right view image of each stereoscopic image in {L i , R i , d i |1≤i≤N}, R The feature vector used to reflect the visual salience of the hth region SP i ,h in i is denoted as Xi ,h , Wherein, the dimensions of X i, h are 105, and the symbol “[]” here is a vector representation symbol.
①-7、采用现有的随机森林回归,对{Li,Ri,di|1≤i≤N}中的所有立体图像的右视点图像中的所有区域的用于反映视觉显著性的特征矢量进行训练,并使得经过训练得到的回归函数值与平均眼动值之间的误差最小,得到最优的随机森林回归训练模型,记为f(Dinp),其中,f()为函数表示形式,Dinp表示随机森林回归训练模型的输入矢量。①-7. Use the existing random forest regression to reflect the visual salience of all regions in the right view point images of all stereoscopic images in {L i , R i , d i |1≤i≤N} The feature vector is trained, and the error between the regression function value obtained through training and the average eye movement value is minimized, and the optimal random forest regression training model is obtained, which is denoted as f(D inp ), where f() is the function Representation, D inp represents the input vector of the random forest regression training model.
测试阶段的具体步骤如下:The specific steps in the testing phase are as follows:
②-1、对于任意一副测试立体图像Stest,将Stest的左视点图像、右视点图像、右视差图像对应记为Ltest、Rtest、dtest;然后采用现有的超像素分割技术将Rtest分割成M个互不重叠的区域,将Rtest中的第h个区域记为SPh',可将Rtest重新表示为M个区域的集合,记为{SPh'};其中,M≥1,在本实施例中取M=400,1≤h≤M。②-1. For any pair of test stereo images S test , record the left viewpoint image, right viewpoint image, and right disparity image of S test as L test , R test , and d test correspondingly; then use the existing superpixel segmentation technology Divide the R test into M non-overlapping areas, record the hth area in the R test as SP h ', and re-express the R test as a collection of M areas, denoted as {SP h '}; where , M≥1, M=400 in this embodiment, 1≤h≤M.
②-2、按照步骤①-3至步骤①-6的过程,以相同的操作方式获取Rtest中的每个区域的用于反映视觉显著性的特征矢量,将Rtest中的第h个区域SPh'的用于反映视觉显著性的特征矢量记为Ftest,h,Ftest,h的获取过程为:计算Rtest中的第h个区域SPh'的对比度特征矢量记为
在此具体实施例中,Ri中的每个像素点的频率响应特征矢量的获取过程为:In this specific embodiment, the acquisition process of the frequency response feature vector of each pixel in R i is:
1)-1、采用Gabor滤波器对Ri进行滤波处理,得到Ri中的每个像素点在不同中心频率和不同方向因子下的频率响应振幅,将Ri中坐标位置为(x,y)的像素点在中心频率为ω和方向因子为θ下的频率响应振幅记为G(x,y;ω,θ),其中,此处(x,y)表示{Li,Ri,di|1≤i≤N}中的立体图像中的像素点的坐标位置,1≤x≤W,1≤y≤H,W和H对应表示{Li,Ri,di|1≤i≤N}中的立体图像的宽度和高度,ω表示Gabor滤波器的中心频率,ω∈Φω,θ表示Gabor滤波器的方向因子,θ∈Φθ,Φω表示Gabor滤波器的所有中心频率的集合,在本实施例中Φω={1.74,2.47,3.49,4.93,6.98},Φθ表示Gabor滤波器的所有方向因子的集合,在本实施例中Φθ={0°,90°,180°,270°}。1)-1. Use the Gabor filter to filter R i to obtain the frequency response amplitude of each pixel in R i at different center frequencies and different direction factors, and set the coordinate position in R i as (x, y ) pixel at the center frequency ω and the direction factor θ is denoted as G(x,y; ω,θ), where (x,y) means {L i ,R i ,d The coordinate position of the pixel in the stereoscopic image in i |1≤i≤N}, 1≤x≤W, 1≤y≤H, W and H correspond to {L i , R i , d i |1≤i ≤ N} in the width and height of the stereo image, ω represents the center frequency of the Gabor filter, ω∈Φ ω , θ represents the direction factor of the Gabor filter, θ∈Φ θ , Φ ω represents all the center frequencies of the Gabor filter The set of, in this embodiment Φ ω ={1.74,2.47,3.49,4.93,6.98}, Φ θ represents the set of all direction factors of the Gabor filter, in this embodiment Φ θ ={0°,90° ,180°,270°}.
1)-2、将Ri中的每个像素点在不同中心频率和不同方向因子下的频率响应振幅按顺序进行排列,构成Ri中的每个像素点的频率响应特征矢量,将Ri中坐标位置为(x,y)的像素点的频率响应特征矢量记为fi(x,y),fi(x,y)为由G(x,y;1.74,0°)、G(x,y;2.47,0°)、G(x,y;3.49,0°)、G(x,y;4.93,0°)、G(x,y;6.98,0°)、G(x,y;1.74,90°)、G(x,y;2.47,90°)、G(x,y;3.49,90°)、G(x,y;4.93,90°)、G(x,y;6.98,90°)、G(x,y;1.74,180°)、G(x,y;2.47,180°)、G(x,y;3.49,180°)、G(x,y;4.93,180°)、G(x,y;6.98,180°)、G(x,y;1.74,270°)、G(x,y;2.47,270°)、G(x,y;3.49,270°)、G(x,y;4.93,270°)、G(x,y;6.98,270°)按顺序排列构成的矢量,其中,fi(x,y)的维数为20。1)-2. Arrange the frequency response amplitudes of each pixel point in R i in different center frequencies and different direction factors in order to form the frequency response feature vector of each pixel point in R i , and set R i The frequency response feature vector of the pixel point whose coordinate position is (x, y) is recorded as f i (x, y), and f i (x, y) is defined by G(x, y; 1.74,0°), G( x,y; 2.47,0°), G(x,y; 3.49,0°), G(x,y; 4.93,0°), G(x,y; 6.98,0°), G(x,y; y; 1.74,90°), G(x,y; 2.47,90°), G(x,y; 3.49,90°), G(x,y; 4.93,90°), G(x,y; 6.98,90°), G(x,y; 1.74,180°), G(x,y; 2.47,180°), G(x,y; 3.49,180°), G(x,y; 4.93, 180°), G(x,y; 6.98,180°), G(x,y; 1.74,270°), G(x,y; 2.47,270°), G(x,y; 3.49,270°) ), G(x,y; 4.93,270°), G(x,y; 6.98,270°) are arranged in order to form a vector, where the dimension of f i (x,y) is 20.
在此具体实施例中,Ri中的每个像素点的颜色特征矢量的获取过程为:In this specific embodiment, the acquisition process of the color feature vector of each pixel in R i is:
2)-1、计算Ri中的每个像素点在不同颜色空间的颜色值,将Ri中坐标位置为(x,y)的像素点在RGB颜色空间的R分量、G分量和B分量的颜色值分别记为R(x,y)、G(x,y)和B(x,y),将Ri中坐标位置为(x,y)的像素点在CIELAB颜色空间的L分量、a分量和b分量的颜色值分别记为L(x,y)、a(x,y)和b(x,y),将Ri中坐标位置为(x,y)的像素点在HVS颜色空间的H分量、V分量和S分量的颜色值分别记为H(x,y)、V(x,y)和S(x,y),其中,此处(x,y)表示{Li,Ri,di|1≤i≤N}中的立体图像中的像素点的坐标位置,1≤x≤W,1≤y≤H,W和H对应表示{Li,Ri,di|1≤i≤N}中的立体图像的宽度和高度。2)-1. Calculate the color value of each pixel point in R i in different color spaces, and use the R component, G component and B component of the pixel point whose coordinate position is (x, y) in R i in the RGB color space The color values of are recorded as R( x , y), G(x, y) and B(x, y) respectively, and the L component, The color values of a component and b component are recorded as L(x, y), a(x, y) and b(x, y) respectively, and the pixel point whose coordinate position in R i is (x, y) is in HVS color The color values of the H component, V component and S component of the space are respectively recorded as H(x, y), V(x, y) and S(x, y), where (x, y) here represents {L i , R i , d i |1≤i≤N} in the coordinate position of the pixel in the stereo image, 1≤x≤W, 1≤y≤H, W and H correspond to {L i ,R i ,d Width and height of stereo images in i |1≤i≤N}.
2)-2、将Ri中的每个像素点在不同颜色空间的颜色值按顺序进行排列,构成Ri中的每个像素点的颜色特征矢量,将Ri中坐标位置为(x,y)的像素点的颜色特征矢量记为ci(x,y),ci(x,y)=[R(x,y),G(x,y),B(x,y),L(x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)],其中,ci(x,y)的维数为9,此处符号“[]”为矢量表示符号。2)-2. Arrange the color values of each pixel in R i in different color spaces in order to form the color feature vector of each pixel in R i , and set the coordinate position in R i as (x, y) is recorded as c i (x, y), c i (x, y) = [R(x, y), G(x, y), B(x, y), L (x,y),a(x,y),b(x,y),H(x,y),V(x,y),S(x,y)], where c i (x,y ) has a dimension of 9, where the symbol “[]” is a vector representation symbol.
以下就利用本发明方法对法国南特大学提供的三维人眼跟踪数据库(3Deye-tracking database)中的Image1、Image2、Image3、Image4和Image5五幅立体图像的三维显著图进行提取。图2a给出了“Image1”的右视点图像、图2b给出了“Image1”的右视点图像的真实眼动图、图2c给出了“Image1”的三维显著图;图3a给出了“Image2”的右视点图像、图3b给出了“Image2”的右视点图像的真实眼动图、图3c给出了“Image2”的三维显著图;图4a给出了“Image3”的右视点图像、图4b给出了“Image3”的右视点图像的真实眼动图、图4c给出了“Image3”的三维显著图;图5a给出了“Image4”的右视点图像、图5b给出了“Image4”的右视点图像的真实眼动图、图5c给出了“Image4”的三维显著图;图6a给出了“Image5”的右视点图像、图6b给出了“Image5”的右视点图像的真实眼动图、图6c给出了“Image5”的三维显著图;图7a给出了“Image6”的右视点图像、图7b给出了“Image6”的右视点图像的真实眼动图、图7c给出了“Image6”的三维显著图;图8a给出了“Image7”的右视点图像、图8b给出了“Image7”的右视点图像的真实眼动图、图8c给出了“Image7”的三维显著图。从图2a至图8c中可以看出,采用本发明方法得到的三维显著图由于考虑了对比度特征、通用特征和背景先验特征,因此能够很好地符合显著语义的特征。Below, the three-dimensional saliency maps of Image1, Image2, Image3, Image4 and Image5 five stereoscopic images in the three-dimensional human eye tracking database (3Deye-tracking database) provided by the University of Nantes in France are extracted by using the method of the present invention. Figure 2a shows the right view image of "Image1", Figure 2b shows the real eye movement image of the right view image of "Image1", Figure 2c shows the 3D saliency map of "Image1"; Figure 3a shows the " The right view image of Image2", Figure 3b shows the real eye movement image of the right view image of "Image2", Figure 3c shows the 3D saliency map of "Image2"; Figure 4a shows the right view image of "Image3" , Figure 4b shows the real eye movement image of the right-viewpoint image of "Image3", and Figure 4c shows the 3D saliency map of "Image3"; Figure 5a shows the right-viewpoint image of "Image4", and Figure 5b shows the The real eye movement image of the right view point image of "Image4", Fig. 5c shows the 3D saliency map of "Image4"; Fig. 6a shows the right view point image of "Image5", and Fig. 6b shows the right view point of "Image5" The real eye movement diagram of the image, Figure 6c shows the 3D saliency map of "Image5"; Figure 7a shows the right viewpoint image of "Image6", and Fig. 7b shows the real eye movement diagram of the right viewpoint image of "Image6" , Figure 7c shows the 3D saliency map of "Image6"; Figure 8a shows the right view image of "Image7", Figure 8b shows the real eye movement image of the right view image of "Image7", and Figure 8c shows The 3D saliency map of "Image7". It can be seen from Figures 2a to 8c that the 3D saliency map obtained by the method of the present invention can well conform to the features of saliency semantics because the contrast feature, general feature and background prior feature are considered.
Claims (6)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510173098.5A CN104822058B (en) | 2015-04-14 | 2015-04-14 | A kind of stereo-picture notable figure extracting method |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201510173098.5A CN104822058B (en) | 2015-04-14 | 2015-04-14 | A kind of stereo-picture notable figure extracting method |
Publications (2)
Publication Number | Publication Date |
---|---|
CN104822058A CN104822058A (en) | 2015-08-05 |
CN104822058B true CN104822058B (en) | 2016-08-17 |
Family
ID=53732209
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201510173098.5A Active CN104822058B (en) | 2015-04-14 | 2015-04-14 | A kind of stereo-picture notable figure extracting method |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN104822058B (en) |
Families Citing this family (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105338335B (en) * | 2015-10-15 | 2017-03-29 | 宁波大学 | A kind of stereo-picture notable figure extracting method |
EP3403403B1 (en) * | 2016-01-12 | 2023-06-07 | Shanghaitech University | Calibration method and apparatus for panoramic stereo video system |
CN109167988B (en) * | 2018-08-29 | 2020-06-26 | 长春理工大学 | Stereo image visual comfort evaluation method based on D + W model and contrast |
CN109523534A (en) * | 2018-11-14 | 2019-03-26 | 北京奇艺世纪科技有限公司 | A kind of salient region detection method and device of view-based access control model angle |
-
2015
- 2015-04-14 CN CN201510173098.5A patent/CN104822058B/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN104822058A (en) | 2015-08-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11830230B2 (en) | Living body detection method based on facial recognition, and electronic device and storage medium | |
Li et al. | A weighted sparse coding framework for saliency detection | |
EP3333768A1 (en) | Method and apparatus for detecting target | |
CN106504276A (en) | The combinations matches cost algorithms of non local Stereo Matching Algorithm and parallax joint filling algorithm | |
CN103020265B (en) | The method and system of image retrieval | |
CN104822058B (en) | A kind of stereo-picture notable figure extracting method | |
CN105389554A (en) | Live body discrimination method and device based on face recognition | |
CN101833767B (en) | Automatic linear matching method based on gradient and color features in digital image | |
CN105869173A (en) | Stereoscopic vision saliency detection method | |
CN104574375A (en) | Image Saliency Detection Method Combining Color and Depth Information | |
CN111160478B (en) | Hyperspectral target significance detection method based on deep learning | |
CN104392233B (en) | A kind of image saliency map extracting method based on region | |
CN106780476A (en) | A kind of stereo-picture conspicuousness detection method based on human-eye stereoscopic vision characteristic | |
CN102129694B (en) | A method of image salient region detection | |
CN106651853A (en) | Establishment method for 3D saliency model based on prior knowledge and depth weight | |
CN104408711A (en) | Multi-scale region fusion-based salient region detection method | |
CN107886507A (en) | A kind of salient region detecting method based on image background and locus | |
CN110807379A (en) | A semantic recognition method, device, and computer storage medium | |
WO2023061195A1 (en) | Image acquisition model training method and apparatus, image detection method and apparatus, and device | |
CN106997478A (en) | RGB-D image salient target detection method based on salient center prior | |
CN104243956B (en) | A kind of stereo-picture visual saliency map extracting method | |
CN104331877A (en) | Color image edge detection method based on fusion color gradient | |
CN111325107A (en) | Detection model training method and device, electronic equipment and readable storage medium | |
CN103632372B (en) | A kind of video saliency image extraction method | |
CN101866422A (en) | A method of extracting image attention based on image multi-feature fusion |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
EXSB | Decision made by sipo to initiate substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20191219 Address after: 313000 Huzhou Youyan Intellectual Property Service Co., Ltd Patentee after: Huzhou You Yan Intellectual Property Service Co., Ltd. Address before: 315211 Zhejiang Province, Ningbo Jiangbei District Fenghua Road No. 818 Patentee before: Ningbo University |
|
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20200117 Address after: 215000 room 16, floor 4, building 21, Tengfei Innovation Park, No. 388, Xinping street, Suzhou Industrial Park, Jiangsu Province Patentee after: ZHONGHE INTERNATIONAL INTELLECTUAL PROPERTY CO., LTD. Address before: 313000 Huzhou Youyan Intellectual Property Service Co., Ltd Patentee before: Huzhou You Yan Intellectual Property Service Co., Ltd. |