WO2019000821A1 - 基于深度图挖掘的后向传播图像视觉显著性检测方法 - Google Patents

基于深度图挖掘的后向传播图像视觉显著性检测方法 Download PDF

Info

Publication number
WO2019000821A1
WO2019000821A1 PCT/CN2017/112788 CN2017112788W WO2019000821A1 WO 2019000821 A1 WO2019000821 A1 WO 2019000821A1 CN 2017112788 W CN2017112788 W CN 2017112788W WO 2019000821 A1 WO2019000821 A1 WO 2019000821A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
depth
region
processing stage
saliency
Prior art date
Application number
PCT/CN2017/112788
Other languages
English (en)
French (fr)
Inventor
李革
朱春彪
王文敏
王荣刚
黄铁军
高文
Original Assignee
北京大学深圳研究生院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京大学深圳研究生院 filed Critical 北京大学深圳研究生院
Priority to US16/336,737 priority Critical patent/US11227178B2/en
Publication of WO2019000821A1 publication Critical patent/WO2019000821A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/56—Extraction of image or video features relating to colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/763—Non-hierarchical techniques, e.g. based on statistics of modelling distributions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing

Definitions

  • the present invention relates to the field of image processing technologies, and in particular, to a multi-stage backward propagation visual saliency detection algorithm for deep mining of depth maps.
  • the human eye's attention is quickly concentrated on a few prominent visual objects, and these objects are prioritized, a process known as visual saliency.
  • Significant detection is the use of this visual biological mechanism of the human eye, using mathematical calculations to simulate the human eye to properly process the image to obtain a significant object of the picture. Since we can prioritize the computational resources required for image analysis and synthesis through the saliency region, it is significant to detect the saliency region of the image by calculation.
  • the extracted saliency images can be widely used in many computer vision applications, including image segmentation of target objects of interest, detection and recognition of target objects, image compression and encoding, image retrieval, and content-aware image editing.
  • the existing saliency detection frameworks are mainly divided into a bottom-up saliency detection method and a top-down saliency detection method.
  • most of the bottom-up saliency detection methods are based on data-driven and independent of specific tasks; and the top-down saliency detection method is subject to consciousness and is related to specific tasks.
  • the bottom-up saliency detection method mostly uses low-level feature information such as color features, distance features, and some heuristic saliency features. Although these methods have their own advantages, they are not accurate enough to be robust enough on challenging data sets in specific scenarios.
  • existing methods have adopted depth information to enhance the accuracy of significant object detection. Although depth information can increase the accuracy of saliency object detection, the accuracy of saliency detection is affected when a significant object has a low contrast with its background.
  • the existing image saliency object detection method is not accurate when detecting significant objects, the method is not robust enough, and it is easy to cause false detection, missed detection, etc. It is difficult to obtain an accurate image saliency detection. As a result, not only the misdetection of the significant object itself is caused, but also a certain error is caused to the application using the significance detection result.
  • the object of the present invention is to solve the shortcomings of the prior art mentioned above, and propose a backward propagation saliency detection algorithm for deep mining of depth maps, which can solve the problem that the existing saliency detection is not accurate enough and is not robust enough.
  • the saliency area is more accurately revealed, providing accurate and useful information for later applications such as target recognition and classification.
  • a backward propagation saliency detection method based on depth map mining which acquires the depth map of the image and the image removed by the four-corner background in the preprocessing stage, and uses the color, depth, and distance information to perform the saliency region of the image in the first processing stage.
  • Position detection detects the preliminary detection results of the saliency objects in the image, and then depth digs the depth map from multiple levels (processing stage) to obtain the corresponding saliency detection results, and then uses the backward propagation mechanism to mine each layer.
  • the results are optimized to obtain a final saliency test result map, the implementation of which includes the following steps:
  • Preprocessing stage For an input image I o , the depth image is first obtained by the Kinect device, defined as I d ; secondly, the quadrilateral background edge of the image is removed by the BSCA algorithm, and the obtained de-square background image is defined as C b .
  • the BSCA algorithm is described in the literature (Qin Y, Lu H, Xu Y, et al. Saliency detection via Cellular Automata [C]//IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 110-119.) That is, based on the background seed information, the background saliency map is obtained according to the comparison of the color and the distance information.
  • the first processing stage using the obtained de-four-corner background image C b and the depth image I d , preliminary saliency detection is performed on the input image I o , and a preliminary saliency monitoring result is obtained, which is defined as: S 1 ;
  • steps 11 - 15 it includes steps 11 - 15:
  • Step 11 The K-means algorithm is used to divide the image into K regions, and each sub-region is calculated by the formula (1).
  • D o (r k , r i ) represents the coordinate position distance of the region k and the region i
  • ⁇ is a range in which a parameter controls W s (r k ).
  • Step 12 Calculate the depth significance value S d (r k ) of the depth map by the equation (3) in the same manner as the color saliency value calculation method:
  • D d (r k , r i ) is the Euclidean distance of the region k and the region i in the depth space.
  • Step 13 the saliency object is located at the center position, and the center of the region k and the depth weight S s (r k ) are calculated by the formula (4):
  • G( ⁇ ) represents Gaussian normalization
  • represents Euclidean distance operation
  • P k is the position coordinate of region k
  • P o is the coordinate center of the image
  • N k is the number of pixels of region k.
  • W d (d k ) is the depth weight and is defined as follows:
  • max ⁇ d ⁇ represents the maximum depth of the depth map
  • d k represents the depth value of the region k
  • ⁇ is a parameter related to the calculated depth map, defined as follows:
  • min ⁇ d ⁇ represents the minimum depth of the depth map.
  • Step 14 Using the formula (7) to obtain a rough saliency detection result S fc (r k ), which is a preliminary saliency detection result that is not optimized in the first processing stage;
  • Step 15 In order to optimize the preliminary significance test results, the depth map I d (d k ) and the de-square back of the pre-processing stage are utilized.
  • Equation 8 The image C b of the scene enhances the result of equation (7), as in Equation 8:
  • S 1 (r k ) refers to the optimization result of S fc (r k ) of the formula 7, that is, the detection result after the optimization in the first processing stage;
  • the second processing stage transforming the depth map into a color map, and using the optimization process of the first processing stage and the optimization of the backward propagation mechanism, the intermediate significance detection result is obtained, which is defined as S 2 .
  • the third processing stage background filtering of the depth map, and then converting the filtered depth map into a color map, and using the second processing stage calculation process and the backward propagation mechanism optimization to obtain the final significant detection result S.
  • the invention provides a multi-layer backward propagation saliency detection algorithm based on depth map mining. Firstly, the depth map and the de-square background image of the image are acquired in the pre-processing layer, and the saliency detection algorithm of the first layer is used based on the image color. , space, depth and other information to calculate the preliminary significant results, then through the second and third layers of the depth map for deep mining and the first layer of calculation for significant detection, and finally for the second and third layers of significance The detection result is excellent in applying the backward propagation mechanism The second significant test result graph and the final significance test result graph are obtained.
  • the invention can detect significant objects more accurately and more robustly. Compared with the prior art, the present invention has the following technical advantages:
  • the present invention can improve the accuracy of detection of significant objects due to multi-layer excavation of depth maps.
  • the present invention provides a backward propagation mechanism to optimize the saliency detection results of the layers.
  • FIG. 1 is a flow chart of the present invention.
  • FIG. 2 is a comparison diagram of a detection result image obtained by using an existing method, detecting an image by using the method of the present invention, and manually calibrating an expected image according to an embodiment of the present invention
  • the first column is an input image
  • the second column is an artificially calibrated desired image
  • the third column is the detection result image of the present invention
  • the fourth to tenth columns are detection result images obtained by other existing methods.
  • the invention provides a multi-layer backward propagation saliency detection algorithm based on depth map mining, which can detect significant objects more accurately and more robustly.
  • the invention first acquires the depth map of the image and the image removed by the four-corner background in the pre-processing layer/stage, and secondly mines the depth map in the first layer, the second layer and the third layer respectively, and obtains corresponding significant detection results. Finally, the backward propagation mechanism is used to optimize the saliency detection results of each layer/processing stage, and the final saliency detection result graph is obtained.
  • FIG. 1 is a flow chart of a method for detecting a significant object provided by the present invention, including the following steps:
  • Step 1 Input an image I o to be detected, and obtain an image C b from which the four-corner background is removed and a depth map I d of the image; wherein, using the literature (Qin Y, Lu H, Xu Y, et al. Saliency detection via Cellular Automata [C] // IEEE Conference on Computer Vision and Pattern Recognition.IEEE, 2015:. 110-119) BSCA algorithm described in FIG removed to obtain a depth image C b corners background, obtained using the apparatus of Kinect image I d ;
  • Step 2 The K-means algorithm is used to divide the image into K regions, and the sub-regions are calculated by the formula (1).
  • D o (r k , r i ) represents the coordinate position distance of the region k and the region i
  • ⁇ is a range in which a parameter controls W s (r k ).
  • Step 3 The same as the color saliency value calculation method, the depth saliency value S d (r k ) of the depth map is calculated by the formula (3):
  • D d (r k , r i ) is the Euclidean distance of the region k and the region i in the depth space.
  • Step 4 Generally speaking, the saliency object is located at the center position, and the center of the region k and the depth weight S s (r k ) are calculated by the formula (4):
  • G( ⁇ ) represents Gaussian normalization
  • represents Euclidean distance operation
  • P k is the position coordinate of region k
  • P o is the coordinate center of the image
  • N k is the number of pixels of region k.
  • W d (d k ) is the depth weight and is defined as follows:
  • max ⁇ d ⁇ represents the maximum depth of the depth map
  • d k represents the depth value of the region k
  • ⁇ is a parameter related to the calculated depth map, defined as follows:
  • min ⁇ d ⁇ represents the minimum depth of the depth map.
  • Step 5 Using the formula (7) to obtain a rough saliency detection result S fc (r k ), which is a preliminary saliency detection result that is not optimized in the first processing stage;
  • Step 6 the result of the formula (7) is enhanced by using the depth map I d (d k ) of the pre-processing stage and the image C b of the de-corner background, as in Equation 8:
  • S 1 (r k ) refers to the optimization result of S fc (r k ) of the formula 7, that is, the detection result after the optimization in the first processing stage;
  • Step 7 Further digging the depth map, first extend the depth map through the formula (9) into a depth-based color map:
  • I e is a depth-based color map after expansion.
  • Step 8 The extended depth-based color map is processed through steps 2 to 5 of the first layer to obtain a second layer of the roughness significance detection result S sc (r k ):
  • Step IX In order to optimize the results of the coarse saliency detection, the back-propagation mechanism is used to optimize the roughness detection result of the second layer by using the preliminary detection result of the first layer (the result calculated by the equation (7)). ), get our intermediate significance test result S 2 (r k ):
  • Step 10 Further digging the depth map, firstly using the formula (12) to perform background filtering on the depth map to obtain the filtered depth map I df :
  • I df is the depth map after background filtering.
  • Step 11 Extend the filtered depth map into a color map by the operation of the formula (9) of the seventh step of the second layer, and define it as I ef .
  • Step 12 The color map I ef of the filtered depth map is processed through the second step to the fifth step of the first layer to obtain a rough saliency detection result S tc (r k ) of the third layer:
  • Step 13 In order to optimize the results of the rough saliency detection, the back-propagation mechanism is used to optimize the rough detection results of the third layer by using the preliminary detection results of the first layer and the second layer, and the method is obtained by the formula (14).
  • the second column is an image obtained by manually calibrating a desired image.
  • the third column is the detection result image of the present invention
  • the fourth column to the tenth column are the detection result images obtained by other existing methods.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Computing Systems (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)

Abstract

一种基于深度图挖掘的后向传播显著性检测方法，对于一个输入图像I o，在预处理阶段，获取图像I o的深度图像I d和四角背景去除的图像C b；在第一处理阶段，利用得到的去四角背景图像C b和深度图像I d，对图像的显著性区域进行定位检测，得到图像中显著性物体的初步检测结果S 1；然后对深度图像I d进行多个处理阶段的深度挖掘，得到相应的显著性检测结果；再利用后向传播机制，对每个处理阶段挖掘的显著性检测结果进行优化，得到最终的显著性检测结果图。该方法能够提高显著性物体检测的精准性。

Description

基于深度图挖掘的后向传播图像视觉显著性检测方法 技术领域
本发明涉及图像处理技术领域，尤其涉及一种对深度图进行深度挖掘的多阶段后向传播视觉显著性检测算法。
背景技术
在面对一个复杂场景时，人眼的注意力会迅速集中在少数几个显著的视觉对象上，并对这些对象进行优先处理，该过程被称为视觉显著性。显著性检测正是利用人眼的这种视觉生物学机制，用数学的计算方法模拟人眼对图像进行适当的处理，从而获得一张图片的显著性物体。由于我们可以通过显著性区域来优先分配图像分析与合成所需要的计算资源，所以，通过计算来检测图像的显著性区域意义重大。提取出的显著性图像可以广泛应用于许多计算机视觉领域的应用，包括对兴趣目标物体的图像分割，目标物体的检测与识别，图像压缩与编码，图像检索，内容感知图像编辑等方面。
通常来说，现有的显著性检测框架主要分为：自底向上的显著性检测方法和自顶向下的显著性检测方法。目前大多采用自底向上的显著性检测方法，它是基于数据驱动的，且独立于具体的任务；而自顶向下的显著性检测方法是受意识支配的，与具体任务相关。
现有方法中，自底向上的显著性检测方法大多使用低水平的特征信息，例如颜色特征、距离特征和一些启发式的显著性特征等。尽管这些方法有各自的优点，但是在一些特定场景下的具有挑战性的数据集上，这些方法表现的不够精确，不够健壮。为了解决这一问题，随着3D图像采集技术的出现，目前已有方法通过采用深度信息来增强显著性物体检测的精准度。尽管深度信息可以增加显著性物体检测的精准度，但是，当一个显著性物体与其背景有着低对比的深度时，还是会影响显著性检测的精准度。
综合来看，现有的图像显著性物体检测方法在检测显著性物体时精准度不高，方法健壮性不够强，容易造成误检、漏检等情况，很难得到一个精确的图像显著性检测结果，不仅造成显著性物体本身的错检，同时也会对利用显著性检测结果的应用造成一定的误差。
发明内容
本发明的目的在于针对上述已有技术的不足，提出了一种对深度图进行深度挖掘的后向传播显著性检测算法，能够解决现有的显著性检测不够精确，不够健壮性的问题，使图像中 的显著性区域更精准地显现出来，为后期的目标识别和分类等应用提供精准且有用的信息。
本发明提供的技术方案是：
一种基于深度图挖掘的后向传播显著性检测方法，在预处理阶段获取图像的深度图以及四角背景去除的图像，在第一处理阶段利用颜色、深度、距离信息对图像的显著性区域进行定位检测，得到图像中显著性物体的初步检测结果，再对深度图从多个层面(处理阶段)进行深度的挖掘，得到相应的显著性检测结果，再利用后向传播机制，对每层挖掘的结果进行优化，得到最终的显著性检测结果图，其实现包括如下步骤：
1)预处理阶段：对于一个输入图像Io，首先利用Kinect设备获取深度图像，定义为Id；其次利用BSCA算法去掉图像的四角背景边缘，将得到的去四角背景图像定义为Cb。其中，BSCA算法在文献(Qin Y,Lu H,Xu Y,et al.Saliency detection via Cellular Automata[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:110-119.)中有记载，也就是基于背景种子信息，根据颜色和距离信息对比得到背景显著性图。
2)第一处理阶段：利用得到的去四角背景图像Cb和深度图像Id，对输入图像Io进行初步的显著性检测，得到初步的显著性监测结果，定义为：S1；
具体包括步骤11-步骤15：
步骤11、利用K-means算法将图像分成K个区域，并通过式(1)计算得到每个子区域
的颜色显著值Sc(rk)：
Figure PCTCN2017112788-appb-000001
其中，rk和ri分别代表区域k和i，Dc(rk,ri)表示区域k和区域i在L*a*b颜色空间上的欧氏距离，Pi代表区域i所占图像区域的比例，Ws(rk)定义如下：
Figure PCTCN2017112788-appb-000002
其中，Do(rk,ri)表示区域k和区域i的坐标位置距离，σ是一个参数控制着Ws(rk)的范围。
步骤12、同颜色显著性值计算方式一样，通过式(3)计算深度图的深度显著性值Sd(rk)：
Figure PCTCN2017112788-appb-000003
其中，Dd(rk,ri)是区域k和区域i在深度空间的欧氏距离。
步骤13、通常来说，显著性物体都位于中心位置，通过式(4)计算区域k的中心和深度权重Ss(rk)：
Figure PCTCN2017112788-appb-000004
其中，G(·)表示高斯归一化，||·||表示欧氏距离操作，Pk是区域k的位置坐标，Po是该图像的坐标中心，Nk是区域k的像素数量。Wd(dk)是深度权重，定义如下：
wd(dk)＝(max{d}-dk)μ       (5)
其中，max{d}表示深度图的最大深度，dk表示区域k的深度值，μ是一个与计算的深度图有关的参数，定义如下：
Figure PCTCN2017112788-appb-000005
其中，min{d}表示深度图的最小深度。
步骤14、利用式(7)得到粗糙的显著性检测结果Sfc(rk)，为第一处理阶段未优化的初步的显著性检测结果；
Sfc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)SS(rk))       (7)
步骤15、为了优化初步的显著性检测结果，利用预处理阶段的深度图Id(dk)和去四角背
景的图像Cb对式(7)的结果进行加强，如式8：
Figure PCTCN2017112788-appb-000006
S1(rk)指的是式7的Sfc(rk)的优化结果，即第一处理阶段优化后的检测结果；
3)第二处理阶段：将深度图转化为彩色图，利用第一处理阶段计算过程和后向传播机制的优化，得到中级显著性检测结果，定义为S2。
3)第三处理阶段：对深度图进行背景过滤，再将过滤后的深度图转化为彩色图，利用第二处理阶段计算过程和后向传播机制的优化，得到最后的显著性检测结果S。
与现有技术相比，本发明的有益效果是：
本发明提供了一种基于深度图挖掘的多层后向传播显著性检测算法，首先在预处理层获取图像的深度图和去四角背景图，再利用第一层的显著性检测算法基于图像颜色、空间、深度等信息计算出初步的显著性结果，然后通过第二层和第三层对深度图进行深度挖掘并用第一层的计算方式进行显著性检测，最后对二、三层的显著性检测结果应用后向传播机制的优 化，得到二次显著性检测结果图和最终显著性检测结果图。
本发明能够更加精准，更加鲁棒地检测出显著性物体。与现有技术相比，本发明具有以下技术优势：
(一)由于对深度图进行多层的挖掘，本发明能够提高显著性物体检测的精准性。
(二)本发明提供了一种后向传播机制来优化各层的显著性检测结果。
附图说明
图1为本发明提供的流程框图。
图2为本发明实施例中对输入图像分别采用现有方法、采用本发明方法检测图像得到的检测结果图像，以及人工标定期望得到图像的对比图；
其中，第一列为输入图像，第二列为人工标定期望得到的图像，第三列为本发明的检测结果图像，第四列至第十列为现有其他方法得到的检测结果图像。
具体实施方式
下面结合图例，通过实施例进一步描述本发明，但不以任何方式限制本发明的范围。
本发明提供了一种基于深度图挖掘的多层后向传播显著性检测算法，能够更加精准，更加鲁棒地检测出显著性物体。本发明首先在预处理层/阶段获取图像的深度图以及四角背景去除的图像，其次在第一层、第二层以及第三层，分别对深度图进行挖掘，得到相应的显著性检测结果，最后利用后向传播机制去优化各层/处理阶段的显著性检测结果，得到最后的显著性检测结果图。图1为本发明提供的显著性物体检测方法的流程框图，包括以下步骤：
步骤一、输入一张待检测的图像Io，得到去除四角背景的图像Cb和该图像的深度图Id；其中，利用文献(Qin Y,Lu H,Xu Y,et al.Saliency detection via Cellular Automata[C]//IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2015:110-119.)记载的BSCA算法得到去除四角背景的图像Cb，利用Kinect设备得到的该图像的深度图Id；
算法第一层操作：(步骤二-步骤六)
步骤二、利用K-means算法将图像分成K个区域，并通过式(1)计算得到每个子区域的
颜色显著值Sc(rk)：
Figure PCTCN2017112788-appb-000007
其中，rk和ri分别代表区域k和i，Dc(rk,ri)表示区域k和区域i在L*a*b颜色空间上的 欧氏距离，Pi代表区域i所占图像区域的比例，Ws(rk)定义如下：
Figure PCTCN2017112788-appb-000008
其中，Do(rk,ri)表示区域k和区域i的坐标位置距离，σ是一个参数控制着Ws(rk)的范围。
步骤三、同颜色显著性值计算方式一样，通过式(3)计算深度图的深度显著性值Sd(rk)：
Figure PCTCN2017112788-appb-000009
其中，Dd(rk,ri)是区域k和区域i在深度空间的欧氏距离。
步骤四、通常来说，显著性物体都位于中心位置，通过式(4)计算区域k的中心和深度权重Ss(rk)：
Figure PCTCN2017112788-appb-000010
其中，G(·)表示高斯归一化，||·||表示欧氏距离操作，Pk是区域k的位置坐标，Po是该图像的坐标中心，Nk是区域k的像素数量。Wd(dk)是深度权重，定义如下：
Wd(dk)＝(max{d}-dk)μ       (5)
其中，max{d}表示深度图的最大深度，dk表示区域k的深度值，μ是一个与计算的深度图有关的参数，定义如下：
Figure PCTCN2017112788-appb-000011
其中，min{d}表示深度图的最小深度。
步骤五、利用式(7)得到粗糙的显著性检测结果Sfc(rk)，为第一处理阶段未优化的初步的显著性检测结果；
Sfc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)Ss(rk))        (7)
步骤六、为了优化初步的显著性检测结果，利用预处理阶段的深度图Id(dk)和去四角背景的图像Cb对式(7)的结果进行加强，如式8：
Figure PCTCN2017112788-appb-000012
S1(rk)指的是式7的Sfc(rk)的优化结果，即第一处理阶段优化后的检测结果；
算法第二层操作：(步骤七-步骤九)
步骤七、进一步挖掘深度图，首先把深度图通过式(9)扩展成基于深度的彩色图：
Ie<R|G|B>＝Io<R|G|B>×Id     (9)
其中，Ie是扩展之后的基于深度的彩色图。
步骤八、将扩展后的基于深度的彩色图通过第一层的步骤二至步骤五操作，得到第二层的粗糙显著性检测结果Ssc(rk)：
Ssc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)Ss(rk)),     (10)
步骤九、为了优化粗糙显著性检测结果，利用后向传播机制，用第一层的初步检测结果(式(7)计算得到的结果)对第二层的粗糙检测结果进行优化，通过式(11)实现，得到我们的中级显著性检测结果S2(rk)：
Figure PCTCN2017112788-appb-000013
算法第三层操作：(步骤十-步骤十三)
步骤十、再进一步挖掘深度图，首先利用公式(12)将深度图进行背景过滤处理，得到过滤后的深度图Idf：
Figure PCTCN2017112788-appb-000014
其中，Idf是背景过滤后的深度图。
步骤十一、将过滤后的深度图通过第二层第七步的公式(9)的操作扩展成彩色图，定义为Ief。
步骤十二、将过滤深度图的彩色图Ief通过第一层的步骤二至步骤五操作，得到第三层的粗糙显著性检测结果Stc(rk)：
Stc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)Ss(rk))         (13)
步骤十三、为了优化粗糙显著性检测结果，利用后向传播机制，用第一层和第二层的初步检测结果对第三层的粗糙检测结果进行优化，通过式(14)实现，得到我们最后的显著性检测结果S(rk)：
Figure PCTCN2017112788-appb-000015
图2为对输入图像分别采用现有方法、本发明方法检测图像得到的检测结果图像，以及人工标定期望得到的图像，其中，第一列为输入图像，第二列为人工标定期望得到的图像， 第三列为本发明的检测结果图像，第四列至第十列为采用现有其他方法得到的检测结果图像。通过图2的图像对比，可以看出，相对其他方法，我们的方法可以检测出的显著性物体，误差率最低，精准度最高，具有很好的鲁棒性。
需要注意的是，公布实施例的目的在于帮助进一步理解本发明，但是本领域的技术人员可以理解：在不脱离本发明及所附权利要求的精神和范围内，各种替换和修改都是可能的。因此，本发明不应局限于实施例所公开的内容，本发明要求保护的范围以权利要求书界定的范围为准。

Claims (4)

  1. 一种基于深度图挖掘的后向传播显著性检测方法，对于一个输入图像Io，
    在预处理阶段，获取图像IO的深度图像Id和四角背景去除的图像Cb；
    在第一处理阶段，利用得到的去四角背景图像Cb和深度图像Id，对图像的显著性区域进行定位检测，得到图像中显著性物体的初步检测结果S1；
    然后对深度图像Id进行多个处理阶段的深度挖掘，得到相应的显著性检测结果；
    再利用后向传播机制，对每个处理阶段挖掘的显著性检测结果进行优化，得到最终的显著性检测结果图。
  2. 如权利要求1所述基于深度图挖掘的后向传播显著性检测方法，其特征是，具体利用Kinect设备获取深度图像Id；利用BSCA算法去掉图像的四角背景边缘，得到去四角背景图像Cb。
  3. 如权利要求1所述基于深度图挖掘的后向传播显著性检测方法，其特征是，第一处理阶段的处理具体包括步骤11-步骤15：
    步骤11、利用K-means算法将图像分成K个区域，并通过式(1)计算得到每个子区域的颜色显著值Sc(rk)：
    Figure PCTCN2017112788-appb-100001
    其中，rk和ri分别代表区域k和i，Dc(rk,ri)表示区域k和区域i在L*a*b颜色空间上的欧氏距离，Pi代表区域i所占图像区域的比例，Ws(rk)通过式(2)得到：
    Figure PCTCN2017112788-appb-100002
    其中，Do(rk,ri)表示区域k和区域i的坐标位置距离，σ是控制着Ws(rk)范围的参数；
    步骤12、通过式(3)计算深度图的深度显著性值Sd(rk)：
    Figure PCTCN2017112788-appb-100003
    其中，Dd(rk,ri)是区域k和区域i在深度空间的欧氏距离；
    步骤13、通过式(4)计算区域k的中心和深度权重Ss(rk)：
    Figure PCTCN2017112788-appb-100004
    其中，G(·)表示高斯归一化，||·||表示欧氏距离操作，Pk是区域k的位置坐标，Po是该 图像的坐标中心，Nk是区域k的像素数量；Wd(dk)是深度权重，通过式(5)计算得到：
    Wd(dk)＝(max{d}-dk)μ         (5)
    其中，max{d}表示深度图的最大深度，dk表示区域k的深度值；μ是与计算的深度图有关的参数，通过式(6)计算得到：
    Figure PCTCN2017112788-appb-100005
    其中，min{d}表示深度图的最小深度；
    步骤14、利用式(7)得到粗糙的显著性检测结果Sfc(rk)：
    Sfc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)Ss(rk))           (7)
    显著性检测结果Sfc(rk)即为第一处理阶段得到的初步的显著性检测结果；
    步骤15、为了优化初步的显著性检测结果，利用预处理阶段的深度图Id(dk)和去四角背景的图像Cb对式(7)的结果进行加强优化，如式8：
    Figure PCTCN2017112788-appb-100006
    即得到第一处理阶段优化后的检测结果S1(rk)。
  4. 如权利要求3所述基于深度图挖掘的后向传播显著性检测方法，其特征是，所述对深度图像Id进行多个处理阶段的深度挖掘，包括第二处理阶段和第三处理阶段的深度挖掘；
    所述第二处理阶段的处理具体包括步骤21-步骤23：
    步骤21、将深度图像Id通过式(9)扩展成基于深度的彩色图Ie：
    Ie〈R|G|B〉＝Io〈R|G|B〉×Id      (9)
    其中，Ie是扩展之后的基于深度的彩色图；
    步骤22、将扩展后的基于深度的彩色图Ie通过所述第一处理阶段对图像的显著性区域进行定位检测的方法，得到第二处理阶段的粗糙显著性检测结果Ssc(rk)，表示为式(10)：
    Ssc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)Ss(rk)),         (10)
    步骤23、利用后向传播机制，用第一处理阶段的初步检测结果对步骤22所述第二处理阶段的粗糙显著性检测结果进行优化，通过式(11)得到中级显著性检测结果S2(rk)：
    Figure PCTCN2017112788-appb-100007
    所述第三处理阶段的处理具体包括步骤31-步骤34：
    步骤31、再进一步挖掘深度图像，利用式(12)将深度图像进行背景过滤处理，得到过滤后的深度图Idf：
    Figure PCTCN2017112788-appb-100008
    其中，Idf是背景过滤后的深度图；
    步骤32、将过滤后的深度图通过式(9)的操作扩展成彩色图，定义为Ief；
    步骤33、将过滤深度图的彩色图Ief通过所述第一处理阶段对图像的显著性区域进行定位检测的方法操作，得到第三处理阶段的粗糙显著性检测结果Stc(rk)，表示为式(13)：
    Stc(rk)＝G(Sc(rk)Ss(rk)+Sd(rk)Ss(rk))       (13)
    步骤34、利用后向传播机制，用第一处理阶段和第二处理阶段的初步检测结果对第三处理阶段的粗糙检测结果进行优化，通过式(14)得到显著性检测结果S(rk)：
    Figure PCTCN2017112788-appb-100009
    由此得到最终的显著性检测结果图。
PCT/CN2017/112788 2017-06-29 2017-11-24 基于深度图挖掘的后向传播图像视觉显著性检测方法 WO2019000821A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/336,737 US11227178B2 (en) 2017-06-29 2017-11-24 Back-propagation image visual saliency detection method based on depth image mining

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201710513077.2 2017-06-29
CN201710513077.2A CN107292923B (zh) 2017-06-29 2017-06-29 基于深度图挖掘的后向传播图像视觉显著性检测方法

Publications (1)

Publication Number Publication Date
WO2019000821A1 true WO2019000821A1 (zh) 2019-01-03

Family

ID=60099950

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/112788 WO2019000821A1 (zh) 2017-06-29 2017-11-24 基于深度图挖掘的后向传播图像视觉显著性检测方法

Country Status (3)

Country Link
US (1) US11227178B2 (zh)
CN (1) CN107292923B (zh)
WO (1) WO2019000821A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115330774A (zh) * 2022-10-12 2022-11-11 南通林多光学仪器有限公司 焊接图像熔池边缘检测方法

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292923B (zh) * 2017-06-29 2019-03-01 北京大学深圳研究生院 基于深度图挖掘的后向传播图像视觉显著性检测方法
CN108615040B (zh) * 2018-04-03 2019-10-29 北京联合大学 一种计算图像局部特征描述子的方法
CN110211115B (zh) * 2019-06-03 2023-04-07 大连理工大学 一种基于深度引导元胞自动机的光场显著性检测实现方法
CN110458797B (zh) * 2019-06-18 2023-05-30 南开大学 一种基于深度图过滤器的显著性物体检测方法
CN113901929A (zh) * 2021-10-13 2022-01-07 河北汉光重工有限责任公司 一种基于显著性的动态目标检测与识别的方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104050682A (zh) * 2014-07-09 2014-09-17 武汉科技大学 一种融合颜色和深度信息的图像分割方法
CN104835146A (zh) * 2015-04-14 2015-08-12 上海大学 基于深度信息和图切割的立体图像中显著对象分割方法
CN105787938A (zh) * 2016-02-25 2016-07-20 上海大学 一种基于深度图的人物分割方法
CN107292923A (zh) * 2017-06-29 2017-10-24 北京大学深圳研究生院 基于深度图挖掘的后向传播图像视觉显著性检测方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140378810A1 (en) * 2013-04-18 2014-12-25 Digimarc Corporation Physiologic data acquisition and analysis
US9390315B1 (en) * 2015-06-25 2016-07-12 A9.Com, Inc. Image match for featureless objects
CN105761238B (zh) * 2015-12-30 2018-11-06 河南科技大学 一种通过灰度统计数据深度信息提取显著目标的方法
CN107145892B (zh) * 2017-05-24 2019-01-22 北京大学深圳研究生院 一种基于自适应融合机制的图像显著性物体检测方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104050682A (zh) * 2014-07-09 2014-09-17 武汉科技大学 一种融合颜色和深度信息的图像分割方法
CN104835146A (zh) * 2015-04-14 2015-08-12 上海大学 基于深度信息和图切割的立体图像中显著对象分割方法
CN105787938A (zh) * 2016-02-25 2016-07-20 上海大学 一种基于深度图的人物分割方法
CN107292923A (zh) * 2017-06-29 2017-10-24 北京大学深圳研究生院 基于深度图挖掘的后向传播图像视觉显著性检测方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
CHUNBIAO ZHU ET AL.: "A Multilayer Backpropagation Saliency Detection Algorithm Based on Depth Mining", 17TH. INTERNATIONAL CONFERENCE ON COMPUTER ANALYSIS OF IMAGES AND PATTERNS, 28 July 2017 (2017-07-28), pages 14 - 23, XP047422891, DOI: 10.1007/978-3-319-64698-5_2 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115330774A (zh) * 2022-10-12 2022-11-11 南通林多光学仪器有限公司 焊接图像熔池边缘检测方法
CN115330774B (zh) * 2022-10-12 2023-05-19 南通林多光学仪器有限公司 焊接图像熔池边缘检测方法

Also Published As

Publication number Publication date
CN107292923B (zh) 2019-03-01
US20210287034A1 (en) 2021-09-16
CN107292923A (zh) 2017-10-24
US11227178B2 (en) 2022-01-18

Similar Documents

Publication Publication Date Title
WO2019000821A1 (zh) 基于深度图挖掘的后向传播图像视觉显著性检测方法
US10776936B2 (en) Point cloud matching method
WO2020078263A1 (zh) 图像分割方法、装置、计算机设备及存储介质
EP3627445B1 (en) Head pose estimation using rgbd camera
US9305240B2 (en) Motion aligned distance calculations for image comparisons
JP2020064637A (ja) 畳み込みニューラルネットワークを介してイメージ偽変造を探知するシステム、方法、及びこれを利用して無補正探知サービスを提供する方法
US20150379371A1 (en) Object Detection Utilizing Geometric Information Fused With Image Data
CN109871823B (zh) 一种结合旋转框和上下文信息的卫星图像舰船检测方法
JP2016508652A (ja) 画像シーケンスにおける物体のオクルージョンの決定
US9418309B2 (en) Method and apparatus for performing a fragmentation assessment of a material
US20210104067A1 (en) Multi-Person Pose Estimation Using Skeleton Prediction
CN105869120A (zh) 一种图像拼接的实时性优化方法
WO2019015344A1 (zh) 基于中心暗通道先验信息的图像显著性物体检测方法
US20230334235A1 (en) Detecting occlusion of digital ink
WO2009123354A1 (en) Method, apparatus, and program for detecting object
CN110910437B (zh) 一种复杂室内场景的深度预测方法
WO2018082308A1 (zh) 一种图像处理方法及终端
US20190205693A1 (en) Scale-Invariant Feature Point Extraction in Edge Map
JPWO2014024655A1 (ja) 画像処理装置、画像処理方法および画像処理プログラム
WO2019029098A1 (zh) 基于对数极空间的图像物体大小和旋转估计计算方法
CN111524117A (zh) 一种基于特征金字塔网络的隧道表面缺陷检测方法
Wang et al. Geometric consistency enhanced deep convolutional encoder-decoder for urban seismic damage assessment by UAV images
JP2016031766A (ja) Ｃａｄデータとｓｅｍ画像との位置合わせ
Hu et al. Supervised multi-scale attention-guided ship detection in optical remote sensing images
CN112509014B (zh) 金字塔遮挡检测块匹配的鲁棒插值光流计算方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17915881

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17915881

Country of ref document: EP

Kind code of ref document: A1