CN109118493B - Method for detecting salient region in depth image - Google Patents

Method for detecting salient region in depth image Download PDF

Info

Publication number
CN109118493B
CN109118493B CN201810757983.1A CN201810757983A CN109118493B CN 109118493 B CN109118493 B CN 109118493B CN 201810757983 A CN201810757983 A CN 201810757983A CN 109118493 B CN109118493 B CN 109118493B
Authority
CN
China
Prior art keywords
value
pixel
depth
saliency
depth image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810757983.1A
Other languages
Chinese (zh)
Other versions
CN109118493A (en
Inventor
李捷
周宏扬
袁夏
赵春霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Science and Technology
Original Assignee
Nanjing University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Science and Technology filed Critical Nanjing University of Science and Technology
Priority to CN201810757983.1A priority Critical patent/CN109118493B/en
Publication of CN109118493A publication Critical patent/CN109118493A/en
Application granted granted Critical
Publication of CN109118493B publication Critical patent/CN109118493B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种深度图像中的显著区域检测方法。首先通过计算深度图中各像素点与邻近点的差异，获得各像素点的梯度特征；再采用全局对比度的方法，由梯度特征计算获得初始的显著性图；然后依据深度图的直方图统计特征进行峰谷检测与零视差区域估计，实施背景区域和前景区域的划分，进而抑制背景部分的显著性，保留前景部分的显著性；最后扩展超像素划分方法进一步优化得到最终的显著区域检测图。本发明先后通过背景区域划分与超像素划分有效的抑制背景部分的显著性、优化前景部分的显著性，为目标检测、目标识别、场景理解等提供可靠的显著区域，提高对图像感兴趣区域的获取能力。

Figure 201810757983

The invention discloses a salient area detection method in a depth image. First, the gradient feature of each pixel is obtained by calculating the difference between each pixel in the depth map and its adjacent points; then the global contrast method is used to obtain the initial saliency map from the gradient feature calculation; then the statistical features of the histogram of the depth map are used Perform peak-valley detection and zero-parallax region estimation, implement the division of background and foreground regions, and then suppress the saliency of the background part and retain the saliency of the foreground part; finally, the superpixel division method is further optimized to obtain the final saliency region detection map. The invention can effectively suppress the saliency of the background part and optimize the saliency of the foreground part through the background area division and the superpixel division successively, so as to provide a reliable salient area for target detection, target recognition, scene understanding, etc., and improve the saliency of the image area of interest acquire ability.

Figure 201810757983

Description

Method for detecting salient region in depth image
Technical Field
The invention relates to a region detection method technology, in particular to a method for detecting a salient region in a depth image.
Background
Depth information is one of important information channels of human visual attention mechanisms, and can help people to quickly find interested areas in complex scenes.
With the progress of sensor technology, the acquisition of depth images is easier, and how to understand a three-dimensional scene represented by the depth images is an important problem to be solved in the fields of intelligent robot navigation, environment modeling, somatosensory games and the like. The salient regions may be used to guide the robot in discovering potential objects in the scene and reduce the computational load of environmental understanding.
The currently common method for calculating the significant area of the depth map is based on the global contrast of the depth value, the depth value is directly used for calculation, and the method is easily interfered by noise data in the depth map, so that the detection result is inaccurate. Furthermore, a change in the depth value range of the different depth map makes the detection result unstable.
Disclosure of Invention
The invention aims to provide a method for detecting a salient region of a depth image, which can accurately and stably estimate the salient region.
The technical solution for realizing the purpose of the invention is as follows: a salient region detection method of a depth image comprises the following steps:
step 1, aiming at each pixel I in the depth image IkSeparately extracting gradient features
Figure GDA0003080270270000011
Step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation mode according to the gradient features extracted in the step 1k) Obtaining an initial saliency map of the same resolution;
step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs;
step 4, estimating a zero parallax area ZPA of the depth image;
step 5, dividing a background area and a foreground area in the depth image according to the peak and trough detection result and the zero parallax area ZPA, adjusting the saliency map obtained in the step 2 according to the background area and the foreground area, and inhibiting the saliency value of the background area to obtain an improved saliency map;
and 6, performing superpixel segmentation on the original image by adopting a superpixel segmentation algorithm, and then optimizing the saliency map obtained in the step 5 to obtain a final saliency area.
Compared with the prior art, the invention has the following remarkable advantages: (1) the method adopts the gradient characteristic of the depth value as the calculation basis of the overall contrast, is less interfered by noise and is not influenced by the change of the range of the depth value, so that the result is more accurate; (2) the invention effectively inhibits the significance of the background and the salient target area by utilizing the method of dividing the background area, and improves the stability of the detection result.
The invention is further described below with reference to the accompanying drawings.
Drawings
Fig. 1 is a flowchart of a salient region detection method of a depth map according to the present invention.
Fig. 2 is a schematic diagram of salient region detection according to an embodiment of the present invention, in which a diagram (a) is an original image, a diagram (b) is a depth image, and a diagram (c) is a detection result diagram.
Detailed Description
A method for detecting a salient region in a depth image comprises the following steps:
step 1, aiming at each pixel I in the depth image IkSeparately extracting gradient features
Figure GDA0003080270270000026
In a further embodiment, extracting the gradient feature of the depth map specifically includes the following steps:
step 1-1, traversing all pixel points of the depth image I to obtain a gradient vector of each pixel point, and counting the pixel points IkN, N is the total number of pixels, and its gradient vector (dr) isk,dck) The calculation formula is as follows:
drk＝(dep(r+1,c)-dep(r-1,c)/2 (1)
dck＝(dep(r,c+1)-dep(r,c-1)/2 (2)
wherein r and c correspond to rows and columns of image coordinates, dep (r, c) represents the depth value of the r-th row and c-th column in the depth image I;
step 1-2, traversing all pixel points to obtain the gradient characteristic of each point, and obtaining a pixel point IkCharacteristic of gradient of
Figure GDA0003080270270000021
The method specifically comprises the following steps:
Figure GDA0003080270270000022
Figure GDA0003080270270000023
where ε is a constant greater than zero and Maximun is specified as
Figure GDA0003080270270000024
And
Figure GDA0003080270270000025
is measured.
Step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation mode according to the gradient features extracted in the step 1k) Obtaining an initial saliency map of the same resolution;
in a further embodiment, an initial saliency value S (I) of each pixel is calculated by a global contrast method according to the gradient characteristicsk) Obtaining an initial saliency map of the same resolution, specifically comprising the following steps:
step 2-1, normalizing two elements of all gradient feature vectors to an interval [0, 255%]Rounding off to obtain integer value and pixel point IkAfter being normalized, the gradient features of
Figure GDA0003080270270000031
Thereby the gradient characteristic values of all pixel points
Figure GDA0003080270270000032
All correspond to [0,255]An integer of up to 256 different values, noted
Figure GDA0003080270270000033
The same can be obtained
Figure GDA0003080270270000034
Step 2-2, obtaining characteristic values according to the calculation mode of the global contrast
Figure GDA0003080270270000035
Corresponding significance values:
Figure GDA0003080270270000036
where n is 256, the total number of feature values extracted from the depth image, and fjRepresents
Figure GDA0003080270270000037
The probability of occurrence in the image is,
Figure GDA0003080270270000038
is composed of
Figure GDA0003080270270000039
And
Figure GDA00030802702700000310
a distance metric function of the two features; for characteristic value
Figure GDA00030802702700000311
In the same way, the corresponding significance values are obtained:
Figure GDA00030802702700000312
step 2-3, the corresponding significance values of the pixels with the same characteristic value are also the same, and the pixel I is subjected tokIf its characteristic value is
Figure GDA00030802702700000313
The initial saliency value for that pixel is then:
Figure GDA00030802702700000314
wherein, waAnd wbIs a weight parameter; for each pixel, according to the characteristic value of the pixel, the significance value of the pixel can be obtained, and therefore the initial significance map of the full resolution is obtained.
Step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs;
in a further embodiment, the peak-valley detection is performed by using the histogram statistical features of the depth map, and the steps are as follows:
step 3-1, dividing the depth values of all pixels in the depth map into 256 intervals, and counting the number of the pixels of the depth values in each interval range to obtain a statistical histogram;
step 3-2, calculating a derivative of the histogram statistic value to obtain the growth rate of each position corresponding to the abscissa of the histogram, and forming a vector alpha ═ alpha1,α2,…,α256}；
Step 3-3, taking alphaiSymbol λ of 1,2, …, 256αiAnd forming them into a vector lambda in orderα＝{λα1,λα2,…,λα256}，αiSign of (a)αiThe concrete formula of (1) is as follows:
Figure GDA0003080270270000041
step 3-4, vector λαCarrying out mean value filtering, and executing the operation of the step 3-3 on the filtered result to obtain a new digital string lambdaβ＝{λβ1,λβ2,…,λβ256}；
Step 3-5, vector lambda is aligned by adopting a template matching modeβCarrying out jump detection; there are 4 types of hopping: [1, -1]，[1,0,-1]The jump position is the peak position Pp；[-1,1]，[-1,0,1]Corresponding to the position P of the wave trought。
Step 4, estimating a Zero Parallax Area (ZPA) of the depth map;
in a further embodiment, the specific step of estimating the zero-disparity region ZPA of the depth image is as follows:
step 4-1, calculating the median of the depth values in the depth image, i.e.
Figure GDA0003080270270000042
And 4-2, taking the median as a center, wherein the area within the range of the distance between the front and the back of the center and the distance between the front and the back of the center is ZPA of the scene:
Figure GDA0003080270270000043
in equation (9), H is the depth-of-field (DOF) of the scene, and σ is the scale parameter.
And 5, dividing a background area and a foreground area in the depth image, adjusting the initial saliency map obtained in the step 2 according to the background area and the foreground area, inhibiting the saliency value of the background area, and obtaining an improved saliency map.
In a further embodiment, the improved saliency map is obtained by the following steps:
step 5-1, determining a depth value corresponding to a peak-valley position which is behind the zero parallax zone ZPA and is closest to the zero parallax zone ZPA, namely a final threshold value T of the background estimation:
T＝min(p),st p∈{Pp,Ptand p > ZPA (10)
Step 5-2, in the depth image, taking an area with a depth value larger than a background threshold value T as a background part, and taking a part with a depth value smaller than T as a foreground area, and thus determining whether a pixel at a corresponding position in the saliency map belongs to the background part or the foreground part; suppressing the significance value of the background part in the significance map, and reserving the significance value of the foreground part in the significance map to obtain an improved significance map, wherein the suppression formula of the significance value of the background part is as follows:
Figure GDA0003080270270000044
in the formula, depkIs a pixel point IkCorresponding to the depth value on the depth image, S (I)k) Is the initial saliency value, S' (I) of the background portionk) The significance value of the background part after inhibition is shown.
Step 6, performing superpixel segmentation on the original image by adopting a superpixel segmentation algorithm, and then optimizing the saliency map obtained in the step 5 to obtain a final saliency area;
in a further embodiment, the step of optimizing the saliency map in step 5 based on the super-pixel pairs is as follows:
step 6-1, initializing a clustering center: setting the number of superpixels as C, length in two-dimensional space
Figure GDA0003080270270000051
For the interval, periodically sampling the depth image, taking each sampling point as an initial clustering center, setting the category labels of all the pixels of the initial clustering centers to be 1,2, … and C, setting the category labels of all the pixels of the non-clustering centers to be-1, setting the distance between the pixels of the non-clustering centers and the clustering centers to be infinite, and setting N to be the total number of the pixels in the whole depth image;
step 6-2, for each clustering center IcC, respectively calculating the cluster center and each pixel point I in the 2s × 2s neighborhood search range of the cluster centeri1, 2., a distance of 2s × 2s, the distance calculation formula is as follows:
Figure GDA0003080270270000052
wherein depcFor clustering central pixel point IcDepth value of uc,νcIs IcAbscissa and ordinate in the image; depiIs a pixel point IiDepth value of ui,νiIs IiThe horizontal and vertical coordinates in the image, m is the compactness adjusting parameter of the super pixel;
each non-clustering center pixel point is searched by a plurality of surrounding clustering center points, the clustering center corresponding to the minimum distance value is taken as the clustering center of the pixel point, and the clustering center is set as a category label same as the clustering center, so that a super-pixel segmentation result is obtained;
6-3, calculating the depth mean value and the horizontal and vertical coordinate mean values of the pixel points in each super pixel, taking the depth mean value and the coordinate mean value of each super pixel as a new clustering center of the super pixel, and repeating the step 6-2 until the clustering center to which each pixel point belongs does not change any more;
step 6-4, counting the number of pixels contained in each super pixel, and merging the number of pixels with the super pixel with the nearest coordinate position in the adjacent super pixels when the number of pixels is smaller than a set minimum value e; after combination, all the super-pixels R are obtainedcC ═ 1,2,. C ', where C' ≦ C;
6-5, according to the super pixel RcOptimizing the significance result obtained in step 5, i.e. if Ik∈RcThen the pixel IkFinal significance value of S ″ (I)k) Comprises the following steps:
Figure GDA0003080270270000053
wherein, | RcIs at the super-pixel RcThe number of pixels contained in (a).
The present invention is further illustrated by the following specific examples.
Example 1
As shown in fig. 1, a method for detecting a salient region in a depth image includes the following steps:
step 1, for a depth image I, for each pixel I in the imagekSeparately extracting gradient features
Figure GDA0003080270270000069
The original image is shown in fig. 2 (a), and the depth image is shown in fig. 2 (b);
step 1-1, traversing all pixel points of the depth image I to obtain a gradient vector of each pixel point, and counting the pixel points IkN, N is the total number of pixels, and its gradient vector (dr) isk,dck) The calculation formula is as follows:
drk＝(dep(r+1,c)-dep(r-1,c)/2 (1)
dck＝(dep(r,c+1)-dep(r,c-1)/2 (2)
wherein r and c correspond to rows and columns of image coordinates, dep (r, c) represents the depth value of the r-th row and c-th column in the depth image I;
step 1-2, traversing all pixel points to obtain the gradient characteristic of each point, and obtaining a pixel point IkCharacteristic of gradient of
Figure GDA0003080270270000061
The method specifically comprises the following steps:
Figure GDA0003080270270000062
Figure GDA0003080270270000063
wherein ε is 0.02 and Maximun is GaAnd GbMaximum value of (a), maximum 600 in this example;
step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation method according to the gradient characteristics in the step 1k) Obtaining an initial saliency map of the same resolution, specifically comprising the following steps:
step 2-1, normalizing two elements of all gradient feature vectors to an interval [0, 255%]Rounding off to obtain integer value and pixel point IkAfter being normalized, the gradient features of
Figure GDA0003080270270000064
Thereby the gradient characteristic values of all pixel points
Figure GDA0003080270270000065
All correspond to [0,255]An integer of up to 256 different values, noted
Figure GDA0003080270270000066
The same can be obtained
Figure GDA0003080270270000067
Step 2-2, obtaining characteristic values according to the calculation mode of the global contrast
Figure GDA0003080270270000068
Corresponding significance values:
Figure GDA0003080270270000071
where n is 256, the total number of feature values extracted from the depth image, and fjRepresents
Figure GDA0003080270270000072
The probability of occurrence in the image is,
Figure GDA0003080270270000073
is composed of
Figure GDA0003080270270000074
And
Figure GDA0003080270270000075
a distance metric function of the two features; for characteristic value
Figure GDA0003080270270000076
In the same way, the corresponding significance values are obtained:
Figure GDA0003080270270000077
step 2-3, the corresponding significance values of the pixels with the same characteristic value are also the same, and the pixel I is subjected tokIf its characteristic value is
Figure GDA0003080270270000078
The initial saliency value for that pixel is then:
Figure GDA0003080270270000079
wherein, waAnd wbAre weight parameters, all set to 0.5; for each pixel, according to the characteristic value of the pixel, the significance value of the pixel can be obtained, and therefore an initial significance map of the full resolution is obtained;
step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs, and comprising the following steps:
step 3-1, dividing the depth values of all pixels in the depth map into 256 intervals, and counting the number of the pixels of the depth values in each interval range to obtain a statistical histogram;
step 3-2, calculating a derivative of the histogram statistic value to obtain the growth rate of each position corresponding to the abscissa of the histogram, and forming a vector alpha ═ alpha1,α2,…,α256}；
Step 3-3, taking alpha according to a formula (8)iSign of (a)αiAnd forming them into a vector lambda in orderα＝{λα1,λα2,…,λα256}：
Figure GDA00030802702700000710
Step 3-4, vector λαCarrying out mean value filtering, repeating the operation of the step 3-3 once on the filtered result to obtain a new digital string lambdaβ＝{λβ1,λβ2,…,λβ256}；
Step 3-5, vector lambda is matched in a template matching modeβCarrying out jump detection; there are 4 types of hopping: [1, -1]，[1,0,-1]The jump position is the peak position Pp；[-1,1]，[-1,0,1]Corresponding to the position P of the wave trought。
Step 4, estimating a Zero Parallax Area (ZPA) of the depth image, which comprises the following steps:
step 4-1, calculating the median of the depth values in the depth image, i.e.
Figure GDA00030802702700000711
And 4-2, determining areas around the median as ZPA of the scene:
Figure GDA0003080270270000081
in equation (9), H is the depth-of-field (DOF) of the scene, and σ is 0.1, which is a scale parameter.
Step 5, dividing a background area and a foreground area in the depth image, adjusting the saliency map obtained in the step 2 according to the background area, suppressing the saliency value of the background area, and obtaining an improved saliency map, wherein the steps are as follows:
step 5-1, determining a depth value corresponding to a peak-valley position which is behind the zero parallax zone ZPA and is closest to the zero parallax zone ZPA, namely a final threshold value T of the background estimation:
T＝min(p),st p∈{Pp,Ptand p > ZPA (10)
Step 5-2, in the depth image, taking an area with a depth value larger than a background threshold value T as a background part, and taking a part with a depth value smaller than T as a foreground area, and thus determining whether a pixel at a corresponding position in the saliency map belongs to the background part or the foreground part; suppressing the significance value of the background part in the significance map, and reserving the significance value of the foreground part in the significance map to obtain an improved significance map, wherein the suppression formula of the significance value of the background part is as follows:
Figure GDA0003080270270000082
in the formula, depkIs a pixel point IkCorresponding to the depth value on the depth image, S (I)k) Is the initial saliency value, S' (I) of the background portionk) For background partial significance value after suppression
And 6, further improving and obtaining a final significant region detection result by adopting a method based on super-pixel division, wherein the steps are as follows:
step 6-1, initializing a clustering center: setting the number of superpixels as the number of superpixels C1600 and the length in the two-dimensional space in the whole depth image
Figure GDA0003080270270000083
Periodically sampling the depth image for intervals, taking each sampling point as an initial clustering center, setting the class labels of all pixels in the initial clustering centers to be 1,2, … and C, setting the class labels of all pixels in non-clustering centers to be-1, setting the distance between the pixels and the clustering centers to be infinite, setting N to be the total number of pixels in the whole depth image, and for a typical depth image with the resolution of 640 multiplied by 480, corresponding interval length to the depth image is equal to
Figure GDA0003080270270000084
Step 6-2, for each clustering center IcC, calculating the cluster center and each pixel point I in the 28 × 28 neighborhood search range respectivelyi1, 2., a distance of 2s × 2s, the distance calculation formula is as follows:
Figure GDA0003080270270000091
wherein depcFor clustering central pixel point IcDepth value of uc,νcIs IcAbscissa and ordinate in the image; depiIs a pixel point IiDepth value of ui,νiIs IiAbscissa and ordinate in the image; the compactness adjusting parameter m of the super pixel is 40;
each non-clustering center pixel point is searched by a plurality of surrounding clustering center points, the clustering center corresponding to the minimum distance value is taken as the clustering center of the pixel point, and the clustering center is set as a category label same as the clustering center, so that a super-pixel segmentation result is obtained;
6-3, calculating the depth mean value and the horizontal and vertical coordinate mean values of the pixel points in each super pixel, taking the depth mean value and the coordinate mean value of each super pixel as a new clustering center of the super pixel, and repeating the step 6-2 until the clustering center to which each pixel point belongs does not change any more; in the embodiment, 10 iterations can obtain ideal effects on most pictures, so that 10 iterations are selected;
step 6-4, setting the minimum value e of the number of pixels contained in the superpixel to be 20, and combining a morphological region smaller than e with a neighborhood thereof; after combination, all the super-pixels R are obtainedcC ═ 1,2,. C ', where C' ≦ C;
6-5, according to the super pixel RcOptimizing the significance result obtained in step 5, i.e. if Ik∈RcThen the pixel IkFinal significance value of S ″ (I)k) Comprises the following steps:
Figure GDA0003080270270000092
wherein, | RcIs at the super-pixel RcThe number of pixels contained in (a). As shown in fig. 2 (c), the closer the area in the graph is to white, the higher the saliency value of the area is, and the closer to black, the lower the saliency value is.
The invention adopts the gradient characteristic of the depth value as the calculation basis of the overall contrast, reduces noise interference, is not influenced by the change of the range of the depth value, and improves the accuracy of the detection result; by using a background area division method, the significance of a background and a highlighted target area is effectively inhibited, and the stability of a detection result is improved; the saliency of the foreground part is optimized by dividing the superpixels, so that a saliency map of a full resolution is obtained through calculation, a reliable saliency region is provided for target detection, target identification, scene understanding and the like, and the acquisition capability of an image region of interest is improved.

Claims (7)

1. A method for detecting a salient region in a depth image is characterized by comprising the following steps:
step 1, aiming at each pixel I in the depth image IkSeparately extracting gradient features
Figure FDA0003080270260000011
Step 2, calculating an initial significance value S (I) of each pixel by adopting a global contrast calculation mode according to the gradient features extracted in the step 1k) Obtaining an initial saliency map of the same resolution;
step 3, utilizing the histogram statistical characteristics of the depth map to detect wave crests and wave troughs;
step 4, estimating a zero parallax area ZPA of the depth image;
step 5, dividing a background area and a foreground area in the depth image according to the peak and trough detection result and the zero parallax area ZPA, adjusting the saliency map obtained in the step 2 according to the background area and the foreground area, and inhibiting the saliency value of the background area to obtain an improved saliency map;
and 6, performing superpixel segmentation on the original image by adopting a superpixel segmentation algorithm, and then optimizing the saliency map obtained in the step 5 to obtain a final saliency area.
2. The method according to claim 1, wherein the extracting gradient features of the depth map in step 1 specifically comprises the following steps:
step 1-1, traversing all pixel points of the depth image I to obtain a gradient vector of each pixel point, and counting the pixel points IkN, N is the total number of pixels, and its gradient vector (dr) isk,dck) The calculation formula is as follows:
drk＝(dep(r+1,c)-dep(r-1,c)) /2 (1)
dck＝(dep(r,c+1)-dep(r,c-1)) /2 (2)
wherein r and c correspond to rows and columns of image coordinates, dep (r, c) represents the depth value of the r-th row and c-th column in the depth image I;
step 1-2, traversing all pixel points to obtain the gradient characteristic of each point, and obtaining a pixel point IkCharacteristic of gradient of
Figure FDA0003080270260000012
The method specifically comprises the following steps:
Figure FDA0003080270260000013
Figure FDA0003080270260000014
where ε is a constant greater than zero and Maximun is specified as
Figure FDA0003080270260000015
And
Figure FDA0003080270260000016
is measured.
3. The method for detecting the salient region in the depth image according to claim 1, wherein in step 2, an initial saliency value S (I) of each pixel is calculated by adopting a global contrast method according to gradient featuresk) Obtaining an initial saliency map of the same resolution, specifically comprising the following steps:
step 2-1, normalizing two elements of all gradient feature vectors to an interval [0, 255%]Rounding off to obtain integer value and pixel point IkAfter being normalized, the gradient features of
Figure FDA0003080270260000021
Thereby the gradient characteristic values of all pixel points
Figure FDA0003080270260000022
All correspond to [0,255]An integer of up to 256 different values, noted
Figure FDA0003080270260000023
The same can be obtained
Figure FDA0003080270260000024
Step 2-2, obtaining characteristic values according to the calculation mode of the global contrast
Figure FDA0003080270260000025
Corresponding significance values:
Figure FDA0003080270260000026
where n is 256, the total number of feature values extracted from the depth image, and fjRepresents
Figure FDA0003080270260000027
The probability of occurrence in the image is,
Figure FDA0003080270260000028
is composed of
Figure FDA0003080270260000029
And
Figure FDA00030802702600000210
a distance metric function of the two features; for characteristic value
Figure FDA00030802702600000211
In the same way, the corresponding significance values are obtained:
Figure FDA00030802702600000212
step 2-3, the corresponding significance values of the pixels with the same characteristic value are also the same, and the pixel I is subjected tokIf its characteristic value is
Figure FDA00030802702600000213
The initial saliency value for that pixel is then:
Figure FDA00030802702600000214
wherein, waAnd wbIs a weight parameter; for each pixel, according to the characteristic value of the pixel, the significance value of the pixel can be obtained, and therefore the initial significance map of the full resolution is obtained.
4. The method for detecting the significant region in the depth image according to claim 1, wherein the step 3 performs peak-valley detection by using histogram statistical features of the depth image, and comprises the following steps:
step 3-1, dividing the depth values of all pixels in the depth map into 256 intervals, and counting the number of the pixels of the depth values in each interval range to obtain a statistical histogram;
step 3-2, calculating a derivative of the histogram statistic value to obtain the growth rate of each position corresponding to the abscissa of the histogram, and forming a vector alpha ═ alpha1,α2,…,α256}；
Step 3-3, taking alphaiSymbol λ of 1,2, …, 256αiAnd forming them into a vector lambda in orderα＝{λα1,λα2,…,λα256}，αiSign of (a)αiThe concrete formula of (1) is as follows:
Figure FDA0003080270260000031
step 3-4, vector λαCarrying out mean value filtering, and executing the operation of the step 3-3 on the filtered result to obtain a new digital string lambdaβ＝{λβ1,λβ2,…,λβ256}；
Step 3-5, vector lambda is aligned by adopting a template matching modeβCarrying out jump detection; a total of 4Hopping type: [1, -1]，[1,0,-1]The jump position is the peak position Pp；[-1,1]，[-1,0,1]Corresponding to the position P of the wave trought。
5. The method according to claim 2, wherein the step 4 of estimating the zero-disparity region ZPA of the depth image comprises the following specific steps:
step 4-1, calculating the median of the depth values in the depth image, i.e.
Figure FDA0003080270260000032
And 4-2, taking the median as a center, wherein an area within a range of the distance between the front and the back of the center and the distance between the front and the back of the center are zero parallax areas ZPA of the depth image:
Figure FDA0003080270260000033
in the formula, H is the depth of field of the scene, and σ is a proportional parameter.
6. The method according to claim 4, wherein in step 5, the background area and the foreground area in the depth image are divided, and the saliency map obtained in step 2 is adjusted accordingly to suppress the saliency value of the background area, so as to obtain an improved saliency map, and the steps are as follows:
step 5-1, determining a depth value corresponding to a peak-valley position which is behind the zero parallax zone ZPA and is closest to the zero parallax zone ZPA, namely a final threshold value T of the background estimation:
T＝min(p),st p∈{Pp,Ptand p > ZPA (10)
Step 5-2, in the depth image, taking an area with a depth value larger than a background threshold value T as a background part, and taking a part with a depth value smaller than T as a foreground area, and thus determining whether a pixel at a corresponding position in the saliency map belongs to the background part or the foreground part; suppressing the significance value of the background part in the significance map, and reserving the significance value of the foreground part in the significance map to obtain an improved significance map, wherein the suppression formula of the significance value of the background part is as follows:
Figure FDA0003080270260000034
in the formula, depkIs a pixel point IkCorresponding to the depth value on the depth image, S (I)k) Is the initial saliency value, S' (I) of the background portionk) The significance value of the background part after inhibition is shown.
7. The method for detecting the salient region in the depth image according to claim 1, wherein the original image is subjected to superpixel segmentation by using a superpixel segmentation algorithm in step 6, and then the salient image obtained in step 5 is optimized to obtain a final salient region, and the specific steps are as follows:
step 6-1, initializing a clustering center: setting the number of superpixels as C, length in two-dimensional space
Figure FDA0003080270260000041
For the interval, periodically sampling the depth image, taking each sampling point as an initial clustering center, setting the category labels of all the pixels of the initial clustering centers to be 1,2, … and C, setting the category labels of all the pixels of the non-clustering centers to be-1, setting the distance between the pixels of the non-clustering centers and the clustering centers to be infinite, and setting N to be the total number of the pixels in the whole depth image;
step 6-2, for each clustering center IcC, respectively calculating the cluster center and each pixel point I in the 2s × 2s neighborhood search range of the cluster centeri1, 2., a distance of 2s × 2s, the distance calculation formula is as follows:
Figure FDA0003080270260000042
wherein depcFor clustering central pixel point IcDepth value of uc,νcIs IcAbscissa and ordinate in the image; depiIs a pixel point IiDepth value of ui,νiIs IiAbscissa and ordinate in the image;
each non-clustering center pixel point is searched by a plurality of surrounding clustering center points, the clustering center corresponding to the minimum distance value is taken as the clustering center of the pixel point, and the clustering center is set as a category label same as the clustering center, so that a super-pixel segmentation result is obtained;
6-3, calculating the depth mean value and the horizontal and vertical coordinate mean values of the pixel points in each super pixel, taking the depth mean value and the coordinate mean value of each super pixel as a new clustering center of the super pixel, and repeating the step 6-2 until the clustering center to which each pixel point belongs does not change any more;
step 6-4, counting the number of pixels contained in each super pixel, and merging the number of pixels with the super pixel with the nearest coordinate position in the adjacent super pixels when the number of pixels is smaller than a set minimum value e; after combination, all the super-pixels R are obtainedcC ═ 1,2,. C ', where C' ≦ C;
6-5, according to the super pixel RcOptimizing the significance result obtained in step 5, i.e. if Ik∈RcThen the pixel IkFinal significance value of S ″ (I)k) Comprises the following steps:
Figure FDA0003080270260000043
wherein, | RcIs at the super-pixel RcThe number of pixels contained in (a).
CN201810757983.1A 2018-07-11 2018-07-11 Method for detecting salient region in depth image Active CN109118493B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810757983.1A CN109118493B (en) 2018-07-11 2018-07-11 Method for detecting salient region in depth image

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810757983.1A CN109118493B (en) 2018-07-11 2018-07-11 Method for detecting salient region in depth image

Publications (2)

Publication Number Publication Date
CN109118493A CN109118493A (en) 2019-01-01
CN109118493B true CN109118493B (en) 2021-09-10

Family

ID=64862700

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810757983.1A Active CN109118493B (en) 2018-07-11 2018-07-11 Method for detecting salient region in depth image

Country Status (1)

Country Link
CN (1) CN109118493B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111986245B (en) * 2019-05-23 2024-11-12 北京猎户星空科技有限公司 Depth information evaluation method, device, electronic device and storage medium
CN114640850B (en) * 2022-02-28 2024-06-18 上海顺久电子科技有限公司 Video image motion estimation method, display device and chip
CN119477891B (en) * 2025-01-07 2025-03-25 天津大学 A method for detecting defects in AC/DC hybrid power lines based on image data

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103208123A (en) * 2013-04-19 2013-07-17 广东图图搜网络科技有限公司 Image segmentation method and system
CN104574375A (en) * 2014-12-23 2015-04-29 浙江大学 Image Saliency Detection Method Combining Color and Depth Information
CN105550678A (en) * 2016-02-03 2016-05-04 武汉大学 Human body motion feature extraction method based on global remarkable edge area
CN107169487A (en) * 2017-04-19 2017-09-15 西安电子科技大学 The conspicuousness object detection method positioned based on super-pixel segmentation and depth characteristic
CN107240096A (en) * 2017-06-01 2017-10-10 陕西学前师范学院 A kind of infrared and visual image fusion quality evaluating method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10095953B2 (en) * 2009-11-11 2018-10-09 Disney Enterprises, Inc. Depth modification for display applications

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103208123A (en) * 2013-04-19 2013-07-17 广东图图搜网络科技有限公司 Image segmentation method and system
CN104574375A (en) * 2014-12-23 2015-04-29 浙江大学 Image Saliency Detection Method Combining Color and Depth Information
CN105550678A (en) * 2016-02-03 2016-05-04 武汉大学 Human body motion feature extraction method based on global remarkable edge area
CN107169487A (en) * 2017-04-19 2017-09-15 西安电子科技大学 The conspicuousness object detection method positioned based on super-pixel segmentation and depth characteristic
CN107240096A (en) * 2017-06-01 2017-10-10 陕西学前师范学院 A kind of infrared and visual image fusion quality evaluating method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
基于空－频域混合分析的RGB-D数据视觉显著性检测方法;岳娟等;《机器人》;20170930;第39卷(第5期);第652-660页 *

Also Published As

Publication number Publication date
CN109118493A (en) 2019-01-01

Similar Documents

Publication Publication Date Title
Lian et al. Density map regression guided detection network for rgb-d crowd counting and localization
Shen et al. Multiobject tracking by submodular optimization
US8045783B2 (en) Method for moving cell detection from temporal image sequence model estimation
Fradi et al. Low level crowd analysis using frame-wise normalized feature for people counting
CN109118493B (en) Method for detecting salient region in depth image
CN104463911A (en) Small infrared moving target detection method based on complicated background estimation
CN109146918B (en) A Block-Based Adaptive Correlation Target Localization Method
CN111951297B (en) Target tracking method based on structured pixel-by-pixel target attention mechanism
CN105809673B (en) Video foreground dividing method based on SURF algorithm and the maximum similar area of merging
JP2012531130A (en) Video copy detection technology
Déniz et al. Fast and accurate global motion compensation
CN111028263A (en) Moving object segmentation method and system based on optical flow color clustering
Nguyen et al. Salient object detection via augmented hypotheses
CN110363197A (en) Video Region of Interest Extraction Method Based on Improved Visual Background Extraction Model
CN112949539A (en) Pedestrian re-identification interactive retrieval method and system based on camera position
CN114511803A (en) Target occlusion detection method for visual tracking task
CN108647605B (en) Human eye gaze point extraction method combining global color and local structural features
CN103077536A (en) Space-time mutative scale moving target detection method
Yu et al. Motion detection in moving background using a novel algorithm based on image features guiding self-adaptive Sequential Similarity Detection Algorithm
CN107067411B (en) Mean-shift tracking method combined with dense features
JP2020086879A (en) Coordinate transformation matrix estimation method and computer program
Lu et al. Edge and color contexts based object representation and tracking
CN116193103A (en) Video picture jitter level assessment method
Liu et al. [Retracted] Mean Shift Fusion Color Histogram Algorithm for Nonrigid Complex Target Tracking in Sports Video
Taalimi et al. Robust multi-object tracking using confident detections and safe tracklets

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant