WO2020206666A1 - 基于散斑图像的深度估计方法及装置、人脸识别系统 - Google Patents

基于散斑图像的深度估计方法及装置、人脸识别系统 Download PDF

Info

Publication number
WO2020206666A1
WO2020206666A1 PCT/CN2019/082373 CN2019082373W WO2020206666A1 WO 2020206666 A1 WO2020206666 A1 WO 2020206666A1 CN 2019082373 W CN2019082373 W CN 2019082373W WO 2020206666 A1 WO2020206666 A1 WO 2020206666A1
Authority
WO
WIPO (PCT)
Prior art keywords
speckle
target
image
window
input
Prior art date
Application number
PCT/CN2019/082373
Other languages
English (en)
French (fr)
Inventor
吴勇辉
刘川熙
詹洁琼
Original Assignee
深圳市汇顶科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市汇顶科技股份有限公司 filed Critical 深圳市汇顶科技股份有限公司
Priority to PCT/CN2019/082373 priority Critical patent/WO2020206666A1/zh
Priority to CN201980000582.4A priority patent/CN112771573B/zh
Publication of WO2020206666A1 publication Critical patent/WO2020206666A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis

Definitions

  • This application relates to the field of computer vision technology, and in particular to a method and device for depth estimation based on speckle images, and a face recognition system.
  • the embodiments of the present application provide a method and device for depth estimation based on speckle images, and a face recognition system to solve the problem that the prior art has low accuracy in detecting speckles in collected speckle images.
  • large errors in subsequent speckle matching and poor disparity map effects of speckles affect the accuracy of the depth image.
  • an embodiment of the present application provides a method for depth estimation based on a speckle image, including: performing speckle detection on an input speckle image to determine target speckle; and combining the input speckle image with a reference speckle image Stereo correction to align the input speckle image with the reference speckle image; construct the target topological structure of the target speckle in the input speckle image; use a window matching algorithm on the reference speckle image Search for a matching reference speckle that is in the same row as each of the target speckles; determine the visual difference between the target speckle and the matched reference speckle according to the reference speckle that matches the target speckle Difference; interpolate the disparity value based on the target topology structure of the target speckle to obtain a disparity map of the input speckle image; convert the disparity map into a depth image of the input speckle image .
  • the performing speckle detection on the input speckle image includes: detecting the input speckle image based on the gray gradient of pixels to determine the preliminary speckle in the input speckle image; A parabolic surface fitting algorithm determines the sub-pixel center point of each preliminary speckle, and uses the sub-pixel center point as the target speckle.
  • the method before performing the speckle detection on the input speckle image to determine the target speckle, the method further includes: preprocessing the collected initial speckle image to obtain the input speckle image.
  • the detecting the input speckle image based on the gray gradient of pixels to determine the preliminary speckle in the input speckle image includes: using each pixel in the input speckle image respectively Is the center point, determine the grayscale gradient of each pixel in the first neighborhood of the center point; if the grayscale gradient of each pixel meets the preset gradient distribution, determine the center point as the input dispersion Preliminary speckles in the speckle image.
  • determining that the center point is the preliminary speckle in the input speckle image includes: if the grayscale gradient in the first neighborhood The pixel gray value of a pixel is inversely proportional to the distance between the pixel and the center point, and the number of pixels meeting the preset gradient direction in the first neighborhood is greater than the preset pixel number threshold, then it is determined The center point is the preliminary speckle in the input speckle image.
  • the method further includes: if each of the preliminary speckles is determined If there are a plurality of preliminary speckles adjacent to each other, a connected area is formed based on the plurality of adjacent preliminary speckles, and only the central point of the connected area is used as the preliminary speckle.
  • the using a quadratic parabolic fitting algorithm to determine the sub-pixel center point of each of the preliminary speckles, and using the sub-pixel center point as the target speckle includes: establishing a second neighborhood based on the preliminary speckles Construct a quadratic function with the position coordinates of each pixel in the second neighborhood and the gray value of the pixel; obtain a fitting surface under the condition that the quadratic function satisfies the quadratic parabolic constraint; The pixel point corresponding to the position coordinate of the projection of the highest point of the curved surface is used as the sub-pixel center point of the preliminary speckle.
  • the using a window matching algorithm to find a matching reference speckle in the same row as each target speckle on the reference speckle image includes: using each of the input speckle images respectively The pixel is the center point, and the grayscale gradient of each pixel in the first neighborhood of the center point is determined; if the grayscale gradient of each pixel meets the preset gradient distribution, the center point is determined to be the Enter the preliminary speckles in the speckle image.
  • the constructing the target topological structure of the target speckle in the input speckle image includes: constructing multiple target triangles based on each of the target speckles in the input speckle image as a vertex, each of which is The target triangle faces do not overlap each other to form a target triangle mesh.
  • the method further includes: constructing a plurality of reference triangles based on each of the reference speckles in the reference speckle image as a vertex Each of the reference triangular surfaces does not overlap each other to form a reference triangular grid;
  • the using a window matching algorithm to find a matching reference speckle in the same row of each target speckle on the reference speckle image includes: establishing a first window with the target speckle as the center of the window; respectively; A second window is established with each reference speckle on the reference speckle image in the same row as the target speckle as the center of the window; and the target triangle meshes in the first window are compared with those in the second window.
  • Each reference triangle grid is evaluated for similarity; the window center of the second window with the highest similarity evaluation is used as the reference speckle matching the target speckle.
  • the interpolating the disparity value based on the target topology structure of the target speckle to obtain the disparity map of the input speckle image includes: taking the center point of each target triangle as Interpolation point, the disparity value of the interpolation point is determined based on the disparity value of the three vertices of the target triangle; the input dispersion is generated based on the disparity value of each target speckle spot and the disparity value of each interpolation point The disparity map of the spot image.
  • an embodiment of the present application also provides a depth estimation device based on a speckle image, including: a speckle detection module for performing speckle detection on an input speckle image to determine a target speckle; a stereo correction module, Used to perform stereo correction on the input speckle image and the reference speckle image, so that the input speckle image and the reference speckle image are aligned; the topology building module is used to construct the input speckle image The target topological structure of the target speckle; a window matching module, which is used to use a window matching algorithm to find a matching reference speckle that is in the same row as each of the target speckles on the reference speckle image; parallax
  • the value determination module is used to determine the disparity value of the target speckle according to the reference speckle that matches the target speckle; the interpolation processing module is used to perform the analysis on the visual field based on the target topology of the target speckle.
  • the difference value is interpolated to obtain a disparity map of the input speckle image; a depth image generation
  • an embodiment of the present application also provides a face recognition system, including the above-mentioned depth estimation device based on speckle images.
  • speckle detection is performed on the input speckle image to determine the target speckle, and then the input speckle image and the reference speckle image are stereo-corrected to make the two Aligning the image rows, and constructing the target topology structure of the target speckle in the input speckle image; then, using a window matching algorithm to find on the reference speckle image the same as each target speckle According to the matched reference speckles of the row, the disparity value of each target speckle is determined according to the reference speckles matched by each target speckle.
  • the disparity map obtained only based on the disparity value of the target speckle is relatively sparse, and the disparity value needs to be interpolated in combination with the target topology of the target speckle constructed before. Obtain a denser disparity map to improve the accuracy of the depth image.
  • the reference topology structure of the reference speckles in the reference speckle image is also constructed, and the The process of finding a matching reference speckle on the reference speckle image that is in the same row as each target speckle on the input speckle image can be combined with consideration of the reference topology and the target topology for matching, so as to improve the matching speed and Accuracy.
  • the target speckle is determined from the input speckle image based on the gray gradient of the pixel and the quadratic parabolic fitting algorithm, wherein the target speckle is determined based on the gradient
  • the distribution requires detecting possible speckles in the input speckle image (i.e. preliminary speckles), and then combining with the quadratic parabolic fitting algorithm to determine the sub-pixel center points of each preliminary speckle (i.e. further accurately estimating the position of the speckles), and Take the sub-pixel center point as the target speckle.
  • the above-mentioned processing method can improve the accuracy of speckle detection in the input speckle image.
  • FIG. 1 is a schematic flowchart of a specific embodiment of a method for depth estimation based on speckle images of the present application
  • FIG. 2a is a schematic diagram of the gray gradient distribution of detected pixels in a depth estimation method based on speckle images of the present application
  • Fig. 2b is a schematic diagram of the fitted curved surface described in a method for depth estimation based on speckle images of the present application;
  • FIG. 3 is a schematic diagram of a target topology structure constructed in a method for depth estimation based on speckle images of the present application
  • FIG. 4 is a schematic diagram before and after interpolation of the disparity value of the target speckle of the input speckle image in the method for depth estimation based on the speckle image of the present application;
  • Figure 5 is a schematic structural diagram of a specific embodiment of a speckle image-based depth estimation device of the present application.
  • Fig. 6 is a schematic structural diagram of a computer device of the present application.
  • Fig. 7 is a schematic structural diagram of an embodiment of a face recognition system of the present application.
  • FIG. 1 is a schematic flowchart of a specific embodiment of a method for depth estimation based on speckle images of the present application. Referring to Figure 1, the method includes:
  • Step 101 Perform speckle detection on the input speckle image to determine the target speckle.
  • Step 102 Perform stereo correction on the input speckle image and the reference speckle image, so that the input speckle image and the reference speckle image are aligned.
  • Step 103 Construct a target topological structure of the target speckle in the input speckle image.
  • Step 104 Use a window matching algorithm to search for matching reference speckles in the same row of each target speckle on the reference speckle image.
  • Step 105 Determine a disparity value between the target speckle and the matched reference speckle according to the reference speckle that matches the target speckle.
  • Step 106 Interpolate the disparity value based on the target topological structure of the target speckle to obtain a disparity map of the input speckle image.
  • Step 107 Convert the disparity map into a depth image of the input speckle image.
  • the reference speckle image is a pre-stored speckle image that is irradiated by a speckle projector onto a flat whiteboard and forms speckles by diffuse reflection of the flat whiteboard, and is collected by an image collector.
  • the depth estimation of the object surface is based on the comparison standard of the position distribution of the speckles on the reference speckle image and the pixel gray value.
  • the input speckle image is compared with the The reference speckle image is matched to determine the disparity value of the target speckle in the input speckle image, and the disparity value of each target speckle is interpolated to obtain a denser disparity map, and then the disparity
  • the image is converted into a depth image of the input speckle image.
  • the depth image is also called the range image, which refers to the image in which the distance from the image collector to each pixel on the surface of the object to be identified (ie depth) is used as the pixel value. It reflects the image to be identified.
  • the geometry of the surface of the object is also called the range image, which refers to the image in which the distance from the image collector to each pixel on the surface of the object to be identified (ie depth) is used as the pixel value. It reflects the image to be identified.
  • step 101 speckle detection is performed on the input speckle image to determine the target speckle.
  • this step includes:
  • Step 1011 Detect the input speckle image based on the gray gradient of the pixel points to determine the preliminary speckle in the input speckle image.
  • Step 1012 Determine the sub-pixel center point of each preliminary speckle by using a quadratic parabolic fitting algorithm, and use the sub-pixel center point as the target speckle.
  • the step 1011 includes:
  • Step 10111 Using each pixel in the input speckle image as a center point, determine the grayscale gradient of each pixel in the first neighborhood of the center point.
  • Step 10112 If the direction of the gray gradient of each pixel meets a preset gradient direction distribution, determine that the center point is a preliminary speckle in the input speckle image.
  • Fig. 2a is a schematic diagram of the gray gradient distribution of detected pixels in a depth estimation method based on speckle images of the present application.
  • the first neighborhood includes 5 ⁇
  • the pixel gray value of each pixel in the 5 pixel matrix except the center point of 24 pixels
  • the input speckle image is a digital image.
  • the image can be regarded as a two-dimensional discrete function.
  • the gray gradient is obtained by deriving the two-dimensional discrete function, which can be calculated by difference. Instead of differentiation, the grayscale gradient of each pixel in the input speckle image is obtained.
  • the pixel gray value of the pixel in the first neighborhood is inversely proportional to the distance of the pixel from the center point. In other words, in the first neighborhood, the farther the pixel points around the center point are from the center point, the smaller the pixel gray value of the pixel point is.
  • the gradient direction of a pixel is a direction obtained after arctangent operation according to the gray gradient of the pixel in the X direction and the gray gradient in the Y direction.
  • the arrow direction of each pixel in FIG. 2a shows the optimal gradient direction distribution state, that is, the grayscale gradient of each pixel in the first neighborhood meets the preset gradient direction at the corresponding position.
  • this optimal gradient direction distribution state may rarely occur. Therefore, in this embodiment, a preset pixel number threshold is set, when the pixels meeting the preset gradient direction in the first neighborhood If the number of points is greater than the preset pixel number threshold, it is considered that the grayscale gradient of the pixel points in the first neighborhood meets the preset gradient direction.
  • a connected area is formed based on the multiple adjacent pixels, and only the center point of the connected area is used as the preliminary speckle.
  • a connected area with a regular shape a pixel corresponding to a geometric center point can be taken as the preliminary scatter
  • a pixel corresponding to the center of gravity of the connected area can be used as the preliminary scatter.
  • the step 1012 includes:
  • Step 10121 Establish a second neighborhood based on the preliminary speckle.
  • Step 10122 Construct a quadratic function based on the position coordinates of each pixel in the second neighborhood and the gray value of the pixel.
  • Step 10123 Obtain a fitted surface under the condition that the quadratic function satisfies the constraint of the quadratic paraboloid.
  • Step 10124 Use the pixel point corresponding to the position coordinate of the projection of the highest point of the fitted curved surface as the sub-pixel center point of the preliminary speckle.
  • the second neighborhood (7 ⁇ 7 pixel matrix) is established based on a preliminary speckle (pixel with a gray value of 234), and the value in the pixel matrix represents the gray scale of the pixel. Degree value).
  • the position coordinates of the preliminary speckle are (3, 3). Assuming that the position coordinate of the highest point projection of the fitted surface is (4.32, 4.98), the pixel point corresponding to the position coordinate is used as the sub-pixel center point of the preliminary speckle.
  • the sub-pixel center point corresponding to each preliminary speckle is determined by the quadratic parabolic fitting algorithm to be a more accurate pixel point than the position of the original preliminary speckle. Therefore, using the sub-pixel as the target speckle can improve The accuracy of speckle detection.
  • step 102 stereo correction is performed on the input speckle image and the reference speckle image, so that the input speckle image and the reference speckle image are aligned.
  • the epipolar constraint can be used to reduce the matching of the corresponding speckle from two-dimensional search to one-dimensional search.
  • the effect of performing stereo correction on the input speckle image and the reference speckle image in this step is to align the two images so that the epipolar lines of the two images are exactly on the same horizontal line. Any point and its corresponding point on another image must have the same line number, and the corresponding point can be matched by one-dimensional search on that line.
  • the reference speckle image is a reference standard when performing stereo correction on the input speckle image, and the input speckle image after the stereo correction is aligned with the reference speckle image.
  • step 103 the target topological structure of the target speckle in the input speckle image is constructed.
  • the purpose of constructing the target topology of the target speckle is:
  • each interpolation point is determined based on the target topology structure.
  • the matching of speckles in the two windows can be judged based on the similarity between the target topology on the input speckle image and the reference topology on the reference speckle image during subsequent window matching. degree.
  • the target topological structure may be a target triangular mesh (for example, a Delaunay triangular mesh).
  • the process of constructing the target triangle mesh is to construct multiple target triangle faces based on each of the target speckle spots in the input speckle image as vertices, and the target triangle faces do not overlap each other to form a target triangle mesh.
  • Fig. 3 is a schematic diagram of a target topology structure constructed in a method for depth estimation based on speckle images of the present application.
  • a plurality of target triangles (target triangles as shown in FIG. 3) are constructed based on each target speckle in the input speckle image as vertices, and these target triangles do not overlap each other, thereby forming a target triangulation Grid (that is, the triangular grid formed by all target triangles in Figure 3).
  • a window matching algorithm is used to search for matching reference speckles that are in the same row as each of the target speckles on the reference speckle image.
  • this step includes:
  • Step 1041a Establish a first window with the target speckle as the center of the window
  • Step 1042a Establish a second window with each reference speckle in the same row of the reference speckle image as the window center;
  • Step 1043a Perform correlation operations on the gray values of pixels in the first window and in the second window to obtain a matching cost
  • Step 1044a Use the window center of the second window corresponding to the extreme value in the matching cost as a reference speckle matching the target speckle.
  • the input speckle image is aligned with the reference speckle image. That is to say, in the process of searching for the reference speckle from the reference speckle image that matches the target speckle on the input speckle image, it is only necessary to search on the same line.
  • a first window (for example, the window size is 4 ⁇ 5) is established with a target speckle on the input speckle image as the center of the window, and then the reference speckle image and the target speckle on the reference speckle image are set to be the same.
  • Each reference speckle of the row establishes a second window for the center of the window (for example, the window size is also 4 ⁇ 5), and then performs correlation operations on the pixel gray values in the first window and the second window to obtain the cost .
  • the calculation process of the correlation operation includes: first inverting the pixel gray values of all pixels in the second window, and then comparing the pixel gray values of the pixels in the first window with the corresponding first window.
  • the pixel gray values of the pixels in the two windows do the "AND operation", and then count the number of overlapping speckles in the two windows to determine the number of overlapping speckles in the window.
  • the proportion of the number of all speckles within, the proportion is the matching cost.
  • the window center of the second window corresponding to the extreme value in the matching cost is used as the reference speckle matching the target speckle.
  • this step includes:
  • Step 1041b Establish a first window with the target speckle as the center of the window
  • Step 1042b Establish a second window with each reference speckle in the same row of the reference speckle image as the center of the window;
  • Step 1043b Perform similarity evaluation on each target triangle mesh in the first window and each reference triangle mesh in the second window;
  • Step 1044b Use the window center of the second window with the highest similarity evaluation as a reference speckle that matches the target speckle.
  • the method further includes:
  • Each of the reference speckle spots in the reference speckle image is used as a vertex to construct a plurality of reference triangles, and the reference triangles do not overlap each other to form a reference triangle mesh. That is, in the reference speckle image, a reference triangular grid is also constructed for each reference speckle.
  • the construction method can refer to the specific description of constructing the target triangular grid in step 103 above, which will not be repeated here.
  • each target triangle mesh in the first window and each reference triangle mesh in the second window are evaluated for similarity, and then the window center of the second window with the highest similarity evaluation is used as the A reference speckle that matches the target speckle. That is to say, in this embodiment, based on the similarity of the triangular meshes constructed in the two windows (that is, the reference triangular mesh and the target triangular mesh), the speckles in the two windows (that is, the reference speckles and the target triangular mesh) are determined. Speckle) whether it matches.
  • window matching based on topological structure can improve the speed and accuracy of matching.
  • the disparity value of the target speckle and the matched reference speckle is determined according to the reference speckle that matches the target speckle.
  • the parallax refers to the horizontal distance between two matching pixels in two images. Therefore, after finding a reference speckle that matches the target speckle on the input speckle image from the reference speckle image, it can be determined that the target speckle matches the target speckle according to the horizontal distance between the two speckles. The parallax value between the reference speckles.
  • the disparity value is interpolated based on the target topology of the target speckle to obtain the disparity map of the input speckle image.
  • the disparity map obtained only based on the disparity value of the target speckle is relatively sparse, and the disparity value needs to be interpolated in combination with the target topology of the target speckle constructed before. Obtain a denser disparity map to improve the accuracy of the depth image.
  • this step includes:
  • Step 1061 Use the center point of each target triangle as an interpolation point, and the disparity value of the interpolation point is determined based on the disparity values of the three vertices of the target triangle.
  • Step 1062 Generate a disparity map of the input speckle image based on the disparity value of each target speckle and the disparity value of each interpolation point.
  • the target topological structure is a target triangle mesh
  • the center point of each target triangle surface in the target triangle mesh (for example, the geometric center of the target triangle surface) is used as an interpolation point.
  • the disparity value of is determined after linear operation (for example, averaging the disparity values of the three vertices) according to the disparity values of the three vertices of the target triangle. In other embodiments, other methods may be used to determine the position of the interpolation point and the disparity value on the interpolation point.
  • Fig. 4 is a schematic diagram before and after interpolation of the disparity value of the target speckle of the input speckle image in the method for depth estimation based on the speckle image of the present application.
  • the disparity map is converted into a depth image of the input speckle image.
  • the depth value and the disparity value are generally inversely proportional, that is, the smaller the depth value, the greater the disparity value. It can be calculated by different conversion formulas in specific applications. In this embodiment, the specific conversion is not performed. The formula is limited, and the existing conversion formula can be used to convert the parallax value into a depth value. Therefore, the disparity map (including the disparity value of the target speckle and the disparity value of the interpolation point) obtained according to the above step 106 is converted into the depth image of the input speckle image through a conversion formula.
  • the input speckle image is obtained after preprocessing the collected initial speckle image by the image collection device.
  • the preprocessing includes processing of noise reduction and background removal on the collected initial speckle image.
  • the noise reduction process is to eliminate the influence of factors such as ambient light
  • the background removal process is to remove other images (ie, image background) other than the object to be recognized on the initial speckle image.
  • the initial speckle image collected by the image acquisition device is a collection image that includes the speckles projected by the speckle projector onto the object to be identified.
  • the initial speckle image includes not only speckle pixels, but also other image pixels, and relative to the entire initial speckle image, the proportion of speckle pixels is relatively small.
  • Fig. 5 is a schematic structural diagram of a specific embodiment of a speckle image-based depth estimation device of the present application. Referring to FIG. 5, the device 5 includes:
  • the speckle detection module 51 is used to perform speckle detection on the input speckle image to determine the target speckle; the stereo correction module 52 is used to perform stereo correction on the input speckle image and the reference speckle image, so that the The input speckle image is aligned with the reference speckle image; the topology building module 53 is used to construct the target topology structure of the target speckle in the input speckle image; the window matching module 54 is used to use a window matching algorithm On the reference speckle image, search for reference speckles that match each of the target speckles in the same row; the disparity value determination module 55 is configured to match the reference speckles with the target speckles Determine the disparity value between the target speckle and the matched reference speckle; the interpolation processing module 56 is configured to interpolate the disparity value based on the target topology structure of the target speckle to obtain the The disparity map of the input speckle image; the depth image generating module 57 is used to convert the disparity map into the depth image of the input speckle image.
  • the speckle detection module 51 is configured to detect the input speckle image based on the gray gradient of pixels to determine the preliminary speckle in the input speckle image; use a quadratic parabolic fitting algorithm to determine each The sub-pixel center point of the preliminary speckle is taken as the target speckle.
  • the speckle detection module 51 is further configured to use each pixel in the input speckle image as a central point to determine the grayscale gradient of each pixel in the first neighborhood of the central point; If the direction of the gray gradient of the pixel meets the preset gradient direction distribution, it is determined that the center point is the preliminary speckle in the input speckle image.
  • the speckle detection module 51 is also configured to: if the pixel gray value of the pixel in the first neighborhood is inversely proportional to the distance from the pixel to the center point, and the first neighborhood meets a preset The number of pixel points in the gradient direction is greater than the preset pixel number threshold, and the central point is determined to be the preliminary speckle in the input speckle image.
  • the device 5 also includes a Unicom area processing module (not shown in FIG. 5), which is used to determine if there are multiple preliminary speckles adjacent to each other in each of the determined preliminary speckles, based on the multiple position phases.
  • the adjacent preliminary speckles constitute a connected area, and only the center point of the connected area is used as the preliminary speckle.
  • the speckle detection module 51 is also configured to establish a second neighborhood based on the preliminary speckle; construct a quadratic function based on the position coordinates of each pixel in the second neighborhood and the gray value of the pixel;
  • the fitted surface is obtained when the quadratic function satisfies the constraint of the quadratic paraboloid; the pixel point corresponding to the position coordinate of the projection of the highest point of the fitted surface is taken as the sub-pixel center point of the preliminary speckle.
  • the window matching module 54 is used to establish a first window with the target speckle as the center of the window; respectively establish a second window with each reference speckle in the same row of the reference speckle image as the window center. Window; perform correlation operations on the pixel gray values in the first window and the second window to obtain the matching cost; use the window center of the second window corresponding to the extreme value in the matching cost as the The reference speckle that matches the target speckle.
  • the topological structure establishment module 53 is configured to construct a plurality of target triangles based on each target speckle in the input speckle image as a vertex, and each of the target triangles does not overlap each other to form a target triangle mesh.
  • the topological structure establishment module 53 is further configured to construct a plurality of reference triangles based on each of the reference speckle spots in the reference speckle image as vertices, and each of the reference triangles does not overlap each other to form a reference triangle grid.
  • the window matching module 54 is further configured to establish a first window with the target speckle as the center of the window; respectively establish a first window with each reference speckle in the same row of the reference speckle image as the window center. Two windows; evaluate the similarity of each target triangular mesh in the first window and each reference triangular mesh in the second window; use the window center of the second window with the highest similarity evaluation as the A reference speckle that matches the target speckle.
  • the interpolation processing module 56 is configured to use the center point of each target triangle as an interpolation point, and the disparity value of the interpolation point is determined based on the disparity values of the three vertices of the target triangle; The disparity value of the target speckle and the disparity value of each interpolation point generate a disparity map of the input speckle image.
  • Fig. 6 is a schematic structural diagram of an embodiment of a computer device of the present application.
  • the computer device may include a memory, a processor, and a computer program stored on the above-mentioned memory and running on the above-mentioned processor.
  • the above-mentioned processor executes the above-mentioned computer program, the depth estimation based on the speckle image provided in the embodiments of the present application can be realized. method.
  • the foregoing computer device may be a server, such as a cloud server, or an electronic device, such as a smart electronic device such as a smart phone, a smart watch, or a tablet computer. This embodiment does not limit the specific form of the foregoing computer device.
  • Fig. 6 shows a block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present application.
  • the computer device 12 shown in FIG. 6 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present application.
  • the computer device 12 takes the form of a general-purpose computing device.
  • the components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).
  • the bus 18 represents one or more of several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any bus structure among multiple bus structures.
  • these architectures include but are not limited to industry standard architecture (Industry Standard Architecture; hereinafter referred to as: ISA) bus, Micro Channel Architecture (hereinafter referred to as: MAC) bus, enhanced ISA bus, video electronics Standards Association (Video Electronics Standards Association; hereinafter referred to as VESA) local bus and Peripheral Component Interconnection (hereinafter referred to as PCI) bus.
  • Industry Standard Architecture hereinafter referred to as: ISA
  • MAC Micro Channel Architecture
  • VESA Video Electronics Standards Association
  • PCI Peripheral Component Interconnection
  • the computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.
  • the system memory 28 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (Random Access Memory; hereinafter referred to as RAM) 30 and/or a cache memory 32.
  • RAM Random Access Memory
  • the computer device 12 may further include other removable/non-removable, volatile/nonvolatile computer system storage media.
  • the storage system 34 may be used to read and write non-removable, non-volatile magnetic media (not shown in FIG. 6 and generally referred to as a "hard drive").
  • a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk”) and a removable non-volatile optical disk (such as a compact disk read-only memory) can be provided.
  • Disc Read Only Memory hereinafter referred to as CD-ROM
  • DVD-ROM Digital Video Disc Read Only Memory
  • each drive can be connected to the bus 18 through one or more data media interfaces.
  • the memory 28 may include at least one program product, the program product having a set (for example, at least one) program modules, and these program modules are configured to perform the functions of the embodiments of the present application.
  • a program/utility tool 40 having a set of (at least one) program module 62 may be stored in, for example, the memory 28.
  • Such program module 62 includes, but is not limited to, an operating system, one or more application programs, and other programs Modules and program data, each of these examples or some combination may include the realization of a network environment.
  • the program module 62 usually executes the functions and/or methods in the embodiments described in this application.
  • the computer device 12 can also communicate with one or more external devices 14 (such as keyboards, pointing devices, displays 24, etc.), and can also communicate with one or more devices that enable users to interact with the computer device 12, and/or communicate with Any device (such as a network card, modem, etc.) that enables the computer device 12 to communicate with one or more other computing devices. This communication can be performed through an input/output (I/O) interface 22.
  • the computer device 12 may also communicate with one or more networks (such as a local area network (Local Area Network; hereinafter referred to as: LAN), a wide area network (Wide Area Network; hereinafter referred to as: WAN) and/or a public network, such as the Internet, through the network adapter 20. ) Communication.
  • the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although not shown in FIG. 6, other hardware and/or software modules can be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape Drives and data backup storage systems, etc.
  • the processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the method for depth estimation based on speckle images provided in the embodiments of the present application.
  • FIG. 7 is a schematic structural diagram of an embodiment of the face recognition system of the present application.
  • the face recognition system 7 includes a speckle projector 71, an image acquisition device 72, and a speckle image-based depth estimation device 73 as provided in the foregoing embodiment. among them,
  • the speckle projector 71 is used to generate laser speckles and projected onto the face; the image acquisition device 72 is used to collect the optical signals formed by the laser speckles reflected on the face to obtain an initial speckle image; The depth estimation device 73 based on the speckle image is used to process the initial speckle image to obtain a depth image.
  • the face recognition system 7 can determine the distance between the face and the image acquisition device 72 according to the obtained depth image, and then recognize the face.
  • the embodiment of the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored.
  • the computer program is executed by a processor, the method for depth estimation based on the speckle image provided in the embodiment of the present application can be implemented.
  • the aforementioned non-transitory computer-readable storage medium may adopt any combination of one or more computer-readable media.
  • the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above.
  • computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (Read Only Memory) ; Hereinafter referred to as: ROM), Erasable Programmable Read Only Memory; hereinafter referred to as EPROM) or flash memory, optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic memory Pieces, or any suitable combination of the above.
  • the computer-readable storage medium can be any tangible medium that contains or stores a program, and the program can be used by or in combination with an instruction execution system, apparatus, or device.
  • the computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and computer-readable program code is carried therein. This propagated data signal can take many forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
  • the computer-readable medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
  • the program code contained on the computer-readable medium can be transmitted by any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
  • the computer program code used to perform the operations of the present application can be written in one or more programming languages or a combination thereof.
  • the programming languages include object-oriented programming languages-such as Java, Smalltalk, C++, and also conventional Procedural programming language-such as "C" language or similar programming language.
  • the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including local area network (Local Area Network; hereinafter referred to as LAN) or Wide Area Network (hereinafter referred to as WAN), or Connect to an external computer (for example, use an Internet service provider to connect via the Internet).
  • LAN Local Area Network
  • WAN Wide Area Network
  • an external computer for example, use an Internet service provider to connect via the Internet.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)

Abstract

一种基于散斑图像的深度估计方法及装置、人脸识别系统。方法包括：对输入散斑图像进行散斑点检测以确定目标散斑点(101)；将所述输入散斑图像与参考散斑图像做立体校正，以使所述待处理散斑图像与参考散斑图像行对准(102)；构建所述输入散斑图像中所述目标散斑点的目标拓扑结构(103)；利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的最匹配的参考散斑点(104)；根据与所述目标散斑点最匹配的参考散斑点确定所述目标散斑点与相匹配的参考散斑点之间的视差值(105)；基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图(106)；将所述视差图转换成所述输入散斑图像的深度图像(107)。该方法解决现有技术对散斑点检测的精度不高，导致散斑点匹配时误差较大以及散斑点的视差图效果不佳影响了深度图像的准确性的问题。

Description

基于散斑图像的深度估计方法及装置、人脸识别系统 技术领域
本申请涉及计算机视觉技术领域，尤其涉及一种基于散斑图像的深度估计方法及装置、人脸识别系统。
背景技术
从图像中恢复深度图像是计算机视觉领域的一个基础问题，随着计算机视觉技术的发展，这一问题得到越来越多的关注。
利用激光散斑的主动深度估计逐渐被研究人员所重视，由于激光模式在不同深度下基本不变，所以利用单独的相机拍摄图像，并与预先存储的参考散斑图像进行匹配从而得到视差图，进而转换成深度图像。但在现有的基于激光散斑的深度估计方法中，一方面对于采集的散斑图像中的散斑点检测的精度不高，导致后续做散斑点匹配时误差较大；另一方面，由于投射器在图像上投射的散斑点分布不均且比较分散，导致计算得到的散斑点的视差图效果不佳(较为稀疏)，影响了深度图像的准确性。
申请内容
有鉴于此，本申请实施例提供了一种基于散斑图像的深度估计方法及装置、人脸识别系统，用以解决现有技术对于采集的散斑图像中的散斑点检测的精度不高，导致后续做散斑点匹配时误差较大以及散斑点的视差图效果不佳影响了深度图像的准确性的问题。
一方面，本申请实施例提供了一种基于散斑图像的深度估计方法，包括：对输入散斑图像进行散斑点检测以确定目标散斑点；将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准；构建所述输入散斑图像中所述目标散斑点的目标拓扑结构；利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点；根据 与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点与相匹配的参考散斑点之间的视差值；基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图；将所述视差图转换成所述输入散斑图像的深度图像。
可选的，所述对输入散斑图像进行散斑点检测包括：基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点；利用二次抛物面拟合算法确定各个所述初步散斑点的亚像素中心点，以所述亚像素中心点作为目标散斑点。
可选的，在执行所述对输入散斑图像进行散斑点检测以确定目标散斑点之前还包括：对采集到的初始散斑图像进行预处理，以得到所述输入散斑图像。
可选的，所述基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点包括：分别以所述输入散斑图像中的各个像素点为中心点，确定在所述中心点的第一邻域内各个像素点的灰度梯度；若所述各个像素点的灰度梯度满足预设梯度分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。
可选的，所述若所述各个像素点的灰度梯度满足预设梯度分布，则确定所述中心点为所述输入散斑图像中的初步散斑点包括：若所述第一邻域内的像素点的像素灰度值与所述像素点距离所述中心点的距离成反比，且所述第一邻域内满足预设梯度方向的像素点的个数大于预设像素个数阈值，则确定所述中心点为所述输入散斑图像中的初步散斑点。
可选的，在执行所述基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点之后还包括：若确定的各个所述初步散斑点中存在多个位置相邻的初步散斑点，则基于所述多个相邻的初步散斑点构成联通区域，并仅将所述联通区域的中心点作为所述初步散斑点。
可选的，所述利用二次抛物面拟合算法确定各个所述初步散斑 点的亚像素中心点，以所述亚像素中心点作为目标散斑点包括：基于所述初步散斑点建立第二邻域；以所述第二邻域内的各个像素点的位置坐标以及像素点灰度值构建二次函数；在所述二次函数满足二次抛物面约束的条件下得到拟合曲面；将所述拟合曲面的最高点投影的位置坐标所对应的像素点作为所述初步散斑点的亚像素中心点。
可选的，所述利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点包括：分别以所述输入散斑图像中的各个像素点为中心点，确定在所述中心点的第一邻域内各个像素点的灰度梯度；若所述各个像素点的灰度梯度满足预设梯度分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。
可选的，所述构建所述输入散斑图像中所述目标散斑点的目标拓扑结构包括：基于所述输入散斑图像中各个所述目标散斑点作为顶点构建多个目标三角面，各个所述目标三角面相互不重叠以形成目标三角网格。
可选的，在执行所述构建所述输入散斑图像中所述目标散斑点的目标拓扑结构之后还包括：基于所述参考散斑图像中各个所述参考散斑点作为顶点构建多个参考三角面，各个所述参考三角面相互不重叠以形成参考三角网格；
所述利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点包括：以所述目标散斑点为窗口中心建立第一窗口；分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；将所述第一窗口内的各个目标三角网格和所述第二窗口内的各个参考三角网格做相似性评价；将所述相似性评价最高的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
可选的，所述基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图包括：以每个所述 目标三角面的中心点作为插值点，该插值点的视差值基于所述目标三角面的三个顶点的视差值确定；基于各个所述目标散斑点的视差值和各个插值点的视差值生成所述输入散斑图像的视差图。
另一方面，本申请实施例还提供了一种基于散斑图像的深度估计装置，包括：散斑点检测模块，用于对输入散斑图像进行散斑点检测以确定目标散斑点；立体校正模块，用于将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准；拓扑结构建立模块，用于构建所述输入散斑图像中所述目标散斑点的目标拓扑结构；窗口匹配模块，用于利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点；视差值确定模块，用于根据与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点的视差值；插值处理模块，用于基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图；深度图像生成模块，用于将所述视差图转换成所述输入散斑图像的深度图像。
再一方面，本申请实施例还提供了一种人脸识别系统，包括上述基于散斑图像的深度估计装置。
与现有技术相比，本技术方案至少具有如下有益效果：
根据本申请实施例提供的基于散斑图像的深度估计方法，对输入散斑图像进行散斑点检测以确定目标散斑点，再将所述输入散斑图像与参考散斑图像做立体校正以使两个图像行对准，并构建所述输入散斑图像中所述目标散斑点的目标拓扑结构；然后，利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点，根据各个目标散斑点相匹配的参考散斑点确定各个目标散斑点的视差值。由于散斑投射器投射的散斑通常比较分散，因此仅基于目标散斑点的视差值得到的视差图较为稀疏，需要结合之前构建的目标散斑点的目标拓扑结构对视差值进行插值，从而得到更为密集的视差图，以提高深度图像的准确性。
进一步，由于在移动终端的应用对散斑点的匹配速度和计算代 价提出了极高的要求，因此在进行窗口匹配之前，还构建参考散斑图像中所述参考散斑点的参考拓扑结构，在从参考散斑图像上寻找与输入散斑图像上每个目标散斑点处于同一行的相匹配的参考散斑点过程可以结合考虑所述参考拓扑结构和目标拓扑结构来做匹配，以提高匹配的速度和精确度。
在对输入散斑图像进行散斑点检测以确定目标散斑点的过程中，依次基于像素点的灰度梯度和二次抛物面拟合算法从所述输入散斑图像中确定目标散斑点，其中基于梯度分布要求检测出输入散斑图像中的可能散斑点(即初步散斑点)，再结合二次抛物面拟合算法确定各个初步散斑点的亚像素中心点(即进一步精确估计散斑点的位置)，并以亚像素中心点作为目标散斑点。经过上述处理方式可以提高对输入散斑图像中散斑点检测的精确度。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1是本申请的一种基于散斑图像的深度估计方法的具体实施例的流程示意图；
图2a是本申请的一种基于散斑图像的深度估计方法中检测像素点的灰度梯度分布示意图；
图2b是本申请的一种基于散斑图像的深度估计方法中所述的拟合曲面的示意图；
图3是本申请的一种基于散斑图像的深度估计方法中构建的目标拓扑结构的示意图；
图4是本申请的一种基于散斑图像的深度估计方法中对输入散斑图像的目标散斑点的视差值进行插值前后的示意图；
图5是本申请的一种基于散斑图像的深度估计装置的具体实施 例的结构示意图；
图6是本申请的一种计算机设备的结构示意图；
图7是本申请的一种人脸识别系统的实施例的结构示意图。
具体实施方式
为了更好的理解本申请的技术方案，下面结合附图对本申请实施例进行详细描述。
应当明确，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其它实施例，都属于本申请保护的范围。
图1是本申请的一种基于散斑图像的深度估计方法的具体实施例的流程示意图。参考图1，所述方法包括：
步骤101、对输入散斑图像进行散斑点检测以确定目标散斑点。
步骤102、将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准。
步骤103、构建所述输入散斑图像中所述目标散斑点的目标拓扑结构。
步骤104、利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点。
步骤105、根据与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点与相匹配的参考散斑点之间的视差值。
步骤106、基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图。
步骤107、将所述视差图转换成所述输入散斑图像的深度图像。
在本实施例中，所述参考散斑图像是预先存储的由散斑投射器 照射到平面白板上经平面白板的漫反射形成散斑，通过图像采集器采集到的散斑图像。
对物体表面的深度估计是以所述参考散斑图像上散斑点的位置分布和像素灰度值为对比标准，根据从物体表面采集的初始散斑图像并经预处理得到的输入散斑图像与所述参考散斑图像进行匹配，从而确定所述输入散斑图像中目标散斑点的视差值，并对各个目标散斑点的视差值进行插值以得到更为密集的视差图，进而将视差图转换成输入散斑图像的深度图像。其中，深度图像(depth image)也称为距离影像(range image)，是指将从图像采集器到待识别物体表面各像素点的距离(即深度)作为像素值的图像，它反映了待识别物体表面的几何形状。
具体来说，如步骤101所述，对输入散斑图像进行散斑点检测以确定目标散斑点。
在本实施例中，本步骤包括：
步骤1011、基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点。
步骤1012、利用二次抛物面拟合算法确定各个所述初步散斑点的亚像素中心点，以所述亚像素中心点作为目标散斑点。
具体地，所述步骤1011包括：
步骤10111、分别以所述输入散斑图像中的各个像素点为中心点，确定在所述中心点的第一邻域内各个像素点的灰度梯度。
步骤10112、若所述各个像素点的灰度梯度的方向满足预设梯度方向分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。
图2a是本申请的一种基于散斑图像的深度估计方法中检测像素点的灰度梯度分布示意图。
参考图2a，以所述输入散斑图像中的一个像素点为中心点(如图2a中所示)，根据该中心点的第一邻域(如图2a所示第一邻域包括5×5像素矩阵中除中心点之外的24个像素点)内每个像素点 的像素灰度值以及每个像素点与其分别在X方向上和Y方向上相邻的两个像素点的像素灰度变化率来确定各个像素点的灰度梯度。所述输入散斑图像是数字图像，本领域技术人员知晓，对于数字图像而言可以将图像看成二维离散函数，灰度梯度就是对该二维离散函数的求导得到，可以用差分来代替微分，求得所述输入散斑图像中各个像素点的灰度梯度。
在本实施例中，若所述各个像素点的灰度梯度的方向满足预设梯度方向分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。其中，所述满足预设梯度分布需要同时满足如下两个条件：
1)所述第一邻域内的像素点的像素灰度值与所述像素点距离所述中心点的距离成反比。换句话说，在所述第一邻域内，所述中心点周围的像素点距离该中心点越远，则该像素点的像素灰度值越小。
2)所述第一邻域内满足预设梯度方向的像素点的个数大于预设像素个数阈值。
具体地，一个像素点的梯度方向是根据该像素点在X方向上的灰度梯度和Y方向上的灰度梯度经过反正切运算之后得到的方向。
如图2a中各个像素点的箭头方向所示的是最理想的梯度方向分布状态，即所述第一邻域内各个像素点的灰度梯度都满足对应位置上的预设梯度方向。但在实际应用中，可能很少出现这种最理想的梯度方向分布状态，因此在本实施例中，设置一个预设像素个数阈值，当所述第一邻域内满足预设梯度方向的像素点的个数大于预设像素个数阈值，就认为所述第一邻域内的像素点的灰度梯度满足预设梯度方向。
当所述第一邻域内的像素点的灰度梯度同时满足上述两个条件，则确定所述中心点为所述输入散斑图像中的初步散斑点。
进一步，若确定的各个所述初步散斑点中存在多个位置相邻的像素点，则需要做进一步处理。
具体来说，由于散斑投射器投射的散斑点都是分散的，因此不 会出现两个或多个位置相邻的像素点都被确定为初步散斑点，当出现存在多个位置相邻的像素点，则基于所述多个位置相邻的像素点构成联通区域，并仅将所述联通区域的中心点作为所述初步散斑点。其中，对于具有规则形状的联通区域可以取几何中心点对应的像素点作为所述初步散斑点，对于不规则形状的联通区域可以联通区域的重心对应的像素点作为初步散斑点。
所述步骤1012包括：
步骤10121、基于所述初步散斑点建立第二邻域。
步骤10122、以所述第二邻域内的各个像素点的位置坐标以及像素点灰度值构建二次函数。
步骤10123、在所述二次函数满足二次抛物面约束的条件下得到拟合曲面。
步骤10124、将所述拟合曲面的最高点投影的位置坐标所对应的像素点作为所述初步散斑点的亚像素中心点。
结合参考下文表一所示的第二邻域像素矩阵以及图2b所示的本申请的一种基于散斑图像的深度估计方法中所述的拟合曲面的示意图。
具体地，如下文表一所示的是基于一个初步散斑点(灰度值为234的像素点)建立第二邻域(7×7像素矩阵，该像素矩阵中的数值表示该像素点的灰度值)。
表一
Figure PCTCN2019082373-appb-000001
以所述第二邻域内的各个像素点的位置坐标以及像素点灰度值 构建二次函数，二次抛物面拟合是将第二邻域内的这些像素点灰度值看作Z，各个像素点的位置坐标作为(X,Y)，构建二次函数z＝f(x,y)。在所述二次函数f(x,y)满足二次抛物面约束的条件下得到的拟合曲面类似于图2b所示的拟合曲面；然后，将所述拟合曲面的最高点投影的位置坐标所对应的像素点作为所述初步散斑点的亚像素中心点。
例如，若以上述表一中左上角的像素点(灰度值为89的像素点为原点)，并以该像素点所在行方向为X轴、所在列方向为Y轴建立XOY坐标系，则初步散斑点(灰度值为234的像素点)的位置坐标为(3,3)。假设拟合曲面最高点投影的位置坐标为(4.32,4.98)，则该位置坐标对应的像素点作为该初步散斑点的亚像素中心点。
可以看出，通过二次抛物面拟合算法确定各个初步散斑点对应的亚像素中心点是比原先初步散斑点的位置更为精确的像素点，因此以所述亚像素点为目标散斑点可以提高散斑点检测的精度。
如步骤102所述，将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准。
由于后续需要将输入散斑图像上的各个目标散斑点与参考散斑图像上的参考散斑点做匹配，然后基于匹配的参考散斑点计算输入散斑图像上的目标散斑点的视差值。然而在二维空间上做窗口匹配非常耗时，为了减少后续窗口匹配搜索范围，可以利用极线约束使得对应散斑点的匹配由二维搜索降为一维搜索。
因此，本步骤将所述输入散斑图像与参考散斑图像做立体校正的作用就是将两幅图像做行对准，使得两幅图像的对极线恰好在同一水平线上，这样一幅图像上任意一点与其在另一幅图像上的对应点就必然具有相同的行号，只需在该行进行一维搜索即可匹配到对应点。其中，所述参考散斑图像是对所述输入散斑图像做立体校正时的参考标准，经过立体校正后的输入散斑图像与所述参考散斑图像行对准。
但在实际应用中，将所述输入散斑图像与参考散斑图像做立体校正之后，可能会有3～5行的行误差(即没有完全行对准)，在后续窗口匹配时，可以设置较大的窗口来消除行误差。
如步骤103所述，构建所述输入散斑图像中所述目标散斑点的目标拓扑结构。
在本实施例中，构建目标散斑点的目标拓扑结构的目的是：
一方面，在后续确定了各个目标散斑点与相匹配的参考散斑点之间的视差值后，对所述视差值进行插值时是基于这个目标拓扑结构来确定各个插值点。
另一方面，构建了目标拓扑结构可以在后续做窗口匹配时，基于输入散斑图像上的目标拓扑结构与参考散斑图像上的参考拓扑结构的相似性来判断两个窗口内散斑点的匹配度。
具体地，所述目标拓扑结构可以是目标三角网格(例如Delaunay三角网格)。构建所述目标三角网格的过程是基于所述输入散斑图像中各个所述目标散斑点作为顶点构建多个目标三角面，各个所述目标三角面相互不重叠以形成目标三角网格。
图3是本申请的一种基于散斑图像的深度估计方法中构建的目标拓扑结构的示意图。参考图3，基于所述输入散斑图像中各个目标散斑点作为顶点构建多个目标三角面(如图3中所示的目标三角面)，这些目标三角面相互不重叠，从而形成目标三角网格(即图3中所有目标三角面构成的三角网格)。
如步骤104所述，利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点。
在一个实施例中，具体地，本步骤包括：
步骤1041a、以所述目标散斑点为窗口中心建立第一窗口；
步骤1042a、分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；
步骤1043a、将所述第一窗口内和所述第二窗口内的像素灰度值做相关运算以得到匹配代价；
步骤1044a、将所述匹配代价中的极值所对应的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
在将所述输入散斑图像与参考散斑图像做立体校正之后，所述输入散斑图像与参考散斑图像行对准。也就是说，在从参考散斑图像上寻找与输入散斑图像上的目标散斑点相匹配的参考散斑点的过程中，只需要在同一行上寻找即可。
具体地，首先以输入散斑图像上的一个目标散斑点为窗口中心建立第一窗口(例如窗口大小为4×5)，然后分别以所述参考散斑图像上与所述目标散斑点处于同一行的各个参考散斑点为窗口中心建立第二窗口(例如窗口大小也为4×5)，然后对所述第一窗口内和所述第二窗口内的像素灰度值做相关运算以得到代价。
其中，相关运算的计算过程包括：先将所述第二窗口内的所有像素点的像素灰度值取反，然后将所述第一窗口内的像素点的像素灰度值分别与对应的第二窗口内的像素点的像素灰度值(取反后的像素灰度值)做“与运算”，然后统计两个窗口内重合的散斑点的个数以确定重合的散斑点个数占窗口内所有散斑点个数的比重值，该比重值即为匹配代价。
然后，将所述匹配代价中的极值所对应的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
在另一个实施例中，具体地，本步骤包括：
步骤1041b、以所述目标散斑点为窗口中心建立第一窗口；
步骤1042b、分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；
步骤1043b、将所述第一窗口内的各个目标三角网格和所述第二窗口内的各个参考三角网格做相似性评价；
步骤1044b、将所述相似性评价最高的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
与上述实施例不同的是，本实施例中，在执行上述步骤103之后(即执行所述构建所述输入散斑图像中所述目标散斑点的目标拓 扑结构之后)，还包括：基于所述参考散斑图像中各个所述参考散斑点作为顶点构建多个参考三角面，各个所述参考三角面相互不重叠以形成参考三角网格。也就是说，在所述参考散斑图像中也对各个参考散斑点构建参考三角网格，其构建方式可以参考上文步骤103构建目标三角网格的具体描述，在此不再赘述。
进一步，将所述第一窗口内的各个目标三角网格和所述第二窗口内的各个参考三角网格做相似性评价，然后将所述相似性评价最高的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。也就是说，本实施例中，基于两个窗口内构建的三角网格(即参考三角网格和目标三角网格)的相似性来确定两个窗口内的散斑点(即参考散斑点和目标散斑点)是否匹配。在实际应用中，基于拓扑结构做窗口匹配可以提高匹配的速度和精确度。
如步骤105所述，根据与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点与相匹配的参考散斑点的视差值。
本领域技术人员知晓，所述视差是指两幅图像中相匹配的两个像素点之间的水平距离。因此，在从参考散斑图像上寻找到了与输入散斑图像上的目标散斑点相匹配的参考散斑点之后，根据这两个散斑点之间的水平距离可以确定所述目标散斑点与其相匹配的参考散斑点之间的视差值。
如步骤106所述，基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图。
由于散斑投射器投射的散斑通常比较分散，因此仅基于目标散斑点的视差值得到的视差图较为稀疏，需要结合之前构建的目标散斑点的目标拓扑结构对视差值进行插值，从而得到更为密集的视差图，以提高深度图像的准确性。
具体地，本步骤包括：
步骤1061、以每个所述目标三角面的中心点作为插值点，该插值点的视差值基于所述目标三角面的三个顶点的视差值确定。
步骤1062、基于各个所述目标散斑点的视差值和各个插值点 的视差值生成所述输入散斑图像的视差图。
在上述步骤1061中，所述目标拓扑结构为目标三角网格，将所述目标三角网格中的每个目标三角面的中心点(例如目标三角面的几何中心)作为插值点，该插值点的视差值是根据所述目标三角面的三个顶点的视差值经过线性运算(例如对三个顶点的视差值求平均值)后确定。在其他实施例中，也可以采用其他方式来确定插值点的位置以及插值点上的视差值。
然后，基于各个所述目标散斑点的视差值(即目标散斑点的真实视差值)和各个插值点的视差值生成所述输入散斑图像的视差图，经过插值运算后得到的视差图更为密集。如图4所示的是本申请的一种基于散斑图像的深度估计方法中对输入散斑图像的目标散斑点的视差值进行插值前后的示意图。
如步骤107所述，将所述视差图转换成所述输入散斑图像的深度图像。
本领域技术人员知晓，通常深度值与视差值是成反比关系，即深度值越小视差值越大，在具体应用中可以通过不同的转换公式来计算，本实施例中不对具体的转换公式做限定，可以采用现有的转换公式讲视差值转换成深度值。因此，根据上述步骤106得到的视差图(包括目标散斑点的视差值和插值点的视差值)通过转换公式转换成所述输入散斑图像的深度图像。
需要说明的是，本实施例中，所述输入散斑图像是通过图像采集设备对采集到的初始散斑图像进行预处理后得到的。所述预处理包括对采集到的初始散斑图像进行降噪、去背景等处理。其中，降噪处理是消除环境光照等因素的影响，去背景处理是去除所述初始散斑图像上待识别物体以外的其他图像(即图像背景)。上述降噪处理和去背景处理都可以采用现有技术来实现，在此不再赘述。
需要说明的是，通过所述图像采集设备采集到的初始散斑图像是包含了由散斑投射器投射到待识别物体上的散斑点的采集图像。也就是说，在所述初始散斑图像中不仅包括散斑像素点，还包括其 他图像像素点，且相对于整个初始散斑图像而言，其中散斑像素点所占比重较小。
基于上述方法实施例，本申请还提供了一种基于散斑图像的深度估计装置。图5是本申请的一种基于散斑图像的深度估计装置的具体实施例的结构示意图。参考图5，所述装置5包括：
散斑点检测模块51，用于对输入散斑图像进行散斑点检测以确定目标散斑点；立体校正模块52，用于将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准；拓扑结构建立模块53，用于构建所述输入散斑图像中所述目标散斑点的目标拓扑结构；窗口匹配模块54，用于利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点；视差值确定模块55，用于根据与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点与相匹配的参考散斑点之间的视差值；插值处理模块56，用于基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图；深度图像生成模块57，用于将所述视差图转换成所述输入散斑图像的深度图像。
其中，所述散斑点检测模块51用于基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点；利用二次抛物面拟合算法确定各个所述初步散斑点的亚像素中心点，以所述亚像素中心点作为目标散斑点。
所述散斑点检测模块51还用于分别以所述输入散斑图像中的各个像素点为中心点，确定在所述中心点的第一邻域内各个像素点的灰度梯度；若所述各个像素点的灰度梯度的方向满足预设梯度方向分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。
所述散斑点检测模块51还用于若所述第一邻域内的像素点的像素灰度值与所述像素点距离所述中心点的距离成反比，且所述第一邻域内满足预设梯度方向的像素点的个数大于预设像素个数阈值，则确定所述中心点为所述输入散斑图像中的初步散斑点。
所述装置5还包括联通区域处理模块(图5中未示出)，用于若确定的各个所述初步散斑点中存在多个位置相邻的初步散斑点，则基于所述多个位置相邻的初步散斑点构成联通区域，并仅将所述联通区域的中心点作为所述初步散斑点。
所述散斑点检测模块51还用于基于所述初步散斑点建立第二邻域；以所述第二邻域内的各个像素点的位置坐标以及像素点灰度值构建二次函数；在所述二次函数满足二次抛物面约束的条件下得到拟合曲面；将所述拟合曲面的最高点投影的位置坐标所对应的像素点作为所述初步散斑点的亚像素中心点。
所述窗口匹配模块54用于以所述目标散斑点为窗口中心建立第一窗口；分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；将所述第一窗口内和所述第二窗口内的像素灰度值做相关运算以得到匹配代价；将所述匹配代价中的极值所对应的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
所述拓扑结构建立模块53用于基于所述输入散斑图像中各个所述目标散斑点作为顶点构建多个目标三角面，各个所述目标三角面相互不重叠以形成目标三角网格。
所述拓扑结构建立模块53还用于基于所述参考散斑图像中各个所述参考散斑点作为顶点构建多个参考三角面，各个所述参考三角面相互不重叠以形成参考三角网格。所述窗口匹配模块54还用于以所述目标散斑点为窗口中心建立第一窗口；分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；将所述第一窗口内的各个目标三角网格和所述第二窗口内的各个参考三角网格做相似性评价；将所述相似性评价最高的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
所述插值处理模块56用于以每个所述目标三角面的中心点作为插值点，该插值点的视差值基于所述目标三角面的三个顶点的视差值确定；基于各个所述目标散斑点的视差值和各个插值点的视差 值生成所述输入散斑图像的视差图。
本实施例所述基于散斑图像的深度估计装置5中各个模块的具体处理过程可以参考上文方法实施例，在此不再赘述。
图6是本申请计算机设备一个实施例的结构示意图。
计算机设备可以包括存储器、处理器及存储在上述存储器上并可在上述处理器上运行的计算机程序，上述处理器执行上述计算机程序时，可以实现本申请实施例提供的基于散斑图像的深度估计方法。
其中，上述计算机设备可以为服务器，例如：云服务器，也可以为电子设备，例如：智能手机、智能手表或平板电脑等智能电子设备，本实施例对上述计算机设备的具体形态不作限定。
图6示出了适于用来实现本申请实施方式的示例性计算机设备12的框图。图6显示的计算机设备12仅仅是一个示例，不应对本申请实施例的功能和使用范围带来任何限制。
如图6所示，计算机设备12以通用计算设备的形式表现。计算机设备12的组件可以包括但不限于：一个或者多个处理器或者处理单元16，系统存储器28，连接不同系统组件(包括系统存储器28和处理单元16)的总线18。
总线18表示几类总线结构中的一种或多种，包括存储器总线或者存储器控制器，外围总线，图形加速端口，处理器或者使用多种总线结构中的任意总线结构的局域总线。举例来说，这些体系结构包括但不限于工业标准体系结构(Industry Standard Architecture；以下简称：ISA)总线，微通道体系结构(Micro Channel Architecture；以下简称：MAC)总线，增强型ISA总线、视频电子标准协会(Video Electronics Standards Association；以下简称：VESA)局域总线以及外围组件互连(Peripheral Component Interconnection；以下简称：PCI)总线。
计算机设备12典型地包括多种计算机系统可读介质。这些介 质可以是任何能够被计算机设备12访问的可用介质，包括易失性和非易失性介质，可移动的和不可移动的介质。
系统存储器28可以包括易失性存储器形式的计算机系统可读介质，例如随机存取存储器(Random Access Memory；以下简称：RAM)30和/或高速缓存存储器32。计算机设备12可以进一步包括其它可移动/不可移动的、易失性/非易失性计算机系统存储介质。仅作为举例，存储系统34可以用于读写不可移动的、非易失性磁介质(图6未显示，通常称为“硬盘驱动器”)。尽管图6中未示出，可以提供用于对可移动非易失性磁盘(例如“软盘”)读写的磁盘驱动器，以及对可移动非易失性光盘(例如：光盘只读存储器(Compact Disc Read Only Memory；以下简称：CD-ROM)、数字多功能只读光盘(Digital Video Disc Read Only Memory；以下简称：DVD-ROM)或者其它光介质)读写的光盘驱动器。在这些情况下，每个驱动器可以通过一个或者多个数据介质接口与总线18相连。存储器28可以包括至少一个程序产品，该程序产品具有一组(例如至少一个)程序模块，这些程序模块被配置以执行本申请各实施例的功能。
具有一组(至少一个)程序模块62的程序/实用工具40，可以存储在例如存储器28中，这样的程序模块62包括——但不限于——操作系统、一个或者多个应用程序、其它程序模块以及程序数据，这些示例中的每一个或某种组合中可能包括网络环境的实现。程序模块62通常执行本申请所描述的实施例中的功能和/或方法。
计算机设备12也可以与一个或多个外部设备14(例如键盘、指向设备、显示器24等)通信，还可与一个或者多个使得用户能与该计算机设备12交互的设备通信，和/或与使得该计算机设备12能与一个或多个其它计算设备进行通信的任何设备(例如网卡，调制解调器等等)通信。这种通信可以通过输入/输出(I/O)接口22进行。并且，计算机设备12还可以通过网络适配器20与一个或者多个网络(例如局域网(Local Area Network；以下简称：LAN)， 广域网(Wide Area Network；以下简称：WAN)和/或公共网络，例如因特网)通信。如图6所示，网络适配器20通过总线18与计算机设备12的其它模块通信。应当明白，尽管图6中未示出，可以结合计算机设备12使用其它硬件和/或软件模块，包括但不限于：微代码、设备驱动器、冗余处理单元、外部磁盘驱动阵列、RAID系统、磁带驱动器以及数据备份存储系统等。
处理单元16通过运行存储在系统存储器28中的程序，从而执行各种功能应用以及数据处理，例如实现本申请实施例提供的基于散斑图像的深度估计方法。
本申请实施例还提供了一种人脸识别系统，图7是本申请的一种人脸识别系统的实施例的结构示意图。
参考图7，所述人脸识别系统7包括散斑投射器71、图像采集设备72以及如上述实施例提供的基于散斑图像的深度估计装置73。其中，
所述散斑投射器71，用于产生激光散斑投射至人脸；所述图像采集设备72，用于采集激光散斑经所述人脸反射形成的光信号以得到初始散斑图像；所述基于散斑图像的深度估计装置73，用于对所述初始散斑图像进行处理以得到深度图像。所述人脸识别系统7根据获得的深度图像可以确定该人脸与所述图像采集设备72的之间距离的远近，进而对人脸进行识别。
本申请实施例还提供一种非临时性计算机可读存储介质，其上存储有计算机程序，上述计算机程序被处理器执行时可以实现本申请实施例提供的基于散斑图像的深度估计方法。
上述非临时性计算机可读存储介质可以采用一个或多个计算机可读的介质的任意组合。计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置 或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机存取存储器(RAM)、只读存储器(Read Only Memory；以下简称：ROM)、可擦式可编程只读存储器(Erasable Programmable Read Only Memory；以下简称：EPROM)或闪存、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本文件中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。
计算机可读的信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括——但不限于——电磁信号、光信号或上述的任意合适的组合。计算机可读的信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。
计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括——但不限于——无线、电线、光缆、RF等等，或者上述的任意合适的组合。
可以以一种或多种程序设计语言或其组合来编写用于执行本申请操作的计算机程序代码，所述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network；以下简称：LAN)或广域网(Wide Area  Network；以下简称：WAN)连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
以上所述仅为本申请的较佳实施例而已，并不用以限制本申请，凡在本申请的精神和原则之内，所做的任何修改、等同替换、改进等，均应包含在本申请保护的范围之内。

Claims (25)

  1. 一种基于散斑图像的深度估计方法，其特征在于，包括：
    对输入散斑图像进行散斑点检测以确定目标散斑点；
    将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准；
    构建所述输入散斑图像中所述目标散斑点的目标拓扑结构；
    利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点；
    根据与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点的视差值；
    基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图；
    将所述视差图转换成所述输入散斑图像的深度图像。
  2. 如权利要求1所述的方法，其特征在于，所述对输入散斑图像进行散斑点检测包括：
    基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点；
    利用二次抛物面拟合算法确定各个所述初步散斑点的亚像素中心点，以所述亚像素中心点作为目标散斑点。
  3. 如权利要求1所述的方法，其特征在于，在执行所述对输入散斑图像进行散斑点检测以确定目标散斑点之前还包括：对采集到的初始散斑图像进行预处理，以得到所述输入散斑图像。
  4. 如权利要求2所述的方法，其特征在于，所述基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点包括：
    分别以所述输入散斑图像中的各个像素点为中心点，确定在所述中心点的第一邻域内各个像素点的灰度梯度；
    若所述各个像素点的灰度梯度的方向满足预设梯度方向分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。
  5. 如权利要求4所述的方法，其特征在于，所述若所述各个像素点的灰度梯度的方向满足预设梯度方向分布，则确定所述中心点为所述输入散斑图像中的初步散斑点包括：
    若所述第一邻域内的像素点的像素灰度值与所述像素点距离所述中心点的距离成反比，且所述第一邻域内满足预设梯度方向的像素点的个数大于预设像素个数阈值，则确定所述中心点为所述输入散斑图像中的初步散斑点。
  6. 如权利要求2所述的方法，其特征在于，在执行所述基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点之后还包括：
    若确定的各个所述初步散斑点中存在多个位置相邻的像素点，则基于所述多个位置相邻的像素点构成联通区域，并仅将所述联通区域的中心点作为所述初步散斑点。
  7. 如权利要求2所述的方法，其特征在于，所述利用二次抛物面拟合算法确定各个所述初步散斑点的亚像素中心点，以所述亚像素中心点作为目标散斑点包括：
    基于所述初步散斑点建立第二邻域；
    以所述第二邻域内的各个像素点的位置坐标以及像素点灰度值构建二次函数；
    在所述二次函数满足二次抛物面约束的条件下得到拟合曲面；
    将所述拟合曲面的最高点投影的位置坐标所对应的像素点作为所述初步散斑点的亚像素中心点。
  8. 如权利要求1所述的方法，其特征在于，所述利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点包括：
    以所述目标散斑点为窗口中心建立第一窗口；
    分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；
    将所述第一窗口内和所述第二窗口内的像素灰度值做相关运算 以得到匹配代价；
    将所述匹配代价中的极值所对应的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
  9. 如权利要求1所述的方法，其特征在于，所述构建所述输入散斑图像中所述目标散斑点的目标拓扑结构包括：
    基于所述输入散斑图像中各个所述目标散斑点作为顶点构建多个目标三角面，各个所述目标三角面相互不重叠以形成目标三角网格。
  10. 如权利要求9所述的方法，其特征在于，在执行所述构建所述输入散斑图像中所述目标散斑点的目标拓扑结构之后还包括：基于所述参考散斑图像中各个所述参考散斑点作为顶点构建多个参考三角面，各个所述参考三角面相互不重叠以形成参考三角网格；
    所述利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点包括：
    以所述目标散斑点为窗口中心建立第一窗口；
    分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；
    将所述第一窗口内的各个目标三角网格和所述第二窗口内的各个参考三角网格做相似性评价；
    将所述相似性评价最高的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
  11. 如权利要求9所述的方法，其特征在于，所述基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图包括：
    以每个所述目标三角面的中心点作为插值点，该插值点的视差值基于所述目标三角面的三个顶点的视差值确定；
    基于各个所述目标散斑点的视差值和各个插值点的视差值生成所述输入散斑图像的视差图。
  12. 一种基于散斑图像的深度估计装置，其特征在于，包括：
    散斑点检测模块，用于对输入散斑图像进行散斑点检测以确定 目标散斑点；
    立体校正模块，用于将所述输入散斑图像与参考散斑图像做立体校正，以使所述输入散斑图像与参考散斑图像行对准；
    拓扑结构建立模块，用于构建所述输入散斑图像中所述目标散斑点的目标拓扑结构；
    窗口匹配模块，用于利用窗口匹配算法在所述参考散斑图像上寻找与每个所述目标散斑点处于同一行的相匹配的参考散斑点；
    视差值确定模块，用于根据与所述目标散斑点相匹配的参考散斑点确定所述目标散斑点与相匹配的参考散斑点的视差值；
    插值处理模块，用于基于所述目标散斑点的目标拓扑结构对所述视差值进行插值，以得到所述输入散斑图像的视差图；
    深度图像生成模块，用于将所述视差图转换成所述输入散斑图像的深度图像。
  13. 如权利要求12所述的装置，其特征在于，所述散斑点检测模块用于基于像素点的灰度梯度对所述输入散斑图像进行检测以确定所述输入散斑图像中的初步散斑点；利用二次抛物面拟合算法确定各个所述初步散斑点的亚像素中心点，以所述亚像素中心点作为目标散斑点。
  14. 如权利要求12所述的装置，其特征在于，还包括预处理模块，用于对采集到的初始散斑图像进行预处理，以得到所述输入散斑图像。
  15. 如权利要求13所述的装置，其特征在于，所述散斑点检测模块还用于分别以所述输入散斑图像中的各个像素点为中心点，确定在所述中心点的第一邻域内各个像素点的灰度梯度；若所述各个像素点的灰度梯度的方向满足预设梯度方向分布，则确定所述中心点为所述输入散斑图像中的初步散斑点。
  16. 如权利要求15所述的装置，其特征在于，所述散斑点检测模块，还用于若所述第一邻域内的像素点的像素灰度值与所述像素点距离所述中心点的距离成反比，且所述第一邻域内满足预设梯度方 向的像素点的个数大于预设像素个数阈值，则确定所述中心点为所述输入散斑图像中的初步散斑点。
  17. 如权利要求13所述的装置，其特征在于，还包括联通区域处理模块，用于若确定的各个所述初步散斑点中存在多个位置相邻的像素点，则基于所述多个位置相邻的像素点构成联通区域，并仅将所述联通区域的中心点作为所述初步散斑点。
  18. 如权利要求13所述的装置，其特征在于，所述散斑点检测模块，还用于基于所述初步散斑点建立第二邻域；以所述第二邻域内的各个像素点的位置坐标以及像素点灰度值构建二次函数；在所述二次函数满足二次抛物面约束的条件下得到拟合曲面；将所述拟合曲面的最高点投影的位置坐标所对应的像素点作为所述初步散斑点的亚像素中心点。
  19. 如权利要求12所述的装置，其特征在于，所述窗口匹配模块用于以所述目标散斑点为窗口中心建立第一窗口；分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；将所述第一窗口内和所述第二窗口内的像素灰度值做相关运算以得到匹配代价；将所述匹配代价中的极值所对应的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
  20. 如权利要求12所述的装置，其特征在于，所述拓扑结构建立模块，用于基于所述输入散斑图像中各个所述目标散斑点作为顶点构建多个目标三角面，各个所述目标三角面相互不重叠以形成目标三角网格。
  21. 如权利要求20所述的装置，其特征在于，所述拓扑结构建立模块，还用于基于所述参考散斑图像中各个所述参考散斑点作为顶点构建多个参考三角面，各个所述参考三角面相互不重叠以形成参考三角网格；
    所述窗口匹配模块，还用于以所述目标散斑点为窗口中心建立第一窗口；分别以所述参考散斑图像上与所述目标散斑点同一行的各个参考散斑点为窗口中心建立第二窗口；将所述第一窗口内的各 个目标三角网格和所述第二窗口内的各个参考三角网格做相似性评价；将所述相似性评价最高的第二窗口的窗口中心作为与所述目标散斑点相匹配的参考散斑点。
  22. 如权利要求20所述的装置，其特征在于，所述插值处理模块用于以每个所述目标三角面的中心点作为插值点，该插值点的视差值基于所述目标三角面的三个顶点的视差值确定；基于各个所述目标散斑点的视差值和各个插值点的视差值生成所述输入散斑图像的视差图。
  23. 一种计算机设备，其特征在于，包括存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时，实现如权利要求1-11中任一所述的方法。
  24. 一种非临时性计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1-11中任一所述的方法。
  25. 一种人脸识别系统，其特征在于，包括：散斑投射器、图像采集设备以及如权利要求12至22中任一项所述的基于散斑图像的深度估计装置；其中，
    所述散斑投射器，用于产生激光散斑投射至人脸；
    所述图像采集设备，用于采集所述激光散斑经所述人脸反射形成的光信号以得到初始散斑图像；
    所述基于散斑图像的深度估计装置，用于对所述初始散斑图像进行处理以得到深度图像。
PCT/CN2019/082373 2019-04-12 2019-04-12 基于散斑图像的深度估计方法及装置、人脸识别系统 WO2020206666A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2019/082373 WO2020206666A1 (zh) 2019-04-12 2019-04-12 基于散斑图像的深度估计方法及装置、人脸识别系统
CN201980000582.4A CN112771573B (zh) 2019-04-12 2019-04-12 基于散斑图像的深度估计方法及装置、人脸识别系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/082373 WO2020206666A1 (zh) 2019-04-12 2019-04-12 基于散斑图像的深度估计方法及装置、人脸识别系统

Publications (1)

Publication Number Publication Date
WO2020206666A1 true WO2020206666A1 (zh) 2020-10-15

Family

ID=72750798

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/082373 WO2020206666A1 (zh) 2019-04-12 2019-04-12 基于散斑图像的深度估计方法及装置、人脸识别系统

Country Status (2)

Country Link
CN (1) CN112771573B (zh)
WO (1) WO2020206666A1 (zh)

Cited By (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112598717A (zh) * 2020-12-14 2021-04-02 珠海欧比特宇航科技股份有限公司 高光谱卫星影像全谱段配准方法及介质
CN112669362A (zh) * 2021-01-12 2021-04-16 四川深瑞视科技有限公司 基于散斑的深度信息获取方法、装置及系统
CN112861764A (zh) * 2021-02-25 2021-05-28 广州图语信息科技有限公司 一种人脸识别活体判断方法
CN113011238A (zh) * 2020-11-24 2021-06-22 腾讯科技（深圳）有限公司 数据处理方法、装置、服务器、终端和存储介质
CN113298785A (zh) * 2021-05-25 2021-08-24 Oppo广东移动通信有限公司 校正方法、电子设备及计算机可读存储介质
CN113409404A (zh) * 2021-06-29 2021-09-17 常熟理工学院 基于新型相关函数约束的cuda架构并行优化立体变形测量方法
CN113936050A (zh) * 2021-10-21 2022-01-14 北京的卢深视科技有限公司 散斑图像生成方法、电子设备及存储介质
CN114283089A (zh) * 2021-12-24 2022-04-05 北京的卢深视科技有限公司 基于跳跃加速的深度恢复方法、电子设备及存储介质
CN114299129A (zh) * 2021-12-31 2022-04-08 北京的卢深视科技有限公司 深度恢复方法、电子设备和计算机可读存储介质
CN115861308A (zh) * 2023-02-22 2023-03-28 山东省林草种质资源中心(山东省药乡林场) 一种元宝枫病害检测方法
CN116067305A (zh) * 2023-02-09 2023-05-05 深圳市安思疆科技有限公司 一种结构光测量系统和测量方法
CN116823809A (zh) * 2023-08-23 2023-09-29 威海迈尼生物科技有限公司 一种微针贴片技术淡斑效果视觉检测方法
CN117409174A (zh) * 2023-12-14 2024-01-16 南昌虚拟现实研究院股份有限公司 散斑图像温度补偿方法、装置、可读介质及电子设备

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113658241B (zh) * 2021-08-16 2022-12-16 合肥的卢深视科技有限公司 单目结构光的深度恢复方法、电子设备及存储介质
CN113888614B (zh) * 2021-09-23 2022-05-31 合肥的卢深视科技有限公司 深度恢复方法、电子设备和计算机可读存储介质
CN113936049A (zh) * 2021-10-21 2022-01-14 北京的卢深视科技有限公司 单目结构光散斑图像深度恢复方法、电子设备及存储介质
CN114387324A (zh) * 2021-12-22 2022-04-22 北京的卢深视科技有限公司 深度成像方法、装置、电子设备和计算机可读存储介质
CN114332014B (zh) * 2021-12-29 2024-05-24 合肥瑞识智能科技有限公司 一种投射器质量评测方法、装置、设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103279982A (zh) * 2013-05-24 2013-09-04 中国科学院自动化研究所 鲁棒的快速高深度分辨率的散斑三维重建方法
US9906793B2 (en) * 2012-08-24 2018-02-27 Microsoft Technology Licensing, Llc Depth data processing and compression
US9959455B2 (en) * 2016-06-30 2018-05-01 The United States Of America As Represented By The Secretary Of The Army System and method for face recognition using three dimensions
CN108734776A (zh) * 2018-05-23 2018-11-02 四川川大智胜软件股份有限公司 一种基于散斑的三维人脸重建方法及设备
CN109461181A (zh) * 2018-10-17 2019-03-12 北京华捷艾米科技有限公司 基于散斑结构光的深度图像获取方法及系统

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103268608B (zh) * 2013-05-17 2015-12-02 清华大学 基于近红外激光散斑的深度估计方法及装置
US20160245641A1 (en) * 2015-02-19 2016-08-25 Microsoft Technology Licensing, Llc Projection transformations for depth estimation
CN105160680B (zh) * 2015-09-08 2017-11-21 北京航空航天大学 一种基于结构光的无干扰深度相机的设计方法
CN106954058B (zh) * 2017-03-09 2019-05-10 深圳奥比中光科技有限公司 深度图像获取系统和方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9906793B2 (en) * 2012-08-24 2018-02-27 Microsoft Technology Licensing, Llc Depth data processing and compression
CN103279982A (zh) * 2013-05-24 2013-09-04 中国科学院自动化研究所 鲁棒的快速高深度分辨率的散斑三维重建方法
US9959455B2 (en) * 2016-06-30 2018-05-01 The United States Of America As Represented By The Secretary Of The Army System and method for face recognition using three dimensions
CN108734776A (zh) * 2018-05-23 2018-11-02 四川川大智胜软件股份有限公司 一种基于散斑的三维人脸重建方法及设备
CN109461181A (zh) * 2018-10-17 2019-03-12 北京华捷艾米科技有限公司 基于散斑结构光的深度图像获取方法及系统

Cited By (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113011238A (zh) * 2020-11-24 2021-06-22 腾讯科技（深圳）有限公司 数据处理方法、装置、服务器、终端和存储介质
CN112598717A (zh) * 2020-12-14 2021-04-02 珠海欧比特宇航科技股份有限公司 高光谱卫星影像全谱段配准方法及介质
CN112598717B (zh) * 2020-12-14 2024-05-17 珠海欧比特卫星大数据有限公司 高光谱卫星影像全谱段配准方法及介质
CN112669362A (zh) * 2021-01-12 2021-04-16 四川深瑞视科技有限公司 基于散斑的深度信息获取方法、装置及系统
CN112669362B (zh) * 2021-01-12 2024-03-29 四川深瑞视科技有限公司 基于散斑的深度信息获取方法、装置及系统
CN112861764B (zh) * 2021-02-25 2023-12-08 广州图语信息科技有限公司 一种人脸识别活体判断方法
CN112861764A (zh) * 2021-02-25 2021-05-28 广州图语信息科技有限公司 一种人脸识别活体判断方法
CN113298785A (zh) * 2021-05-25 2021-08-24 Oppo广东移动通信有限公司 校正方法、电子设备及计算机可读存储介质
CN113409404A (zh) * 2021-06-29 2021-09-17 常熟理工学院 基于新型相关函数约束的cuda架构并行优化立体变形测量方法
CN113409404B (zh) * 2021-06-29 2023-06-16 常熟理工学院 基于新型相关函数约束的cuda架构并行优化立体变形测量方法
CN113936050A (zh) * 2021-10-21 2022-01-14 北京的卢深视科技有限公司 散斑图像生成方法、电子设备及存储介质
CN114283089A (zh) * 2021-12-24 2022-04-05 北京的卢深视科技有限公司 基于跳跃加速的深度恢复方法、电子设备及存储介质
CN114299129B (zh) * 2021-12-31 2023-01-31 合肥的卢深视科技有限公司 深度恢复方法、电子设备和计算机可读存储介质
CN114299129A (zh) * 2021-12-31 2022-04-08 北京的卢深视科技有限公司 深度恢复方法、电子设备和计算机可读存储介质
CN116067305A (zh) * 2023-02-09 2023-05-05 深圳市安思疆科技有限公司 一种结构光测量系统和测量方法
CN115861308A (zh) * 2023-02-22 2023-03-28 山东省林草种质资源中心(山东省药乡林场) 一种元宝枫病害检测方法
CN116823809A (zh) * 2023-08-23 2023-09-29 威海迈尼生物科技有限公司 一种微针贴片技术淡斑效果视觉检测方法
CN116823809B (zh) * 2023-08-23 2023-11-24 威海迈尼生物科技有限公司 一种微针贴片技术淡斑效果视觉检测方法
CN117409174A (zh) * 2023-12-14 2024-01-16 南昌虚拟现实研究院股份有限公司 散斑图像温度补偿方法、装置、可读介质及电子设备
CN117409174B (zh) * 2023-12-14 2024-03-15 南昌虚拟现实研究院股份有限公司 散斑图像温度补偿方法、装置、可读介质及电子设备

Also Published As

Publication number Publication date
CN112771573A (zh) 2021-05-07
CN112771573B (zh) 2023-01-20

Similar Documents

Publication Publication Date Title
WO2020206666A1 (zh) 基于散斑图像的深度估计方法及装置、人脸识别系统
US10810734B2 (en) Computer aided rebar measurement and inspection system
US10930005B1 (en) Profile matching of buildings and urban structures
WO2019170164A1 (zh) 基于深度相机的三维重建方法、装置、设备及存储介质
WO2020228643A1 (zh) 交互控制方法、装置、电子设备及存储介质
US10455219B2 (en) Stereo correspondence and depth sensors
WO2018205803A1 (zh) 位姿估计方法和装置
US20140153816A1 (en) Depth Map Stereo Correspondence Techniques
CN113256712A (zh) 定位方法、装置、电子设备和存储介质
US8565557B2 (en) Free view generation in ray-space
CN110288710B (zh) 一种三维地图的处理方法、处理装置及终端设备
CN113129352B (zh) 一种稀疏光场重建方法及装置
CN113793370B (zh) 三维点云配准方法、装置、电子设备及可读介质
WO2023227011A1 (zh) 一种线性轮廓仪外参的标定方法、装置及电子设备
US20230154027A1 (en) Spatial construction using guided surface detection
WO2023155387A1 (zh) 多传感器目标检测方法、装置、电子设备以及存储介质
CN115409808A (zh) 焊缝识别方法、装置、焊接机器人及存储介质
WO2021169704A1 (zh) 手势深度确定方法、装置、设备及存储介质
US20180114339A1 (en) Information processing device and method, and program
CN112509135B (zh) 元素标注方法、装置、设备、存储介质及计算机程序产品
US11315265B2 (en) Fingertip detection method, fingertip detection device, and medium
US11783501B2 (en) Method and apparatus for determining image depth information, electronic device, and media
CN115031635A (zh) 测量方法及装置、电子设备及存储介质
CN115346020A (zh) 点云处理方法、避障方法、装置、机器人和存储介质
CN115375740A (zh) 位姿确定方法和三维模型的生成方法、装置、设备、介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19923835

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19923835

Country of ref document: EP

Kind code of ref document: A1