CN115457615A

CN115457615A - Method and device for correcting depth map, computer equipment and storage medium

Info

Publication number: CN115457615A
Application number: CN202110649351.5A
Authority: CN
Inventors: 洪哲鸣; 张晓翼; 赵伟; 王少鸣; 张勇; 郭润增
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2021-06-08
Filing date: 2021-06-08
Publication date: 2022-12-09

Abstract

The present application relates to computer vision technologies, and in particular, to a method and an apparatus for correcting a depth map, a computer device, and a storage medium. The method comprises the following steps: acquiring a depth map obtained based on structured light projected by a projector, and a projection temperature of the projector when the structured light is projected; determining a center wavelength of the structured light projected by the projector based on the projection temperature; for each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained; correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor; and the corrected depth map is used for carrying out living body detection. The method can shorten the time of in vivo detection.

Description

Method and device for correcting depth map, computer equipment and storage medium

Technical Field

The present application relates to the field of computer vision technologies, and in particular, to a method and an apparatus for correcting a depth map, a computer device, and a storage medium.

Background

With the continuous development of intelligent devices and image processing technologies, more and more users execute services related to personal sensitive information through the intelligent devices, and because of the related personal sensitive information, before face recognition is performed on a user initiating the services, living body detection is usually performed on the user, so that the user is prevented from using images of other people obtained in advance to perform face recognition.

Because the depth map contains depth information, when living body detection is carried out, the depth map can be collected for living body detection. In the conventional in-vivo detection scheme, generally, when the temperature of the projector reaches a steady state, a depth map is collected by structured light projected by the projector after stabilization, and then in-vivo detection is performed based on the collected depth map. However, since it takes a certain time for the temperature of the projector to reach the steady state, the time for the living body detection is prolonged.

Disclosure of Invention

In view of the above, it is necessary to provide a method and an apparatus for correcting a depth map, a computer device, and a storage medium, which can shorten the time for living body detection.

A method of correcting a depth map, the method comprising:

acquiring a depth map based on the structured light projected by a projector, and a projection temperature of the projector when projecting the structured light;

determining a center wavelength of the structured light projected by the projector based on the projection temperature;

for each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of a pixel point adjacent to the current pixel point and the center wavelength until the depth correction factor of each pixel point in the depth map is obtained;

correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor; and the corrected depth map is used for carrying out living body detection.

An apparatus for correcting a depth map, the apparatus comprising:

the device comprises an acquisition module, a processing module and a control module, wherein the acquisition module is used for acquiring a depth map obtained based on structured light projected by a projector and a projection temperature of the projector when the structured light is projected;

a first determination module for determining a center wavelength of the structured light projected by the projector based on the projection temperature;

a second determining module, configured to determine, for each pixel point in the depth map, a depth correction factor of the current pixel point based on a pixel value of the current pixel point in the depth map, a pixel value of a pixel point adjacent to the current pixel point, and the center wavelength until the depth correction factor of each pixel point in the depth map is obtained;

the correction module is used for correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor; and the corrected depth map is used for carrying out living body detection.

In one embodiment, the acquiring module is further configured to collect speckle structural light reflected from the projection of the structured light onto the target object by the projector; generating a speckle structure light map based on the speckle structure light; performing speckle matching calculation on the speckle structure light pattern and a reference structure light pattern of the target object to obtain a parallax between the speckle structure light pattern and the reference structure light pattern; determining distances between the projector and points in the target object based on the disparity; generating a depth map for the target object according to the distance.

In one embodiment, the disparity is a disparity between each target speckle point in the speckle structural light map and a matching speckle point in the reference structural light map; the distances include at least two target distances; at least two of the target distances are determined based on a disparity between at least two adjacent of the target speckle points and the matched speckle point;

the acquisition module is further configured to determine a distance difference value of each target distance for each target scattered spot and a target distance corresponding to each matched scattered spot; adjusting the parallax between each target speckle point and each matched speckle point according to the distance difference; determining a distance between the projector and a point in the target object that reflects the structured light as a function of the adjusted parallax.

In one embodiment, the obtaining module is further configured to determine image depth information according to the distance; rendering an image based on the image depth information to obtain an initial depth map; and denoising and filtering the initial depth map to obtain the depth map of the target object.

In one embodiment, the first determining module is further configured to obtain a conversion relationship between the projection temperature and the wavelength; and determining the central wavelength of the structured light projected by the projector according to the conversion relation and the projection temperature.

In one embodiment thereof, the apparatus further comprises:

the first detection module is used for detecting the wavelength of the projection structured light of the projector at different calibration temperatures;

the fitting module is used for fitting the wavelengths of the structured light at different calibration temperatures to obtain a wavelength change curve; and taking the wavelength change curve as a conversion relation between the projection temperature and the wavelength.

In one embodiment, the fitting module is further configured to use different calibration temperatures and corresponding wavelengths as data nodes, respectively; determining a temperature step between adjacent data nodes; determining a wavelength adjustment parameter based on different data nodes, the temperature step length and the node coefficient; and respectively constructing wavelength change curves of different data nodes according to the wavelength adjusting parameters and the calibration temperatures in different data nodes.

In one embodiment, the second determining module is further configured to perform power operation on a difference between a pixel value of a current pixel point in the depth map and a pixel value of a pixel point adjacent to the current pixel point to obtain at least two pixel power components; determining correction parameters corresponding to at least two of the pixel power components based on the center wavelength and a preset conversion coefficient; and determining a depth correction factor of the current pixel point according to the correction parameter, at least two pixel power components and a reference parameter.

In one embodiment thereof, the depth map is an image of the projector projecting structured light onto a target object; the device further comprises:

the second detection module is used for carrying out living body detection on the target object based on the corrected depth map to obtain a first detection result; acquiring an infrared image corresponding to the corrected depth map, and performing living body detection on the target object based on the infrared image to obtain a second detection result; determining that the target object is a living body based on the first detection result and the second detection result.

In one embodiment thereof, the apparatus further comprises:

a processing module for acquiring a color image corresponding to the corrected depth map when the target object is a living body; extracting image features of the target object from the color image; if the image characteristics are matched with the image characteristics stored in a characteristic library and the resource transfer operation is detected, transferring the resources with the specified limit from the resource account of the target object; or transferring the resource with the specified quota to the resource account of the target object.

A computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:

for each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained;

A computer-readable storage medium, on which a computer program is stored which, when executed by a processor, carries out the steps of:

acquiring a depth map obtained based on structured light projected by a projector, and a projection temperature of the projector when the structured light is projected;

According to the correction method, the device, the computer equipment and the storage medium of the depth map, the central wavelength of the structured light is determined based on the projection temperature of the projector when the structured light projects, then the depth correction factor of the current pixel point is calculated according to the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength, further the depth correction factor of each pixel point in the depth map is obtained, then the pixel value of each pixel point in the depth map is corrected according to the obtained depth correction factor, so that the temperature of the projector does not need to reach a stable state, only the depth maps of the previous frames need to be corrected, the in-vivo detection can be carried out by using the corrected depth map, the waiting time in the in-vivo detection process is avoided, the in-vivo detection time is shortened, and the in-vivo detection efficiency is improved.

Drawings

FIG. 1 is a diagram of an application environment of a method for correcting a depth map according to an embodiment;

FIG. 2 is a flowchart illustrating a method for correcting a depth map according to an embodiment;

FIG. 3 is a schematic diagram of obtaining a depth map from projected structured light in one embodiment;

FIG. 4 is a schematic illustration of wavelength variation with temperature variation in one embodiment;

FIG. 5 is a schematic diagram of coordinate transformation between different cameras in one embodiment;

FIG. 6 is a diagram illustrating facial feature points according to one embodiment;

FIG. 7 is a diagram illustrating an exemplary depth map correction scheme;

FIG. 8 is a flowchart illustrating a depth map correction method applied to an account registration scenario in one embodiment;

FIG. 9 is a schematic diagram of a page for account registration in one embodiment;

FIG. 10 is a flow diagram that illustrates a method for depth map correction applied to a pay scene, in accordance with an embodiment;

FIG. 11 is a schematic diagram of a payment related page in one embodiment;

FIG. 12 is a schematic diagram of an embodiment of a door entry scenario;

FIG. 13 is a block diagram showing the structure of a device for correcting a depth map in one embodiment;

FIG. 14 is a block diagram showing the structure of a depth map correction apparatus according to another embodiment;

FIG. 15 is a diagram showing an internal structure of a computer device in one embodiment;

fig. 16 is an internal structural view of a computer device in another embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

An Artificial Intelligence (AI) technology is a comprehensive subject, and relates to the field of extensive technologies, namely a hardware technology and a software technology. The artificial intelligence infrastructure generally includes technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and the like.

The scheme provided by the embodiment of the application relates to the technologies of artificial intelligence, such as computer vision, machine learning and the like, and is specifically explained by the following embodiments:

the method for correcting the depth map can be applied to the application environment shown in fig. 1. In the application environment, a terminal 102 and a server 104 are included. The method for correcting the depth map may be executed by the terminal 102, by the server 104, or by the terminal 102 and the server 104 in a coordinated manner.

Taking the case that the correction method of the depth map is executed by the terminal 102, the terminal 102 initiates living body detection, acquires a depth map based on the structured light projected by the projector, and a projection temperature of the projector when projecting the structured light; determining a center wavelength of the structured light projected by the projector based on the projection temperature; aiming at each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained; and correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor, and then carrying out living body detection based on the corrected depth map. After the detection result is obtained, if the target object is a living body, a color image is obtained, face recognition is performed on the target object based on the color image, and when the target object is determined to be the user owner, resource transfer is initiated to the server 104, for example, the server 104 is requested to transfer the resource to a resource account of the target object.

As an example, the correction method of the depth map is executed by the server 104, and the server 104 acquires a depth map obtained based on the structured light projected by the projector in the terminal 102, and a projection temperature of the projector when projecting the structured light; determining a center wavelength of the structured light projected by the projector based on the projection temperature; aiming at each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained; and correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor, then carrying out living body detection based on the corrected depth map, and then sending the detection result to the terminal 102 so that the terminal 102 carries out corresponding operation according to the detection result.

The terminal 102 may be, but is not limited to, a camera, a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart watch, and the like. It is noted that the terminal 102 integrates a projector (e.g., a laser) that can project structured light onto the surface of the target object and a sensor that images based on the reflected structured light and then recovers 3D coordinate information of the surface of the target object to obtain a depth map.

The server 104 may be an independent physical server or a service node in a block chain system, where the service nodes in the block chain system form a Peer-To-Peer (P2P) network, and the P2P Protocol is an application layer Protocol running on a Transmission Control Protocol (TCP).

In addition, the server 104 may also be a server cluster composed of a plurality of physical servers, and may be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, web services, cloud communication, middleware services, domain name services, security services, content Delivery Networks (CDNs), and big data and artificial intelligence platforms.

The terminal 102 and the server 104 may be connected through communication connection manners such as bluetooth, USB (Universal Serial Bus), or network, which is not limited herein.

In one embodiment, as shown in fig. 2, a method for correcting a depth map is provided, which is described by taking the method as an example applied to the terminal 102 in fig. 1, and includes the following steps:

s202, acquiring a depth map obtained based on the structured light projected by the projector and the projection temperature of the projector when the structured light is projected.

The projector may be a laser integrated in the terminal for projecting structured light, such as an infrared light projector and a laser projector (i.e. a laser). The structured light may refer to lattice light arranged according to a certain structural rule, such as structured laser (e.g., lattice laser) and lattice infrared light. The projection temperature may refer to the temperature of the projector when projecting the structured light.

The depth map may be an image representing the distance between the projector and the target object. For example, the pixel value of each pixel point in the depth map may be used as image depth information to indicate the vertical distance between the projector and each point on the surface of the target object.

In one embodiment, when the terminal receives a living body detection request or a resource transfer request, the terminal projects structured light to the target object through the projector, then collects the reflected structured light through a sensor in the terminal to obtain speckle structured light, and then analyzes the speckle structured light to obtain a depth map, as shown in fig. 3.

The step of analyzing the speckle structure light to obtain the depth map may specifically include: the terminal generates a speckle structure light pattern according to the speckle structure light; and then carrying out speckle matching calculation on the speckle structure light image and the reference structure light image of the target object to obtain the parallax between the speckle structure light image and the reference structure light image, and generating a depth map aiming at the target object based on the parallax.

For the speckle matching calculation, the method specifically includes: the terminal extracts texture features from the speckle structure light pattern and the reference structure light pattern respectively to obtain a first texture feature and a second texture feature in sequence, then carries out matching based on the first texture feature pattern and the second texture feature pattern in the speckle structure light pattern and the reference structure light pattern, and then calculates parallax according to the matched texture features.

For the generation of the depth map, the method may specifically include: the terminal acquires camera internal parameters configured in the terminal after acquiring the parallax between the speckle structure light pattern and the reference structure light pattern, determines the distance between the projector and each point in the target object according to the parallax and the camera internal parameters, and then generates a depth map for the target object according to the distance.

The terminal can determine image depth information according to the distance, and then perform image rendering based on the image depth information to obtain a depth map. In addition, after the terminal performs image rendering, denoising and filtering processing may be performed on the obtained depth map (the depth map at this time may be referred to as an initial depth map), so as to obtain a depth map of the target object. The image depth information may be pixel values of a depth map.

For the acquisition of the projection temperature, the temperature of the projector projecting the structural light is collected every time the projector projects the structural light, so that the projection temperature is obtained.

And S204, determining the central wavelength of the structured light projected by the projector based on the projection temperature.

The central wavelength may refer to a central point of a band in which energy of the structured light is concentrated. For example, a laser generally has a line width of ten and several nanometers, and if the energy of the laser is mainly concentrated in the range of 795 to 805nm, the center wavelength of the laser is 800nm.

In one embodiment, the terminal acquires a conversion relationship between the projection temperature and the wavelength; and determining the central wavelength of the structured light projected by the projector according to the conversion relation and the projection temperature. Wherein the conversion relationship may be a curve fitting the wavelengths at different temperatures.

For the obtaining of the conversion relation, the specific process comprises the following steps: the terminal detects the wavelength of the structural light projected by the projector at different calibration temperatures; fitting the wavelengths at different calibration temperatures to obtain a wavelength change curve; the wavelength variation curve is used as the conversion relation between the projection temperature and the wavelength.

Wherein, the calibration temperature may refer to a projection temperature recorded by the projector when projecting the structured light, and the calibration temperature may be 0 to 60 degrees, as shown in fig. 4.

In one embodiment, the terminal detects the wavelength at which the projector projects structured light at different calibration temperatures; fitting the wavelengths at different calibration temperatures to obtain a wavelength change curve; the wavelength variation curve is taken as the conversion relation between the projection temperature and the wavelength.

When the projector projects the structured light at 0-60 degrees, the terminal can measure the wavelengths of the structured light projected by the projector through the integrated spectrometer, and then fit the wavelengths, so that the trend of the wavelength changing along with the calibrated temperature can be recorded, as shown in fig. 4. In the fitting, the wavelength may be fitted by means of spline fitting. It is noted that fitting the wavelength refers to fitting the wavelength values at different temperatures.

In an embodiment, the step of fitting the wavelengths of the structured light at different calibration temperatures to obtain a wavelength variation curve may specifically include: the terminal respectively takes different calibration temperatures and corresponding wavelengths as data nodes; determining a temperature step between adjacent data nodes; determining a wavelength adjustment parameter based on different data nodes, temperature step lengths and node coefficients; and respectively constructing wavelength change curves of different data nodes according to the wavelength adjusting parameters and the calibration temperatures in different data nodes.

Wherein the node coefficient may be a quadratic differential value. As shown in FIG. 4, assuming temperature and wavelength as data nodes, there are n +1 data nodes (x) ₀ ,y ₀ )，(x ₁ ,y ₁ )，(x ₂ ,y ₂ )，…，(x _n ,y _n ) Inputting each data node into matrix equation y = a _i +b _i x+c _i x ² +d _i x ³ In (3), solving the matrix equation so as to obtain a quadratic differential value m _i 。

And S206, aiming at each pixel point in the depth map, determining the depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained.

Wherein, current pixel can refer to: and in the process of calculating the depth correction factor of each pixel point in the depth map, calculating the current pixel point. For example, assuming that the depth map has m × n (if m × n = P) pixels, when the depth correction factor calculation is performed on the i × i (if i × i = K) pixels, the K-th pixel is the current pixel. Correspondingly, the adjacent pixel points may be the pixel points near the Kth pixel point, such as the (i-1) xi, (i + 1) xi, ixx (i-1) and ixx (i + 1) pixel points, or other adjacent pixel points.

In one embodiment, the terminal inputs the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the center wavelength into the depth correction function, and calculates the depth correction factor of the current pixel point according to the depth correction function. Wherein the depth correction function is as follows:

S(x _i )＝a+b(x _i -t)+c(x _i -t) ² +d(x _i -t) ³

wherein x is _i Representing a current pixel point i in the depth map, t representing a pixel point (which may be an adjacent pixel point or a plurality of adjacent pixel points) adjacent to the current pixel point, S _i (x) And the depth correction factor of the current pixel point i in the depth map is represented, a represents a reference parameter, and b, c and d are correction parameters of different power components respectively. Further, b =0.3 (λ -940), c =0.13 (λ -940), and d =0.024 (λ -940).

Specifically, the terminal performs power operation on a difference value between a pixel value of a current pixel point in the depth map and a pixel value of a pixel point adjacent to the current pixel point to obtain at least two pixel power components; determining correction parameters corresponding to at least two pixel power components based on the central wavelength and a preset conversion coefficient; and determining the depth correction factor of the current pixel point according to the correction parameter, the at least two pixel power components and the reference parameter.

Wherein, the pixel power component may refer to: the resulting value obtained by performing different powers on the difference between the pixel values of the pixels adjacent to the pixel value of the current pixel may include a first-order pixel power component such as (x) _i -t), second order pixel power components such as (x) _i -t) ² And a third-order pixel power component such as (x) _i -t) ³ And the like.

For example, assuming that the correction parameters are b, c, and d, respectively, b =0.3 (λ -940), c =0.13 (λ -940), and d =0.024 (λ -940), then different pixel power components are weighted based on these correction parameters, and then the weighted result and the reference parameter are summed to obtain the depth correction factor of the current pixel point.

And S208, correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor.

Wherein the corrected depth map is used for performing a living body detection, such as a living body detection of the target object based on the corrected depth map.

In an embodiment, the terminal may weight the pixel value of each pixel in the depth map according to the obtained depth correction factor, so as to obtain the depth map formed by the weighted pixel values, thereby implementing the correction of the depth map.

After the depth map is corrected, the terminal may further perform living body detection based on the corrected depth map, so as to determine whether the target object is a living body, and then perform a corresponding operation according to a result of the determination, such as an operation of face recognition. For example, in the process of face payment, the terminal corrects the pixel value of each pixel point in the depth maps of the first several frames, then performs living body detection through the corrected depth maps, so as to determine that the target object is a living body, and then continues to recognize the target object through the color image corresponding to the corrected depth maps to determine whether the target object is the user himself, if so, then payment is performed. By the method, the depth maps of the first 5-10 frames can be corrected, and the acquisition speed of the depth map per second is 12.5, so that the time of each frame of depth map is 80ms, and the depth maps of the first 5-10 frames are corrected, so that the response time of 5 x 80 ms-10 x 80ms (namely 400 ms-800 ms) can be prolonged, the living body detection time is greatly shortened, and the user experience is improved.

In the above embodiment, the center wavelength of the structured light is determined based on the projection temperature of the projector when projecting the structured light, then the depth correction factor of the current pixel point is calculated according to the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the center wavelength, so as to obtain the depth correction factor of each pixel point in the depth map, and then the pixel value of each pixel point in the depth map is corrected according to the obtained depth correction factor, so that the temperature of the projector does not need to be in a stable state, and the in-vivo detection can be performed only by using the corrected depth map, thereby avoiding the waiting time in the in-vivo detection process, shortening the in-vivo detection time, and improving the in-vivo detection efficiency.

In an embodiment, the obtaining of the depth map obtained based on the structured light projected by the projector in S202 may specifically include: collecting the reflected speckle structural light in the process that the terminal projects the structural light to the target object through the projector; generating a speckle structure light map based on the speckle structure light; performing speckle matching calculation on the speckle structure light pattern and a reference structure light pattern of the target object to obtain a parallax between the speckle structure light pattern and the reference structure light pattern; determining distances between the projector and points in the target object based on the parallax; a depth map for the target object is generated from the distances.

Wherein speckle structured light may refer to structured light that is reflected back at the surface of the target object. The reference structured light pattern may refer to: the projector projects structured light at a fixed location toward a target object and then forms a structured light pattern based on the reflected speckle structured light.

When the speckle Matching calculation is performed on the speckle structure light pattern and the reference structure light pattern, the speckle Matching calculation may be performed based on a local Matching algorithm (Block Matching), a Global Matching algorithm (Global Matching), or a Semi-Global Matching algorithm (Semi-Global Matching).

Specific steps for the speckle matching calculation may include: the terminal can search the reference structure light pattern for a speckle point matching a target speckle point in the speckle structure light pattern, and then determine the parallax between the speckle structure light pattern and the reference structure light pattern according to the target speckle point and the matching speckle point. Or the terminal can determine a matching area in the speckle structure light image so as to obtain a target scattered spot in the matching area, then search a speckle point matched with the target speckle point in the reference structure light image so as to quickly search speckle points matched with each other in the two images, and then determine the parallax between the speckle structure light image and the reference structure light image according to the target scattered spot and the matched speckle point.

The target scattered spots can be all scattered spots in the speckle structure light pattern, or scattered spots in a matching area in the speckle structure light pattern. The speckle point corresponds to the speckle structure light, for example, a beam of structured infrared light is received, and when the speckle structure light pattern is generated, a scattered spot can be generated in the speckle structure light pattern according to the beam of infrared light.

In one embodiment, the above-mentioned disparity is a disparity between each target speckle point in the speckle structural light pattern and a matching speckle point in the reference structural light pattern; the distances include at least two target distances; at least two target distances are determined based on the disparity between at least two adjacent target speckle points and the matching speckle point. Therefore, the step of determining the distance between the projector and each point in the target object based on the parallax may specifically include: the terminal determines a distance difference value of each target distance according to each target scattered spot and the target distance corresponding to each matched scattered spot; adjusting the parallax between each target speckle point and each matched scattered spot according to the distance difference value; the distance between the projector and the point in the target object at which the structured light is reflected is determined in dependence on the adjusted parallax. Wherein the reference structured light pattern may be a speckle structured light pattern acquired by a projector at a known distance (e.g., 0.5 meters) from the target object.

In the above embodiment, the depth map may be generated according to the speckle structure light, so as to correct the depth map of the first several frames and perform the in vivo detection, which is beneficial to improving the time efficiency of the in vivo detection. In addition, in the process of generating the depth map based on the speckle structure light, error compensation is performed by considering error factors, namely, the parallax is adjusted, so that the accuracy of the depth information in the depth map is improved.

In an embodiment, the step of generating the depth map for the target object according to the distance may specifically include: the terminal determines image depth information according to the distance; rendering the image based on the image depth information to obtain an initial depth map; and denoising and filtering the initial depth map to obtain a depth map of the target object.

The denoising process may be to remove information, which does not belong to the target object, in the initial depth map as noise. The filtering process may be effective filtering of edges in the depth map to strengthen the edges of the depth map. In addition, contrast enhancement processing may also be performed on the initial depth map.

When the terminal has a plurality of collectors (such as cameras) for collecting depth maps and the depth maps need to be aligned, coordinate values of the target object in a first coordinate system (i.e., a coordinate system corresponding to the first camera) can be determined, and then the coordinate values of the first coordinate system are converted to a second coordinate system (i.e., a coordinate system corresponding to the second camera) through external parameters of the camera, as shown in fig. 5, so that alignment of the depth maps is achieved. The camera extrinsic parameters include a rotation matrix R and a translation matrix t.

In the embodiment, the initial depth map is subjected to denoising and filtering processing, so that background noise can be effectively removed and the edge of the depth map can be effectively strengthened, the obtained depth map is more accurate, and the accuracy of living body detection based on the depth map is improved.

In one embodiment, after the depth map is corrected, a living body detection operation may also be performed, which is as follows: performing living body detection on the target object based on the corrected depth map to obtain a first detection result; wherein the depth map is an image obtained by projecting the structured light to the target object by the projector; acquiring an infrared image corresponding to the corrected depth map, and performing living body detection on the target object based on the infrared image to obtain a second detection result; based on the first detection result and the second detection result, the target object is determined to be a living body.

The target object may be a real person, or a paper photograph, an electronic photograph, or a human body model or a mask. It is noted that the depth map and the corresponding ir image are aligned in time and space, i.e. the depth map and the ir image may be acquired at the same time instant, and the resolution, field of view (FOV) and pixels of the depth map and the ir image are the same.

Based on the living body detection of the depth map, it is possible to determine the distance between each point of the terminal and the target object, and if the target object is a living body, the surface of the target object should not be horizontal, and the distance between each point of the terminal and the target object should be different, so it is possible to determine whether the target object is a person in a paper photograph or an electronic photograph.

Based on the living body detection of the infrared image, it can be determined whether or not the target object emits infrared light, and if the target object is a living body, infrared light is emitted, so that it can be determined whether or not the target object is a human body model or a mask worn by a user.

In the embodiment, based on the living body detection of the depth map and the infrared image, whether the target object is a living body can be accurately judged, and the accuracy of the living body detection is improved.

In one embodiment, after the living body detection is performed, the image recognition may be further performed, that is, whether the target object is the user himself or herself is recognized, which is as follows: when the target object is a living body, the terminal acquires a color image corresponding to the corrected depth map; extracting image features of the target object from the color image; if the image characteristics are matched with the image characteristics stored in the characteristic library and the resource transfer operation is detected, transferring the resources with the specified limit from the resource account of the target object; or transferring the resource with the specified quota to the resource account of the target object.

The color image may refer to an RGB (red, green, and blue) image obtained by image-capturing the target object. When the target object is determined to be a living body, the image feature may be a human face feature, or a key point of the human face, such as at least one feature of a face edge feature point, a left eyebrow feature point, a right eyebrow feature point, a nose feature point, a left eye feature point, a right eye feature point, and a lip feature point. As shown in fig. 6, 1 to 17 in fig. 6 indicate face edge feature points, 18 to 22, and 23 to 27 indicate left and right brow feature points of the user correspondingly, 28 to 36 indicate nose feature points of the user, 37 to 42 indicate left eye feature points of the user, 43 to 48 indicate right eye feature points of the user, and 49 to 68 indicate lip feature points of the user.

Before extracting the image features, the terminal can also determine a face region, and then extract the image features of the target object in the face region.

When image features are matched, the terminal can acquire corresponding image features from the feature library according to a user account, then compares the extracted image features with the image features acquired from the feature library, so as to determine the similarity between the two, and determines whether the target object is the user according to the size of the similarity. In addition, the terminal can compare the extracted image features with the image features in the feature library one by one, so as to determine the similarity between the extracted image features and the image features, determine whether the target object is a registered user according to the similarity, and further determine the identity of the target object.

In the above embodiment, when the living body detection is performed based on the depth map and the infrared image and the target object is determined to be the living body, the face recognition is performed through the color image to recognize whether the target object is the user, and only when the target object is the user, the resource transfer operation is performed, so that potential safety hazards of resources are avoided.

As an example, the method is described with reference to a correction framework of a depth map, as shown in fig. 7, fig. 7 is a schematic diagram of the correction framework of the depth map, where the correction framework of the depth map is introduced first, and then the method for correcting the depth map is described with reference to the correction framework, which specifically includes the following steps:

(1) And (4) correcting the depth map.

The basic principle is as follows: firstly, temperature calibration is carried out, the relation between temperature and wavelength is represented through a temperature correction wavelength model, then a depth correction factor is calculated by utilizing the center wavelength of the structured laser and a depth correction function, and finally the depth correction factor is used for correcting a depth map.

A projector: refers to a laser capable of emitting a regular structured laser for measuring distance.

And (3) wavelength measurement: refers to measuring the center wavelength of the structured laser using a spectrometer.

Temperature corrected wavelength model: the method is characterized in that a model of wavelength variation along with temperature is represented by fitting a wavelength variation curve through fitting the central wavelength of the structured laser measured at different temperatures.

And (3) correcting the depth map: means that a depth correction factor is calculated from the relationship between wavelength and depth, and the depth map is corrected according to the depth correction factor.

(2) And correcting the depth map.

First, a wavelength measurement is performed. Under different temperature conditions (temperature can be selected from-10 to 60 degrees), the wavelength of the structured laser emitted by the projector is measured by using a spectrometer, and the trend of the wavelength along with the temperature is recorded, as shown in fig. 4.

Then, a temperature correction wavelength model is constructed, and the construction steps are as follows:

as shown in FIG. 4, assuming temperature and wavelength as data nodes, there are n +1 data nodes (x) ₀ ,y ₀ )，(x ₁ ,y ₁ )，(x ₂ ,y ₂ )，…，(x _n ,y _n ). Wherein x is ₀ 、x ₁ 、x ₂ 、…、x _n Respectively represent different temperatures, and y ₀ 、y ₁ 、y ₂ 、…、y _n Respectively, the wavelength of the structured laser light projected by the projector at different temperatures.

1, calculating the temperature step length h _i ＝x _i+1 -x _i ；

2, substituting the data node and the specified head end point condition into the matrix equation y = a _i +b _i x+c _i x ² +d _i x ³ ；

3, solving the matrix equation to obtain a quadratic differential value m _i . The coefficient matrix in the matrix equation is a tri-diagonal matrix, and LU decomposition can be performed on the coefficient matrix to obtain a unit lower triangular matrix and a unit upper triangular matrix. Namely that

B＝Ax＝(LU)x＝L(Ux)＝Ly

And 4, calculating coefficients of the matrix equation.

a _i ＝y _i

Wherein, a _i 、b _i 、c _i And d _i Are coefficients of a matrix equation, m _i Is a quadratic differential value.

5, in each subinterval x _i ≤x≤x _i+1 Creating a temperature corrected wavelength model as follows:

g(x)＝a _i +b _i (x-x _i )+c _i (x-x _i ) ² +d _i (x-x _i ) ³

where x denotes the projection temperature of the projector when projecting the structured laser light, x _i Is the calibrated temperature.

And finally, calculating the center wavelength of the structured laser according to the temperature correction wavelength model and the projection temperature of the projector, calculating a depth correction factor by combining a depth correction function, and correcting the depth map by using the depth correction factor:

S(x _i )＝a+b(x _i -t)+c(x _i -t) ² +d(x _i -t) ³

wherein x is _i Representing a current pixel point i in the depth map, t representing a pixel point (which may be an adjacent pixel point or a plurality of adjacent pixel points) adjacent to the current pixel point, and S _i (x) And a, b, c and d are correction parameters of different power components respectively. Further, b =0.3 (λ -940), c =0.13 (λ -940), and d =0.024 (λ -940).

By the method, the depth maps of the first 5-10 frames can be corrected, and the acquisition speed of the depth map per second is 12.5, so that the time of each frame of depth map is 80ms, and the depth maps of the first 5-10 frames are corrected, so that the response time of 5 x 80 ms-10 x 80ms (namely 400 ms-800 ms) can be prolonged, the living body detection time is greatly shortened, and the user experience is improved.

In order to more intuitively and clearly understand the scheme of the application, the application also provides an application scenario for resource account registration, and the application scenario applies the correction method of the depth map. Specifically, as shown in fig. 8, the application of the depth map correction method in the application scenario is as follows:

s802, the terminal detects the operation of resource account registration.

As shown in fig. 9, the terminal displays user information input by the user, including the user's name, identification number, and mailbox, through the registration initiation page 90A of fig. 9 (b), and then detects an operation in which the user clicks the registration button 902 in real time.

S804, the terminal collects the depth map.

Wherein, the terminal may project the structured light to the target object through the projector, then generate the depth map through the sensor in common, and acquire a projection temperature of the projector when projecting the structured light. In addition, during the process of acquiring the depth map, the terminal may also acquire a color image of the target object and an infrared image, and as shown in fig. 9 (B), the color image of the target object is displayed in an image acquisition frame 904 of the image acquisition page 90B.

S806, the terminal corrects the depth map.

Specifically, the terminal determines the center wavelength of the structured light projected by the projector based on the projection temperature; aiming at each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained; and correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor.

And S808, the terminal detects the living body based on the corrected depth map.

And S810, the terminal performs face recognition by using the color image.

The terminal extracts the face features from the color image, compares the extracted face features with the face features stored in the feature library corresponding to the user information, and if the extracted face features are consistent with the face features stored in the feature library, the terminal performs face recognition and then executes S812.

S812, the terminal initiates a registration request to the server.

Wherein, the registration request carries the user information.

S814, the terminal receives the resource account information sent by the server.

The server generates resource account information after receiving a registration request sent by the terminal, registers the user information and the resource account information to the resource account management system, and feeds back the resource account information to the terminal. After receiving the resource account information, the terminal displays the registration result and the resource account information on a result display page, as shown in fig. 9 (c).

In addition, the server can also perform face recognition according to the color image acquired by the terminal, for example, the terminal sends the color image and user information to the server, and the server performs face recognition on the color image and generates resource account information after the face recognition passes. And the server registers the user information and the resource account information in the resource account management system and simultaneously feeds back the resource account information to the terminal.

In the embodiment, the terminal performs living body detection by using the corrected depth map, and initiates a registration request to the server when the target object is determined to be a living body and the target object is the user owner, so that the server generates resource account information and feeds back the resource account information to the client, thereby effectively preventing an illegal user from pretending to be the identity of a legal user to register a resource account, ensuring the legal use of the resource account, and improving the registration efficiency of the resource account.

The application also provides a paid application scene, and the application scene applies the correction method of the depth map. Specifically, as shown in fig. 10, the application of the depth map correction method in the application scenario is as follows:

s1002, detecting the operation of confirming the product and the price.

As shown in fig. 11, the terminal displays the purchased product and the corresponding price through the product and price display page 1102 in fig. 11 (b), and then detects an operation of the user clicking the confirm button 1104 in real time.

And S1004, the terminal collects the depth map.

Wherein, the terminal may project the structured light to the target object through the projector, then generate the depth map through the sensor in common, and acquire a projection temperature of the projector when projecting the structured light. In addition, during the process of acquiring the depth map, the terminal may also acquire a color image of the target object and an infrared image, and as shown in fig. 11 (b), the color image of the target object is displayed in an image acquisition page 1106.

S1006, the terminal corrects the depth map.

And S1008, the terminal carries out living body detection based on the corrected depth map.

And S1010, the terminal performs face recognition by using the color image.

The terminal extracts the face features from the color image, compares the extracted face features with the face features stored in the feature library corresponding to the user information, and if the extracted face features are consistent with the face features stored in the feature library, performs face recognition and then executes S1012.

S1012, the terminal pays the merchant through the server.

When the target object is determined to be the user, payment is made to the merchant through the account of the target object, namely, the account of the merchant is transferred; alternatively, the transfer is made to the third party platform via the account of the target object, such that the third party platform transfers funds to the merchant's account.

S1014, the terminal receives the information of successful payment.

After receiving the information of successful payment, the terminal displays the word of successful payment on the result display page 1108, as shown in fig. 11 (c).

In the embodiment, the terminal performs living body detection by using the corrected depth map, and pays the money of the product when the target object is determined to be the living body and the target object is the user, so that the safety of the user fund is ensured.

The application also provides an application scene of opening the entrance guard, under the application scene, the terminal can be an entrance guard device integrated with an entrance guard system, and the entrance guard device comprises a data acquisition end (integrating a display and a camera) 120 and a living body detection end (comprising a gate) 122. Specifically, as shown in fig. 12, the application scenario applies the above-mentioned depth map correction method as follows:

when a target object approaches an access control device integrated with an access control system, a living body detection end 122 in the access control device is triggered to carry out living body detection on the target object, and a data acquisition end 120 in the access control device displays an image acquisition page, so that the data acquisition end 120 can acquire a depth map of the target object through the image acquisition page and can acquire a color image and an infrared image of the target object.

When the depth map is acquired, the data acquisition terminal 120 transmits the depth map to the living body detection terminal 122. The living body detection end 122 determines the center wavelength of the structured light projected by the projector based on the projection temperature; aiming at each pixel point in the depth map, determining a depth correction factor of the current pixel point based on the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point and the central wavelength until the depth correction factor of each pixel point in the depth map is obtained; and correcting the pixel value of each pixel point in the depth map according to the obtained depth correction factor.

After correcting the depth map, the terminal performs living body detection based on the corrected depth map, and then performs face recognition using the color image. The process of face recognition may include: and the terminal extracts the face features from the color image and then compares the extracted face features with the face features which correspond to the user information and are stored in the feature library. If the detected images are consistent with each other, the living body detection end 122 opens the gate to allow the target object to enter through face recognition.

In the embodiment, the living body detection is performed on the target object requesting to open the gate, so as to verify whether the target object is a living body or not and the identity, and the gate is opened when the target object is determined to be the living body, so that the safety of the access control system is ensured, compared with the manual verification of the target object, the cost is saved, and the passing efficiency is improved.

It should be understood that although the steps in the flowcharts of fig. 2, 8, 10 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Moreover, at least some of the steps in fig. 2, 8, and 10 may include multiple steps or multiple stages, which are not necessarily performed at the same time, but may be performed at different times, which are not necessarily performed in sequence, but may be performed alternately or alternately with other steps or at least some of the other steps or stages.

In one embodiment, as shown in fig. 13, there is provided an apparatus for correcting a depth map, which may be a part of a computer device using a software module or a hardware module, or a combination of the two modules, and specifically includes: an obtaining module 1302, a first determining module 1304, a second determining module 1306, and a correcting module 1308, wherein:

an obtaining module 1302, configured to obtain a depth map obtained based on the structured light projected by the projector, and a projection temperature of the projector when projecting the structured light;

a first determining module 1304 for determining a center wavelength of the structured light projected by the projector based on the projection temperature;

a second determining module 1306, configured to determine, for each pixel point in the depth map, a depth correction factor of the current pixel point based on a pixel value of the current pixel point in the depth map, a pixel value of a pixel point adjacent to the current pixel point, and the center wavelength until the depth correction factor of each pixel point in the depth map is obtained;

a correcting module 1308, configured to correct a pixel value of each pixel point in the depth map according to the obtained depth correction factor; the corrected depth map is used for performing the living body detection.

In the above embodiment, the central wavelength of the structured light is determined based on the projection temperature of the projector when projecting the structured light, the depth correction factor of the current pixel point is calculated according to the pixel value of the current pixel point in the depth map, the pixel value of the pixel point adjacent to the current pixel point, and the central wavelength, so as to obtain the depth correction factor of each pixel point in the depth map, and the pixel value of each pixel point in the depth map is corrected according to the obtained depth correction factor, so that the temperature of the projector does not need to reach a stable state, and the in-vivo detection can be performed only by using the corrected depth map, thereby avoiding the waiting time in the in-vivo detection process, shortening the in-vivo detection time, and improving the in-vivo detection efficiency.

In one embodiment, the obtaining module 1302 is further configured to collect speckle structure light reflected from the projection of the structured light onto the target object by the projector; generating a speckle structure light map based on the speckle structure light; performing speckle matching calculation on the speckle structure light pattern and a reference structure light pattern of the target object to obtain the parallax between the speckle structure light pattern and the reference structure light pattern; determining distances between the projector and points in the target object based on the parallax; a depth map for the target object is generated from the distances.

In one embodiment, the disparity is the disparity between each target speckle point in the speckle structural light map and a matching speckle point in the reference structural light map; the distance comprises at least two target distances; at least two target distances determined based on a disparity between at least two adjacent target speckle points and the matching speckle point;

the obtaining module 1302 is further configured to determine a distance difference value of each target distance for each target scattered spot and a target distance corresponding to each matched scattered spot; adjusting the parallax between each target speckle point and each matched speckle point according to the distance difference; the distance between the projector and the point in the target object at which the structured light is reflected is determined in dependence on the adjusted parallax.

In the embodiment, the depth map can be generated according to the speckle structure light, so that the depth maps of the first frames can be corrected and the living body detection can be carried out, and the time efficiency of the living body detection can be improved. In addition, in the process of generating the depth map based on the speckle structure light, error compensation is performed by considering error factors, namely, the parallax is adjusted, so that the accuracy of the depth information in the depth map is improved.

In one embodiment, the obtaining module 1302 is further configured to determine image depth information according to the distance; rendering the image based on the image depth information to obtain an initial depth map; and denoising and filtering the initial depth map to obtain a depth map of the target object.

In one embodiment, the first determining module 1304 is further configured to obtain a conversion relationship between the projection temperature and the wavelength; and determining the central wavelength of the structured light projected by the projector according to the conversion relation and the projection temperature.

In one embodiment, as shown in fig. 14, the apparatus further comprises:

a first detection module 1310 for detecting wavelengths of the structured light projected by the projector at different calibration temperatures;

the fitting module 1312 is configured to fit wavelengths of the structured light at different calibration temperatures to obtain a wavelength variation curve; the wavelength variation curve is used as the conversion relation between the projection temperature and the wavelength.

In one embodiment, the fitting module 1312 is further configured to use different calibration temperatures and corresponding wavelengths as data nodes, respectively; determining a temperature step between adjacent data nodes; determining a wavelength adjustment parameter based on different data nodes, temperature step lengths and node coefficients; and respectively constructing wavelength change curves of different data nodes according to the wavelength adjustment parameters and the calibration temperatures in different data nodes.

In one embodiment, the second determining module 1306 is further configured to perform a power operation on a difference between a pixel value of a current pixel point in the depth map and a pixel value of a pixel point adjacent to the current pixel point, so as to obtain at least two pixel power components; determining correction parameters corresponding to at least two pixel power components based on the central wavelength and a preset conversion coefficient; and determining the depth correction factor of the current pixel point according to the correction parameter, the at least two pixel power components and the reference parameter.

In one embodiment, the depth map is an image of a projector projecting structured light onto a target object; as shown in fig. 14, the apparatus further includes:

a second detection module 1314, configured to perform in vivo detection on the target object based on the corrected depth map, so as to obtain a first detection result; acquiring an infrared image corresponding to the corrected depth map, and performing living body detection on the target object based on the infrared image to obtain a second detection result; based on the first detection result and the second detection result, the target object is determined to be a living body.

In one embodiment, as shown in fig. 14, the apparatus further comprises:

a processing module 1316, configured to, when the target object is a living body, acquire a color image corresponding to the corrected depth map; extracting image features of the target object from the color image; if the image characteristics are matched with the image characteristics stored in the characteristic library and the resource transfer operation is detected, transferring the resources with the specified limit from the resource account of the target object; or transferring the resource with the designated quota to the resource account of the target object.

For the specific definition of the correction device for the depth map, reference may be made to the above definition of the correction method for the depth map, which is not described herein again. The modules in the correction device for the depth map can be wholly or partially realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as shown in fig. 15. The computer device includes a processor, a memory, and a network interface connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The database of the computer device is used for storing the depth map. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a method of correcting a depth map.

In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as shown in fig. 16. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used for communicating with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a method of correcting a depth map. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.

It will be appreciated by those skilled in the art that the configurations shown in fig. 15 and 16 are block diagrams of only some of the configurations relevant to the present disclosure, and do not constitute a limitation on the computing devices to which the present disclosure may be applied, and that a particular computing device may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

In an embodiment, a computer device is further provided, which includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above method embodiments when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, in which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned method embodiments.

In one embodiment, a computer program product or computer program is provided that includes computer instructions stored in a computer-readable storage medium. The computer instructions are read by a processor of a computer device from a computer-readable storage medium, and the computer instructions are executed by the processor to cause the computer device to perform the steps in the above-mentioned method embodiments.

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above may be implemented by hardware instructions of a computer program, which may be stored in a non-volatile computer-readable storage medium, and when executed, may include the processes of the embodiments of the methods described above. Any reference to memory, storage, database or other medium used in the embodiments provided herein can include at least one of non-volatile and volatile memory. Non-volatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical storage, or the like. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), among others.

The technical features of the above embodiments can be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the above embodiments are not described, but should be considered as the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A method for correcting a depth map, the method comprising:

2. The method of claim 1, wherein obtaining a depth map based on structured light projected by a projector comprises:

collecting speckle structural light reflected back in the process of projecting the structural light to a target object by a projector;

generating a speckle structure light map based on the speckle structure light;

performing speckle matching calculation on the speckle structure light pattern and a reference structure light pattern of the target object to obtain a parallax between the speckle structure light pattern and the reference structure light pattern;

determining distances between the projector and points in the target object based on the disparity;

generating a depth map for the target object according to the distance.

3. The method of claim 2, wherein the disparity is a disparity between each target speckle point in the speckle structural light map and a matching speckle point in the reference structural light map; the distances include at least two target distances; at least two of the target distances are determined based on a disparity between at least two adjacent of the target speckle points and the matched speckle point;

the determining distances between the projector and points in the target object based on the disparity comprises:

determining a distance difference value of each target distance according to each target scattered spot and a target distance corresponding to each matched speckle point;

adjusting the parallax between each target speckle point and each matched speckle point according to the distance difference;

determining a distance between the projector and a point of the target object that reflects the structured light as a function of the adjusted parallax.

4. The method of claim 2, wherein the generating the depth map for the target object according to the distance comprises:

determining image depth information according to the distance;

rendering an image based on the image depth information to obtain an initial depth map;

and denoising and filtering the initial depth map to obtain the depth map of the target object.

5. The method of claim 1, wherein the determining the center wavelength of the structured light projected by the projector based on the projection temperature comprises:

acquiring a conversion relation between the projection temperature and the wavelength;

and determining the central wavelength of the structured light projected by the projector according to the conversion relation and the projection temperature.

6. The method of claim 5, further comprising:

detecting the wavelength of the structural light projected by the projector at different calibration temperatures;

fitting the wavelengths of the structured light at different calibration temperatures to obtain a wavelength change curve;

and taking the wavelength change curve as the conversion relation between the projection temperature and the wavelength.

7. The method according to claim 6, wherein the fitting the wavelengths of the structured light at the different calibration temperatures to obtain a wavelength variation curve comprises:

respectively taking the different calibration temperatures and the corresponding wavelengths as data nodes;

determining a temperature step between adjacent data nodes;

determining a wavelength adjustment parameter based on different data nodes, the temperature step length and the node coefficient;

and respectively constructing wavelength change curves of different data nodes according to the wavelength adjusting parameters and the calibration temperatures in different data nodes.

8. The method of claim 1, wherein determining the depth correction factor for the current pixel based on the pixel value of the current pixel in the depth map, the pixel values of pixels neighboring the current pixel, and the center wavelength comprises:

performing power operation on a difference value between a pixel value of a current pixel point in the depth map and a pixel value of a pixel point adjacent to the current pixel point to obtain at least two pixel power components;

determining correction parameters corresponding to at least two of the pixel power components based on the central wavelength and a preset conversion coefficient;

and determining a depth correction factor of the current pixel point according to the correction parameter, at least two pixel power components and a reference parameter.

9. The method of any one of claims 1 to 8, wherein the depth map is an image of the projector projecting structured light onto a target object; the method further comprises the following steps:

performing living body detection on the target object based on the corrected depth map to obtain a first detection result;

acquiring an infrared image corresponding to the corrected depth map, and performing living body detection on the target object based on the infrared image to obtain a second detection result;

determining that the target object is a living body based on the first detection result and the second detection result.

10. The method of claim 9, further comprising:

when the target object is a living body, acquiring a color image corresponding to the corrected depth map;

extracting image features of the target object from the color image;

if the image characteristics are matched with the image characteristics stored in a characteristic library and the resource transfer operation is detected, transferring the resources with the specified limit from the resource account of the target object; or transferring the resource with the specified quota to the resource account of the target object.

11. An apparatus for correcting a depth map, the apparatus comprising:

12. The apparatus of claim 11, wherein the acquisition module is further configured to collect speckle structured light reflected from the projection device during projection of the structured light onto the target object; generating a speckle structure light map based on the speckle structure light; performing speckle matching calculation on the speckle structure light pattern and a reference structure light pattern of the target object to obtain a parallax between the speckle structure light pattern and the reference structure light pattern; determining distances between the projector and points in the target object based on the disparity; generating a depth map for the target object according to the distance.

13. The apparatus of claim 12, wherein the disparity is a disparity between each target speckle point in the speckle structured light map and a matching speckle point in the reference structured light map; the distances include at least two target distances; at least two of the target distances are determined based on a disparity between at least two adjacent of the target speckle points and the matched speckle point;

the acquisition module is further configured to determine a distance difference value of each target distance for each target scattered spot and a target distance corresponding to each matched scattered spot; adjusting the parallax between each target speckle point and each matched speckle point according to the distance difference; determining a distance between the projector and a point of the target object that reflects the structured light as a function of the adjusted parallax.

14. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor realizes the steps of the method of any one of claims 1 to 10 when executing the computer program.

15. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 10.