WO2022052366A1 - Fused depth measurement method and measurement device - Google Patents

Abstract

A fused depth measurement method and measurement device, measuring a distance of a scene region (30). Said measurement method comprises: emitting a pulse beam (10) to a scene region (30); receiving a reflected signal (20) of the pulse beam (10), and outputting an electrical signal of a transit time of the pulse beam (10) and the reflected signal (20) back and forth; acquiring a left image of the scene region (30), and acquiring a right image of the scene region (30); processing the electrical signal of the transit time to obtain a first depth image, and converting the first depth image into a first parallax image; performing stereo matching between the left image and the right image, so as to obtain a second parallax image; and fusing the first parallax image and the second parallax image, so as to obtain a target image. The fused depth measurement method and measurement device solve the technical problems of long distance measurement calculation time and low target recognition accuracy caused by a complex algorithm in traditional distance measurement methods.

Description

A fusion depth measurement method and measurement device technical field

The present application relates to the technical field of image processing and optical measurement, and in particular, to a fusion depth measurement method and measurement device.

Background technique

The depth measurement device can be used to obtain the depth image of the object, and further can perform 3D modeling, skeleton extraction, face recognition, etc., and has a very wide range of applications in the fields of 3D measurement and human-computer interaction. The current depth measurement technologies mainly include TOF ranging technology and binocular ranging technology.

The full name of TOF is Time-of-Flight, that is, time of flight. TOF ranging technology is a technology that achieves precise ranging by measuring the round-trip flight time of a pulsed beam between the transmitting/receiving device and the target object. It is divided into direct measurement distance technology and indirect distance measurement technology. Among them, direct ranging technology measures the time difference between the emission and reception of pulsed beams. Compared with traditional image sensors, direct ranging technology uses single-photon avalanche diode (SPAD) image sensors to achieve high-sensitivity light detection, and uses time-correlated single Photonic technology approach to achieve picosecond time precision. However, there are still many limitations in the manufacturing process and chip design of the SPAD image sensor, so that the resolution of the image sensor is very low.

The binocular ranging technology uses the triangular ranging method to calculate the distance from the measured object to the camera; specifically, when the same object is observed from two cameras, the position of the observed object in the images captured by the two cameras will be different. There is a certain position difference, that is, parallax; the closer the measured object is to the camera, the greater the parallax; the farther away, the smaller the parallax. When the relative positional relationship such as the distance between the two cameras is known, the distance from the subject to the camera can be calculated through the principle of similar triangles. However, this ranging method has high requirements on processor hardware, and needs to rely on complex algorithms. The algorithm takes a long time to calculate, and the recognition effect is not good for targets with inconspicuous features, and the recognition accuracy is low.

Therefore, the traditional ranging method has technical problems such as long ranging calculation time and low target recognition accuracy due to the complex algorithm.

SUMMARY OF THE INVENTION

The purpose of the present application is to provide an integrated depth measurement method and measurement device, which aims to solve the technical problems of the traditional ranging method that the algorithm is complex, which leads to long ranging calculation time and low target recognition accuracy.

A first aspect of the embodiments of the present application provides a fusion depth measurement method, which measures the distance of a scene area, including:

emitting a pulsed light beam to the scene area;

receiving the reflected signal of the pulsed beam, and outputting an electrical signal of the transit time between the pulsed beam and the reflected signal;

collecting left and right images of the scene area;

processing the electrical signal of the transit time to obtain a first depth image, and converting the first depth image into a first parallax image;

performing stereo matching on the left image and the right image to obtain a second parallax image;

The first parallax image and the second parallax image are fused to obtain a target image.

In one embodiment, emitting a pulsed light beam to the scene area includes:

generating the pulsed beam;

adjusting the divergence of the pulsed beam;

The pulsed light beam is directed to instruct the pulsed light beam to diverge in various directions to the scene area.

A second aspect of the embodiments of the present application provides a fused depth measurement device, including:

The emission module is used to emit pulsed beams to the scene area;

a detection module, configured to receive the reflected signal of the pulsed beam, and output an electrical signal of the transit time between the pulsed beam and the reflected signal;

an acquisition module, configured to acquire the left image and the right image of the scene area;

a processing module, configured to process the electrical signal of the transit time to obtain a first depth image, and convert the first depth image into a first parallax image;

a conversion module that performs stereo matching on the left image and the right image to obtain a second parallax image; and

A fusion module, configured to fuse the first parallax image and the second parallax image to obtain a target image.

In one embodiment, the transmitting module includes:

an array of light sources for generating pulsed light beams;

a lens element for adjusting the divergence of the pulsed beam; and

The beam scanning element is used for guiding the pulsed beam to indicate that the pulsed beam diverges to various directions of the scene area.

In one embodiment, the detection module includes a single-photon avalanche diode image sensor;

In the processing module, converting the first depth image into a first parallax image specifically includes:

Using the left image as a reference image, calculate the first parallax image corresponding to the first depth image:

In the formula, _PD (x ₀ , y ₀ ) is the disparity value of the depth value Z (x ₀ , y ₀ ) at the point (x ₀ , y ₀ ) on the first depth image, and f is the single The focal length of the photon avalanche diode image sensor, T _lt is the baseline length of the depth camera-left camera system, and H _lt is the homography matrix calibrated by the depth camera.

In one of the embodiments, converting the first depth image into the first parallax image further includes:

The parallax surface of the first parallax image is fitted by the following binary cubic equation to obtain a smooth parallax surface:

d(x,y)=a ₁ +a ₂ x+a ₃ y+a ₄ x ² +a ₅ xy+a ₆ y ² +a ₇ x ³ +a ₈ x ² y+a ₉ xy ² +a ₁₀ y ³

In the formula, d(x, y) is a three-dimensional parallax surface, a ₁ , a ₂ , ..., a ₁₀ are coefficients, and x and y are pixel coordinates.

In one embodiment, before converting the first depth image into a first parallax image, the processing module further includes:

Perform joint calibration on the single-photon avalanche diode image sensor and the left image sensor or the right image sensor:

Converting the first depth image obtained by the single-photon avalanche diode image sensor into point cloud data, and mapping the point cloud data to the camera of the left image sensor or the right image sensor through a jointly calibrated transformation matrix In the coordinate system, a plane two-dimensional point with the left image sensor or the right image sensor as a reference is obtained.

In one embodiment, in the conversion module, performing stereo matching on the left image and the right image to obtain the second parallax image specifically includes:

Select the pixel points on the first parallax image as seed points, and guide the left image and the right image to perform stereo matching to obtain the second parallax image, specifically using the following formula:

In the formula, (x, y ₀ ) is the pixel point within each parallax range of the right image, and θ is the selected parameter.

In one embodiment, the first parallax image and the second parallax image are fused to obtain a depth image, which specifically includes:

obtaining a first credibility function according to the first parallax image;

obtaining a second credibility function according to the second parallax image;

According to the different weights formed by the first credibility function and the second credibility function, the first parallax image and the second parallax image are fused to obtain pixel fusion parallax;

Three-dimensional reconstruction is performed on the scene area according to the pixel fusion disparity, so as to obtain the depth image.

In one embodiment, the first parallax image and the second parallax image are fused to obtain pixel fusion parallax, which specifically includes:

According to the first credibility function and the second credibility function, different weights are formed to fuse the two parallaxes to obtain the pixel fusion parallax. Specifically, the following formula is used:

d=w _t ·d _t +w _s ·d _s

w _t =1-w _s

In the formula, d _t is the first disparity value, d _s is the second disparity value, w _t is the weight according to the first disparity value, and _ws is the weight according to the second disparity value.

In the above-mentioned fusion depth measurement method and measurement device in the embodiment of the present invention, a first depth image is acquired by a single-photon avalanche diode image sensor, and the pixel points of the first depth image are used as seed points to guide the left image sensor and the right image sensor. The image sensor performs stereo matching to obtain a second parallax image, and by converting the first depth image into a first parallax image, and forming different weights according to the first credibility function and the second credibility function, to fuse the first The parallax image and the second parallax image are obtained by merging the parallax image to restore the three-dimensional reconstruction of the scene area, and obtain a high-resolution target image with high recognition accuracy.

Description of drawings

FIG. 1 is a schematic diagram of specific flow steps of a fusion depth measurement method provided by an embodiment of the present application;

2 is a schematic flowchart of converting a first depth image into a first parallax image in a fusion depth measurement method provided by an embodiment of the present application;

FIG. 3 is a schematic flowchart of fusing a first parallax image and a second parallax image in a fusion depth measurement method provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a fused depth measurement apparatus according to an embodiment of the present application.

detailed description

In order to make the technical problems, technical solutions and beneficial effects to be solved by the present application clearer, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

Fig. 1 is the concrete flow chart of the depth measurement method of a kind of fusion provided by an embodiment of the application, for the convenience of explanation, only shows the part relevant to the present embodiment, and details are as follows:

In an embodiment, the first aspect provides a fusion depth measurement method, which measures the distance of a scene area, including the following steps:

S101. Control the emission module to emit a pulse beam to the scene area.

In one embodiment, the emission module includes a light source array, a lens element and a beam scanning element.

The light source array is used to generate pulsed beams. In one embodiment, the light source array uses a semiconductor LED, a semiconductor laser, or a VCSEL (Vertical-Cavity Surface-Emitting Laser) array as the light source, and the light source array can also use a parallel light source array. The edge-emitting laser on the surface of the resonant cavity is used to emit light beams with wavelengths such as infrared and ultraviolet. Preferably, in this embodiment, the light source array is a VCSEL array. The VCSEL has the characteristics of small size, small pulse beam emission angle, and good stability. Hundreds of thousands of VCSEL sub-light sources are arranged on a semiconductor substrate with an area of 1mm×1mm. Thus, a VCSEL array light-emitting chip is formed, which is small in size and low in power consumption. The VCSEL array light-emitting chip can be a bare chip with a smaller volume and thickness, or a packaged light-emitting chip with better stability and convenient connection.

In one embodiment, the lens element is used to adjust the divergence of the pulsed light beam, and the lens element adopts a single lens, a lens combination or a microlens array. Divergence can be determined as an angular value associated with one or more cross-sectional axes of the pulsed beam, and adjustment of the divergence can be at least partially controlled to mitigate non-uniformity in the cross-section of the pulsed beam to improve device detection at various locations in a defined field of view The accuracy of the attributes of reflection points, and improve the correlation between each group of reflection points in the light source array and the point cloud of each object, etc. The light source array has certain design requirements for lens elements, such as the emission pulse beam density. Therefore, it is often difficult for a single lens element to meet the requirements, and multiple lenses are required to form a lens posture to meet the design requirements. In the specific implementation process, in addition to the basic design requirements of the lens, it is also necessary to consider some other factors that the lens element will encounter during use, which is not limited here, and can be designed according to specific conditions.

In one embodiment, the beam scanning element is used to guide the pulsed beam to indicate that the pulsed beam is diverged to various directions in the scene area, and the beam scanning element is made of MEMS (Micro Electro Mechanical ayatems, Micro Electro Mechanical System) technology. Micromirrors that controllably emit pulsed beams in all directions and scan the target scene area.

S102. Receive the reflected signal of the pulsed light beam through the single-photon avalanche diode image sensor, and output the electric signal of the transit time of the round-trip pulsed light beam and the reflected signal.

In one embodiment, the emitting module emits a pulsed light beam of a line beam or an area array beam that is pulse-modulated in time sequence, and the SPAD (Single Photon Avalanche Diode, single-photon avalanche diode) image sensor detects the single photon, thereby generating a digital pulse, The time when the digital pulse is generated is recorded by the TDC (Time to Digital Convert) circuit, and the accumulated value of the single photon count in the corresponding time interval is added by one, and multiple sets of photon flight time information are output, and the same measurement is repeated in large numbers. After that, a lot of time data will be obtained, and these time data will be accumulated in the corresponding time interval in the same way, and then the corresponding single photon counting histogram will be obtained through the TCSPC (Time-correlated single-photon counting) circuit. , judge the peak value of the single photon count histogram, determine the electrical signal of the transit time required for the photon to fly between the scene area and the emission module, and output it.

S103. Use the left image sensor to collect the left image of the scene area, and use the right image sensor to collect the right image of the scene area.

In one embodiment, the left image sensor and the right image sensor respectively collect the left infrared image and the right infrared image of the scene area under the illumination of the emission module; both the left image sensor and the right image sensor can use infrared image sensors, and because the infrared The image sensor needs a continuous active light source (such as infrared light) for illumination during imaging, so the emission module also includes infrared light sources such as infrared floodlights and infrared projectors.

In another embodiment, the left image sensor and the right image sensor may also use visible light image sensors, the left image sensor and the right image sensor may use two RGB image sensors, or the one RGB image sensor and one grayscale sensor are combined, Under the illumination of ambient light, it is used to collect the left visible light image and the right visible light image of the scene area respectively.

S104. Process the electrical signal of the transit time to obtain a first depth image, and convert the first depth image into a first parallax image.

In one embodiment, for a parallel stereo vision system, the optical axes of the left camera and the right camera formed by the left image sensor and the right image sensor are parallel, so the depth information of the scene area can be obtained according to the principle of triangulation; The projection coordinates of point P in the fixed scene area on the left camera and the right camera are P _l (x _l , y _l ) and P _r (x _r , y _r ) respectively, and the depth of the scene area point P is obtained according to the principle of triangulation ranging information:

Further, obtain the depth value at point P:

In the formula, z is the depth value of point P, f is the focal length of the left camera and the right camera, b is the baseline between the left image sensor and the right image sensor, and d is the parallax between the left image and the right image.

FIG. 2 is a schematic flowchart of converting a first depth image into a first parallax image in a fusion depth measurement method provided by an embodiment of the present application. For the convenience of description, only parts related to this embodiment are shown. as follows:

Specifically, converting the first depth image into the first parallax image includes the following steps:

S1041. Jointly calibrate the single-photon avalanche diode image sensor with the left image sensor or the right image sensor.

In this embodiment, the left image is used as the reference image, so the SPAD image sensor and the left image sensor are jointly calibrated; the first depth image obtained by the SPAD image sensor is converted into point cloud data, and the point cloud passes through the jointly calibrated transformation matrix [ R,T] maps to the camera coordinate system of the left image sensor, where R is the rotation matrix and T is the translation vector. That is to transform the three-dimensional coordinates X _Ti (i=1,2,...,N) of the points in the SPAD image sensor to the three-dimensional points X _Li (i=1,2,...,N) of the binocular system with the left image sensor as a reference ), the three-dimensional point X _Li of the binocular system is projected into the coordinate system of the left image sensor through the internal parameter matrix of the left image sensor, so as to form a lattice x _Li (i=1,2,...,N), and the left image is obtained. The sensor is a plane two-dimensional point of reference.

S1042. Convert the first depth image into a first parallax image.

Taking the left image as the reference image, according to formula (2), the following formula is used to calculate the first parallax image corresponding to the first depth image obtained by the depth camera formed by the SPAD image sensor:

In the formula, _PD (x ₀ , y ₀ ) is the parallax value of the depth value Z (x ₀ , y ₀ ) at the point (x ₀ , y ₀ ) on the first depth image, and f is the focal length of the SPAD image sensor , T _lt is the baseline length of the depth camera-left camera system, and H _lt is the homography matrix calibrated by the depth camera.

It should be understood that, in this embodiment, the left image is selected as the reference image, and the right image may also be selected as the reference image, which is not limited here.

S1043. Use a binary cubic equation to fit the parallax surface of the first parallax image to obtain a smooth parallax surface.

Since the first parallax image obtained in the above steps is sparse, it is necessary to fill in the missing data. According to the sparse known parallax surface, a smooth parallax surface is fitted in the parallax space, and the unknown parallax can be filled by upsampling according to the parallax surface. In order to ensure that the fitted parallax surface is as close to the real scene as possible, this implementation For example, the parallax surface of the first parallax image is fitted by a binary cubic equation, as follows:

d(x,y)=a ₁ +a ₂ x+a ₃ y+a ₄ x ² +a ₅ xy+a ₆ y ² +a ₇ x ³ +a ₈ x ² y+a ₉ xy ² +a ₁₀ y ³ (4)

In the formula, d(x, y) is a three-dimensional parallax surface, a ₁ , a ₂ , ..., a ₁₀ are coefficients, and x and y are pixel coordinates.

S105. Perform stereo matching on the left image and the right image to obtain a second parallax image.

In one embodiment, performing stereo matching on the left image and the right image to obtain the second parallax image specifically includes:

The pixel points on the first parallax image are selected as seed points, and the left image and the right image are guided to perform stereo matching to obtain the second parallax image.

In one embodiment, since a depth camera composed of a SPAD image sensor cannot obtain accurate depth information in a low-reflection area or an area with strong refraction lines, for the pixel points (x, y) within each parallax range of the right image ₀ ) The depth distance cost function is:

In the formula, θ is a parameter selected according to experience, which is used to adjust the range of the depth distance cost function to set the threshold of C(x, y _{0 )} . If the value of the depth distance cost function is within the threshold range, the pixel is the seed point, and the disparity value between the left image and the right image is calculated to obtain the second disparity image; if a certain pixel (x, y) on the right image ₀ ) The value of the depth distance cost function is outside the threshold range, then the point is a non-seed point, then during binocular stereo matching, it will search for the nearest seed point along its horizontal direction and assign its disparity value to this point.

In one embodiment, the binocular camera composed of the left image sensor and the right image sensor is placed horizontally, ensuring that the optical axes of the cameras are parallel, and the imaging is also on the same horizontal plane, and there is no parallax in the vertical direction, so the ordinate is y ₀ is the same, and the parallax search range is also determined according to the baseline of the binocular camera. ₀ , y ₀ ), will find another point within the [(x ₀ -m,y ₀ ),(x ₀ +m,y ₀ )] disparity search range in the corresponding right image target image.

Assuming that a point (x, y ₀ ) on the target image of the right image, that is, the right image, is the matching point of the reference image, that is, the left image point (x ₀ , y ₀ ), using the depth distance cost function C(x, y ₀ ), guide the Stereo matching is performed on the binocular camera, and the parallax is obtained as D=|(xx ₀ )|, so as to obtain a second parallax image.

S106. Fusion of the first parallax image and the second parallax image to obtain a target image.

FIG. 3 is a schematic flowchart of a fusion depth measurement method for a first parallax image and a second parallax image in a fusion depth measurement method provided by an embodiment of the present application. For the convenience of description, only parts related to this embodiment are shown. Details are as follows:

In one embodiment, fusing the first parallax image and the second parallax image to obtain the target image includes the following steps:

S1061. Obtain a first reliability function according to the first parallax image.

For the first depth image acquired by the SPAD image sensor, the depth value Z(x ₀ , y ₀ ) of the point (x ₀ , y ₀ ) on the first depth image, set its standard deviation as σ _z , according to formula (3) , the corresponding disparity standard deviation can be obtained as σ _d , namely

Further, get the standard deviation of the parallax at the point (x ₀ , y ₀ ):

Among them, define the minimum threshold σ _min and the maximum threshold σ _max of the parallax standard deviation, where σ _min is the standard deviation of the brightest pixel at the nearest received depth of field, and σ _max is the received standard deviation of the darkest pixel farthest from the depth of field . When the standard deviation of the disparity value of a pixel calculated on the first depth image is less than σ _min , the point is considered to be completely stable, and the reliability is 1; when the standard deviation is greater than σ _max , the point is unreliable and reliable degree is 0; when the standard deviation is between the two thresholds, set the reliability range to (0,1), thus obtaining the first reliability function, as follows:

S1062. Obtain a second reliability function according to the second parallax image.

In one embodiment, the second parallax image obtained by the binocular camera is calculated according to an adaptive weighting algorithm, and the second reliability function is obtained, that is,

In the formula,

and

are the minimum matching cost and the second smallest matching cost obtained by the binocular camera in the adaptive weighting algorithm, respectively, T _c is a constant parameter, in order to avoid

is 0, set to T _c >0.

S1063. According to the different weights formed by the first credibility function and the second credibility function, fuse the first parallax image and the second parallax image to obtain pixel fusion parallax.

Suppose the first parallax value obtained by the SPAD image sensor is d _t , and the second parallax value obtained by the left image sensor and the right image sensor is d _s , then according to the first reliability function and the second reliability function , constitute different weights to fuse the two disparities, and obtain the pixel fusion disparity d as

d=w _t ·d _t +w _s ·d _s (10)

In the formula, w _t is the weight of the disparity value obtained from the depth image obtained by the SPAD image sensor, and _ws is the weight of the disparity obtained by stereo matching of the left image and the right image obtained from the left image sensor and the right image sensor.

Further, from formula (10), we can get:

w _t =1-w _s (12)

It should be understood that the present application does not limit the method of calculating the weight, and any method of calculating the weight in the prior art can be applied to the present application.

S1064. Perform three-dimensional reconstruction on the scene area according to the pixel fusion parallax to obtain a target image.

A second aspect of the embodiments of the present application provides a fused depth measurement device, including:

FIG. 4 is a schematic structural diagram of a fused depth measurement apparatus provided by an embodiment of the present application. For the convenience of description, only parts related to this embodiment are shown, and the details are as follows:

In one embodiment, a fused depth measurement apparatus includes a transmitting module 100 , a receiving module 200 and a control and processing module 300 .

The transmitting module 100 is used for transmitting the pulsed light beam 10 to the scene area 30 .

Specifically, the emission module 100 includes a light source array 101 , a lens element 102 and a beam scanning element 103 .

The light source array 101 is used to generate pulsed beams; the lens element 102 is used to adjust the divergence of the pulsed beams;

The receiving module 200 includes a detection module and a collection module.

The detection module includes a single-photon avalanche diode image sensor 201, and the single-photon avalanche diode image sensor 201 is used to receive the reflected signal 20 of the pulsed beam 10, and output an electrical signal of the transit time of the round-trip pulsed beam 10 and the reflected signal 20.

The acquisition module includes a left image sensor 202 and a right image sensor 203 . The left image sensor 202 is used to collect the left image of the scene area 30 ; the right image sensor 203 is used to collect the right image of the scene area 30 .

The control and processing module 300 includes a processing module, a conversion module and a fusion module. The processing module, the conversion module and the fusion module can be independent modules that implement corresponding functions in each module, or can be an integrated processor that implements all functions in each module.

The processing module is configured to process the electrical signal of the transit time to obtain a first depth image, and convert the first depth image into a first parallax image.

The conversion module is used for stereo matching the left image and the right image to obtain a second parallax image.

The fusion module is used for fusing the first parallax image and the second parallax image to obtain the target image.

It should be noted that a fused depth measurement device in this embodiment is an embodiment of a measurement device corresponding to the above-mentioned fused depth measurement method. Therefore, the specific implementation of the software method in each module of the measurement device can be Referring to the embodiments in FIG. 1 to FIG. 3 , detailed descriptions are omitted here.

In the above-mentioned fusion depth measurement method and measurement device in the embodiment of the present invention, a first depth image is acquired by a single-photon avalanche diode image sensor, and the pixel points of the first depth image are used as seed points to guide the left image sensor and the right image sensor. The image sensor performs stereo matching to obtain a second parallax image, and by converting the first depth image into a first parallax image, and forming different weights according to the first credibility function and the second credibility function, to fuse the first The parallax image and the second parallax image are obtained to obtain a fusion parallax image to restore the three-dimensional reconstruction of the scene area, and obtain a high-resolution depth image with high recognition accuracy.

Various embodiments are described herein for various devices, circuits, apparatus, systems, and/or methods. Numerous specific details are set forth to provide a thorough understanding of the general structure, function, manufacture, and use of the embodiments as described in the specification and illustrated in the accompanying drawings. It will be understood by those skilled in the art, however, that embodiments may be practiced without such specific details. In other instances, well-known operations, components and elements have been described in detail so as not to obscure the implementations in the specification. It will be understood by those skilled in the art that the embodiments herein and shown are non-limiting examples, and therefore, it may be appreciated that the specific structural and functional details disclosed herein may be representative and do not necessarily limit the embodiments range.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the above-mentioned embodiments, those of ordinary skill in the art should understand that: it can still be used for the above-mentioned implementations. The technical solutions described in the examples are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions in the embodiments of the application, and should be included in the within the scope of protection of this application.

Claims (10)

1. A fusion depth measurement method for measuring the distance of a scene area, wherein the measurement method comprises:

   emitting a pulsed light beam to the scene area;

   receiving the reflected signal of the pulsed beam, and outputting an electrical signal of the transit time between the pulsed beam and the reflected signal;

   collecting left and right images of the scene area;

   processing the electrical signal of the transit time to obtain a first depth image, and converting the first depth image into a first parallax image;

   performing stereo matching on the left image and the right image to obtain a second parallax image;

   The first parallax image and the second parallax image are fused to obtain a target image.
2. The measurement method according to claim 1, characterized in that, transmitting a pulsed beam to the scene area comprises:

   generating the pulsed beam;

   adjusting the divergence of the pulsed beam;

   The pulsed light beam is directed to instruct the pulsed light beam to diverge in various directions to the scene area.
3. A fusion depth measurement device, comprising:

   The emission module is used to emit pulsed beams to the scene area;

   a detection module, configured to receive the reflected signal of the pulsed beam, and output an electrical signal of the transit time between the pulsed beam and the reflected signal;

   an acquisition module, configured to acquire the left image and the right image of the scene area;

   a processing module, configured to process the electrical signal of the transit time to obtain a first depth image, and convert the first depth image into a first parallax image;

   a conversion module that performs stereo matching on the left image and the right image to obtain a second parallax image; and

   A fusion module, configured to fuse the first parallax image and the second parallax image to obtain a target image.
4. The measuring device according to claim 3, wherein the transmitting module comprises:

   an array of light sources for generating pulsed light beams;

   a lens element for adjusting the divergence of the pulsed beam; and

   The beam scanning element is used for guiding the pulsed beam to indicate that the pulsed beam is diverged to various directions of the scene area.
5. The measurement device according to claim 4, wherein the detection module comprises a single photon avalanche diode image sensor;

   In the processing module, converting the first depth image into the first parallax image specifically includes:

   Using the left image as a reference image, calculate the first parallax image corresponding to the first depth image:

   

   In the formula, _PD (x ₀ , y ₀ ) is the disparity value of the depth value Z (x ₀ , y ₀ ) at the point (x ₀ , y ₀ ) on the first depth image, and f is the single The focal length of the photon avalanche diode image sensor, T _lt is the baseline length of the depth camera-left camera system, and H _lt is the homography matrix calibrated by the depth camera.
6. The measuring device according to claim 5, wherein, in the processing module, converting the first depth image into the first parallax image further comprises:

   The parallax surface of the first parallax image is fitted by the following binary cubic equation to obtain a smooth parallax surface:

   d(x,y)=a ₁ +a ₂ x+a ₃ y+a ₄ x ² +a ₅ xy+a ₆ y ² +a ₇ x ³ +a ₈ x ² y+a ₉ xy ² +a ₁₀ y ³

   In the formula, d(x, y) is a three-dimensional parallax surface, a ₁ , a ₂ , ···, a ₁₀ are coefficients, and x and y are pixel coordinates.
7. The measuring device according to claim 6, wherein, in the processing module, before converting the first depth image into a first parallax image, further comprising:

   Perform joint calibration on the single-photon avalanche diode image sensor and the left image sensor or the right image sensor:

   Converting the first depth image obtained by the single-photon avalanche diode image sensor into point cloud data, and mapping the point cloud data to the camera of the left image sensor or the right image sensor through a jointly calibrated transformation matrix In the coordinate system, a plane two-dimensional point with the left image sensor or the right image sensor as a reference is obtained.
8. The measuring device according to claim 7, wherein, in the conversion module, performing stereo matching on the left image and the right image to obtain the second parallax image specifically comprises:

   Select the pixel points on the first parallax image as seed points, and guide the left image and the right image to perform stereo matching to obtain the second parallax image, specifically using the following formula:

   

   In the formula, (x, y ₀ ) is the pixel point within each parallax range of the right image, and θ is the selected parameter.
9. The measuring device according to claim 8, wherein the first parallax image and the second parallax image are fused to obtain a depth image, which specifically includes:

   obtaining a first credibility function according to the first parallax image;

   obtaining a second credibility function according to the second parallax image;

   According to the different weights formed by the first credibility function and the second credibility function, the first parallax image and the second parallax image are fused to obtain pixel fusion parallax;

   Three-dimensional reconstruction is performed on the scene area according to the pixel fusion parallax to obtain the target image.
10. The measurement device according to claim 9, wherein the fusion of the first parallax image and the second parallax image to obtain pixel fusion parallax specifically includes:

    According to the first credibility function and the second credibility function, different weights are formed to fuse the two parallaxes to obtain the pixel fusion parallax. Specifically, the following formula is used:

    d=w _t ·d _t +w _s ·d _s

    

    w _t =1-w _s

    In the formula, d _t is the first disparity value, d _s is the second disparity value, w _t is the weight of the first disparity value, and _ws is the weight of the second disparity value.

Images