WO2022166723A1

WO2022166723A1 - Depth measurement method, chip, and electronic device

Info

Publication number: WO2022166723A1
Application number: PCT/CN2022/074100
Authority: WO
Inventors: 李明采
Original assignee: 深圳市汇顶科技股份有限公司
Priority date: 2021-02-08
Filing date: 2022-01-26
Publication date: 2022-08-11
Also published as: CN112954230A; CN112954230B

Abstract

Embodiments of the present application relate to the field of ranging, and disclosed therein are a depth measurement method, a chip, and an electronic device. The depth measurement method comprises: according to preset N exposure durations respectively corresponding to N phases, acquiring N subframe images of a target scene, N being a natural number greater than or equal to two; and determining depth information of the target scene according to the N subframe images. The N phases comprise a reference phase and an extended phase, and the N exposure durations comprise a reference exposure duration corresponding to the reference phase and an extended exposure duration corresponding to the extended phase; the reference exposure duration is less than or equal to the maximum duration for which an image in the reference phase and a preset reference distance is not overexposed; the extended exposure duration is between the reference exposure duration and the maximum duration for which an image in the extended phase and the reference distance is not overexposed; and the extended exposure duration is different from the reference exposure duration, so that power consumption can be reduced while improving the dynamic range of ranging.

Description

Depth measurement method, chip and electronic device

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on the Chinese patent application with the application number 202110184297.1 and the filing date on February 8, 2021, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is hereby incorporated by reference into this application.

technical field

The embodiments of the present application relate to the field of ranging, and in particular, to a depth measurement method, a chip, and an electronic device.

Background technique

The acquisition of depth information is currently widely used in many fields. Time of flight (TOF) technology is a way to obtain depth information. The principle is to calculate the distance from the measuring device (such as TOF camera) to the target object through the flight time of light in the air. The Metal Oxide Semiconductor (Complementary Metal Oxide Semiconductor, CMOS) pixel array is used as the light sensor at the receiving end, and the modulated light is used as the light source for measurement, for example, the modulated single pulse or continuously modulated light signal is emitted by the transmitting end, and the receiving end receives the target. The light reflected back by the object is calculated by calculating the phase difference between the emitted light and the received light to calculate the current distance of the object.

At present, in order to expand the dynamic range of the TOF camera ranging, the method used is to collect two exposure data for fusion. The two exposure data are the exposure data at low exposure and the exposure data at high exposure. Taking the four-phase time-of-flight ranging as an example, the exposure data at low exposure includes: 4 sub-phases collected at low exposure Frame image, the exposure data at high exposure includes: 4 sub-frame images of different phases collected at high exposure.

However, the inventor found that there are at least the following problems in the related art: in order to expand the dynamic range, it is necessary to collect two exposure data for fusion to obtain one frame of depth data, and the power consumption is relatively large.

SUMMARY OF THE INVENTION

The purpose of the embodiments of the present application is to provide a depth measurement method, a chip and an electronic device, so that the power consumption can be reduced while improving the dynamic range of ranging.

In order to solve the above technical problem, an embodiment of the present application provides a depth measurement method, including: acquiring N sub-frame images of a target scene according to preset N exposure durations corresponding to N phases respectively; wherein, the N is a natural number greater than or equal to 2, and the N phases are N phases with different phase differences from the phase of the emitted light; according to the N sub-frame images, determine the depth information of the target scene; wherein, the The N phases include a reference phase and an extended phase, and the N exposure durations include a reference exposure duration corresponding to the reference phase and an extended exposure duration corresponding to the extended phase; the reference exposure duration is less than or equal to the reference exposure duration in the reference phase. The maximum duration for which the image is not exposed under the phase and the preset reference distance, and the extended exposure duration is between the reference exposure duration and the maximum duration for which the image is not exposed under the extended phase and the reference distance.

Embodiments of the present application further provide a depth measurement method, including: emitting emission light for depth measurement at a reference phase; collecting a first subframe image of a target scene according to a first phase and a reference exposure duration; wherein the The phase difference between the first phase and the reference phase is 0 degrees; the second subframe image of the target scene is collected according to the second phase; wherein, the phase difference between the second phase and the reference phase is 180 degrees; The third subframe image of the target scene is collected in three phases; wherein, the phase difference between the third phase and the reference phase is 90 degrees; the fourth subframe image of the target scene is collected according to the fourth phase; The phase difference between the four-phase and the reference phase is 270 degrees; wherein, the exposure duration used to collect the second subframe image, the exposure duration used to collect the third subframe image, and the exposure duration used to collect the fourth subframe image The exposure duration used by a frame image is respectively greater than the reference exposure duration, and the first subframe image, the second subframe image, the third subframe image and the fourth subframe image are used to determine a Frame depth image.

Embodiments of the present application further provide a chip, which is set in an electronic device, the chip is connected to a memory in the electronic device, and the memory stores an instruction that can be executed by the chip, and the instruction is executed by the device. The chip is executed, so that the chip can execute the above-mentioned depth measurement method.

Embodiments of the present application also provide an electronic device, including the above-mentioned chip and a memory connected to the chip.

In this embodiment of the present application, N sub-frame images of the target scene are acquired according to the preset N exposure durations corresponding to the N phases, where N is a natural number greater than or equal to 2, and the N phases are the phase differences from the phase of the emitted light. N different phases; according to the N sub-frame images, the depth information of the target scene is determined; the N phases include the reference phase and the extended phase, and the N exposure durations include the reference exposure duration corresponding to the reference phase and the extended phase Extended exposure duration; the reference exposure duration is less than or equal to the maximum duration that the image will not be exposed under the reference phase and the preset reference distance, to ensure that the sub-frame images of the acquired target scene will not be exposed according to the reference exposure duration corresponding to the reference phase. The extended exposure duration is between the reference exposure duration and the maximum duration that the image will not be exposed under the extended phase and the reference distance, ensuring that the sub-frame images of the acquired target scene will not be exposed according to the extended exposure duration corresponding to the extended phase. In addition, since the extended exposure duration is different from the reference exposure duration, it is beneficial to determine the depth information at different distances in the target scene under different exposure durations. In the extended exposure time and the reference exposure time, a longer exposure time is conducive to accurately determining the depth information of the distant point in the target scene, and a shorter exposure time is conducive to accurately determining the target scene. The closer point is the depth information of the near point, which is beneficial to improve the dynamic range of ranging without exposure, and because one exposure duration is used in each phase to obtain one sub-frame image, it is not necessary to The phase adopts 2 different exposure durations to obtain 2 sub-frame images, so there is no need to fuse the sub-frame images obtained based on 2 different exposure durations for each phase, so the dynamic range of ranging can be improved while reducing the range. Small power consumption.

In addition, the method for determining the maximum duration of no exposure of the image in the extended phase and the reference distance is as follows: within the same exposure duration, the number of photons of the sub-frame image in the reference phase and the extended duration are obtained respectively. The number of photons of the sub-frame image in the phase; Calculate the first ratio of the photon number of the sub-frame image in the reference phase to the photon number of the sub-frame image in the extended phase; Based on the modulation frequency of the optical signal in the depth measurement , the extended phase and the reference distance, calculate the farthest measurement distance at which the number of photons of the subframe image under the extended phase reaches the limit value; Calculate the second ratio of the farthest measurement distance and the reference distance Square; select the smallest value among the squares of the first ratio and the second ratio, and multiply the smallest value by the reference phase and the maximum duration of the image under the preset reference distance without exposure, It is the maximum duration for which the image is not exposed under the extended phase and the reference distance. By taking the product of the smallest value among the squares of the first ratio and the second ratio and the maximum duration of the image not being exposed under the reference phase and the preset reference distance as the maximum duration of the image not being exposed under the extended phase and the reference distance, it is beneficial to Considering the ratio relationship between different dimensions, one dimension is to ensure that the image is not exposed when measuring close range, and the other dimension is to ensure that the image is not exposed when measuring long distance, so that the image can be more reasonably determined under the extended phase and reference distance. The maximum duration allows the final extended phase and the maximum duration of the image not to be exposed at the reference distance to meet the requirements of non-exposure when measuring close range and non-exposure when measuring long distance.

In addition, the spread phase includes two of the first type spread phases and one of the second type spread phases, and the phase difference between the two first type spread phases is π. That is, in the embodiment of the present application, according to the preset four exposure durations corresponding to the four phases, four sub-frame images of the target scene are acquired, and the depth information of the target scene is determined by using the four sub-frame images, and the four-phase sampling method is adopted. In this way, ranging accuracy, power consumption and speed can be taken into account at the same time, that is, the power consumption and speed are not greatly affected while the ranging accuracy is improved.

In addition, the depth information of the target scene includes depth information of multiple pixels in the target scene, and after the determining the depth information of the target scene according to the N sub-frame images, further includes: according to the N sub-frame images sub-frame images, and determine multiple confidence levels corresponding to the depth information of the multiple pixels; If the extended exposure duration does not reach the maximum duration for which the image is not exposed under the extended phase and the reference distance, the extended exposure duration is increased. It is understandable that the depth measurement accuracy of several pixels in the target scene is low, indicating that the extended exposure time setting is more likely to be unreasonable, and the image is not exposed when the extended exposure time does not reach the extended phase and the reference distance. The maximum duration indicates that the current extended exposure duration still has room to increase. Therefore, in this embodiment, the number of confidence levels that are less than the preset confidence level threshold among the multiple confidence levels exceeds the preset number threshold and the extended exposure duration does not reach the maximum duration of the extended phase and the reference distance that the image will not be exposed. Large extended exposure duration can increase the extended exposure duration at a reasonable time. It is not necessary to set the extended exposure duration longer at the beginning, which is beneficial to obtain the minimum extended exposure duration that can make the confidence greater than the confidence threshold. The power consumption is further reduced while the dynamic square range and ranging accuracy are improved.

In addition, the emission light used for depth measurement is emitted in the reference phase; the first sub-frame image of the target scene is collected according to the first phase and the reference exposure duration; wherein, the phase difference between the first phase and the reference phase is 0 degree ; Acquire the second subframe image of the target scene according to the second phase; wherein, the phase difference between the second phase and the reference phase is 180 degrees; Acquire the third subframe image of the target scene according to the third phase; wherein, The phase difference between the third phase and the reference phase is 90 degrees; the fourth subframe image of the target scene is collected according to the fourth phase; wherein, the phase difference between the fourth phase and the reference phase is 270 degrees; Wherein, the exposure duration used for collecting the second subframe image, the exposure duration used for collecting the third subframe image, and the exposure duration used for collecting the fourth subframe image are respectively greater than the reference exposure duration , the first subframe image, the second subframe image, the third subframe image and the fourth subframe image are used to determine a frame of depth image. In the embodiment of the present invention, one frame of depth image can be determined based on four sub-frame images, because the exposure duration used for collecting the second sub-frame image, the exposure duration used for collecting the third sub-frame image and the fourth The exposure durations used by the subframe images are respectively longer than the reference exposure duration, so the depth information of the distant point in the target scene can be determined based on the second, third, and fourth subframe images compared with the first subframe image. The image can determine depth information for near points in the target scene. Therefore, in this embodiment, the depth information of the far point and the depth information of the near point in the target scene can be obtained at the same time based on the four sub-frame images, thereby greatly improving the frame rate and reducing the dynamic range of distance measurement while improving the dynamic range. power consumption. At the same time, the depth image determined based on the four sub-frame images can also improve the measurement accuracy and also ensure that the time required for measurement is not very long, that is, the speed of determining a frame of depth image is increased.

Description of drawings

One or more embodiments are exemplified by the pictures in the corresponding drawings, and these exemplifications do not constitute limitations of the embodiments, and elements with the same reference numerals in the drawings are denoted as similar elements, Unless otherwise stated, the figures in the accompanying drawings do not constitute a scale limitation.

1 is a block diagram of the electronic device for depth measurement mentioned in the embodiment of the present application;

2 is a waveform diagram of an optical signal transmitted, an optical signal received, and an optical signal sampled based on 4 phases in the related art mentioned in the embodiments of the present application;

3 is a flowchart of the depth measurement method mentioned in the embodiment of the present application;

4 is a schematic diagram of different exposure durations corresponding to 4 phases when 0° is the reference phase and the reference distance is 0 mentioned in the embodiment of the present application;

5 is a schematic diagram of different exposure durations corresponding to 4 phases when 0° is the reference phase and the reference distance is greater than 0 mentioned in the embodiment of the present application;

6 is a flowchart of a method for determining the maximum duration that the image is not exposed under the extended phase and the reference distance mentioned in the embodiment of the present application;

7 is a flowchart of the depth measurement method in an example mentioned in the embodiment of the present application;

8 is a flowchart of a depth measurement method mentioned in an embodiment of the present application;

FIG. 9 is a schematic structural diagram of the electronic device mentioned in the embodiment of the present application.

Detailed ways

In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, those of ordinary skill in the art can understand that, in each embodiment of the present application, many technical details are provided for the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be realized. The following divisions of the various embodiments are for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the various embodiments may be combined with each other and referred to each other on the premise of not contradicting each other.

The embodiment of the present application relates to a depth measurement method, which is applied to an electronic device. The electronic device is used to measure depth information, and the depth information can be understood as the distance between the object to be measured and the electronic device. The electronic device may be a TOF ranging device, and may be a TOF camera in a specific implementation. The depth information measured by the TOF camera can be understood as the distance between the object to be measured and the TOF camera. The implementation details of the depth measurement method in this embodiment will be specifically described below, and the following contents are only provided for the convenience of understanding, and are not necessary for implementing this solution.

In order to facilitate the understanding of this embodiment, the depth measurement principle involved in this embodiment is described below:

The depth measurement technology involved in this embodiment is to periodically modulate the emitted optical signal, by measuring the phase delay of the reflected optical signal relative to the emitted optical signal, and then calculating the depth according to the phase delay and the speed of light. This measurement technology It can be called indirect-time-of-flight (indirect-TOF, iTOF) technology. For the module diagram of the electronic device for depth measurement, refer to FIG. 1 , including: a transmitting module 101 , a receiving module 102 , and a processing module 103 .

The transmitting module 101, for transmitting the optical signal modulated based on the modulation frequency f, may use a laser transmitter as a light source.

The receiving module 102 is used to receive the light reflected by the target object (reflected light for short), and obtain sub-frame images corresponding to different phases by sampling the reflected light at different phases. The receiving module mainly uses a CMOS image sensor to detect the reflected light signal. . An image sensor may also be referred to as a photodetector, which generally includes a plurality of pixels distributed in an array. For example, in the commonly used 4-phase sampling method, through detection by the pixel array of the image sensor, 4 different phase subframes, that is, 4 subframe images with different phases, can be obtained. For each pixel in the 4 subframe images, it corresponds to The respective photon numbers are Q1, Q2, Q3, and Q4, respectively. In the related art, reference may be made to FIG. 2 for waveform diagrams of the transmitted optical signal, the received optical signal, and the optical signal sampled based on 4 phases, that is, the 4 phases adopt the same exposure duration. The above phase can be understood as the phase offset of the starting position of the integration window relative to the starting position of the emission window, wherein the phase corresponding to the starting position of the emission window can be regarded as 0° as a reference, and the integration window can also be called the receive window. The four phases in Figure 2 are 0°, 90°, 180°, and 270°, respectively. From Figure 2, it can be seen that the phase offset of the starting position of the integration window of 0° relative to the starting position of the emission window is 0°, that is, the integration window of 0° and the emission window are completely coincident; the phase offset of the starting position of the integration window of 90° relative to the starting position of the emission window is 90°; the starting position of the integration window of 180° The phase offset relative to the starting position of the emission window is 180°; the phase offset of the starting position of the 270° integration window relative to the starting position of the emission window is 270°.

The integration window can be understood as: the window occupied by the high level of the working timing diagram of the photosensitive control switch of the pixel in the photodetector. The integration window includes an effective integration window and an invalid integration window. The valid integration window is the window in the integration window that actually receives the reflected light, that is, the shadow part of each integration window in Figure 2; the invalid integration window is the integration window that does not actually receive the light. The window for reflected light, i.e. the unshaded portion of each integration window in Figure 2. In the waveform diagram corresponding to each phase sub-frame in Figure 2, the photosensitive control switch is turned on to a high level, and the high level indicates that it is allowed to start receiving reflected light. After the specified exposure duration ends, one phase subframe sampling is completed. In the specific implementation, there are usually multiple integration windows within the specified exposure duration. For example, there are 3 integration windows in the exposure duration corresponding to the 270° subframe in Figure 2, and the number of photons received in each integration window is q4, 3 After each integration window has received the number of photons, it can be considered that the 270° phase subframe sampling is completed, and one phase subframe can also be referred to as a subframe image. A sub-frame image can also be understood as an image formed by each pixel after all integration windows are integrated within a specified exposure duration.

The processing module 103 is configured to send the image acquisition command to the receiving module 102, and the receiving module 102 sends the image acquisition command to the transmitting module 101, so that the transmitting module 101 emits an optical signal. The processing module 103 is further configured to receive the phase data sent by the receiving module 102, the phase data may include the image data of the 4 subframe images obtained by the above-mentioned 4 phase downsampling, and the image data may be embodied as the number of photons of the 4 subframe images. , denoted as Q1, Q2, Q3, Q4 in turn. Therefore, the processing module 103 can calculate the depth d by the following formula:

Q=sinφ=Q3-Q4; Q3=n3*q3, Q4=n4*q4

I=cosφ=Q1-Q2; Q1=n1*q1, Q2=n2*q2

tanφ=sinφ/cosφ=(Q3-Q4)/(Q1-Q2)

A=1/2(Q ² +I ² ) ^1/2

d=(c/2f)×(φ/2π)

Among them, f is the modulation frequency of the optical signal, φ is the phase delay of the reflected optical signal relative to the emitted optical signal, c is the speed of light, and A is the confidence of the measured depth d. Referring to FIG. 2 , q1 is the number of photons received by the receiving module 102 in each integration window in the 0° phase subframe, n1 is the number of integration windows in the exposure duration corresponding to the 0° phase subframe, and Q1 is the 0° phase subframe. The sum of the number of photons received by the receiving module 102 in all integration windows in a frame. q2 is the number of photons received by the receiving module 102 in each integration window in the 180° phase subframe, n2 is the number of integration windows in the exposure duration corresponding to the 180° phase subframe, and Q2 is the receiving module in the 180° phase subframe 102 The sum of the number of photons received in all integration windows. q3 is the number of photons received by the receiving module 102 in each integration window in the 90° phase subframe, n3 is the number of integration windows in the exposure duration corresponding to the 90° phase subframe, and Q3 is the receiving module in the 90° phase subframe 102 The sum of the number of photons received in all integration windows. q4 is the number of photons received by the receiving module 102 in each integration window in the 270° phase subframe, n4 is the number of integration windows in the exposure duration corresponding to the 270° phase subframe, and Q4 is the receiving module in the 270° phase subframe 102 The sum of the number of photons received in all integration windows. In FIG. 2, n1=n2=n3=n4=3.

In a specific implementation, each pixel on the photodetector in the receiving module 103 can independently obtain the number of photons, and the processing module 103 can calculate the depth information of a pixel in the target scene based on a pixel on the photodetector. Multiple pixels on the detector can calculate the depth information of multiple pixels in the target scene. Among them, the number of photons acquired by the pixel can be understood as: the reflected light signal is imaged on the pixel to accumulate photocharge during the exposure time, that is, the number of photons generated by the accumulated reflected light in the pixel during the exposure time can also be understood as: The sum of the number of photons received under all integration windows during the exposure duration.

In one implementation, two exposure data (4 sub-frame images of low exposure + 4 sub-frame images of high exposure) are used for fusion to expand the dynamic measurement range, and the 4 phases under low exposure use the same exposure duration, and the high exposure The same exposure duration is used for the 4 phases under exposure. The difference between high exposure and low exposure is that the exposure time used in high exposure is longer than the exposure time used in low exposure. That is, the exposure data is collected once, and the exposure durations corresponding to different phases remain the same. Assuming that the exposure durations corresponding to 0°, 180°, 90°, and 270° are t1, t2, t3, and t4, respectively, then t1=t2=t3=t4. A total of 8 subframe images need to be collected, and 2 different exposure durations (the exposure duration of low exposure and the exposure duration of high exposure) are used in each phase to obtain 2 subframe images, and 8 subframe images can be obtained in 4 phases. In order to improve the dynamic range, each phase needs to be sampled based on 2 different exposure durations to obtain 2 sub-frame images, and 8 sub-frame images obtained based on 4 phase sampling are fused to output a frame of depth information, but this significantly reduces the The frame rate increases and the power consumption increases. Among them, the fusion of 8 sub-frame images can be understood as: obtaining a frame of low-exposure depth information based on the low-exposure 4 sub-frame images, obtaining a frame of high-exposure depth information based on the high-exposure 4 sub-frame images, and then combining the low-exposure depth information The depth information of , and the depth information of high exposure are fused to obtain the final measured depth information.

In this embodiment, in order to reduce power consumption while expanding the dynamic measurement range, the following depth measurement method is provided. Refer to FIG. 3 for a flowchart of the depth measurement method in this embodiment, including:

Step 301: Acquire N sub-frame images of the target scene according to the preset N exposure durations corresponding to the N phases respectively.

Step 302: Determine the depth information of the target scene according to the N subframe images.

Wherein, N is a natural number greater than or equal to 2, and the N phases are N phases with different phase differences from the phase of the emitted light. In each phase, one exposure duration is used to obtain one sub-frame image, and it is not necessary to adopt two different exposure durations for each phase to obtain two sub-frame images. In this embodiment, in order to improve the dynamic range, each phase is sampled based on one exposure duration to obtain one subframe image, and one frame of depth information is output based on N subframe images collected under N different phases.

In one example, N is a natural number greater than or equal to 2 and less than 8, that is, at most one frame of depth information can be output according to 7 subframe images obtained based on 7 different phase samplings. Fusion to output a frame of depth information can increase the frame rate and reduce power consumption.

In an example, N=4, this embodiment can output one frame of depth information according to 4 subframe images obtained based on 4 different phase samplings, that is, determine the target scene according to the 4 subframe images obtained in sequence depth information. Considering that the more phases, the higher the measurement accuracy, but at the same time, the power consumption and measurement time will be relatively more, and the frame rate will be reduced accordingly. N is 4 to balance the dynamic range, frame rate, measurement accuracy, and power consumption. As well as the time it takes to measure, to a certain extent, it can improve the frame rate and measurement accuracy while improving the dynamic range, and it can also ensure that the power consumption will not be large, and the measurement time will not be very long.

The N different phases include the reference phase and the extended phase, and the N exposure durations include the reference exposure duration corresponding to the reference phase and the extended exposure duration corresponding to the extended phase; the reference exposure duration is less than or equal to the reference phase and the preset reference distance. The maximum duration for which the lower image will not be exposed, to ensure that the sub-frame images of the acquired target scene will not be overexposed according to the benchmark exposure duration corresponding to the benchmark phase. The extended exposure duration is between the reference exposure duration and the maximum duration that the image will not be exposed under the extended phase and the reference distance, ensuring that the sub-frame images of the acquired target scene will not be overexposed according to the extended exposure duration corresponding to the extended phase.

In specific implementation, the reference phase can be selected according to actual needs, and the reference phase corresponds to the reference distance. For example, when the reference phase is 0°, the reference distance corresponding to 0° is the shortest distance range0 of the application ranging, and range0 can be understood as the shortest distance that needs to be measured in the ranging scenario applied by the TOF ranging device. It can also be understood that, assuming that the closest distance of the application ranging is represented as range0, then referring to Figure 4, when the integration window is completely aligned with the emission window, the phase is recorded as the reference phase 0°, that is, the reference phase is the phase difference with the emitted light phase is 0° phase. As can be seen from Figure 4, range0 is 0 meters, and the number of photons received in the integration window of the 0° phase subframe is full photons, that is to say, the integration windows of the 0° phase subframe are all valid integration windows, and the 180° phase subframe is an effective integration window. The number of photons received in the integration window of the bit subframe is 0, that is to say, the integration windows of the 180° phase subframe are invalid integration windows. Among them, full photons can be understood as the total number of photons emitted in an emission window. In FIG. 4, Q1=3q1, Q2=0, Q3=6q3, Q4=5q4.

In one example, the closest distance range0 to which ranging is applied is greater than 0, such as may be 0.3 meters. When range0 is greater than 0, you can refer to Figure 5. The number of photons received in the integration window of the 0° phase subframe is less than full photons, that is to say, the integration windows of the 0° phase subframe are not all valid integration windows but also invalid ones. Integration window. The number of photons received in the integration window of the 180° phase subframe is greater than 0, that is to say, the integration windows of the 180° phase subframe are not all invalid integration windows but also include a small part of the valid integration windows. The shaded parts in FIG. 5 are all effective integration windows. It should be noted that since the effective integration window of the 180° phase subframe is very small, q2 in FIG. 5 is not standard in the shaded area. In FIG. 5, Q1=3q1, Q2=6q2, Q3=6q3, Q4=5q4.

For the convenience of description, the maximum duration for which the image is not exposed under the reference phase and the preset reference distance is abbreviated as t0, and the reference exposure duration is abbreviated as t1, and t1 is less than or equal to t0. In one example, t0 can be determined as follows:

Under the reference distance, adjust the distance between the starting point of the integration window of the reference phase and the starting point of the emitted optical signal to the reference phase, gradually increase the preset initial exposure duration, and obtain the The number of photons in the reference phase, until the number of photons acquired is the same as the number of photons acquired last time, that is, the number of photons acquired does not increase, indicating that the number of photons has reached the limit value, and the exposure duration based on the number of photons acquired last time is taken as t0. It is understandable that the TOF ranging device includes a photosensitive chip. The number of photons that the photosensitive chip can receive is limited. If the limit value is reached, the photosensitive chip is considered to be saturated. If the exposure time is increased, the number of photons obtained will no longer be obtained. Increase. Therefore, t0 can be understood as the critical exposure duration that prevents the number of photons from increasing as the exposure duration increases.

For example, when the reference phase is 0° and the reference distance is range0, t0 can be determined by:

In range0, the starting point of the integration window of the reference phase is adjusted to be separated from the starting point of the emitted optical signal by 0°, that is, the starting point of the integration window of the reference phase is aligned with the starting point of the emitted optical signal. For example, referring to Figure 4, align the 0° frame to the modulated wave emitted by the transmitter module, so that the integration window of the 0° frame is the largest, and then increase the exposure time from small to large, for example, the exposure time gradually increases from 1ms to 10ms. After the photosensitive chip receives The number of photons will not increase any more. It can be considered that the maximum duration of image overexposure at 0° phase and range0 is 10ms, that is, t0=10ms, and the exposure duration t1 corresponding to 0° can be set to 10ms, that is, t1=t0.

In an example, the method of determining the maximum duration of the image without exposure under the extended phase and the reference distance can refer to FIG. 6 , including:

Step 601 : within the same exposure duration, obtain the photon number of the sub-frame image under the reference phase and the photon number of the sub-frame image under the extended phase, respectively.

Step 602: Calculate a first ratio of the photon number of the subframe image in the reference phase to the photon number of the subframe image in the extended phase.

Step 603: Calculate the farthest measurement distance at which the number of photons of the subframe image under the extended phase reaches a limit value based on the modulation frequency, the extended phase and the reference distance of the optical signal in the depth measurement.

Step 604: Calculate the square of the second ratio of the farthest measured distance to the reference distance.

Step 605: Select the smallest value among the squares of the first ratio and the second ratio, and use the product of the smallest value and the reference phase and the maximum duration of the image not to be exposed under the preset reference distance as the extended phase and the reference distance. The maximum length of time that an image will not be exposed.

For ease of understanding, the reference phase is taken as an example of 0°, and the following takes the extended phase of 90° and 180° as examples to illustrate the method of determining the maximum duration of the image without exposure under the extended phase and the reference distance:

The maximum duration for which the image is not exposed under the extended phase of 90° and the reference distance is abbreviated as t3, and the determination method of t3 is as follows:

First of all, referring to FIG. 4 , it can be seen that within the same exposure duration (such as T), the number of photons in the subframe image at 90° is at most half of the number of photons in the subframe image at 0°, that is, the above-mentioned first ratio is 2.

Secondly, assuming that the modulation frequency of the optical signal in the depth measurement is f=100Mhz, the maximum measurement distance d=c/2f=1.5m when f=100Mhz, and assuming that the reference distance is 0.3m, the number of photons at 90° reaches the maximum limit value. The far measurement distance is: 0.3+1.5/(90°/360°)=0.3+1.5/4=0.675m. Then the second ratio is: 0.675/0.3=2.25, and the square of the second ratio is approximately equal to 5.

Finally, select the smallest value, namely 2, between the first ratio (2) and the square of the second ratio (5), then t3=2t0, and when the reference exposure duration t1=t0, t3=2t1. That is, t3 is between t1 and 2t1, and t1<t3≤2t1.

The maximum duration for which the image is not exposed under the extended phase of 180° and the reference distance is abbreviated as t2, and the determination method of t2 is as follows:

First, referring to FIG. 4 , it can be seen that within the same exposure duration (such as T), the number of photons in a sub-frame image at 180° is 0, that is, no photons are received, so the above-mentioned first ratio is theoretically infinite.

Secondly, assuming that the modulation frequency of the optical signal in the depth measurement is f=100Mhz, the maximum measurement distance d=c/2f=1.5m when f=100Mhz, and assuming that the reference distance is 0.3m, the number of photons at 180° reaches the maximum limit of the limit value. The far measurement distance is: 0.3+1.5/(180°/360°)=0.3+1.5/2=1.05m≈1m. Then the square of the second ratio is: 1/0.3≈3, and the square of the second ratio is approximately equal to 9.

Finally, select the smallest value 9 among the first ratio (infinity) and the square of the second ratio (9), then t2=9t0, and when the reference exposure duration t1=t0, t2=9t1. That is, t2 is between t1 and 9t1, and t1<t2≤9t1.

It should be noted that the above example is only for the convenience of understanding that the expansion phases are selected to be 90° and 180° respectively. In specific implementation, when selecting other phases, the maximum duration of the image without exposure can also be calculated based on the above method. For example, if 270° is selected, the maximum duration for which the image is not exposed under the extended phase of 270° and the reference distance is abbreviated as t4, and it can be calculated that t1<t4≤2t1. If 45° is selected, the maximum duration for which the image is not exposed under the extended phase of 45° and the reference distance is abbreviated as t5, and it can be calculated that t1<t5≤4/3t1.

It can be understood that, according to the principle of modulated light, on range0, the number of photons received by the receiving window of the 0° phase subframe (also called the integration window) is the largest, and the corresponding 90° and 270° receive at most 1/2 of the photons. Photons, that is, the above-mentioned first ratio is 2, so the maximum can be set to t3=t4=2t1. On range0, by calculating the first ratio, the determined t3 and t4 are beneficial to ensure that the measurement at close range is not overexposed.

In addition, on the basis of the reference distance, as the distance increases, the number of photons received in the receiving window of the 180° phase subframe will gradually increase. Considering the light emitted by a point light source, at any distance from the light source The illuminance decays with the square of the distance, that is, the decay of the number of photons is inversely proportional to the square of the distance. Therefore, by calculating the square of the above-mentioned second ratio, the determined extended phase of 180° and the maximum duration t2 during which the image is not exposed at the reference distance is beneficial to ensure that the image is not exposed when measuring long distances.

By taking the product of the smallest value among the squares of the first ratio and the second ratio and the maximum duration of the image not being exposed under the reference phase and the preset reference distance as the maximum duration of the image not being exposed under the extended phase and the reference distance, it is beneficial to Considering the ratio relationship between different dimensions, one dimension is to ensure that the image is not exposed when measuring close range, and the other dimension is to ensure that the image is not exposed when measuring long distance, so that the image can be more reasonably determined under the extended phase and reference distance. The maximum duration allows the final extended phase and the maximum duration of the image not to be exposed at the reference distance to meet the requirements of non-exposure when measuring close range and non-exposure when measuring long distance.

In one example, the spread phase includes a first type of spread phase and/or a second type of spread phase. The following describes how to determine the maximum duration of the image without exposure under the first type of extended phase and the reference distance, and the method for determining the maximum duration of the image without exposure under the second type of extended phase and reference distance:

The phase difference between the first type of extended phase and the reference phase is greater than 0 and less than or equal to π/2. The first type of extended phase and the maximum duration of the image without exposure at the reference distance is determined as follows: TOF ranging equipment is exposed at the same exposure. During the time period, obtain the photon number of the sub-frame image under the reference phase and the photon number of the sub-frame image under the first type of extended phase respectively, and calculate the photon number of the sub-frame image under the reference phase and the sub-frame image under the first type of extended phase. The first ratio of the number of photons of the frame image, and the product of the first ratio and the reference phase and the maximum duration of the image under the preset reference distance is not exposed as the maximum duration of the image under the extended phase and the reference distance. It can be understood that the manner of determining the above-mentioned first ratio is similar to steps 501 to 502, and details are not repeated here.

For example, if the reference phase is 0°, the first type of extended phase can be selected in two ranges of [270°, 0°) and (0°, 90°].

The phase difference between the second type of extended phase and the reference phase is greater than π/2 and less than or equal to π. The second type of extended phase and the maximum duration of the image under the reference distance without exposure is determined as follows: based on the optical signal in the depth measurement. Modulation frequency, the second type of extended phase and the reference distance, calculate the farthest measurement distance where the number of photons of the subframe image under the second type of extended phase reaches the limit value; calculate the square of the second ratio of the farthest measurement distance to the reference distance, The product of the square of the second ratio and the maximum duration of no exposure of the image under the reference phase and the preset reference distance is taken as the maximum duration of the image under the second type of extended phase and the reference distance without exposure. It can be understood that the manner of determining the square of the second ratio is similar to that in steps 503 to 504 , and details are not repeated here.

For example, if the reference phase is 0°, the second type of extended phase can be selected within the interval range of (90°, 180°).

By distinguishing the types of extended phases, namely the first type of extended phase and the second type of extended phase, different types of extended phases use different methods to calculate the maximum duration of the image without exposure, which can be combined with the characteristics of different types of extended phases. Calculates a ratio that can more quickly and reasonably calculate the maximum time the image will not be exposed.

In one example, N is 4, the 4 different phases include: a reference phase and three extension phases, the extension phases include two first-type extension phases and one second-type extension phase, and two first-type extension phases The phase difference between is π. For example, if the reference phase is 0°, the two first-type extended phases can be selected in the range of [270°, 0°), (0°, 90°], for example, 90° and 270° are selected. . The second type of extended phase is selected within the range of (90°, 180°], for example, 180° is selected. Referring to Figure 4, the reference exposure duration corresponding to 0° is t1, and t1 can be equal to the image at 0° and range0. The maximum duration of no exposure. The exposure durations corresponding to 180°, 90°, and 270° are set as t2, t3, and t4, respectively, where t1<t3=t4≤2t1; t1<t2≤9t1. In this example, the above method is used to set The 4-phase exposure time of the 4-phase can ensure that the close distance will not be overexposed, and the long distance because 90°, 270°, and 180° increase the exposure time compared to t1, so the dynamic range of ranging is improved, and the maximum can be increased. And it is equivalent to only need to collect 4 sub-frame images, compared with the same kind of high dynamic range scheme (8 sub-frame images are collected, and the fusion of 4 sub-frame images with high exposure and 4 sub-frame images with low exposure is required), The frame rate is greatly improved (only 4 subframes are collected, no need to collect and merge twice), and the power consumption is greatly reduced.

In an example, when the extended phase is greater than the reference phase, and the extended exposure duration is between the reference exposure duration and the maximum duration of no-exposure images under the extended phase and the reference distance, it means: the extended exposure duration is greater than the reference exposure duration and less than or equal to The maximum length of time that the image will not be exposed under the extended phase and reference distance. For example, when the reference phase is 0°, the extension phase is 90°, the reference exposure duration is t1, and the extension exposure duration is t3, then t3 is between the reference exposure duration t1 and the maximum duration 2t1 where the image is not exposed under the extension phase and reference distance. Between means: t1<t3≤2t1.

In another example, when the extended phase is smaller than the reference phase, and the extended exposure duration is between the reference exposure duration and the maximum duration of no image exposure under the extended phase and the reference distance, it means: the extended exposure duration is greater than or equal to the extended phase and the reference distance. The maximum length of time that the image will not be exposed at the reference distance and less than the reference exposure time. For example, if the reference phase is 90°, the extension phase is 0°, the reference exposure duration is t3, and the extension exposure duration is t1, then t1 is between the reference exposure duration t3 and the maximum duration t3/ Between 2 means: t3/2≤t1<t3.

It should be noted that Fig. 4 only takes 4-phase sampling as an example. In the specific implementation, it is not limited to 4-phase sampling, but can also be 2-phase sampling (such as 0° and 90°), 8-phase sampling (such as 0°) , 45°, 90°, 135°, 180°, 225°, 270°, 315°), etc. 4-phase sampling can take into account ranging accuracy, power consumption and speed at the same time, that is, it will not have much impact on power consumption and speed while improving ranging accuracy.

In one example, if it is 2-phase sampling, in step 301, the TOF ranging device acquires 2 sub-frame images of the target scene according to the preset 2 exposure durations corresponding to the 2 phases respectively. In step 302, the TOF ranging device determines the depth information of the target scene according to the two subframe images. For example, two subframe images are called 0° phase subframe and 90° phase subframe respectively, the image data of the 0° phase subframe is the number of photons Q1, and the image data of the 90° phase subframe is the number of photons Q3. The following formula calculates the depth information in the target scene:

tanφ=sinφ/cosφ=Q3/Q1

d=(c/2f)×(φ/2π)

Among them, d is the calculated depth, f is the modulation frequency of the optical signal, φ is the phase delay of the reflected optical signal relative to the emitted optical signal, and c is the speed of light.

In another example, if it is 4-phase sampling, in step 301, the TOF ranging device acquires 4 sub-frame images of the target scene according to the preset 4 exposure durations corresponding to the 4 phases respectively. In step 302, the depth information of the target scene is determined according to the four subframe images. For example, the 4 subframe images may be respectively referred to as a 0° phase subframe, a 180° phase subframe, a 90° phase subframe, and a 270° phase subframe. Referring to Figure 4, the image data of the four phase subframes are: Q1, Q2, Q3, Q4, and the depth information in the target scene can be calculated by the following formula:

tanφ=sinφ/cosφ=(Q3-Q4)/(Q1-Q2)

d=(c/2f)×(φ/2π)

In one example, the flowchart of the depth measurement method can refer to FIG. 7, including:

Step 701: Emit the emission light for depth measurement at the reference phase.

Wherein, the reference phase may be 0° mentioned in the above example, but is not limited thereto. The TOF ranging device can emit emitted light for depth measurement in a reference phase.

Step 702: Collect a first sub-frame image of the target scene according to the first phase and the reference exposure duration.

The phase difference between the first phase and the reference phase is 0 degrees. In an example, when the reference phase is 0°, the first phase is 0°, and the first subframe image is the above-mentioned 0° phase subframe. Since the difference between the reference phase and the first phase is 0 degrees, the first phase can also be understood as the reference phase.

In a specific implementation, the reference exposure duration is less than or equal to the maximum duration during which the image is not exposed under the reference phase and the preset reference distance. The preset reference distance can be understood as the closest distance to be measured in the ranging scenario applied by the above TOF ranging device, and can also be understood as the minimum detection distance of the depth measurement method.

Step 703: Collect a second subframe image of the target scene according to the second phase.

The phase difference between the second phase and the reference phase is 180 degrees. In an example, when the reference phase is 0°, the second phase is 180°, and the second subframe image is the aforementioned 180° phase subframe.

Step 704: Collect a third subframe image of the target scene according to the third phase.

The phase difference between the third phase and the reference phase is 90 degrees. In an example, when the reference phase is 0°, the third phase is 90°, and the third subframe image is the above-mentioned 90° phase subframe.

Step 705: Collect a fourth subframe image of the target scene according to the fourth phase.

The phase difference between the fourth phase and the reference phase is 270 degrees. In an example, when the reference phase is 0°, the fourth phase is 270°, and the fourth subframe image is the above-mentioned 270° phase subframe.

In a specific implementation, the exposure duration (eg, t2 in FIG. 4 ) used for collecting the second subframe image, the exposure duration (eg, t3 in FIG. 4 ) used for collecting the third subframe image, and the The exposure duration (such as t4 in FIG. 4 ) used for the fourth subframe image is respectively greater than the reference exposure duration, the first subframe image, the second subframe image, and the third subframe image The image and the fourth sub-frame image are used to determine a frame of depth image. That is, one frame of depth image may be output according to the first subframe image, the second subframe image, the third subframe image and the fourth subframe image.

The 4 subframe images collected in the above steps 702 to 705 can be understood as when N is 4 in step 301, according to the 4 preset exposure durations corresponding to the 4 phases respectively, the acquired 4 subframes in the target scene image. The first sub-frame image, the second sub-frame image, the third sub-frame image, and the fourth sub-frame image are collected in sequence. The first phase mentioned in the above steps 702 to 705 may be understood as the reference phase, and the second phase, the third phase, and the fourth phase may be understood as three extended phases.

In one example, the exposure duration used for collecting the third sub-frame image (such as t3 in FIG. 4 ) and the exposure time used for collecting the fourth sub-frame image (such as t4 in FIG. 4 ) are both shorter than those used for collecting the second sub-frame image The exposure duration used by the frame image (eg t2 in Figure 4). Referring to FIG. 4, that is, t1<t3<t2, t1<t4<t2, and t1<t2. That is to say, in the 4 exposure durations,

The exposure duration t2 used to collect the second sub-frame image is the largest, and when the phase difference between the second phase and the reference phase is 180 degrees, the depth that can be measured based on the second sub-frame image collected based on the second phase is the largest, so t2 is the largest and can be measured. Increase the maximum depth of the measurement to further increase the dynamic range of the measurement.

In one example, the exposure duration (eg, t3 in FIG. 4 ) used to acquire the third sub-frame image is shorter than the exposure duration (eg, t4 in FIG. 4 ) used to acquire the fourth sub-frame image, refer to FIG. 4 , that is, t1<t3<t4.

In one example, t1, t2, t3, and t4 are different. Since one subframe image collected based on one exposure duration can accurately obtain the depth information within one depth range in the target scene, the depth information collected based on four exposure durations can be accurately obtained. The 4 sub-frame images can accurately obtain the depth information in 4 depth ranges in the target scene (for example, nearer spots, near spots, far spots, and far spots in the target scene), which is beneficial to improve the measurement dynamic range. At the same time, the measurement accuracy in different depth ranges is improved.

In this embodiment, the reference exposure duration is less than or equal to the maximum duration that the image will not be exposed under the reference phase and the preset reference distance, so as to ensure that the subframe image of the acquired target scene is not exposed according to the reference exposure duration corresponding to the reference phase. The extended exposure duration is between the reference exposure duration and the maximum duration that the image will not be exposed under the extended phase and the reference distance, ensuring that the sub-frame images of the acquired target scene will not be exposed according to the extended exposure duration corresponding to the extended phase. In addition, since the extended exposure duration is different from the reference exposure duration, it is beneficial to determine the depth information at different distances in the target scene under different exposure durations. In the extended exposure time and the reference exposure time, a longer exposure time is conducive to accurately determining the depth information of the distant point in the target scene, and a shorter exposure time is conducive to accurately determining the target scene. The closer point is the depth information of the near point, which is beneficial to improve the dynamic range of ranging without exposure, and because one exposure duration is used in each phase to obtain one sub-frame image, it is not necessary to The phase adopts 2 different exposure durations to obtain 2 sub-frame images, so there is no need to fuse the sub-frame images obtained based on 2 different exposure durations for each phase, so the dynamic range of ranging can be improved while reducing the range. Small power consumption.

The embodiments of the present application relate to a depth measurement method. The implementation details of the depth measurement method of this embodiment are described below in detail. The following contents are only provided for the convenience of understanding, and are not necessary for implementing this solution.

In this embodiment, the flowchart of the depth measurement method in this embodiment may refer to FIG. 8 , including:

Step 801: Acquire N sub-frame images of the target scene according to the preset N exposure durations corresponding to the N phases respectively.

Step 802: Determine the depth information of the target scene according to the N subframe images.

Wherein, steps 801 to 802 are substantially the same as steps 301 to 302 in the above-mentioned embodiment, and the repetition is not avoided to be repeated here.

Step 803 : Determine a plurality of confidence levels corresponding to the depth information of the plurality of pixels according to the N sub-frame images.

Wherein, the TOF ranging device may determine the depth information of multiple pixels in the target scene based on multiple pixels on the photodetector and N sub-frame images. In a specific implementation, multiple pixel points in the target scene may include near points and far points, that is, the TOF ranging device can measure and obtain the depth information of the near point and the far point in the target scene. The confidence level corresponding to the depth information of each pixel point can represent the credibility of the depth information of the pixel point, that is, whether the measured depth information of the pixel point is accurate. For the manner of determining the confidence level, reference may be made to the relevant description in the first embodiment, which will not be repeated in this embodiment to avoid repetition.

Step 804: If the number of confidence levels that are less than the preset confidence level threshold in the multiple confidence levels exceeds the preset number threshold and the extended exposure duration does not reach the maximum duration of the extended phase and the reference distance that the image will not be exposed, increase the extended exposure duration.

Wherein, when the confidence corresponding to the depth information of a pixel in the target scene is less than the preset confidence threshold, it means that the depth measured for the pixel is less accurate, and if the multiple confidences are smaller than the preset confidence The number of confidence levels of the threshold exceeds the preset number threshold, indicating that the depth accuracy of several pixel points in the target scene is low. On the premise of ensuring that the image is not exposed, increasing the exposure time can improve the confidence level. The premise of the image not being exposed is to determine that the current extended exposure duration does not reach the maximum duration of the image not being exposed under the extended phase and the reference distance. In specific implementation, the preset confidence threshold and the preset quantity threshold can be set according to actual needs. For example, if the accuracy of the ranging is desired to be high, the confidence threshold can be set high and the quantity threshold can be set small.

In an example, referring to FIG. 4 , the reference phase is 0°, and the extended phase includes: 180°, 90°, and 270°. The reference exposure duration is equal to the maximum duration that the image will not be exposed at 0° and range0, that is, the reference exposure duration is t1, then theoretically, the value range of the extended exposure duration t2 corresponding to 180° can be: t1<t2≤9t1, 90° The value ranges of the extended exposure durations t3 and t4 corresponding to 270° may be: t1<t3=t4≤2t1. When setting t2, t3, and t4, you can gradually increase t2, t3, and t4 in their respective value ranges to find the minimum exposure duration that can make the confidence greater than the confidence threshold. It is not necessary to set t2, t3, The setting of t4 is relatively large. For example, if it is set to the maximum duration of no exposure, it is not necessary to directly set t2 to 9t1 at the beginning, and set t3 and t4 to 2t1, which can improve the dynamic range and accuracy of ranging, and at the same time, further reduce power consumption.

It is understandable that the depth measurement accuracy of several pixels in the target scene is low, indicating that the extended exposure time setting is more likely to be unreasonable, and the image is not exposed when the extended exposure time does not reach the extended phase and the reference distance. The maximum duration indicates that the current extended exposure duration still has room to increase. Therefore, in this embodiment, the number of confidence levels that are less than the preset confidence level threshold among the multiple confidence levels exceeds the preset number threshold and the extended exposure duration does not reach the maximum duration of the extended phase and the reference distance that the image will not be exposed. Large extended exposure duration can increase the extended exposure duration at a reasonable time. It is not necessary to set the extended exposure duration longer at the beginning, which is beneficial to obtain the minimum extended exposure duration that can make the confidence greater than the confidence threshold. The power consumption is further reduced while the dynamic square range and ranging accuracy are improved.

The steps of the above various methods are divided only for the purpose of describing clearly. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as the same logical relationship is included, they are all within the protection scope of this patent. ;Adding insignificant modifications to the algorithm or process or introducing insignificant designs, but not changing the core design of the algorithm and process are all within the scope of protection of this patent.

The embodiment of the present application relates to a chip. As shown in FIG. 9 , the chip 901 is connected to a memory 902 in an electronic device. The memory 902 stores instructions that can be executed by the chip 901 , and the instructions are executed by the chip 901 . So that the chip 901 can perform the depth measurement method in the above embodiment.

The memory 902 and the chip 901 are connected by a bus, and the bus may include any number of interconnected buses and bridges, and the bus connects one or more chips 901 and various circuits of the memory 902 together. The bus may also connect together various other circuits, such as peripherals, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides the interface between the bus and the transceiver. A transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a means for communicating with various other devices over a transmission medium. The data processed by the chip 901 is transmitted on the wireless medium through the antenna, and further, the antenna also receives the data and transmits the data to the chip 901 .

Chip 901 is responsible for managing the bus and general processing, and may also provide various functions including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 902 can be used to store data used by the chip 901 when performing operations.

The embodiment of the present application relates to an electronic device, as shown in FIG. 9 , including the chip 901 described in the foregoing embodiment and a memory connected to the chip 901 .

The embodiments of the present application relate to a computer-readable storage medium storing a computer program. The above method embodiments are implemented when the computer program is executed by the processor.

That is, those skilled in the art can understand that all or part of the steps in the method for implementing the above embodiments can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium and includes several instructions to make a device ( It may be a single chip microcomputer, a chip, etc.) or a processor (processor) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Those of ordinary skill in the art can understand that the above-mentioned embodiments are specific embodiments for realizing the present application, and in practical applications, various changes in form and details can be made without departing from the spirit and the spirit of the present application. scope.

Claims

A depth measurement method, comprising:

According to the preset N exposure durations corresponding to the N phases, N sub-frame images of the target scene are acquired; wherein, the N is a natural number greater than or equal to 2, and the N phases are the phases of the emitted light phase N phases with different differences;

determining the depth information of the target scene according to the N subframe images;

Wherein, the N phases include a reference phase and an extended phase, and the N exposure durations include a reference exposure duration corresponding to the reference phase and an extended exposure duration corresponding to the extended phase; the reference exposure duration is less than or equal to The maximum duration for which the image is not exposed under the reference phase and the preset reference distance, and the extended exposure duration is between the reference exposure duration and the maximum duration for which the image is not exposed under the extended phase and the reference distance. , the extended exposure duration is different from the reference exposure duration.
The depth measurement method according to claim 1, wherein the extended phase comprises a first type of extended phase and/or a second type of extended phase;

The phase difference between the first type of extended phase and the reference phase is greater than 0 and less than or equal to π/2, and the first type of extended phase and the maximum duration of the image not exposed at the reference distance are determined as follows: :

Within the same exposure duration, obtain the photon number of the sub-frame image under the reference phase and the photon number of the sub-frame image under the first type of extended phase respectively;

Calculate the first ratio of the number of photons of the subframe image under the reference phase to the number of photons of the subframe image under the first type of extended phase, and compare the first ratio to the reference phase and the preset The product of the maximum duration that the image is not exposed at the reference distance is taken as the first type of extended phase and the maximum duration of the image that is not exposed at the reference distance;

The phase difference between the second type of extended phase and the reference phase is greater than π/2 and less than or equal to π, the second type of extended phase and the maximum duration of the image under the reference distance without exposure is determined as follows: :

Based on the modulation frequency of the optical signal in the depth measurement, the second type of extended phase and the reference distance, calculate the farthest measurement distance at which the number of photons of the subframe image under the second type of extended phase reaches a limit value;

Calculate the square of the second ratio of the farthest measurement distance and the reference distance, and use the product of the square of the second ratio and the reference phase and the maximum duration of the image not to be exposed under the preset reference distance as The second type of extended phase and the maximum duration for which the image is not exposed at the reference distance.
The depth measurement method according to claim 2, wherein the spread phase comprises two spread phases of the first type and one spread phase of the second type, and the distance between the two spread phases of the first type is The phase difference is π.
The depth measurement method according to claim 3, wherein the two first-type extended phases are respectively 90° and 270°, and one of the second-type extended phases is 180°.
The depth measurement method according to any one of claims 1 to 4, wherein the reference phase is 0°.
The depth measurement method according to any one of claims 1 to 5, wherein the reference exposure duration is equal to the maximum duration during which the image is not exposed under the reference phase and a preset reference distance.
The depth measurement method according to claim 1, wherein the method for determining the maximum duration of the image without exposure under the extended phase and the reference distance is:

In the same exposure duration, respectively obtain the photon number of the subframe image under the reference phase and the photon number of the subframe image under the extended phase;

calculating a first ratio of the number of photons of the subframe image under the reference phase to the number of photons of the subframe image under the extended phase;

Based on the modulation frequency of the optical signal in the depth measurement, the extended phase and the reference distance, calculate the farthest measurement distance at which the number of photons of the subframe image under the extended phase reaches a limit value;

calculating the square of the second ratio of the farthest measured distance to the reference distance;

The smallest value is selected from the square of the first ratio and the second ratio, and the product of the smallest value and the reference phase and the maximum time period during which the image is not exposed under the preset reference distance is used as the The maximum duration for which the image is not exposed under the extended phase and the reference distance.
The depth measurement method according to any one of claims 1 to 7, wherein the depth information of the target scene includes the depth information of a plurality of pixels in the target scene. After the image determines the depth information of the target scene, it further includes:

determining, according to the N sub-frame images, a plurality of confidence levels corresponding to the depth information of the plurality of pixels respectively;

If the number of confidence levels among the plurality of confidence levels that are less than a preset confidence level threshold exceeds a preset number threshold and the extended exposure duration does not reach the extended phase and the maximum duration for the image not to be exposed at the reference distance , and increase the extended exposure duration.
A depth measurement method, comprising:

emit light for depth measurement at a reference phase;

The first sub-frame image of the target scene is collected according to the first phase and the reference exposure duration; wherein, the phase difference between the first phase and the reference phase is 0 degrees;

The second subframe image of the target scene is collected according to the second phase; wherein, the phase difference between the second phase and the reference phase is 180 degrees;

The third subframe image of the target scene is collected according to the third phase; wherein, the phase difference between the third phase and the reference phase is 90 degrees;

The fourth subframe image of the target scene is collected according to the fourth phase; wherein, the phase difference between the fourth phase and the reference phase is 270 degrees;

Wherein, the exposure duration used for collecting the second subframe image, the exposure duration used for collecting the third subframe image, and the exposure duration used for collecting the fourth subframe image are respectively greater than the reference exposure duration , the first subframe image, the second subframe image, the third subframe image and the fourth subframe image are used to determine a frame of depth image.
The depth measurement method according to claim 9, wherein the exposure time used for collecting the third subframe image and the exposure time used for collecting the fourth subframe image are both smaller than the exposure time used for collecting the first subframe image. Exposure duration used for two subframe images.
The depth measurement method according to claim 10, wherein the exposure duration used for collecting the third subframe image is shorter than the exposure duration used for collecting the fourth subframe image.
The depth measurement method according to any one of claims 9 to 11, wherein the first subframe image, the second subframe image, the third subframe image, and the fourth subframe image Images are acquired sequentially.
The depth measurement method according to any one of claims 9 to 12, wherein the reference exposure duration is equal to the maximum duration for which the image is not exposed under the reference phase and the minimum detection distance of the depth measurement method.
A chip, characterized in that it is arranged in an electronic device, the chip is connected to a memory in the electronic device, the memory stores an instruction that can be executed by the chip, and the instruction is executed by the chip, so that the chip can perform the depth measurement method as claimed in any one of claims 1 to 13 .
An electronic device, characterized by comprising the chip according to claim 14 and a memory connected to the chip.