WO2022204895A1 - 获取深度图的方法、装置、计算设备和可读存储介质 - Google Patents

获取深度图的方法、装置、计算设备和可读存储介质 Download PDF

Info

Publication number
WO2022204895A1
WO2022204895A1 PCT/CN2021/083673 CN2021083673W WO2022204895A1 WO 2022204895 A1 WO2022204895 A1 WO 2022204895A1 CN 2021083673 W CN2021083673 W CN 2021083673W WO 2022204895 A1 WO2022204895 A1 WO 2022204895A1
Authority
WO
WIPO (PCT)
Prior art keywords
resolution
pixel
depth map
modulation frequency
value
Prior art date
Application number
PCT/CN2021/083673
Other languages
English (en)
French (fr)
Inventor
罗鹏飞
董晨
周鸿彬
唐样洋
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Priority to CN202180096494.6A priority Critical patent/CN117083533A/zh
Priority to PCT/CN2021/083673 priority patent/WO2022204895A1/zh
Publication of WO2022204895A1 publication Critical patent/WO2022204895A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00

Definitions

  • the present application relates to the field of image technology, and in particular, to a method, apparatus, computing device, and readable storage medium for acquiring a depth map.
  • TOF imaging technology the light source in the TOF three-dimensional imaging system (referred to as TOF system) emits light to illuminate the target object, and the light reflected by the target object returns to the TOF
  • TOF chip ie image sensor
  • each pixel in the pixel array of the TOF chip can receive the light signal, and based on the phase difference between the emitted light signal and the received light signal, each pixel in the pixel array and the target object are determined distance to obtain a depth map.
  • phase delay value Due to the noise interference in the TOF system, the phase delay value will be affected by the noise and fluctuate around the ideal value. When the phase delay value fluctuates greatly due to interference, the phase delay value is inaccurate, which will lead to inaccurate unwrapping coefficients, and thus inaccurate depth maps obtained.
  • Embodiments of the present application provide a method, apparatus, computing device, and readable storage medium for acquiring a depth map, so that an accurate depth map can be obtained.
  • the present application provides a method for obtaining a depth map, the method comprising: performing down-sampling processing on raw data of multiple modulation frequencies of a measured object to obtain a depth map of a first resolution, wherein, The raw data of each modulation frequency is obtained by multiple exposures at each modulation frequency when the TOF system measures the distance of the object to be measured, and the depth map of the first resolution is the depth map that is determined to be the pixel without abnormal unwrapping coefficients.
  • the resolution of the pixel array in the TOF system is the second resolution, and the first resolution is lower than the second resolution; using the depth map of the first resolution and the phase delay value of each pixel of the second resolution, the second resolution is calculated to obtain the second resolution.
  • the unwrapping coefficient of each pixel of the resolution wherein the phase delay value of each pixel of the second resolution is obtained by using raw data of multiple modulation frequencies; the unwrapping coefficient of each pixel of the second resolution and the second The phase delay value of each pixel of the resolution is calculated to obtain the target depth map of the second resolution corresponding to the measured object.
  • the object to be measured is any object, such as a car.
  • the modulation frequency is a frequency used for intensity modulation of an optical signal of a certain frequency.
  • phase delay values of each pixel of the second resolution which are the phase delay values of each pixel at respective modulation frequencies.
  • unwrapping coefficients for each pixel of the second resolution which are respectively the unwrapping coefficients of each pixel at each modulation frequency.
  • Anomalous unwrapping coefficients can also be referred to as false unwrapping coefficients.
  • the method for obtaining a depth map can be executed by a ranging device.
  • the TOF system When measuring the distance of the measured object, the TOF system performs multiple exposures at any modulation frequency to obtain raw data of the modulation frequency. Then, down-sampling processing is performed on the raw data of multiple modulation frequencies to obtain a depth map of the first resolution, and each pixel in the depth map is determined to be a pixel without abnormal unwrapping coefficients. Then, using the depth map of the first resolution and the phase delay value of each pixel of the second resolution, the unwrapping coefficient of each pixel of the second resolution is obtained by calculation.
  • a target depth map of the second resolution corresponding to the object to be measured is obtained by calculation.
  • the unwrapping coefficients of the second resolution are calculated using the depth map of the first resolution, so this solution is used.
  • the target depth map of the second resolution calculated by the entanglement coefficient is also relatively accurate, which ensures the accuracy of the target depth map of the second resolution.
  • performing down-sampling processing on raw data of multiple modulation frequencies of the measured object to obtain a depth map with a first resolution including: using the first down-sampling parameter to The raw data of each modulation frequency is subjected to down-sampling processing to obtain the grayscale image of the target resolution and the depth map of the target resolution corresponding to the raw data of each modulation frequency; For the pixels of the first threshold corresponding to the modulation frequency, it is determined that the depth map of the target resolution is the depth map of the first resolution; if there is a pixel whose gray value is less than the first threshold corresponding to the modulation frequency in at least one grayscale image, Then, the first down-sampling parameter is updated, and based on the updated down-sampling parameter, the raw data of multiple modulation frequencies is subjected to down-sampling processing until a depth map of the first resolution is obtained.
  • the first downsampling parameter is a parameter for reducing the resolution.
  • the first downsampling parameter is 2*2, 2*2 means 4, and the original 2*2 pixels are combined into one pixel during processing.
  • the first down-sampling parameter may also be referred to as a first binning parameter, and the down-sampling process may also be referred to as a binning process.
  • the ranging device uses the first downsampling parameter to downsample the raw data of each modulation frequency to obtain a grayscale image of the target resolution corresponding to each modulation frequency, and can obtain a target Resolution depth map.
  • the target resolution is lower than the second resolution. Then it is judged whether there is a pixel whose gray value is less than the first threshold corresponding to the modulation frequency in the grayscale image of each target resolution, and if not, it is determined that the depth map of the target resolution is the grayscale image of the first resolution .
  • the first downsampling parameter is updated to make the first downsampling parameter larger, that is to say, more pixels are merged into one pixel.
  • the updated down-sampling parameters down-sampling the raw data of multiple modulation frequencies again to obtain multiple grayscale images and a depth map, and the resolutions of the multiple grayscale images are lower than the target resolution.
  • performing down-sampling processing on raw data of multiple modulation frequencies of the measured object to obtain a depth map with a first resolution including: using the first down-sampling parameter to The raw data of each modulation frequency is subjected to down-sampling processing to obtain a grayscale image of the target resolution corresponding to the raw data of each modulation frequency; if there is no grayscale value in each grayscale image that is smaller than the first threshold corresponding to the modulation frequency , then use the first downsampling parameter to downsample the raw data of multiple modulation frequencies to obtain a depth map of the first resolution; For pixels with the first threshold, the first down-sampling parameters are updated, and based on the updated down-sampling parameters, the raw data of multiple modulation frequencies is subjected to down-sampling processing until a depth map of the first resolution is obtained.
  • the ranging device uses the first down-sampling parameter to perform down-sampling processing on the raw data of each modulation frequency to obtain a grayscale image of the target resolution corresponding to each modulation frequency.
  • the target resolution is lower than the second resolution.
  • a downsampling process is performed to obtain a grayscale image of the first resolution.
  • the first downsampling parameter is updated to make the first downsampling parameter larger, that is to say, more pixels are merged into one pixel.
  • the updated down-sampling parameters down-sampling the raw data of multiple modulation frequencies again to obtain multiple grayscale images, the resolutions of which are lower than the target resolution. Determine whether there are pixels in the multiple grayscale images whose grayscale values are less than the first threshold corresponding to the modulation frequency. If not, use the updated downsampling parameters to downsample the raw data corresponding to the multiple modulation frequencies.
  • the first threshold corresponding to each modulation frequency is the lowest unwrapped received signal strength in the corresponding range noise curve obtained in the TOF system.
  • each modulation frequency corresponds to a distance noise curve.
  • the horizontal axis is the received optical signal intensity
  • the vertical axis is the multiple times of a certain optical signal intensity.
  • performing down-sampling processing on raw data of multiple modulation frequencies of the measured object to obtain a depth map of the first resolution including: using raw data of multiple modulation frequencies and multiple modulation frequencies obtain the unwrapping coefficient of each pixel in the initial depth map of the second resolution; use the unwrapping coefficient of each pixel in the initial depth map and the phase delay value of each pixel of the second resolution to calculate and obtain the initial depth The depth value of each pixel in the figure; down-sampling the initial depth map to obtain a depth map of the first resolution.
  • the phase delay map is a map of the correspondence between the phase delay value and the unwrapping coefficient, and in the phase delay map, the phase delay value can be used to correspond to the unwrapping coefficient.
  • the ranging device can substitute the raw data of each modulation frequency into the phase delay value calculation formula to determine the phase delay value of each pixel of the second resolution at each modulation frequency. Then, for any pixel, use the multiple phase delay values of the pixel to determine the line segment closest to the position of the multiple phase delay values in the phase delay map, determine the unwrapping coefficient corresponding to the serial number of the line segment, and then unwrap the line segment. The coefficients are determined as the corresponding unwrapping coefficients for the pixel at multiple frequencies.
  • N kj is the unwrapping coefficient corresponding to the pixel k at the modulation frequency j
  • U j is the phase delay value of pixel k at modulation frequency j.
  • the depth values of each pixel at multiple modulation frequencies are averaged, and the average value corresponding to each pixel is determined as the depth value of each pixel of the second resolution.
  • the depth value of each pixel of the second resolution is formed into an initial depth map of the second resolution. Then, the initial depth map of the second resolution is down-sampled to obtain the depth map of the first resolution, where the first resolution is lower than the second resolution.
  • performing downsampling processing on the initial depth map to obtain a depth map with a first resolution includes: using a second downsampling parameter to perform downsampling processing on the initial depth map to obtain a third resolution determine the gradient map corresponding to the depth map of the third resolution; if the gradient map satisfies the gradient conditions, then determine that the depth map of the third resolution is the depth map of the first resolution; if the gradient map does not meet the gradient conditions, Then update the second down-sampling parameter, and perform down-sampling processing on the initial depth map based on the updated down-sampling parameters until the target gradient map that satisfies the gradient condition is obtained, and the depth map obtained from the target gradient map is determined as the depth of the second resolution.
  • the gradient condition is that the absolute value of the gradient of the pixels whose distance interval is less than the second threshold is less than or equal to the third threshold.
  • the second downsampling parameter is a parameter for reducing the resolution.
  • the second downsampling parameter is 2*2, 2*2 means 4, and the original 2*2 pixels are combined into one pixel during processing.
  • the second downsampling parameter may also be referred to as a second binning parameter, and the downsampling process may also be referred to as a binning process.
  • the ranging apparatus may use the second downsampling parameter to divide the initial depth map of the second resolution into multiple pixel combinations, and the number of pixels in each pixel combination is the same.
  • the depth value of each pixel in each pixel combination is averaged to obtain the depth value when the pixels in each pixel combination are combined into one pixel, and the combined depth value is formed into a depth map of the third resolution.
  • the ranging device determines the relationship between the absolute value of the gradient of the pixels whose distance interval is less than the second threshold in the gradient map and the third threshold, if the distance interval in the gradient map is less than The absolute value of the gradient of the pixels of the second threshold is less than or equal to the third threshold, then it is determined that the gradient map satisfies the gradient condition, and it can be determined that the depth map of the third resolution is the depth map of the first resolution, that is, the third resolution rate equal to the first resolution.
  • the second downsampling parameter may be updated by using a preset rule based on the second downsampling parameter.
  • the updated downsampling parameters to downsample the initial depth map of the second resolution, obtain the updated gradient map, and determine whether the updated gradient map satisfies the gradient conditions. If the updated gradient map satisfies the gradient conditions, then It is determined that the depth map obtained from the updated gradient map is the depth map of the first resolution. If the updated gradient map does not satisfy the gradient conditions, the gradient map will continue to be updated on the basis of the downsampling parameters after the last update, until the target gradient map that satisfies the gradient conditions is obtained. The depth map obtained from the target gradient map is determined as the depth map of the first resolution.
  • the absolute value of the gradient is relatively large, and the gradient is related to the depth value error.
  • the depth value The error is usually less than a certain value, so the gradient condition can be used to obtain the depth map of the first resolution.
  • the unwrapping coefficient of pixel k of the second resolution is: Among them, modulation frequency j belongs to multiple modulation frequencies, round[] is the rounding operation, DL is the depth value corresponding to pixel k in the depth map of the first resolution, and N kj is the unwrapping coefficient of pixel k at modulation frequency j , is the phase delay value of pixel k at modulation frequency j, and U j is the blurring distance corresponding to modulation frequency j.
  • the pixels of the second resolution are The depth value DL corresponding to k in the depth map of the first resolution is the depth value of a pixel formed by a plurality of pixels including pixel k.
  • the unwrapping coefficient of each pixel of the second resolution and the phase delay value of each pixel of the second resolution are used to calculate and obtain the target depth map of the second resolution corresponding to the measured object, Including: using the unwrapping coefficient of each pixel of the second resolution and the phase delay value of each pixel of the second resolution, calculating the depth value of each pixel of the second resolution for each modulation frequency; using the first resolution the depth map and the depth value of each pixel of the second resolution of each modulation frequency, determine the depth value error of each pixel of the second resolution of each modulation frequency; use each pixel of the first resolution of each modulation frequency The depth value error of , calculates and obtains the target depth map of the second resolution corresponding to the measured object.
  • the ranging device can convert N kj , and the fuzzy distance U j into the formula Obtain the depth value D kj of the pixel k at the modulation frequency j, where N kj is the unwrapping coefficient corresponding to the pixel k at the modulation frequency j, is the phase delay value of pixel k at modulation frequency j. In this way, the depth value of each pixel at each modulation frequency can be determined.
  • the ranging apparatus may then use the depth map of the first resolution and the depth value of each pixel of the second resolution of each modulation frequency to determine the error of the depth value of each pixel of the second resolution of each modulation frequency.
  • the distance measuring device may use the depth value error of each pixel of the second resolution of each modulation frequency to determine the target depth map of the second resolution corresponding to the measured object.
  • the depth value of each pixel of the second resolution of each modulation frequency is determined by using the depth map of the first resolution and the depth value of each pixel of the second resolution of each modulation frequency
  • using the depth value error of each pixel of the first resolution of each modulation frequency to calculate and obtain a target depth map of the second resolution corresponding to the measured object including: for multiple modulation frequencies In the modulation frequency j, among the pixels of the second resolution of the modulation frequency j, determine the target pixel whose depth value error is greater than the fourth threshold corresponding to the modulation frequency j, where j takes a value from 1 to n, and n is a plurality of modulation The number of frequencies; using the depth values of the adjacent pixels of the target pixel in the depth map of the first resolution, update the unwrapping coefficients of the target pixels of the second resolution of the modulation frequency j; use the updated unwrapping coefficients of the target pixels and the phase delay value of the target pixel of each modulation frequency, update the depth value of the target pixel of the second resolution of the modulation frequency j; the depth of the pixels other than the target pixel in the pixels of the second resolution of the modulation frequency j
  • the value is combined
  • the ranging device can obtain the fourth threshold corresponding to the modulation frequency j, and then the ranging device can determine whether the error of the depth value in the pixels of the second resolution of the modulation frequency j is greater than the modulation frequency For the fourth threshold value corresponding to j, in the depth map of the second resolution with the modulation frequency j, the target pixels whose depth value error is greater than the fourth threshold value are obtained. Then, in the depth map of the first resolution, the depth values of adjacent pixels of the target pixel are determined.
  • the ranging device uses the depth value of the adjacent pixel P and the target of the second resolution with the modulation frequency j
  • the phase delay value corresponding to the pixel and the blurring distance corresponding to the modulation frequency j are calculated to obtain the unwrapping coefficient corresponding to the target pixel of the second resolution at the modulation frequency j when the depth value of the adjacent pixel P is used.
  • the ranging device uses the unwrapping coefficient corresponding to the target pixel at the modulation frequency j, the phase delay value corresponding to the target pixel at the modulation frequency j, and the fuzzy distance corresponding to the modulation frequency j to calculate and obtain the target pixel of the second resolution at the modulation frequency j. depth value. Determines the depth value error of the target pixel for this calculation. Based on the above method, each adjacent pixel of the target pixel in the depth map of the first resolution is used to determine the depth value error of the target pixel corresponding to each adjacent pixel.
  • the ranging device determines the depth value when the error of the depth value is the smallest and less than the fourth threshold corresponding to the modulation frequency j, as the depth value of the target pixel at the modulation frequency j. Then, the ranging device combines the depth values of pixels other than the target pixel among the pixels of the second resolution at the modulation frequency j with the updated depth value of the target pixel to obtain the second resolution of the measured object at the modulation frequency j.
  • Target depth map Based on the manner in which the second resolution target depth map of the measured object at modulation frequency j is determined, the second resolution target depth map of the measured object at each modulation frequency is determined.
  • the correct depth value of the adjacent pixels can be used to calculate the depth value of each pixel in the region with the wrong depth value, so it can be guaranteed that the depth values of all pixels can be calculated correctly, and the obtained Correct depth map at native resolution.
  • the error of the phase delay value exceeds one-half of the minimum phase distance, it may lead to the determination of the unwrapping coefficient of the neighbor line segment, and then determine the wrong number of unwrapping, and the error of the phase delay value is smaller than the minimum phase distance.
  • the distance is half of the distance, the probability of determining the unwrapping coefficient of the neighbor line segment is relatively small, so the distance corresponding to half of the minimum phase distance can be used as the threshold of the depth value error.
  • the present application provides an apparatus for acquiring a depth map, where the apparatus includes one or more modules for implementing the method for acquiring a depth map described in the first aspect.
  • the present application provides a computing device for acquiring a depth map, the computing device comprising a processor and a memory, wherein: the memory stores computer instructions; the processor executes the computer instructions to achieve Methods of the first aspect and possible implementations thereof.
  • the present application provides a computer-readable storage medium, where computer instructions are stored in the computer-readable storage medium, and when the computer instructions in the computer-readable storage medium are executed by a computing device, the computing device executes the computer instructions.
  • the present application provides a computer program product comprising instructions which, when run on a computing device, cause the computing device to perform the method of the above-described first aspect and possible implementations thereof.
  • FIG. 1 is a relationship diagram of a phase delay value and distance provided by an exemplary embodiment of the present application
  • FIG. 2 is a schematic diagram of unwrapping coefficients in a phase delay value map provided by an exemplary embodiment of the present application
  • FIG. 3 is a relationship diagram between a phase delay value and an unwrapping coefficient provided by an exemplary embodiment of the present application
  • FIG. 4 is a schematic structural diagram of a TOF system provided by an exemplary embodiment of the present application.
  • FIG. 5 is a schematic flowchart of a method for acquiring a depth map provided by an exemplary embodiment of the present application
  • FIG. 6 is a schematic diagram of downsampling provided by an exemplary embodiment of the present application.
  • FIG. 7 is a relationship diagram between received optical signal strength and distance noise provided by an exemplary embodiment of the present application.
  • FIG. 8 is a schematic diagram of downsampling provided by an exemplary embodiment of the present application.
  • FIG. 9 is a schematic flowchart of a method for acquiring a depth map provided by an exemplary embodiment of the present application.
  • FIG. 10 is a schematic flowchart of a method for acquiring a depth map provided by an exemplary embodiment of the present application
  • FIG. 11 is a schematic structural diagram of an apparatus for acquiring a depth map provided by an exemplary embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a computing device provided by an exemplary embodiment of the present application.
  • time of flight refers to the round-trip time.
  • the round-trip time is the round-trip time between the optical signal sent by the transmitter in the TOF system and the receiver in the TOF system.
  • the depth map can be calculated through the round-trip time.
  • the depth map includes multiple pixels, and the pixel value of each pixel is used to record the distance between the TOF system and a certain point of the measured object, that is, the pixel value of each pixel is the depth value.
  • the fuzzy distance for a TOF system with a single modulation frequency, represents the maximum measurable distance, and is expressed as:
  • f represents the modulation frequency
  • u represents the fuzzy distance corresponding to the modulation frequency f
  • c represents the speed of light.
  • the modulation frequency here is a frequency used for intensity modulation of an optical signal of a certain frequency.
  • the beat frequency refers to the frequency corresponding to the distance obtained by the joint measurement of the multiple modulation frequencies when using multiple modulation frequencies to measure the distance.
  • the striking frequency is the greatest common divisor of the plurality of modulation frequencies.
  • the striking frequency is relatively low relative to the plurality of modulation frequencies, and a longer measurement distance can be extended.
  • it represents the relationship between the phase delay value of two modulation frequencies within 7.5m of 80MHz and 100MHz and the distance.
  • the fuzzy distance is extended to 7.5m. If there is no noise in the TOF system, the fuzzy distance is within the fuzzy distance.
  • the two modulation frequency phase delay values for each pixel Corresponding to the distance one-to-one, in the TOF system of 80MHz and 100MHz, there are 8 unwrapped regions. In Figure 2, only the unwrapped regions of odd-numbered segments are marked. As shown in Table 1, the unwrapping coefficients of the 8-segment unwrapping regions of the TOF system at 80 MHz and 100 MHz are given.
  • Unwrapping coefficient N1 80MHz
  • Unwrapping coefficient N2 100MHz 1 0 0 2 0 1 3 1 1 4 1 2 5 2 2 6 2 3 7 3 3 8 3 4
  • phase delay values corresponding to the two modulation frequencies are drawn in the Cartesian coordinate system (xoy) according to the xy coordinates.
  • the unwrapping coefficients (N 1 , N 2 ) corresponding to a certain pixel at the two modulation frequencies determine the position point of the phase delay value corresponding to the pixel in Fig. 3, and determine the line segment ( 3
  • the line segment can be called the nearest line segment).
  • the unwrapping coefficient corresponding to the serial number of the nearest line segment is determined in Table 1, and the unwrapping coefficient is determined as the unwrapping coefficient corresponding to the pixel at the two modulation frequencies respectively.
  • the calculated phase delay value will jump around the ideal value. If the phase delay value of a pixel is subject to relatively small noise interference and the beating is not large, as long as the line segment closest to the position of the phase delay value is determined, the accurate unwrapping coefficient can be determined. If the phase delay value of a pixel is delayed The noise interference of the value is relatively large, and the position of the phase delay value deviates from the correct line segment, the pixel will be given the wrong unwrapping coefficient, which will lead to the wrong distance for the pixel is determined based on equations (2) and (3). , so that the depth map obtained by measuring the measured object is wrong.
  • an embodiment of the present application provides a method for acquiring a depth map.
  • the method may be performed by a device for acquiring a depth map, which may be referred to as a distance measuring device in the following.
  • the ranging device may be a hardware device, such as a computing device such as a server and a terminal, or a software device, such as a set of software programs running on the hardware device.
  • the ranging device may acquire raw data of multiple modulation frequencies from the TOF system, and determine the depth map based on the raw data of multiple modulation frequencies, and the detailed process will be described later.
  • the ranging device may be a part of the TOF system, or may exist independently of the TOF system.
  • the TOF system includes a TOF chip 101, a lens 102, a laser driver 103, a laser 104, etc.
  • the TOF chip 101 includes a controller 1011, a digital-to-analog converter (analog to digital) converter, ADC) 1012 and pixel array 1013.
  • the TOF chip 101 may be referred to as an image sensor.
  • the laser 104 can be a light-emitting diode (LED) laser, and can sequentially emit light signals of multiple modulation frequencies.
  • the controller 1011 is used to control the laser 104 to emit light signals through the laser driver 103, and the controller 1011 is also used to control the pixel array 1013 to expose for a period of time to obtain the exposure value.
  • the pixel array 1013 transmits the exposure value to the ADC 1012, the ADC 1012 performs analog-to-digital conversion, and sends the exposure value after the analog-to-digital conversion, that is, raw data, to the ranging device.
  • the four exposure values correspond to 0 degrees, 90 degrees, 180 degrees and 270 degrees, respectively, see equations (7) to (10).
  • the transmitted optical signal is an intensity-modulated cosine signal, and the signal is represented by formula (4):
  • equation (4) s(t) represents the emitted light signal, and ⁇ represents the angular velocity.
  • the transmitted optical signal is reflected back by the measured object with an offset (due in part to the illumination of the background light), and a modulated cosine signal containing a phase delay, the expression for the reflected optical signal It is expressed by formula (5):
  • Equation (5) g(t) represents the reflected light signal, ⁇ represents the angular velocity, a represents the attenuation coefficient, represents the phase delay value, and b represents the offset.
  • the process of demodulating the reflected optical signal uses the correlation function method: using the transmitted optical signal as the reference signal, the modulated transmitted optical signal and the reflected optical signal are obtained.
  • the correlation function between the returned optical signals is expressed by equation (6):
  • c( ⁇ ) represents the correlation function
  • Equation (7) to (10) The final expressions of the demodulated optical signals are represented in equations (7) to (10), and K is an offset, representing the background light included in the reflected back signal.
  • K is an offset, representing the background light included in the reflected back signal.
  • the four exposure values mentioned are C( ⁇ 0 ), C( ⁇ 1 ), C( ⁇ 2 ) and C( ⁇ 3 ).
  • TOF systems can be applied to autonomous driving, collision avoidance and self-braking.
  • assisted driving systems supported by the principle of TOF technology
  • the assisted driving system can accurately detect the position of the driver's body and head, and even capture his blinking action when he wears glasses or sunglasses to determine whether the driver is Focus enough, whether you are driving fatigued, etc., so as to activate the corresponding countermeasures.
  • Countermeasures can be vibrating the seat or sounding a warning sound.
  • quick and accurate responses are possible, with driver assistance systems and emergency braking systems automatically activated before a potential emergency occurs.
  • TOF technology can also control in-vehicle entertainment systems or car air conditioners through hand movements or body gestures, and even realize new auxiliary and safety functions outside the car, such as door opening assistance equipment, when opening the door in a parking lot or home basement, Prevent the door from hitting other objects (such as vehicles, walls, ceilings, etc.) when the door is opened.
  • Scenario 2 Human-computer interaction scenario
  • the TOF system provides a real-time distant image that can be easily used to record human movements. Therefore, electronic products have a new interactive mode, and electronic products can be TVs, mobile phones, tablets, etc.
  • the depth value of the object captured by each pixel can be obtained through TOF, and objects with different depths can be distinguished.
  • the TOF system is installed on the robot, and the robot classifies and precisely positions the objects through the TOF system.
  • it can also be applied to face recognition and geomorphological mapping.
  • the method for acquiring a depth map provided by the embodiment of the present application will be described below with reference to FIG. 5 , and the method may be executed by a ranging apparatus. As shown in Figure 5, the processing flow of the method is as follows:
  • Step 501 Perform down-sampling processing on the raw data of multiple modulation frequencies of the measured object to obtain a depth map of the first resolution, wherein the raw data of each modulation frequency is when the TOF system measures the distance of the measured object at each time. Obtained by multiple exposures at each modulation frequency, the depth map of the first resolution is the depth map that is judged to have no abnormal unwrapping coefficients in the pixels, the resolution of the pixel array in the TOF system is the second resolution, and the first resolution is low at the second resolution.
  • the object to be measured is an arbitrary object, such as a car.
  • the number of the plurality of modulation frequencies is greater than 1, for example, the plurality of modulation frequencies may be 2 modulation frequencies or 3 modulation frequencies.
  • the pixel value of each pixel in the depth map can be referred to as a depth value.
  • the depth value represents the distance from the reflected light signal on the measured object to the position of the pixel and the TOF system.
  • the first resolution is lower than the second resolution, which is the resolution of the pixel array in the TOF system. Relative to the first resolution, the second resolution is a resolution without downsampling, so it can also be referred to as an original resolution, and the first resolution is a low resolution relative to the second resolution.
  • the absence of abnormal unwrapping coefficients for a pixel means that when the depth value of the pixel is used to determine the unwrapping coefficients, the unwrapping coefficients will not be calculated incorrectly.
  • the controller in the TOF system is driven by the laser, and controls the laser to transmit the optical signal of each modulation frequency in turn to obtain the raw data of each modulation frequency, as follows to transmit the modulation frequency j (the modulation frequency j is a plurality of modulation
  • the optical signal of any modulation frequency in the frequency) is used as an example to illustrate:
  • the controller is driven by the laser to control the laser to emit an optical signal with modulation frequency j, and irradiate it to the measured object.
  • the optical signal is an intensity-modulated optical signal.
  • the measured object reflects the light signal
  • the controller controls the pixel array to expose for a period of time, and receives the light signal reflected by the measured object.
  • the exposure values of any pixel in the pixel array are C( ⁇ 0 ) j , C( ⁇ 1 ) j , C( ⁇ 2 ) j and C( ⁇ 3 ) j , respectively corresponding to formulas (7) to (10) in the foregoing.
  • the pixel array transmits the exposure value of each pixel to the ADC, and the ADC performs analog-to-digital conversion processing on the exposure value of each pixel to obtain DCS( ⁇ 0 ) j , DCS( ⁇ 1 ) j , DCS( ⁇ 2 ) j , DCS ( ⁇ 3 ) j , which is transmitted to the ranging device.
  • the multiple exposures of each pixel and the quantized values DCS( ⁇ 0 ) j , DCS( ⁇ 1 ) j , DCS( ⁇ 2 ) j , and DCS( ⁇ 3 ) j constitute the raw data of the modulation frequency j. In this way, the ranging device obtains raw data of the modulation frequency j.
  • raw data of a plurality of modulation frequencies are obtained based on sequentially transmitting optical signals of a plurality of modulation frequencies.
  • the ranging device may down-sample the raw data of multiple modulation frequencies to obtain a depth map of a first resolution, and the depth map of the first resolution is a depth map that is determined to be a pixel without abnormal unwrapping coefficients. In this way, a depth map in which there are no abnormal disentanglement coefficients for pixels can be obtained.
  • Step 502 using the depth map of the first resolution and the phase delay value of each pixel of the second resolution, calculate and obtain the unwrapping coefficient of each pixel of the second resolution, wherein the phase delay of each pixel of the second resolution is Values are calculated using raw data for multiple modulation frequencies.
  • the ranging device can calculate the phase delay value through equations (7) to (10). Using formula (11), it can be expressed as:
  • DCS( ⁇ 0 ) kj , DCS( ⁇ 1 ) kj , DCS( ⁇ 2 ) kj , and DCS( ⁇ 3 ) kj represent raw data of pixel k at modulation frequency j.
  • the second resolution at each modulation frequency can be obtained by formula (11). Phase delay value for each pixel.
  • the distance measuring device can obtain the pre-stored fuzzy distance corresponding to each modulation frequency, or substitute each modulation frequency into Equation (1) to obtain the fuzzy distance corresponding to each modulation frequency.
  • the ranging device uses the depth map of the first resolution, the phase delay value of each pixel of the second resolution, and the blurring distance corresponding to each modulation frequency to calculate and obtain the unwrapping coefficient of each pixel of the second resolution.
  • Step 503 using the unwrapping coefficient of each pixel of the second resolution and the phase delay value of each pixel of the second resolution, calculate and obtain a target depth map of the second resolution corresponding to the object to be measured.
  • the ranging device may use the N kj determined in step 502, and U j are substituted into equation (12) to obtain the distance D kj (that is, the depth value) of the pixel k.
  • the depth value of each pixel of the second resolution of the modulation frequency j is determined, and these depth values are formed into a target depth map of the second resolution corresponding to the object to be measured at the modulation frequency j.
  • N kj is the unwrapping coefficient of the pixel k at the modulation frequency j
  • U j is the ambiguity distance of the modulation frequency j.
  • the target depth map of the second resolution corresponding to the measured object at the modulation frequency j is determined.
  • the unwrapping coefficients of each pixel of the second resolution are calculated and obtained using the depth map of the first resolution, so the second resolution
  • the unwrapping coefficient of each pixel of the rate is relatively accurate, and the depth map of the second resolution calculated based on the unwrapping coefficient is also relatively accurate, which ensures the accuracy of the depth map of the second resolution.
  • step 503 the target depth map is to be distinguished from the initial depth map in the following, which actually represents the depth map.
  • the depth values of the corresponding pixels in the depth map of the second resolution corresponding to the measured object with multiple modulation frequencies can also be averaged or weighted averaged according to a certain set weight sequence to obtain the measured object.
  • step 501 There are many ways to realize step 501, and three feasible implementation ways are given as follows:
  • Method 1 Use the first downsampling parameter to downsample the raw data of multiple modulation frequencies of the object under test, and obtain the grayscale image of the target resolution and the depth of the target resolution corresponding to the raw data of each modulation frequency Figure; if there is no pixel whose gray value is less than the first threshold corresponding to the modulation frequency in each grayscale, then determine that the depth map of the target resolution is the depth map of the first resolution; if at least one grayscale map If there is a pixel whose gray value is less than the first threshold corresponding to the modulation frequency, the first down-sampling parameter is updated, and the raw data of multiple modulation frequencies is down-sampled based on the updated down-sampling parameter until the first resolution is obtained. depth map.
  • the first threshold value corresponding to each modulation frequency is a preset value, which is stored in the ranging device.
  • the first downsampling parameter is the downsampling parameter used when the downsampling process is performed for the first time.
  • the first downsampling parameter is a parameter for reducing the resolution.
  • the first downsampling parameter is 2*2, 2*2 means 4, and the original 2*2 pixels are combined into one pixel during processing.
  • the first down-sampling parameter may also be referred to as the first binning parameter, and the down-sampling process may also be referred to as the binning process.
  • the distance measuring device may use the first downsampling parameter to downsample the raw data of each modulation frequency to obtain a grayscale image of the target resolution and the target resolution corresponding to the raw data of each modulation frequency rate depth map.
  • the first downsampling parameter the plurality of pixels in the raw data of the modulation frequency j are divided into a plurality of pixel combinations, and the number of pixels included in each pixel combination is equal to the first downsampling parameter. For example, if the first downsampling parameter is 2*2, then each pixel combination here includes 2*2 pixels, and the first downsampling parameter is 3*3, then each pixel combination here includes 3*3 pixels pixel.
  • the DCS( ⁇ 0 ) j of each pixel in the pixel combination in the raw data of the modulation frequency j is averaged to obtain the DCS( ⁇ 0 ) j0 that the pixels in the pixel combination are combined into one pixel; Average the DCS( ⁇ 1 ) j of each pixel in the pixel combination in the raw data of the modulation frequency j, and obtain the DCS( ⁇ 1 ) j1 that the pixels in the pixel combination are merged into a pixel; the raw data of the modulation frequency j
  • the DCS( ⁇ 2 ) j of each pixel in this pixel combination is averaged to obtain the DCS( ⁇ 2 ) j2 that the pixel in this pixel combination is merged into a pixel; DCS( ⁇ 3 ) j is averaged to obtain DCS( ⁇ 3 ) j3 in which the pixels in the pixel combination are combined into one pixel.
  • the pixels in the pixel combination in the raw data of the second resolution are merged into one pixel, it is equivalent to down-sampling the raw data of the second resolution to obtain raw data lower than the second resolution, which is lower than the second resolution.
  • the raw data of the resolution may be referred to as the raw data of the target resolution.
  • the second resolution is 8*8
  • the first downsampling parameter is 2*2
  • the target resolution is 4*4
  • each box in FIG. 6 represents one pixel.
  • a grayscale image of the target resolution at the modulation frequency j can be obtained, and by following the above processing for each modulation frequency, the grayscale image of the target resolution corresponding to the raw data of each modulation frequency can be obtained.
  • the line segments of the unwrapped regions of multiple modulation frequencies can be obtained.
  • the phase delay value corresponding to each modulation frequency and the unwrapped regions of multiple modulation frequencies are used. of line segments to obtain the unwrapping coefficients for each pixel.
  • the multiple modulation frequencies are two modulation frequencies (modulation frequency 1 and modulation frequency 2), for pixel i of the target resolution, use the phase delay value of the pixel i and ( represents the phase delay value of the pixel i at modulation frequency 1, represents the phase delay value of the pixel i at modulation frequency 2), find the distance in Figure 3 and For the line segment with the nearest position point, the unwrapping coefficient corresponding to the serial number of the line segment is determined as the unwrapping coefficient corresponding to the pixel i.
  • the depth value of each pixel of the target resolution under multiple modulation frequencies is averaged or weighted to obtain the depth value of each pixel of the target resolution.
  • the depth value of each pixel of the target resolution at any modulation frequency among the plurality of modulation frequencies may also be determined as the depth value of each pixel of the target resolution. In this way, a depth map at the target resolution can be obtained.
  • the ranging device obtains the stored first threshold value corresponding to each modulation frequency, and in the grayscale map of the target resolution of each modulation frequency, determines whether the grayscale value of each pixel is greater than or equal to the first threshold value corresponding to the corresponding modulation frequency. threshold. If both are greater than or equal to the first threshold corresponding to the modulation frequency, it is determined that the depth map of the target resolution is the depth map of the first resolution. If there are pixels in the grayscale image of at least one target resolution whose grayscale values are less than the first threshold corresponding to the modulation frequency, the downsampling parameters may be updated according to a preset rule based on the first downsampling parameters.
  • the ranging device performs down-sampling processing on the raw data of multiple modulation frequencies by using the updated binning parameters, and obtains the grayscale map of the fourth resolution and the depth map of the fourth resolution of each modulation frequency.
  • the method judges whether the grayscale value of each pixel in each grayscale image of the fourth resolution is greater than or equal to the first threshold value corresponding to the modulation frequency to which it belongs. If the gray value of each pixel is greater than or equal to the first threshold, it is determined that the depth map of the fourth resolution is the depth map of the first resolution. If there are pixels in at least one grayscale image of the fourth resolution whose grayscale value is less than the first threshold corresponding to the modulation frequency, the downsampling parameters after the last update can be updated according to preset rules. sampling parameters, and then perform down-sampling processing on raw data of multiple modulation frequencies until the gray value of each pixel in each grayscale image is obtained to be greater than or equal to the first threshold corresponding to the modulation frequency.
  • the downsampling parameters are updated according to a preset rule
  • the preset rule may be to increase the preset value in both the horizontal direction and the vertical direction on the basis of the original downsampling parameters.
  • the preset value is 1, the first downsampling parameter is 2*2, the downsampling parameter after the first update is 3*3, and the downsampling parameter after the second update is 4*4.
  • the gray value can reflect the signal reception strength, the signal reception strength is relatively high, and the signal-to-noise ratio is relatively high, the unwrapping can generally be performed accurately, and the accurate unwrapping coefficient can be determined, so the gray value can be obtained by grayscale.
  • the gray value in the degree map is used to judge whether the unwrapping can be performed accurately.
  • Method 2 Use the first downsampling parameter to downsample the raw data of multiple modulation frequencies of the measured object to obtain a grayscale image of the target resolution corresponding to the raw data of each modulation frequency; if each grayscale If there is no pixel whose gray value is less than the first threshold corresponding to the modulation frequency, the first downsampling parameter is used to downsample the raw data of multiple modulation frequencies to obtain a depth map of the first resolution; If there is a pixel whose grayscale value is less than the first threshold corresponding to the modulation frequency in at least one grayscale image, the first downsampling parameter is updated, and the raw data of the plurality of modulation frequencies is subjected to downsampling processing based on the updated downsampling parameter, until a depth map of the first resolution is obtained.
  • the processing process of the second method is similar to that of the first method, the difference is that: in the first method, the grayscale image and the depth map are determined together in the downsampling process;
  • the downsampling parameter used in the degree map is the target downsampling parameter. After the grayscale value of each pixel in the target grayscale image is greater than or equal to the first threshold corresponding to the modulation frequency to which it belongs, then based on the target downsampling parameter, multiple modulation
  • the raw data of the frequency is subjected to down-sampling processing to obtain a depth map of the first resolution.
  • the first threshold corresponding to each modulation frequency is the lowest unwrapped received signal strength in the corresponding distance-to-noise curve obtained in the TOF system.
  • the fuzzy distances of the TOF systems with multiple modulation frequencies are relatively large, and the reflectivity of the measured object may vary greatly within the spatial range from 0 to the fuzzy distance.
  • the distance between the TOF system and the measured object is different, the measured The intensity of the light signal reflected by the object is also different, so some pixels in some pixel arrays in the TOF system can receive the light signal with a relatively high signal-to-noise ratio, but other pixels will receive the light signal with a relatively low signal-to-noise ratio. .
  • the x-axis represents the received optical signal intensity
  • the y-axis represents the measured object that measures the same distance multiple times under a certain optical signal intensity. Standard deviation of depth values.
  • the modulation frequency is 100MHz
  • the unit of the received optical signal intensity is the digital quantization value (digital number). , DN)
  • the distance noise is increased from 5 to 50
  • the unit of distance noise is mm.
  • the relationship between the phase delay value and the unwrapping coefficient similar to Figure 3 can be obtained.
  • the minimum phase distance phi ith between two neighboring line segments can be obtained by geometric calculation,
  • the phase delay value corresponding to each modulation frequency can be calculated, and the phase delay value corresponding to each modulation frequency contains errors.
  • the phase delay value corresponding to the modulation frequency j is expressed as is the real phase delay value of modulation frequency j, is the error of the phase delay value of the modulation frequency j.
  • phase standard deviation can be obtained as:
  • n is the number of a plurality of modulation frequencies.
  • the first threshold corresponding to the modulation frequency j is obtained by dividing A th by g, where g is the target downsampling parameter mentioned above, and the target downsampling parameter is the downsampling parameter used when obtaining the depth map of the first resolution.
  • the reason for dividing A th by g here is: when calculating the first threshold, what is calculated is the gray value of each pixel before the original downsampling process (the minimum signal-to-noise ratio requirement that the gray value satisfies), but After the downsampling process, (g*g) pixels are merged into one pixel, the signal-to-noise ratio is increased by g times, and the minimum signal reception intensity of a newly formed pixel can become 1/g of the minimum signal reception intensity of the original pixel. .
  • Mode 3 Use raw data of multiple modulation frequencies and phase delay maps of multiple modulation frequencies to obtain the unwrapping coefficients of each pixel in the initial depth map of the second resolution; use the unwrapping coefficients of each pixel in the initial depth map and Calculate the phase delay value of each pixel of the second resolution to obtain the depth value of each pixel in the initial depth map; perform downsampling processing on the initial depth map to obtain the depth map of the first resolution.
  • the ranging device may substitute the raw data of the modulation frequency j into Equation (11) to determine the phase delay value of each pixel of the second resolution under the modulation frequency j, so that the second resolution
  • Each pixel of the rate corresponds to a phase delay value at each modulation frequency.
  • use the multiple phase delay values of the pixel to determine the line segment closest to the location of the multiple phase delay values in the corresponding relationship diagram between the phase delay value and the unwrapping coefficient, and determine the sequence number corresponding to the line segment.
  • the unwrapping coefficients of are the corresponding unwrapping coefficients of the pixel at multiple frequencies.
  • the multiple modulation frequencies are two modulation frequencies.
  • the depth value of each pixel at multiple modulation frequencies is averaged, and the average value corresponding to each pixel is determined as the depth value of each pixel of the second resolution.
  • the depth value of each pixel of the second resolution is formed into an initial depth map of the second resolution.
  • the initial depth map of the second resolution is down-sampled to obtain the depth map of the first resolution, where the first resolution is lower than the second resolution.
  • the process of downsampling the initial depth map of the second resolution is as follows:
  • the second downsampling parameter uses the second downsampling parameter to downsample the initial depth map to obtain a depth map of the third resolution; determine the gradient map corresponding to the depth map of the third resolution; if the gradient map satisfies the gradient condition, determine the third resolution
  • the depth map of the first resolution is the depth map of the first resolution; if the gradient map does not meet the gradient conditions, the second downsampling parameters are updated, and the initial depth map is downsampled based on the updated downsampling parameters until the gradient conditions are met.
  • the target gradient map is obtained, and the depth map obtained from the target gradient map is determined as the depth map of the second resolution; wherein, the gradient condition is that the absolute value of the gradient of the pixels whose distance interval is less than the second threshold value is less than or equal to the third threshold value.
  • the second downsampling parameter may be the same as or different from the first downsampling parameter.
  • the third resolution is lower than the second resolution. Where the second downsampling parameter is the same as the first downsampling parameter, the third resolution is equal to the target resolution.
  • the second threshold is a preset threshold, which may be 3, 4, or the like.
  • the third threshold is a preset value.
  • the ranging apparatus may divide the initial depth map of the second resolution into multiple pixel combinations according to the second down-sampling parameter, and the number of pixels in each pixel combination is the same.
  • the depth value of each pixel in each pixel combination is averaged to obtain the depth value when the pixels in each pixel combination are combined into one pixel.
  • the combined depth value of each pixel is formed into a depth map of the third resolution.
  • the ranging device determines the relationship between the absolute value of the gradient of the pixels whose distance interval is less than the second threshold in the gradient map and the third threshold, if the distance interval in the gradient map is less than The absolute value of the gradient of the pixels of the second threshold is less than or equal to the third threshold, then it is determined that the gradient map satisfies the gradient condition, and it can be determined that the depth map of the third resolution is the depth map of the first resolution, that is, the third resolution equal to the first resolution.
  • the down-sampling parameter can be updated by using a preset rule based on the second down-sampling parameter . Use the updated downsampling parameters to downsample the initial depth map of the second resolution, obtain the updated gradient map, and determine whether the updated gradient map satisfies the gradient conditions. If the updated gradient map satisfies the gradient conditions, then It is determined that the depth map obtained from the updated gradient map is the depth map of the first resolution.
  • the gradient map will continue to be updated on the basis of the downsampling parameters after the last update, until the target gradient map that satisfies the gradient conditions is obtained.
  • the depth map obtained from the target gradient map is determined as the depth map of the first resolution.
  • the second downsampling parameter is 2*2, and there are two adjacent pixels in the gradient map (such as the 3rd and 4th pixels in the second row), the absolute value of the gradient is greater than the third threshold, then the downsampling parameter is changed from 2 *2 updated to 3*3.
  • the second downsampling parameter is 2*2, and there are 2 gradients in the gradient graph with an interval of 1 pixel (such as the 3rd and 5th pixels in the second row) whose absolute values of gradients are both greater than the third threshold, then the downsampling parameter is changed from 2*2 is updated to 3*3.
  • the gradient of the pixel can be seen as a pixel with a sudden change. If the gradient of the pixel has a sudden change, the absolute value of the gradient will be relatively large. For example, there are 5 pixels in the depth map, which are [0 0 100 0 0], and the third pixel is [0 0 100 0 0]. The pixel is an abnormal pixel. After the gradient map is calculated, [0 50 0-50 0] is obtained, and the gradient map does not meet the gradient conditions.
  • the third threshold value is related to the depth value.
  • the third threshold value may be thD min f /m, m is a positive integer not greater than a certain value, and the certain value is an empirical value, such as 10 and so on.
  • thD min f U min f *ph ith /(2 ⁇ *2).
  • U min f is the fuzzy distance corresponding to the smallest modulation frequency among the multiple modulation frequencies
  • thD min f is the depth value error threshold corresponding to the smallest modulation frequency
  • ph ith is the minimum value in the phase delay map of n modulation frequencies
  • Phase distance the minimum phase distance is the minimum phase distance between neighboring line segments in the phase delay graph, and the neighboring line segments belong to the unwrapped region in the phase delay graph.
  • the reason for selecting the depth value error threshold of the minimum modulation frequency here is: the U min f of the minimum modulation frequency is relatively large, then the depth value error threshold of the minimum modulation frequency is also relatively large. If the depth value error of the minimum modulation frequency is all can be smaller than the depth value error threshold, the gradient map also satisfies the gradient condition, so the third threshold is set to thD min f /m.
  • the gradient value of any position (x, y) in the depth map of the third resolution can be calculated using the following formula:
  • G(x,y) represents the gradient value at the pixel (x,y) in the depth map of the third resolution
  • dx(i,j) [I(i+1,j)-I(i-1 ,j)]/2
  • dy(i,j) [I(i,j+1)-I(i,j-1)]/2
  • I() represents the depth value of the corresponding pixel, such as I(i +1,j) is the depth value at pixel (i+1,j).
  • the depth value can be directly determined as the gradient value, or it can be calculated in other ways.
  • the number of times to update the downsampling parameters is limited. If the number of updates reaches a certain number and the depth map of the first resolution has not been obtained, it is considered that the obtained raw data is incorrect, and the raw data can be obtained again. data.
  • step 502 may be:
  • DL is the depth value corresponding to pixel k in the depth map of the first resolution
  • N kj is the unwrapping coefficient of pixel k at modulation frequency j
  • U j is the blurring distance corresponding to modulation frequency j.
  • the ranging device may use the formula (19) to determine the unwrapping coefficient of the pixel k at the frequency j in the second resolution.
  • formula (19) can be derived from formula (2), in formula (19), express right Perform rounding operation, D L is the depth value corresponding to pixel k in the depth map of the first resolution, N kj is the unwrapping coefficient of pixel k at modulation frequency j, is the phase delay value corresponding to pixel k at modulation frequency j, and U j is the blurring distance corresponding to modulation frequency j.
  • the unwrapping coefficient of pixel k at modulation frequency 1 is:
  • the unwrapping coefficient of pixel k at modulation frequency 2 is:
  • DL is the depth value of pixel k in the depth map of the first resolution, and are the phase delay values of the pixel k at modulation frequency 1 and modulation frequency 2 , respectively, and U1 and U2 are the fuzzy distances corresponding to modulation frequency 1 and modulation frequency 2, respectively.
  • the second The depth value DL corresponding to the pixel k of the resolution in the depth map of the first resolution is the depth value of a pixel formed by a plurality of pixels including the pixel k.
  • some pixels in the TOF system capture the edge of the object to be measured (scenes with both near and far views).
  • the depth value DL calculated by these pixels and adjacent pixels will have a large error with the real depth value of these pixels before downsampling. Therefore, formula (19) is directly used to calculate the solution of the pixel of the second resolution
  • the entanglement coefficient will make the calculation of the unwrapping coefficient wrong, so the unwrapping coefficient needs to be recalculated for the edge area of the object.
  • the TOF system uses two modulation frequencies (80MHz and 100MHz) to vertically photograph two whiteboards at different distances without noise, and the left 3 columns of pixels are photographed 2.5 meters away , the 5 columns of pixels on the right are taken to the whiteboard 3.5 meters away, when the packing parameter is 2*2, the second column after down-sampling processing (ie the 3rd and 4th columns of the left picture, pixels A and C)
  • the second column after down-sampling processing ie the 3rd and 4th columns of the left picture, pixels A and C
  • pixels B and D in the fourth column contain pixels with two depth values (column 3 2.5 meters, column 4 3.5 meters), assuming that the output depth value D after the two modulation frequencies are unwrapped L (such as 2.49 meters), the depth value will then use formula (19) to calculate the unwrapping coefficients of 2*2 pixels in the edge area, which will cause the calculation of the unwrapping coefficients of pixels B and D to be wrong. Therefore, an edge error detection function can be introduced, which
  • step 503 may be:
  • each pixel of the second resolution uses the unwrapping coefficient of each pixel of the second resolution and the phase delay value of each pixel of the second resolution; calculate the depth value of each pixel of the second resolution for each modulation frequency; use the depth map of the first resolution and the depth value of each pixel of the second resolution of each modulation frequency to determine the depth value error of each pixel of the second resolution of each modulation frequency; using the depth of each pixel of the first resolution of each modulation frequency The value error is calculated to obtain the target depth map of the second resolution corresponding to the measured object.
  • the distance measuring device can substitute the unwrapping coefficients of each pixel of the second resolution of the modulation frequency j, the phase delay value of each pixel at the modulation frequency j, and the fuzzy distance corresponding to the modulation frequency j into Equation (12) , obtain the depth value of each pixel at modulation frequency j. In this way, the depth value of each pixel at each modulation frequency can be determined.
  • the ranging device may then use the depth map of the first resolution and the depth value of each pixel of the second resolution of each modulation frequency to determine the error of the depth value of each pixel of the second resolution of each modulation frequency.
  • the ranging device may use the depth value error of each pixel at the second resolution at each modulation frequency to determine the target depth map of the object to be measured at the second resolution at each modulation frequency.
  • the process of determining the depth value error of each pixel may be:
  • the ranging device can use the formula (22) to determine the depth value error of the pixel k at the second resolution at the modulation frequency j as:
  • E kj represents the absolute value of the error between the test distance D kj and DL of the pixel k after unwrapping at the modulation frequency j .
  • DL is the depth value corresponding to pixel k in the depth map of the first resolution
  • D kj is the depth value of pixel k at the second resolution at modulation frequency j
  • N kj is the unwrapping coefficient of pixel k at modulation frequency j
  • U j is the blurring distance corresponding to modulation frequency j.
  • the depth value corresponding to the pixel k of the second resolution in the depth map of the first resolution is the depth value of the pixel formed by the pixels including the pixel k when downsampling is performed.
  • pixel k is pixel B in FIG. 8
  • the corresponding depth value of pixel B in the depth map of the first resolution is the depth value of the pixel composed of pixels A, B, C and D.
  • the process of determining the target depth map of the second resolution of each modulation frequency may be:
  • the target pixel whose depth value error is greater than the fourth threshold corresponding to the modulation frequency j where j takes a value from 1 to n, n is the number of multiple modulation frequencies; use the depth values of the adjacent pixels of the target pixel in the depth map of the first resolution to update the unwrapping coefficient of the target pixel of the second resolution of the modulation frequency j; use the updated target The unwrapping coefficient of the pixel and the phase delay value of the target pixel of each modulation frequency, update the depth value of the target pixel of the second resolution of the modulation frequency j; divide the pixels of the second resolution of the modulation frequency j by the amount of the target pixel.
  • the depth value of the outer pixel is combined with the updated depth value of the target pixel to obtain the target depth map of the second resolution corresponding to the modulation frequency j of the measured object.
  • the fourth threshold is a preset value, which can be stored in the distance measuring device.
  • the ranging device can obtain the fourth threshold corresponding to the modulation frequency j, and then the ranging device can determine whether the error of the depth value in the pixels of the second resolution of the modulation frequency j is greater than the modulation frequency j
  • the fourth threshold in the depth map of the second resolution with the modulation frequency j, the target pixels whose depth value error is greater than the fourth threshold are obtained.
  • the depth map of the first resolution the depth values of adjacent pixels of the target pixel are determined. For example, in FIG.
  • the target pixel of the second resolution is pixel B
  • the adjacent pixels of pixel B in the depth map of the first resolution are pixels P, M and N
  • the target pixel of the second resolution is A
  • the adjacent pixels of pixel A in the depth map of the first resolution are also pixels P, M and N.
  • the ranging device uses the depth value of the adjacent pixel P, the target pixel of the second resolution at the modulation frequency j
  • the phase delay value of the pixel and the fuzzy distance corresponding to the modulation frequency j are substituted into Equation (19) to obtain the unwrapping coefficient of the target pixel at the second resolution at the modulation frequency j when the depth value of the adjacent pixel P is used.
  • the distance measuring device substitutes the unwrapping coefficient of the target pixel at modulation frequency j, the phase delay value of the target pixel at modulation frequency j and the fuzzy distance corresponding to modulation frequency j into formula (12) to obtain the second resolution at modulation frequency j.
  • the depth value of the target pixel, the depth value is obtained based on the depth value of the adjacent pixel P.
  • the depth value of the adjacent pixel P and the depth value of the target pixel determined above at the modulation frequency j are substituted into formula (22) to obtain the depth value error of the target pixel corresponding to the adjacent pixel P.
  • the depth value error of the target pixel corresponding to each adjacent pixel is determined.
  • the ranging device determines the depth value when the error of the depth value is the smallest and less than the fourth threshold corresponding to the modulation frequency j, as the depth value of the target pixel at the modulation frequency j.
  • the depth value of the target pixel at the first frequency corresponding to any depth value error is determined as the depth value of the target pixel at the modulation frequency j.
  • the target pixel is pixel B
  • the adjacent pixels of pixel B in the depth map of the first resolution are pixels P, M, and N. Since pixel B and pixel M are closer to the TOF system, so The depth value calculated based on the neighboring pixels M has the smallest error.
  • the depth value error of each adjacent pixel corresponding to the target pixel is greater than the fourth threshold, it means that the raw data obtained this time is incorrect, and the raw data is re-acquired to calculate the depth map.
  • the ranging device combines the depth values of pixels other than the target pixel among the pixels of the second resolution at the modulation frequency j with the updated depth value of the target pixel to obtain the second resolution of the measured object at the modulation frequency j.
  • Target depth map The target depth map of the second resolution of the measured object for each modulation frequency can be determined in the above-mentioned manner.
  • the correct depth value of the adjacent pixels can be used to calculate the depth value of each pixel in the region with the wrong depth value, so it can be guaranteed that the depth values of all pixels can be calculated correctly, and the obtained Correct depth map at native resolution.
  • the minimum phase distance between neighboring line segments in the phase delay graph, which belong to the unwrapped region in the phase delay graph, and j ranges from 1 to n.
  • the fourth threshold represents the minimum depth value error when unwrapping errors occur. For any modulation frequency, if the depth value error of a certain pixel is greater than the fourth threshold thD corresponding to the modulation frequency, an unwrapping error occurs, and an incorrect unwrapping coefficient is determined.
  • the error of the phase delay value exceeds one-half of the minimum phase distance, it may lead to the determination of the unwrapping coefficient of the neighbor line segment, and the wrong number of unwrapping times will be determined.
  • the probability of determining the unwrapping coefficient of the neighbor line segment is relatively small, so the distance corresponding to half of the minimum phase distance can be used as the threshold of the depth value error.
  • step 901 starts.
  • Step 902 from the TOF system, obtain raw data of multiple modulation frequencies
  • Step 903 using the first down-sampling parameter as the current down-sampling parameter, down-sampling the raw data of multiple modulation frequencies to obtain the raw data of the target resolution of each modulation frequency, and using the raw data of multiple modulation frequencies. data, determine the phase delay value of each pixel of the second resolution of each modulation frequency;
  • Step 904 based on the raw data of the target resolution of each modulation frequency, determine the depth map of the target resolution and the grayscale map of the target resolution of each modulation frequency;
  • Step 905 judging whether the grayscale value of each pixel in the grayscale image of each target resolution is greater than or equal to the first threshold of the modulation frequency to which it belongs;
  • Step 906 if the determination result in step 905 is yes, determine the depth map of the target resolution as the depth map of the first resolution, and execute step 907 . If the judgment result of step 905 is no, return to update the downsampling parameters, use the updated downsampling parameters as the current downsampling parameters, and perform the process of downsampling the raw data of multiple modulation frequencies until the first resolution is obtained. rate depth map;
  • Step 907 using the depth map of the first resolution, calculate the unwrapping coefficients of each pixel of the second resolution at each modulation frequency;
  • Step 908 using the unwrapping coefficient of each pixel of the second resolution, the depth map of the first resolution and the phase delay value of each pixel of the second resolution, calculate the difference of each pixel of the second resolution at each modulation frequency. depth value error;
  • Step 909 judging whether there is a pixel whose depth value error is greater than the fourth threshold corresponding to the corresponding modulation frequency in each pixel of the second resolution of each modulation frequency;
  • Step 910 if the judgment result of step 909 is yes, then output the depth map of the second resolution corresponding to the measured object at each modulation frequency;
  • Step 911 if the judgment result of step 909 is no, then recalculate the unwrapping coefficient of the target pixel based on the adjacent pixels of the target pixel in the depth map of the first resolution, and return to perform step 908 until the first time of each modulation frequency.
  • the depth value error of each pixel of the two resolutions is less than or equal to the fourth threshold of the corresponding modulation frequency, and the depth map of the second resolution corresponding to the measured object at each modulation frequency is output.
  • the target pixel has a depth value error greater than this
  • the modulation frequency corresponds to the pixel of the fourth threshold.
  • the pixels obtained after downsampling do not have the depth map of the first resolution with abnormal unwrapping coefficients, so based on the first resolution
  • the unwrapping coefficient of each pixel in the second resolution calculated from the depth map of the rate is accurate, and the depth map of the second resolution calculated based on the unwrapping coefficient is also more accurate, which ensures the accuracy of the depth map of the second resolution.
  • step 1001 starts.
  • Step 1002 from the TOF system, obtain raw data of multiple modulation frequencies
  • Step 1003 calculate the unwrapping coefficient of each pixel of the second resolution at each modulation frequency, the phase delay value of each pixel at each modulation frequency, and use the unwrapping coefficient of each pixel of the second resolution, the second resolution
  • the phase delay value of each pixel of calculates the depth map of the second resolution
  • Step 1004 taking the second down-sampling parameter as the current down-sampling parameter, down-sampling the depth map of the second resolution, obtaining the depth map of the third resolution, and determining the gradient map of the depth map of the third resolution;
  • Step 1005 judging whether the gradient map of the depth map of the third resolution satisfies the gradient condition
  • Step 1006 when the gradient map of the depth map of the third resolution satisfies the gradient condition, determine that the depth map of the third resolution is the depth map of the first resolution, and perform step 1007; In the case that the gradient map does not meet the gradient condition, return to update the downsampling parameters, use the updated downsampling parameters as the current downsampling parameters, and execute step 1004;
  • Step 1007 using the depth map of the third resolution to calculate the unwrapping coefficients of each pixel of the second resolution at each modulation frequency;
  • Step 1008 Determine the second resolution based on the unwrapping coefficient of each pixel of the second resolution at each modulation frequency, the depth map of the first resolution and the phase delay value of each pixel of the second resolution at each modulation frequency. The depth value error of each pixel of the rate at each modulation frequency;
  • Step 1009 judging whether there is a pixel whose depth value error is greater than the fourth threshold in each pixel of the second resolution
  • Step 1010 if the judgment result of Step 1009 is yes, output the depth map of the second resolution corresponding to the measured object at each modulation frequency;
  • Step 1011 if the judgment result of step 1009 is no, then recalculate the unwrapping coefficient of the target pixel based on the adjacent pixels of the target pixel in the depth map of the first resolution, and return to perform step 908 until the first pixel of each modulation frequency.
  • the depth value error of each pixel of the two resolutions is less than or equal to the fourth threshold of the corresponding modulation frequency, and the depth map of the second resolution corresponding to the measured object at each modulation frequency is output.
  • the target pixel has a depth value error greater than this Pixels of the fourth threshold of modulation frequency.
  • the depth map with abnormally unwrapped pixels is down-sampled, so that the down-sampled depth map of the first resolution has no pixels with abnormal unwrapping coefficients, so the second resolution calculated based on the depth map of the first resolution
  • the unwrapping coefficient of each pixel is accurate, and the depth map of the second resolution calculated based on the unwrapping coefficient is also relatively accurate, which ensures the accuracy of the depth map of the second resolution.
  • the determined unwrapping coefficient is wrong, which in turn leads to the problem of reducing the distance accuracy, and the low resolution of the pixel without abnormal unwrapping coefficient is determined.
  • the depth map of , and the unwrapping coefficient of each pixel of the original resolution is calculated by the low-resolution depth map, and the accurate unwrapping coefficient can be determined, and then the accurate depth map can be determined.
  • the correct depth value of the adjacent pixels of the pixel is used to determine the depth value, so that the depth values of all pixels are correct depth values.
  • all calculation processes involved may be obtained by using a floating-point calculator, or obtained by looking up a table.
  • FIG. 11 is a structural diagram of an apparatus for acquiring a depth map provided by an embodiment of the present application.
  • the apparatus can be implemented by software, hardware or a combination of the two to become a part or all of the apparatus.
  • the apparatus provided by the embodiment of the present application can implement the process described in FIG. 5 in the embodiment of the present application.
  • the apparatus includes: a downsampling module 1110, a disentanglement coefficient determination module 1120, and a depth map determination module 1130, wherein:
  • the down-sampling module 1110 is configured to perform down-sampling processing on the raw data of multiple modulation frequencies of the measured object to obtain a depth map of the first resolution, wherein the raw data of each modulation frequency is the TOF system for the raw data.
  • the measured object is obtained by multiple exposures at each modulation frequency during ranging, and the depth map of the first resolution is a depth map that is determined to be a pixel without abnormal unwrapping coefficients.
  • the resolution is the second resolution, and the first resolution is lower than the second resolution, which can specifically be used to implement the downsampling function of step 501 and perform the implicit steps included in step 501;
  • an unwrapping coefficient determination module 1120 configured to use the depth map of the first resolution and the phase delay value of each pixel of the second resolution to calculate and obtain the unwrapping coefficient of each pixel of the second resolution, wherein, the phase delay value of each pixel of the second resolution is calculated and obtained by using the raw data of the multiple modulation frequencies, which can be specifically used to realize the function of determining the unwrapping coefficient in step 502 and to execute the implicit information contained in step 502. step;
  • the depth map determination module 1130 is further configured to use the unwrapping coefficient of each pixel of the second resolution and the phase delay value of each pixel of the second resolution to calculate and obtain the corresponding value of the measured object.
  • the target depth map of the second resolution can be specifically used to implement the depth map determination function in step 503 and perform the implicit steps included in step 503 .
  • the downsampling module 1110 is configured to:
  • down-sampling processing is performed on the raw data of multiple modulation frequencies of the measured object to obtain a grayscale image of the target resolution corresponding to the raw data of each of the modulation frequencies and a grayscale image of the target resolution.
  • the depth map of the target resolution is the depth map of the first resolution
  • the first downsampling parameter is updated, and the plurality of modulations are modulated based on the updated downsampling parameter.
  • the raw data of the frequency is subjected to down-sampling processing until the depth map of the first resolution is obtained.
  • the downsampling module 1110 is configured to:
  • down-sampling is performed on the raw data of multiple modulation frequencies of the measured object to obtain a grayscale image of the target resolution corresponding to the raw data of each of the modulation frequencies;
  • the first downsampling parameter is used to downsample the raw data of the plurality of modulation frequencies. Sampling processing to obtain the depth map of the first resolution;
  • the first downsampling parameter is updated, and the plurality of modulations are modulated based on the updated downsampling parameter.
  • the raw data of the frequency is subjected to down-sampling processing until the depth map of the first resolution is obtained.
  • the first threshold corresponding to each modulation frequency is the lowest unwrapped received signal strength in the corresponding range noise curve obtained in the TOF system.
  • the downsampling module 1110 is configured to:
  • the downsampling module 1110 is configured to:
  • the gradient map satisfies the gradient condition, determining that the depth map of the third resolution is the depth map of the first resolution
  • the gradient map does not satisfy the gradient condition, update the second downsampling parameter, and perform downsampling processing on the initial depth map based on the updated downsampling parameter until a target gradient satisfying the gradient condition is obtained , determining the depth map obtained from the target gradient map as the depth map of the second resolution;
  • the gradient condition is that the absolute value of the gradient of the pixels whose distance interval is smaller than the second threshold is smaller than or equal to the third threshold.
  • the disentanglement coefficient determination module 1120 is configured to:
  • the unwrapping coefficient of the pixel k of the second resolution that determines the modulation frequency j is:
  • the modulation frequency j belongs to the plurality of modulation frequencies
  • round[] is the rounding operation
  • D L is the depth value corresponding to the pixel k in the depth map of the first resolution
  • N kj is the the unwrapping coefficient of the pixel k at the modulation frequency j
  • U j is the blurring distance corresponding to the modulation frequency j.
  • the depth map determining module 1130 is configured to:
  • a target depth map of the second resolution corresponding to the measured object is obtained by calculation.
  • DL is the depth value corresponding to the pixel k in the depth map of the first resolution
  • D kj is the depth value of the pixel k of the second resolution of the modulation frequency j
  • N kj is the unwrapping coefficient of the pixel k at the modulation frequency j
  • U j is the fuzzy distance corresponding to the modulation frequency j
  • j takes a value 1 to n, where n is the number of the plurality of modulation frequencies.
  • the depth map determining module 1130 is configured to:
  • modulation frequency j among the plurality of modulation frequencies, in the pixels of the second resolution of the modulation frequency j, determine the target pixel whose depth value error is greater than the fourth threshold corresponding to the modulation frequency j, wherein, j takes a value from 1 to n, where n is the number of the plurality of modulation frequencies;
  • the division of modules in the embodiments of the present application is schematic, and is only a logical function division. In actual implementation, there may also be other division methods.
  • the functional modules in the various embodiments of the present application may be integrated into one
  • the processor may also exist physically alone, or two or more modules may be integrated into one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules.
  • the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solutions of the present application can be embodied in the form of software products in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, and the computer software products are stored in a storage medium , including several instructions to make a terminal device (which may be a personal computer, a mobile phone, or a network device, etc.) or a processor (processor) to execute all or part of the steps of the method in each embodiment of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes .
  • FIG. 12 exemplarily provides a possible architectural diagram of a computing device 1200 .
  • Computing device 1200 includes memory 1201 , processor 1202 , communication interface 1203 , and bus 1204 .
  • the memory 1201 , the processor 1202 , and the communication interface 1203 are connected to each other through the bus 1204 for communication.
  • the memory 1201 may be ROM, static storage device, dynamic storage device or RAM.
  • the memory 1201 may store programs, and when the programs stored in the memory 1201 are executed by the processor 1202, the processor 1202 and the communication interface 1203 are used to execute the method for acquiring a depth map.
  • the memory 1201 may also store raw data.
  • the processor 1202 can be a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), a graphics processor (graphics processing unit, GPU), or one or more integrated circuit.
  • CPU central processing unit
  • ASIC application specific integrated circuit
  • GPU graphics processing unit
  • the processor 1202 may also be an integrated circuit chip with signal processing capability. In the implementation process, part or all of the functions of the apparatus for acquiring a depth map of the present application may be implemented by hardware integrated logic circuits in the processor 1202 or instructions in the form of software.
  • the above-mentioned processor 1202 can also be a general-purpose processor, a digital signal processor (digital signal processing, DSP), an application-specific integrated circuit, an off-the-shelf programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, discrete gates. Or transistor logic devices, discrete hardware components.
  • DSP digital signal processing
  • FPGA field programmable gate array
  • the methods, steps and logic block diagrams disclosed in the above embodiments of the present application can be implemented or executed.
  • a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
  • the steps of the methods disclosed in conjunction with the embodiments of the present application may be directly embodied as executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor.
  • the software module may be located in random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers and other storage media mature in the art.
  • the storage medium is located in the memory 1201, and the processor 1202 reads the information in the memory 1201, and performs part of the functions of the apparatus for acquiring a depth map of the embodiment of the present application in combination with its hardware.
  • the communication interface 1203 uses a transceiver module such as, but not limited to, a transceiver to enable communication between the computing device 1200 and other devices or a communication network.
  • a transceiver module such as, but not limited to, a transceiver to enable communication between the computing device 1200 and other devices or a communication network.
  • the data set can be obtained through the communication interface 1203 .
  • Bus 1204 may include pathways for communicating information between various components of computing device 1200 (eg, memory 1201, processor 1202, communication interface 1203).
  • the above-mentioned embodiments may be implemented in whole or in part by software, hardware, firmware or any combination thereof, and when implemented in software, it may be implemented in whole or in part in the form of a computer program product.
  • the computer program product includes one or more computer instructions, and when the computer program instructions are loaded and executed on a server or terminal, all or part of the processes or functions described in the embodiments of the present application are generated.
  • the computer instructions may be stored in or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line) or wireless (eg, infrared, wireless, microwave, etc.).
  • the computer-readable storage medium may be any available medium that can be accessed by a server or a terminal, or a data storage device such as a server, a data center, or the like that includes the integration of one or more available media.
  • the usable medium may be a magnetic medium (such as a floppy disk, a hard disk, and a magnetic tape, etc.), an optical medium (such as a digital video disk (DVD), etc.), or a semiconductor medium (such as a solid-state disk, etc.).
  • a magnetic medium such as a floppy disk, a hard disk, and a magnetic tape, etc.
  • an optical medium such as a digital video disk (DVD), etc.
  • a semiconductor medium such as a solid-state disk, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Image Processing (AREA)
  • Radar Systems Or Details Thereof (AREA)

Abstract

本申请提供了一种获取深度图的方法、装置、计算设备和可读存储介质，属于图像技术领域。该方法包括：对被测物体的多个调制频率的裸数据进行下采样处理，获得第一分辨率的深度图，每个调制频率的裸数据为TOF系统对被测物体测距时在每个调制频率下多次曝光获得，第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图，TOF系统像素阵列的分辨率为第二分辨率，第一分辨率低于第二分辨率，使用第一分辨率的深度图和第二分辨率的各像素的相位延迟值，获得第二分辨率的各像素的解缠系数；使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，获得被测物体对应的第二分辨率的目标深度图。采用本申请，提升深度图的准确率。

Description

获取深度图的方法、装置、计算设备和可读存储介质 技术领域
本申请涉及图像技术领域，特别涉及一种获取深度图的方法、装置、计算设备和可读存储介质。
背景技术
目前常常通过飞行时间(time of flight，TOF)成像技术测量距离，在TOF成像技术中，TOF三维成像系统(简称为TOF系统)中光源发出光线照射目标物体，经过目标物体反射后的光线返回TOF三维成像系统中的TOF芯片(即图像传感器)，TOF芯片的像素阵列中各像素可以接收光信号，基于发射光信号和接收到的光信号的相位差，确定出像素阵列中各像素与目标物体的距离，获得深度图。
由于相位每隔2π就会重叠，所以仅使用一种调制频率的光线测量距离，会导致测量的距离不准确，因此可以使用多种调制频率的光线测量距离。例如，使用两种调制频率的光线测量距离，在使用两种调制频率的光线测量距离时，对于像素阵列中的每个像素，确定该像素在两种调制频率分别对应的相位延迟值，并且确定每种调制频率对应的模糊距离，然后使用在该像素的相位延迟值，确定出在该像素两种调制频率对应的解缠系数。使用相位延迟值、模糊距离以及解缠系数确定出在两种调制频率下测量获得的距离。
由于TOF系统存在噪声干扰，所以相位延迟值会受到噪声的影响，在理想值附近波动。在相位延迟值由于受到干扰波动比较大时，相位延迟值不准确，会导致解缠系数不准确，进而导致获得的深度图不准确。
发明内容
本申请实施例提供了一种获取深度图的方法、装置、计算设备和可读存储介质，可以获得准确的深度图。
第一方面，本申请提供一种获取深度图的方法，该方法包括：对被测物体的多个调制频率的裸(raw)数据进行下采样处理，获得第一分辨率的深度图，其中，每个调制频率的raw数据为TOF系统对被测物体测距时在每个调制频率下多次曝光获得，第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图，TOF系统中像素阵列的分辨率为第二分辨率，第一分辨率低于第二分辨率；使用第一分辨率的深度图和第二分辨率的各像素的相位延迟值，计算获得第二分辨率的各像素的解缠系数，其中，第二分辨率的各像素的相位延迟值是使用多个调制频率的raw数据计算获得；使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算获得被测物体对应的第二分辨率的目标深度图。
其中，被测物体为任一物体，如汽车等。调制频率为对某一频率的光信号进行强度调制使用的频率。第二分辨率的每个像素的相位延迟值是多个，分别为每个像素在各个调制频率下的相位延迟值。第二分辨率的每个像素的解缠系数为多个，分别为每个像素在各个调制频率下的解缠系数。异常解缠系数也可以称为是错误解缠系数。
本申请所示的方案，获取深度图的方法的执行主体可以是测距装置，在对被测物体测距时，在任一调制频率下TOF系统进行多次曝光，获得该调制频率的raw数据。然后对多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，该深度图中各像素被判定为像素不存在异常解缠系数。然后使用第一分辨率的深度图和第二分辨率的各像素的相位延迟值，计算获得第二分辨率的各像素的解缠系数。使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算获得被测物体对应的第二分辨率的目标深度图。这样，由于下采样后的第一分辨率的深度图中不存在异常解缠系数的像素，第二分辨率的解缠系数在计算时，使用了第一分辨率的深度图，所以使用该解缠系数计算出的第二分辨率的目标深度图也比较准确，保证了第二分辨率的目标深度图的精度。
在一种可能的实现方式中，对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，包括：使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个调制频率的raw数据对应的目标分辨率的灰度图和目标分辨率的深度图；若每个灰度图中不存在灰度值小于所属调制频率对应的第一阈值的像素，则确定目标分辨率的深度图为第一分辨率的深度图；若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则更新第一下采样参数，基于更新后的下采样参数对多个调制频率的raw数据进行下采样处理，直至获得第一分辨率的深度图。
其中，第一下采样参数为用于降低分辨率的参数，例如，第一下采样参数为2*2，2*2表示4，在处理时将原来的2*2个像素合并为一个像素。另外第一下采样参数也可以称为是第一装箱(binning)参数，下采样处理也可以称为是binning处理。
本申请所示的方案，测距装置使用第一下采样参数，对每个调制频率的raw数据进行下采样处理，获得每个调制频率对应的目标分辨率的灰度图，并且可以获得一个目标分辨率的深度图。目标分辨率低于第二分辨率。然后判断每个目标分辨率的灰度图中是否存在灰度值小于所属调制频率对应的第一阈值的像素，若不存在，则确定目标分辨率的深度图为第一分辨率的灰度图。若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则更新第一下采样参数，使得第一下采样参数变大，也就是说使得更多的像素合并为一个像素。使用更新后的下采样参数重新对多个调制频率的raw数据进行下采样处理，再次获得多个灰度图和一个深度图，该多个灰度图的分辨率低于目标分辨率。判断多个灰度图中是否存在灰度值小于所属调制频率对应的第一阈值的像素，若不存在，则将该一个深度图，确定为第一分辨率的深度图，若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则继续更新下采样参数，执行上述处理，直至获得第一分辨率的深度图。
这样，可以获取到像素不存在异常解缠系数的深度图。
在一种可能的实现方式中，对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，包括：使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个调制频率的raw数据对应的目标分辨率的灰度图；若每个灰度图中不存在灰度值小于所属调制频率对应的第一阈值的像素，则使用第一下采样参数，对多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图；若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则更新第一下采样参数，基于更新后的下采样参数对多个调制频率的raw数据进行下采样处理，直至获得第一分辨率的深度图。
本申请所示的方案，测距装置使用第一下采样参数，对每个调制频率的raw数据进行下 采样处理，获得每个调制频率对应的目标分辨率的灰度图。目标分辨率低于第二分辨率。然后判断每个目标分辨率的灰度图中是否存在灰度值小于所属调制频率对应的第一阈值的像素，若不存在，则使用第一下采样参数，对多个调制频率对应的raw数据进行下采样处理，获得第一分辨率的灰度图。若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则更新第一下采样参数，使得第一下采样参数变大，也就是说使得更多的像素合并为一个像素。使用更新后的下采样参数重新对多个调制频率的raw数据进行下采样处理，再次获得多个灰度图，该多个灰度图的分辨率低于目标分辨率。判断多个灰度图中是否存在灰度值小于所属调制频率对应的第一阈值的像素，若不存在，则使用更新后的下采样参数，对多个调制频率对应的raw数据进行下采样处理，获得第一分辨率的深度图，若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则继续更新下采样参数，执行上述处理，直至获得第一分辨率的深度图。
这样，可以获取到像素不存在异常解缠系数的深度图。
在一种可能的实现方式中，每个所述调制频率对应的第一阈值是对应的距离噪声曲线中解缠无误的最低信号接收强度，所述距离噪声曲线是在所述TOF系统中获得。
其中，在TOF系统确定后，每个调制频率都对应有一条距离噪声曲线，距离噪声曲线绘制在直角坐标系后，横轴为接收的光信号强度，纵轴为某个光信号强度下多次测量同一距离的被测物体的深度值标准差。在深度值标准差满足一定条件时，对应的信号接收强度即为解缠无误的最低信号接收强度。
在一种可能的实现方式中，对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，包括：使用多个调制频率的raw数据和多个调制频率的相位延迟图，获得第二分辨率的初始深度图中各像素的解缠系数；使用初始深度图中各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算获得初始深度图中各像素的深度值；对初始深度图进行下采样处理，获得第一分辨率的深度图。
其中，相位延迟图为相位延迟值与解缠系数的对应关系图，在该相位延迟图中，可以使用相位延迟值对应到解缠系数。
本申请所示的方案，测距装置可以将每个调制频率的raw数据代入相位延迟值计算公式，确定在每个调制频率，第二分辨率的各像素的相位延迟值。然后对于任一像素，使用该像素的多个相位延迟值在相位延迟图中，确定距离多个相位延迟值所在位置点最近的线段，确定该线段的序号对应的解缠系数，将该解缠系数确定为该像素在多个频率下对应的解缠系数。然后对于第二分辨率的像素k，将N kj、
Figure PCTCN2021083673-appb-000001
和模糊距离U j代入公式
Figure PCTCN2021083673-appb-000002
获得在调制频率j下像素k的深度值D kj，N kj为在调制频率j像素k对应的解缠系数，
Figure PCTCN2021083673-appb-000003
为在调制频率j像素k相位延迟值。基于这种方式，获得在调制频率j，每个像素的深度值。将每个像素的多个调制频率下的深度值取平均值，将每个像素对应的平均值确定为第二分辨率的每个像素的深度值。将第二分辨率的每个像素的深度值组成第二分辨率的初始深度图。然后将第二分辨率的初始深度图进行下采样处理，获得第一分辨率的深度图，第一分辨率低于第二分辨率。
这样，可以获取到第一分辨率的深度图。
在一种可能的实现方式中，对初始深度图进行下采样处理，获得第一分辨率的深度图， 包括：使用第二下采样参数，对初始深度图进行下采样处理，获得第三分辨率的深度图；确定第三分辨率的深度图对应的梯度图；若梯度图满足梯度条件，则确定第三分辨率的深度图为第一分辨率的深度图；若梯度图不满足梯度条件，则更新第二下采样参数，基于更新后的下采样参数对初始深度图进行下采样处理，直至获得满足梯度条件的目标梯度图，将获得目标梯度图的深度图确定为第二分辨率的深度图；其中，梯度条件为距离间隔小于第二阈值的像素的梯度绝对值小于或等于第三阈值。
其中，第二下采样参数为用于降低分辨率的参数，例如，第二下采样参数为2*2，2*2表示4，在处理时将原来的2*2个像素合并为一个像素。另外第二下采样参数也可以称为是第二binning参数，下采样处理也可以称为是binning处理。
本申请所示的方案，测距装置可以使用第二下采样参数，将第二分辨率的初始深度图中划分为多个像素组合，每个像素组合中的像素数目相同。将每个像素组合中各像素的深度值取平均值，获得每个像素组合中的像素合并为一个像素时的深度值，将合并后的深度值，组成第三分辨率的深度图。
然后确定第三分辨率的深度图对应的梯度图，测距装置判断该梯度图中距离间隔小于第二阈值的像素的梯度绝对值与第三阈值的大小关系，若该梯度图中距离间隔小于第二阈值的像素的梯度绝对值均小于或等于第三阈值，则确定该梯度图满足梯度条件，可以确定第三分辨率的深度图为第一分辨率的深度图，也就是说第三分辨率等于第一分辨率。反之确定梯度图不满足梯度条件，可以在第二下采样参数的基础上使用预设的规则，更新第二下采样参数。使用更新后的下采样参数对第二分辨率的初始深度图进行下采样处理，获得更新后的梯度图，判断更新后的梯度图是否满足梯度条件，若更新后的梯度图满足梯度条件，则确定获得更新后的梯度图的深度图为第一分辨率的深度图。若更新后的梯度图不满足梯度条件，则在上一次更新后的下采样参数的基础上，继续更新梯度图，直至获得满足梯度条件的目标梯度图。将获得目标梯度图的深度图确定为第一分辨率的深度图。
这样，在像素的梯度存在突变时，说明梯度绝对值比较大，梯度又与深度值误差相关，对于任一像素，在能获取到该像素在某个调制频率的正确解缠系数时，深度值误差通常小于一定数值，所以可以使用梯度条件，获取到第一分辨率的深度图。
在一种可能的实现方式中，使用第一分辨率的深度图和第二分辨率的各像素的相位延迟值，计算获得第二分辨率的各像素的解缠系数，包括：确定调制频率j的第二分辨率的像素k的解缠系数为：
Figure PCTCN2021083673-appb-000004
其中，调制频率j属于多个调制频率，round[]为四舍五入操作，D L为像素k在第一分辨率的深度图中对应的深度值，N kj为在调制频率j像素k的解缠系数，
Figure PCTCN2021083673-appb-000005
为在调制频率j像素k的相位延迟值，U j为调制频率j对应的模糊距离。
其中，由于第一分辨率低于第二分辨率，且第一分辨率的像素由第二分辨率下的多个像素下采样处理形成，所以在计算解缠系数时，第二分辨率的像素k在第一分辨率的深度图中对应的深度值D L为包括像素k的多个像素形成的像素的深度值。
在一种可能的实现方式中，使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算获得被测物体对应的第二分辨率的目标深度图，包括：使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算每个调制频率的第二分辨率 的各像素的深度值；使用第一分辨率的深度图和每个调制频率的第二分辨率的各像素的深度值，确定每个调制频率的第二分辨率的各像素的深度值误差；使用每个调制频率的第一分辨率的各像素的深度值误差，计算获得被测物体对应的第二分辨率的目标深度图。
本申请所示的方案，对于多个调制频率中的调制频率j、第二分辨率的像素k，测距装置可以将N kj、
Figure PCTCN2021083673-appb-000006
和模糊距离U j代入公式
Figure PCTCN2021083673-appb-000007
获得在调制频率j下像素k的深度值D kj，N kj为在调制频率j像素k对应的解缠系数，
Figure PCTCN2021083673-appb-000008
为在调制频率j像素k相位延迟值。基于这种方式，可以确定出在每个调制频率各像素的深度值。
然后测距装置可以使用第一分辨率的深度图和每个调制频率的第二分辨率的各像素的深度值，确定每个调制频率的第二分辨率的各像素的深度值误差。测距装置可以使用每个调制频率的第二分辨率的各像素的深度值误差，确定被测物体对应的第二分辨率的目标深度图。
这样，由于考虑了深度值误差，所以可以提升目标深度图的精度。
在一种可能的实现方式中，使用第一分辨率的深度图和每个调制频率的第二分辨率的各像素的深度值，确定每个调制频率的第二分辨率的各像素的深度值误差，包括：确定调制频率j的第二分辨率的像素k的深度值误差为：E kj＝|D L-D kj|；其中，D L为像素k在第一分辨率的深度图中对应的深度值，
Figure PCTCN2021083673-appb-000009
D kj为调制频率j的第二分辨率的像素k的深度值，
Figure PCTCN2021083673-appb-000010
为在调制频率j像素k的相位延迟值，N kj为在调制频率j像素k的解缠系数，U j为调制频率j对应的模糊距离，j取值为1至n，n为多个调制频率的数目。
在一种可能的实现方式中，使用每个调制频率的第一分辨率的各像素的深度值误差，计算获得被测物体对应的第二分辨率的目标深度图，包括：对于多个调制频率中调制频率j，在调制频率j的第二分辨率的像素中，确定深度值误差大于调制频率j对应的第四阈值的目标像素，其中，j取值为1至n，n为多个调制频率的数目；使用第一分辨率的深度图中目标像素的相邻像素的深度值，更新调制频率j的第二分辨率的目标像素的解缠系数；使用更新后的目标像素的解缠系数和每个调制频率的目标像素的相位延迟值，更新调制频率j的第二分辨率的目标像素的深度值；将调制频率j的第二分辨率的像素中除目标像素之外的像素的深度值和更新后的目标像素的深度值组合，获得被测物体对应的调制频率j的第二分辨率的目标深度图。
本申请所示的方案，对于调制频率j，测距装置可以获取调制频率j对应的第四阈值，然后测距装置可以判断调制频率j的第二分辨率的像素中深度值误差是否大于调制频率j对应的第四阈值，在调制频率j的第二分辨率的深度图中，获得深度值误差大于第四阈值的目标像素。然后在第一分辨率的深度图中，确定目标像素的相邻像素的深度值。
假设目标像素为像素B，对于目标像素在第一分辨率的深度图中的任一相邻像素P，测距装置使用该相邻像素P的深度值、调制频率j的第二分辨率的目标像素对应的相位延迟值和调制频率j对应的模糊距离，计算获得使用该相邻像素P的深度值时，在调制频率j第二分辨率的目标像素对应的解缠系数。
然后测距装置使用在调制频率j目标像素对应的解缠系数、在调制频率j目标像素对应的相位延迟值和调制频率j对应的模糊距离，计算获得在调制频率j第二分辨率的目标像素的深度值。确定此次计算的目标像素的深度值误差。基于上述方式，使用目标像素在第一分辨率 的深度图的每个相邻像素，确定出该每个相邻像素对应的目标像素的深度值误差。测距装置将深度值误差最小，且小于调制频率j对应的第四阈值时的深度值，确定为目标像素在调制频率j的深度值。然后测距装置将调制频率j的第二分辨率的像素中除目标像素之外的像素的深度值与目标像素更新后的深度值组合，获得在调制频率j被测物体的第二分辨率的目标深度图。基于确定在调制频率j被测物体的第二分辨率的目标深度图的方式，确定在每个调制频率被测物体的第二分辨率的目标深度图。
这样，由于在获得深度值出错的区域后，可以使用相邻像素的正确深度值来计算深度值出错的区域的每个像素的深度值，所以可以保证所有像素的深度值均可以计算正确，获得原始分辨率的正确深度图。
在一种可能的实现方式中，调制频率j对应的第四阈值为：thD j＝U j*ph ith/(2π*2)；U j为调制频率j对应的模糊距离，ph ith为多个调制频率的相位延迟图中解缠区域的邻居线段之间的最小相位距离，j取值为1至n，n为多个调制频率的数目。
其中，如果相位延迟值的误差超过二分之一最小相位距离，就有可能会导致确定为邻居线段的解缠系数，进而确定出错误的解缠次数，而在相位延迟值的误差小于最小相位距离的一半时，确定为邻居线段的解缠系数的概率比较小，所以可以将最小相位距离的一半对应的距离，作为深度值误差的阈值。
第二方面，本申请提供了一种获取深度图的装置，该装置包括一个或多个模块，用于实现第一方面所述的获取深度图的方法。
第三方面，本申请提供了一种获取深度图的计算设备，所述计算设备包括处理器和存储器，其中：所述存储器中存储有计算机指令；所述处理器执行所述计算机指令，以实现第一方面及其可能的实现方式的方法。
第四方面，本申请提供了一种计算机可读存储介质，所述计算机可读存储介质存储有计算机指令，当所述计算机可读存储介质中的计算机指令被计算设备执行时，使得计算设备执行第一方面及其可能的实现方式的方法，或者使得计算设备实现上述第二方面及其可能的实现方式的装置的功能。
第五方面，本申请提供了一种包含指令的计算机程序产品，当其在计算设备上运行时，使得计算设备执行上述第一方面及其可能的实现方式的方法。
附图说明
图1是本申请一个示例性实施例提供的相位延迟值与距离的关系图；
图2是本申请一个示例性实施例提供的相位延迟值图中解缠系数的示意图；
图3是本申请一个示例性实施例提供的相位延迟值与解缠系数的关系图；
图4是本申请一个示例性实施例提供的TOF系统的结构示意图；
图5是本申请一个示例性实施例提供的获取深度图的方法的流程示意图；
图6是本申请一个示例性实施例提供的下采样的示意图；
图7是本申请一个示例性实施例提供的接收的光信号强度与距离噪声的关系图；
图8是本申请一个示例性实施例提供的下采样的示意图；
图9是本申请一个示例性实施例提供的获取深度图的方法的流程示意图；
图10是本申请一个示例性实施例提供的获取深度图的方法的流程示意图；
图11是本申请一个示例性实施例提供的获取深度图的装置的结构示意图；
图12是本申请一个示例性实施例提供的计算设备的结构示意图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚，下面将结合附图对本申请实施方式作进一步地详细描述。
为了便于对本申请实施例的理解，下面首先介绍所涉及到的名词的概念：
TOF，飞行时间指往返时间，往返时间为TOF系统中发射器发送的光信号在被测物体与TOF系统中接收器之间的往返时间，通过往返时间可以计算出深度图。
深度图，包括多个像素，每个像素的像素值用于记录TOF系统与被测物体的某一点的距离，即每个像素的像素值为深度值。
模糊距离，对于单个调制频率的TOF系统，表示最大的可测量距离，用公式表示为：
Figure PCTCN2021083673-appb-000011
其中，在式(1)中，f表示调制频率，u表示调制频率f对应的模糊距离，c表示光速。此处调制频率是对某一频率的光信号进行强度调制使用的频率。
击打频率(beat frequency)，是指使用多个调制频率测量距离时，多个调制频率共同测量获得的距离对应的频率。击打频率是该多个调制频率的最大公约数。击打频率相对于该多个调制频率比较低，可以扩展更长测量距离。例如，如图1所示，表示80MHz和100MHz在7.5m内的两个调制频率相位延迟值与距离的关系，在调制频率100MHz下，模糊距离为u 100MHz＝1.5m，在调制频率80MHz下，模糊距离为u 80MHz＝1.875m，使用两个调制频率80MHz和100MHz时，模糊距离为7.5m。说明最大测量距离扩展为7.5m，此处单位“m”表示“米”。
在相关技术中，使用两种调制频率获得深度图时，如图2所示，使用80MHz和100MHz两个调制频率后，模糊距离扩展为7.5m，若TOF系统中没有噪声，则在模糊距离内，每个像素的两个调制频率相位延迟值
Figure PCTCN2021083673-appb-000012
与距离一一对应，80MHz和100MHz的TOF系统中，存在8段解缠区域，在图2中，仅标注了奇数段的解缠区域。如表一所示，给出了80MHz和100MHz的TOF系统8段解缠区域的解缠系数。
表一
序号 解缠系数N1(80MHz) 解缠系数N2(100MHz)
① 0 0
② 0 1
③ 1 1
④ 1 2
⑤ 2 2
⑥ 2 3
⑦ 3 3
⑧ 3 4
对于任一像素，在已知两个调制频率分别对应的解缠系数(N 1，N 2)和相位延迟值
Figure PCTCN2021083673-appb-000013
后，使用式(2)和式(3)计算在两个调制频率下测量获得的距离(D 1，D 2)：
Figure PCTCN2021083673-appb-000014
Figure PCTCN2021083673-appb-000015
可见，为了计算(D 1，D 2)，首先需要计算获得在两个调制频率对应的解缠系数(N 1，N 2)。相关技术中，如图3所示，将两个调制频率对应的相位延迟值按照xy坐标方式画在直角坐标系(x-o-y)中，图3中给出了没有噪声干扰的情况下，80MHz和100MHz的TOF系统8段解缠区域的线段，横轴表示80MHz的相位延迟值，纵轴表示100MHz的相位延迟值。在计算在两个调制频率某个像素对应的解缠系数(N 1，N 2)时，确定该像素对应的相位延迟值在图3中的位置点，确定与该位置点距离最近的线段(③所在线段，可以称为是最近线段)。在表一中确定该最近线段的序号对应的解缠系数，将该解缠系数确定为在两个调制频率该像素分别对应的解缠系数。
由于实际的80MHz和100MHz的TOF系统中存在噪声干扰，所以计算出的相位延迟值会在理想值附近跳动。若某个像素的相位延迟值受到的噪声干扰比较小，跳动不大，只要判断出距离相位延迟值的位置点最近的线段，即可确定出准确的解缠系数，若某个像素的相位延迟值受到的噪声干扰比较大，相位延迟值的位置点偏离正确的线段，该像素会被赋予错误的解缠系数，进而会导致对于该像素基于式(2)和(3)确定出错误的距离，使得测量被测物体获得的深度图错误。
基于上述原因，本申请实施例提供了一种获取深度图的方法。该方法可以由获取深度图的装置执行，后续可以简称为测距装置。该测距装置可以是一个硬件装置，如服务器、终端等计算设备等，也可以是一个软件装置，如是运行在硬件装置上的一套软件程序等。
测距装置可以从TOF系统中获取多个调制频率的raw数据，基于多个调制频率的raw数据，确定深度图，详细过程在后文中描述。在本申请实施例中测距装置可以是TOF系统的一部分，也可以是独立于TOF系统存在。
在测距装置独立于TOF系统时，如图4所示，TOF系统包括TOF芯片101、镜头102、激光器驱动103、激光器104等，TOF芯片101包括控制器1011、数模转换器(analog to digital converter，ADC)1012和像素阵列1013。TOF芯片101可以称为是图像传感器。激光器104可以是发光二极管(light-emitting diode，LED)激光器，可以依次发出多个调制频率的光信号。控制器1011用于通过激光器驱动103控制激光器104发射光信号，同时控制器1011还用于控制像素阵列1013曝光一段时间，获得曝光值。像素阵列1013将曝光值传输给ADC1012，ADC1012进行模数转换，将进行模数转换后的曝光值，即raw数据，发送至测距装置。此处，对于多个调制频率的TOF系统，每次控制器1011控制激光器104发射一种频率的光信号，针对每个调制频率，任一像素对应有四个曝光值。四个曝光值分别对应0度、90度、180度和270度，见式(7)至(10)。
在本申请实施例中，假设发射的光信号为强度调制后的余弦信号，该信号使用式(4)表示：
s(t)＝cos(ωt)   (4)
其中，在式(4)中，s(t)表示发射的光信号，ω表示角速率。
发射的光信号经过被测物体反射回来的光信号会产生一个偏移量(部分原因为由于背景光的照明)，以及一个包含有相位延迟的已调制余弦信号，反射回来的光信号的表达式用式(5)表示：
Figure PCTCN2021083673-appb-000016
其中，在式(5)中，g(t)表示反射回来的光信号，ω表示角速率，a表示衰减系数，
Figure PCTCN2021083673-appb-000017
表示相位延迟值，b表示偏移量。
通过解调反射回来的光信号，间接的求出距离，解调反射回来的光信号的过程使用相关函数法：以发射的光信号作为参考信号，求取经过调制后的发射的光信号与反射回来的光信号之间的相关函数，用式(6)表示：
Figure PCTCN2021083673-appb-000018
其中，在式(6)中，c(τ)表示相关函数，对于发射的光信号选定4个不同相位延迟，即选取4个不同的τ值：τ 0＝0°，τ 1＝90°，τ 2＝180°，τ 3＝270°，分别代入式(6)获得：
Figure PCTCN2021083673-appb-000019
Figure PCTCN2021083673-appb-000020
Figure PCTCN2021083673-appb-000021
Figure PCTCN2021083673-appb-000022
在式(7)至式(10)中表示解调后的光信号的最终表达式，K为偏移量，表示反射回的信号中的包括的背景光。前文中针对每个像素，提到的四个曝光值，分别为C(τ 0)、C(τ 1)、C(τ 2)和C(τ 3)。
本申请中获取深度图的方法可以应用于多种场景中，如下给出三种可能的场景：
场景一：汽车场景
在汽车应用中，TOF系统可以被应用于自动驾驶、防撞自刹车等。例如，应用于辅助驾驶系统中，在TOF技术原理支持下，辅助驾驶系统可精确检测驾驶员身体和头部位置，甚至在其戴眼镜或太阳镜的情况下捕获其眨眼动作，以判断驾驶员是否注意力足够集中、是否正在疲劳驾驶等，从而启动相应的对策。对策可以是振动座椅或者发出警告音等。另外，还可以快速和准确地做出响应，辅助驾驶系统和紧急制动系统可在潜在紧急情况发生之前自动激活。
此外，TOF技术还可以通过手部运动或者身体姿态控制车载娱乐系统或车用空调，甚至在车外实现全新的辅助和安全功能，例如，开门辅助设备，在停车场或者家用地库开门时，防止车门打开后撞上其他物体(如车辆、墙壁、天花板等)。
场景二：人机交互场景
TOF系统提供了一种实时的远方影像，使得可以简单地用来记录人体动作。因此，使得 电子类产品有了全新的交互模式，电子类产品可以是电视、手机、平板等。
例如，可以通过TOF获得每个像素拍摄的物体的深度值，将不同深度的物体区别开来。
场景三：测量与机器视觉场景
在工业机器视觉的应用中，TOF系统安装在机器人上，机器人通过TOF系统对物体进行分类和精准定位安置。另外，还可以应用于人脸识别和地球地貌测绘方面等。
在介绍完应用场景后，下面将结合图5对本申请实施例提供的获取深度图的方法进行说明，该方法可以由测距装置执行。如图5所示，该方法的处理流程如下：
步骤501，对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，其中，每个调制频率的raw数据为TOF系统对被测物体测距时在每个调制频率下多次曝光获得，第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图，TOF系统中像素阵列的分辨率为第二分辨率，第一分辨率低于第二分辨率。
其中，被测物体是任意的物体，如汽车等。多个调制频率的数目大于1，如多个调制频率可以是2个调制频率，也可以是3个调制频率等。深度图中每个像素的像素值可以称为是深度值，对于任一像素的深度值，该深度值表示被测物体上反射光信号至该像素的位置点与TOF系统的距离。第一分辨率低于第二分辨率，第二分辨率为TOF系统中像素阵列的分辨率。第二分辨率相对于第一分辨率为未下采样处理的分辨率，所以也可以称为是原始分辨率，第一分辨率相对于第二分辨率为低分辨率。像素不存在异常解缠系数指在使用该像素的深度值确定解缠系数时，解缠系数不会计算错误。
在本实施例中，TOF系统中控制器通过激光器驱动，控制激光器依次发射每个调制频率的光信号，获得每个调制频率的raw数据，如下以发射调制频率j(调制频率j为多个调制频率中任一个调制频率)的光信号为例进行说明：
TOF系统中控制器通过激光器驱动，控制激光器发射调制频率j的光信号，照射至被测物体上，该光信号是经过强度调制的光信号。被测物体对光信号进行反射，控制器控制像素阵列曝光一段时间，接收被测物体反射回的光信号。在多次曝光(例如，相位延迟值依次为0度、90度、180度、270度)结束后，像素阵列中任一像素的曝光值为C(τ 0) j、C(τ 1) j、C(τ 2) j和C(τ 3) j，分别对应前文中的式(7)至(10)。像素阵列将每个像素的曝光值传输至ADC，ADC对每个像素的曝光值进行模数转换处理后获得DCS(τ 0) j、DCS(τ 1) j、DCS(τ 2) j、DCS(τ 3) j，传输至测距装置。每个像素的多次曝光、量化后得到的数值DCS(τ 0) j、DCS(τ 1) j、DCS(τ 2) j、DCS(τ 3) j组成调制频率j的raw数据。这样，测距装置获取到调制频率j的raw数据。
这样，基于依次发射多个调制频率的光信号，获得多个调制频率的raw数据。
然后测距装置可以对多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图。这样，可以获得像素不存在异常解缠系数的深度图。
步骤502，使用第一分辨率的深度图和第二分辨率的各像素的相位延迟值，计算获得第二分辨率的各像素的解缠系数，其中，第二分辨率的各像素的相位延迟值是使用多个调制频率的raw数据计算获得。
在本实施例中，对于调制频率j下第二分辨率的任一像素k，测距装置通过式(7)至式(10)可以计算出相位延迟值
Figure PCTCN2021083673-appb-000023
使用式(11)表示为：
Figure PCTCN2021083673-appb-000024
其中，在式(11)中DCS(τ 0) kj、DCS(τ 1) kj、DCS(τ 2) kj、DCS(τ 3) kj表示在调制频率j像素k的raw数据。这样，由于TOF系统中像素阵列的分辨率为第二分辨率，且未对各个调制频率对应的raw数据进行下采样，所以可以通过式(11)可以获得在每个调制频率第二分辨率的各像素的相位延迟值。
然后测距装置可以获取预先存储的每个调制频率对应的模糊距离，或者将每个调制频率代入式(1)获得每个调制频率对应的模糊距离。测距装置使用第一分辨率的深度图、第二分辨率的各像素的相位延迟值以及每个调制频率对应的模糊距离，计算获得第二分辨率的各像素的解缠系数。
步骤503，使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算获得被测物体对应的第二分辨率的目标深度图。
在本实施例中，对于调制频率j的第二分辨率的任一像素k，测距装置可以将步骤502确定出的N kj、
Figure PCTCN2021083673-appb-000025
和U j代入式(12)，获得该像素k的距离D kj(即深度值)。基于这种方式确定出调制频率j的第二分辨率的每个像素的深度值，将这些深度值组成在调制频率j被测物体对应的第二分辨率的目标深度图。
Figure PCTCN2021083673-appb-000026
其中，在式(12)中，N kj为在调制频率j该像素k的解缠系数，
Figure PCTCN2021083673-appb-000027
为在调制频率j该像素k的相位延迟值，U j为调制频率j的模糊距离。
基于确定在调制频率j被测物体对应的第二分辨率的目标深度图的方式，确定出在每个调制频率被测物体对应的第二分辨率的目标深度图。
这样，由于下采样后的第一分辨率的深度图中不存在异常解缠系数的像素，第二分辨率的各像素的解缠系数使用第一分辨率的深度图计算获得，所以第二分辨率的各像素的解缠系数比较准确，进而基于该解缠系数计算出的第二分辨率的深度图也比较准确，保证了第二分辨率的深度图的精度。
此处需要说明的是，步骤503中，目标深度图是为了与后文中的初始深度图进行区分，实际其就是表示深度图。
另外，在步骤503之后，也可以将在多个调制频率被测物体对应的第二分辨率的深度图中对应像素的深度值取平均值或按某一设定权重序列加权平均，获得被测物体对应的第二分辨率的一张深度图。
如下针对图5的流程进行补充说明：
在步骤501中有多种方式可以实现，如下给出三种可行的实现方式：
方式一：使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个调制频率的raw数据对应的目标分辨率的灰度图和目标分辨率的深度图；若每个灰度图中不存在灰度值小于所属调制频率对应的第一阈值的像素，则确定目标分辨率的深度图为第一分辨率的深度图；若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则更新第一下采样参数，基于更新后的下采样参数对多个调制频率的raw数据进行 下采样处理，直至获得第一分辨率的深度图。
其中，每个调制频率对应的第一阈值为预设数值，存储在测距装置中。第一下采样参数为第一次进行下采样处理时使用的下采样参数。第一下采样参数为用于降低分辨率的参数，例如，第一下采样参数为2*2，2*2表示4，在处理时将原来的2*2个像素合并为一个像素。另外第一下采样参数也可以称为是第一binning参数，下采样处理也可以称为是binning处理。
在本实施例中，测距装置可以使用第一下采样参数，对每个调制频率的raw数据进行下采样处理，获得每个调制频率的raw数据对应的目标分辨率的灰度图和目标分辨率的深度图。示例性的，按照第一下采样参数，将调制频率j的raw数据中多个像素划分为多个像素组合，每个像素组合中包括的像素数目等于第一下采样参数。例如，第一下采样参数为2*2，那么此处每个像素组合中包括2*2个像素，第一下采样参数为3*3，那么此处每个像素组合中包括3*3个像素。
然后对于任一像素组合，将调制频率j的raw数据中该像素组合中各像素的DCS(τ 0) j求平均，得到该像素组合中的像素合并为一个像素的DCS(τ 0) j0；将调制频率j的raw数据中该像素组合中各像素的DCS(τ 1) j求平均，得到该像素组合中的像素合并为一个像素的DCS(τ 1) j1；将调制频率j的raw数据中该像素组合中各像素的DCS(τ 2) j求平均，得到该像素组合中的像素合并为一个像素的DCS(τ 2) j2；将调制频率j的raw数据中像素组合中各像素的DCS(τ 3) j求平均，得到该像素组合中的像素合并为一个像素的DCS(τ 3) j3。由于第二分辨率的raw数据中像素组合中的像素合并为一个像素，所以相当于是对第二分辨率的raw数据进行下采样处理，获得低于第二分辨率的raw数据，低于第二分辨率的raw数据可以称为是目标分辨率的raw数据。例如，如图6所示，第二分辨率为8*8，第一下采样参数为2*2，目标分辨率为4*4，在图6中每个方框表示一个像素。
然后对于目标分辨率的像素i，将像素i的DCS(τ 0) ij0、DCS(τ 1) ij1、DCS(τ 2) ij2和DCS(τ 3) ij3代入式(13)，获得目标分辨率的像素i的灰度值A ij，这样，在调制频率j下，对于目标分辨率的每个像素均对应一个灰度值，那么即获得在调制频率j目标分辨率的灰度图。
Figure PCTCN2021083673-appb-000028
基于上述处理过程，可以获得在调制频率j目标分辨率的灰度图，对于每个调制频率都按照上述处理，即可获得每个调制频率的raw数据对应的目标分辨率的灰度图。
并且，对于目标分辨率的像素i，将像素i的DCS(τ 0) ij0、DCS(τ 1) ij1、DCS(τ 2) ij2和DCS(τ 3) ij3代入式(14)，获得目标分辨率的像素i的相位延迟值
Figure PCTCN2021083673-appb-000029
这样，在调制频率j下，对于目标分辨率的每个像素均对应一个相位延迟值。
Figure PCTCN2021083673-appb-000030
在每个调制频率固定后，就能得到多个调制频率的解缠区域的线段，对于目标分辨率的每个像素，使用每个调制频率对应的相位延迟值和多个调制频率的解缠区域的线段，获得每个像素的解缠系数。例如，多个调制频率为两个调制频率(调制频率1和调制频率2)，对于 目标分辨率的像素i，使用该像素i的相位延迟值
Figure PCTCN2021083673-appb-000031
和
Figure PCTCN2021083673-appb-000032
(
Figure PCTCN2021083673-appb-000033
表示该像素i在调制频率1的相位延迟值，
Figure PCTCN2021083673-appb-000034
表示该像素i在调制频率2的相位延迟值)，在图3中找到距离
Figure PCTCN2021083673-appb-000035
和
Figure PCTCN2021083673-appb-000036
组成的位置点最近的线段，将该线段的序号对应的解缠系数确定为该像素i对应的解缠系数。
将在调制频率j目标分辨率的像素i的解缠系数N ij、调制频率j对应的模糊距离U j、在调制频率j像素i的相位延迟值
Figure PCTCN2021083673-appb-000037
代入式(15)，获得在调制频率j，目标分辨率的像素i的深度值D ij。
Figure PCTCN2021083673-appb-000038
基于式(15)确定出目标分辨率的每个像素在调制频率i的深度值。
然后将目标分辨率每个像素在多个调制频率下的深度值取平均或加权平均，获得目标分辨率的每个像素的深度值。当然也可以将多个调制频率中任一调制频率下目标分辨率的每个像素的深度值，确定为目标分辨率的每个像素的深度值。这样，就可以获得目标分辨率的深度图。
测距装置获取存储的每个调制频率对应的第一阈值，在每个调制频率的目标分辨率的灰度图中，判断各像素的灰度值是否均大于或等于所属调制频率对应的第一阈值。若均大于或等于所属调制频率对应的第一阈值，则确定目标分辨率的深度图为第一分辨率的深度图。若至少一个目标分辨率的灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则可以在第一下采样参数的基础上，按照预设的规则更新下采样参数。测距装置使用更新后的装箱参数对多个调制频率的raw数据进行下采样处理，获得每个调制频率的第四分辨率的灰度图和第四分辨率的深度图，按照前文中的方式判断每个第四分辨率的灰度图中各像素的灰度值是否均大于或等于所属调制频率对应的第一阈值。若各像素的灰度值均大于或等于第一阈值，则确定第四分辨率的深度图为第一分辨率的深度图。若至少一个第四分辨率的灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则可以在上次更新后的下采样参数的基础上，按照预设的规则更新下采样参数，再对多个调制频率的raw数据进行下采样处理，直至获得每个灰度图中各像素的灰度值均大于或等于所属调制频率对应的第一阈值。
另外，在上述处理中是按照预设的规则，更新下采样参数，该预设的规则可以是在原来下采样参数的基础上，在水平方向和垂直方向均增加预设数值。例如，预设数值为1，第一下采样参数为2*2，第一次更新后的下采样参数为3*3，第二次更新后的下采样参数为4*4。
此处需要说明的是，由于灰度值可以反映出信号接收强度，信号接收强度比较高，信噪比比较高，一般可以准确地进行解缠，确定出准确的解缠系数，所以可以通过灰度图中的灰度值，判断能否准确地进行解缠。
方式二，使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个调制频率的raw数据对应的目标分辨率的灰度图；若每个灰度图中不存在灰度值小于所属调制频率对应的第一阈值的像素，则使用第一下采样参数，对多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图；若至少一个灰度图中存在灰度值小于所属调制频率对应的第一阈值的像素，则更新第一下采样参数，基于更新后的下采样参数对多个调制频率的raw数据进行下采样处理，直至获得第一分辨率的深度图。
方式二的处理过程与方式一的处理过程类似，区别在于：方式一中是在下采样处理时一并确定出灰度图和深度图，方式二中是先确定目标出灰度图，获得目标灰度图使用的下采样 参数为目标下采样参数，在目标灰度图中各像素的灰度值均大于或等于所属调制频率对应的第一阈值后，再基于目标下采样参数，对多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图。
可选的，每个调制频率对应的第一阈值是对应的距离噪声曲线中解缠无误的最低信号接收强度，距离噪声曲线是在TOF系统中获得。
在本实施例中，多个调制频率的TOF系统的模糊距离比较大，在0至模糊距离空间范围内被测物体的反射率可能差别比较大，TOF系统与被测物体的距离不同时被测物体反射回来的光信号强度也不相同，因此TOF系统中某些像素阵列中有的像素可以接收到信噪比比较高的光信号，但是另一些像素会接收到信噪比比较低的光信号。对于TOF系统的TOF芯片，在每个调制频率都有一条固定的距离噪声曲线，x轴表示接收的光信号强度，y轴表示在某个光信号强度下多次测量同一距离的被测物体的深度值标准差。如图7所示，在调制频率为100MHz，对于像素阵列的某个像素来说，接收到的光信号强度从800降低至100后，接收到的光信号强度的单位为数字量化值(digital number，DN)，距离噪声从5升高为50，距离噪声的单位为mm。
在选定多个调制频率后，可以得到类似图3的相位延迟值与解缠系数的关系图，在该关系图中，通过几何计算可以获得两个邻居线段之间的最小相位距离ph ith，对于每个调制频率，可以计算得到每个调制频率对应的相位延迟值，每个调制频率对应的相位延迟值中包含误差，假设调制频率j对应的相位延迟值表达式为
Figure PCTCN2021083673-appb-000039
为调制频率j的真实相位延迟值，
Figure PCTCN2021083673-appb-000040
为调制频率j的相位延迟值的误差。
在相位延迟值误差不是很大时，解缠系数判断就不会出错，所以有
Figure PCTCN2021083673-appb-000041
时，解缠系数判断不会出错，而当
Figure PCTCN2021083673-appb-000042
时，解缠系数可能会判断出错。假设每个调制频率对应的相位延迟值中包含的误差相同，则存在相位阈值
Figure PCTCN2021083673-appb-000043
由相位阈值和正态分布3sigma原则可以得到相位标准差为：
Figure PCTCN2021083673-appb-000044
在上述n为多个调制频率的数目。
然后将相位标准差代入式(1)和距离
Figure PCTCN2021083673-appb-000045
(t为从TOF系统发射的光信号到接收到被测物体反射回的光信号的时长，c为光速，f为调制频率j)，确定出调制频率j的深度值标准差为：
Figure PCTCN2021083673-appb-000046
在式(17)中U j为调制频率j的模糊距离。
然后使用调制频率j的深度值标准差，在调制频率j的距离噪声曲线中，找到深度值标准差为σ(d j)时的信号接收强度，此信号接收强度为在调制频率j解缠无误的最低信号接收强 度A th。将A th除以g得到调制频率j对应的第一阈值，g为上述提到的目标下采样参数，目标下采样参数为获得第一分辨率的深度图时使用的下采样参数。
此处A th除以g的原因为：在计算第一阈值时，计算的是原来未进行下采样处理之前每个像素的灰度值(该灰度值满足的最低信噪比要求)，但是在下采样处理后，是(g*g)个像素合并一个像素，信噪比提升了g倍，新组成的一个像素的最低信号接收强度可以变为原来一个像素时最低信号接收强度的1/g。
方式三：使用多个调制频率的raw数据和多个调制频率的相位延迟图，获得第二分辨率的初始深度图中各像素的解缠系数；使用初始深度图中各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算获得初始深度图中各像素的深度值；对初始深度图进行下采样处理，获得第一分辨率的深度图。
在本实施例中，测距装置可以将调制频率j的raw数据，代入式(11)，确定出在调制频率j下，第二分辨率的每个像素的相位延迟值，这样，第二分辨率的各像素在每个调制频率对应有一个相位延迟值。然后对于任一像素，使用该像素的多个相位延迟值在相位延迟值与解缠系数的对应关系图中，确定距离该多个相位延迟值所在位置点最近的线段，确定该线段的序号对应的解缠系数为该像素在多个频率下对应的解缠系数。例如，多个调制频率为两个调制频率，在图3中，确定出距离两个相位延迟值所在位置点最近的线段，确定该线段的序号对应的解缠系数，将该解缠系数确定为该像素在两个调制频率对应的解缠系数。然后将像素k的N kj、
Figure PCTCN2021083673-appb-000047
和U j代入式(12)，获得在调制频率j下像素k的深度值，N kj为在调制频率j该像素k的解缠系数，
Figure PCTCN2021083673-appb-000048
为在调制频率j该像素k的相位延迟值，U j为调制频率j的模糊距离。按照这种方式确定在调制频率j每个像素的深度值。
将在多个调制频率每个像素的深度值取平均值，将每个像素对应的平均值确定为第二分辨率的每个像素的深度值。将第二分辨率的每个像素的深度值组成第二分辨率的初始深度图。
然后将第二分辨率的初始深度图进行下采样处理，获得第一分辨率的深度图，第一分辨率低于第二分辨率。
可选的，对第二分辨率的初始深度图进行下采样处理的过程为：
使用第二下采样参数，对初始深度图进行下采样处理，获得第三分辨率的深度图；确定第三分辨率的深度图对应的梯度图；若梯度图满足梯度条件，则确定第三分辨率的深度图为第一分辨率的深度图；若梯度图不满足梯度条件，则更新第二下采样参数，基于更新后的下采样参数对初始深度图进行下采样处理，直至获得满足梯度条件的目标梯度图，将获得目标梯度图的深度图确定为第二分辨率的深度图；其中，梯度条件为距离间隔小于第二阈值的像素的梯度绝对值小于或等于第三阈值。
其中，第二下采样参数可以与第一下采样参数相同，或者不相同。第三分辨率低于第二分辨率。在第二下采样参数与第一下采样参数相同的情况下，第三分辨率等于目标分辨率。第二阈值为预设的阈值，可以为3、4等。第三阈值为预设的数值。
在本实施例中，测距装置可以按照第二下采样参数，将第二分辨率的初始深度图中划分为多个像素组合，每个像素组合中的像素数目相同。将每个像素组合中各像素的深度值取平均值，获得每个像素组合中的像素合并为一个像素时的深度值。将合并后的每个像素的深度值组成第三分辨率的深度图。
然后确定第三分辨率的深度图对应的梯度图，测距装置判断该梯度图中距离间隔小于第 二阈值的像素的梯度绝对值与第三阈值的大小关系，若该梯度图中距离间隔小于第二阈值的像素的梯度绝对值小于或等于第三阈值，则确定该梯度图满足梯度条件，可以确定第三分辨率的深度图为第一分辨率的深度图，也就是说第三分辨率等于第一分辨率。若梯度图中距离间隔小于第二阈值的像素的梯度绝对值大于第三阈值，则确定梯度图不满足梯度条件，可以在第二下采样参数的基础上使用预设的规则，更新下采样参数。使用更新后的下采样参数对第二分辨率的初始深度图进行下采样处理，获得更新后的梯度图，判断更新后的梯度图是否满足梯度条件，若更新后的梯度图满足梯度条件，则确定获得更新后的梯度图的深度图为第一分辨率的深度图。若更新后的梯度图不满足梯度条件，则在上一次更新后的下采样参数的基础上，继续更新梯度图，直至获得满足梯度条件的目标梯度图。将获得目标梯度图的深度图确定为第一分辨率的深度图。
例如，第二下采样参数为2*2，梯度图中存在两个相邻像素(如第2行第3、4个像素)的梯度绝对值均大于第三阈值，则将下采样参数从2*2更新为3*3。再例如，第二下采样参数为2*2，梯度图中存在2个间隔1像素(如第2行第3、5个像素)的梯度绝对值均大于第三阈值，则将下采样参数从2*2更新为3*3。
这样，像素的梯度可以看出突变的像素，若像素的梯度存在突变，则梯度的绝对值会比较大，例如，深度图存在5个像素，分别为[0 0 100 0 0]，第三个像素为异常像素，计算完梯度图后，得到[0 50 0-50 0]，梯度图不满足梯度条件。
另外，由于梯度与深度值误差相关，在能正确确定出解缠系数时，像素在每个调制频率的深度值误差通常都小于所属调制频率对应的第四阈值thD，所以第三阈值与深度值误差相关，第三阈值取值可以为thD min f/m，m为不大于一定数值的正整数，该一定数值为经验值，如10等。此处thD min f＝U min f*ph ith/(2π*2)。其中，U min f为多个调制频率中最小的调制频率对应的模糊距离，thD min f为该最小的调制频率对应的深度值误差阈值，ph ith为n个调制频率的相位延迟图中的最小相位距离，该最小相位距离为该相位延迟图中邻居线段之间的最小相位距离，该邻居线段属于相位延迟图中解缠区域。
此处选取最小的调制频率的深度值误差阈值原因为：最小的调制频率的U min f比较大，那么最小的调制频率的深度值误差阈值也比较大，如果最小的调制频率的深度值误差都能小于深度值误差阈值，则梯度图也满足梯度条件，所以将第三阈值取值为thD min f/m。
在本实施例中，计算第三分辨率的深度图的梯度图时，第三分辨率的深度图中任一位置(x,y)的梯度值可以使用如下式子计算：
G(x,y)＝dx(i,j)+dy(i,j)(18)
其中，G(x,y)表示第三分辨率的深度图中像素(x，y)处的梯度值，dx(i,j)＝[I(i+1,j)-I(i-1,j)]/2，dy(i,j)＝[I(i,j+1)-I(i,j-1)]/2，I()表示对应像素的深度值，如I(i+1,j)为像素(i+1,j)处的深度值。对于边缘位置的像素，可以直接将深度值确定为梯度值，也可以使用其它方式进行计算。
此处仅是一种可能的计算梯度图的方式，任何计算梯度图的方式均可以应用本申请实施例，此处不再赘述。
需要说明的是，下采样处理时，更新下采样参数的次数有限定，如果更新次数达到一定数目，还未获得第一分辨率的深度图，则认为获取的raw数据有误，可以重新获取raw数据。
在一种可能的实现方式中，步骤502的处理可以为：
确定第二分辨率的像素k在调制频率j的解缠系数为：
Figure PCTCN2021083673-appb-000049
其中，round[]为四舍五入操作，D L为像素k在第一分辨率的深度图中对应的深度值，N kj为在调制频率j像素k的解缠系数，
Figure PCTCN2021083673-appb-000050
为在调制频率j像素k对应的相位延迟值，U j为调制频率j对应的模糊距离。
在本实施例中，测距装置可以使用式(19)确定第二分辨率中像素k在频率j的解缠系数。
Figure PCTCN2021083673-appb-000051
其中，式(19)可以通过式(2)推导出，在式(19)中，
Figure PCTCN2021083673-appb-000052
表示对
Figure PCTCN2021083673-appb-000053
进行四舍五入操作，D L为像素k在第一分辨率的深度图中对应的深度值，N kj为在调制频率j像素k的解缠系数，
Figure PCTCN2021083673-appb-000054
为在调制频率j像素k对应的相位延迟值，U j为调制频率j对应的模糊距离。
例如，在存在两个调制频率时，像素k在调制频率1的解缠系数为：
Figure PCTCN2021083673-appb-000055
像素k在调制频率2的解缠系数为：
Figure PCTCN2021083673-appb-000056
其中，在式(20)和式(21)中D L为第一分辨率的深度图中像素k的深度值，
Figure PCTCN2021083673-appb-000057
和
Figure PCTCN2021083673-appb-000058
分别为在调制频率1和调制频率2像素k的相位延迟值，U 1和U 2分别为调制频率1和调制频率2分别对应的模糊距离。
此处需要说明的是，由于第一分辨率低于第二分辨率，且第一分辨率的像素由第二分辨率下的多个像素下采样形成，所以在计算解缠系数时，第二分辨率的像素k在第一分辨率的深度图中对应的深度值D L为包括像素k的多个像素形成的像素的深度值。
在一种可能的实现方式中，在TOF系统中某些像素拍摄到了被测物体边沿(既有近景也有远景的场景)，当近景和远景与TOF系统的距离差异比较大时，下采样处理后的这些像素与相邻像素共同计算出的深度值D L，会与下采样前这些像素的真实深度值误差较大，因此，再直接使用式(19)来计算第二分辨率的像素的解缠系数，会使得解缠系数计算出错，所以需要针对物体边沿区域重新计算解缠系数。例如，如图8所示，给出了TOF系统在没有噪声的情况下，使用两个调制频率(80MHz和100MHz)垂直拍摄两个不同距离的白板的场景，左边3列像素拍摄到2.5米远的白板，右侧5列像素拍摄到3.5米远的白板，当装箱参数为2*2时，下采样处理后的第2列(即左图的第3、4列像素，像素A和C在第三列，像素B和D在第四列)包含了两个深度值的像素(第3列2.5米，第4列3.5米)，假设两个调制频率解缠后输出的深度值为D L(如2.49米)，该深度值之后会用式(19)来计算边沿区域的2*2个像素的解缠系数，会使得像素B和D的解缠系数计算出错。因此，可以引入一个边沿错误检测功能，详细说明如下：
相应的，步骤503的处理可以为：
使用第二分辨率的各像素的解缠系数和第二分辨率的各像素的相位延迟值，计算每个调 制频率的第二分辨率的各像素的深度值；使用第一分辨率的深度图和每个调制频率的第二分辨率的各像素的深度值，确定每个调制频率的第二分辨率的各像素的深度值误差；使用每个调制频率的第一分辨率的各像素的深度值误差，计算获得被测物体对应的第二分辨率的目标深度图。
在本实施例中，测距装置可以将调制频率j的第二分辨率的各像素解缠系数、在调制频率j各像素的相位延迟值、调制频率j对应的模糊距离，代入式(12)，获得在调制频率j各像素的深度值。基于这种方式，可以确定出在每个调制频率各像素的深度值。
然后测距装置可以使用第一分辨率的深度图和各调制频率的第二分辨率的各像素的深度值，确定在每个调制频率第二分辨率的各像素的深度值误差。测距装置可以使用在每个调制频率第二分辨率的各像素的深度值误差，确定被测物体在各调制频率下的第二分辨率的目标深度图。
在一种可能的实现方式中，确定各像素的深度值误差的处理可以为：
测距装置可以使用式(22)确定在调制频率j第二分辨率的像素k的深度值误差为：
E kj＝|D L-D kj|   (22)
在式(22)中，E kj表示在调制频率j下，像素k在解缠后的测试距离D kj与D L的误差的绝对值，E kj越大，说明深度值误差越大，需要重新计算像素k的深度值。D L为像素k在第一分辨率的深度图中对应的深度值，
Figure PCTCN2021083673-appb-000059
D kj为在调制频率j第二分辨率的像素k的深度值，
Figure PCTCN2021083673-appb-000060
为在调制频率j像素k对应的相位延迟值，N kj为在调制频率j像素k的解缠系数，U j为调制频率j对应的模糊距离。
此处需要说明的是，第二分辨率的像素k在第一分辨率的深度图中对应的深度值为包括像素k在内的像素在进行下采样时组成的像素的深度值。例如，在图8中，像素k为图8中的像素B，像素B在第一分辨率的深度图中对应的深度值为由像素A、B、C和D组成的像素的深度值。
在一种可能的实现方式中，确定各调制频率的第二分辨率的目标深度图的处理可以为：
对于多个调制频率中调制频率j，在调制频率j的第二分辨率的像素中，确定深度值误差大于调制频率j对应的第四阈值的目标像素，其中，j取值为1至n，n为多个调制频率的数目；使用第一分辨率的深度图中目标像素的相邻像素的深度值，更新调制频率j的第二分辨率的目标像素的解缠系数；使用更新后的目标像素的解缠系数和每个调制频率的目标像素的相位延迟值，更新调制频率j的第二分辨率的目标像素的深度值；将调制频率j的第二分辨率的像素中除目标像素之外的像素的深度值和更新后的目标像素的深度值组合，获得被测物体对应的调制频率j的第二分辨率的目标深度图。
其中，第四阈值为预设的数值，可以存储在测距装置中。
在本实施例中，对于调制频率j，测距装置可以获取调制频率j对应的第四阈值，然后测距装置可以判断调制频率j的第二分辨率的像素中深度值误差是否大于调制频率j对应的第四阈值，在调制频率j的第二分辨率的深度图中，获得深度值误差大于第四阈值的目标像素。然后在第一分辨率的深度图中，确定目标像素的相邻像素的深度值。例如，在图8中，第二分辨率的目标像素为像素B，像素B在第一分辨率的深度图中相邻像素为像素P、M和N， 第二分辨率的目标像素为A，像素A在第一分辨率的深度图中相邻像素也为像素P、M和N。
假设目标像素为像素B，对于目标像素在第一分辨率的深度图中的任一相邻像素P，测距装置将该相邻像素P的深度值、在调制频率j第二分辨率的目标像素的相位延迟值和调制频率j对应的模糊距离代入式(19)，获得使用该相邻像素P的深度值时，在调制频率j第二分辨率的目标像素的解缠系数。
然后测距装置将在调制频率j目标像素的解缠系数、在调制频率j目标像素的相位延迟值和调制频率j对应的模糊距离代入式(12)，获得在调制频率j第二分辨率的目标像素的深度值，该深度值是基于相邻像素P的深度值获得。
然后将相邻像素P的深度值和上述确定的目标像素在调制频率j的深度值，代入式(22)可以获得相邻像素P对应的目标像素的深度值误差。
基于上述方式，使用目标像素在第一分辨率的深度图的每个相邻像素，确定出该每个相邻像素对应的目标像素的深度值误差。
测距装置将深度值误差最小，且小于调制频率j对应的第四阈值时的深度值，确定为目标像素在调制频率j的深度值。或者，若确定的目标像素的深度值误差均小于第四阈值，则将任一深度值误差对应的目标像素在第一频率的深度值，确定为目标像素在调制频率j的深度值。例如，在图8中，目标像素为像素B，像素B在第一分辨率的深度图中相邻像素有像素P、M、N，由于像素B与像素M距离TOF系统的距离更接近，所以基于相邻像素M计算的深度值误差最小。另外，每个相邻像素对应目标像素的深度值误差均大于第四阈值，则说明本次获取的raw数据有误，重新获取raw数据，计算深度图。
然后测距装置将调制频率j的第二分辨率的像素中除目标像素之外的像素的深度值与目标像素更新后的深度值组合，获得在调制频率j被测物体的第二分辨率的目标深度图。每个调制频率被测物体的第二分辨率的目标深度图均可以按照上述方式确定。
这样，由于在获得深度值出错的区域后，可以使用相邻像素的正确深度值来计算深度值出错的区域中每个像素的深度值，所以可以保证所有像素的深度值均可以计算正确，获得原始分辨率的正确深度图。
可选的，调制频率j对应的第四阈值为：thD j＝U j*ph ith/(2π*2)；其中，U j为调制频率j对应的模糊距离，ph ith为n个调制频率的相位延迟图中邻居线段之间的最小相位距离，该邻居线段属于相位延迟图中的解缠区域，j取值为1至n。第四阈值表示出现解缠错误时的最小深度值误差。对于任一调制频率下，如果某个像素的深度值误差大于该调制频率对应的第四阈值thD，则会出现解缠错误，确定出错误的解缠系数。
此处需要说明的是，如果相位延迟值的误差超过二分之一最小相位距离，就有可能会导致确定为邻居线段的解缠系数，确定出错误的解缠次数，而在相位延迟值的误差小于最小相位距离的一半时，确定为邻居线段的解缠系数的概率比较小，所以可以将最小相位距离的一半对应的距离，作为深度值误差的阈值。
另外，为了更好的理解本申请实施例，还提供了针对前文中方式一的流程：
如图9所示，步骤901，开始。
步骤902，从TOF系统，获取多个调制频率的raw数据；
步骤903，将第一下采样参数作为当前的下采样参数，对多个调制频率的raw数据进行 下采样处理，获得每个调制频率的目标分辨率的raw数据，并且使用多个调制频率的raw数据，确定每个调制频率的第二分辨率的各像素的相位延迟值；
步骤904，基于每个调制频率的目标分辨率的raw数据，确定目标分辨率的深度图和每个调制频率的目标分辨率的灰度图；
步骤905，判断每个目标分辨率的灰度图中各像素的灰度值是否均大于或等于所属调制频率的第一阈值；
步骤906，若步骤905的判断结果为是，将目标分辨率的深度图确定为第一分辨率的深度图，执行步骤907。若步骤905的判断结果为否，返回更新下采样参数，将更新后的下采样参数作为当前的下采样参数，执行对多个调制频率的raw数据进行下采样处理的过程，直至获得第一分辨率的深度图；
步骤907、使用第一分辨率的深度图，计算第二分辨率的各像素在每个调制频率的解缠系数；
步骤908，使用第二分辨率的各像素的解缠系数，第一分辨率的深度图和第二分辨率的各像素的相位延迟值，计算第二分辨率的各像素在每个调制频率的深度值误差；
步骤909，判断每个调制频率的第二分辨率的各像素中是否存在深度值误差大于所属调制频率对应的第四阈值的像素；
步骤910，若步骤909的判断结果为是，则输出在每个调制频率被测物体对应的第二分辨率的深度图；
步骤911，若步骤909的判断结果为否，则基于目标像素在第一分辨率的深度图中的相邻像素重新计算目标像素的解缠系数，返回执行步骤908，直至每个调制频率的第二分辨率的各像素的深度值误差均小于或等于所属调制频率的第四阈值，输出在每个调制频率被测物体对应的第二分辨率的深度图，目标像素为深度值误差存在大于该调制频率对应的第四阈值的像素。
图9中各步骤的详细描述可参见前文中的描述，此处不再赘述。
这样，通过对多个调制频率的raw数据在计算解缠系数之前增加下采样的处理，使得下采样后获得的像素不存在异常解缠系数的第一分辨率的深度图，所以基于第一分辨率的深度图计算的第二分辨率中各像素的解缠系数准确，进而基于该解缠系数计算出的第二分辨率的深度图也比较准确，保证了第二分辨率的深度图的精度。
另外，为了更好的理解本申请实施例，还提供了针对前文中方式三的流程：
如图10所示，步骤1001，开始。
步骤1002，从TOF系统，获取多个调制频率的raw数据；
步骤1003，计算第二分辨率的各像素在每个调制频率的解缠系数、各像素在每个调制频率的相位延迟值，使用第二分辨率的各像素的解缠系数、第二分辨率的各像素的相位延迟值，计算第二分辨率的深度图；
步骤1004，将第二下采样参数作为当前的下采样参数，对第二分辨率的深度图进行下采样，获得第三分辨率的深度图，确定第三分辨率的深度图的梯度图；
步骤1005，判断第三分辨率的深度图的梯度图是否满足梯度条件；
步骤1006，在第三分辨率的深度图的梯度图满足梯度条件的情况下，确定第三分辨率的深度图为第一分辨率的深度图，执行步骤1007；在第三分辨率的深度图的梯度图不满足梯度 条件的情况下，返回更新下采样参数，将更新后的下采样参数作为当前的下采样参数，执行步骤1004；
步骤1007，使用第三分辨的深度图计算第二分辨率的各像素在每个调制频率的解缠系数；
步骤1008，基于第二分辨率的各像素在每个调制频率的解缠系数，第一分辨率的深度图和第二分辨率的各像素在每个调制频率的相位延迟值，确定第二分辨率的各像素在每个调制频率的深度值误差；
步骤1009，判断第二分辨率的各像素中是否存在深度值误差大于第四阈值的像素；
步骤1010，若步骤1009的判断结果为是，则输出在每个调制频率被测物体对应的第二分辨率的深度图；
步骤1011，若步骤1009的判断结果为否，则基于目标像素在第一分辨率的深度图中的相邻像素重新计算目标像素的解缠系数，返回执行步骤908，直至每个调制频率的第二分辨率的各像素的深度值误差均小于或等于所属调制频率的第四阈值，输出在每个调制频率被测物体对应的第二分辨率的深度图，目标像素为深度值误差存在大于该调制频率的第四阈值的像素。
图10中各步骤的详细描述可参见前文中的描述，此处不再赘述。
这样，对存在异常解缠像素的深度图进行下采样，使得降采样后的第一分辨率的深度图无异常解缠系数的像素，所以基于第一分辨率的深度图计算的第二分辨率中各像素的解缠系数准确，进而基于该解缠系数计算出的第二分辨率的深度图也比较准确，保证了第二分辨率的深度图的精度。
通过本申请实施例，针对多个调制频率的TOF系统中相位延迟值误差过大而导致确定出的解缠系数错误，进而导致距离精度降低的问题，确定像素无异常解缠系数的低分辨率的深度图，通过该低分辨率的深度图计算原始分辨率的每个像素的解缠系数，可以确定出准确的解缠系数，进而可以确定出准确的深度图。
另外，对于深度值出错的像素，使用该像素的相邻像素的正确深度值确定深度值，可以使得所有像素的深度值均是正确的深度值。
在本申请实施例中，涉及的所有计算过程，可以是使用浮点数计算器获得，也可以是查表获得。
图11是本申请实施例提供的获取深度图的装置的结构图。该装置可以通过软件、硬件或者两者的结合实现成为装置中的部分或者全部。本申请实施例提供的装置可以实现本申请实施例图5所述的流程，该装置包括：下采样模块1110、解缠系数确定模块1120和深度图确定模块1130，其中：
下采样模块1110，用于对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，其中，每个所述调制频率的raw数据为TOF系统对所述被测物体测距时在每个所述调制频率下多次曝光获得，所述第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图，所述TOF系统中像素阵列的分辨率为第二分辨率，所述第一分辨率低于所述第二分辨率，具体可以用于实现步骤501的下采样功能以及执行步骤501包含的隐含步骤；
解缠系数确定模块1120，用于使用所述第一分辨率的深度图和所述第二分辨率的各像素 的相位延迟值，计算获得所述第二分辨率的各像素的解缠系数，其中，所述第二分辨率的各像素的相位延迟值是使用所述多个调制频率的raw数据计算获得，具体可以用于实现步骤502的解缠系数确定功能以及执行步骤502包含的隐含步骤；
所述深度图确定模块1130，还用于使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述被测物体对应的所述第二分辨率的目标深度图，具体可以用于实现步骤503的深度图确定功能以及执行步骤503包含的隐含步骤。
在一种可能的实现方式中，所述下采样模块1110，用于：
使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个所述调制频率的raw数据对应的目标分辨率的灰度图和所述目标分辨率的深度图；
若每个所述灰度图中不存在灰度值小于所属所述调制频率对应的第一阈值的像素，则确定所述目标分辨率的深度图为所述第一分辨率的深度图；
若至少一个所述灰度图中存在灰度值小于所属所述调制频率对应的第一阈值的像素，则更新所述第一下采样参数，基于更新后的下采样参数对所述多个调制频率的raw数据进行下采样处理，直至获得所述第一分辨率的深度图。
在一种可能的实现方式中，所述下采样模块1110，用于：
使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个所述调制频率的raw数据对应的目标分辨率的灰度图；
若每个所述灰度图中不存在灰度值小于所属所述调制频率对应的第一阈值的像素，则使用所述第一下采样参数，对所述多个调制频率的raw数据进行下采样处理，获得所述第一分辨率的深度图；
若至少一个所述灰度图中存在灰度值小于所属所述调制频率对应的第一阈值的像素，则更新所述第一下采样参数，基于更新后的下采样参数对所述多个调制频率的raw数据进行下采样处理，直至获得所述第一分辨率的深度图。
在一种可能的实现方式中，每个所述调制频率对应的第一阈值是对应的距离噪声曲线中解缠无误的最低信号接收强度，所述距离噪声曲线是在所述TOF系统中获得。
在一种可能的实现方式中，所述下采样模块1110，用于：
使用所述多个调制频率的raw数据和所述多个调制频率的相位延迟图，获得所述第二分辨率的初始深度图中各像素的解缠系数；
使用所述初始深度图中各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述初始深度图中各像素的深度值；
对所述初始深度图进行下采样处理，获得所述第一分辨率的深度图。
在一种可能的实现方式中，所述下采样模块1110，用于：
使用第二下采样参数，对所述初始深度图进行下采样处理，获得第三分辨率的深度图；
确定所述第三分辨率的深度图对应的梯度图；
若所述梯度图满足梯度条件，则确定所述第三分辨率的深度图为所述第一分辨率的深度图；
若所述梯度图不满足所述梯度条件，则更新所述第二下采样参数，基于更新后的下采样参数对所述初始深度图进行下采样处理，直至获得满足所述梯度条件的目标梯度图，将获得所述目标梯度图的深度图确定为所述第二分辨率的深度图；
其中，所述梯度条件为距离间隔小于第二阈值的像素的梯度绝对值小于或等于第三阈值。
在一种可能的实现方式中，所述解缠系数确定模块1120，用于：
确定调制频率j的所述第二分辨率的像素k的解缠系数为：
Figure PCTCN2021083673-appb-000061
其中，所述调制频率j属于所述多个调制频率，round[]为四舍五入操作，D L为所述像素k在所述第一分辨率的深度图中对应的深度值，N kj为在所述调制频率j所述像素k的解缠系数，
Figure PCTCN2021083673-appb-000062
为在所述调制频率j所述像素k的相位延迟值，U j为所述调制频率j对应的模糊距离。
在一种可能的实现方式中，所述深度图确定模块1130，用于：
使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算每个所述调制频率的所述第二分辨率的各像素的深度值；
使用所述第一分辨率的深度图和每个所述调制频率的所述第二分辨率的各像素的深度值，确定每个所述调制频率的所述第二分辨率的各像素的深度值误差；
使用每个所述调制频率的所述第一分辨率的各像素的深度值误差，计算获得所述被测物体对应的所述第二分辨率的目标深度图。
在一种可能的实现方式中，所述深度图确定模块1130，用于：确定调制频率j的所述第二分辨率的像素k的深度值误差为：E kj＝|D L-D kj|；
其中，D L为所述像素k在所述第一分辨率的深度图中对应的深度值，
Figure PCTCN2021083673-appb-000063
D kj为所述调制频率j的所述第二分辨率的所述像素k的深度值，
Figure PCTCN2021083673-appb-000064
为在所述调制频率j所述像素k的相位延迟值，N kj为在所述调制频率j所述像素k的解缠系数，U j为所述调制频率j对应的模糊距离，j取值为1至n，n为所述多个调制频率的数目。
在一种可能的实现方式中，所述深度图确定模块1130，用于：
对于所述多个调制频率中调制频率j，在所述调制频率j的所述第二分辨率的像素中，确定深度值误差大于所述调制频率j对应的第四阈值的目标像素，其中，j取值为1至n，n为所述多个调制频率的数目；
使用所述第一分辨率的深度图中所述目标像素的相邻像素的深度值，更新所述调制频率j的所述第二分辨率的所述目标像素的解缠系数；
使用更新后的所述目标像素的解缠系数和每个所述调制频率的所述目标像素的相位延迟值，更新所述调制频率j的所述第二分辨率的所述目标像素的深度值；
将所述调制频率j的所述第二分辨率的像素中除所述目标像素之外的像素的深度值和更新后的所述目标像素的深度值组合，获得所述被测物体对应的所述调制频率j的所述第二分辨率的目标深度图。
在一种可能的实现方式中，所述调制频率j对应的第四阈值为：thD j＝U j*ph ith/(2π*2)；U j为所述调制频率j对应的模糊距离，ph ith为所述多个调制频率的相位延迟图中解缠区域的邻居线段之间的最小相位距离，j取值为1至n，n为所述多个调制频率的数目。
本申请实施例中对模块的划分是示意性的，仅仅为一种逻辑功能划分，实际实现时也可 以有另外的划分方式，另外，在本申请各个实施例中的各功能模块可以集成在一个处理器中，也可以是单独物理存在，也可以两个或两个以上模块集成为一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。
该集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台终端设备(可以是个人计算机，手机，或者网络设备等)或处理器(processor)执行本申请各个实施例该方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
本申请实施例中还提供了一种获取深度图的计算设备。图12示例性的提供了计算设备1200的一种可能的架构图。
计算设备1200包括存储器1201、处理器1202、通信接口1203以及总线1204。其中，存储器1201、处理器1202、通信接口1203通过总线1204实现彼此之间的通信连接。
存储器1201可以是ROM，静态存储设备，动态存储设备或者RAM。存储器1201可以存储程序，当存储器1201中存储的程序被处理器1202执行时，处理器1202和通信接口1203用于执行获取深度图的方法。存储器1201还可以存储raw数据。
处理器1202可以采用通用的中央处理器(central processing unit，CPU)，微处理器，应用专用集成电路(application specific integrated circuit，ASIC)，图形处理器(graphics processing unit，GPU)或者一个或多个集成电路。
处理器1202还可以是一种集成电路芯片，具有信号的处理能力。在实现过程中，本申请的获取深度图的装置的部分或全部功能可以通过处理器1202中的硬件的集成逻辑电路或者软件形式的指令完成。上述的处理器1202还可以是通用处理器、数字信号处理器(digital signal processing，DSP)、专用集成电路、现成可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。可以实现或者执行本申请上述实施例中的公开的各方法、步骤及逻辑框图。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。结合本申请实施例所公开的方法的步骤可以直接体现为硬件译码处理器执行完成，或者用译码处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器1201，处理器1202读取存储器1201中的信息，结合其硬件完成本申请实施例的获取深度图的装置的部分功能。
通信接口1203使用例如但不限于收发器一类的收发模块，来实现计算设备1200与其他设备或通信网络之间的通信。例如，可以通过通信接口1203获取数据集。
总线1204可包括在计算设备1200各个部件(例如，存储器1201、处理器1202、通信接口1203)之间传送信息的通路。
上述各个附图对应的流程的描述各有侧重，某个流程中没有详述的部分，可以参见其他流程的相关描述。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现，当 使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令，在服务器或终端上加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴光缆、光纤、数字用户线)或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是服务器或终端能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(如软盘、硬盘和磁带等)，也可以是光介质(如数字视盘(digital videodisk，DVD)等)，或者半导体介质(如固态硬盘等)。
以上所述仅为本申请的示例性实施例，并不用以限制本申请，凡在本申请的构思和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (24)

  1. 一种获取深度图的方法，其特征在于，所述方法包括：
    对被测物体的多个调制频率的裸raw数据进行下采样处理，获得第一分辨率的深度图，其中，每个所述调制频率的raw数据为飞行时间TOF系统对所述被测物体测距时在每个所述调制频率下多次曝光获得，所述第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图，所述TOF系统中像素阵列的分辨率为第二分辨率，所述第一分辨率低于所述第二分辨率；
    使用所述第一分辨率的深度图和所述第二分辨率的各像素的相位延迟值，计算获得所述第二分辨率的各像素的解缠系数，其中，所述第二分辨率的各像素的相位延迟值是使用所述多个调制频率的raw数据计算获得；
    使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述被测物体对应的所述第二分辨率的目标深度图。
  2. 根据权利要求1所述的方法，其特征在于，所述对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，包括：
    使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个所述调制频率的raw数据对应的目标分辨率的灰度图和所述目标分辨率的深度图；
    若每个所述灰度图中不存在灰度值小于所属所述调制频率对应的第一阈值的像素，则确定所述目标分辨率的深度图为所述第一分辨率的深度图；
    若至少一个所述灰度图中存在灰度值小于所属所述调制频率对应的第一阈值的像素，则更新所述第一下采样参数，基于更新后的下采样参数对所述多个调制频率的raw数据进行下采样处理，直至获得所述第一分辨率的深度图。
  3. 根据权利要求1所述的方法，其特征在于，所述对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，包括：
    使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个所述调制频率的raw数据对应的目标分辨率的灰度图；
    若每个所述灰度图中不存在灰度值小于所属所述调制频率对应的第一阈值的像素，则使用所述第一下采样参数，对所述多个调制频率的raw数据进行下采样处理，获得所述第一分辨率的深度图；
    若至少一个所述灰度图中存在灰度值小于所属所述调制频率对应的第一阈值的像素，则更新所述第一下采样参数，基于更新后的下采样参数对所述多个调制频率的raw数据进行下采样处理，直至获得所述第一分辨率的深度图。
  4. 根据权利要求2或3所述的方法，其特征在于，每个所述调制频率对应的第一阈值是对应的距离噪声曲线中解缠无误的最低信号接收强度，所述距离噪声曲线是在所述TOF系统中获得。
  5. 根据权利要求1所述的方法，其特征在于，所述对被测物体的多个调制频率的raw数据进行下采样处理，获得第一分辨率的深度图，包括：
    使用所述多个调制频率的raw数据和所述多个调制频率的相位延迟图，获得所述第二分辨率的初始深度图中各像素的解缠系数；
    使用所述初始深度图中各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述初始深度图中各像素的深度值；
    对所述初始深度图进行下采样处理，获得所述第一分辨率的深度图。
  6. 根据权利要求5所述的方法，其特征在于，所述对所述初始深度图进行下采样处理，获得所述第一分辨率的深度图，包括：
    使用第二下采样参数，对所述初始深度图进行下采样处理，获得第三分辨率的深度图；
    确定所述第三分辨率的深度图对应的梯度图；
    若所述梯度图满足梯度条件，则确定所述第三分辨率的深度图为所述第一分辨率的深度图；
    若所述梯度图不满足所述梯度条件，则更新所述第二下采样参数，基于更新后的下采样参数对所述初始深度图进行下采样处理，直至获得满足所述梯度条件的目标梯度图，将获得所述目标梯度图的深度图确定为所述第二分辨率的深度图；
    其中，所述梯度条件为距离间隔小于第二阈值的像素的梯度绝对值小于或等于第三阈值。
  7. 根据权利要求1至6任一项所述的方法，其特征在于，所述使用所述第一分辨率的深度图和第二分辨率的各像素的相位延迟值，计算获得所述第二分辨率的各像素的解缠系数，包括：
    确定调制频率j的所述第二分辨率的像素k的解缠系数为：
    Figure PCTCN2021083673-appb-100001
    其中，所述调制频率j属于所述多个调制频率，round[]为四舍五入操作，D L为所述像素k在所述第一分辨率的深度图中对应的深度值，N kj为在所述调制频率j所述像素k的解缠系数，
    Figure PCTCN2021083673-appb-100002
    为在所述调制频率j所述像素k的相位延迟值，U j为所述调制频率j对应的模糊距离。
  8. 根据权利要求1至7任一项所述的方法，其特征在于，所述使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述被测物体对应的所述第二分辨率的目标深度图，包括：
    使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算每个所述调制频率的所述第二分辨率的各像素的深度值；
    使用所述第一分辨率的深度图和每个所述调制频率的所述第二分辨率的各像素的深度值，确定每个所述调制频率的所述第二分辨率的各像素的深度值误差；
    使用每个所述调制频率的所述第一分辨率的各像素的深度值误差，计算获得所述被测物体对应的所述第二分辨率的目标深度图。
  9. 根据权利要求8所述的方法，其特征在于，所述使用所述第一分辨率的深度图和每个所述调制频率的所述第二分辨率的各像素的深度值，确定每个所述调制频率的所述第二分辨率的各像素的深度值误差，包括：
    确定调制频率j的所述第二分辨率的像素k的深度值误差为：E kj＝|D L-D kj|；
    其中，D L为所述像素k在所述第一分辨率的深度图中对应的深度值，
    Figure PCTCN2021083673-appb-100003
    D kj为所述调制频率j的所述第二分辨率的所述像素k的深度值，
    Figure PCTCN2021083673-appb-100004
    为在所述调制频率j所述像素k的相位延迟值，N kj为在所述调制频率j所述像素k的解缠系数，U j为所述调制频率j对应的模糊距离，j取值为1至n，n为所述多个调制频率的数目。
  10. 根据权利要求8或9所述的方法，其特征在于，所述使用每个所述调制频率的所述第一分辨率的各像素的深度值误差，计算获得所述被测物体对应的所述第二分辨率的目标深度图，包括：
    对于所述多个调制频率中调制频率j，在所述调制频率j的所述第二分辨率的像素中，确定深度值误差大于所述调制频率j对应的第四阈值的目标像素，其中，j取值为1至n，n为所述多个调制频率的数目；
    使用所述第一分辨率的深度图中所述目标像素的相邻像素的深度值，更新所述调制频率j的所述第二分辨率的所述目标像素的解缠系数；
    使用更新后的所述目标像素的解缠系数和每个所述调制频率的所述目标像素的相位延迟值，更新所述调制频率j的所述第二分辨率的所述目标像素的深度值；
    将所述调制频率j的所述第二分辨率的像素中除所述目标像素之外的像素的深度值和更新后的所述目标像素的深度值组合，获得所述被测物体对应的所述调制频率j的所述第二分辨率的目标深度图。
  11. 根据权利要求10所述的方法，其特征在于，所述调制频率j对应的第四阈值为：thD j＝U j*ph ith/(2π*2)；U j为所述调制频率j对应的模糊距离，ph ith为所述多个调制频率的相位延迟图中解缠区域的邻居线段之间的最小相位距离，j取值为1至n，n为所述多个调制频率的数目。
  12. 一种获取深度图的装置，其特征在于，所述装置包括：
    下采样模块，用于对被测物体的多个调制频率的裸raw数据进行下采样处理，获得第一分辨率的深度图，其中，每个所述调制频率的raw数据为飞行时间TOF系统对所述被测物体测距时在每个所述调制频率下多次曝光获得，所述第一分辨率的深度图为被判定为像素不存在异常解缠系数的深度图，所述TOF系统中像素阵列的分辨率为第二分辨率，所述第一分辨率低于所述第二分辨率；
    解缠系数确定模块，用于使用所述第一分辨率的深度图和所述第二分辨率的各像素的相 位延迟值，计算获得所述第二分辨率的各像素的解缠系数，其中，所述第二分辨率的各像素的相位延迟值是使用所述多个调制频率的raw数据计算获得；
    深度图确定模块，用于使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述被测物体对应的所述第二分辨率的目标深度图。
  13. 根据权利要求12所述的装置，其特征在于，所述下采样模块，用于：
    使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个所述调制频率的raw数据对应的目标分辨率的灰度图和所述目标分辨率的深度图；
    若每个所述灰度图中不存在灰度值小于所属所述调制频率对应的第一阈值的像素，则确定所述目标分辨率的深度图为所述第一分辨率的深度图；
    若至少一个所述灰度图中存在灰度值小于所属所述调制频率对应的第一阈值的像素，则更新所述第一下采样参数，基于更新后的下采样参数对所述多个调制频率的raw数据进行下采样处理，直至获得所述第一分辨率的深度图。
  14. 根据权利要求12所述的装置，其特征在于，所述下采样模块，用于：
    使用第一下采样参数，对被测物体的多个调制频率的raw数据进行下采样处理，获得每个所述调制频率的raw数据对应的目标分辨率的灰度图；
    若每个所述灰度图中不存在灰度值小于所属所述调制频率对应的第一阈值的像素，则使用所述第一下采样参数，对所述多个调制频率的raw数据进行下采样处理，获得所述第一分辨率的深度图；
    若至少一个所述灰度图中存在灰度值小于所属所述调制频率对应的第一阈值的像素，则更新所述第一下采样参数，基于更新后的下采样参数对所述多个调制频率的raw数据进行下采样处理，直至获得所述第一分辨率的深度图。
  15. 根据权利要求13或14所述的装置，其特征在于，每个所述调制频率对应的第一阈值是对应的距离噪声曲线中解缠无误的最低信号接收强度，所述距离噪声曲线是在所述TOF系统中获得。
  16. 根据权利要求12所述的装置，其特征在于，所述下采样模块，用于：
    使用所述多个调制频率的raw数据和所述多个调制频率的相位延迟图，获得所述第二分辨率的初始深度图中各像素的解缠系数；
    使用所述初始深度图中各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算获得所述初始深度图中各像素的深度值；
    对所述初始深度图进行下采样处理，获得所述第一分辨率的深度图。
  17. 根据权利要求16所述的装置，其特征在于，所述下采样模块，用于：
    使用第二下采样参数，对所述初始深度图进行下采样处理，获得第三分辨率的深度图；
    确定所述第三分辨率的深度图对应的梯度图；
    若所述梯度图满足梯度条件，则确定所述第三分辨率的深度图为所述第一分辨率的深度 图；
    若所述梯度图不满足所述梯度条件，则更新所述第二下采样参数，基于更新后的下采样参数对所述初始深度图进行下采样处理，直至获得满足所述梯度条件的目标梯度图，将获得所述目标梯度图的深度图确定为所述第二分辨率的深度图；
    其中，所述梯度条件为距离间隔小于第二阈值的像素的梯度绝对值小于或等于第三阈值。
  18. 根据权利要求12至17任一项所述的装置，其特征在于，所述解缠系数确定模块，用于：
    确定调制频率j的所述第二分辨率的像素k的解缠系数为：
    Figure PCTCN2021083673-appb-100005
    其中，所述调制频率j属于所述多个调制频率，round[]为四舍五入操作，D L为所述像素k在所述第一分辨率的深度图中对应的深度值，N kj为在所述调制频率j所述像素k的解缠系数，
    Figure PCTCN2021083673-appb-100006
    为在所述调制频率j所述像素k的相位延迟值，U j为所述调制频率j对应的模糊距离。
  19. 根据权利要求12至18任一项所述的装置，其特征在于，所述深度图确定模块，用于：
    使用所述第二分辨率的各像素的解缠系数和所述第二分辨率的各像素的相位延迟值，计算每个所述调制频率的所述第二分辨率的各像素的深度值；
    使用所述第一分辨率的深度图和每个所述调制频率的所述第二分辨率的各像素的深度值，确定每个所述调制频率的所述第二分辨率的各像素的深度值误差；
    使用每个所述调制频率的所述第一分辨率的各像素的深度值误差，计算获得所述被测物体对应的所述第二分辨率的目标深度图。
  20. 根据权利要求19所述的装置，其特征在于，所述深度图确定模块，用于：
    确定调制频率j的所述第二分辨率的像素k的深度值误差为：E kj＝|D L-D kj|；
    其中，D L为所述像素k在所述第一分辨率的深度图中对应的深度值，
    Figure PCTCN2021083673-appb-100007
    D kj为所述调制频率j的所述第二分辨率的所述像素k的深度值，
    Figure PCTCN2021083673-appb-100008
    为在所述调制频率j所述像素k的相位延迟值，N kj为在所述调制频率j所述像素k的解缠系数，U j为所述调制频率j对应的模糊距离，j取值为1至n，n为所述多个调制频率的数目。
  21. 根据权利要求19或20所述的装置，其特征在于，所述深度图确定模块，用于：
    对于所述多个调制频率中调制频率j，在所述调制频率j的所述第二分辨率的像素中，确定深度值误差大于所述调制频率j对应的第四阈值的目标像素，其中，j取值为1至n，n为所述多个调制频率的数目；
    使用所述第一分辨率的深度图中所述目标像素的相邻像素的深度值，更新所述调制频率j的所述第二分辨率的所述目标像素的解缠系数；
    使用更新后的所述目标像素的解缠系数和每个所述调制频率的所述目标像素的相位延迟值，更新所述调制频率j的所述第二分辨率的所述目标像素的深度值；
    将所述调制频率j的所述第二分辨率的像素中除所述目标像素之外的像素的深度值和更新后的所述目标像素的深度值组合，获得所述被测物体对应的所述调制频率j的所述第二分辨率的目标深度图。
  22. 根据权利要求21所述的装置，其特征在于，所述调制频率j对应的第四阈值为：thD j＝U j*ph ith/(2π*2)；U j为所述调制频率j对应的模糊距离，ph ith为所述多个调制频率的相位延迟图中解缠区域的邻居线段之间的最小相位距离，j取值为1至n，n为所述多个调制频率的数目。
  23. 一种获取深度图的计算设备，其特征在于，所述计算设备包括处理器和存储器，其中：
    所述存储器中存储有计算机指令；
    所述处理器执行所述计算机指令，以使所述计算设备执行所述权利要求1至11中任一项权利要求所述的方法。
  24. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有计算机指令，当所述计算机可读存储介质中的计算机指令被计算设备执行时，使得所述计算设备执行所述权利要求1至11中任一项权利要求所述的方法。
PCT/CN2021/083673 2021-03-29 2021-03-29 获取深度图的方法、装置、计算设备和可读存储介质 WO2022204895A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202180096494.6A CN117083533A (zh) 2021-03-29 2021-03-29 获取深度图的方法、装置、计算设备和可读存储介质
PCT/CN2021/083673 WO2022204895A1 (zh) 2021-03-29 2021-03-29 获取深度图的方法、装置、计算设备和可读存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2021/083673 WO2022204895A1 (zh) 2021-03-29 2021-03-29 获取深度图的方法、装置、计算设备和可读存储介质

Publications (1)

Publication Number Publication Date
WO2022204895A1 true WO2022204895A1 (zh) 2022-10-06

Family

ID=83457421

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/083673 WO2022204895A1 (zh) 2021-03-29 2021-03-29 获取深度图的方法、装置、计算设备和可读存储介质

Country Status (2)

Country Link
CN (1) CN117083533A (zh)
WO (1) WO2022204895A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230010725A1 (en) * 2021-07-12 2023-01-12 SK Hynix Inc. Image acquisition method for time of flight camera

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101706577A (zh) * 2009-12-01 2010-05-12 中南大学 InSAR监测高速公路路面沉降方法
US20180011195A1 (en) * 2016-07-07 2018-01-11 Microsoft Technology Licensing, Llc Multi-frequency unwrapping
CN108519588A (zh) * 2018-04-12 2018-09-11 视缘(上海)智能科技有限公司 一种多频相位解缠方法及装置
US20190302244A1 (en) * 2018-03-30 2019-10-03 Microsoft Technology Licensing, Llc Analytical-adaptive multifrequency error minimization unwrapping
CN111208512A (zh) * 2020-01-15 2020-05-29 电子科技大学 一种基于视频合成孔径雷达的干涉测量方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101706577A (zh) * 2009-12-01 2010-05-12 中南大学 InSAR监测高速公路路面沉降方法
US20180011195A1 (en) * 2016-07-07 2018-01-11 Microsoft Technology Licensing, Llc Multi-frequency unwrapping
US20190302244A1 (en) * 2018-03-30 2019-10-03 Microsoft Technology Licensing, Llc Analytical-adaptive multifrequency error minimization unwrapping
CN108519588A (zh) * 2018-04-12 2018-09-11 视缘(上海)智能科技有限公司 一种多频相位解缠方法及装置
CN111208512A (zh) * 2020-01-15 2020-05-29 电子科技大学 一种基于视频合成孔径雷达的干涉测量方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230010725A1 (en) * 2021-07-12 2023-01-12 SK Hynix Inc. Image acquisition method for time of flight camera
US11915393B2 (en) * 2021-07-12 2024-02-27 SK Hynix Inc. Image acquisition method for time of flight camera

Also Published As

Publication number Publication date
CN117083533A (zh) 2023-11-17

Similar Documents

Publication Publication Date Title
US10341633B2 (en) Systems and methods for correcting erroneous depth information
US10671862B2 (en) Method and system for detecting obstacles by autonomous vehicles in real-time
JP6881307B2 (ja) 情報処理装置、情報処理方法、及び、プログラム
US20170344888A1 (en) Information processing apparatus and information processing method
US11222438B2 (en) Information processing apparatus, vehicle, and information processing method for presence probability of object
EP2642447A2 (en) Device for the calibration of a stereo camera
US11204610B2 (en) Information processing apparatus, vehicle, and information processing method using correlation between attributes
JP2004037239A (ja) 同一対象物判断方法および装置、並びに、位置ずれ補正方法および装置
US11954918B2 (en) Object detection device, object detection method, and storage medium
US11769267B2 (en) Object distance measurement apparatus and method
US11543513B2 (en) Information processing apparatus, information processing method and program
WO2022204895A1 (zh) 获取深度图的方法、装置、计算设备和可读存储介质
US20210174528A1 (en) Three-dimensional point cloud generation using a polarimetric camera in a drive assistance system equipped vehicle
JP2017524122A (ja) モバイルプラットフォームの変位を計測する方法及び装置
CN112284416A (zh) 一种自动驾驶定位信息校准装置、方法及存储介质
CN109031333B (zh) 距离测量方法和装置、存储介质、电子设备
JP2021051347A (ja) 距離画像生成装置及び距離画像生成方法
CN112689842B (zh) 一种目标检测方法以及装置
Vaida et al. Automatic extrinsic calibration of LIDAR and monocular camera images
CN113014899B (zh) 一种双目图像的视差确定方法、装置及系统
CN116087987A (zh) 确定目标对象高度的方法、装置、电子设备及存储介质
WO2021077358A1 (zh) 测距方法、测距装置及计算机可读存储介质
JP7064400B2 (ja) 物体検知装置
CN113052886A (zh) 一种采用双目原理的双tof相机深度信息获取方法
US20190250260A1 (en) Electronic device and positioning method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21933581

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202180096494.6

Country of ref document: CN

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21933581

Country of ref document: EP

Kind code of ref document: A1