WO2023071307A1 - 一种三维成像的方法和装置 - Google Patents

一种三维成像的方法和装置 Download PDF

Info

Publication number
WO2023071307A1
WO2023071307A1 PCT/CN2022/106360 CN2022106360W WO2023071307A1 WO 2023071307 A1 WO2023071307 A1 WO 2023071307A1 CN 2022106360 W CN2022106360 W CN 2022106360W WO 2023071307 A1 WO2023071307 A1 WO 2023071307A1
Authority
WO
WIPO (PCT)
Prior art keywords
groups
optical signals
optical
signals
pixels
Prior art date
Application number
PCT/CN2022/106360
Other languages
English (en)
French (fr)
Inventor
陈宇宸
邹冰
邓宁
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023071307A1 publication Critical patent/WO2023071307A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/08—Systems determining position data of a target for measuring distance only
    • G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • G01S17/18—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves wherein range gates are used
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06—Systems determining position data of a target
    • G01S17/46—Indirect determination of position data
    • G01S17/48—Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88—Lidar systems specially adapted for specific applications
    • G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
    • G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/514—Depth or shape recovery from specularities

Definitions

  • the embodiments of the present application relate to the technical field of 3D imaging, and more specifically, to a method and device for 3D imaging.
  • the 3D sensing system can obtain the complete geometric information of the 3D scene in the real world, so that the image with depth information can be used to realize the accurate digitization of the scene, and realize high-precision recognition, positioning and reconstruction , scene understanding and other functions.
  • time of flight is a method of measuring the round-trip flight time of actively emitted light pulses between the transmitting device, the target object, and the receiving device, and based on Depth measurement technology that obtains accurate distance information at the speed of light.
  • photodetection devices used to measure time-of-flight in TOF technology such as indirect photon time-of-flight (i-TOF) image sensors and single-photon avalanche diodes ( single photon avalanche diode (SPAD) arrays cannot achieve high-pixel imaging at a lower cost.
  • i-TOF indirect photon time-of-flight
  • SPAD single photon avalanche diode
  • Embodiments of the present application provide a method and device for three-dimensional imaging, which can obtain a three-dimensional image with high spatial resolution at a relatively low cost under existing process conditions.
  • a method for three-dimensional imaging including: acquiring N groups of optical signals, each group of optical signals includes M optical signals, each optical signal corresponds to a pixel of an optical modulation device one by one, and in the same group of optical signals
  • the M optical signals of the detector correspond to the M pixels of the detection device respectively, N and M are positive integers greater than or equal to 2, and at least one group of optical signals in the N groups of optical signals has a different acquisition time from the remaining groups of optical signals; based on the N A group of optical signals is used to determine a three-dimensional image, the three-dimensional image includes N sub-three-dimensional images, and the N sub-three-dimensional images are in one-to-one correspondence with the N groups of optical signals.
  • the embodiment of the present application obtains corresponding N sub-3D images based on N groups of optical signals in sequence in terms of acquisition time, so that the N sub-3D images are combined to form a 3D image, which can be used in the existing Under the condition of the technology, the three-dimensional image with high spatial resolution can be obtained at a lower cost.
  • the acquiring N groups of optical signals includes: determining N groups of modulation signals, each group of modulation signals is used to turn on a corresponding group of pixels in the light modulation device.
  • Optical paths using the N groups of modulation signals to open the optical paths corresponding to the pixels in the light modulation device, and obtain the N groups of optical signals.
  • the embodiment of the present application can obtain a high-resolution three-dimensional image under the existing process conditions.
  • the acquisition time of each group of optical signals in the N groups of optical signals is different.
  • the embodiment of the present application can acquire N groups of optical signals in N time periods, and obtain independent N sub-3D images based on the N groups of optical signals, and combine the independent N sub-3D images The sub-3D images are combined to obtain a 3D image with a resolution of N*M, thereby improving the resolution of the 3D image.
  • the pixels of the light modulation device are arranged in M matrices, each matrix includes N pixels, and each pixel is connected to one optical signal in each group of optical signals
  • using N sets of modulation signals, opening the optical path corresponding to the pixel in the light modulation device, and obtaining the N sets of optical signals includes: using the N sets of modulation signals, and simultaneously turning on the pixels corresponding to a set of modulation signals in M matrices corresponding optical paths, and turn on the optical paths corresponding to the pixels corresponding to the N groups of modulation signals in the M matrices in sequence.
  • M pixels corresponding to N groups of modulation signals in M matrices will be turned on at the same time, and there are only optical paths corresponding to pixels of corresponding modulation signals in each matrix is turned on, and the optical paths corresponding to the remaining pixels in each matrix remain closed.
  • the embodiment of the present application can reduce the influence of variegated light, and can avoid crosstalk between different groups of optical signals. In this way, it can further improve The resolution of the 3D image.
  • a ratio N of pixels of the light modulation device to pixels of the detection device is a square number.
  • the embodiments of the present application can control that the aspect ratio of the final three-dimensional image will not be changed.
  • each optical signal is obtained by focusing signal light in a corresponding external area through a lens.
  • the light modulation device is any one of the following: a liquid crystal light valve, a digital micromirror device, and a liquid crystal on silicon.
  • a three-dimensional imaging device including: an optical modulation device, configured to obtain N groups of optical signals, each group of optical signals includes M optical signals, and each optical signal is connected to a pixel of the optical modulation device one by one
  • the M optical signals in the same group of optical signals correspond to the M pixels of the detection device respectively
  • N and M are positive integers greater than or equal to 2
  • the acquisition time is different; the detection device is used to determine a three-dimensional image based on the N groups of optical signals, the three-dimensional image includes N sub-three-dimensional images, and the N sub-three-dimensional images correspond to the N groups of optical signals one by one.
  • the device further includes: a control module, configured to determine N sets of modulation signals, each set of modulation signals is used to turn on a corresponding set of pixels in the light modulation device Corresponding optical paths; the control module is further configured to use the N groups of modulation signals to open the optical paths corresponding to the pixels in the light modulation device, and obtain the N groups of optical signals.
  • a control module configured to determine N sets of modulation signals, each set of modulation signals is used to turn on a corresponding set of pixels in the light modulation device Corresponding optical paths; the control module is further configured to use the N groups of modulation signals to open the optical paths corresponding to the pixels in the light modulation device, and obtain the N groups of optical signals.
  • the acquisition time of each group of optical signals in the N groups of optical signals is different.
  • the pixels of the light modulation device are arranged in M matrices, each matrix includes N pixels, and each pixel is connected to one light in each group of light signals
  • the control module is used to: use the N groups of modulation signals, simultaneously open the optical paths corresponding to the pixels corresponding to a group of modulation signals in the M matrices, and sequentially open the light paths corresponding to the pixels corresponding to the N groups of modulation signals in the M matrices light path.
  • the ratio N of the pixels of the light modulator to the pixels of the detection device is a square number.
  • the transposition further includes: a lens, configured to focus signal light in a corresponding external area to obtain each optical signal.
  • the light modulation device is any one of the following: a liquid crystal light valve, a digital micromirror device, and a liquid crystal on silicon.
  • the apparatus further includes: a transmitting device, configured to emit an amplitude-modulated light beam, and the light beam is used to measure depth information of the target object.
  • a third aspect provides a laser radar system
  • the laser radar system includes the device described in the second aspect and any possible implementation, the laser radar system also includes: a transmitter, a receiver, a signal processing device and image processing device.
  • a vehicle in a fourth aspect, includes the device described in the second aspect and any possible implementation manner, and the vehicle further includes: a wireless communication device, a display screen, and an image processing device.
  • an indoor three-dimensional sensing system includes the device described in the second aspect and any possible implementation manner, the indoor three-dimensional sensing system further includes: an image processing device, Transmitter and receiver.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • Fig. 2 is a schematic flowchart of a three-dimensional imaging method according to an embodiment of the present application.
  • Fig. 3 is a schematic block diagram of a pixel correspondence relationship between a light modulation device and a detection device provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a sequence provided by an embodiment of the present application.
  • Fig. 5 is a schematic block diagram of a three-dimensional imaging device provided by an embodiment of the present application.
  • the 3D sensing system can obtain complete geometric information from 3D scenes in the real world, and based on these geometric information, or depth information, realize the construction of a complete 3D scene about the real world.
  • FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application.
  • the three-dimensional sensing system measures the round-trip flight time of light pulses between the transmitting device, the target object (such as a residence, office building, bus, hotel, etc.) and the receiving device, And obtain accurate distance information based on the speed of light, and realize the construction of a three-dimensional scene about the surrounding environment based on the accurate distance information.
  • the 3D sensing system measures the light pulses going to and from the house, and based on the speed of light, obtains the depth information about the house, thereby constructing a 3D scene about the house; or, the 3D sensing system measures the light pulses going to and from the bus , and based on the speed of light, obtain the depth information about the bus, so as to construct a three-dimensional scene about the bus, and so on.
  • One of the core elements of a 3D sensing system to construct a 3D scene about the real world is to obtain a 3D image with high spatial resolution, and construct a 3D scene about the real world based on the 3D image.
  • the existing technical solution is to estimate the accurate depth information of the area between the SPAD pixels by fusing the low spatial resolution image of the SPAD array with the two-dimensional RGB high spatial resolution image information, thereby improving the three-dimensional sensing system of the three-dimensional sensing system.
  • the spatial resolution of the image result.
  • the accuracy of the 3D image obtained by the 3D sensing system based on the above-mentioned technical means is limited, and additional RGB image information needs to be provided.
  • more powerful computing power support and longer image processing time are required to obtain 3D images with high spatial resolution and high accuracy, in other words, this increases the cost of the 3D sensing system to acquire high-pixel 3D images.
  • the present application provides a method and device for three-dimensional imaging, which can obtain three-dimensional images with high spatial resolution at a relatively low cost under existing technological conditions.
  • the application scenario shown in Figure 1 is only understood as an example, but it cannot limit the field where the 3D imaging method described in the embodiment of the present application can actually be applied.
  • the 3D imaging method described in the embodiment of the present application does not It is limited to measurements based on amplitude-modulated light beams, for example, measurements based on optical pulses, measurements based on sine waves, measurements based on square waves, etc., can also be measured in other ways, the embodiment of the present application Not specifically limited.
  • the detection device in the embodiment of the present application may be a SPAD array, an i-TOF image sensor, or other sensors, which are not specifically limited in the embodiment of the present application.
  • the three-dimensional imaging method provided by the present application will be described below with reference to FIG. 2 .
  • FIG. 2 is a schematic flowchart of a three-dimensional imaging method provided by the present application. The specific content is shown in Figure 2.
  • the method #200 includes:
  • each group of optical signals includes M optical signals, each optical signal corresponds to the pixels of the optical modulation device one by one, and the M optical signals in the same group of optical signals correspond to M pixels of the detection device respectively , the acquisition time of at least one group of optical signals in the N groups of optical signals is different from that of the other groups of optical signals.
  • the acquisition time of at least one group of optical signals in the N groups of optical signals is different from that of the other groups of optical signals.
  • the N groups of optical signals are N groups of optical signals with timing.
  • the N groups of optical signals with timing can be understood as: there is a time sequence among the N groups of optical signals. More specifically, the N groups of optical signals are not acquired simultaneously within one period, but are acquired multiple times in multiple periods. , that is, among the N groups of optical signals, the acquisition time of at least one group of optical signals is different from the acquisition time of other groups of optical signals.
  • the time sequence may be that each group of optical signals is acquired sequentially within a corresponding time period.
  • the first group of optical signals is obtained in the first period
  • the second group of optical signals is obtained in the second period
  • the ninth group of optical signals is obtained in the ninth period
  • the timing can be understood as a sequential arrangement , or, a period corresponds to a group of optical signals.
  • the timing may be to simultaneously acquire multiple groups of optical signals within a period of time.
  • the simultaneous acquisition of multiple groups of optical signals within one time period may be understood as: the number of groups of multiple groups of optical signals simultaneously acquired in different time periods may be the same or different.
  • the number of groups is the same, for example, the first group of optical signals, the second group of optical signals and the third group of optical signals are simultaneously acquired in the first period, and the fourth group of optical signals and the fifth group of optical signals are simultaneously acquired in the second period.
  • the embodiment of the present application can also increase the resolution of the obtained three-dimensional image by 3 times.
  • time sequence can be divided into two categories: one type is: one time period corresponds to a group of optical signals, and the other type is: one time period corresponds to multiple groups of optical signals, and the multiple groups of optical signals are among the N groups of optical signals Some groups of optical signals, not all N groups of optical signals.
  • each time period may be the same or different, and at the same time, there may or may not be an interval between each time period, which is not specifically limited in this embodiment of the present application.
  • the specific arrangement manner of the N groups of pixels of the light modulation device may be arranged in an aggregated manner or in a crossed manner.
  • the M pixels included in each group of pixels are all aggregated together.
  • the N groups of pixels are divided into N rows and arranged, and each row corresponds to a group of pixels; or, the N groups of pixels are divided into N arrays, and each array includes M pixels.
  • N groups of pixels are divided into M matrices, and each matrix includes one pixel in each group of pixels, that is, the pixels included in each matrix are formed by combining N groups of pixels.
  • the embodiment of the present application does not limit the specific arrangement manner of each group of pixels in the N groups of pixels.
  • the embodiment of the present application regulates the number and sequence of optical signals captured by the detection device in a time sequence and grouping manner, so that N groups of optical signals that exist in sequence at the acquisition time can be obtained, which can effectively Increase the resolution of 3D images.
  • each optical signal is in one-to-one correspondence with the pixels of the optical modulation device, which can be understood as: the number of pixels of the optical modulation device determines the total number of optical signals in the N groups of optical signals, and each pixel corresponds to an optical signal, When the optical channel corresponding to each pixel is turned on or turned on, the corresponding optical signal can be obtained.
  • each group of optical signals correspond to M pixels of the detection device, which can be understood as: when the number of pixels of the detection device is M, then the number of optical signals included in each group of optical signals is M , and each optical signal corresponds to each pixel one-to-one.
  • S220 Determine a three-dimensional image based on the N groups of optical signals, where the three-dimensional image includes N sub-three-dimensional images, and the N sub-three-dimensional images are in one-to-one correspondence with the N groups of optical signals.
  • one group of optical signals corresponds to one sub-3D image
  • N groups of optical signals correspond to N sub-3D images. Since the number of optical signals included in each group of optical signals is M, the number of pixels of the detection device is M, and the resolution of a sub-3D image is M*1, the resolution of the obtained 3D image composed of N sub-3D images is generally It is N*M.
  • the present application can realize acquiring three-dimensional images with different resolutions.
  • the total resolution of the three-dimensional image is N*M; or, when the timing corresponds to multiple groups of optical signals for one period, the total resolution of the three-dimensional image is It is larger than M*1 and smaller than N*M, but both can improve the total resolution of the three-dimensional image.
  • the embodiment of the present application obtains corresponding N sub-3D images based on N groups of optical signals in sequence in terms of acquisition time, so that the N sub-3D images are combined to form a 3D image, which can be used in the existing Under the condition of the technology, the three-dimensional image with high spatial resolution can be obtained at a lower cost.
  • the embodiment of the present application regulates the number and sequence of the optical signals captured by the detection device in a time sequence and grouping manner, so that N groups of optical signals that are sequenced in the acquisition time can be obtained, and thus The resolution of the three-dimensional image can be effectively improved.
  • obtaining N groups of optical signals includes:
  • each group of modulation signals is used to open the optical path corresponding to a corresponding group of pixels in the light modulation device;
  • N groups of modulation signals open the light path corresponding to the pixel in the light modulation device, and obtain N groups of light signals.
  • the timing between the N groups of modulation signals is the same as the timing between the N groups of optical signals, because a group of modulation signals corresponds to a group of optical signals one by one.
  • the embodiment of the present application controls the light path corresponding to the corresponding pixel in the light modulation device in a manner of modulating a signal, and acquires a corresponding light signal. Therefore, the timing between the N groups of modulation signals is the same as the timing between the N groups of optical signals.
  • each group of modulation signals in the N groups of modulation signals can control the optical path corresponding to a group of pixels in the light modulation device, in other words, each group of modulation signals can control the optical path corresponding to each pixel in the group of pixels
  • the turn-on and turn-off of the corresponding group of optical signals are controlled to obtain or not to obtain.
  • the modulation signal may be a high-low level signal.
  • the high level is used to open the optical channel corresponding to the pixel, and the low level is used to close the optical channel corresponding to the pixel, or the low level is used to open the optical channel corresponding to the pixel, and the high level is used to close the corresponding optical channel of the pixel.
  • the modulating signal can also be a sine wave signal, a triangular wave signal, and so on.
  • the embodiment of the present application does not specifically limit the specific type of the modulation signal, as long as it can control the opening and closing of the optical path corresponding to the pixel.
  • timings existing in the N groups of modulation signals are the same as the timings existing in the N groups of optical signals.
  • the embodiment of the present application achieves the acquisition of N groups of optical signals in the order of acquisition time in the form of modulated signals, but other implementation methods that can achieve the same effect are not excluded, such as , by setting N groups of timers, and each group of timers includes M timers, each timer is used to control the opening of the optical channel corresponding to a corresponding pixel, so that the acquisition time exists N groups of optical signals in sequence.
  • the acquisition time of each group of optical signals in the N groups of optical signals is different.
  • N groups of corresponding optical signals are acquired within N time periods, N sub-3D images are determined based on the N groups of optical signals, and N sub-3D images are combined based on the time sequence to obtain a complete 3D image.
  • the resolution of the three-dimensional image can be further significantly improved.
  • the pixels of the optical modulation device are arranged in M matrices, each matrix includes N pixels, and each pixel corresponds to one optical signal in each group of optical signals, then N groups of modulation signals are used, Open the optical path corresponding to the pixel in the optical modulation device to obtain N groups of optical signals, including:
  • the optical paths corresponding to the M pixels corresponding to the group of modulation signals in the M matrices are simultaneously opened, and The optical paths corresponding to the pixels corresponding to N groups of modulation signals in the M matrices are sequentially opened, and finally N groups of optical signals are obtained.
  • each of the four matrices includes four pixels, which are the first pixel, the second pixel, the third pixel and the fourth pixel, and the first pixel corresponds to an optical signal of the first group of optical signals, and the first pixel corresponds to an optical signal of the first group of optical signals, and The second pixel corresponds to an optical signal of the second group of optical signals, the third pixel corresponds to an optical signal of the third group of optical signals, and the fourth pixel corresponds to an optical signal of the fourth group of optical signals, then at the time of the first group of modulation signals During the period, the optical path corresponding to the first pixel in each of the four matrices is simultaneously turned on, and the optical paths corresponding to the remaining pixels remain closed.
  • each of the four matrices The light path corresponding to the second pixel in the first matrix is turned on at the same time, and the light path corresponding to the other pixels remains closed, and so on, the light path corresponding to the pixel in each of the four matrices will be turned on at the same time, but only The light paths corresponding to the pixels corresponding to the modulated signal are turned on, and the light paths corresponding to the remaining pixels are kept off.
  • the optical paths corresponding to the pixels corresponding to a group of modulation signals in M matrices are simultaneously opened to obtain a corresponding group of optical signals, and a corresponding sub-3D image is obtained, and according to The opening sequence of the optical channels corresponding to each pixel in each matrix is combined according to the opening sequence of the obtained N sub-3D images to obtain a high-resolution 3D image.
  • the embodiment of the present application can realize the acquisition of a three-dimensional image with a higher resolution, and at the same time reduce the influence of variegated light, and can avoid the crosstalk phenomenon between different groups of optical signals, so that the resolution of the three-dimensional image can be further improved Rate.
  • the M pixels corresponding to the group of modulation signals in the M matrices will be turned on at the same time, and there are only optical paths corresponding to the pixels of the corresponding modulation signals in each matrix is turned on, and the optical paths corresponding to the remaining pixels in each matrix remain closed.
  • the embodiment of the present application can reduce the influence of variegated light, and can avoid crosstalk between different groups of optical signals. In this way, it can further improve The resolution of the 3D image.
  • FIG. 2 The three-dimensional imaging method shown in FIG. 2 will be further described below in conjunction with FIG. 3 and FIG. 4 .
  • Fig. 3 is a schematic block diagram of a pixel correspondence relationship between a light modulation device and a detection device provided by an embodiment of the present application. Specifically shown in Figure 3.
  • each matrix includes 9 pixels, wherein each The pixels are in one-to-one correspondence with one light signal in each group of light signals in the nine groups of light signals.
  • the numbers 1-9 shown in FIG. 3 represent different groups of optical signals, for example, the number 1 represents the first group of optical signals, the number 2 represents the second group of optical signals, and so on, and the number 9 represents the ninth group of optical signals.
  • groups of optical signals, and each group of optical signals includes 4 optical signals.
  • each dotted box represents a pixel of the light modulation device
  • each solid line represents a pixel of the detection device
  • nine pixels of the light modulation device correspond to one pixel of the detection device, that is, the light
  • the pixels represented by nine dotted-line frames surrounded by the solid-line frame of the modulation device correspond to the pixels represented by one solid-line frame of the detection device.
  • FIG. 4 is a schematic diagram of a sequence provided by an embodiment of the present application. Specifically shown in Figure 4.
  • T1 shown in Figure 4 represents the first time period
  • T2 represents the second time period
  • T9 represents the ninth time period
  • only one group of optical signals can be obtained in each time period, for example, the first group is obtained in the T1 time period
  • the second group of optical signals is obtained during the T2 period
  • the ninth group of optical signals is obtained within the T9 period.
  • the T1 period indicates that the first group of modulation signals is in use
  • the T2 period indicates that the second group of modulation signals is in use
  • the T9 period indicates that the ninth group of modulation signals is in use, and only one group of modulation signals can be used in each period Signal.
  • Modulating signal, the second group of modulating signal...the ninth group of modulating signal then only the optical path corresponding to one pixel in each matrix is opened, for example, when the first group of modulating signal is used, the number in each matrix is 1
  • the optical channels corresponding to the pixels in each matrix are turned on, and the optical channels corresponding to the remaining pixels in each matrix are closed, then the detection device obtains a sub-3D image with a resolution of 4*1; then, turn on each matrix in turn
  • the detection device obtains the remaining eight sub-3D images with a resolution of 4*1, and combines the obtained nine sub-3D images according to the order in which the optical channels corresponding to the pixels are opened Get up and get a 3D image with a resolution of 4*9.
  • the ratio N between the pixels of the light modulator and the pixels of the detection device is a square number.
  • the pixel ratio of the two is 3 2 : 1, in other words, the pixel ratio of the light modulation device to the detection device is 3 2 , then N
  • the value of M is 9, and the value of M is 4; or, if the pixel of the light modulation device is 64, and the pixel of the detection device is 4, then the pixel ratio of the two is 4 2 : 1, in other words, the light modulation device and the detection device
  • the pixel ratio of is 4 2 , then the value of N is 16, and the value of M is 4.
  • the embodiments of the present application can control that the aspect ratio of the final imaged three-dimensional image will not be changed.
  • each optical signal is obtained by focusing the signal light in a corresponding external area through a lens.
  • the embodiment of the present application can make objects correspond to images one by one.
  • the outer corresponding area can be understood as the corresponding imaging area of the pixel.
  • the embodiments of the present application take the light modulation device as a liquid crystal light valve and digital micromirror device (digital micromirrordevice, DMD) respectively, and the detection device as an example of a SPAD array. , to describe the three-dimensional imaging method of the embodiment of the present application.
  • the light modulation device is a liquid crystal light valve
  • the 36 pixels of the liquid crystal light valve can be divided into 9 groups of pixels, and each group of pixels includes 4 pixels, and the 36 pixels of the light modulation device are arranged in four matrices, and each matrix includes one pixel in each group of pixels.
  • one pixel of the SPAD array corresponds to nine pixels of the liquid crystal light valve.
  • the specific corresponding relationship can be referred to the schematic diagram of the matrix arrangement of the pixels of the light modulation device shown in FIG. 3 , which will not be repeated here.
  • each group of modulation signals is used to control the opening and closing of the light path corresponding to a group of pixels.
  • the first group of modulation signals The signal is used to control the opening and closing of the light path corresponding to the first group of pixels, and so on, the ninth group of modulation signals is used to control the opening and closing of the light path corresponding to the ninth group of pixels.
  • the embodiment of the present application uses a group of modulation signals to open the optical paths corresponding to a group of pixels within a period of time, but the pixels included in each group of pixels are distributed in four In the matrix, therefore, only the optical path corresponding to one pixel in each matrix is opened, and so on, nine groups of modulation signals are required to open the optical path corresponding to the corresponding pixel in each matrix in sequence, and then The nine sub-3D images are obtained sequentially. After the nine sub-3D images are obtained, the embodiment of the present application splices the nine sub-3D images according to the order in which the modulation signals are used to form a complete high-resolution 3D image.
  • each group of modulation signals includes 4 modulation signals, and the modulation signals are used to rotate the liquid crystal molecules corresponding to each pixel of the liquid crystal light valve, so as to realize the opening of the corresponding optical path of the pixel, and then obtain the corresponding light signal.
  • the specific imaging method is as above, the difference is that the modulation signal is used to deflect each micromirror of the DMD, so as to realize the opening of the corresponding light path of the pixel, and then obtain the corresponding light signal.
  • the three-dimensional imaging device provided by the embodiment of the present application will be described below with reference to FIG. 5 .
  • Fig. 5 is a schematic block diagram of a three-dimensional imaging device provided by an embodiment of the present application.
  • the unit includes:
  • the light modulation device is configured to perform the aforementioned step S210, and for specific content, refer to the detailed content described in the aforementioned method embodiment.
  • the detection device is configured to execute the aforementioned step S220, and for specific content, please refer to the detailed content described in the aforementioned method embodiment.
  • the device also includes:
  • control module is configured to execute the steps or methods involved in modulating signals in the foregoing method embodiments, and for specific content, please refer to the detailed content described in the foregoing method embodiments.
  • control module may be a system on chip (system of chip, SOC).
  • the device further includes: a lens, configured to focus the signal light in the corresponding external area to obtain each optical signal.
  • the device further includes: a transmitting device, configured to emit an amplitude-modulated light beam, where the light beam is used to measure depth information of the target object.
  • each optical signal acquired by the optical modulation device is a signal carrying the depth information of the target object returned by the light beam emitted by the emitting device after reaching the target object, and the detection device is based on these The optical signal enables the construction of a three-dimensional image about the target object.
  • the device further includes: a receiving device, configured to receive the light beam carrying the depth information of the target object.
  • a receiving device configured to receive the light beam carrying the depth information of the target object.
  • the light beam comprises an optical signal.
  • each device and module included in the three-dimensional imaging device shown in FIG. 5 is used to implement the corresponding methods and steps in the foregoing method embodiments.
  • An embodiment of the present application further provides a laser radar system, which includes the three-dimensional imaging device shown in FIG. 5 , and the laser radar system further includes: a transmitter, a receiver, a signal processing device, and an image processing device.
  • An embodiment of the present application also provides a vehicle, the vehicle includes the three-dimensional imaging device shown in FIG. 5 , and the vehicle further includes: a wireless communication device, a display screen, and an image processing device.
  • the embodiment of the present application also provides an indoor three-dimensional sensing system, the indoor three-dimensional sensing system includes the three-dimensional imaging device shown in FIG. 5 , and the indoor three-dimensional sensing system further includes: an image processing device, a transmitting device and a receiving device.
  • the disclosed systems, devices and methods may be implemented in other ways.
  • the device embodiments described above are only illustrative.
  • the division of the units is only a logical function division. In actual implementation, there may be other division methods.
  • multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the functions described above are realized in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (read-only memory, ROM), random access memory (random access memory, RAM), magnetic disk or optical disc and other media that can store program codes. .

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Electromagnetism (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)

Abstract

一种三维成像的方法和装置，该三维成像的方法包括：获取N组光信号，每组光信号包括M个光信号，每个光信号与光调制器件的像素一一对应，同一组光信号中的M个光信号分别对应探测器件的M个像素，N、M为大于或等于2的正整数，N组光信号中的至少一组光信号与其余组光信号的获取时间不同；基于N组光信号，确定三维图像，该三维图像包括N个子三维图像，N个子三维图像与N组光信号一一对应。基于在获取时间上存在先后顺序的N组光信号，获得对应的N个子三维图像，从而将该N个子三维图像组合起来构成一个三维图像，从而能够在现有的工艺条件下，实现以较低的成本获取高空间分辨率的三维图像。

Description

一种三维成像的方法和装置
本申请要求于2021年10月26日提交中国国家知识产权局、申请号202111248357.8、申请名称为“一种三维成像的方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及三维成像技术领域，更具体地，涉及一种三维成像的方法和装置。
背景技术
三维传感系统作为各类智能设备的眼睛，能够获取现实世界中三维场景完整的几何信息，从而利用带有深度信息的图像来实现对场景的精准数字化，并实现高精度的识别、定位、重建、场景理解等功能。
作为三维传感系统领域中最为常用的技术方案，飞行时间(time of flight，TOF)是一种通过测量主动发出的光脉冲在发射装置、目标物体以及接收装置之间的往返飞行时间，并基于光速获取精确距离信息的深度测量技术。
然而，受限于工艺水平或读出处理电路尺寸，TOF技术中用于测量飞行时间的光探测器件，例如间接光子飞行时间(indirect time of flight，i-TOF)图像传感器和单光子雪崩二极管(single photon avalanche diode，SPAD)阵列，无法以较低的成本实现高像素成像。
发明内容
本申请实施例提供一种三维成像的方法和装置，能够在现有的工艺条件下，实现以较低的成本获取高空间分辨率的三维图像。
第一方面，提供了一种三维成像的方法，包括：获取N组光信号，每组光信号包括M个光信号，每个光信号与光调制器件的像素一一对应，同一组光信号中的M个光信号分别对应探测器件的M个像素，N、M为大于或等于2的正整数，N组光信号中的至少一组光信号与其余组光信号的获取时间不同；基于该N组光信号，确定三维图像，该三维图像包括N个子三维图像，N个子三维图像与该N组光信号一一对应。
通过该方案，本申请实施例基于在获取时间上存在先后顺序的N组光信号，获得对应的N个子三维图像，从而将该N个子三维图像组合起来构成一个三维图像，从而能够在现有的工艺条件下，实现以较低的成本获取高空间分辨率的三维图像。
结合第一方面，在第一方面的某些可能实现方式中，该获取N组光信号，包括：确定N组调制信号，每组调制信号用于开启光调制器件中对应的一组像素对应的光通路；使用该N组调制信号，开启光调制器件中像素对应的光通路，并获取该N组光信号。
通过以调制信号的方式获取在获取时间上存在先后顺序的的N组光信号，继而根据该N组光信号获得对应的N个子三维图像，并将该N个子三维图像组合起来，获得一张三维图像， 本申请实施例能够在现有的工艺条件下获得高分辨率的三维图像。
结合第一方面，在第一方面的某些可能实现方式中，N组光信号中的每组光信号的获取时间均不相同。
通过在一个时段内获取一组光信号的时序方式，本申请实施例能够在N个时段内获取N组光信号，并基于N组光信号获取独立的N个子三维图像，并将该独立的N个子三维图像组合起来，获得分辨率为N*M的三维图像，从而提高三维图像的分辨率。
结合第一方面，在第一方面的某些可能实现方式中，光调制器件的像素以M个矩阵排布，每个矩阵包括N个像素，每个像素与每组光信号中的一个光信号对应，该使用N组调制信号，开启光调制器件中像素对应的光通路，并获取该N组光信号，包括：使用该N组调制信号，同时开启M个矩阵中对应一组调制信号的像素对应的光通路，且依次开启M个矩阵中对应N组调制信号的像素对应的光通路。
在每次仅获取一组光信号的时间顺序下，M个矩阵中与N组调制信号对应的M个像素会同时开启，且每个矩阵内仅有与对应的调制信号的像素对应的光通路得到开启，每个矩阵内其余像素对应的光通路保持关闭状态，如此，本申请实施例能够减少杂色光的影响，并且能够避免不同组光信号之间发生的串扰现象，如此，能够进一步的提高三维图像的分辨率。
结合第一方面，在第一方面的某些可能实现方式中，光调制器件的像素与探测器件的像素的比值N为平方数。
通过控制光调制器件与探测器件的像素比值为一个平放数，本申请实施例能够控制最终成像的三维图像的长宽比不会被改变。
结合第一方面，在第一方面的某些可能实现方式中，每个光信号是通过透镜对外部对应区域的信号光聚焦所得。
结合第一方面，在第一方面的某些可能实现方式中，光调制器件为以下任意一个：液晶光阀、数字微镜器件和硅基液晶。
第二方面，提供了一种三维成像的装置，包括：光调制器件，用于获取N组光信号，每组光信号包括M个光信号，每个光信号与该光调制器件的像素一一对应，同一组光信号中的M个光信号分别对应探测器件的M个像素，N、M为大于或等于2的正整数，N组光信号中的至少一组光信号与其余组光信号的获取时间不同；探测器件，用于基于该N组光信号，确定三维图像，该三维图像包括N个子三维图像，该N个子三维图像与该N组光信号一一对应。
结合第二方面，在第二方面的某些可能实现方式中，该装置还包括：控制模块，用于确定N组调制信号，每组调制信号用于开启该光调制器件中对应的一组像素对应的光通路；该控制模块，还用于使用该N组调制信号，开启该光调制器件中像素对应的光通路，并获取该N组光信号。
结合第二方面，在第二方面的某些可能实现方式中，该N组光信号中的每组光信号的获取时间均不相同。
结合第二方面，在第二方面的某些可能实现方式中，该光调制器件的像素以M个矩阵排布，每个矩阵包括N个像素，每个像素与每组光信号中的一个光信号对应，控制模块，用于：使用该N组调制信号，同时开启该M个矩阵中对应一组调制信号的像素对应的光通路，依次开启M个矩阵中对应N组调制信号的像素对应的光通路。
结合第二方面，在第二方面的某些可能实现方式中，光调制器的像素与探测器件的像素的比值N为平方数。
结合第二方面，在第二方面的某些可能实现方式中，该转置还包括：透镜，用于将外部对应区域的信号光聚焦而得到所述每个光信号。
结合第二方面，在第二方面的某些可能实现方式中，光调制器件为以下任意一个：液晶光阀、数字微镜器件和硅基液晶。
结合第二方面，在第二方面的某些可能实现方式中，该装置还包括：发射器件，用于发射振幅被调制的光束，光束用于测量目标物体的深度信息。
第三方面，提供了一种激光雷达系统，该激光雷达系统包括第二方面以及任一种可能实现方式中所述的装置，该激光雷达系统还包括：发射机、接收机、信号处理装置与图像处理装置。
第四方面，提供了一种车辆，该车辆包括第二方面以及任一种可能实现方式中所述的装置，该车辆还包括：无线通信装置、显示屏与图像处理装置。
第五方面，提供了一种室内三维传感系统，该室内三维传感系统包括第二方面以及任一种可能实现方式中所述的装置，该室内三维传感系统还包括：图像处理装置、发射装置与接收装置。
附图说明
图1是本申请实施例提供的一种应用场景的示意图。
图2是本申请提供实施例的一种三维成像的方法的示意流程图。
图3是本申请实施例提供的一种光调制器件与探测器件的像素对应关系的示意框图。
图4是本申请实施例提供的一种时序的示意图。
图5是本申请实施例提供的一种三维成像的装置的示意框图。
具体实施方式
下面将结合附图，对本申请中的技术方案进行描述。
作为各类智能设备的眼睛，三维传感系统能够获取来自现实世界中三维场景的完整的几何信息，并基于这些几何信息，或者说是深度信息，实现构建关于现实世界的完整的三维场景。
图1是本申请实施例提供的一种应用场景的示意图。在图1所示的应用场景示意图中，三维传感系统通过测量光脉冲在发射装置、目标物体(例如，住宅、办公大楼、大巴、酒店，等等)以及接收装置之间的往返飞行时间，并基于光速获得精确距离信息，并基于精确距离信息实现构建关于周边环境的三维场景。示例性地，三维传感系统通过测量往返于住宅的光脉冲，并基于光速，获取关于住宅的深度信息，从而构建关于住宅的三维场景；或者，三维传感系统通过测量往返于大巴的光脉冲，并基于光速，获取关于大巴的深度信息，从而构建关于大巴的三维场景，不一而足。
三维传感系统构建关于现实世界的三维场景的核心要素之一是要获取高空间分辨率的三维图像，并基于该三维图像构建关于现实世界的三维场景。现有的技术方案是通过将SPAD阵列的低空间分辨率图像与二维的RGB高空间分辨率图像信息进行融合，以估计SPAD像素 之间区域的精确深度信息，从而提高三维传感系统的三维图像结果的空间分辨率。
然而，三维传感系统基于上述技术手段所得出的三维图像的结果的精确性有限，且需要提供额外的RGB图像信息，另外，还需要更强大的算力支撑以及更长的图像处理时间才能获得具有高空间分辨率高和高准确性的三维图像，换言之，这增加了三维传感系统获取高像素的三维图像的成本。
鉴于上述技术问题，本申请提供了一种三维成像的方法和装置，能够在现有的工艺条件下，实现以较低的成本获取高空间分辨率的三维图像。
需要说明的是，图1所示的应用场景仅作为示例性理解，但不能限定本申请实施例描述的三维成像的方法实际所能应用的领域，例如，本申请实施例描述的三维成像方法不限定于基于振幅被调制的光束的测量，例如，基于光脉冲的测量，还有基于正弦波的测量，还有基于方波的测量，等等，也可以其他的方式进行测量，本申请实施例不做具体限定。
需要说明的是，本申请实施例中的探测器件可以是SPAD阵列，也可以是i-TOF图像传感器，也还可以是其他的传感器，本申请实施例不做具体限定。
下文将结合图2对本申请提供的三维成像的方法做出描述。
图2是本申请提供的一种三维成像的方法的示意流程图。具体内容如图2所示。该方法#200包括：
S210，获取N组光信号，每组光信号包括M个光信号，每个光信号与光调制器件的像素一一对应，同一组光信号中的M个光信号分别对应探测器件的M个像素，N组光信号中的至少一组光信号与其余组光信号的获取时间不同。
具体而言，N组光信号中的至少一组光信号与其余组光信号的获取时间不同，可以理解为N组光信号为存在时序的N组光信号。该存在时序的N组光信号，可以理解为：N组光信号之间存在时间先后顺序，更具体地说，N组光信号并非在一个时段内同时获取，而是分多个时段多次获取，即N组光信号之中，至少有一组光信号的获取时间与其余组光信号的获取时间不同。
示例性地，当存在九组光信号时，则该时序可以是每组光信号在对应时段内依序获取。例如，在第一时段内获取第一组光信号，在第二时段内获取第二组光信号，以此类推，在第九时段内获取第九组光信号，则该时序可以理解为顺序排列，或者，一个时段对应一组光信号。
又示例性地，当存在九组光信号时，该时序可以是在一个时段内同时获取多组光信号。应理解，该一个时段内同时获取多组光信号，可以理解为：不同时段同时获取的多组光信号的组数可以相同，也可以不同。当组数相同时，例如，在第一时段内同时获取第一组光信号、第二组光信号与第三组光信号，在第二时段内同时获取第四组光信号、第五组光信号与第六组光信号，在第三时段内同时获取第七组光信号、第八组光信号与第九组光信号，如此，每个子三维图像的像素是均匀一致的，则本申请实施例能够使获得的三维图像的分辨率得到均匀的提升，即得到3倍的均匀提升；当组数不相同时，例如，在第一时段内获取第一组光信号与第二组光信号，在第二时段内获取第三组光信号、第四组光信号与第五组光信号，在第三时段内获取剩余的第六组光信号、第七组光信号、第八组光信号与第九组光信号，则本申请实施例也能够使获得的三维图像的分辨率得到3倍的提升。
应理解，该时序可以分为两大类：一类是：一个时段对应一组光信号，另一类是：一个 时段对应多组光信号，且该多组光信号是N组光信号中的部分组光信号，并非全部的N组光信号。
需要说明的是，每个时段对应的时间长度可以是相同的，也可以是不同的，同时，每个时段之间可以存在间隔，也可以不存在间隔，本申请实施例不做具体限定。
应理解，本申请实施例对时序不做具体限定，仅需使得N组光信号并非在同一个时段内同时获取即可。
应理解，光调制器件的N组像素的具体排列方式，可以是以聚合方式排列与交叉方式排列。当以聚合方式排列时，例如，每组像素包括的M个像素全部聚集在一起。示例性地，将N组像素分成N行排布，每行对应一组像素；或者，将N组像素分成N个阵列，每个阵列包括M个像素。当以交叉方式排列时，例如，将N组像素分成M个矩阵，且每个矩阵包括每组像素中的一个像素，即每个矩阵所包括的像素是由N组像素组合而成的。本申请实施例对N组像素的每组像素之间的具体排列方式不做限定。
应理解，本申请实施例以时序与分组的方式，调控探测器件所摄入的光信号的数量与先后顺序，如此，便能获取在获取时间存在先后顺序的N组光信号，如此便能有效提高三维图像的分辨率。
应理解，每个光信号与光调制器件的像素一一对应，可以理解为：光调制器件的像素数决定了N组光信号中的光信号的总数，并且每个像素对应着一个光信号，当每个像素所对应的光通路打开或者开启时，才能获取对应的光信号。
应理解，每组光信号中的M个光信号分别对应探测器件的M个像素，可以理解为：探测器件的像素个数为M时，则每组光信号所包括的光信号的数量为M个，且每个光信号与每个像素一一对应。
S220，基于N组光信号，确定三维图像，该三维图像包括N个子三维图像，该N个子三维图像与N组光信号一一对应。
具体而言，一组光信号对应一个子三维图像，N组光信号对应N个子三维图像。由于每组光信号所包括的光信号数量为M，探测器件的像素数为M，一个子三维图像的分辨率为M*1，则获得的由N个子三维图像组成的三维图像的分辨率一般为N*M。
应理解，依据不同的时序方式，本申请能够实现获取不同分辨率的三维图像。示例性地，当时序为一个时段对应一组光信号时，三维图像的总的分辨率为N*M；或者，当时序为一个时段对应多组光信号，该三维图像的总的分辨率是大于M*1且小于N*M，但是均能提高三维图像的总的分辨率。
通过该方案，本申请实施例基于在获取时间上存在先后顺序的N组光信号，获得对应的N个子三维图像，从而将该N个子三维图像组合起来构成一个三维图像，从而能够在现有的工艺条件下，实现以较低的成本获取高空间分辨率的三维图像。
更具体地说，本申请实施例以时序与分组的方式，调控探测器件所摄入的光信号的数量与先后顺序，如此，便能获取在获取时间上存在先后顺序的N组光信号，如此便能有效提高三维图像的分辨率。
作为一种可能的实现方式，获取N组光信号，包括：
确定N组调制信号，每组调制信号用于开启光调制器件中对应的一组像素对应的光通路；
使用N组调制信号，开启光调制器件中像素对应的光通路，并获取N组光信号。
具体而言，N组调制信号之间存在的时序与N组光信号之间存在的时序是相同的，这是由于一组调制信号与一组光信号一一对应。示例性地，本申请实施例以调制信号的方式控制光调制器件中对应的像素对应的光通路，并获取对应的光信号。因此，N组调制信号之间的时序与该N组光信号之间的时序是相同的。
应理解，N组调制信号中的每组调制信号能够控制光调制器件中对应的一组像素所对应的光通路，换言之，每组调制信号能够控制该一组像素中每个像素对应的光通路的导通与关闭，从而控制对应的一组光信号的获取与不获取。
应理解，该调制信号可以是一个高低电平信号。例如，高电平用于开启像素对应的光通路，低电平用于关闭像素对应的光通路，又或者，低电平用于开启像素对应的光通路，高电平用于关闭像素对应的光通路；该调制信号也可以是一个正弦波信号和三角波信号，等等。
需要说明的是，本申请实施例对调制信号的具体类别不做具体限定，只要其能够起到控制像素对应光通路的开启和关闭即可。
应理解，该N组调制信号中存在的时序与该N组光信号之间存在的时序是相同的。
需要说明的是，在本申请实施例中，本申请实施例以调制信号的方式实现获取在获取时间上存在先后顺序的N组光信号，但是并不排除其他能够实现相同效果的实现方式，例如，通过设置N组定时器的方式，且每组定时器包括M个定时器，每个定时器用于控制对应的一个像素所对应的光通路的开启，如此，也能实现获取在获取时间上存在先后顺序的N组光信号。
作为一种可能的实现方式，N组光信号中的每组光信号的获取时间均不相同。
本申请实施例通过在N个时段内获取对应的N组光信号，并基于N组光信号确定N个子三维图像，并基于该时序将N个子三维图像组合起来获得一个完整的三维图像，如此就能够进一步的显著提高三维图像的分辨率。
作为一种可能的实现方式，光调制器件的像素以M个矩阵排布，每个矩阵包括N个像素，每个像素与每组光信号中的一个光信号对应，则使用N组调制信号，开启光调制器件中像素所对应的光通路，获取N组光信号，包括：
使用该N组调制信号，同时开启该M个矩阵中对应一组调制信号的像素对应的光通路，依次开启M个矩阵中对应N组调制信号的像素对应的光通路。
具体而言，当光调制器件的像素以M个矩阵进行排布时，则依据N组调制信号的顺序，同时开启M个矩阵中与该组调制信号对应的M个像素对应的光通路，并依次开启M个矩阵中对应N组调制信号的像素对应的光通路，并最终获取N组光信号。
示例性地，若N的取值为4，M的取值为4，且N组调制信号的顺序为第一组调制信号、第二组调制信号、第三组调制信号与第四组调制信号，则4个矩阵中的每个矩阵均包括四个像素，分别是第一像素、第二像素、第三像素与第四像素，且第一像素对应第一组光信号的一个光信号，第二像素对应第二组光信号的一个光信号，第三像素对应第三组光信号的一个光信号，第四像素对应第四组光信号的一个光信号，则在第一组调制信号的时间段内，四个矩阵中的每个矩阵中的第一像素对应的光通路同时开启，其余像素对应的光通路保持关闭，之后，在第二调制信号的时间段内，四个矩阵中的每个矩阵中的第二像素对应的光通路同时开启，其余像素对应的光通路保持关闭，以此类推，四个矩阵中的每个矩阵的对应像素所对应的光通路会同时开启，但仅限于与调制信号对应的像素对应的光通路会开启，剩余像素对 应的光通路会保持关闭。
换言之，以一个时段对应一组调制信号的方式，同时开启M个矩阵中对应一组调制信号的像素对应的光通路，获取对应的一组光信号，然而获得一个对应的子三维图像，并根据每个矩阵中每个像素所对应的光通路的开启顺序，将获得的N个子三维图像按照该开启顺序组合起来，获得一个高分辨率的三维图像。如此，本申请实施例能够实现获取分辨率更高的三维图像，同时还能减少杂色光的影响，并且能够避免不同组光信号之间发生的串扰现象，如此，能够进一步的提高三维图像的分辨率。
在每次仅获取一组光信号的时间顺序下，M个矩阵中与该组调制信号对应的M个像素会同时开启，且每个矩阵内仅有与对应的调制信号的像素对应的光通路得到开启，每个矩阵内其余像素对应的光通路保持关闭状态，如此，本申请实施例能够减少杂色光的影响，并且能够避免不同组光信号之间发生的串扰现象，如此，能够进一步的提高三维图像的分辨率。
下文将结合图3与图4对图2所示的三维成像的方法作进一步的描述。
图3是本申请实施例提供的一种光调制器件与探测器件的像素对应关系的示意框图。具体如图3所示。
假设光调制器件的像素数是36，N的取值为9，M的取值为4，则光调制器件的所有像素可以分成4个矩阵，且每个矩阵包括9个像素，其中，每个像素与9组光信号中的每组光信号中的一个光信号一一对应。
应理解，图3所示的数字1-9，分别表示不同的组光信号，例如，数字1表示第一组光信号，数字2表示第二组光信号，以此类推，数字9表示第九组光信号，且每组光信号包括4个光信号。
在图3所示的示意框图中，每个虚线框表示光调制器件的一个像素，每个实线框表示探测器件的一个像素，光调制器件的九个像素对应探测器件的一个像素，即光调制器件的实线框所圈住的九个虚线框表示的像素对应着探测器件的一个实线框表示的像素。
图4是本申请实施例提供的一种时序的示意图。具体如图4所示。图4所示的T1表示第一时段，T2表示第二时段，以此类推，T9表示第九时段，在每个时段内仅能获取一组光信号，例如，在T1时段内获取第一组光信号，在T2时段内获取第二组光信号，以此类推，在T9时段内获取第九组光信号。
应理解，该时序也适用于N组调制信号。例如，T1时段表示第一组调制信号在使用，T2时段表示第二组调制信号在使用，以此类推，T9时段表示第九组调制信号在使用，在每个时段内仅能使用一组调制信号。
结合图3与图4可知，根据一个时段对应一组光信号，或者，一个时段对应一组调制信号的时序，本申请实施例依次使用N组调制信号的每组调制信号，依次使用第一组调制信号、第二组调制信号…第九组调制信号，则每个矩阵内仅有一个像素对应的光通路得到开启，例如，在第一组调制信号使用时，每个矩阵内的编号为1的像素对应的光通路得到开启，每个矩阵内的剩余像素对应的光通路处于关闭状态，则该探测器件获得一张分辨率为4*1的子三维图像；然后，依次开启每个矩阵内剩余的每个像素对应的光通路，则该探测器件获得剩余的八个分辨率均为4*1的子三维图像，并将该获得的九个子三维图像根据像素对应的光通路的开启顺序组合起来，获得一张分辨率为4*9的三维图像。通过依次开启每个矩阵内的每个像素，本申请实施例能够减少杂色光的影响，并且能够避免不同组光信号之间发生的串扰现 象，如此，能够进一步的提高三维图像的分辨率。
作为一种可能的实现方式，光调制器的像素与探测器件的像素之间的比值N为平方数。
示例性地，若光调制器件的像素为36，探测器件的像素为4，则二者的像素比值为3 2：1，换言之，光调制器件与探测器件的像素比值为3 2，则N的取值为9，M的取值为4；或者，若光调制器件的像素为64，探测器件的像素为4，则二者的像素比值为4 2:1，换言之，光调制器件与探测器件的像素比值为4 2，则N的取值为16，M的取值为4。
通过控制光调制器件与探测器件的像素比值N为一个平方数，本申请实施例能够控制最终成像的三维图像的长宽比不会被改变。
作为一种可能的实现方式，每个光信号是通过透镜对外部对应区域的信号光聚焦所得。如此，通过透镜成像的效果，本申请实施例能够让物与像一一对应。应理解，该外部对应区域可以理解为该像素的对应成像区域。
应理解，为了更好的理解本申请实施例揭示的三维成像方法，本申请实施例以光调制器件分别为液晶光阀与数字微镜器件(digital micromirrordevice，DMD)、探测器件为SPAD阵列为例，对本申请实施例的三维成像方法进行描述。
当光调制器件为液晶光阀时，假设液晶光阀的像素数为36，SPAD阵列的像素数为4，则可以将液晶光阀的36个像素分为9组像素，且每组像素包括4个像素，同时将光调制器件的36个像素以4个矩阵排布，每个矩阵包括每组像素中的一个像素。换言之，SPAD阵列的一个像素对应液晶光阀的9个像素，具体对应关系可参见图3所示的光调制器件的像素的矩阵排布示意图，在此不再赘述。
在建立SPAD阵列与液晶光阀之间的像素对应关系之后，通过设置9组调制信号，每组调制信号用于控制一组像素对应的光通路的开启与关闭，具体的说，第一组调制信号用于控制第一组像素对应的光通路的开启与关闭，以此类推，第九组调制信号用于控制第九组像素对应的光通路的开启与关闭。
在基于图3所示的矩阵排布的基础之上，本申请实施例在一个时段内，使用一组调制信号开启一组像素对应的光通路，但是每组像素所包括的像素分布在4个矩阵之中，因此，每个矩阵之中仅有一个像素对应的光通路得到开启，以此类推，则需要九组调制信号分次序的开启每个矩阵中的对应像素所对应的光通路，继而分次序的获得九个子三维图像，在获取9个子三维图像之后，本申请实施例按调制信号的使用顺序，将九个子三维图像拼接起来，构成一张完整的高分辨率的三维图像。
应理解，在上述方案中，每组调制信号包括4个调制信号，且该调制信号用于使液晶光阀各个像素对应的液晶分子旋转，从而实现开启该像素对应的光通路，继而获取对应的光信号。
当光调制器件为DMD时，具体的成像方法如上所述，区别在于，该调制信号用于使DMD的各个微镜发生偏转，从而实现开启该像素对应的光通路，继而获取对应的光信号。
下文将结合图5对本申请实施例提供的三维成像的装置进行描述。
图5是本申请实施例提供的一种三维成像的装置的示意框图。该装置包括：
光调制器件，用于执行前述步骤S210，具体内容可以参见前述方法实施例描述的详细内容。
探测器件，用于执行前述步骤S220，具体内容可以参见前述方法实施例描述的详细内容。
作为一种可能的实现方式，该装置还包括：
控制模块，用于执行前述方法实施例中涉及调制信号的步骤或者方法，具体内容也可以参见前述方法实施例描述的详细内容。示例性地，该控制模块可以是片上系统芯片(system of chip，SOC)。
作为一种可能的实现方式，该装置还包括：透镜，用于对外部对应区域的信号光聚焦而得到每个光信号。
作为一种可能的实现方式，该装置还包括：发射器件，用于发射振幅被调制的光束，该光束用于测量目标物体的深度信息。
需要说明的是，在本申请实施例中，光调制器件获取的每个光信号均是发射器件发出的光束在抵达目标物体之后，返回的携带目标物体的深度信息的信号，探测器件并基于这些光信号实现构建关于该目标物体的三维图像。
作为一种可能的实现方式，该装置还包括：接收器件，用于接收携带目标物体的深度信息的光束。应理解，该光束包括光信号。
需要说明的是，图5所示的三维成像的装置所包括的每个器件和模块均用于实现前述方法实施例中对应的方法和步骤。
本申请实施例还提供一种激光雷达系统，该激光雷达系统包括图5所示的三维成像装置，该激光雷达系统还包括：发射机、接收机、信号处理装置与图像处理装置。
本申请实施例还提供一种车辆，该车辆包括图5所示的三维成像装置，该车辆还包括：无线通信装置、显示屏与图像处理装置。
本申请实施例还提供一种室内三维传感系统，该室内三维传感系统包括图5所示的三维成像装置，该室内三维传感系统还包括：图像处理装置、发射装置与接收装置。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (17)

  1. 一种三维成像的方法，其特征在于，包括：
    获取N组光信号，每组光信号包括M个光信号，每个光信号与光调制器件的像素一一对应，同一组光信号中的M个光信号分别对应探测器件的M个像素，所述N、M为大于或等于2的正整数，所述N组光信号中的至少一组光信号与其余组光信号的获取时间不同；
    基于所述N组光信号，确定三维图像，所述三维图像包括N个子三维图像，所述N个子三维图像与所述N组光信号一一对应。
  2. 根据权利要求1所述的方法，其特征在于，所述获取N组光信号，包括：
    确定N组调制信号，每组调制信号用于开启所述光调制器件中对应的一组像素对应的光通路；
    使用所述N组调制信号，开启所述光调制器件中像素对应的光通路，并获取所述N组光信号。
  3. 根据权利要求1或2所述的方法，其特征在于，所述N组光信号中的每组光信号的获取时间均不相同。
  4. 根据权利要求3所述的方法，其特征在于，所述光调制器件的像素以M个矩阵排布，每个矩阵包括N个像素，每个像素与每组光信号中的一个光信号对应，
    所述使用N组调制信号，开启所述光调制器件中像素对应的光通路，并获取所述N组光信号，包括：
    使用所述N组调制信号，同时开启所述M个矩阵中对应一组调制信号的像素对应的光通路，且依次开启M个矩阵中对应N组调制信号的像素对应的光通路。
  5. 根据权利要求1至4中任一项所述的方法，其特征在于，所述光调制器的像素与所述探测器件的像素的比值N为平方数。
  6. 根据权利要求1至5中任一项所述的方法，其特征在于，所述每个光信号是通过透镜对外部对应区域的信号光聚焦所得。
  7. 根据权利要求1至6中任一项所述的方法，其特征在于，所述光调制器件为以下任意一个：
    液晶光阀、数字微镜器件和硅基液晶。
  8. 一种三维成像的装置，其特征在于，包括：
    光调制器件，用于获取N组光信号，每组光信号包括M个光信号，每个光信号与所述光调制器件的像素一一对应，同一组光信号中的M个光信号分别对应探测器件的M个像素，所述N、M为大于或等于2的正整数，所述N组光信号中的至少一组光信号与其余组光信号的获取时间不同；
    所述探测器件，用于基于所述N组光信号，确定三维图像，所述三维图像包括N个子三维图像，所述N个子三维图像与所述N组光信号一一对应。
  9. 根据权利要求8所述的装置，其特征在于，所述装置还包括：
    控制模块，用于确定N组调制信号，每组调制信号用于开启所述光调制器件中对应的一组像素对应的光通路；
    所述控制模块，还用于使用所述N组调制信号，开启所述光调制器件中像素对应的光通路，并获取所述N组光信号。
  10. 根据权利要求8或9所述的装置，其特征在于，所述N组光信号中的每组光信号的获取时间均不相同。
  11. 根据权利要求10所述的装置，其特征在于，所述光调制器件的像素以M个矩阵排布，每个矩阵包括N个像素，每个像素与每组光信号中的一个光信号对应，所述调制模块，用于：
    使用所述N组调制信号，同时开启所述M个矩阵中对应一组调制信号的像素对应的光通路，依次开启M个矩阵中对应N组调制信号的像素对应的光通路。
  12. 根据权利要求8至11中任一项所述的装置，其特征在于，所述光调制器的像素与所述探测器件的像素的比值N为平方数。
  13. 根据权利要求8至12中任一项所述的装置，其特征在于，所述装置还包括：
    透镜，用于将外部对应区域的信号光聚焦而得到所述每个光信号。
  14. 根据权利要求8至13中任一项所述的装置，其特征在于，所述光调制器件为以下任意一个：
    液晶光阀、数字微镜器件和硅基液晶。
  15. 根据权利要求8至14中任一项所述的装置，其特征在于，所述装置还包括：
    发射器件，用于发射振幅被调制的光束，所述光束用于测量目标物体的深度信息。
  16. 一种激光雷达系统，其特征在于，所述激光雷达系统包括权利要求8至15中任一项所述的装置，
    所述激光雷达系统还包括：
    发射机、接收机、信号处理装置与图像处理装置。
  17. 一种车辆，其特征在于，所述车辆包括权利要求8至15中任一项所述的装置，
    所述车辆还包括：
    无线通信装置、显示屏与图像处理装置。
PCT/CN2022/106360 2021-10-26 2022-07-19 一种三维成像的方法和装置 WO2023071307A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111248357.8A CN116027345A (zh) 2021-10-26 2021-10-26 一种三维成像的方法和装置
CN202111248357.8 2021-10-26

Publications (1)

Publication Number Publication Date
WO2023071307A1 true WO2023071307A1 (zh) 2023-05-04

Family

ID=86069598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/106360 WO2023071307A1 (zh) 2021-10-26 2022-07-19 一种三维成像的方法和装置

Country Status (2)

Country Link
CN (1) CN116027345A (zh)
WO (1) WO2023071307A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102891969A (zh) * 2011-07-21 2013-01-23 三星电子株式会社 图像感测设备和操作其的方法
CN110554401A (zh) * 2018-06-04 2019-12-10 英飞凌科技股份有限公司 混合lidar接收器和lidar方法
CN110998365A (zh) * 2017-07-05 2020-04-10 奥斯特公司 具有电子扫描发射器阵列和同步传感器阵列的光测距装置
US20200120299A1 (en) * 2018-10-10 2020-04-16 Northwestern University Method and system for time-of-flight imaging with high lateral resolution
CN111427052A (zh) * 2020-06-09 2020-07-17 深圳市汇顶科技股份有限公司 基于飞行时间的测距方法和相关测距系统
CN111458717A (zh) * 2020-04-20 2020-07-28 深圳奥比中光科技有限公司 一种tof深度测量装置、方法及电子设备
CN111798503A (zh) * 2019-03-21 2020-10-20 英飞凌科技股份有限公司 利用飞行时间相机同时进行数据传输和深度图像记录

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102891969A (zh) * 2011-07-21 2013-01-23 三星电子株式会社 图像感测设备和操作其的方法
CN110998365A (zh) * 2017-07-05 2020-04-10 奥斯特公司 具有电子扫描发射器阵列和同步传感器阵列的光测距装置
CN110554401A (zh) * 2018-06-04 2019-12-10 英飞凌科技股份有限公司 混合lidar接收器和lidar方法
US20200120299A1 (en) * 2018-10-10 2020-04-16 Northwestern University Method and system for time-of-flight imaging with high lateral resolution
CN111798503A (zh) * 2019-03-21 2020-10-20 英飞凌科技股份有限公司 利用飞行时间相机同时进行数据传输和深度图像记录
CN111458717A (zh) * 2020-04-20 2020-07-28 深圳奥比中光科技有限公司 一种tof深度测量装置、方法及电子设备
CN111427052A (zh) * 2020-06-09 2020-07-17 深圳市汇顶科技股份有限公司 基于飞行时间的测距方法和相关测距系统

Also Published As

Publication number Publication date
CN116027345A (zh) 2023-04-28

Similar Documents

Publication Publication Date Title
EP3593166B1 (en) Integrated lidar illumination power control
CN110596721B (zh) 双重共享tdc电路的飞行时间距离测量系统及测量方法
CN110596722B (zh) 直方图可调的飞行时间距离测量系统及测量方法
CN110596725B (zh) 基于插值的飞行时间测量方法及测量系统
Niclass et al. A 0.18-$\mu $ m CMOS SoC for a 100-m-Range 10-Frame/s 200$\,\times\, $96-Pixel Time-of-Flight Depth Sensor
US10739444B2 (en) LIDAR signal acquisition
CN110596724B (zh) 动态直方图绘制飞行时间距离测量方法及测量系统
CN110596723B (zh) 动态直方图绘制飞行时间距离测量方法及测量系统
CN113780349B (zh) 训练样本集的获取方法、模型训练方法及相关装置
CN112689776A (zh) 使用彩色图像数据校准深度感测阵列
WO2022042013A1 (zh) 一种dTOF传感模组、终端设备及测距方法
WO2022110947A1 (zh) 电子装置的控制方法、电子装置及计算机可读存储介质
WO2022241942A1 (zh) 一种深度相机及深度计算方法
WO2021058016A1 (zh) 激光雷达及生成激光点云数据的方法
CN115552276A (zh) 模拟用于测试调频连续波(FMCW)光检测和测距(LiDAR)系统的环境的系统
WO2023071307A1 (zh) 一种三维成像的方法和装置
US20240230911A1 (en) Highly parallel large memory histogramming pixel for direct time of flight lidar
US11470261B2 (en) Three-dimensional distance measuring method and device
WO2023061386A1 (zh) 激光雷达、接收系统、发射系统以及控制方法
WO2024113328A1 (zh) 探测方法、阵列探测器、阵列发射器、探测装置及终端
US12094140B2 (en) Apparatus and method of generating depth map
WO2023056848A1 (zh) 一种控制探测方法、控制装置、激光雷达及终端设备
CN117233787B (zh) 点云图像获取方法、装置和激光雷达
CN117043639A (zh) 一种控制方法、激光雷达及终端设备
CN111726548A (zh) 图像传感器的像素和图像传感器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22885211

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE