WO2021212916A1 - Tof depth measurement apparatus and method, and electronic device - Google Patents

Tof depth measurement apparatus and method, and electronic device Download PDF

Info

Publication number
WO2021212916A1
WO2021212916A1 PCT/CN2020/141870 CN2020141870W WO2021212916A1 WO 2021212916 A1 WO2021212916 A1 WO 2021212916A1 CN 2020141870 W CN2020141870 W CN 2020141870W WO 2021212916 A1 WO2021212916 A1 WO 2021212916A1
Authority
WO
WIPO (PCT)
Prior art keywords
target object
light
light beam
line
light source
Prior art date
Application number
PCT/CN2020/141870
Other languages
French (fr)
Chinese (zh)
Inventor
王兆民
许星
Original Assignee
奥比中光科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 奥比中光科技集团股份有限公司 filed Critical 奥比中光科技集团股份有限公司
Publication of WO2021212916A1 publication Critical patent/WO2021212916A1/en

Links

Images

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/02Systems using the reflection of electromagnetic waves other than radio waves
    • G01S17/06Systems determining position data of a target
    • G01S17/08Systems determining position data of a target for measuring distance only
    • G01S17/10Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S17/00Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
    • G01S17/88Lidar systems specially adapted for specific applications
    • G01S17/89Lidar systems specially adapted for specific applications for mapping or imaging
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481Constructional features, e.g. arrangements of optical elements
    • G01S7/4814Constructional features, e.g. arrangements of optical elements of transmitters alone
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/481Constructional features, e.g. arrangements of optical elements
    • G01S7/4817Constructional features, e.g. arrangements of optical elements relating to scanning
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01SRADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483Details of pulse systems
    • G01S7/484Transmitters

Definitions

  • This application relates to the field of three-dimensional imaging technology, and in particular to a TOF depth measurement device, method, and electronic equipment.
  • TOF ranging technology is a technology that achieves precise ranging by measuring the round-trip flight time of light pulses between the transmitting/receiving device and the target object.
  • D-TOF direct-TOF
  • direct TOF ranging technology the technology that directly measures the optical time of flight
  • I-TOF (Indirect-TOF) technology the measurement technique in which the phase delay of the emitted light signal is measured, and the flight time is calculated by the phase delay.
  • Existing TOF measurement devices based on I-TOF technology usually include a launching module and a collection module.
  • the launching module provides flood lighting/lattice illumination to the target space, and the collection module images the reflected light beam.
  • the depth measuring device calculates the phase difference based on the reflected light signal to obtain the distance of the object.
  • the solution disclosed in Chinese Patent Application No. 201911032055.X proposes that the emission module provides floodlighting to the target space. Since floodlighting can illuminate the field of view to the greatest extent, every pixel in the acquisition module can be obtained. Relatively effective light signal and can calculate the depth information of the response. However, the TOF measurement device using floodlight is susceptible to the interference of ambient light and the influence of multipath, resulting in low measurement accuracy.
  • the Chinese patent application No. 201811393403.1 proposes that the launch module provides lattice illumination to the target space.
  • the lattice illumination can be more concentrated through a single point, and the distribution between points is sparse, which can effectively improve the image
  • the signal-to-noise ratio also reduces the impact of multipath.
  • the TOF depth measurement device using dot matrix illumination cannot cover all pixels, so that only some pixels can measure effective depth data, thereby reducing the resolution of the image.
  • the purpose of the present application is to provide a TOF depth measurement device, method and electronic equipment to solve at least one of the above-mentioned background technical problems.
  • the embodiment of the present application provides a TOF depth measuring device, including: a transmitting module for emitting a light beam toward a given area of a target object; wherein the transmitting module includes a light source and a beam scanner, which is scanned by the light beam The device deflects the light beam emitted by the light source to illuminate a given area of the target object; a collection module is used to collect the light beam reflected by the target object; wherein, the collection module includes a pixel array composed of Image sensor; control and processor, respectively connected with the transmitting module and the acquisition module, for controlling the beam scanner to deflect the beam to illuminate a given area of the target object, and according to the deflection The resulting light beam activates the pixels in the corresponding area of the image sensor to respond to the photocharge accumulated by the light beam reflected by the given area, and calculate the phase difference based on the photocharge to obtain the distance of the target object and output The depth image of the target object.
  • a transmitting module for emitting a light beam toward a given
  • the emission module further includes a lens, the light source generates a line beam through the lens, and the line beam is line scanned by the beam scanner to illuminate the target object.
  • the lens is a cylindrical lens
  • the light beam emitted by the light source passes through the cylindrical lens to form a first line beam
  • the first line beam is deflected by the beam scanner for multiple times.
  • a projection pattern composed of a second line of light beams.
  • the transmitting module further includes a collimating lens and a diffractive optical element; wherein the light beam emitted by the light source is collimated by the collimating lens and then emits a collimated light beam, and the collimated light beam passes through The diffractive optical element is diffracted to form a first line string beam or a first lattice beam, and the first line string beam or the first lattice beam is deflected by the beam scanner multiple times to obtain a plurality of second line strings A projection pattern composed of a light beam or a second lattice beam.
  • the second lattice beams are regularly arranged.
  • the example of this application also provides a TOF depth measurement method, which includes the following steps:
  • the emission module emits a light beam toward a given area of the target object; wherein the emission module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate the target object.
  • the collection module includes an image sensor composed of a pixel array
  • the control and processor activates the pixels in the corresponding area of the image sensor according to the deflected beam to respond to the photocharge accumulated by the reflected beam, and calculates the phase difference based on the photocharge to obtain the The distance of the target object.
  • the emission module further includes a cylindrical lens, the light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam is obtained after being deflected by the beam scanner multiple times A projection pattern composed of multiple second line beams.
  • the transmitting module further includes a collimating lens and a diffractive optical element.
  • the light beam emitted by the light source is collimated by the collimating lens and then emits a collimated light beam, and the collimated light beam passes through the collimated light beam.
  • the diffractive optical element diffracts to form a first line string beam or a first lattice beam.
  • the first line string beam or the first lattice beam is deflected by the beam scanner multiple times to obtain a plurality of second line beams or multiple The projection pattern composed of the second lattice beam.
  • control and processor controls the number of deflection, the deflection angle, and the deflection sequence of the beam scanner in each direction to obtain projection patterns with different densities and different field angles.
  • An embodiment of the present application further provides an electronic device, including: a housing, a screen, and a TOF depth measuring device; wherein the TOF depth measuring device includes: a transmitting module for transmitting a light beam toward a given area of a target object; wherein , The transmitting module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate a given area of the target object; The reflected light beam; wherein, the collection module includes an image sensor composed of a pixel array; a control and processor, respectively connected with the emission module and the collection module, for controlling the beam scanning The device deflects the light beam to illuminate a given area of the target object, and activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection, in response to the light accumulated by the light beam reflected back by the given area Charge, calculate the phase difference based on the photocharge to obtain the distance of the target object and output the depth image of the target object; It
  • the embodiment of the application provides a TOF depth measurement device, including: a transmitting module, a collection module, and a control and processor; wherein the transmitting module is used to emit a light beam, which includes a light source and a beam scanner, and is deflected by the beam scanner The light beam emitted by the light source illuminates a given area of the target object; the acquisition module is used to collect the light beam reflected by the target object, and it includes an image sensor composed of a pixel array; the control and processor is used to control the deflection of the beam scanner The light beam illuminates a given area of the target object, and activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection, in response to the accumulated photocharges of the light beam reflected by the given area, The phase difference is calculated based on the photocharge to obtain the distance of the target object and output a depth image of the target object.
  • the TOF depth measurement device of the present application can improve the image signal-to-noise ratio while
  • Fig. 1 is a schematic structural diagram of a TOF depth measuring device according to an embodiment of the present application.
  • FIG. 2 is a schematic diagram of the transmitting module of the TOF depth measuring device in the embodiment of FIG. 1.
  • 3a-3c are schematic diagrams of projection patterns according to an embodiment of the present application.
  • Fig. 4 is a schematic diagram of an image sensor pixel array according to an embodiment of the present application.
  • Fig. 5 is a flowchart of a TOF depth measurement method according to another embodiment of the present application.
  • Fig. 6 is a diagram of an electronic device using the TOF depth measuring device of the embodiment of Fig. 1.
  • connection can be used for fixing or circuit connection.
  • first and second are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with “first” and “second” may explicitly or implicitly include one or more of these features.
  • “plurality” means two or more, unless otherwise specifically defined.
  • FIG. 1 is a schematic structural diagram of a TOF depth measuring device according to an embodiment of the application
  • FIG. 2 is a schematic diagram of a transmitting module of the TOF depth measuring device.
  • the TOF depth measurement device 10 includes a transmitting module 11, an acquisition module 12, and a control and processor 13 connected to the transmitting module 11 and the acquisition module 12, respectively.
  • the transmitting module 11 is used to emit a light beam toward a given area of the target object, and it includes a light source 101 and a beam scanner 102; wherein the light source 101 is used to emit a light beam, and the beam scanner 102 is used to receive the light emitted from the light source.
  • the light beam is deflected and projected to the target object 20;
  • the acquisition module includes an image sensor 121 composed of a pixel array for collecting the light beam 40 reflected by the target object 20;
  • the control and processor 13 controls the light beam
  • the scanner 102 deflects the light beam to illuminate a given area of the target object 20, and activates the pixels in the corresponding area of the image sensor 121 according to the deflected light beam to respond to the accumulated photocharges of the light beam reflected by the given area.
  • the photocharge calculates the phase difference to obtain the distance of the target object 20 and output the depth image of the target object 20.
  • the emission module 11 also includes a light source driver (not shown in the figure), which drives the light source to emit a light beam; wherein the light source can be a light emitting diode (LED), an edge emitting laser (EEL), or a vertical cavity surface emitting laser (VCSEL)
  • the light source can also be a light source array composed of multiple light sources, and the light beam emitted by the light source can be visible light, infrared light, ultraviolet light, etc.
  • the light source 101 emits a light beam
  • the beam scanner 102 receives the light beam and emits the light beam to the target object 20 by rotating along a single axis or multiple axes.
  • the beam scanner 102 may be a Liquid Crystal Polarization Grating (LCPG), a Micro-Electro Mechanical System (MEMS) scanner, or the like.
  • the beam scanner 102 adopts a MEMS scanner. Since MEMS has a very high scanning frequency and a small volume, the transmitting module 11 can have a small volume and high performance.
  • the MEMS scanner can scan at a frequency of 1 MHz to 20 MHz, so it can provide sufficient spatial and temporal resolution.
  • the light beam emitted by the light source 101 can be spatially and temporally modulated to generate a variety of patterned beams, such as regular spot patterns, line beam patterns, and line string beam patterns.
  • the acquisition module 12 includes a TOF image sensor 121, a lens unit, and a filter (not shown in the figure); wherein the lens unit receives and images at least part of the light beam reflected by the target object 20 on at least part of the TOF image sensor,
  • the filter is a narrow-band filter that matches the wavelength of the light source and is used to suppress background light noise in the remaining wavelength bands.
  • TOF image sensor can be an image sensor composed of charge coupled device (CCD), complementary metal oxide semiconductor (CMOS), avalanche diode (AD), single photon avalanche diode (SPAD), etc.
  • the size of the array represents the resolution of the depth camera , Such as 320x240, etc.
  • connected to the image sensor 121 also includes a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure). ).
  • a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure).
  • the TOF image sensor includes at least one pixel, and each pixel includes two or more taps for storing and reading or discharging charge signals generated by incident photons under the control of a corresponding electrode, For example, including 2 taps, the taps are sequentially switched in a certain order within a single frame period (or within a single exposure time) to collect corresponding photons for receiving optical signals and converting them into electrical signals.
  • the control and processor 13 can be an independent dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc. composed of CPU, memory, bus, etc., or a general processing circuit, such as when the depth camera is integrated into When in a smart terminal such as a mobile phone, a TV, a computer, etc., the processing circuit in the terminal can be used as at least a part of the control and processor 13.
  • the control and processor 13 is used to provide the emission signal required when the light source emits laser light, and the light source emits a light beam to the target object 20 under the control of the emission signal.
  • control and processor 13 provides the demodulated signal (collection signal) of each tap in each pixel of the TOF image sensor, and the tap collects the reflected light beam reflected by the target object 20 under the control of the demodulated signal. electric signal.
  • the electrical signal is related to the intensity of the reflected light beam, and the control and processor 13 processes the electrical signal and calculates the phase difference to obtain the distance of the target object 20.
  • the transmitting module 11 includes a lens (not shown), the light source 101 generates a line beam through the lens, and the line beam is line scanned by the beam scanner 102 to illuminate the target object 20.
  • the lens is a cylindrical lens
  • the light source 101 generates a first line beam through the cylindrical lens
  • the beam scanner 102 receives the first line beam and deflects the first line beam to form
  • the second line beam 301 assuming that the angle at which the first line beam is deflected by the beam scanner 102 for the first time is 0 degrees, the second line beam 301 is represented by the solid line in Figure 3a; then the beam scanner 102 performs the first line beam deflection It is deflected again and has a certain deflection angle to form yet another second line beam 301 as indicated by the dashed line in FIG.
  • the projection pattern 30 composed of multiple second line beams formed after multiple deflection has a larger field of view than the first line beam without the beam scanner 102, so that a high signal-to-noise ratio and high resolution can be obtained.
  • the lens may also be another combined lens that can generate a line beam.
  • the transmitting module 11 includes a lens and diffractive optical elements (DOE) (not shown).
  • DOE diffractive optical elements
  • the light source 101 is collimated by the lens to emit a collimated beam, and the collimated beam is diffracted by the DOE to form a collimated beam including A string of light beams formed by a plurality of spots are connected to illuminate the target object 20.
  • the light source 101 is collimated by a lens to emit a collimated light beam, and the collimated light beam is then diffracted by DOE to form a first line string beam composed of multiple spots.
  • the beam scanner 102 receives the first beam.
  • a line string beam and the first line string beam are deflected to form a second line string beam.
  • the formed second line string beam is the line string beam 302 represented by the solid line in FIG. 3b; Subsequently, the beam scanner 102 deflects the first line string beam again, and has a certain deflection angle, and the second line string beam is formed, that is, the line string beam 302 represented by the dashed line in FIG. 3b. It can be understood that all the line-string beams are the same, and they are all the second line-string beams formed by the deflection of the beam scanner 102.
  • the dashed and solid lines are used to separate the beams in the figure for the convenience of description. Therefore, the projection pattern 30 composed of multiple second line string beams formed after multiple deflections has a larger field of view than the first line string beam without the beam scanner 102, thereby obtaining high signal-to-noise ratio and resolution. High image.
  • the light source 101 is collimated by a lens to emit a collimated light beam, and the collimated light beam is then diffracted by DOE to form a lattice pattern composed of multiple spots to illuminate the target object 20.
  • the light source 101 is collimated by the lens to emit a collimated beam, and the collimated beam is diffracted by the DOE to form a first lattice beam.
  • the beam scanner 102 receives the first lattice beam and the first lattice beam. Perform deflection to form a second lattice beam.
  • the formed lattice beam is the second lattice beam formed by the solid circle 303 in Figure 3c; then the beam scanner 102 targets the first point The array beam is deflected again, and has a certain deflection angle, and the formed lattice beam is the second lattice beam formed by the dashed circle 303 in FIG. 3c. It can be understood that all the lattice beams are indistinguishable, and they are all the second lattice beams formed by the deflection of the beam scanner 102.
  • the projection pattern 30 composed of multiple second lattice beams formed after multiple deflection has a higher density and a larger field of view than the first lattice beam without the beam scanner 102, thereby obtaining information.
  • the second lattice beams can be arranged in one dimension or two dimensions, and can be regular or irregular. Optionally, regular arrangements are used to make the depth value distribution more regular.
  • the above-mentioned light source can be a single light source or multiple light sources. If it is a multiple light source, multiple first beams can be formed at one time, and the multiple first beams are deflected by the beam scanner 102 to form multiple light sources. A second light beam, so that the scanning angle is small, and the scanning speed is fast, so that the measurement accuracy can be improved. It is understandable that, according to the needs of the actual use scene, the number of deflection, deflection angle, and deflection sequence of each direction of the beam scanner 102 can be controlled to achieve projections with different densities and different angles of view to obtain high signal-to-noise ratio and resolution. High rate images.
  • the second line beam 301 formed by the first line beam in Fig. 3a deflected by the beam scanner 102 is controlled and processed below.
  • the pixels in the corresponding area of the image sensor 121 are activated in a time-sharing manner for specific description.
  • the light source 101 generates a first line beam through a cylindrical lens, and the beam scanner 102 receives the first line beam and deflects the first line beam to form a plurality of second line beams 302.
  • the control and processor 13 controls the second line beam 301 to irradiate a given area of the target object 20, and activates the pixels in the corresponding area of the image sensor 121 based on the second line beam 301, as shown in FIG. 4.
  • the control and processor 13 activates the pixels in another corresponding area of the image sensor 121 based on the next second line beam 301 , In order to respond to the light beam reflected by another area of the target object 20 and accumulate the photoelectric charge.
  • the power consumption can be greatly reduced, and multiple deflection of the beam scanner 102 can illuminate multiple areas of the target object 20, so that multiple first The projection pattern formed by the two-line beam completely covers the target object.
  • the pixels in each corresponding area of the image sensor 121 can sequentially collect the photocharges accumulated by the light beam reflected back by the target object 20, and control the processor 13 to calculate the phase difference based on the photocharges to obtain the distance of the target object 20, so as to output signal noise Depth image with high ratio and high resolution.
  • FIG. 5 shows the flow of a TOF depth measurement method.
  • the measurement method includes the following steps:
  • S501 Transmit a light beam toward a given area of the target object through a transmitting module; wherein the transmitting module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate the given area of the target object;
  • the transmitting module further includes a lens.
  • the lens is a cylindrical lens.
  • the light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam is formed after multiple deflection by the beam scanner.
  • a projection pattern composed of multiple second-line light speeds, and the projection pattern has a larger field of view than the first-line beam.
  • the transmitting module further includes a collimating lens and a diffractive optical element.
  • the light beam emitted by the light source is collimated by the lens and then emits a collimated beam.
  • the collimated beam is diffracted by the DOE to form a first string beam or a first point.
  • the first linear beam or the first lattice beam is deflected by the beam scanner for multiple times to form a projection pattern composed of multiple second linear beams or multiple second lattice beams, and the projection pattern is larger than the first
  • the line string beams or the first lattice beams have higher density and/or larger field of view, and the second lattice beams are arranged regularly.
  • the pixels in each corresponding area of the image sensor can sequentially collect the light charges accumulated by the light beam reflected by the target object.
  • the control and processor activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection to respond to the photocharge accumulated by the reflected light beam, and calculates the phase difference based on the photocharge to obtain the distance of the target object.
  • control and processor activates the image sensor composed of the pixel array by time sharing to collect the photocharges accumulated by the light beam reflected back by the given area, and calculates the phase difference based on the photocharges collected by the pixels to obtain the target object Distance, and output the depth image of the target object.
  • the control and processor since the projection pattern formed by multiple deflected beams completely covers the target object, a depth image with high signal-to-noise ratio and high resolution can be output.
  • an electronic device may be a desktop, a desktop-mounted device, a portable device, a wearable device or a vehicle-mounted device, a robot, and the like.
  • the device may be a notebook computer or an electronic device to allow gesture recognition or biometric recognition.
  • the device may be a head-mounted device to identify objects or hazards in the user's surrounding environment to ensure safety.
  • a virtual reality system that obstructs the user's vision of the environment can detect objects or hazards in the surrounding environment. To provide users with warnings about nearby objects or obstacles.
  • the electronic device 600 may be a mixed reality system that mixes virtual information and images with the user's surrounding environment, and can detect objects or people in the user's environment to integrate the virtual information with the physical environment and objects.
  • it may also be a device used in fields such as unmanned driving.
  • the electronic device 600 includes a housing 61, a screen 62, and the TOF depth measuring device described in the foregoing embodiment; wherein, the transmitting module of the TOF depth measuring device 11 and the collection module 12 are arranged on the same surface of the electronic device 600, and are used to emit a light beam to the target object and receive the light beam reflected by the target object and form an electrical signal.
  • the embodiments of the present application also provide a storage medium for storing a computer program, which at least executes the method described in any of the foregoing embodiments when the computer program is executed.
  • the storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof.
  • the non-volatile memory can be read-only memory (ROM, Read Only Memory), programmable read-only memory (PROM, Programmable Read-Only Memory), and erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only).
  • Memory Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), Magnetic Random Access Memory (FRAM, Ferromagnetic Random Access Memory), Flash Memory (Flash Memory), Magnetic Surface Memory, Optical Disks, Or CD-ROM (Compact Disc Read-Only Memory); magnetic surface memory can be disk storage or tape storage.
  • the volatile memory may be a random access memory (RAM, Random Access Memory), which is used as an external cache.
  • RAM random access memory
  • SRAM static random access memory
  • SSRAM synchronous static random access memory
  • Synchronous Static Random Access Memory Synchronous Static Random Access Memory
  • DRAM Dynamic Random Access Memory
  • SDRAM Synchronous Dynamic Random Access Memory
  • DDRSDRAM Double Data Rate Synchronous Dynamic Random Access Memory
  • ESDRAM Enhanced Synchronous Dynamic Random Access Memory
  • SLDRAM synchronous connection dynamic random access memory
  • SLDRAM SyncLink Dynamic Random Access Memory
  • DRAM Direct Rambus Random Access Memory
  • the storage media described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memory.

Abstract

A TOF depth measurement apparatus (10) and method, and an electronic device. The apparatus (10) comprises: an emitting module (11), an acquisition module (12), and a control and processor (13). The emitting module (11) is used for emitting a beam, and comprises a light source (101) and a beam scanner (102). A beam emitted by the light source (101) is deflected by the beam scanner (102) to irradiate a given region of a target object (20). The acquisition module (12) is used for acquiring a beam reflected back by the target object (20), and comprises an image sensor (121) composed of a pixel array. The control and processor (13) is used for controlling the beam scanner (102) to deflect the beam to irradiate the given region of the target object (20), activating a pixel in a corresponding region of the image sensor (121) according to the beam formed after deflection so as to respond to a phtocharge accumulated by the beam reflected back by the given region, and calculating a phase difference on the basis of the photocharge to obtain the distance of the target object (20) and output a depth image of the target object (20). The TOF depth measurement apparatus can increase the image resolution and reduce the power consumption while increasing the signal-to-noise ratio of the image.

Description

一种TOF深度测量装置、方法及电子设备A TOF depth measuring device, method and electronic equipment
本申请要求于2020年4月20日提交中国专利局,申请号为202010311680.4,发明名称为“一种TOF深度测量装置、方法及电子设备”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 20, 2020, the application number is 202010311680.4, and the invention title is "a TOF depth measuring device, method and electronic equipment", the entire content of which is incorporated by reference In this application.
技术领域Technical field
本申请涉及三维成像技术领域,尤其涉及一种TOF深度测量装置、方法及电子设备。This application relates to the field of three-dimensional imaging technology, and in particular to a TOF depth measurement device, method, and electronic equipment.
背景技术Background technique
TOF的全称是Time-of-Flight,即飞行时间,TOF测距技术是一种通过测量光脉冲在发射/接收装置和目标物体间的往返飞行时间来实现精确测距的技术。在TOF技术中直接对光飞行时间进行测量的技术被称为D-TOF(direct-TOF),也称为直接TOF测距;而对发射光信号进行周期性调制,通过对反射光信号相对于发射光信号的相位延迟进行测量,再由相位延迟对飞行时间进行计算的测量技术被成为I-TOF(Indirect-TOF)技术,也成为间接TOF测距或者相位式TOF测距。The full name of TOF is Time-of-Flight, that is, time of flight. TOF ranging technology is a technology that achieves precise ranging by measuring the round-trip flight time of light pulses between the transmitting/receiving device and the target object. In TOF technology, the technology that directly measures the optical time of flight is called D-TOF (direct-TOF), also known as direct TOF ranging; while the emitted light signal is periodically modulated, and the reflected light signal is compared with The measurement technique in which the phase delay of the emitted light signal is measured, and the flight time is calculated by the phase delay is called I-TOF (Indirect-TOF) technology, which is also called indirect TOF ranging or phase TOF ranging.
现有的基于I-TOF技术的TOF测量装置通常包含一个发射模组以及采集模组,发射模组向目标空间提供泛光照明/点阵照明,采集模组则对反射回的光束进行成像,深度测量装置基于反射光信号计算相位差以获取物体的距离。Existing TOF measurement devices based on I-TOF technology usually include a launching module and a collection module. The launching module provides flood lighting/lattice illumination to the target space, and the collection module images the reflected light beam. The depth measuring device calculates the phase difference based on the reflected light signal to obtain the distance of the object.
中国专利申请第201911032055.X号公开的方案中提出了发射模组向目标空间提供泛光照明,由于泛光照明可以最大程度上对视场进行照明,采集模组中的每个像素都可以获取相对有效的光信号并可以计算出响应的深度信息。但是采用泛光照明的TOF测量装置由于容易受到环境光的干扰以及多路径的影响, 从而导致测量精度低。The solution disclosed in Chinese Patent Application No. 201911032055.X proposes that the emission module provides floodlighting to the target space. Since floodlighting can illuminate the field of view to the greatest extent, every pixel in the acquisition module can be obtained. Relatively effective light signal and can calculate the depth information of the response. However, the TOF measurement device using floodlight is susceptible to the interference of ambient light and the influence of multipath, resulting in low measurement accuracy.
而中国专利申请第201811393403.1号方案中提出了发射模组向目标空间提供点阵照明,点阵照明可以通过单点的能量更加集中,且点与点之间分布较稀疏,这样可以有效提高图像的信噪比的同时减小多路径的影响。但是采用点阵照明的TOF深度测量装置由于点阵无法全部覆盖所有像素,导致仅部分像素可以测量到有效的深度数据,从而降低图像的分辨率。The Chinese patent application No. 201811393403.1 proposes that the launch module provides lattice illumination to the target space. The lattice illumination can be more concentrated through a single point, and the distribution between points is sparse, which can effectively improve the image The signal-to-noise ratio also reduces the impact of multipath. However, the TOF depth measurement device using dot matrix illumination cannot cover all pixels, so that only some pixels can measure effective depth data, thereby reducing the resolution of the image.
发明内容Summary of the invention
本申请的目的在于提供一种TOF深度测量装置、方法及电子设备,以解决上述背景技术问题中的至少一种问题。The purpose of the present application is to provide a TOF depth measurement device, method and electronic equipment to solve at least one of the above-mentioned background technical problems.
本申请实施例提供一种TOF深度测量装置,包括:发射模组,用于朝向目标物体的给定区域发射光束;其中,所述发射模组包括有光源以及光束扫描器,通过所述光束扫描器偏转所述光源发出的光束以照射所述目标物体的给定区域;采集模组,用于采集经所述目标物体反射回的光束;其中,所述采集模组包括有由像素阵列组成的图像传感器;控制与处理器,分别与所述发射模组和所述采集模组连接,以用于控制所述光束扫描器偏转所述光束以照射所述目标物体的给定区域,并根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应由所述给定区域反射回的光束而累积的光电荷,基于该光电荷计算相位差以获得所述目标物体的距离并输出所述目标物体的深度图像。The embodiment of the present application provides a TOF depth measuring device, including: a transmitting module for emitting a light beam toward a given area of a target object; wherein the transmitting module includes a light source and a beam scanner, which is scanned by the light beam The device deflects the light beam emitted by the light source to illuminate a given area of the target object; a collection module is used to collect the light beam reflected by the target object; wherein, the collection module includes a pixel array composed of Image sensor; control and processor, respectively connected with the transmitting module and the acquisition module, for controlling the beam scanner to deflect the beam to illuminate a given area of the target object, and according to the deflection The resulting light beam activates the pixels in the corresponding area of the image sensor to respond to the photocharge accumulated by the light beam reflected by the given area, and calculate the phase difference based on the photocharge to obtain the distance of the target object and output The depth image of the target object.
在一些实施例中,所述发射模组还包括有透镜,所述光源通过所述透镜产生线光束,所述线光束通过所述光束扫描器进行线扫描以照射目标物体。In some embodiments, the emission module further includes a lens, the light source generates a line beam through the lens, and the line beam is line scanned by the beam scanner to illuminate the target object.
在一些实施例中,所述透镜为柱透镜,所述光源发出的光束经过所述柱透镜后形成第一线光束,所述第一线光束经过所述光束扫描器多次偏转后得到由多条第二线光束组成的投影图案。In some embodiments, the lens is a cylindrical lens, the light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam is deflected by the beam scanner for multiple times. A projection pattern composed of a second line of light beams.
在一些实施例中,所述发射模组还包括有准直透镜和衍射光学元件;其中,所述光源发出的光束经过所述准直透镜准直后发出准直光束,所述准直光束通 过所述衍射光学元件衍射后形成第一线串光束或第一点阵光束,所述第一线串光束或第一点阵光束经过所述光束扫描器多次偏转后得到由多条第二线串光束或第二点阵光束组成的投影图案。In some embodiments, the transmitting module further includes a collimating lens and a diffractive optical element; wherein the light beam emitted by the light source is collimated by the collimating lens and then emits a collimated light beam, and the collimated light beam passes through The diffractive optical element is diffracted to form a first line string beam or a first lattice beam, and the first line string beam or the first lattice beam is deflected by the beam scanner multiple times to obtain a plurality of second line strings A projection pattern composed of a light beam or a second lattice beam.
在一些实施例中,所述第二点阵光束为规则排列。In some embodiments, the second lattice beams are regularly arranged.
本申请实例还提供一种TOF深度测量方法,包括如下步骤:The example of this application also provides a TOF depth measurement method, which includes the following steps:
通过发射模组朝向目标物体的给定区域发射光束;其中,所述发射模组包括有光源以及光束扫描器,通过所述光束扫描器偏转所述光源发出的光束以照射所述目标物体的给定区域;The emission module emits a light beam toward a given area of the target object; wherein the emission module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate the target object. Fixed area
通过采集模组采集经所述目标物体反射回的光束;其中,所述采集模组包括由像素阵列组成的图像传感器;Collecting the light beam reflected by the target object through a collection module; wherein, the collection module includes an image sensor composed of a pixel array;
控制与处理器根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应于由所述反射回的光束而累积的光电荷,并基于所述光电荷计算相位差以获取所述目标物体的距离。The control and processor activates the pixels in the corresponding area of the image sensor according to the deflected beam to respond to the photocharge accumulated by the reflected beam, and calculates the phase difference based on the photocharge to obtain the The distance of the target object.
在一些实施例中,所述发射模组还包括有柱透镜,所述光源发出光束经过所述柱透镜形成第一线光束,所述第一线光束经过所述光束扫描器多次偏转后得到由多条第二线光束组成的投影图案。In some embodiments, the emission module further includes a cylindrical lens, the light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam is obtained after being deflected by the beam scanner multiple times A projection pattern composed of multiple second line beams.
在一些实施例中,所述发射模组还包括有准直透镜以及衍射光学元件,所述光源发出的光束经过所述准直透镜准直后发出准直光束,所述准直光束通过所述衍射光学元件衍射形成第一线串光束或第一点阵光束,所述第一线串光束或第一点阵光束经过所述光束扫描器多次偏转后得到由多条第二线光束或多个第二点阵光束组成的投影图案。In some embodiments, the transmitting module further includes a collimating lens and a diffractive optical element. The light beam emitted by the light source is collimated by the collimating lens and then emits a collimated light beam, and the collimated light beam passes through the collimated light beam. The diffractive optical element diffracts to form a first line string beam or a first lattice beam. The first line string beam or the first lattice beam is deflected by the beam scanner multiple times to obtain a plurality of second line beams or multiple The projection pattern composed of the second lattice beam.
在一些实施例中,所述控制与处理器控制所述光束扫描器每个方向的偏转次数、偏转角度以及偏转顺序以获得不同密度和不同视场角的投影图案。In some embodiments, the control and processor controls the number of deflection, the deflection angle, and the deflection sequence of the beam scanner in each direction to obtain projection patterns with different densities and different field angles.
本申请实施例还提供一种电子设备,包括:壳体、屏幕以及TOF深度测量装置;其中,所述TOF深度测量装置包括:发射模组,用于朝向目标物体的给定区域发射光束;其中,所述发射模组包括有光源以及光束扫描器,通过所述 光束扫描器偏转所述光源发出的光束以照射所述目标物体的给定区域;采集模组,用于采集经所述目标物体反射回的光束;其中,所述采集模组包括有由像素阵列组成的图像传感器;控制与处理器,分别与所述发射模组和所述采集模组连接,以用于控制所述光束扫描器偏转所述光束以照射所述目标物体的给定区域,并根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应由所述给定区域反射回的光束而累积的光电荷,基于该光电荷计算相位差以获得所述目标物体的距离并输出所述目标物体的深度图像;所述TOF深度测量装置的发射模组与采集模组设置于电子设备的同一面,以用于向目标物体发射光束以及接收目标物体反射回来的光束并形成电信号。An embodiment of the present application further provides an electronic device, including: a housing, a screen, and a TOF depth measuring device; wherein the TOF depth measuring device includes: a transmitting module for transmitting a light beam toward a given area of a target object; wherein , The transmitting module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate a given area of the target object; The reflected light beam; wherein, the collection module includes an image sensor composed of a pixel array; a control and processor, respectively connected with the emission module and the collection module, for controlling the beam scanning The device deflects the light beam to illuminate a given area of the target object, and activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection, in response to the light accumulated by the light beam reflected back by the given area Charge, calculate the phase difference based on the photocharge to obtain the distance of the target object and output the depth image of the target object; It is used to emit light beams to the target object and receive the light beam reflected by the target object and form an electrical signal.
本申请实施例提供一种TOF深度测量装置,包括:发射模组、采集模组以及控制与处理器;其中发射模组用于发射光束,其包括有光源以及光束扫描器,通过光束扫描器偏转光源发出的光束以照射目标物体的给定区域;采集模组用于采集经目标物体反射回的光束,其包括有由像素阵列组成的图像传感器;控制与处理器用于控制所述光束扫描器偏转所述光束以照射所述目标物体的给定区域,并根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应由所述给定区域反射回的光束而累积的光电荷,基于该光电荷计算相位差以获得所述目标物体的距离并输出所述目标物体的深度图像。本申请TOF深度测量装置在提升图像信噪比的同时可以提高图像的分辨率且降低功耗。The embodiment of the application provides a TOF depth measurement device, including: a transmitting module, a collection module, and a control and processor; wherein the transmitting module is used to emit a light beam, which includes a light source and a beam scanner, and is deflected by the beam scanner The light beam emitted by the light source illuminates a given area of the target object; the acquisition module is used to collect the light beam reflected by the target object, and it includes an image sensor composed of a pixel array; the control and processor is used to control the deflection of the beam scanner The light beam illuminates a given area of the target object, and activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection, in response to the accumulated photocharges of the light beam reflected by the given area, The phase difference is calculated based on the photocharge to obtain the distance of the target object and output a depth image of the target object. The TOF depth measurement device of the present application can improve the image signal-to-noise ratio while increasing the resolution of the image and reducing power consumption.
附图说明Description of the drawings
为了更清楚地说明本申请实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动性的前提下,还可以根据这些附图获得其他的附图。In order to more clearly describe the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.
图1是根据本申请一个实施例TOF深度测量装置的结构示意图。Fig. 1 is a schematic structural diagram of a TOF depth measuring device according to an embodiment of the present application.
图2是图1实施例TOF深度测量装置的发射模组的示意图。FIG. 2 is a schematic diagram of the transmitting module of the TOF depth measuring device in the embodiment of FIG. 1.
图3a-3c是根据本申请一个实施例的投影图案的示意图。3a-3c are schematic diagrams of projection patterns according to an embodiment of the present application.
图4是根据本申请一个实施例的图像传感器像素阵列的示意图。Fig. 4 is a schematic diagram of an image sensor pixel array according to an embodiment of the present application.
图5是根据本申请另一个实施例TOF深度测量方法的流程图。Fig. 5 is a flowchart of a TOF depth measurement method according to another embodiment of the present application.
图6是采用图1实施例TOF深度测量装置的电子设备的图示。Fig. 6 is a diagram of an electronic device using the TOF depth measuring device of the embodiment of Fig. 1.
具体实施方式Detailed ways
为了使本申请实施例所要解决的技术问题、技术方案及有益效果更加清楚明白,以下结合附图及实施例,对本申请进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本申请,并不用于限定本申请。In order to make the technical problems, technical solutions, and beneficial effects to be solved by the embodiments of the present application clearer, the following further describes the present application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not used to limit the present application.
需要说明的是,当元件被称为“固定于”或“设置于”另一个元件,它可以直接在另一个元件上或者间接在该另一个元件上。当一个元件被称为是“连接于”另一个元件,它可以是直接连接到另一个元件或间接连接至该另一个元件上。另外,连接即可以是用于固定作用也可以是用于电路连通作用。It should be noted that when an element is referred to as being "fixed to" or "disposed on" another element, it can be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it can be directly connected to the other element or indirectly connected to the other element. In addition, the connection can be used for fixing or circuit connection.
需要理解的是,术语“长度”、“宽度”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”“内”、“外”等指示的方位或位置关系为基于附图所示的方位或位置关系,仅是为了便于描述本申请实施例和简化描述,而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作,因此不能理解为对本申请的限制。It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top" The orientation or positional relationship indicated by "bottom", "inner", "outer", etc. are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the embodiments of the application and simplifying the description, rather than indicating or implying what is meant. The device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present application.
此外,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多该特征。在本申请实施例的描述中,“多个”的含义是两个或两个以上,除非另有明确具体的限定。In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the embodiments of the present application, "plurality" means two or more, unless otherwise specifically defined.
参照图1、图2所示,图1为本申请一个实施例TOF深度测量装置的结构示意图,图2为TOF深的测量装置的发射模组的示意图。TOF深度测量装置10包括发射模组11、采集模组12以及分别与发射模组11和采集模组12连接的控制与处理器13。如图2所示,发射模组11用于朝向目标物体的给定区域发射光 束,其包括光源101、光束扫描器102;其中,光源101用于发出光束,光束扫描器102用于接收光源发出的光束并对该光束进行偏转后投射到目标物体20;采集模组包括由像素阵列组成的图像传感器121,以用于采集经目标物体20反射的光束40;控制与处理器13控制所述光束扫描器102偏转光束以照射目标物体20的给定区域,并根据该偏转光束激活所述图像传感器121中相应区域的像素,以响应由给定区域反射回的光束而累积的光电荷,基于该光电荷计算相位差以获取所述目标物体20的距离并输出目标物体20的深度图像。Referring to FIGS. 1 and 2, FIG. 1 is a schematic structural diagram of a TOF depth measuring device according to an embodiment of the application, and FIG. 2 is a schematic diagram of a transmitting module of the TOF depth measuring device. The TOF depth measurement device 10 includes a transmitting module 11, an acquisition module 12, and a control and processor 13 connected to the transmitting module 11 and the acquisition module 12, respectively. As shown in FIG. 2, the transmitting module 11 is used to emit a light beam toward a given area of the target object, and it includes a light source 101 and a beam scanner 102; wherein the light source 101 is used to emit a light beam, and the beam scanner 102 is used to receive the light emitted from the light source. The light beam is deflected and projected to the target object 20; the acquisition module includes an image sensor 121 composed of a pixel array for collecting the light beam 40 reflected by the target object 20; the control and processor 13 controls the light beam The scanner 102 deflects the light beam to illuminate a given area of the target object 20, and activates the pixels in the corresponding area of the image sensor 121 according to the deflected light beam to respond to the accumulated photocharges of the light beam reflected by the given area. The photocharge calculates the phase difference to obtain the distance of the target object 20 and output the depth image of the target object 20.
发射模组11还包括有光源驱动器(图中未示出),光源驱动器驱动光源发出光束;其中,光源可以是发光二极管(LED)、边发射激光器(EEL)、垂直腔面发射激光器(VCSEL)等光源,也可以是多个光源组成的光源阵列,光源所发射的光束可以是可见光、红外光、紫外光等。The emission module 11 also includes a light source driver (not shown in the figure), which drives the light source to emit a light beam; wherein the light source can be a light emitting diode (LED), an edge emitting laser (EEL), or a vertical cavity surface emitting laser (VCSEL) The light source can also be a light source array composed of multiple light sources, and the light beam emitted by the light source can be visible light, infrared light, ultraviolet light, etc.
光源101发出光束,光束扫描器102接收该光束并通过沿单轴或多轴旋转以将光束发射到目标物体20。在一些实施例中,光束扫描器102可以是液晶偏振光栅(Liquid Crystal Polarization Grating,LCPG)、微机电系统(Micro-Electro Mechanical System,MEMS)扫描器等。可选地,光束扫描器102采用MEMS扫描器,由于MEMS具有极高的扫描频率以及较小的体积,可以使得发射模组11具有较小的体积及较高的性能。在一些实施例中,MEMS扫描器可以以1MHz~20MHz的频率进行扫描,因此可以提供足够的空间及时间分辨率。通过光源驱动器以及光束扫描器102的配置,可以对光源101发出的光束进行空间以及时间调制以产生多种图案光束射出,比如规则斑点图案、线光束图案、线串光束图案等。The light source 101 emits a light beam, and the beam scanner 102 receives the light beam and emits the light beam to the target object 20 by rotating along a single axis or multiple axes. In some embodiments, the beam scanner 102 may be a Liquid Crystal Polarization Grating (LCPG), a Micro-Electro Mechanical System (MEMS) scanner, or the like. Optionally, the beam scanner 102 adopts a MEMS scanner. Since MEMS has a very high scanning frequency and a small volume, the transmitting module 11 can have a small volume and high performance. In some embodiments, the MEMS scanner can scan at a frequency of 1 MHz to 20 MHz, so it can provide sufficient spatial and temporal resolution. Through the configuration of the light source driver and the beam scanner 102, the light beam emitted by the light source 101 can be spatially and temporally modulated to generate a variety of patterned beams, such as regular spot patterns, line beam patterns, and line string beam patterns.
采集模组12包括TOF图像传感器121、透镜单元、以及滤光片(图中未示出);其中,透镜单元接收并将由目标物体20反射回的至少部分光束成像在至少部分TOF图像传感器上,滤光片为与光源波长相匹配的窄带滤光片,用于抑制其余波段的背景光噪声。TOF图像传感器可以是电荷耦合元件(CCD)、互补金属氧化物半导体(CMOS)、雪崩二极管(AD)、单光子雪崩二极管(SPAD)等组成的图 像传感器,阵列大小代表着该深度相机的分辨率,比如320x240等。一般地,与图像传感器121连接的还包括由信号放大器、时数转换器(TDC)、模数转换器(ADC)等器件中的一种或多种组成的读出电路(图中未示出)。The acquisition module 12 includes a TOF image sensor 121, a lens unit, and a filter (not shown in the figure); wherein the lens unit receives and images at least part of the light beam reflected by the target object 20 on at least part of the TOF image sensor, The filter is a narrow-band filter that matches the wavelength of the light source and is used to suppress background light noise in the remaining wavelength bands. TOF image sensor can be an image sensor composed of charge coupled device (CCD), complementary metal oxide semiconductor (CMOS), avalanche diode (AD), single photon avalanche diode (SPAD), etc. The size of the array represents the resolution of the depth camera , Such as 320x240, etc. Generally, connected to the image sensor 121 also includes a readout circuit composed of one or more of a signal amplifier, a time-to-digital converter (TDC), an analog-to-digital converter (ADC) and other devices (not shown in the figure). ).
在一些实施例中,TOF图像传感器包括至少一个像素,每个像素包含两个以上的抽头(tap),以用于在相应电极的控制下存储并读取或者排出由入射光子产生的电荷信号,比如:包括2个抽头,在单个帧周期(或单次曝光时间内)内以一定的次序依次切换抽头以采集相应的光子,以用于接收光信号并转换成电信号。In some embodiments, the TOF image sensor includes at least one pixel, and each pixel includes two or more taps for storing and reading or discharging charge signals generated by incident photons under the control of a corresponding electrode, For example, including 2 taps, the taps are sequentially switched in a certain order within a single frame period (or within a single exposure time) to collect corresponding photons for receiving optical signals and converting them into electrical signals.
控制与处理器13可以是独立的专用电路,比如包含CPU、存储器、总线等组成的专用SOC芯片、FPGA芯片、ASIC芯片等等,也可以包含通用处理电路,比如当该深度相机被集成到如手机、电视、电脑等智能终端中时,终端中的处理电路可以作为该控制与处理器13的至少一部分。The control and processor 13 can be an independent dedicated circuit, such as a dedicated SOC chip, FPGA chip, ASIC chip, etc. composed of CPU, memory, bus, etc., or a general processing circuit, such as when the depth camera is integrated into When in a smart terminal such as a mobile phone, a TV, a computer, etc., the processing circuit in the terminal can be used as at least a part of the control and processor 13.
控制与处理器13用于提供光源发射激光时所需的发射信号,光源在发射信号的控制下向目标物体20发射光束。The control and processor 13 is used to provide the emission signal required when the light source emits laser light, and the light source emits a light beam to the target object 20 under the control of the emission signal.
在一些实施例中,控制与处理器13提供TOF图像传感器各像素中各抽头的解调信号(采集信号),抽头在解调信号的控制下采集由目标物体20反射回的反射光束所产生的电信号。该电信号与反射光束的强度相关,控制与处理器13对该电信号进行处理并计算出相位差以获得目标物体20的距离。In some embodiments, the control and processor 13 provides the demodulated signal (collection signal) of each tap in each pixel of the TOF image sensor, and the tap collects the reflected light beam reflected by the target object 20 under the control of the demodulated signal. electric signal. The electrical signal is related to the intensity of the reflected light beam, and the control and processor 13 processes the electrical signal and calculates the phase difference to obtain the distance of the target object 20.
在一些实施例中,发射模组11包括透镜(未图示),光源101通过透镜产生线光束,线光束通过光束扫描器102进行线扫描以照射目标物体20。如图3a所示,在本申请实施例中,所述透镜为柱透镜,光源101通过柱透镜产生第一线光束,光束扫描器102接收第一线光束并对第一线光束进行偏转以形成第二线光束301,假设光束扫描器102对第一线光束进行第一次偏转的角度为0度,形成第二线光束301如图3a中实线表示;随后光束扫描器102对第一线光束进行再次偏转,并且具有一定的偏转角度,形成又一个第二线光束301如图3a中虚线表示。可以理解的是,所有第二线光束并无区别,都是经过光束扫描器102偏转形成的第二线光束。由此经过多次偏转之后形成的多个第二线光束组成的 投影图案30相对于没有光束扫描器102时的第一线光束具有更大的视场,从而可以获得信噪比高且分辨率高的图像。当然可以理解的是,所述透镜也可以是其他可产生线光束的组合型透镜。In some embodiments, the transmitting module 11 includes a lens (not shown), the light source 101 generates a line beam through the lens, and the line beam is line scanned by the beam scanner 102 to illuminate the target object 20. As shown in FIG. 3a, in the embodiment of the present application, the lens is a cylindrical lens, the light source 101 generates a first line beam through the cylindrical lens, and the beam scanner 102 receives the first line beam and deflects the first line beam to form The second line beam 301, assuming that the angle at which the first line beam is deflected by the beam scanner 102 for the first time is 0 degrees, the second line beam 301 is represented by the solid line in Figure 3a; then the beam scanner 102 performs the first line beam deflection It is deflected again and has a certain deflection angle to form yet another second line beam 301 as indicated by the dashed line in FIG. 3a. It is understandable that all the second line beams are indistinguishable, and they are all the second line beams formed by the deflection of the beam scanner 102. Therefore, the projection pattern 30 composed of multiple second line beams formed after multiple deflection has a larger field of view than the first line beam without the beam scanner 102, so that a high signal-to-noise ratio and high resolution can be obtained. Image. Of course, it can be understood that the lens may also be another combined lens that can generate a line beam.
在一些实施例中,发射模组11包括透镜和衍射光学元件(Diffractive Optical Elements,DOE)(未图示),光源101通过透镜准直后发出准直光束,准直光束通过DOE衍射后形成包括有多个斑点连成的线串光束以照射目标物体20。具体的,如图3b所示,光源101通过透镜准直后发出准直光束,准直光束再通过DOE衍射后形成包括有多个斑点连成的第一线串光束,光束扫描器102接收第一线串光束并对第一线串光束进行偏转以形成第二线串光束,假定第一次偏转角度为0度,所形成的第二线串光束即图3b中实线表示的线串光束302;随后光束扫描器102对第一线串光束进行再次偏转,并且具有一定的偏转角度,所形成得第二线串光束即图3b中虚线表示的线串光束302。可以理解的是,所有线串光束并无区别,且都是经过光束扫描器102偏转形成的第二线串光束,图中用虚实线分开是为了方便描述说明。由此经过多次偏转之后形成的多个第二线串光束组成的投影图案30相对于没有光束扫描器102时的第一线串光束具有更大的视场,从而获取信噪比高且分辨率高的图像。In some embodiments, the transmitting module 11 includes a lens and diffractive optical elements (DOE) (not shown). The light source 101 is collimated by the lens to emit a collimated beam, and the collimated beam is diffracted by the DOE to form a collimated beam including A string of light beams formed by a plurality of spots are connected to illuminate the target object 20. Specifically, as shown in FIG. 3b, the light source 101 is collimated by a lens to emit a collimated light beam, and the collimated light beam is then diffracted by DOE to form a first line string beam composed of multiple spots. The beam scanner 102 receives the first beam. A line string beam and the first line string beam are deflected to form a second line string beam. Assuming that the first deflection angle is 0 degrees, the formed second line string beam is the line string beam 302 represented by the solid line in FIG. 3b; Subsequently, the beam scanner 102 deflects the first line string beam again, and has a certain deflection angle, and the second line string beam is formed, that is, the line string beam 302 represented by the dashed line in FIG. 3b. It can be understood that all the line-string beams are the same, and they are all the second line-string beams formed by the deflection of the beam scanner 102. The dashed and solid lines are used to separate the beams in the figure for the convenience of description. Therefore, the projection pattern 30 composed of multiple second line string beams formed after multiple deflections has a larger field of view than the first line string beam without the beam scanner 102, thereby obtaining high signal-to-noise ratio and resolution. High image.
在一些实施例中,光源101通过透镜准直后发出准直光束,准直光束再通过DOE衍射后形成包括多个斑点组成的点阵图案以照射目标物体20。如图3c所示,光源101通过透镜准直后发出准直光束,准直光束再通过DOE衍射后形成包括第一点阵光束,光束扫描器102接收第一点阵光束对第一点阵光束进行偏转以形成第二点阵光束,假定第一次偏转角度为0度,所形成的点阵光束即图3c中实心圆圈303组成的第二点阵光束;随后光束扫描器102对第一点阵光束进行再次偏转,并且具有一定的偏转角度,所形成的点阵光束即图3c中虚线圆圈303组成的第二点阵光束。可以理解的是,所有的点阵光束并无区别,且都是光束扫描器102偏转形成的第二点阵光束。由此经过多次偏转之后形成的多个第二点阵光束组成的投影图案30相对于没有光束扫描器102时的第一点阵光 束具有更高的密度和更大的视场,从而获取信噪比高且分辨率高的图像。可以理解的是,第二点阵光束可以是一维排列,也可以是二维排列,可以是规则排列也可以是不规则排列,可选地,采用规则排列,从而使得深度值分布更规整。In some embodiments, the light source 101 is collimated by a lens to emit a collimated light beam, and the collimated light beam is then diffracted by DOE to form a lattice pattern composed of multiple spots to illuminate the target object 20. As shown in Fig. 3c, the light source 101 is collimated by the lens to emit a collimated beam, and the collimated beam is diffracted by the DOE to form a first lattice beam. The beam scanner 102 receives the first lattice beam and the first lattice beam. Perform deflection to form a second lattice beam. Assuming that the first deflection angle is 0 degrees, the formed lattice beam is the second lattice beam formed by the solid circle 303 in Figure 3c; then the beam scanner 102 targets the first point The array beam is deflected again, and has a certain deflection angle, and the formed lattice beam is the second lattice beam formed by the dashed circle 303 in FIG. 3c. It can be understood that all the lattice beams are indistinguishable, and they are all the second lattice beams formed by the deflection of the beam scanner 102. Therefore, the projection pattern 30 composed of multiple second lattice beams formed after multiple deflection has a higher density and a larger field of view than the first lattice beam without the beam scanner 102, thereby obtaining information. An image with a high noise ratio and high resolution. It can be understood that the second lattice beams can be arranged in one dimension or two dimensions, and can be regular or irregular. Optionally, regular arrangements are used to make the depth value distribution more regular.
如图3a-图3c所示,上述光源可以是单光源也可以是多光源,若为多光源,则一次可以形成多个第一光束,多个第一光束通过光束扫描器102偏转后形成多个第二光束,如此使得扫描的角度较小,且扫描速度快,从而可以提高测量精度。可以理解的是,根据实际使用场景的需求,可以控制光束扫描器102每个方向的偏转次数、偏转角度以及偏转顺序从而实现不同密度以及不同视场角的投影,以获取信噪比高且分辨率高的图像。As shown in Figures 3a-3c, the above-mentioned light source can be a single light source or multiple light sources. If it is a multiple light source, multiple first beams can be formed at one time, and the multiple first beams are deflected by the beam scanner 102 to form multiple light sources. A second light beam, so that the scanning angle is small, and the scanning speed is fast, so that the measurement accuracy can be improved. It is understandable that, according to the needs of the actual use scene, the number of deflection, deflection angle, and deflection sequence of each direction of the beam scanner 102 can be controlled to achieve projections with different densities and different angles of view to obtain high signal-to-noise ratio and resolution. High rate images.
根据上述图3a-图3c方案记载的发射模组11向目标物体20投射的投影图案30,下面以控制与处理器基于图3a中第一线光束经光束扫描器102偏转形成的第二线光束301,分时激活图像传感器121相应区域中的像素进行具体说明。According to the projection pattern 30 projected by the emission module 11 to the target object 20 according to the above-mentioned schemes in Figs. 3a-3c, the second line beam 301 formed by the first line beam in Fig. 3a deflected by the beam scanner 102 is controlled and processed below. , The pixels in the corresponding area of the image sensor 121 are activated in a time-sharing manner for specific description.
光源101通过柱透镜产生第一线光束,光束扫描器102接收第一线光束并对第一线光束进行偏转以形成多个第二线光束302。控制与处理器13控制第二线光束301照射到目标物体20的给定区域,并基于第二线光束301激活图像传感器121相应区域中的像素,如图4所示。假定当光束扫描器102经过一次偏转形成一个偏转光束301照射到目标物体20的给定区域,控制与处理器13基于偏转光束301激活图像传感器121中相应区域401的像素,以响应于由给定区域反射回的光束而累积的光电荷。图4中显示每一个第二线光束大致占13x2=26个像素,实际上可以是其它大小,在本申请实施例中不作特别限定。The light source 101 generates a first line beam through a cylindrical lens, and the beam scanner 102 receives the first line beam and deflects the first line beam to form a plurality of second line beams 302. The control and processor 13 controls the second line beam 301 to irradiate a given area of the target object 20, and activates the pixels in the corresponding area of the image sensor 121 based on the second line beam 301, as shown in FIG. 4. Assuming that when the beam scanner 102 undergoes a deflection to form a deflected beam 301 to irradiate a given area of the target object 20, the control and processor 13 activates the pixels in the corresponding area 401 of the image sensor 121 based on the deflected beam 301 to respond to the given area The accumulated photocharges of the beam reflected by the area. It is shown in FIG. 4 that each second line beam roughly occupies 13×2=26 pixels, which can actually be of other sizes, which is not particularly limited in the embodiment of the present application.
同样地,当经过光束扫描器102偏转形成下一个第二线光束301照射到目标物体20的另一个区域,控制与处理器13基于下一个第二线光束301激活图像传感器121中另一个相应区域的像素,以响应于由目标物体20的另一个区域反射回的光束而累积的光电荷。Similarly, when the next second line beam 301 is deflected by the beam scanner 102 to irradiate another area of the target object 20, the control and processor 13 activates the pixels in another corresponding area of the image sensor 121 based on the next second line beam 301 , In order to respond to the light beam reflected by another area of the target object 20 and accumulate the photoelectric charge.
可以理解的是,通过分时激活图像传感器121中相应区域的像素,可以大大减小功耗,且通过光束扫描器102的多次偏转可以照射目标物体20的多个区 域,以使得多个第二线光束形成的投影图案完整覆盖目标物体。图像传感器121各相应区域的像素可以依次采集到由目标物体20反射回的光束而累积的光电荷,控制与处理器13基于光电荷计算相位差以获取目标物体20的距离,从而可以输出信噪比高且分辨率高的深度图像。It can be understood that by activating the pixels in the corresponding area of the image sensor 121 in time sharing, the power consumption can be greatly reduced, and multiple deflection of the beam scanner 102 can illuminate multiple areas of the target object 20, so that multiple first The projection pattern formed by the two-line beam completely covers the target object. The pixels in each corresponding area of the image sensor 121 can sequentially collect the photocharges accumulated by the light beam reflected back by the target object 20, and control the processor 13 to calculate the phase difference based on the photocharges to obtain the distance of the target object 20, so as to output signal noise Depth image with high ratio and high resolution.
基于上述各实施例中的TOF深度测量装置,本申请另一实施例还提供一种TOF深度测量方法。参照图5所试,图5示出了一种TOF深度测量方法的流程,测量方法包括如下步骤:Based on the TOF depth measurement device in the foregoing embodiments, another embodiment of the present application also provides a TOF depth measurement method. Refer to Figure 5, which shows the flow of a TOF depth measurement method. The measurement method includes the following steps:
S501、通过发射模组朝向目标物体的给定区域发射光束;其中,发射模组包括有光源以及光束扫描器,通过光束扫描器偏转光源发出的光束以照射目标物体的给定区域;S501: Transmit a light beam toward a given area of the target object through a transmitting module; wherein the transmitting module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate the given area of the target object;
具体的,发射模组还包括有透镜,在本申请实施例中,所述透镜为柱透镜,光源发出光束经过柱透镜形成第一线光束,第一线光束经过光束扫描器多次偏转后形成多条第二线光速组成的投影图案,该投影图案比第一线光束拥有更大的视场角。Specifically, the transmitting module further includes a lens. In the embodiment of the present application, the lens is a cylindrical lens. The light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam is formed after multiple deflection by the beam scanner. A projection pattern composed of multiple second-line light speeds, and the projection pattern has a larger field of view than the first-line beam.
在一些实施例中,发射模组还包括有准直透镜以及衍射光学元件,光源发出的光束经过透镜准直后发出准直光束,准直光束通过DOE衍射形成第一线串光束或第一点阵光束,第一线串光束或第一点阵光束经过光束扫描器多次偏转后形成多条第二线光束或多个第二点阵光束组成的投影图案,所述投影图案比所述第一线串光束或第一点阵光束拥有更高的密度和/或更大的视场角,且第二点阵光束为规则排列。In some embodiments, the transmitting module further includes a collimating lens and a diffractive optical element. The light beam emitted by the light source is collimated by the lens and then emits a collimated beam. The collimated beam is diffracted by the DOE to form a first string beam or a first point. The first linear beam or the first lattice beam is deflected by the beam scanner for multiple times to form a projection pattern composed of multiple second linear beams or multiple second lattice beams, and the projection pattern is larger than the first The line string beams or the first lattice beams have higher density and/or larger field of view, and the second lattice beams are arranged regularly.
S502、通过采集模组采集经目标物体反射回的光束;其中,采集模组包括由像素阵列组成的图像传感器;S502. Collect the light beam reflected by the target object through the collection module; wherein the collection module includes an image sensor composed of a pixel array;
在一些实施例中,图像传感器中各相应区域的像素可以依次采集由目标物体反射回的光束而累积的光电荷。In some embodiments, the pixels in each corresponding area of the image sensor can sequentially collect the light charges accumulated by the light beam reflected by the target object.
S503、控制与处理器根据偏转后形成的光束激活图像传感器中相应区域的像素,以响应于由反射回的光束而累积的光电荷,并基于光电荷计算相位差以 获取目标物体的距离。S503. The control and processor activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection to respond to the photocharge accumulated by the reflected light beam, and calculates the phase difference based on the photocharge to obtain the distance of the target object.
具体地,控制与处理器通过分时激活由像素阵列组成的图像传感器,以采集由给定区域反射回的光束而累积的光电荷,并基于像素采集的光电荷计算相位差以获取目标物体的距离,并输出目标物体的深度图像。本申请实施例中,由于多个偏转光束形成的投影图案完整覆盖目标物体,因此可以输出信噪比高且分辨率高的深度图像。Specifically, the control and processor activates the image sensor composed of the pixel array by time sharing to collect the photocharges accumulated by the light beam reflected back by the given area, and calculates the phase difference based on the photocharges collected by the pixels to obtain the target object Distance, and output the depth image of the target object. In the embodiment of the present application, since the projection pattern formed by multiple deflected beams completely covers the target object, a depth image with high signal-to-noise ratio and high resolution can be output.
作为本申请另一个实施例,还提供一种电子设备,所述电子设备可以是台式、桌面安装设备、便携式设备、可穿戴设备或车载设备以及机器人等。具体的,设备可以是笔记本电脑或电子设备,以允许手势识别或生物识别。在其他示例中,设备可以是头戴式设备,以用于标识用户周围环境的对象或危险,以确保安全,例如,阻碍用户对环境视觉的虚拟现实系统,可以检测周围环境中的对象或危险,以向用户提供关于附近对象或障碍物的警告。在另一些示例中,可以是将虚拟信息和图像与用户周围环境相混合的混合现实系统,可检测用户环境中的对象或人,以将虚拟信息与物理环境和对象集成。在其它示例中,还可以是应用在无人驾驶等领域的设备。参照图6所示,以手机为例进行说明,所述电子设备600包括壳体61、屏幕62、以及前述实施例所述的TOF深度测量装置;其中,所述TOF深度测量装置的发射模组11与采集模组12设置于电子设备600的同一面,用于向目标物体发射光束以及接收目标物体反射回来的光束并形成电信号。As another embodiment of the present application, an electronic device is also provided. The electronic device may be a desktop, a desktop-mounted device, a portable device, a wearable device or a vehicle-mounted device, a robot, and the like. Specifically, the device may be a notebook computer or an electronic device to allow gesture recognition or biometric recognition. In other examples, the device may be a head-mounted device to identify objects or hazards in the user's surrounding environment to ensure safety. For example, a virtual reality system that obstructs the user's vision of the environment can detect objects or hazards in the surrounding environment. To provide users with warnings about nearby objects or obstacles. In other examples, it may be a mixed reality system that mixes virtual information and images with the user's surrounding environment, and can detect objects or people in the user's environment to integrate the virtual information with the physical environment and objects. In other examples, it may also be a device used in fields such as unmanned driving. Referring to FIG. 6, taking a mobile phone as an example for description, the electronic device 600 includes a housing 61, a screen 62, and the TOF depth measuring device described in the foregoing embodiment; wherein, the transmitting module of the TOF depth measuring device 11 and the collection module 12 are arranged on the same surface of the electronic device 600, and are used to emit a light beam to the target object and receive the light beam reflected by the target object and form an electrical signal.
本申请实施例还提供一种存储介质,用于存储计算机程序,该计算机程序被执行时至少执行前述任一实施例中所述的方法。The embodiments of the present application also provide a storage medium for storing a computer program, which at least executes the method described in any of the foregoing embodiments when the computer program is executed.
所述存储介质可以由任何类型的易失性或非易失性存储设备、或者它们的组合来实现。其中,非易失性存储器可以是只读存储器(ROM,Read Only Memory)、可编程只读存储器(PROM,Programmable Read-Only Memory)、可擦除可编程只读存储器(EPROM,ErasableProgrammable Read-Only Memory)、电可擦除可编程只读存储器(EEPROM,ElectricallyErasable Programmable Read-Only  Memory)、磁性随机存取存储器(FRAM,FerromagneticRandom Access Memory)、快闪存储器(Flash Memory)、磁表面存储器、光盘、或只读光盘(CD-ROM,Compact Disc Read-Only Memory);磁表面存储器可以是磁盘存储器或磁带存储器。易失性存储器可以是随机存取存储器(RAM,Random Access Memory),其用作外部高速缓存。通过示例性但不是限制性说明,许多形式的RAM可用,例如静态随机存取存储器(SRAM,Static Random Access Memory)、同步静态随机存取存储器(SSRAM,SynchronousStatic Random Access Memory)、动态随机存取存储器(DRAM,Dynamic Random AccessMemory)、同步动态随机存取存储器(SDRAM,Synchronous Dynamic Random AccessMemory)、双倍数据速率同步动态随机存取存储器(DDRSDRAM,Double Data RateSynchronous Dynamic Random Access Memory)、增强型同步动态随机存取存储器(ESDRAM,Enhanced Synchronous Dynamic Random Access Memory)、同步连接动态随机存取存储器(SLDRAM,SyncLink Dynamic Random Access Memory)、直接内存总线随机存取存储器(DRRAM,Direct Rambus Random Access Memory)。本申请实施例描述的存储介质旨在包括但不限于这些和任意其它适合类型的存储器。The storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be read-only memory (ROM, Read Only Memory), programmable read-only memory (PROM, Programmable Read-Only Memory), and erasable programmable read-only memory (EPROM, Erasable Programmable Read-Only). Memory), Electrically Erasable Programmable Read-Only Memory (EEPROM, Electrically Erasable Programmable Read-Only Memory), Magnetic Random Access Memory (FRAM, Ferromagnetic Random Access Memory), Flash Memory (Flash Memory), Magnetic Surface Memory, Optical Disks, Or CD-ROM (Compact Disc Read-Only Memory); magnetic surface memory can be disk storage or tape storage. The volatile memory may be a random access memory (RAM, Random Access Memory), which is used as an external cache. By way of exemplary but not restrictive description, many forms of RAM are available, such as static random access memory (SRAM, Static Random Access Memory), synchronous static random access memory (SSRAM, Synchronous Static Random Access Memory), and dynamic random access memory. (DRAM, Dynamic Random Access Memory), Synchronous Dynamic Random Access Memory (SDRAM, Synchronous Dynamic Random Access Memory), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM, Double Data Rate Synchronous Dynamic Access Memory), Enhanced Synchronous Dynamic Random Access Memory Access memory (ESDRAM, Enhanced Synchronous Dynamic Random Access Memory), synchronous connection dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), direct memory bus random access memory (DRRAM, Direct Rambus Random Access Memory). The storage media described in the embodiments of the present application are intended to include, but are not limited to, these and any other suitable types of memory.
可以理解的是,以上内容是结合具体/优选的实施方式对本申请所作的进一步详细说明,不能认定本申请的具体实施只局限于这些说明。对于本申请所属技术领域的普通技术人员来说,在不脱离本申请构思的前提下,其还可以对这些已描述的实施方式做出若干替代或变型,而这些替代或变型方式都应当视为属于本申请的保护范围。在本说明书的描述中,参考术语“一种实施例”、“一些实施例”、“优选实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本申请的至少一个实施例或示例中。It is understandable that the above content is a further detailed description of the application in combination with specific/preferred implementations, and it cannot be considered that the specific implementation of the application is limited to these descriptions. For those of ordinary skill in the technical field to which this application belongs, without departing from the concept of this application, they can also make several substitutions or modifications to the described implementations, and these substitutions or modifications should be regarded as It belongs to the protection scope of this application. In the description of this specification, reference to the description of the terms "one embodiment", "some embodiments", "preferred embodiment", "examples", "specific examples", or "some examples" etc. means to incorporate the implementation The specific features, structures, materials or characteristics described by the examples or examples are included in at least one embodiment or example of the present application.
在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术 人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。尽管已经详细描述了本申请的实施例及其优点,但应当理解,在不脱离由所附权利要求限定的范围的情况下,可以在本文中进行各种改变、替换和变更。In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the described specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification without mutual contradiction. Although the embodiments of the present application and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the scope defined by the appended claims.
此外,本申请的范围不旨在限于说明书中所述的过程、机器、制造、物质组成、手段、方法和步骤的特定实施例。本领域普通技术人员将容易理解,可以利用执行与本文所述相应实施例基本相同功能或获得与本文所述实施例基本相同结果的目前存在的或稍后要开发的上述披露、过程、机器、制造、物质组成、手段、方法或步骤。因此,所附权利要求旨在将这些过程、机器、制造、物质组成、手段、方法或步骤包含在其范围内。In addition, the scope of the present application is not intended to be limited to the specific embodiments of the processes, machines, manufacturing, material composition, means, methods, and steps described in the specification. A person of ordinary skill in the art will easily understand that the above-mentioned disclosures, processes, machines, processes, machines, processes that currently exist or will be developed later that perform substantially the same functions as the corresponding embodiments described herein or obtain substantially the same results as the embodiments described herein can be used. Manufacturing, material composition, means, method, or step. Therefore, the appended claims intend to include these processes, machines, manufacturing, material compositions, means, methods, or steps within their scope.

Claims (10)

  1. 一种TOF深度测量装置,其特征在于,包括:A TOF depth measuring device, which is characterized in that it comprises:
    发射模组,用于朝向目标物体的给定区域发射光束;其中,所述发射模组包括有光源以及光束扫描器,通过所述光束扫描器偏转所述光源发出的光束以照射所述目标物体的给定区域;A transmitting module for emitting a light beam toward a given area of a target object; wherein the transmitting module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate the target object Of a given area;
    采集模组,用于采集经所述目标物体反射回的光束;其中,所述采集模组包括有由像素阵列组成的图像传感器;The collection module is used to collect the light beam reflected by the target object; wherein, the collection module includes an image sensor composed of a pixel array;
    控制与处理器,分别与所述发射模组和所述采集模组连接,以用于控制所述光束扫描器偏转所述光束以照射所述目标物体的给定区域,并根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应由所述给定区域反射回的光束而累积的光电荷,基于该光电荷计算相位差以获得所述目标物体的距离并输出所述目标物体的深度图像。The control and processor are respectively connected with the transmitting module and the collecting module to control the beam scanner to deflect the beam to irradiate a given area of the target object, and the result is formed according to the deflection The light beam activates the pixels in the corresponding area of the image sensor to respond to the light charge accumulated by the light beam reflected by the given area, and calculate the phase difference based on the light charge to obtain the distance of the target object and output the target Depth image of the object.
  2. 如权利要求1所述的TOF深度测量装置,其特征在于:所述发射模组还包括有透镜,所述光源通过所述透镜产生线光束,所述线光束通过所述光束扫描器进行线扫描以照射所述目标物体。The TOF depth measuring device according to claim 1, wherein the transmitting module further comprises a lens, the light source generates a line beam through the lens, and the line beam is scanned by the beam scanner To illuminate the target object.
  3. 如权利要求2所述的TOF深度测量装置,其特征在于:所述透镜为柱透镜,所述光源发出的光束经过所述柱透镜后形成第一线光束,所述第一线光束经过所述光束扫描器多次偏转后得到由多条第二线光束组成的投影图案。The TOF depth measuring device according to claim 2, wherein the lens is a cylindrical lens, the light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam passes through the The beam scanner is deflected multiple times to obtain a projection pattern composed of a plurality of second line beams.
  4. 如权利要求2所述的TOF深度测量装置,其特征在于:所述发射模组还包括有准直透镜和衍射光学元件;其中,所述光源发出的光束经过所述准直透镜准直后发出准直光束,所述准直光束通过所述衍射光学元件衍射后形成第一线串光束或第一点阵光束,所述第一线串光束或第一点阵光束经过所述光束扫描器多次偏转后得到由多条第二线串光束或第二点阵光束组成的投影图案。The TOF depth measuring device according to claim 2, wherein the transmitting module further comprises a collimating lens and a diffractive optical element; wherein the light beam emitted by the light source is collimated by the collimating lens and then emitted A collimated light beam, the collimated light beam is diffracted by the diffractive optical element to form a first line string beam or a first lattice beam, and the first line string beam or the first lattice beam passes through the beam scanner. After the second deflection, a projection pattern composed of multiple second line string beams or second lattice beams is obtained.
  5. 如权利要求4所述的TOF深度测量装置,其特征在于:所述第二点阵光束呈规则排列。The TOF depth measuring device according to claim 4, wherein the second lattice beams are arranged regularly.
  6. 一种TOF深度测量方法,其特征在于,包括如下步骤:A TOF depth measurement method is characterized in that it comprises the following steps:
    通过发射模组朝向目标物体的给定区域发射光束;其中,所述发射模组包括有光源以及光束扫描器,通过所述光束扫描器偏转所述光源发出的光束以照射所述目标物体的给定区域;The emission module emits a light beam toward a given area of the target object; wherein the emission module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate the target object. Fixed area
    通过采集模组采集经所述目标物体反射回的光束;其中,所述采集模组包括由像素阵列组成的图像传感器;Collecting the light beam reflected by the target object through a collection module; wherein, the collection module includes an image sensor composed of a pixel array;
    控制与处理器根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应于由所述反射回的光束而累积的光电荷,并基于所述光电荷计算相位差以获得所述目标物体的距离。The control and processor activates the pixels in the corresponding area of the image sensor according to the deflected light beam to respond to the photocharge accumulated by the reflected light beam, and calculates the phase difference based on the photocharge to obtain the The distance of the target object.
  7. 如权利要求6所述的TOF深度测量方法,其特征在于:所述发射模组还包括有柱透镜,所述光源发出光束经过所述柱透镜形成第一线光束,所述第一线光束经过所述光束扫描器多次偏转后得到由多条第二线光束组成的投影图案。The TOF depth measurement method according to claim 6, wherein the transmitting module further comprises a cylindrical lens, the light beam emitted by the light source passes through the cylindrical lens to form a first line beam, and the first line beam passes through The beam scanner is deflected multiple times to obtain a projection pattern composed of a plurality of second line beams.
  8. 如权利要求6所述的TOF深度测量方法,其特征在于:所述发射模组还包括有准直透镜以及衍射光学元件,所述光源发出的光束经过所述准直透镜准直后发出准直光束,所述准直光束通过所述衍射光学元件衍射形成第一线串光束或第一点阵光束,所述第一线串光束或第一点阵光束经过所述光束扫描器多次偏转后得到由多条第二线光束或多个第二点阵光束组成的投影图案。7. The TOF depth measurement method according to claim 6, wherein the transmitting module further comprises a collimating lens and a diffractive optical element, and the light beam emitted by the light source is collimated by the collimating lens and then emitted. The collimated beam is diffracted by the diffractive optical element to form a first line string beam or a first lattice beam, and the first line string beam or the first lattice beam is deflected multiple times by the beam scanner A projection pattern composed of multiple second line beams or multiple second lattice beams is obtained.
  9. 如权利要求6所述的TOF深度测量方法,其特征在于:所述控制与处理器控制所述光束扫描器每个方向的偏转次数、偏转角度以及偏转顺序以获得不同密度和不同视场角的投影图案。The TOF depth measurement method according to claim 6, wherein the control and processor controls the number of deflection, deflection angle, and deflection sequence of the beam scanner in each direction to obtain different density and different field of view angles. Projection pattern.
  10. 一种电子设备,其特征在于,包括:壳体、屏幕以及TOF深度测量装置;其中,所述TOF深度测量装置包括:发射模组,用于朝向目标物体的给定区域发射光束;其中,所述发射模组包括有光源以及光束扫描器,通过所述光束扫描器偏转所述光源发出的光束以照射所述目标物体的给定区域;采集模组,用于采集经所述目标物体反射回的光束;其中,所述采集模组包括有由像素阵 列组成的图像传感器;控制与处理器,分别与所述发射模组和所述采集模组连接,以用于控制所述光束扫描器偏转所述光束以照射所述目标物体的给定区域,并根据偏转后形成的光束激活所述图像传感器中相应区域的像素,以响应由所述给定区域反射回的光束而累积的光电荷,基于该光电荷计算相位差以获得所述目标物体的距离并输出所述目标物体的深度图像;所述发射模组与采集模组设置于电子设备的同一面,以用于向目标物体发射光束以及接收目标物体反射回来的光束并形成电信号。An electronic device, characterized by comprising: a housing, a screen, and a TOF depth measuring device; wherein the TOF depth measuring device comprises: a transmitting module for transmitting a light beam toward a given area of a target object; wherein The emission module includes a light source and a beam scanner, and the light beam emitted by the light source is deflected by the beam scanner to illuminate a given area of the target object; The beam; wherein the acquisition module includes an image sensor composed of a pixel array; a control and processor, respectively connected with the emission module and the acquisition module, for controlling the deflection of the beam scanner The light beam illuminates a given area of the target object, and activates the pixels in the corresponding area of the image sensor according to the light beam formed after the deflection, in response to the accumulated photocharges of the light beam reflected by the given area, Calculate the phase difference based on the photoelectric charge to obtain the distance of the target object and output the depth image of the target object; the transmitting module and the collecting module are arranged on the same surface of the electronic device for emitting light beams to the target object And to receive the light beam reflected by the target object and form an electrical signal.
PCT/CN2020/141870 2020-04-20 2020-12-30 Tof depth measurement apparatus and method, and electronic device WO2021212916A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010311680.4 2020-04-20
CN202010311680.4A CN111458717A (en) 2020-04-20 2020-04-20 TOF depth measuring device and method and electronic equipment

Publications (1)

Publication Number Publication Date
WO2021212916A1 true WO2021212916A1 (en) 2021-10-28

Family

ID=71685994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/141870 WO2021212916A1 (en) 2020-04-20 2020-12-30 Tof depth measurement apparatus and method, and electronic device

Country Status (2)

Country Link
CN (1) CN111458717A (en)
WO (1) WO2021212916A1 (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111458717A (en) * 2020-04-20 2020-07-28 深圳奥比中光科技有限公司 TOF depth measuring device and method and electronic equipment
CN112729164B (en) * 2020-12-21 2023-06-30 革点科技(深圳)有限公司 MEMS-based self-adaptive lattice structured light projection method
CN113534596B (en) * 2021-07-13 2022-09-27 盛景智能科技(嘉兴)有限公司 RGBD stereo camera and imaging method
CN116027345A (en) * 2021-10-26 2023-04-28 华为技术有限公司 Three-dimensional imaging method and device
CN115144842B (en) * 2022-09-02 2023-03-14 深圳阜时科技有限公司 Transmitting module, photoelectric detection device, electronic equipment and three-dimensional information detection method

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106772407A (en) * 2016-12-02 2017-05-31 深圳市镭神智能系统有限公司 Laser radar system based on MEMS micromirror scanning
CN206331115U (en) * 2016-12-02 2017-07-14 深圳市镭神智能系统有限公司 The laser radar system scanned based on MEMS micromirror
US20180203102A1 (en) * 2017-01-18 2018-07-19 Analog Devices Global Depth sensing with multiple light sources
CN108490420A (en) * 2018-06-12 2018-09-04 深圳市镭神智能系统有限公司 A kind of micro mirror scanning optics
US20180284284A1 (en) * 2017-03-29 2018-10-04 Luminar Technologies, Inc. Optical resolution in front of a vehicle
CN109188451A (en) * 2018-10-15 2019-01-11 北京径科技有限公司 A kind of laser radar system
CN109963138A (en) * 2019-02-15 2019-07-02 深圳奥比中光科技有限公司 A kind of depth camera and image acquiring method
CN110824490A (en) * 2019-09-27 2020-02-21 深圳奥锐达科技有限公司 Dynamic distance measuring system and method
CN111458717A (en) * 2020-04-20 2020-07-28 深圳奥比中光科技有限公司 TOF depth measuring device and method and electronic equipment

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106772407A (en) * 2016-12-02 2017-05-31 深圳市镭神智能系统有限公司 Laser radar system based on MEMS micromirror scanning
CN206331115U (en) * 2016-12-02 2017-07-14 深圳市镭神智能系统有限公司 The laser radar system scanned based on MEMS micromirror
US20180203102A1 (en) * 2017-01-18 2018-07-19 Analog Devices Global Depth sensing with multiple light sources
US20180284284A1 (en) * 2017-03-29 2018-10-04 Luminar Technologies, Inc. Optical resolution in front of a vehicle
CN108490420A (en) * 2018-06-12 2018-09-04 深圳市镭神智能系统有限公司 A kind of micro mirror scanning optics
CN109188451A (en) * 2018-10-15 2019-01-11 北京径科技有限公司 A kind of laser radar system
CN109963138A (en) * 2019-02-15 2019-07-02 深圳奥比中光科技有限公司 A kind of depth camera and image acquiring method
CN110824490A (en) * 2019-09-27 2020-02-21 深圳奥锐达科技有限公司 Dynamic distance measuring system and method
CN111458717A (en) * 2020-04-20 2020-07-28 深圳奥比中光科技有限公司 TOF depth measuring device and method and electronic equipment

Also Published As

Publication number Publication date
CN111458717A (en) 2020-07-28

Similar Documents

Publication Publication Date Title
WO2021212916A1 (en) Tof depth measurement apparatus and method, and electronic device
CN110596721B (en) Flight time distance measuring system and method of double-shared TDC circuit
CN110596722B (en) System and method for measuring flight time distance with adjustable histogram
EP3433578B1 (en) Lidar based 3-d imaging with varying illumination intensity
JP6854387B2 (en) Synchronous spinning LIDAR and rolling shutter camera system
JP7203217B2 (en) Time-of-flight sensor with structured light illumination
JP6977045B2 (en) Systems and methods for determining the distance to an object
US11328446B2 (en) Combining light-field data with active depth data for depth map generation
CA3017817C (en) Lidar based 3-d imaging with varying illumination field density
WO2021212917A1 (en) Device and method for measuring tof depth
CN110596725B (en) Time-of-flight measurement method and system based on interpolation
CN212694038U (en) TOF depth measuring device and electronic equipment
CN110596724B (en) Method and system for measuring flight time distance during dynamic histogram drawing
JP7028878B2 (en) A system for measuring the distance to an object
CN110596723B (en) Dynamic histogram drawing flight time distance measuring method and measuring system
CN108139483A (en) For determining the system and method for the distance of object
CN109791205A (en) For the method from the exposure value of the pixel unit in imaging array subduction bias light and for the pixel unit of this method
CN102947726A (en) Scanning 3d imager
WO2019076072A1 (en) Optical distance measurement method and apparatus
CN111427230A (en) Imaging method based on time flight and 3D imaging device
CN115023627A (en) Efficient algorithm for projecting world points onto rolling shutter images
CN110780312A (en) Adjustable distance measuring system and method
CN212135134U (en) 3D imaging device based on time flight
CN113780349A (en) Method for acquiring training sample set, model training method and related device
WO2019021887A1 (en) Optical radar device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20932133

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20932133

Country of ref document: EP

Kind code of ref document: A1