CN109597211B - Projection module, depth camera and depth image acquisition method - Google Patents
Projection module, depth camera and depth image acquisition method Download PDFInfo
- Publication number
- CN109597211B CN109597211B CN201811589922.5A CN201811589922A CN109597211B CN 109597211 B CN109597211 B CN 109597211B CN 201811589922 A CN201811589922 A CN 201811589922A CN 109597211 B CN109597211 B CN 109597211B
- Authority
- CN
- China
- Prior art keywords
- light
- sub
- light source
- structured
- patterned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/42—Diffraction optics, i.e. systems including a diffractive element being designed for providing a diffractive effect
- G02B27/4205—Diffraction optics, i.e. systems including a diffractive element being designed for providing a diffractive effect having a diffractive optical element [DOE] contributing to image formation, e.g. whereby modulation transfer function MTF or optical aberrations are relevant
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B17/00—Details of cameras or camera bodies; Accessories therefor
- G03B17/55—Details of cameras or camera bodies; Accessories therefor with provision for heating or cooling, e.g. in aircraft
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10141—Special mode during image acquisition
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Aviation & Aerospace Engineering (AREA)
- Length Measuring Devices By Optical Means (AREA)
- Measurement Of Optical Distance (AREA)
Abstract
The invention relates to the technical field of optics and electronics, and provides a projection module, a structured light depth camera and a structured light image acquisition method, wherein the projection module comprises a light source, an optical lens and a diffraction optical element which are arranged along a light path, the light source comprises a plurality of sub light sources, each sub light source comprises a plurality of light-emitting elements, and the plurality of sub light sources are used for alternately generating patterned light beams in a projection period; the diffraction optical element respectively receives the patterned light beams alternately generated by the plurality of sub light sources and correspondingly generates mutually superposed sub structured light patterns; in a projection period, the plurality of sub-light sources alternately generate patterned light beams, and all light-emitting elements in the light sources do not need to be lightened simultaneously, so that the overall power can be greatly reduced; because a plurality of sub-light sources are lighted in a time-sharing manner, the lighting time of each sub-light source can not last for one projection period, so that the power consumption of the light source is greatly reduced, and the requirement of the intelligent equipment adopting the projection module on low power consumption can be effectively met.
Description
Technical Field
The invention relates to the technical field of optics and electronics, in particular to a projection module, a depth camera and a depth image acquisition method.
Background
As the demand for 3D imaging functionality from smart devices (e.g., mobile terminals) is increasing, depth cameras will be increasingly embedded in smart devices, which also requires the development of depth cameras toward small size and high performance. The projection module is one of the core components in the depth camera and is also the most important factor affecting the performance of the depth camera. The projection module generally includes a laser source, a lens, and other optical elements, and is used for projecting a light beam into a space.
Along with smart machine is stricter and stricter to the control of consumption, need control the consumption of projection module in the degree of depth camera. The conventional scheme reduces power consumption by reducing the number of light sources or the power of the light sources, but this method may cause performance degradation of the depth camera, for example, the resolution of the structured light is reduced, and further, accuracy of the obtained depth image is reduced, and the use requirement cannot be met. Therefore, a solution with low power consumption and high performance is still lacking.
Disclosure of Invention
The invention aims to provide a projection module to solve the technical problem that the conventional depth camera cannot meet the requirements of low power consumption and high performance at the same time.
In order to achieve the purpose, the invention adopts the technical scheme that: providing a projection module, which comprises a light source, an optical lens and a diffraction optical element arranged along a light path;
the light source comprises a plurality of sub light sources, each sub light source comprises a plurality of light-emitting elements, and the plurality of sub light sources are used for alternately generating patterned light beams in one projection period;
the diffraction optical elements respectively receive the patterned light beams generated by the plurality of sub light sources alternately and correspondingly generate the mutually-superposed sub-structured light patterns.
In one embodiment, a plurality of the sub-structured light patterns partially overlap;
alternatively, the first and second electrodes may be,
a plurality of the sub-structured light patterns are complementary.
In one embodiment, the light source comprises a first sub-light source comprising a plurality of first light emitting elements and a second sub-light source comprising a plurality of second light emitting elements;
the first light-emitting element is used for generating a first patterned light beam, the second light-emitting element is used for generating a second patterned light beam, and the first patterned light beam and the second patterned light beam are generated alternately;
the diffraction optical element receives the first patterned light beam and the second patterned light beam respectively and correspondingly generates a first structured light pattern and a second structured light pattern which can be superposed with each other.
In one embodiment, the first light emitting element includes at least one of a light emitting diode, an edge-emitting laser diode, and a vertical cavity surface-emitting laser;
the second light emitting element includes at least one of a light emitting diode, an edge-emitting laser diode, and a vertical cavity surface-emitting laser.
In one embodiment, the first sub-light source and the second sub-light source are at least partially coincident, and the first light emitting elements and the second light emitting elements are distributed in a staggered manner;
or, the first sub light source and the second sub light source are not overlapped with each other.
The invention also provides a depth camera, which comprises the projection module, the imaging module and a processor;
the processor is connected with the imaging module and the projection module;
the processor is used for controlling the plurality of sub light sources in the projection module to alternately generate patterned light beams and controlling the imaging module to respectively acquire a plurality of sub structured light patterns so as to acquire the structured light patterns.
In one embodiment, the depth camera further comprises a heat sink, and the plurality of sub-light sources are respectively in heat-conducting connection with the heat sink.
In one embodiment, the depth camera further comprises a color camera module connected with the processor for collecting color patterns.
The invention also aims to provide a depth image acquisition method, which comprises the following steps:
controlling the projection module to alternately project a plurality of groups of sub-structure light patterns which can be mutually overlapped through the first group of time sequence control signals;
controlling an imaging module to collect the multiple groups of sub-structure light patterns through a second group of time sequence control signals, wherein the first group of time sequence control signals and the second group of time sequence control signals have the same period;
and obtaining the structured light pattern according to the superposed areas of the plurality of groups of the sub-structured light patterns.
In one embodiment, the number of the sub-structure patterns is two, and the first group of timing control signals includes a first timing signal and a second timing signal;
the step of controlling the projection module to alternately project a plurality of groups of structured light patterns which can be mutually superposed through the first group of time sequence control signals comprises the following steps:
controlling a first light-emitting element in the projection module to generate a first patterned light beam through the first timing signal;
controlling a second light-emitting element in the projection module to generate a second patterned light beam through the second timing signal, wherein the first patterned light beam and the second patterned light beam are generated alternately;
and respectively receiving the first patterned light beam and the second patterned light beam through a diffraction optical element in the projection module, and correspondingly generating a first structured light pattern and a second structured light pattern.
The projection module provided by the invention has the beneficial effects that: the light source is divided into the plurality of sub-light sources to be respectively controlled, and the plurality of sub-light sources alternately generate the patterned light beams in one projection period, so that only the light-emitting element in at most one sub-light source is lightened when the light source projects the patterned light beam at each moment, and all the light-emitting elements in the light source are not required to be lightened simultaneously, thereby greatly reducing the overall power and further reducing the design requirements of a matched driving circuit and the like. Moreover, because a plurality of sub-light sources are lighted in a time-sharing manner, the lighting time of each sub-light source does not last for one projection period, so that the power consumption of the light source in one period can be greatly reduced, and the requirement of the intelligent equipment adopting the projection module on low power consumption can be effectively met. Therefore, not only can higher resolution be obtained, but also the power consumption of the projection module can be effectively reduced.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed for the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings without creative efforts.
FIG. 1 is a schematic structural diagram of a depth camera according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a projection module according to an embodiment of the invention;
fig. 3 is a first schematic structural diagram of a light source in a projection module according to an embodiment of the present invention;
fig. 4 is a second schematic structural diagram of a light source in a projection module according to an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating a timing control signal sent by a processor in a depth camera according to an embodiment of the present invention;
fig. 6 is a first schematic structural diagram of a superimposed image acquired by an imaging module in the depth camera according to the embodiment of the present invention;
fig. 7 is a schematic structural diagram of a second superimposed image acquired by an imaging module in the depth camera according to the embodiment of the present invention;
fig. 8 is a schematic flowchart of a depth image obtaining method according to an embodiment of the present invention;
fig. 9 is a schematic flowchart of step S10 in the depth image obtaining method according to the embodiment of the present invention.
Wherein, in the figures, the respective reference numerals:
10-a projection module; 100-window;
11-a light source; 111-a first sub-light source;
1110-a first light emitting element; 112-a second sub-light source;
1120 — a second light emitting element; 12-an optical lens;
13-a diffractive optical element; 14-a lens holder;
20-an imaging module; 30-a control circuit board;
40-a processor; 50-color camera module.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more clearly apparent, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
It will be understood that when an element is referred to as being "secured to" or "disposed on" another element, it can be directly or indirectly secured to the other element. When an element is referred to as being "connected to" another element, it can be directly or indirectly connected to the other element. The terms "upper", "lower", "left", "right", "front", "rear", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like indicate orientations or positions based on the orientations or positions shown in the drawings, and are for convenience of description only and not to be construed as limiting the technical solution. The terms "first", "second" and "first" are used merely for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features. The meaning of "plurality" is two or more unless specifically limited otherwise.
Referring to fig. 2 to 4, a projection module 10 includes a light source 11, an optical lens 12, and a Diffractive Optical Element (DOE)13 disposed along an optical path, where the light source 11 includes a plurality of sub-light sources, each of which includes a plurality of light emitting elements, and the plurality of sub-light sources are used for alternately generating a patterned light beam in a projection period; the diffractive optical element 13 receives the patterned light beams alternately generated by the plurality of sub-light sources, and correspondingly generates sub-structured light patterns which can be overlapped with each other. The number of the sub light sources can be set according to needs, and is not limited herein.
Referring to fig. 1, in one embodiment, the projection module 10 can be applied in a depth camera for projecting a structured light pattern. The sub-light sources alternately generate the patterned light beams means that in a projection period, each sub-light source sequentially generates the patterned light beams, the sequentially generated patterned light beams sequentially reach the diffractive optical element 13, the diffractive optical element 13 sequentially projects the sub-structured light patterns forward according to the received patterned light beams, the sub-structured light patterns can be sequentially received by the imaging module 20 in the depth camera in an acquisition period, and finally the structured light patterns superposed by the sub-structured light patterns are obtained.
When the projection module 10 generates a patterned light beam to obtain a structured light pattern, all light sources in the projection module 10 are turned on simultaneously in one acquisition period of the imaging module 20, and the duration of the acquisition period is a projection period, which is matched with the projection period. On the one hand, in order to obtain higher resolution, the light source 11 in the projection module 10 usually includes a large number of light emitting elements, so the overall power of the large number of light emitting elements is larger, and the design requirement for a matching driving circuit and the like is higher; on the other hand, because the light-on time of the light source in the projection module 10 needs to last a projection cycle, can lead to the consumption greatly increased of projection module 10, when the smart machine adopted this projection module 10, can't satisfy the requirement of smart machine to low-power consumption.
This embodiment provides a new design method, which not only can obtain higher resolution, but also can effectively reduce the power consumption of the projection module 10. By dividing the light source 11 into a plurality of sub-light sources for respective control, the plurality of sub-light sources alternately generate patterned light beams in one projection period, so that at most only one light emitting element in one sub-light source is lighted when the light source projects the patterned light beam at each moment, and all the light emitting elements in the light source do not need to be lighted at the same time, thereby greatly reducing the overall power and further reducing the design requirements of a matched driving circuit and the like. Moreover, because a plurality of sub-light sources are lighted in a time-sharing manner, and the lighting time of each sub-light source does not last for one projection period, the power consumption of the light source in one period can be greatly reduced, and the requirement of the intelligent device adopting the projection module 10 on low power consumption can be effectively met.
In one embodiment, the projection module 10 is used in a depth camera. Fig. 1 shows a schematic structural diagram of a depth camera provided in this embodiment. The depth camera includes a projection module 10, an imaging module 20, a control circuit board 30 and a processor 40, wherein the projection module 10, the imaging module 20 and the processor 40 are all integrated on the control circuit board 30. The control circuit board 30 may be a PCB (printed circuit board) for providing support and electrical connection for the projection module 10, the imaging module 20 and the processor 40. The processor 40 controls the projection module 10 and the imaging module 20 via the control circuit board 30 via corresponding interfaces, which include various interfaces, such as an I2C interface. It is understood that the control circuit board 30 may be any other type, such as a Flexible Printed Circuit (FPC), a rigid-flex board, or a combination of other metals, ceramics, semiconductor materials, SOI (Silicon-On-Insulator), and the like.
Referring to fig. 2, in an embodiment, the projection module 10 further includes a lens holder 14, and the light source 11, the optical lens 12 and the diffractive optical element 13 are disposed in the lens holder 14, so as to ensure good fixation. The light source 11 further includes a semiconductor substrate on which a plurality of sub-light sources are distributed, the semiconductor substrate being disposed on the control circuit board 30 through electrical connection. It will be appreciated that the electrical connections and control circuit board 30 may be coupled to the light source 11 by wires to control the operation of the light source 11 and provide a source of electrical current that causes the light source 11 to emit a patterned beam of light.
Referring to fig. 3, in an embodiment, the light source 11 includes two independently controllable first sub-light sources 111 and second sub-light sources 112, and the first sub-light sources 111 and the second sub-light sources 112 may be disposed on the same or different semiconductor substrates. The first sub-light sources 111 include a plurality of first light emitting elements 1110, the second sub-light sources 112 include a plurality of second light emitting elements 1120, and the light emitting elements on each sub-light source are integrally controlled and turned on or off simultaneously. As shown in fig. 3, the shapes of the first light emitting element 1110 and the second light emitting element 1120 are different for visual distinction, and actually, the shapes of the first light emitting element 1110 and the second light emitting element 1120 may be the same or different, and may be set according to actual needs, which is not limited herein.
Wherein the first light emitting element 1110 is used to generate a first patterned beam and the second light emitting element 1120 is used to generate a second patterned beam, and the first patterned beam and the second patterned beam are alternately generated by the individual control of the first sub light source 111 and the second sub light source 112. Alternatively, the wavelength of the patterned light beam emitted from the first light emitting element 1110 is the same as the wavelength of the patterned light beam emitted from the second light emitting element 1120.
The diffractive optical element 13 receives the first patterned beam and the second patterned beam, respectively, and correspondingly generates a first structured light pattern and a second structured light pattern which can be superimposed on each other. Referring to fig. 5, for example, when the structured light pattern needs to be projected, the processor 40 sends a first group of timing control signals S51 to the light source 11 in the projection module 10, where the first group of timing control signals S51 is sent in a pulse form, and each projection period includes a first timing signal S511 and a second timing signal S512 that are sent in sequence, where the first timing signal S511 is used to control the first sub-light source 111 to emit the first patterned light beam, and the second timing signal S512 is used to control the second sub-light source 112 to emit the second patterned light beam, so that the first group of timing control signals S51 can control the light source 11 to alternately generate the first patterned light beam and the second patterned light beam. The processor 40 sends a second set of timing control signals S52 to the imaging module 20, the second set of timing control signals S52 is sent in a pulse form, wherein the second set of timing control signals S52 is synchronized with the first set of timing control signals S51, and each acquisition cycle of the second set of timing control signals S52 is the same as the projection cycle. Therefore, in one acquisition cycle, the imaging module 20 can acquire the first structured light pattern and the second structured light pattern respectively, so as to obtain a structured light pattern superposed by the first structured light pattern and the second structured light pattern. Of course, in other embodiments, the timing control signal from processor 40 may take other suitable forms.
When the first timing signal S511 is at a high level, the first light emitting element 1110 in the first sub-light source 111 emits the patterned light beam; when the second timing signal S512 is at a high level, the second light emitting element 1120 in the second sub-light source 112 emits the patterned light beam. It is understood that the patterned light beams emitted by the first light emitting element 1110 and the second light emitting element 1120 can be visible light, infrared light, ultraviolet light, or light in other wavelength bands. The pattern projected by the projection module 10 may also include a coded projection scheme composed of different patterns, such as a speckle pattern, a block pattern, a cross pattern, a stripe pattern, a specific symbol pattern, and the like. For example, in one embodiment, the structured light pattern projected by the projection module 10 is an infrared speckle pattern, and the infrared speckle pattern has the characteristics of high irrelevance and uniform distribution.
When the second set of timing control signals S52 is at a high level, the imaging module 20 is turned on to collect the structured light pattern projected by the projection module 10. Since the first timing signal S511 and the second timing signal S512 are synchronized with the second timing control signal S52 and are alternately emitted within a single frame period of the imager module 10, the structured-light pattern finally formed by the imager module 20 includes the superimposed portion of the sub-structured-light patterns generated by the first light-emitting device 1110 and the second light-emitting device 1120. The overlapping mode can be set according to the requirement, and two possible overlapping modes are provided below.
Referring to fig. 6, a first stacking method:
through reasonable design of the light source 11 and the diffractive optical element 13, during the period that the first light-emitting element 1110 is turned on, the first structured-light pattern 601 projected by the projection module 10 is composed of a plurality of sub-units a in the figure (for example, 16 sub-units a are arranged in 4 rows and 4 columns); during the period when the second light emitting device 1120 is turned on, the second structure pattern 602 projected by the projection module 10 is composed of a plurality of sub-units B (e.g. 16 sub-units B are arranged in 4 rows and 4 columns). Since the first light emitting element 1110 and the second light emitting element 1120 are continuously turned on in one collection period, the structured light pattern 603 finally formed is a portion where the first structured light pattern 601 and the second structured light pattern 602 are superimposed, and the density of the superimposed structured light pattern 603 is greater than that of the structured light pattern formed by independently turning on the first light emitting element 1110 and the second light emitting element 1120.
Referring to fig. 7, a second stacking method:
through reasonable design of the light source 11 and the diffractive optical element 13, during the period that the first light emitting element 1110 is turned on, the first structured light pattern 701 projected by the projection module 10 is composed of a plurality of sub-units a in the figure (for example, 8 sub-units a are arranged in 4 rows, each row is provided with two sub-units a, and a gap is formed between two sub-units a in the same row); during the period that the second light emitting element 1120 is turned on, the second structured-light pattern 702 projected by the projection module 10 is composed of a plurality of sub-units B (e.g. 8 sub-units B are arranged in 4 rows, each row is provided with two sub-units B, and two sub-units B in the same row have a gap therebetween and are alternately staggered with the sub-units a, so that the gap between two sub-units a can form exactly one sub-unit B, and the gap between two sub-units B can form exactly one sub-unit a). Since the first 1110 and second 1120 light-emitting elements are turned on continuously during one acquisition period, the resulting structured light pattern 703 will be a superposition of the two sub-structured light patterns, the effect of the superposition being that sub-unit a and sub-unit B complement to fill the void region.
It is to be understood that the above two stacking manners are only illustrative and are not limited thereto. The structured light pattern finally formed by superposition is determined by a specific scheme.
In one embodiment, the first light emitting element 1110 and the second light emitting element 1120 include at least one of a light emitting diode, an edge emitting laser diode, and a VCSEL (vertical cavity surface emitting laser), and the types of the first light emitting element 1110 and the second light emitting element 1120 may be the same or different. For example, the first light emitting elements 1110 are VCSEL arrays distributed in an irregular two-dimensional pattern on a semiconductor substrate; the second light emitting elements 1120 are LED arrays distributed in an irregular two-dimensional pattern on the semiconductor substrate. For another example, the first light emitting elements 1110 and the second light emitting elements 1120 are VCSEL arrays and are distributed on the semiconductor substrate in a regular or irregular two-dimensional pattern. It should be understood that the above is by way of example only and is not intended as limiting.
It is understood that the power of the first light emitting element 1110 may be the same as or different from that of the second light emitting element 1120, and may be set according to actual needs; likewise, the number of the first light emitting elements 1110 may be the same as or different from the number of the second light emitting elements 1120. It is understood that the arrangement of the first light emitting elements 1110 is at least partially complementary to the arrangement of the second light emitting elements 1120, and when the first light emitting elements 1110 and the second light emitting elements 1120 are alternately turned on in a single frame acquisition period of the imaging module 20, sub-structured light patterns formed by the patterned light beams emitted by the first light emitting elements 1110 and the patterned light beams emitted by the second light emitting elements 1120 are arranged in the imaging module 20 to intersect with each other to generate partially overlapping or complementary overlapping patterns, so as to obtain the structured light patterns. For example, referring to fig. 3, the plurality of first light emitting elements 1110 in the first sub-light source 111 and the plurality of sub-light sources 1120 in the second sub-light source 112 are distributed in a staggered manner, so that the arrangement portions of the two are complementary, and a mutually overlapped or complementary overlapping pattern can be generated. For another example, referring to fig. 4, the first sub-light source 111 and the second sub-light source 112 are not overlapped with each other (for example, the first light emitting element 1110 and the second light emitting element 1120 are respectively arranged in two regions arranged side by side), and at this time, the two light emitting elements may form a complementary overlapping pattern, and a desired structured light pattern may also be obtained.
The imaging module 20 includes a combination of an image sensor (e.g., a lens unit disposed on a light-in path of the image sensor), which may be a CCD (charge coupled device) image sensor, a CMOS (Complementary Metal Oxide Semiconductor) image sensor or other types, and other optical elements, which are not limited herein. The image sensor is connected to the processor 40, and is configured to collect the sub-structured light patterns alternately generated by the first light emitting element 1110 and the second light emitting element 1120 in one collection period under the control of a second set of timing control signals S52 sent by the processor 40, and generate the structured light patterns. It will be appreciated that a filter may be provided in the light path of the image sensor to allow only the structured-light pattern projected by the projection module 10 to pass through, thereby avoiding interference from ambient light. For example, when the structured light pattern projected by the projection module 10 is an infrared speckle pattern, an infrared filter may be disposed on the light incident surface of the image sensor, so that the image sensor only collects the infrared speckle pattern.
Referring to fig. 1, in one embodiment, in order to provide the depth camera with more functions, the depth camera may further include a color camera module 50 for collecting color patterns. The projection module 10, the imaging module 20, and the color camera module 50 are generally mounted on the same plane of the depth camera and on the same base line, and each of the modules corresponds to a window 100 for light to enter or exit. The color camera module 50 is installed between the projection module 10 and the imaging module 20, and is closer to the projection module 10. In one embodiment, the color camera module 50 may be an RGB camera. Therefore, the depth camera equipped with the color camera module 50 has the capability of synchronously acquiring the depth pattern and the RGB pattern. The structured light pattern collected by the imaging module 20 and the color image collected by the color camera module 50 may be further transmitted to the processor 40, and the processor 40 may perform further processing, such as calculating a depth image, according to the received structured light pattern and color pattern, and may perform operations such as face detection, face recognition, security payment, and the like according to the depth image.
In some embodiments, to accelerate the heat dissipation of the depth camera, a heat sink may be further attached to the heat-generating portion of the projection module 10. For example, when the light source of the projection module 10 is a VCSEL array, a hole may be formed in the control circuit board 30 supporting the projection module 10 to allow the heat sink to contact the ceramic under the VCSEL to dissipate heat, so as to avoid an excessively high operating temperature of the VCSEL, prolong the service life of the projection module 10, accelerate the heat dissipation of the depth camera, and provide advantages for the depth camera to be integrated into various mobile terminals. It will be appreciated that the heat sink is attached to the heat generating portion of the projection module 10, and thus the heat sink is very small and does not substantially increase the size of the depth camera. The heat sink may be made of aluminum or other materials, which is not limited herein.
Referring to fig. 8, the present embodiment is further directed to a depth image obtaining method, including the following steps:
step S10: and controlling the projection module to alternately project a plurality of groups of sub-structure light patterns which can be mutually overlapped through the first group of time sequence control signals.
The projection module 10 includes a light source 11, the light source 11 includes a plurality of sub-light sources independently controlled from each other, and the processor 40 in the depth camera can independently control each sub-light source, so that the plurality of sub-light sources alternately project the sub-structured light patterns in one projection period, and the plurality of sub-structured light patterns can be mutually overlapped.
For example, the number of the sub-structured light patterns is two, in this case, the light source 11 includes the first sub-light source 111 and the second sub-light source 112, the first sub-light source 111 includes a plurality of first light emitting elements 1110, and the plurality of first light emitting elements 1110 are controlled in a unified manner; the second sub-light sources 112 include a plurality of second light emitting elements 1120, and the plurality of second light emitting elements 1120 are controlled in a unified manner. The first set of timing control signals S51 sent by the processor 40 includes a first timing signal S511 and a second timing signal S512 to control the first light emitting element 1110 and the second light emitting element 1120, respectively. Referring to fig. 9, the step S10 includes:
step S101: controlling a first light-emitting element in the projection module to generate a first patterned light beam through a first timing signal;
step S102: controlling a second light-emitting element in the projection module to generate a second patterned light beam through a second timing signal, wherein the first patterned light beam and the second patterned light beam are generated alternately;
step S103: the first patterned light beam and the second patterned light beam are respectively received by a diffraction optical element in the projection module, and a first structured light pattern and a second structured light pattern are correspondingly generated.
Of course, the number of the sub-light sources in the light source 11 may also be other values, and is not limited to the above two cases, and is not limited herein.
The processor 40 controls the projection module 10 to project the plurality of sub-structured light patterns to the space, and also synchronously controls the imaging module 20 to collect the plurality of sub-structured light patterns. For example, please refer to FIG. 8:
step S20: and controlling the imaging module to collect a plurality of groups of sub-structure light patterns through a second group of time sequence control signals, wherein the first group of time sequence control signals and the second group of time sequence control signals have the same period and are synchronous.
When the light source 11 includes the first sub light source 111 and the second sub light source 112, the imaging module 20 obtains the first structured light pattern and the second structured light pattern in one capturing period under the control of the second group timing control signal S52, so as to obtain the structured light pattern obtained by superimposing the first structured light pattern and the second structured light pattern. In order to ensure that the imaging module 20 can capture the first and second structured light patterns in one capturing period, the first group of timing control signals S51 includes the first and second timing signals S511 and S512 in one period, and the first and second group of timing control signals S51 and S52 are synchronous and have the same period. Of course, in other embodiments, the timing control signal from processor 40 may take other suitable forms.
Step S30: and obtaining the structured light pattern according to the superposed areas of the plurality of groups of the sub-structured light patterns. The overlapping mode can be set according to the requirement, for example, the overlapping mode can be used, and the overlapped area is the structured light pattern; the complementary pattern is also possible, and the complementary regions are the structured light pattern.
In the depth image obtaining method provided in this embodiment, the light source 11 is divided into a plurality of sub light sources for respective control, and the processor 40 controls the plurality of sub light sources to alternately generate the patterned light beams in one projection period through the first timing control signal S51, so that at most only one light emitting element in one sub light source is turned on when the light source projects the patterned light beam at each moment, and all the light emitting elements in the light source do not need to be turned on at the same time, thereby greatly reducing the overall power, and reducing the design requirements of the supporting driving circuit and the like. Moreover, because a plurality of sub-light sources are lighted in a time-sharing manner, and the lighting time of each sub-light source does not last for one projection period, the power consumption of the light source in one period can be greatly reduced, and the requirement of the intelligent device adopting the projection module 10 on low power consumption can be effectively met. Therefore, by the depth image obtaining method of the embodiment, not only a higher resolution can be obtained, but also the power consumption of the projection module 10 can be effectively reduced.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (7)
1. A projection module, its characterized in that: the optical system comprises a light source, an optical lens and a diffraction optical element which are arranged along an optical path;
the light source comprises a first sub light source and a second sub light source, wherein the first sub light source comprises a plurality of first light emitting elements, the second sub light source comprises a plurality of second light emitting elements, the first light emitting elements are used for generating first patterned light beams, the second light emitting elements are used for generating second patterned light beams, the first patterned light beams and the second patterned light beams are generated alternately, the number of the first light emitting elements is the same as that of the second light emitting elements, and the power of the first light emitting elements is the same as that of the second light emitting elements;
the diffraction optical element receives the first patterned light beam and the second patterned light beam respectively and correspondingly generates a first structured light pattern and a second structured light pattern which can be mutually superposed, and the first structured light pattern and the second structured light pattern are complementary; the first sub light source and the second sub light source are at least partially overlapped, and the first light-emitting elements and the second light-emitting elements are distributed in a staggered mode; the first light-emitting element and the second light-emitting element are sequentially lightened in one acquisition period; when the light source projects the patterned light beam at each moment, the first sub-light source and the second sub-light source are turned on in a time-sharing manner, and the turn-on time of the first sub-light source and the second sub-light source does not last for one projection period.
2. The projection module of claim 1 wherein: the first light emitting element includes at least one of a light emitting diode, an edge-emitting laser diode, and a vertical cavity surface-emitting laser;
the second light emitting element includes at least one of a light emitting diode, an edge-emitting laser diode, and a vertical cavity surface-emitting laser.
3. The projection module of claim 1 wherein:
the first sub light source and the second sub light source are not coincident with each other.
4. A depth camera, comprising the projection module, the imaging module and the processor of any one of claims 1 to 3;
the processor is connected with the imaging module and the projection module;
the processor is used for controlling the plurality of sub light sources in the projection module to alternately generate patterned light beams and controlling the imaging module to respectively acquire a plurality of sub structured light patterns so as to acquire the structured light patterns.
5. The depth camera of claim 4, wherein: the depth camera further comprises a heat radiating fin, and the plurality of sub-light sources are respectively in heat conduction connection with the heat radiating fin.
6. The depth camera of claim 4, wherein: the depth camera further comprises a color camera module, and the color camera module is connected with the processor and used for collecting color patterns.
7. A depth image acquisition method using the projection module of claim 1, comprising:
the method comprises the following steps of controlling a projection module to alternately project a plurality of groups of sub-structured light patterns which can be mutually superposed through a first group of time sequence control signals, wherein the number of the sub-structured light patterns is two, the first group of time sequence control signals comprises a first time sequence signal and a second time sequence signal, and the step of controlling the projection module to alternately project the plurality of groups of the structured light patterns which can be mutually superposed through the first group of time sequence control signals comprises the following steps: controlling a first light-emitting element in the projection module to generate a first patterned light beam through the first timing signal, and controlling a second light-emitting element in the projection module to generate a second patterned light beam through the second timing signal, wherein the first light-emitting element and the second light-emitting element are distributed in a staggered manner, and the first light-emitting element and the second light-emitting element are sequentially lightened in an acquisition period; when the light source projects the patterned light beam at each moment, the first sub-light source and the second sub-light source are lighted in a time-sharing mode, and the lighting time of the first sub-light source and the second sub-light source does not last for one projection period; the first patterned light beam and the second patterned light beam are generated alternately, the first patterned light beam and the second patterned light beam are received by a diffractive optical element in the projection module respectively, a first structured light pattern and a second structured light pattern are generated correspondingly, and the first structured light pattern and the second structured light pattern are complementary;
controlling an imaging module to acquire the first structured light pattern and the second structured light pattern through a second group of time sequence control signals, wherein the periods of the first group of time sequence control signals and the second group of time sequence control signals are the same;
obtaining a structured light pattern from a superimposed area of the first structured light pattern and the second structured light pattern.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811589922.5A CN109597211B (en) | 2018-12-25 | 2018-12-25 | Projection module, depth camera and depth image acquisition method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811589922.5A CN109597211B (en) | 2018-12-25 | 2018-12-25 | Projection module, depth camera and depth image acquisition method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109597211A CN109597211A (en) | 2019-04-09 |
| CN109597211B true CN109597211B (en) | 2022-01-14 |
Family
ID=65964269
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811589922.5A Active CN109597211B (en) | 2018-12-25 | 2018-12-25 | Projection module, depth camera and depth image acquisition method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109597211B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110221309B (en) * | 2019-04-30 | 2021-08-17 | 深圳市光鉴科技有限公司 | 3D imaging device and electronic equipment based on asynchronous ToF discrete point cloud |
| CN110244318B (en) * | 2019-04-30 | 2021-08-17 | 深圳市光鉴科技有限公司 | 3D imaging method based on asynchronous ToF discrete point cloud |
| CN110290376B (en) * | 2019-05-24 | 2023-08-08 | 深圳市光鉴科技有限公司 | Display device with 3D camera module and electronic equipment |
| CN110275381B (en) * | 2019-06-26 | 2021-09-21 | 业成科技（成都）有限公司 | Structural light emission module and depth sensing equipment using same |
| CN111246073B (en) * | 2020-03-23 | 2022-03-25 | 维沃移动通信有限公司 | Imaging device, method and electronic equipment |
| CN111398977B (en) * | 2020-04-10 | 2021-12-03 | 深圳市灵明光子科技有限公司 | Imaging device capable of improving resolution, imaging method thereof and detection equipment |
| CN114111634A (en) * | 2021-11-12 | 2022-03-01 | 先临三维科技股份有限公司 | Structured light projection module and three-dimensional scanning device |
Family Cites Families (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060023172A1 (en) * | 2004-07-28 | 2006-02-02 | Sanyo Electric Co. | Illuminating device and projection type video display |
| JP4183663B2 (en) * | 2004-07-28 | 2008-11-19 | 三洋電機株式会社 | Illumination device and projection display device |
| JP5074278B2 (en) * | 2008-04-21 | 2012-11-14 | オリンパス株式会社 | Illumination device and illumination imaging device |
| JP4711021B2 (en) * | 2009-06-30 | 2011-06-29 | カシオ計算機株式会社 | Projection device |
| CN101737701B (en) * | 2010-01-07 | 2013-03-20 | 苏州佳世达光电有限公司 | Light-emitting diode lighting system and control method thereof, and projector |
| CN104730825B (en) * | 2012-03-15 | 2019-04-02 | 苹果公司 | Photoelectricity projection device |
| US20160377414A1 (en) * | 2015-06-23 | 2016-12-29 | Hand Held Products, Inc. | Optical pattern projector |
| JP6642817B2 (en) * | 2015-09-18 | 2020-02-12 | カシオ計算機株式会社 | Light source device and projection device |
| CN108604053B (en) * | 2015-10-21 | 2021-11-02 | 普林斯顿光电子股份有限公司 | Coding pattern projector |
| TWI601449B (en) * | 2015-11-27 | 2017-10-01 | 高準精密工業股份有限公司 | Lighting apparatus |
| CN107424188B (en) * | 2017-05-19 | 2020-06-30 | 深圳奥比中光科技有限公司 | Structured light projection module based on VCSEL array light source |
| CN106997603B (en) * | 2017-05-19 | 2020-04-17 | 深圳奥比中光科技有限公司 | Depth camera based on VCSEL array light source |
| CN108388071B (en) * | 2018-02-07 | 2024-03-08 | 奥比中光科技集团股份有限公司 | Depth camera and projection module thereof |
| CN108594453B (en) * | 2018-03-23 | 2019-12-13 | 深圳奥比中光科技有限公司 | Structured light projection module and depth camera |
| CN108828887B (en) * | 2018-06-05 | 2023-10-24 | 奥比中光科技集团股份有限公司 | Projector and depth camera |
-
2018
- 2018-12-25 CN CN201811589922.5A patent/CN109597211B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109597211A (en) | 2019-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109597211B (en) | Projection module, depth camera and depth image acquisition method | |
| US11320666B2 (en) | Integrated structured-light projector | |
| US20190068951A1 (en) | Synchronization of projected illumination with rolling shutter of image sensor | |
| US10931935B2 (en) | Structured light projection module based on VCSEL array light source | |
| CN106997603B (en) | Depth camera based on VCSEL array light source | |
| KR101926321B1 (en) | Integrated structured-light projector | |
| KR101941247B1 (en) | A surface array of light emitting elements for angular range sweep-out | |
| EP2926422B1 (en) | Laser device for projecting a structured light pattern onto a scene | |
| KR20210032359A (en) | Projectors of structured light | |
| KR102113752B1 (en) | Staggered array of light-emitting elements to sweep the angular range | |
| JP2005507571A5 (en) | ||
| JP6203436B2 (en) | Light source device | |
| CN211378086U (en) | Camera module type sensor device and camera module | |
| CN114428437A (en) | 3D projector and electronic equipment that structured light and floodlight illumination closed and put | |
| WO2022242225A1 (en) | Time of flight camera module and electronic device | |
| WO2020122628A1 (en) | Camera device | |
| TW201721271A (en) | Lighting module and application thereof | |
| CN209330218U (en) | Electronic equipment with floodlight | |
| CN111721233B (en) | Three-dimensional sensing device, light emitting module and control method thereof | |
| CN208239782U (en) | Irradiate mould group and its Multifunctional imaging device | |
| CN210093398U (en) | 3D imaging device and electronic equipment | |
| CN113572024A (en) | Light emitter, depth module and terminal | |
| KR20220015133A (en) | Distance measuring camera | |
| RU2781814C1 (en) | Camera assembly and electronic apparatus | |
| KR20170066009A (en) | Depth information extraction Apparatus and camera module includes thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 11-13 / F, joint headquarters building, high tech Zone, 63 Xuefu Road, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Applicant after: Obi Zhongguang Technology Group Co., Ltd Address before: 12 / F, joint headquarters building, high tech Zone, 63 Xuefu Road, Yuehai street, Nanshan District, Shenzhen, Guangdong 518000 Applicant before: SHENZHEN ORBBEC Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |