WO2022017441A1 - 深度数据测量设备和结构光投射装置 - Google Patents

深度数据测量设备和结构光投射装置 Download PDF

Info

Publication number
WO2022017441A1
WO2022017441A1 PCT/CN2021/107785 CN2021107785W WO2022017441A1 WO 2022017441 A1 WO2022017441 A1 WO 2022017441A1 CN 2021107785 W CN2021107785 W CN 2021107785W WO 2022017441 A1 WO2022017441 A1 WO 2022017441A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
imaging
structured light
projection
image
Prior art date
Application number
PCT/CN2021/107785
Other languages
English (en)
French (fr)
Inventor
王敏捷
梁雨时
Original Assignee
上海图漾信息科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202010712929.2A external-priority patent/CN111721239A/zh
Priority claimed from CN202110423509.7A external-priority patent/CN115218820A/zh
Priority claimed from CN202110423508.2A external-priority patent/CN115218812A/zh
Application filed by 上海图漾信息科技有限公司 filed Critical 上海图漾信息科技有限公司
Priority to US18/016,670 priority Critical patent/US20230296372A1/en
Priority to JP2023504300A priority patent/JP7508150B2/ja
Priority to EP21846474.1A priority patent/EP4187197A4/en
Publication of WO2022017441A1 publication Critical patent/WO2022017441A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2513—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object with several lines being projected in more than one direction, e.g. grids, patterns
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/22—Measuring arrangements characterised by the use of optical techniques for measuring depth
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
    • G01B11/2536—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object using several gratings with variable grating pitch, projected on the object with the same angle of incidence
    • G—PHYSICS
    • G02—OPTICS
    • G02F—OPTICAL DEVICES OR ARRANGEMENTS FOR THE CONTROL OF LIGHT BY MODIFICATION OF THE OPTICAL PROPERTIES OF THE MEDIA OF THE ELEMENTS INVOLVED THEREIN; NON-LINEAR OPTICS; FREQUENCY-CHANGING OF LIGHT; OPTICAL LOGIC ELEMENTS; OPTICAL ANALOGUE/DIGITAL CONVERTERS
    • G02F1/00—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics
    • G02F1/01—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour 
    • G02F1/13—Devices or arrangements for the control of the intensity, colour, phase, polarisation or direction of light arriving from an independent light source, e.g. switching, gating or modulating; Non-linear optics for the control of the intensity, phase, polarisation or colour  based on liquid crystals, e.g. single liquid crystal display cells
    • G02F1/133—Constructional arrangements; Operation of liquid crystal cells; Circuit arrangements
    • G02F1/136—Liquid crystal cells structurally associated with a semi-conducting layer or substrate, e.g. cells forming part of an integrated circuit
    • G02F1/1362—Active matrix addressed cells
    • G02F1/136277—Active matrix addressed cells formed on a semiconductor substrate, e.g. of silicon
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/00—2D [Two Dimensional] image generation
    • G06T11/003—Reconstruction from projections, e.g. tomography
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/521—Depth or shape recovery from laser ranging, e.g. using interferometry; from the projection of structured light
    • H—ELECTRICITY
    • H01—ELECTRIC ELEMENTS
    • H01S—DEVICES USING THE PROCESS OF LIGHT AMPLIFICATION BY STIMULATED EMISSION OF RADIATION [LASER] TO AMPLIFY OR GENERATE LIGHT; DEVICES USING STIMULATED EMISSION OF ELECTROMAGNETIC RADIATION IN WAVE RANGES OTHER THAN OPTICAL
    • H01S5/00—Semiconductor lasers
    • H01S5/10—Construction or shape of the optical resonator, e.g. extended or external cavity, coupled cavities, bent-guide, varying width, thickness or composition of the active region
    • H01S5/18—Surface-emitting [SE] lasers, e.g. having both horizontal and vertical cavities
    • H01S5/183—Surface-emitting [SE] lasers, e.g. having both horizontal and vertical cavities having only vertical cavities, e.g. vertical cavity surface-emitting lasers [VCSEL]
    • H01S5/18355—Surface-emitting [SE] lasers, e.g. having both horizontal and vertical cavities having only vertical cavities, e.g. vertical cavity surface-emitting lasers [VCSEL] having a defined polarisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/207—Image signal generators using stereoscopic image cameras using a single 2D image sensor
    • H04N13/211—Image signal generators using stereoscopic image cameras using a single 2D image sensor using temporal multiplexing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/239—Image signal generators using stereoscopic image cameras using two 2D image sensors having a relative position equal to or related to the interocular distance
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/204—Image signal generators using stereoscopic image cameras
    • H04N13/254—Image signal generators using stereoscopic image cameras in combination with electromagnetic radiation sources for illuminating objects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H04N23/13—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths with multiple sensors
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N2013/0074—Stereoscopic image analysis
    • H04N2013/0081—Depth or disparity estimation from stereoscopic image signals

Definitions

  • the invention relates to the field of three-dimensional imaging, in particular to a depth data measurement device and a structured light projection device.
  • a depth camera is a collection device that collects the depth information of a target object. This type of camera is widely used in 3D scanning, 3D modeling and other fields. For example, more and more smartphones are now equipped with depth sensors for face recognition. camera device.
  • a technical problem to be solved by the present disclosure is to provide an improved depth data measurement device, which uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data.
  • LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios.
  • the VCSEL structure can be adopted to achieve low power consumption and miniaturization of the projection device.
  • a depth data measurement device comprising: a projection device for projecting structured light to a photographic object; an imaging device for photographing the photographed object to obtain the structure A two-dimensional image frame under light irradiation, wherein the structured light projection device comprises: a laser generator for generating laser light; a liquid crystal on silicon (LCOS) device for acquiring the laser light and generating a structure for projection Light.
  • LCOS liquid crystal on silicon
  • pixel-level precision projection pattern control is performed using LCOS.
  • the switching of each pixel of the LCOS device can be controlled by, for example, a processing device to generate different projected structured light patterns. This expands the field of application of the device.
  • the laser generator includes a vertical cavity surface emitting laser (VCSEL) for generating the laser light.
  • VCSEL vertical cavity surface emitting laser
  • the vertical emission performance of the VCSEL can be utilized to further reduce the volume, power consumption and heat generation.
  • polarized light can be generated by utilizing the characteristics of the VCSEL, and the LCOS device can control the reflection of the light by adjusting the phase difference of the liquid crystal corresponding to each pixel.
  • the VCSEL may be a light-emitting array including a plurality of light-emitting units, and the VCSEL may turn off a specific row, column or light-emitting unit according to the projected structured light pattern.
  • the VCSEL itself can present various light-emitting patterns by turning on and off the light-emitting units.
  • the device may be a monocular imaging device, so the imaging device further includes: an image sensor with a fixed relative distance from the projection device, wherein the image sensor captures the two-dimensional image of the structured light obtained by shooting Image frames are used for comparison with reference structured light image frames to obtain depth data for the subject.
  • the device may also be a binocular imaging device, so the imaging device may further include: first and second image sensors with a fixed relative distance from the projection device, used for photographing the photographed object to obtain first and second two-dimensional image frames under the illumination of the structured light, wherein based on the first and second two-dimensional image frames and predetermined relative positions between the first and second image sensors The relationship obtains the depth data of the photographed object.
  • the structured light projected by the projection device is infrared structured light
  • the imaging device further includes: a visible light sensor for photographing the photographed object to obtain a two-dimensional image frame under visible light irradiation. This provides color information of the subject.
  • the LCOS device may be used to project a two-dimensionally distributed encoded discrete light spot, and the imaging device may be configured to synchronously photograph the projected two-dimensionally distributed structured light to obtain all the the two-dimensional image frame.
  • the LCOS device can also be used to project a group of structured lights with different fringe codes respectively, and the imaging device is used for synchronously photographing each projected structured light to obtain a group of two-dimensional image frames, the group of two The three-dimensional image frames are jointly used to obtain the depth data of the photographed object once.
  • the LCOS device may be used for: scanning and projecting the fringe code
  • the imaging device includes: a rolling shutter sensor that synchronously turns on pixel columns in the fringe direction corresponding to the current scanning position to perform imaging.
  • the laser generator is a VCSEL including a light-emitting array composed of a plurality of light-emitting units, and is used for: partially lighting the light-emitting unit column of the VCSEL
  • the imaging device includes: synchronously turning on and current The illuminated light-emitting unit column illuminates the pixel column in the stripe direction corresponding to the position of the rolling shutter sensor for imaging.
  • the projection device is used for: projecting the fringe coding pattern in a plurality of time periods in one imaging period, wherein each time period projects a part of the pattern, and the pattern parts projected in the plurality of time periods can be combined into one picture.
  • the imaging device is used for:
  • a pixel column of a corresponding portion of the projection pattern is turned on for imaging the projection pattern portion, and other pixel columns are turned on for imaging ambient light.
  • the projection device is used for: scanning and projecting a group of structured lights with different patterns to the shooting area, the group of structured lights includes at least two structured lights with different patterns, and the image sensor included in the imaging device is used for In: photographing the photographing object to obtain a set of image frames under the illumination of the set of structured light, which are used for single-shot depth data calculation of the photographing area, wherein the image sensor includes at least a shared At least two sub-image sensors in part of the optical path are respectively used to image the structured light of different patterns successively projected by the projection device.
  • the apparatus may further include a synchronization device for causing the at least two sub-image sensors to project at least two different patterns of structured light at a first interval smaller than a frame imaging interval of the sub-image sensor while the projection device projects the at least two sub-image sensors.
  • the image sensor simultaneously images the at least two different patterns of structured light in succession at the first interval.
  • the synchronization device may be configured to: make each sub-image sensor perform its own next frame imaging at a second interval not less than the frame imaging interval of the sub-image sensor, and synchronize with the projection of the projection device.
  • the image sensor includes: a lens unit for receiving the incident returning structured light; an optical path transforming device for changing the light path to deliver the incident returning structured light to the first sub-image sensor and the first sub-image sensor ;
  • the first sub-image sensor and the second sub-image sensor are used to image different patterns at different times.
  • a structured light projection device comprising: a vertical cavity surface emitting laser (VCSEL) for generating the laser light.
  • VCSEL vertical cavity surface emitting laser
  • LCOS liquid crystal on silicon
  • the device may further include: a diffuser, arranged on the propagation light path of the laser light, to convert the laser light generated by the VCSEL into a surface light source; a shaping optical component for converting the surface light source generated by the diffuser provided to the LCOS device; and a lens group for projecting the structured light generated by the LCOS device outward. Therefore, it can be used for structured light projection of various depth data computing devices.
  • the depth data measurement device of the present invention uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data, and is especially suitable for depth data measurement of tiny objects or details.
  • LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios.
  • a VCSEL structure can be used to achieve low power consumption and miniaturization of the projection device.
  • the VCSEL can have an array structure and can emit light partially to further reduce power consumption and device heat generation. Further, multiple sub-image sensors arranged coaxially can be used to realize fast imaging in a multi-pattern merging scenario.
  • FIG. 1 shows a schematic diagram of an example of an object to be measured.
  • Figure 2 shows a schematic diagram of discrete spots projected by the laser beam onto the surface of the object to be measured.
  • FIG. 3 shows a schematic structural diagram of a depth data measurement device according to an embodiment of the present invention.
  • Figure 4 illustrates the principle of depth imaging with fringe-encoded structured light.
  • FIG. 5 shows a schematic diagram of the composition of a depth data measurement device according to an embodiment of the present invention.
  • Figure 6 shows an example of segmented projection of a fringe coded pattern.
  • FIG. 7 shows a schematic composition diagram of a depth data measurement device according to an embodiment of the present invention.
  • FIG. 8 shows a schematic diagram of the composition of a depth data measuring head according to an embodiment of the present invention.
  • FIG. 9 shows a schematic diagram of the composition of a depth data measuring head according to an embodiment of the present invention.
  • FIG. 10 shows a comparison timing diagram of coaxial two-group binocular imaging and single-group binocular imaging.
  • FIG. 11 shows the timing diagram of coaxial three-group binocular imaging.
  • the three-dimensional measurement method based on structured light detection adopted in the present invention can perform three-dimensional measurement on the surface of an object in real time.
  • the three-dimensional measurement method based on structured light detection is a method that can perform real-time three-dimensional detection on the surface of moving objects.
  • the measurement method first projects a two-dimensional laser texture pattern with encoded information on the surface of a natural body, such as a discretized speckle pattern, and the laser texture is continuously collected by another relatively fixed image acquisition device. Compare the collected laser texture pattern with the reference surface texture pattern with known depth distance stored in the memory in advance, and calculate the projection on the surface of the natural body according to the difference between the collected texture pattern and the known reference texture pattern. The depth distance of each laser texture sequence segment is obtained, and the three-dimensional data of the surface of the object to be measured is obtained by further measurement.
  • the three-dimensional measurement method based on structured light detection adopts the method of parallel image processing, so it can detect moving objects in real time, and has the advantages of fast and accurate three-dimensional measurement, which is especially suitable for use environments with high real-time measurement requirements.
  • FIG. 1 shows a schematic diagram of an example of an object to be measured.
  • the figure schematically shows the human hand as the object to be tested.
  • Figure 2 shows a schematic diagram of discrete spots projected by the laser beam onto the surface of the object to be measured.
  • the discrete spot image shown in Figure 2 can be compared with the reference standard image to calculate the depth data of each discrete spot, and thus integrate the overall image of the object to be measured. depth data.
  • depth data As can be seen from Figure 2, since there is a certain distance between the discrete laser spots, more spot information cannot be emitted for the narrower position of the projection surface, so it is easy to lose part of the real depth information.
  • the present invention provides an improved depth data measurement device, which uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data.
  • LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios.
  • the VCSEL structure can be adopted to achieve low power consumption and miniaturization of the projection device.
  • FIG. 3 shows a schematic structural diagram of a depth data measurement device according to an embodiment of the present invention.
  • the depth data measurement apparatus 300 may include a projection device 310 and an imaging device 320 .
  • the projection device 310 is used for projecting structured light to the photographed object.
  • the imaging device 320 is used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of the structured light.
  • FIG. 3 does not show the housing and/or the fixing parts of the projection device 310, and the above-mentioned housings and/or fixing parts can be used to fix the relative positions of the various components shown in the figure, and It can have the effect of protecting the device from external contamination and external shock damage.
  • the projection device for projecting structured light mainly includes two devices: a laser generator 311 and a liquid crystal on silicon (LCOS) device 312 .
  • a laser generator 311 and a liquid crystal on silicon (LCOS) device 312 .
  • LCOS liquid crystal on silicon
  • the laser generator 311 is used to generate laser light.
  • a liquid crystal-on-silicon (LCOS) device can be used as a generator of projected patterns for capturing the laser and generating structured light for projection.
  • LCOS liquid crystal-on-silicon
  • extremely high-precision projection pattern control is performed using LCOS.
  • the switching of each pixel of the LCOS device can be controlled by, for example, a processing device inside or outside the device, so as to generate different projected structured light patterns. This expands the application scenarios of the device.
  • LCOS Liquid Crystal on Silicon
  • liquid crystal on silicon also called liquid crystal on silicon
  • CMOS complementary metal-oxide-semiconductor
  • LCOS can use a CMOS integrated circuit chip coated with liquid crystal silicon as a substrate of a reflective LCD. It is polished with advanced technology and then coated with aluminum as a mirror to form a CMOS substrate. Then, the CMOS substrate is attached to the glass substrate containing the transparent electrode, and then injected into the liquid crystal package. LCOS places the control circuit behind the display device, which can improve light transmittance, resulting in greater light output and higher resolution.
  • LCOS can be regarded as a kind of LCD.
  • the traditional LCD is made on a glass substrate, and the LCOS is made on a silicon wafer. Due to the reflective projection, the light utilization efficiency can reach more than 40%.
  • the structure of LCOS panel is similar to that of TFT LCD. After partitions are distributed between the upper and lower substrates for isolation, liquid crystal is filled between the substrates to form a light valve. The switch of the circuit drives the rotation of the liquid crystal molecules to determine the projected brightness. with dark.
  • the upper substrate of the LCOS panel may be ITO conductive glass, and the lower substrate may be a CMOS substrate coated with liquid crystal silicon.
  • the material of the lower substrate is single-crystal silicon, it has good electron mobility, and single-crystal silicon can form thin lines, so high resolution can be achieved.
  • the pixel pitch (ie, the horizontal distance between two pixels of the same color) of existing LCOS devices can be extremely small, eg, 8 to 20 micrometers (10-6).
  • the laser generator projects light of a single wavelength, such as projecting infrared light, (for example, infrared light of 940 nm), it is different from the LCOS panel commonly used to display RGB three colors in the prior art.
  • the LCOS device is used to generate projections for a pattern at one wavelength (ie, only "monochromatic"). Therefore, the LCOS device of the present invention can have a smaller pixel pitch, thereby realizing the projection of extremely fine structured light patterns.
  • the laser generator 311 may comprise or be implemented by a vertical cavity surface emitting laser (VCSEL).
  • VCSEL vertical cavity surface emitting laser
  • a VCSEL can be used to generate the laser.
  • the vertical emission performance of the VCSEL can be utilized to further reduce the volume, power consumption and heat generation.
  • the projection device 310 may further include: a diffuser 313 arranged on the propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source.
  • the LCOS device 312 is provided with the background light it needs.
  • the projection device may further include: a shaping optical component 314 for shaping the surface light source generated by the diffusing sheet (for example, shaping into a shape conforming to the LCOS device) to provide the LCOS device.
  • the projection device 310 may further include: a lens group for projecting the structured light generated by the LCOS device.
  • the laser generator and the projection lens group can be arranged on the folded optical path as shown, thereby contributing to the compactness and miniaturization of the device.
  • the laser light emitted by the laser generator 311, such as VCSEL, is sent to the LCOS 312 via the diffuser 313 and the shaping component 314, and is projected and sent out by the lens group 315 after being reflected by the relevant liquid crystal inside the LCOS 312.
  • the diffuser 313, shaping optics 314, and lens group 315 for projection are shown, in other embodiments, one or more of the above components may be omitted (eg, such that the VCSEL 311
  • the outgoing shape of the LCOS directly conforms to the cross-sectional shape required by the LCOS, so as to omit the shaping optical component 314), or replace or add other components. All of these conventional optical modifications are within the scope of the present invention.
  • the VCSEL 311 can be made to directly generate polarized light, and the LCOS device controls the reflection of light by adjusting the phase difference of the liquid crystal corresponding to each pixel. Since the LCOS 312 projects polarized light through the lens group 315, the adverse effect of specular reflection on the imaging quality of the imaging device 320 can be reduced, thereby improving the imaging quality. Further, the device can also be used for high precision flaw inspection of reflective surfaces (eg glass surfaces).
  • the projected pattern can be precisely controlled by controlling the "switch" of each pixel (eg, controlling the angle of the liquid crystal in the pixel to the incident polarized light).
  • the VCSEL 311 may also include a matrix structure, such as a light-emitting array composed of a plurality of light-emitting units. To this end, in some embodiments, the VCSEL 311 can also turn off a specific row, column or light-emitting unit according to the projected structured light pattern when emitting laser light.
  • the VCESL 311 is used as the surface light source of the LCOS 312, the light-emitting pattern of the VCESL 311 still has a certain correlation with the pattern of the surface light source received by the LCOS 312, and can be precisely fine-tuned by the LCOS 312.
  • the projection device 310 may project a fringe pattern as structured light and finely image it.
  • the determined scanning angle can be realized by LCOS.
  • Figure 4 illustrates the principle of depth imaging with fringe-encoded structured light.
  • the coding principle of stripe structured light is briefly described in the figure with two-gray-level three-bit binary time coding.
  • the projection device can sequentially project three patterns as shown in the figure to the measured object in the shooting area, and the projection space is divided into 8 areas by light and dark grayscale respectively in the three patterns.
  • Each area corresponds to its own projection angle, wherein it can be assumed that the bright area corresponds to the code "1", and the dark area corresponds to the code "0".
  • VCESL 311 may be fully lit and projected by LCOS 312 by turning off the left column of pixels corresponding to 0-3.
  • the VCESL 311 may be partially lit, eg, the lighting corresponds to the right side portion (typically need not be an exact 4-7, but could be a wider range of 3-7 segments), thereby ensuring that the LCOS 312 The pixel columns corresponding to 4-7 receive sufficient backlight and are projected by LCOS 312 by turning off the left pixel column corresponding to 0-3.
  • the power consumption of the VCSEL can be further reduced, thereby reducing the heat generated by the device, and obtaining more rest time for each light-emitting unit of the VCSEL. Therefore, it is especially suitable for use in heat-sensitive scenarios, and can prolong the life of the VCSEL.
  • the fringe light pattern projections of FIGS. 5 and 6 will be described in detail below.
  • the depth data measurement device of the present invention may be a monocular device, that is, only one camera is included to capture structured light.
  • the imaging device 320 may be implemented as an image sensor with a fixed relative distance from the projection device, wherein the two-dimensional image frame of the structured light captured by the image sensor is used for comparison with the reference structured light image frame , to obtain the depth data of the photographed object.
  • the depth data measurement device of the present invention may be a binocular device, that is, it includes two cameras to capture structured light synchronously, and uses the parallax in the two images to obtain depth data.
  • the imaging device further includes: first and second image sensors with a fixed relative distance from the projection device, used for photographing the photographed object to obtain the first and second image sensors under the illumination of the structured light. and a two-dimensional image frame, wherein the depth data of the photographed object is obtained based on the predetermined relative positional relationship between the first and second two-dimensional image frames and the first and second image sensors.
  • the above-described decoding process such as the fringe encoding shown in FIG. 4 can be simplified by directly matching the encoded values of the respective points in the first and second image sensors.
  • the number of projected patterns in the time encoding can be increased, such as a five-bit binary time encoding with two gray levels.
  • the five encoding patterns are equivalent to achieving higher-precision image matching at a higher time-domain cost. This is still quite desirable when the original projection rate of the projection device is extremely high (eg, fast switching of the LCOS projection pattern).
  • the structured light projected by the projection device is preferably infrared structured light, so as to avoid the interference of visible light.
  • the depth data measurement device of the present invention may further include: a visible light sensor, used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of visible light.
  • a visible light sensor used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of visible light.
  • an RGB sensor may be included to acquire color 2D information of the subject to combine with the acquired depth information, such as to obtain 3D information, or as a supplement or correction for deep learning.
  • the laser generator can also generate laser light in the visible light band, so that the projection device projects the structured light in the visible light band.
  • the LCOS device can be used for: projecting a group of structured lights with different fringe codes (for example, three groups as shown in FIG. 4, or more groups of fringe patterns), and the imaging device is used for synchronously photographing each projected A structured light is used to obtain a group of two-dimensional image frames, and the group of two-dimensional image frames is jointly used to obtain the depth data of the photographed object once.
  • a group of structured lights with different fringe codes for example, three groups as shown in FIG. 4, or more groups of fringe patterns
  • the projection device ie, the LCOS device combined with the laser generator
  • the LCOS device may be used in conjunction with a laser generator implemented as a VCSEL to scan project the fringe code (here, "scan projection" means that the entire image is not projected at the same time, but only projected at each time A part of the complete pattern and projections within a specific period can synthesize a complete projection pattern)
  • the imaging device includes: a rolling shutter sensor that simultaneously turns on pixel columns in the stripe direction corresponding to the current scanning position for imaging.
  • a VCSEL can turn on some of its own arrays in turn, and cooperate with the LCOS's turn-by-turn reflection (ie, the LCOS projects a structured light pattern of several turns lit up), and synchronizes with the pixel arrays of the rolling shutter sensor.
  • the heat dissipation of the VCSEL is further reduced, and the interference of ambient light on structured light imaging is avoided.
  • FIG. 5 shows a schematic diagram of the composition of a depth data measurement device according to an embodiment of the present invention.
  • the depth data measurement head 500 includes a projection device 510 and two image sensors 520_1 and 520_2.
  • the projection device 510 is used for scanning and projecting structured light with fringe coding to the shooting area. For example, in successive 3 image frame projection periods, the projection device 510 can successively project the three patterns shown in FIG. 4 , and the imaging results of the three patterns can be used to generate depth data.
  • the first and second image sensors 520_1 and 520_2 respectively have a predetermined relative positional relationship, and are used for photographing the photographing area to obtain first and second two-dimensional image frames under the illumination of structured light, respectively.
  • the first and second image sensors 520_1 and 520_2 may respectively perform the projection of the three patterns projected with the three patterns in three synchronized image frame imaging periods.
  • the photographing area (for example, the imaging plane in FIG. 5 and the area within a certain range before and after it) is imaged.
  • the projection device 510 may project linear light extending in the x-direction in the z-direction (i.e., toward the photographing area). Specifically, one or more pixel columns (line light) may be reflected from the LCOS in the projection device 510 .
  • the projected linear light can move continuously in the y-direction to cover the entire imaging area.
  • the mirrors that are turned on column by column in the LCOS can realize the continuous movement of the linear light in the y direction.
  • the LCOS will keep the mirrors in the dark fringe part off based on the currently projected fringe pattern.
  • the lower part of Figure 5 gives a more understandable illustration of the scanning of the line-shaped light for the perspective view of the shot area.
  • the direction in which the light exits the measuring head is defined as the z direction
  • the vertical direction of the photographing plane is the x direction
  • the horizontal direction is the y direction. Therefore, the striped structured light projected by the projection device may be the result of the linear light extending in the x direction moving in the y direction.
  • it is also possible to perform synchronization and imaging processing on the stripe structured light obtained by moving the linear light extending in the horizontal y direction in the x direction but in the embodiment of the present invention, it is still preferable to use vertical stripe light Be explained.
  • the measuring head 500 also includes a synchronization device 550, which can be implemented by, for example, the following processing device.
  • the synchronization device 550 is connected to the projection device 510 (including both VCSEL and LCOS) and the first and second image sensors 520_1 and 520_2, respectively, to achieve precise synchronization among the three.
  • the synchronization device 550 may, based on the scanning position of the projection device 510, simultaneously turn on the pixel columns in the stripe direction corresponding to the current scanning position in the first and second image sensors 520_1 and 520_2 to perform imaging. As shown in Figure 5, the current stripe is being scanned to the center of the shot area.
  • pixel columns eg, 3 adjacent pixel columns located in the central region are turned on for imaging.
  • the pixel columns in image sensors 520_1 and 520_2 that are turned on for imaging also move synchronously (as shown above the matrix in the upper left block diagram of FIG. 5 ) arrows).
  • the one-dimensional characteristic of the fringe image can be used to control the range of the pixel column for imaging at each moment, thereby reducing the adverse effect of ambient light on the measurement result.
  • the projection device is particularly suitable for projecting light that is not easily confused with ambient light, such as infrared light.
  • ambient light such as infrared light.
  • the range (and the corresponding number) of pixel rows that are turned on synchronously each time can be based on calibration, for example. operation to confirm.
  • the scanning and projection of the linear light can be realized by turning on the mirrors one by one or multiple columns of the LCOS as described above, or the scanning and projection of the linear light can be realized by the column-by-column or partial lighting of the VCSEL. , or a combination of the two.
  • a VCSEL can include a light-emitting array composed of a plurality of light-emitting units, and can turn off a specific row, column or light-emitting unit according to the projected structured light pattern when emitting laser light.
  • the light-emitting unit column of the VCSEL may be partially illuminated, and the rolling shutter sensor simultaneously turns on the pixel column in the stripe direction corresponding to the lighting position of the currently-lit light-emitting unit column for imaging. Partial lighting can be column-by-column lighting, or multiple columns (adjacent or spaced) lighting together, as long as the superposition of multiple lightings can illuminate the entire pattern range.
  • the LCOS can maintain the switch shape of the image to be projected, and is lit by the VCSEL column by column or block (ie, multiple columns are simultaneously lit) to realize the scanning of the fringe pattern projection.
  • the pixel columns corresponding to 0, 2, 4, and 6 in the LCOS are turned off, and the pixel columns corresponding to 1, 3, 5, and 7 are turned on.
  • the light-emitting units of the VCSEL can be lit column by column to complete linear light scanning in the y-direction with the cooperation of the LCOS.
  • the projected "line-shaped light” at this time can have a larger line width (the "bar-shaped light” separated by the dotted line in Figure 6 below), and a
  • the lighting time of the light-emitting column corresponds to the sum of the exposure times of the plurality of pixel columns in the image sensor.
  • the LCOS may also follow the portion illuminated by the light-emitting unit column to perform partial turn-on corresponding to the stripe pattern.
  • the LCOS device can also be used to project the coded discrete light spots in a two-dimensional distribution, and the imaging device is used for synchronously photographing the projected structured light in the two-dimensional distribution to obtain the two-dimensional distribution.
  • dimensional image frame For example, an LCOS device can project a discrete spot of light like the one shown in Figure 2 (but with much greater precision and typically a much smaller subject).
  • the structured light projected by the projection device is preferably infrared structured light, so as to avoid the interference of visible light.
  • the depth data measurement device of the present invention may further include: a visible light sensor, used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of visible light.
  • a visible light sensor used for photographing the photographed object to obtain a two-dimensional image frame under the illumination of visible light.
  • an RGB sensor may be included to obtain color two-dimensional information of the photographed object, so as to combine with the obtained depth information, for example, to obtain three-dimensional information, or as a supplement or correction of deep learning.
  • the structured light projected by the projection device can also be located in the visible light band, and since the instantaneous light intensity of the projected light is much larger than that of the ambient light, the structured light can still be imaged well.
  • the structured light projected by the projection device can also be located in the non-visible light band, for example, to project infrared structured light.
  • the filter in front of the image sensor can be removed, or a filter with a wider band that can pass both structured light and ambient light can be selected. .
  • the image sensor needs to be a specially fabricated sensor whose pixels can be controlled separately, and the projection device needs to be The segmented form projects a full striped structured light pattern.
  • the projection device can be used to project the fringe coding pattern in a plurality of time periods in one imaging cycle, wherein each time period projects a part of the pattern, and the pattern parts projected by the plurality of time periods can be combined into a complete fringe coding pattern.
  • the imaging device may be configured to turn on a pixel column of a corresponding portion of the projection pattern for imaging the projection pattern portion, and turn on other pixel columns for imaging ambient light in each period.
  • Figure 6 shows an example of segmented projection of a fringe coded pattern.
  • the projection period of one image frame of the projection device is 1ms, and correspondingly, the imaging period of one image frame of the imaging device is also 1ms, which is used to image a complete fringe coding pattern (shown in FIG. 6 ).
  • the example is to project the third of the three fringe patterns shown in Figure 4).
  • the projection device can divide 1 ms into 10 periods.
  • the projection device projects the pattern shown in the grid line area, ie, nothing is projected, and the pixels in the part of the imaging device corresponding to the imaging grid line area are turned on for imaging, and the remaining right Pixels in the side nine-tenths portion image ambient light.
  • the projection device projects the pattern shown in the area marked 0.2, that is, only the pattern on the right side is lit, and the pixels in the part of the imaging device corresponding to the 0.2 area are turned on for imaging, and the rest Pixels in nine-tenths of the fraction image ambient light. Then, by analogy, the projection and imaging of a complete fringe coding pattern are completed in an imaging period of 1 ms.
  • the above-mentioned segmented projection of the projection device can be realized by a laser generator, by an LCOS device, or by a combination of the two.
  • the LCOS device keeps the pixel columns corresponding to the projection pattern on and off for a period of 1ms, and the VCSEL turns on its own 10 (or 10 groups) light-emitting columns in turn, thereby realizing the completeness of the pattern. projection.
  • the light-emitting illumination area of the VCSEL only needs to cover the projection area, and the LCOS device only turns on the pixel columns in the corresponding area that should be lit in each 0.1 ms period.
  • both the light-emitting area of the VCSEL and the projection area of the LCOS change synchronously with the area of the pattern that should be projected during the period.
  • the imaging device can realize simultaneous imaging of structured light and ambient light through various control or structural schemes.
  • the imaging device has columns of pixels that are individually controllable to read, and each pixel includes a memory cell. In the 1ms imaging period, all pixels can be kept exposed, and the imaging device only needs to read the memory cells twice before and after the corresponding area is irradiated by structured light, and the ambient light with 0.9ms exposure duration is read before irradiation. Imaging information, the structured light imaging information of 0.1ms exposure duration is read after irradiation.
  • each pixel of the imaging device includes two storage units, the first storage unit is used for storing structured light exposure information, and the second storage unit is used for storing ambient light exposure information.
  • the imaging device may switch to receive exposure information from the first storage unit when the corresponding area is illuminated by structured light, and switch to receive exposure information from the second storage unit during other periods.
  • the imaging device may have a finer pixel exposure control function, so that some pixels in the same segmented exposure area are used for structured light imaging, and some pixels are used for ambient light imaging, so as to reduce the resolution of the pixels. It can realize the simultaneous imaging of structured light and ambient light of the same imaging device.
  • the imaging device may specify that odd-numbered pixel columns image structured light and even-numbered pixel columns image ambient light. When the structured light pattern is projected on the corresponding area, the odd-numbered columns are turned on for exposure, and the even-numbered columns are turned on for exposure during other periods.
  • the structured light may be further exposed for different durations, thereby achieving an HDR imaging effect.
  • odd-numbered pixels in odd-numbered pixel columns may be exposed for full-time structured light (eg, 0.1 ms), even-numbered pixels may be exposed for half-time structured light (eg, 0.05 ms), and during structured light imaging image composition, Select unexposed pixel values for display or calculation.
  • the apparatus may be implemented as a measuring head only for capturing functions, or may contain processing and computing means.
  • the measuring head and the processing and computing device can be packaged in the same housing, or connected separately via a signal transmission device.
  • the depth data measuring apparatus of the present invention may further include: a processing device (control function) connected to the projection device and the imaging device for controlling the projection of the projection device and the imaging Imaging of the device.
  • a processing device control function
  • the processing device may be used to: control the switching of pixels of the LCOS device to generate different projected structured light patterns.
  • processing device may further include a computing function and be configured to: obtain depth data of the photographed object by using the two-dimensional image frame photographed by the imaging device.
  • the depth data measuring apparatus of the present invention may further comprise: a casing for accommodating the projection device and the imaging device, and for fixing the relative positions of the projection device and the imaging device.
  • the fixture 330 shown in Figure 3 may be part of the housing.
  • processing means for control and/or computation may be included within the housing. But in some cases, the camera and processor need to be set up separately.
  • the apparatus may include: a signal transmission device connected to the projection device and the imaging device through the casing, for inwardly transmitting a control signal for the projection device and the imaging device, and transmitting the two-dimensional image frame outward.
  • the above-mentioned signal transmission device may be a signal connection line with the processing device, such as an optical fiber or a coaxial cable.
  • the above-mentioned signal transmission device may be a connection interface with the processing device.
  • FIG. 7 shows a schematic composition diagram of a depth data measurement device according to an embodiment of the present invention.
  • the depth data measurement apparatus includes a separate measurement head 700 , a signal transmission device 740 and a processor 750 .
  • the figure schematically shows a perspective view of the measuring head 700 , a schematic diagram of a cable of a signal transmission device (transmission cable) 740 and a schematic diagram of a symbol of the processor 750 .
  • the processor 750 may be surrounded by a separate processor housing, or inserted into other devices, such as the computing motherboard of the acquisition device described below, or fixed in other ways. There is no restriction on this disclosure.
  • the measuring head completes the active projection of structured light and the binocular measurement for structured light.
  • the measurement head 600 may include a structured light projection device 710 , first and second image sensors 720_1 and 720_2 having a predetermined relative positional relationship, and a housing 730 .
  • the structured light projection device 710 can be used to project structured light to the photographed object, and includes the structure of the VCSEL combined with the LCOS as described above.
  • the first and second image sensors 720_1 and 720_2 are used for photographing the photographed object to obtain first and second two-dimensional image frames under the illumination of the structured light, respectively.
  • the housing 730 is used to accommodate the structured light projection device and the first and second image sensors, and to fix the relative positions of the structured light projection device and the first and second image sensors.
  • the signal transmission device 740 can be connected with the structured light projection device and the first and second image sensors through the casing 730, and is used for transmitting (inside the casing) a signal for the structured light projection device 710 And the control signals of the first and second image sensors, and the first and second two-dimensional image frames captured by the image sensors are transmitted to the outside (outside the casing).
  • the processor 750 is connected to the signal transmission device 740 and located outside the housing 730, and is used for sending the control signal through the signal transmission device, and based on the continuously acquired first and second two-dimensional image frames and The predetermined relative positional relationship between the first and second image sensors calculates motion data of the photographed object.
  • the depth data measurement apparatus of the present invention can set the measurement head to be miniaturized, light weight and low heat dissipation by separating the measurement head from the processor (eg, processing circuit), thereby facilitating imaging in, for example, medical imaging equipment installation in the space.
  • the processor eg, processing circuit
  • the signal transmission device 740 may include a coaxial cable, whereby control signals and image data are directly transmitted through electrical signals. Further, in a high magnetic field environment such as MRI acquisition, in order to avoid using iron-nickel materials, an optical fiber can be used as the signal transmission device 740 .
  • the structured light projection device, the image sensor and the processor may each include a photoelectric converter for converting an optical signal transmitted by the optical fiber into an electrical signal, or converting a signal to be transmitted into an optical signal.
  • the present invention can also be implemented as a structured light projection device.
  • the apparatus may include: a vertical cavity surface emitting laser (VCSEL) for generating the laser light; and a liquid crystal on silicon (LCOS) device for capturing the laser light and generating structured light for projection.
  • the device may further include: a diffusion sheet, arranged on the propagation light path of the laser light, to convert the laser light generated by the VCSEL into a surface light source; a shaping optical component for converting the surface light generated by the diffusion sheet a light source is provided to the LCOS device; and a lens group for projecting the structured light generated by the LCOS device outward.
  • the above structured light projection device can cooperate with various imaging devices to realize depth data measurement and calculation for various scenes.
  • the depth data measuring apparatus uses LCOS to perform fine projection of structured light, thereby improving the imaging accuracy of depth data, and is especially suitable for measuring the depth data of tiny objects or details.
  • LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios.
  • a VCSEL structure can be used to achieve low power consumption and miniaturization of the projection device, the VCSEL can have an array structure, and can partially emit light to further reduce power consumption and device heat generation.
  • the structured light projected by LCOS is similar to the stripe light shown in Figure 4
  • multiple sets of patterns are required due to the properties of time-domain imaging of stripe light imaging, and the more patterns used to synthesize a depth image, the more The more depth information contained in the depth image.
  • the structured light projected by LCOS is similar to the discrete spot image shown in Figure 2
  • the projection of different discrete patterns for the same object can cover the projection More area in the region, and can provide higher confidence for a smaller matching window, so it is also conducive to the need for multiple projection pattern imaging to synthesize a depth image in a speckle scene.
  • multiple coaxial sensor structures can be used to image different patterns respectively, so as to improve the overall frame rate of obtaining depth data based on multiple frames, thereby solving the problem in the prior art. Due to the long acquisition time of multiple frames, dynamic imaging cannot be performed and the frame rate of depth data is too low. In addition, since an imaging device with a conventional frame rate is introduced, the use of a high-cost and high-frame-rate imaging device can be avoided while improving the depth imaging accuracy and frame rate.
  • the projection device of the depth data measurement device may be used to scan and project a group of structured lights with different patterns to the shooting area, where the group of structured lights includes at least two structured lights with different patterns.
  • the image sensor included in the imaging device can be used to: photograph the photographed object to obtain a set of image frames under the illumination of the set of structured light, so as to be used for single-shot depth data calculation of the photographed area .
  • the image sensor includes at least two sub-image sensors sharing at least part of the optical path, and the at least two sub-image sensors are respectively used to image the structured light of different patterns successively projected by the projection device.
  • the sub-image sensors included in the image sensor that share at least part of the optical path can be used to image successively projected patterns in a group of structured lights projected by the projection device in turn, until all patterns in a group of structured lights are imaged.
  • the projection device projects a group of three structured lights, as shown in FIG. 4
  • the image sensor includes two sub-image sensors.
  • the first sub-image sensor images the first pattern
  • the second sub-image sensor images the second pattern
  • the first sub-image sensor images the first pattern.
  • different patterns can be imaged by different image sensors included in the image sensor.
  • the number of sub-image sensors contained in the image sensor needs to perform secondary imaging, that is, the same sub-image sensor at different times Image different patterns.
  • the projected pattern may be a discrete light spot pattern, eg, different discrete light spot patterns resulting from switching on pixels by LCOS transformation.
  • a set of discrete light spot patterns used for synthesizing a depth image preferably have light spot distributions projected into the same area without overlapping each other, so as to jointly acquire as many deep learnings of the same object as possible.
  • imaging with fringe light patterns can be regarded as a temporal modulation scheme, that is, a depth data map is synthesized with different fringes projected at different times, with a sufficiently high QE. (Quantum Efficiency), which can charge the exposed pixels of the image sensor in microseconds. For this reason, a depth measurement device with a high anti-interference coefficient can be realized especially in combination with a rolling shutter type image sensor.
  • the image sensor may be a single image sensor as described above, ie implemented as a monocular system.
  • the image sensors may be two image sensors, ie implemented as a binocular system.
  • the monocular and binocular implementations in which the same image sensor includes multiple sub-image sensors will be described below with reference to FIG. 8 and FIG. 9 respectively.
  • FIG. 8 shows a schematic diagram of the composition of a depth data measuring head according to an embodiment of the present invention.
  • the depth data measurement head 800 based on the monocular principle includes a projection device 810 and an image sensor 820 .
  • the projection device 810 may be a device including LCOS similar to 310 shown in FIG. 3 , for scanning and projecting a group of structured lights with different patterns to the shooting area, and the group of structured lights includes at least two different patterns of structured light. Structured light.
  • the image sensor 820 is used for photographing the photographing area to obtain a group of image frames under the illumination of the group of structured lights, which are used for single-shot depth data calculation of the photographing area.
  • the projection device 810 sequentially projects three patterns as shown in FIG. 4 .
  • the three patterns are taken as a group, and each pattern is imaged by the image sensor, thereby obtaining a group of image frames including 3 frames.
  • these three frames of images can be compared with their corresponding reference image frames, and used together to calculate the depth data for the shooting area once, that is, a depth image of one frame can be calculated.
  • the image sensor 820 shown in FIG. 8 includes at least two sub-image sensors that share at least part of the optical path. , the at least two sub-image sensors are respectively used to image the structured light of different patterns successively projected by the projection device.
  • FIG. 8 shows an example in which the image sensor 820 includes two sub-image sensors (photosensitive units).
  • the image sensor 820 includes sub-image sensors 823 and 824 .
  • the sub-image sensors 823 and 824 share the optical path up to the beam splitting surface of the beam splitting device 822, and are at the same distance from the above-mentioned beam splitting area, or at least approximately the same distance.
  • the present invention introduces sensor structures that are coaxial with each other.
  • the sub-image sensor 823 can be used to image the structured light of, for example, the first pattern among the three patterns in FIG. 1 .
  • the structured light such as the second of the three patterns in FIG. 1 , can then be imaged using the sub-image sensor 824 .
  • the sub-image sensor 824 with the same optical path (or a completely equivalent optical path) is in place, and the imaging of the latter pattern structured light is performed instead of 823 . Therefore, the imaging interval of two adjacent frames can be performed at a smaller interval without depending on the frame interval of each image sensor.
  • FIG. 9 shows a schematic diagram of the composition of a depth data measurement head according to an embodiment of the present invention.
  • An example of the composition of the image sensor is given in more detail than the schematically shown projection device.
  • the depth data measurement head 900 based on the binocular principle includes a projection device 910 and a first image sensor 920 and a second image sensor 930 having a predetermined relative positional relationship.
  • the measuring head 900 may further include a housing for enclosing the above-mentioned device, and the connection structure 940 shown in FIG. 9 can be regarded as a mechanism for fixing the above-mentioned device and connecting to the housing.
  • the connection structure 940 may be a circuit board that includes control circuitry thereon. It should be understood that, in other implementations, the above-mentioned apparatuses 910-930 may be connected to the housing in other ways, and perform corresponding data transmission and instruction reception operations.
  • the projection device 910 is configured to scan and project a group of structured lights with different patterns to the shooting area, and the group of structured lights includes at least two structured lights with different patterns.
  • the first image sensor 920 and the second image sensor 930 having a predetermined relative positional relationship are used for photographing the photographing area to obtain a set of image frame pairs under the illumination of the set of structured lights for use in Single-shot depth data calculation of the shooting area.
  • the projection device 910 projects three patterns as shown in FIG. 4 in sequence (in other embodiments, it may also be three discrete light spot patterns).
  • the three patterns are taken as a group, and each pattern is imaged by the image sensor, thereby obtaining a group of image frame pairs including three pairs (6 frames).
  • These 6 frames of images are jointly used for the calculation of one-time depth data for the shooting area, that is, a depth image of one frame can be calculated.
  • each of the first and second image sensors includes only one photosensitive unit, and each photosensitive unit performs three images respectively to obtain three pairs (6 frames) of a set of image frame pairs
  • Figure 9 The illustrated first and second image sensors each include at least two sub-image sensors sharing at least part of the optical path, and the at least two sub-image sensors are respectively used for imaging different patterns of structured light successively projected by the projection device.
  • FIG. 9 shows an example in which each of the first and second image sensors includes two sub-image sensors (photosensitive units).
  • the first image sensor 920 includes sub-image sensors 923 and 924
  • the second image sensor 930 includes sub-image sensors 933 and 934 .
  • the sub-image sensors 923 and 924 share the optical path up to the beam splitting surface of the beam splitting device 922, and the distance from the above-mentioned beam splitting area is equal.
  • the sub-image sensors 933 and 934 share the optical path up to the beam-splitting surface of the beam-splitting device 932, and are equally spaced from the above-mentioned beam-splitting area.
  • the present invention introduces sets of binocular structures that are coaxial with each other.
  • the sub-image sensors 923 and 933 can be regarded as the first group of image sensors (the first group of binoculars) for imaging the structured light of the first pattern among the three patterns in FIG. 4 .
  • sub-image sensors 924 and 934 which can be regarded as a second group of image sensors (a second group of binoculars), can be used to image structured light of, for example, the second pattern of the three patterns in FIG. 4 .
  • the sub-image sensors 924 and 934 coaxial with 923 and 933 respectively are in place, and the imaging of the latter pattern structured light is performed instead of 923 and 933 . Therefore, the imaging interval of two adjacent frames can be performed at a smaller interval without depending on the frame interval of each image sensor.
  • the measuring head 800 or 900 may further include: a synchronization device for projecting at least two different patterns of structured light at a first interval smaller than a frame imaging interval of the sub-image sensor, while the projection device projects structured light of different patterns
  • the image sensor 820 or the first and second image sensors 920 and 930 each includes at least two sub-image sensors synchronously and successively imaging the at least two different patterns of structured light at the first interval, respectively. Accordingly, each sub-image sensor still performs its own next frame imaging at a second interval not less than the frame imaging interval of the sub-image sensor (for example, imaging at its own frame interval), and the above-mentioned imaging operations can be synchronized
  • the synchronization of the device is synchronized with the projection of the projection device.
  • Figure 10 shows a comparison timing diagram of coaxial two-group imaging and single-group imaging.
  • the frame rate of each photosensitive unit can be set to 100 frames/s
  • the frame interval can be set to 10ms
  • the exposure time required by each photosensitive unit can be set to 1ms.
  • the image sensor 820 or the first and second image sensors 920 and 930 are conventional image sensors including only a single photosensitive unit, when depth data calculation is to be performed using, for example, the three patterns shown in FIG. 4 , the lower part of FIG. 10 As shown, three imagings are required at 0th, 10th and 20th milliseconds. To this end, compositing each depth data image requires the subject to remain motionless for 21ms (so it is more difficult to capture moving subjects), and the frame rate has also dropped from 100 frames/s to 33.3 frames/s.
  • the image sensor includes two photosensitive units, when, for example, three patterns are to be used for depth data calculation, as shown in the upper part of FIG. Imaging, followed by the second group of photosensitive units to image pattern 2 at the 1st millisecond, and then after an interval of 10ms, the first group of photosensitive units to image pattern 3 at the 10th millisecond, thus completing a Three imaging required for sub-depth data images. Then, in the 11th millisecond, the second group of photosensitive units can start the next round of imaging for pattern 1. At the 20th millisecond, the first group of photosensitive cells imaged pattern 2 . At the 21st millisecond, the second group of photosensitive units perform imaging for pattern 3 again.
  • the imaging interval of different groups of photosensitive units only needs the time required for imaging (for example, 1 ms), and the re-imaging interval of the same group of photosensitive units still follows the frame interval time corresponding to the frame rate (for example, 10 ms).
  • the frame rate can be kept close to 66.6 fps/ s.
  • each of the first and second image sensors may further include more photosensitive units.
  • FIG. 11 shows the timing diagram of coaxial three-group binocular imaging.
  • each of the first and second image sensors may include three photosensitive units (sub-image sensors) that are coaxial.
  • the first group of photosensitive units imaged pattern 1 at the 0th millisecond, followed by the second group of photosensitive units to image pattern 2 at the 1st millisecond, followed by the third group of photosensitive units.
  • the group photosensitive unit performs imaging for pattern 3 at the second millisecond.
  • the next round of three-group imaging starts at the 10th millisecond
  • the next round of three-group imaging starts at the 20th millisecond
  • so on at this time, by introducing three sets of coaxial binoculars, it only takes 3ms to obtain three sets (6 frames) of images required to synthesize a depth data image, that is, the object only needs to remain motionless for 3ms, so the improvement is greatly improved.
  • the shooting level for moving objects, and the frame rate can be kept close to 100 frames/s (in this example, it takes 1003ms to shoot 100 frames, or 1.003 seconds).
  • the frame rate of depth data based on multi-frame synthesis can be doubled and the imaging of each frame can be shortened time.
  • coaxial binocular structures with the same number of sets of images projected by the projection device can be arranged, so that the framing time of each depth frame and the frame interval of the sensor are only related to the multiple of the exposure time (when the frame interval is greater than In the case of exposure time x number of coaxial structure groups).
  • the optical path needs to be designed.
  • the example of FIG. 9 a coaxial arrangement based on beam splitting is shown (the example of FIG. 8 also has a similar structure).
  • the first image sensor 920 may include: a mirror unit 921 for receiving the incident returning structured light; a beam splitting device 922 for dividing the incident returning structured light into at least a first light beam and a second light beam ; the first sub-image sensor 923 is used to image the first light beam; the second sub-image sensor 924 is used to image the second light beam of the returning structured light corresponding to different patterns.
  • the beam splitting device 922 is an optical prism, such as a square prism or a triangular prism.
  • the reflected infrared light in the incident light reaches the second sub-image sensor 924 , and the unreflected visible light in the incident light can propagate to the first sub-image sensor 923 in a straight line.
  • the beam splitting device 922 in the form of a prism can split the incident light into two beams whose propagation directions are perpendicular to each other.
  • the first sub-image sensor 923 and the second sub-image sensor 924 may also be vertically arranged so as to receive incident visible light and infrared light beams at a vertical angle, respectively.
  • the components in the incident light need to have the same optical path.
  • the first sub-image sensor 923 and the second sub-image sensor 924 may be arranged at equal distances from the beam splitting area of the beam splitting device 922 .
  • the distance between the two photosensitive units and the beam splitting device 922, especially the distance from the beam splitting area can be flexibly adjusted according to the ratio of the refractive index of air to the prism material.
  • Pixel-level alignment or approximate alignment between the first sub-image sensor 923 and the second sub-image sensor 924 can be theoretically achieved by making the incident light share most of the optical path and have the same optical path.
  • the actual arrangement of the first sub-image sensor 923 and the second sub-image sensor 924 cannot present ideal vertical and equidistant conditions, resulting in a deviation between the images of the two.
  • forced software correction can be performed on the manufactured image sensor.
  • true pixel-level correction is achieved by introducing a calibration target and aligning the images of both the first sub-image sensor 923 and the second sub-image sensor 924 with the calibration target.
  • the light beam before entering the first sub-image sensor 923 and the second sub-image sensor 924, the light beam may also pass through a filter to further filter out the influence of light of other wavelengths.
  • the projection device can project infrared laser light, so the filter arranged in the image sensor can be a corresponding infrared light transmission unit for transmitting infrared light in a specific frequency range, for example, the wavelength of 780- 1100nm infrared light.
  • the projection device can also project visible light, such as red laser or blue laser, such as red light at 635 nm or blue light at 450 nm.
  • the QE of 635nm red light is as high as 90% to 95%.
  • the ambient light may also include red light or blue light, due to the short exposure time and the powerful instantaneous laser light, imaging with high signal-to-noise ratio can also be performed with the help of the corresponding red or blue light filter.
  • the first and second sub-image sensors can also be implemented as corresponding visible light sensors.
  • the beam splitting device is a square prism
  • one side of the filter can be in physical contact with the square prism directly, and the other side is in physical contact with the photosensitive unit, and the photosensitive unit and the square prism are clamped in the housing, This ensures a high degree of invariance in the relative positions of the components.
  • additional visible light photosensitive cells may also be arranged in the image sensor as described above (Fig. (not shown in the figure) is used to capture the image information of the measured object, so that the image captured by the image sensor contains both the image information of the measured object and the depth information.
  • the visible light sensing unit can be a grayscale sensor or a color sensor. The grayscale sensor only captures the brightness information, and the color sensor can be used to capture the color information of the measured object.
  • the visible light sensing unit can be composed of three primary color sensing units, of which the three primary colors can be red, green, and blue (RGB) or cyan, red and yellow. Three primary colors (CMY).
  • the second image sensor 930 may also have the same structure.
  • 923 and 933 can be regarded as the first set of binoculars
  • 924 and 934 can be regarded as the second set of binoculars
  • 923 and 934 can also be regarded as the first set of binoculars
  • 924 and 934 can be regarded as the first set of binoculars.
  • the image sensors may each include: a lens unit for receiving the incident return structured light; a light path conversion device for delivering the incident return structured light to at least the first sub-path and the second sub-path; the first sub-image The sensor is used to image the returned structured light on the first sub-path; the second sub-image sensor is used to image the returned structured light corresponding to different patterns on the second sub-path.
  • the optical path conversion device may be a rotating mirror, which may reflect the incident light to the photosensitive unit 923 at the 0th millisecond, reflect the incident light to the photosensitive unit 924 at the 1st millisecond, and so on.
  • the optical path conversion device may also be a device for performing optical path conversion based on other mechanical, chemical or electrical principles.
  • the above-mentioned beam splitting device or optical path conversion device can be regarded as an optical path conversion device for changing the optical path to deliver the incident returning structured light to the first sub-image sensor and the first sub-image sensor.
  • other optical path transformation devices such as fiber guides, may also be utilized.
  • the above scheme of coaxially arranging multiple sub-image sensors can be implemented as a monocular or binocular fringe light projection scheme, or a binocular discrete light spot projection scheme.
  • the stripe pattern projection of linear light is especially suitable for combining with rolling shutter exposure, thereby realizing depth data measurement with high anti-interference coefficient, especially suitable for measuring target objects outdoors and in sunlight, for example, you can It can be realized as a car damage calibrator and so on.
  • the speckle measurement solution is suitable for depth measurement of continuous planes, for example, for loading and unloading picking or welding seam inspection in shipyards.
  • a visible light sensor may also be included in the plurality of sub-image sensors arranged coaxially, so as to be turned on, for example, when the corresponding pixel is not irradiated by actively projected infrared light, so as to obtain a visible light two-dimensional image of the photographed area.
  • the structured light projection device and the depth data measurement device of the present invention can cooperate with multiple pairs of binocular sensors sharing a light path, thereby further shortening the frame interval and improving the quality of the depth fusion data.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Optics & Photonics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Nonlinear Science (AREA)
  • Theoretical Computer Science (AREA)
  • Electromagnetism (AREA)
  • Mathematical Physics (AREA)
  • Chemical & Material Sciences (AREA)
  • Crystallography & Structural Chemistry (AREA)
  • Microelectronics & Electronic Packaging (AREA)
  • Condensed Matter Physics & Semiconductors (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

一种深度数据测量设备，包括：投射装置（310），用于向拍摄对象投射结构光；成像装置（320），用于对拍摄对象进行拍摄以获得在结构光照射下的二维图像帧，其中，结构光投射装置（310）包括：激光器发生器（311），用于生成激光；硅基液晶(LCOS)器件（312），用于获取激光并生成用于进行投射的结构光。使用LCOS进行结构光的精细投影，改善深度数据的成像精度。LCOS还可以变换包括散斑或是条纹在内的各种投影编码，从而满足各种成像场景。进一步地，可以结合采用VCSEL结构来实现投影装置的低功耗和小型化。还公开一种结构光投射装置。

Description

深度数据测量设备和结构光投射装置 技术领域
本发明涉及三维成像领域，具体地说，涉及一种深度数据测量设备和结构光投射装置。
背景技术
深度摄像头是一种采集目标物体深度信息的采集设备，这类摄像头广泛应用于三维扫描、三维建模等领域，例如，现在越来越多的智能手机上配备了用于进行人脸识别的深度摄像装置。
虽然三维成像已经是领域内多年研究的热点，但现有的深度摄像头依然具有功耗高、体积大、抗干扰能力差、无法实现精细实时成像等诸多问题。
为此，需要一种改进的深度数据测量设备。
发明内容
本公开要解决的一个技术问题是提供一种改进的深度数据测量设备，该设备使用LCOS进行结构光的精细投影，从而改善深度数据的成像精度。LCOS还可以变换包括散斑或是条纹在内的各种投影编码，从而满足各种成像场景。进一步地，可以采用VCSEL结构来实现投影装置的低功耗和小型化。
根据本公开的第一个方面，提供了一种深度数据测量设备，包括：投射装置，用于向拍摄对象投射结构光；成像装置，用于对所述拍摄对象进行拍摄以获得在所述结构光照射下的二维图像帧，其中，所述结构光投射装置包括：激光发生器，用于生成激光；硅基液晶(LCOS)器件，用于获取所述激光并生成用于进行投射的结构光。由此，利用LCOS进行像素级精度的投影图案控制。进一步地，可由例如处理装置来控制LCOS器件的每个像素开合，以产生不同的投射结构光图案。由此拓展该设备的应用 领域。
可选地，激光发生器包括：垂直腔面发射激光器(VCSEL)，用于生成所述激光。由此，能够利用VCSEL垂直发射的性能，进一步缩减体积、功耗和发热。
可选地，可以利用VCSEL的特性，产生偏振光，并且所述LCOS器件通过调整每个像素对应液晶的相位差来控制光的反射。
可选地，VCSEL可以是包括多个发光单元组成的发光阵列，并且所述VCSEL根据投射的结构光图案，关闭特定行、列或是发光单元。换句话说，VCSEL本身可以通过发光单元的亮灭呈现各种发光图案。
可选地，所述设备可以是单目成像设备，于是所述成像装置还包括：与所述投射装置相对距离固定的一个图像传感器，其中，该图像传感器拍摄获得的所述结构光的二维图像帧被用于与参考结构光图像帧相比较，以求取所述拍摄对象的深度数据。作为替换，所述设备也可以是双目成像设备，于是，所述成像装置还可以包括：与所述投射装置相对距离固定的第一和第二图像传感器，用于对所述拍摄对象进行拍摄以获得在所述结构光照射下的第一和第二二维图像帧，其中，基于所述第一和第二二维图像帧以及所述第一和第二图像传感器之间的预定相对位置关系求取所述拍摄对象的深度数据。
可选地，所述投射装置投射的结构光为红外结构光，并且所述成像装置还包括：可见光传感器，用于对所述拍摄对象进行拍摄以获得在可见光照射下的二维图像帧。由此提供拍摄对象的彩色信息。
在不同的实现中，所述LCOS器件可以用于：投射呈二维分布的经编码的离散光斑，并且，所述成像装置用于同步拍摄投射的所述呈二维分布的结构光以获取所述二维图像帧。所述LCOS器件也可以用于：分别投射具有不同条纹编码的一组结构光，并且，所述成像装置用于同步拍摄投射的每一种结构光以获取一组二维图像帧，该组二维图像帧共同用于求取一次所述拍摄对象的深度数据。
具体地，所述LCOS器件可以用于：扫描投射所述条纹编码，并且所述成像装置包括：同步开启与当前扫描位置相对应的条纹方向上的像素列进行成像的卷帘传感器。
作为补充或是替换，所述激光发生器是包括由多个发光单元组成的发光阵列的VCSEL，并且用于：部分点亮所述VCSEL的发光单元列，所述成像装置包括：同步开启与当前点亮的发光单元列照亮位置相对应的条纹方向上的像素列进行成像的卷帘传感器。
可选地，所述投影装置用于：在一个成像周期内分多个时段投射条纹编码图案，其中每个时段投射所述图案的一部分，并且所述多个时段投射的图案部分能够合并成一幅完整的条纹编码图案，所述成像装置用于：
在每个时段中开启投射图案对应部分的像素列用于对投射图案部分进行成像，并且开启其他像素列用于对环境光进行成像。
可选地，所述投影装置用于：向拍摄区域扫描投射不同图案的一组结构光，所述一组结构光中包含至少两个不同图案的结构光，所述成像装置包括的图像传感器用于：对所述拍摄对象进行拍摄以获得在所述一组结构光照射下的一组图像帧，以用于所述拍摄区域的单次深度数据计算，其中，所述图像传感器包括至少共用部分光路的至少两个子图像传感器，所述至少两个子图像传感器分别用于对所述投影装置相继投射的不同图案的结构光进行成像。
此时，设备还可以包括同步装置，用于在所述投影装置以小于所述子图像传感器的帧成像间隔的第一间隔投射至少两个不同图案的结构光的同时，使得所述至少两个子图像传感器同步地以所述第一间隔相继分别对所述至少两个不同图案的结构光进行成像。
进一步地，所述同步装置可以用于：使得每个子图像传感器以不小于所述子图像传感器的帧成像间隔的第二间隔进行自身的下一帧成像，并与所述投影装置的投射同步。
可选地，所述图像传感器包括：镜片单元，用于接收入射的返回结构光；光路变换装置，用于改变光路以将入射的返回结构光输送到第一子图像传感器和第一子图像传感器；第一子图像传感器和第二子图像传感器，用于在不同时刻对不同的图案进行成像。
根据本公开的第二个方面，提供了一种结构光投射装置，包括：垂直腔面发射激光器(VCSEL)，用于生成所述激光。硅基液晶(LCOS)器件，用于获取所述激光并生成用于进行投射的结构光。进一步地，该装置 还可以包括：扩散片，布置在所述激光的传播光路上，以将所述VCSEL生成的激光转换为面光源；整形光学组件，用于将所述扩散片产生的面光源提供给所述LCOS器件；以及透镜组，用于向外投射由所述LCOS器件生成的结构光。由此可以用于各类深度数据计算设备的结构光投射。
由此，本发明的深度数据测量设备使用LCOS进行结构光的精细投影，从而改善深度数据的成像精度，尤其适用于对微小对象或是细节的深度数据测量。LCOS还可以变换包括散斑或是条纹在内的各种投影编码，从而满足各种成像场景。可以采用VCSEL结构来实现投影装置的低功耗和小型化，VCSEL可以具有阵列结构，并且可以部分发光，以进一步降低功耗和器件发热。进一步地，可以利用同轴布置的多个子图像传感器，实现多图案合并场景下的快速成像。
附图说明
通过结合附图对本公开示例性实施方式进行更详细的描述，本公开的上述以及其它目的、特征和优势将变得更加明显，其中，在本公开示例性实施方式中，相同的参考标号通常代表相同部件。
图1示出了待测对象的一个示例的示意图。
图2示出了激光束投影到待测对象表面的离散斑点示意图。
图3示出了根据本发明一个实施例的深度数据测量设备的结构示意图。
图4示出了利用条纹编码的结构光进行深度成像的原理。
图5示出了根据本发明一个实施例的深度数据测量设备的组成示意图。
图6示出了分段投射一幅条纹编码图案的例子。
图7示出了根据本发明一个实施例的深度数据测量设备的组成示意图。
图8示出了根据本发明一个实施例的深度数据测量头的组成示意图。
图9示出了根据本发明一个实施例的深度数据测量头的组成示意图。
图10示出了同轴两组双目成像和单组双目成像的对比时序图。
图11示出了同轴三组双目成像的时序图。
具体实施方式
下面将参照附图更详细地描述本公开的优选实施方式。虽然附图中显示了本公开的优选实施方式，然而应该理解，可以以各种形式实现本公开而不应被这里阐述的实施方式所限制。相反，提供这些实施方式是为了使本公开更加透彻和完整，并且能够将本公开的范围完整地传达给本领域的技术人员。
本发明采用的基于结构光检测的三维测量方法能够实时地对物体表面进行三维测量。
基于结构光检测的三维测量方法是一种能够对运动物体表面进行实时三维检测的方法。简单地说，该测量方法首先向自然体表面投射带有编码信息的二维激光纹理图案，例如离散化的散斑图，由另一位置相对固定的图像采集装置对激光纹理进行连续采集，处理单元将采集的激光纹理图案与预先存储在存储器内的已知纵深距离的参考面纹理图案进行比较，根据所采集到的纹理图案和已知的参考纹理图案之间的差异，计算出投射在自然体表面的各个激光纹理序列片段的纵深距离，并进一步测量得出待测物表面的三维数据。基于结构光检测的三维测量方法采用并行图像处理的方法，因此能够对运动物体进行实时检测，具有能够快速、准确进行三维测量的优点，特别适用于对实时测量要求较高的使用环境。
图1示出了待测对象的一个示例的示意图。图中示意性地给出了人手作为待测对象。图2示出了激光束投影到待测对象表面的离散斑点示意图。在单目成像的场景中，拍摄得到的图2所示的离散斑点图像可以与参考标准图像进行比对，由此计算出每个离散斑点的深度数据，并由此整合出待测对象整体的深度数据。从图2中可以看出，由于离散的各个激光光斑间有一定距离，因此针对投射面较细窄的位置无法发射较多的光斑信息，这样就容易丢失部分真实深度信息。
现有技术中，缺乏能够进行精细投影的结构光投射装置，因此也就无法对精细对象进行高精度的深度数据测量。
为此，本发明提供了一种改进的深度数据测量设备，该设备使用LCOS进行结构光的精细投影，从而改善深度数据的成像精度。LCOS还可以变 换包括散斑或是条纹在内的各种投影编码，从而满足各种成像场景。进一步地，可以采用VCSEL结构来实现投影装置的低功耗和小型化。
图3示出了根据本发明一个实施例的深度数据测量设备的结构示意图。如图所示，深度数据测量设备300可以包括投射装置310和成像装置320。
投射装置310用于向拍摄对象投射结构光。成像装置320则用于对所述拍摄对象进行拍摄以获得在所述结构光照射下的二维图像帧。
为了显示出投射装置310的内部结构，图3并未示出投射装置310的壳体和/或固定件，上述壳体和/或固定件可以用于固定图示的各个器件的相对位置，并且可以起到保护器件免于外部污染和外部冲击损害的效果。
如图所示，用于投射结构光的投射装置主要包括两个器件：激光发生器311和硅基液晶(LCOS)器件312。
在此，激光发生器311用于生成激光。硅基液晶(LCOS)器件则可用作投射图案的发生装置，用于获取所述激光并生成用于进行投射的结构光。由此，利用LCOS进行极高精度的投影图案控制。进一步地，可由例如设备内部或是外部的处理装置来控制LCOS器件的每个像素开合，以产生不同的投射结构光图案。由此拓展该设备的应用场景。
在此，LCOS(Liquid Crystal on Silicon)，即液晶附硅，也叫硅基液晶，是一种基于反射模式，尺寸非常小的矩阵液晶显示装置。这种矩阵采用CMOS技术在硅芯片上加工制作而成。
具体地，LCOS可以采用涂有液晶硅的CMOS集成电路芯片作为反射式LCD的基片。用先进工艺磨平后镀上铝当作反射镜，形成CMOS基板，然后将CMOS基板与含有透明电极之上的玻璃基板相贴合，再注入液晶封装而成。LCOS将控制电路放置于显示装置的后面，可以提高透光率，从而达到更大的光输出和更高的分辨率。
LCOS可视为LCD的一种，传统的LCD是做在玻璃基板上，LCOS则是做在硅晶圆上，并且由于采用反射式投射，光利用效率可达40％以上。LCOS面板的结构类似TFT LCD，是在上下二层基板中间分布隔板以加以隔绝后，再填充液晶于基板间形成光阀，藉由电路的开关以推动液晶分子的旋转，以决定投射的明与暗。LCOS面板的上基板可以是ITO导电玻璃， 下基板则可以是涂有液晶硅的CMOS基板。由于下基板的材质是单晶硅，因此拥有良好的电子移动率，而且单晶硅可形成较细的线路，因此能够实现高解析度。现有的LCOS器件的像素间距(即，两个相同颜色像素之间的水平距离)可以极小，例如8至20微米(10-6)。
在本发明中，由于激光发生器投射单波长的光，比如投射红外光，(例如，940nm的红外光)，因此不同于现有技术中常用于显示RGB三色的LCOS面板，本发明中使用的LCOS器件用于生成针对一种波长下的图案(即，仅需“单色”)的投影。因此，本发明的LCOS器件可以具有更小的像素间距，从而实现极为精细的结构光图案的投影。
在一个实施例中，激光发生器311可以包括垂直腔面发射激光器(VCSEL)或由其实现。VCSEL可用于生成所述激光。由此，能够利用VCSEL垂直发射的性能，进一步缩减体积、功耗和发热。
进一步地，如图3所示，投射装置310还可以包括：扩散片(diffuser)313，布置在所述激光的传播光路上，以将所述VCSEL生成的激光转换为面光源。由此，为LCOS器件312提供其所需的背景光。进一步地，投射装置还可以包括：整形光学组件314，用于将所述扩散片产生的面光源进行整形(例如，整形成符合LCOS器件的形状)提供给所述LCOS器件。
另外，投射装置310还可以包括：透镜组，用于投射由所述LCOS器件生成的结构光。
由于采用了利用反射原理进行投影的LCOS，所以激光发生器和投影透镜组可以如图所示被布置在折叠的光路上，从而有助于设备的紧凑和小型化。由激光发生器311，例如VCSEL发出的激光，经由扩散片313和整形组件314，送至LCOS 312，在经LCOS 312内部相关液晶的反射后，由透镜组315投射送出。
应该理解的是，虽然图中示出了扩散片313、整形光学组件314和用于投射的透镜组315，但在其他实施例中，可以省略如上的一个或多个部件(例如，使得VCSEL 311的出射形状直接符合LCOS所需的截面形状，以省略整形光学组件314)，或是替换、添加其他的组件。所有这些常规的光学更改都位于本发明的范围之内。
进一步地，基于LCOS反射偏振光的原理，可以使得VCSEL 311直 接生成偏振光，并且所述LCOS器件通过调整每个像素对应液晶的相位差来控制光的反射。由于LCOS 312经由透镜组315投射的是偏振光，因此能够降低镜面反射对成像装置320成像质量的不利影响，从而提升成像质量。进一步地，该设备还可以用于反射表面(例如，玻璃表面)的高精度瑕疵检验。
另外，虽然LCOS 312本身是由多个像素组成的像素矩阵，并且可以通过控制每个像素的“开关”(例如，控制像素中的液晶与入射偏振光的角度)来精确控制投射图案。但是，另一方面，VCSEL 311同样可以包括矩阵结构，例如包括由多个发光单元组成的发光阵列。为此，在某些实施例中，VCSEL 311也可以在发射激光时根据投射的结构光图案，关闭特定行、列或是发光单元。换句话说，虽然VCESL 311用作LCOS 312的面光源，但是VCESL 311的发光图案与LCOS 312接收到的面光源的图案仍有一定的相关性，并且可由LCOS 312进行精确微调。
例如，在某些情况下，投射装置310可以投射条纹图案作为结构光并精细成像。根据结构光测量原理可知，能否精确地确定扫描角α是整个条纹图案测量系统的关键，在本发明中可由LCOS实现确定的扫描角，而图像编码及解码的意义就在于确定编码结构光即面结构光系统的扫描角。图4示出了利用条纹编码的结构光进行深度成像的原理。为了方便理解，图中以两灰度级三位二进制时间编码简要说明条纹结构光的编码原理。投射装置可以向拍摄区域中的被测对象依次投射如图所示的三幅图案，三幅图案中分别用亮暗两灰度将投射空间分为8个区域。每个区域对应各自的投射角，其中可以假设亮区域对应编码“1”，暗区域对应编码“0”。将投射空间中景物上一点在三幅编码图案中的编码值按投射次序组合，得到该点的区域编码值，由此确定该点所在区域进而解码获得该点的扫描角。
在投射图4最左侧图案时，在一个实施例中，VCESL 311可以完全点亮，并由LCOS 312通过关闭左侧对应于0-3的像素列来进行投射。在另一个实施例中，VCESL 311可以部分点亮，例如，点亮对应于右侧部分(通常无需是精确的4-7，而可以是范围更大的3-7部分)，由此确保LCOS 312与4-7对应的像素列接收到足够的背光，并由LCOS 312通过关闭左侧对应于0-3的像素列来进行投射。
由此，通过在投影时关闭VCSEL的部分发光单元，可以进一步地降低VCSEL的功耗，由此降低设备生成的热量，并且为VCSEL的每个发光单元获取了更多的休息时间。由此，尤其适用于在热量敏感的场景下使用，并且能够延长VCSEL的寿命。在下文中将结合图5和6的条纹光图案投射进行详述。
如图3所示，本发明的深度数据测量设备可以是单目设备，即，只包括一个摄像头来拍摄结构光。为此，成像装置320可以实现为与所述投射装置相对距离固定的一个图像传感器，其中，该图像传感器拍摄获得的所述结构光的二维图像帧被用于与参考结构光图像帧相比较，以求取所述拍摄对象的深度数据。
作为替换，本发明的深度数据测量设备可以是双目设备，即，包括两个摄像头来同步拍摄结构光，并利用两幅图中的视差求取深度数据。为此，成像装置还包括：与所述投射装置相对距离固定的第一和第二图像传感器，用于对所述拍摄对象进行拍摄以获得在所述结构光照射下的第一和第二二维图像帧，其中，基于所述第一和第二二维图像帧以及所述第一和第二图像传感器之间的预定相对位置关系求取所述拍摄对象的深度数据。
在双目成像系统中，例如图4所示条纹编码的上述解码过程可以通过对第一和第二图像传感器中的各个点的编码值直接进行匹配而得以简化。为了提升匹配精度，可以增加时间编码中投射图案的数量，例如两灰度级的五位二进制时间编码。在双目成像的应用场景下，这意味着例如左右每幅图像帧中的每个像素都包含5个或0或1的区域编码值，由此能够以更高精度(例如，像素级)实现左右图像匹配。在投射装置的投射速率不变的情况下，相比于图4的三幅编码图案，五幅编码图案相当于以更高的时域代价实现了更高精度的图像匹配。这在投射装置原本的投射速率极高的情况下(如，LCOS投射图案的快速切换)，仍然是相当可取的。
如前所述，所述投射装置投射的结构光优选为红外结构光，由此避免可见光的干扰。此时，本发明的深度数据测量设备还可以包括：可见光传感器，用于对所述拍摄对象进行拍摄以获得在可见光照射下的二维图像帧。例如，可以包括RGB传感器，以获取拍摄对象的彩色二维信息，以便与求取的深度信息组合，例如得到三维信息，或是作为深度学习的补充 或是修正。在其他实施例中，由于投射结构光的瞬间光强通常远大于环境光，因此激光发生器也可以生成位于可见光波段的激光，由此投射装置投射位于可见光波段的结构光。
LCOS器件可以用于：分别投射具有不同条纹编码的一组结构光(例如，图4所示的三组，或是更多组条纹图案)，并且，所述成像装置用于同步拍摄投射的每一种结构光以获取一组二维图像帧，该组二维图像帧共同用于求取一次所述拍摄对象的深度数据。
在某些情况下，投射装置(即，LCOS器件结合激光发生器)可以每次投射一幅完整的图案。在其他情况下，LCOS器件可以结合实现为VCSEL的激光发生器用于扫描投射所述条纹编码(在此，“扫描投射”指代整幅图像不是在同一时刻投射的，而是每一时刻仅投射完整图案的一部分并且在特定时段内的投射能够合成一幅完整的投射图案)，并且所述成像装置包括：同步开启与当前扫描位置相对应的条纹方向上的像素列进行成像的卷帘传感器。例如，VCSEL可以轮番点亮自己的某几个数列，并且配合LCOS的轮番反射(即，LCOS投射轮番点亮的几个列的结构光图案)，同时与卷帘传感器的像素列开启进行同步。由此，进一步降低VCSEL的散热，并且避免环境光对结构光成像的干扰。
图5示出了根据本发明一个实施例的深度数据测量设备的组成示意图。如图5所示，深度数据测量头500包括投影装置510以及两个图像传感器520_1和520_2。
投影装置510用于向拍摄区域扫描投射具有条纹编码的结构光。例如，在相继的3个图像帧投射周期内，投影装置510可以接连投射如图4所示的三个图案，这三个图案的成像结果可用于深度数据的生成。可以分别称为第一和第二图像传感器的520_1和520_2则具有预定相对位置关系，用于对拍摄区域进行拍摄以分别获得在结构光照射下的第一和第二二维图像帧。例如，在投影装置510投射如图1所示的三个图案的情况下，第一和第二图像传感器520_1和520_2可以在三个同步的图像帧成像周期内分别对投射有这三个图案的拍摄区域(例如，图5中的成像平面及其前后一定范围内的区域)进行成像。
如图5所示，投射装置510可以在z方向上(即，朝向拍摄区域)投 射在x方向上延伸的线型光。具体地，可由投射装置510中的LCOS反射出一个或多个像素列(线型光)。投射的线型光可以在y方向上持续移动，以覆盖整个成像区域。LCOS逐列开启的反射镜可以实现线型光在y方向上的持续移动。当然应该理解的是，LCOS在逐列开启反射镜的过程中会基于当前投射的条纹图案保持暗纹部分的反射镜关闭。图5下部针对拍摄区域的透视图对线型光的扫描给出了更易理解的图示。
在本发明的实施例中，将光线出射测量头的方向约定为z方向，拍摄平面的竖直方向为x方向，水平方向为y方向。于是，投射装置所投射的条纹结构光，可以是在x方向延伸的线型光在y方向上移动的结果。虽然在其他实施例中，也可以针对水平y方向上延伸的线型光在x方向上移动得到的条纹结构光进行同步和成像处理，但在本发明的实施例中仍优选使用竖直条纹光进行说明。
进一步地，测量头500还包括同步装置550，例如，可由下文的处理装置实现。同步装置550分别与投影装置510(包括VCSEL和LCOS两者)以及第一和第二图像传感器520_1和520_2相连接，以实现三者之间的精确同步。具体地，同步装置550可以基于投影装置510的扫描位置，同步开启第一和第二图像传感器520_1和520_2中与当前扫描位置相对应的条纹方向上的像素列进行成像。如图5所示，当前条纹正扫描至拍摄区域的中心区域。为此，图像传感器520_1和520_2中，位于中心区域的像素列(例如，3个相邻的像素列)被开启进行成像。随着条纹在y方向上的移动(如图5下部透视图中的箭头所示)，图像传感器520_1和520_2中开启用于成像的像素列也相应地同步移动(如图5左上框图中矩阵上方的箭头所示)。由此，可以利用条纹图像的一维特性，控制每一时刻进行成像的像素列的范围，从而降低环境光对测量结果的不利影响。为了进一步降低环境光的影响，投射装置尤其适用于投射不易与环境光混淆的光，例如红外光。另外，由于像素列与扫描光的对应关系受到投射光的宽度、功率、速度、图像传感器的感光效率等诸多因素的影响，因此每次同步开启的像素列范围(及对应数量)例如可以基于标定操作来确定。
在图5的例子中，可以如上所述由LCOS的逐列或多列开启反射镜来实现线型光的扫描投射，也可以是VCSEL的逐列或部分点亮来实现线型 光的扫描投射，也可以是两者的结合。
如前所述，VCSEL可以包括多个发光单元组成的发光阵列，并且可以在发射激光时根据投射的结构光图案，关闭特定行、列或是发光单元。为此，在某些实施例中，可以部分点亮VCSEL的发光单元列，而卷帘传感器则同步开启与当前点亮的发光单元列照亮位置相对应的条纹方向上的像素列进行成像。部分点亮可以是逐列点亮，也可以是多列(相邻或是间隔)一起点亮，只要多次点亮的叠加能够照射完整图案范围即可。
此时，在一整幅图像的成像时间内，LCOS可以保持所要投射图像的开关形状，并由VCSEL的逐列或是分块(即多个列同时点亮)点亮来实现条纹图案的扫描投射。例如，在投射装置需要投射图4的第三幅图案时，LCOS中对应于0、2、4、6的像素列关闭，对应于1、3、5、7的像素列打开。VCSEL的发光单元则可被逐列点亮，以在LCOS的配合下完成y方向的线型光扫描。由于VCSEL的发光单元列数远小于图像传感器的像素列时，此时投射的“线型光”可以具有更大的线宽(如下图6由虚线分隔出的“条型光”)，并且一个发光列的点亮时间对应于图像传感器中多个像素列的曝光时间之和。作为替换，LCOS也可以跟随发光单元列照亮的部分进行对应条纹图案的部分开启。在其他实施例中，LCOS器件也可以用于投射呈二维分布的经编码的离散光斑，并且，所述成像装置用于同步拍摄投射的所述呈二维分布的结构光以获取所述二维图像帧。例如，LCOS器件可以投射图2所示的离散光斑(但精度要高得多，拍摄对象也通常要小得多)。
如前所述，所述投射装置投射的结构光优选为红外结构光，由此避免可见光的干扰。此时，本发明的深度数据测量设备还可以包括：可见光传感器，用于对所述拍摄对象进行拍摄以获得在可见光照射下的二维图像帧。例如，可以包括RGB传感器，以获取拍摄对象的彩色二维信息，以便与求取的深度信息组合，例如得到三维信息，或是作为深度学习的补充或是修正。
在某些实施例中，甚至可以使用同一个图像传感器(例如，同一块CMOS成像器件)来实现在同一个成像周期内对结构光图案和环境光图像的同时获取。为此，投射装置投射的结构光也可以位于可见光波段，并且 由于投射光的瞬间光强要远大于环境光，因此仍然能够对结构光进行良好成像。投射装置投射的结构光也可以位于非可见光波段，例如，投射红外结构光，此时可以去除图像传感器前的滤波器，或者选取通过波段更宽且能够使得结构光和环境光都通过的滤波器。
在使用同一个图像传感器实现在同一个成像周期内对结构光图案和环境光图像的同时获取的实施例中，图像传感器需要是像素能被分开控制的被特别制造的传感器，并且投射装置需要以分段形式投射完整条纹结构光图案。
由此，投影装置可以用于在一个成像周期内分多个时段投射条纹编码图案，其中每个时段投射所述图案的一部分，并且所述多个时段投射的图案部分能够合并成一幅完整的条纹编码图案。相应地，成像装置可以用于在每个时段中开启投射图案对应部分的像素列用于对投射图案部分进行成像，并且开启其他像素列用于对环境光进行成像。
图6示出了分段投射一幅条纹编码图案的例子。在此，为了方便说明，可以假设投射装置的一个图像帧投射周期为1ms，相应地成像装置的一个图像帧成像周期也为1ms，用于为一幅完整的条纹编码图案成像(图6示出的例子是投射图4所示三幅条纹图案中的第三幅)。投影装置可以将1ms分为10个时段。在第0.1ms中，投影装置投射网格线区域内的所示图案，即什么也不投射，而成像装置中对应于成像网格线区域那一部分中的像素被开启进行成像，剩下的右侧十分之九部分中的像素对环境光进行成像。在第0.2ms中，投影装置投射标注为0.2的区域内的所示图案，即仅右侧被点亮的图案，而成像装置中对应于0.2区域那一部分中的像素被开启进行成像，剩下的十分之九部分中的像素对环境光进行成像。随后由此类推，从而在1ms的成像周期中完成对一幅完整的条纹编码图案的投影和成像。
投射装置的上述分段投射，可由激光发生器实现、由LCOS器件实现，或者由两者配合实现。在一个实施例中，LCOS器件在1ms的时间段中保持与投射图案相对应的像素列开启和关闭，VCSEL则依次点亮自己的10个(或10组)发光列，由此实现图案的完整投射。在另一个实施例中，VCSEL的发光照射区域仅需涵盖投射区域，LCOS器件在每个0.1ms的时段中仅开启对应区域内应该被点亮部分的像素列。在又一个实施例中， VCSEL的发光区域以及LCOS的投射区域都与该时段内应投射图案区域同步变化。
相应地，成像装置可以通过各种控制或结构方案来实现对结构光和环境光的同时成像。
在一个最简单的实施例中，成像装置具有可单独控制读取的像素列，并且每个像素包括一个存储单元。在1ms的成像周期中，全部像素可以保持曝光，并且成像装置在对应区域被结构光照射前和照射后仅需两次存储单元的读取，照射前读取的是0.9ms曝光时长的环境光成像信息，照射后读取的是0.1ms曝光时长的结构光成像信息。
在另一个实施例中，成像装置的每个像素包括两个存储单元，第一存储单元用于存储结构光曝光信息，第二存储单元用于存储环境光曝光信息。成像装置可以在对应区域被结构光照射时切换为由第一存储单元接收曝光信息，并在其他时段切换为由第二存储单元接收曝光信息。
在又一个实施例中，成像装置可以具有更为精细的像素曝光控制功能，使得同一分段曝光区域内的部分像素用于结构光成像，部分像素用于环境光成像，从而以降低分辨率的代价来实现同一成像装置的结构光与环境光同时成像。例如，成像装置可以规定奇数像素列成像结构光，偶数像素列成像环境光。在对应区域被投射结构光图案时，奇数列被开启曝光，其他时段则偶数列开启曝光。在此实施例中，还可以进一步对结构光进行不同时长的曝光，由此实现HDR成像效果。例如，奇数像素列中的奇数像素可以进行全时长的结构光曝光(例如，0.1ms)，偶数像素进行半时长的结构光曝光(例如，0.05ms)，并且在进行结构光成像图像合成时，选取未过曝的像素值进行显示或是计算。
在不同的实施例中，该设备可以实现为仅用于实现拍摄功能的测量头，也可以包含处理和计算装置。另外，在包含处理和计算设备的情况下，根据不同的应用场合，测量头与处理和计算装置可以封装在同一个壳体内，或是经由信号传输装置分开连接。
虽然图3未示出，本发明的深度数据测量设备还可以包括：与所述投射装置和所述成像装置相连的处理装置(控制功能)，用于控制所述投射装置的投影和所述成像装置的成像。例如，所述处理装置可以用于：控制 所述LCOS器件的像素开合，以产生不同的投射结构光图案。
另外，处理装置还可以包括计算功能，并且用于：利用所述成像装置拍摄的所述二维图像帧求取所述拍摄对象的深度数据。
进一步地，本发明的深度数据测量设备还可以包括：壳体，用于容纳所述投射装置和所述成像装置，并固定所述投射装置和所述成像装置的相对位置。图3所示的固定装置330可以是壳体的一部分。
在某些实施例中，用于控制和/或计算的处理装置可以包括在壳体内部。但是在某些情况下，需要将摄像头和处理器进行分立设置。此时，设备可以包括：穿过所述壳体与所述投射装置和所述成像装置连接的信号传输装置，用于用于向内传输针对所述投射装置和所述成像装置的控制信号，以及向外传输所述二维图像帧。在本发明的深度数据测量设备包括处理装置时，上述信号传输装置可以是与处理装置的信号连接线，例如光纤或是同轴电缆。在设备自身不包括处理功能时，上述信号传输装置可以是与处理装置的连接接口。
图7示出了根据本发明一个实施例的深度数据测量设备的组成示意图。
如图所示，深度数据测量设备包括单独的测量头700、信号传输装置740和处理器750。图中示意性的示出了测量头700的透视图，以及信号传输装置(传输线缆)740的线缆示意以及处理器750的符号示意图。应该理解的是，在不同的实现中，处理器750可以被单独的处理器外壳包围，或是插入其他设备，例如下文所述的采集设备的计算主板上，或是以其他方式被固定，本公开对此不作限制。
测量头在此完成结构光主动投射以及针对结构光的双目测量功能。测量头600可以包括结构光投射装置710、具有预定相对位置关系的第一和第二图像传感器720_1和720_2、以及壳体730。
结构光投射装置710可以用于向拍摄对象投射结构光，并且包括如前所述的VCSEL结合LCOS的结构。第一和第二图像传感器720_1和720_2用于对所述拍摄对象进行拍摄以各自获得在所述结构光照射下的第一和第二二维图像帧。壳体730则用于容纳所述结构光投射装置和所述第一和第二图像传感器，并固定所述结构光投射装置和所述第一和第二图像传感 器的相对位置。
信号传输装置740可以穿过所述壳体730与所述结构光投射装置和所述第一和第二图像传感器连接，用于向(壳体内)内传输针对所述所述结构光投射装置710以及第一和第二图像传感器的控制信号，以及向(壳体外)外传输图像传感器拍摄的第一和第二二维图像帧。
处理器750与信号传输装置740相连且位于所述壳体730之外，用于通过所述信号传输装置发送所述控制信号，并基于继续获取的所述第一和第二二维图像帧以及所述第一和第二图像传感器之间的所述预定相对位置关系，计算所述拍摄对象的运动数据。
由此，本发明的深度数据测量设备通过将测量头与处理器(例如，处理电路)分离，能够对测量头进行小型化、轻量化和低散热的设置，从而方便在例如医学成像设备的成像空间内的安装。
在此，信号传输装置740可以包括同轴电缆，由此直接通过电信号来进行控制信号以及图像数据从传输。进一步地，在诸如MRI采集等的高磁场环境中，为了避免采用铁镍材料，可以使用光纤作为信号传输装置740。此时，结构光投射装置、图像传感器和处理器可以各自包括光电转换器，用于将所述光纤传输的光信号转换为电信号，或是将要发送的信号转换为光信号。
在另一个实施例中，本发明也可以实现为一种结构光投射装置。该装置可以包括：垂直腔面发射激光器(VCSEL)，用于生成所述激光；以及硅基液晶(LCOS)器件，用于获取所述激光并生成用于进行投射的结构光。进一步地，该装置，还可以包括：扩散片，布置在所述激光的传播光路上，以将所述VCSEL生成的激光转换为面光源；整形光学组件，用于将所述扩散片产生的面光源提供给所述LCOS器件；以及透镜组，用于向外投射由所述LCOS器件生成的结构光。上述结构光投射装置可以与各种成像装置相配合，以实现用于各种场景的深度数据测量和计算。
上文中已经参考附图详细描述了根据本发明的深度数据测量设备和组成该设备的结构光投射装置。本发明使用LCOS进行结构光的精细投影，从而改善深度数据的成像精度，尤其适用于对微小对象或是细节的深度数据测量。LCOS还可以变换包括散斑或是条纹在内的各种投影编码，从而 满足各种成像场景。进一步地，可以采用VCSEL结构来实现投影装置的低功耗和小型化，VCSEL可以具有阵列结构，并且可以部分发光，以进一步降低功耗和器件发热。
在LCOS投射的结构光是类似图4所示条纹光的情况下，由于条纹光成像时域成像的属性而本身需要多组图案，并且用于合成一幅深度图像的图案越多，合成得到的深度图像所包含的深度信息也就越多。而在LCOS投射的结构光是类似如图2所示的离散光斑图像的情况下，虽然一幅图案就能够用于深度信息的求取，但针对同一拍摄对象的不同离散图案的投射能够覆盖投射区域内更多的面积，并且能够为更小的匹配窗口提供更高的置信度，因此在散斑场景下也有利于多幅投射图案成像来合成一幅深度图像的需求。
为此，在本发明的优选实施例中，还可以利用同轴的多组传感器结构分别对不同图案进行成像，以提升基于多帧求取深度数据的总体帧率，从而解决了现有技术中由于多帧获取时间过长而导致的无法动态成像以及深度数据帧率过低等问题。另外，由于引入的是常规帧率的成像设备，因此能够在提高深度成像精度和帧率的同时，避免对高成本高帧率成像设备的使用。
为此，在一个实施例中，深度数据测量设备的投影装置可以用于：向拍摄区域扫描投射不同图案的一组结构光，所述一组结构光中包含至少两个不同图案的结构光。所述成像装置包括的图像传感器可以用于：对所述拍摄对象进行拍摄以获得在所述一组结构光照射下的一组图像帧，以用于所述拍摄区域的单次深度数据计算。在这其中，图像传感器包括至少共用部分光路的至少两个子图像传感器，所述至少两个子图像传感器分别用于对所述投影装置相继投射的不同图案的结构光进行成像。
在此，图像传感器中包括的至少共用部分光路的子图像传感器可以用于对投影装置投射的一组结构光中相继投射出的图案进行轮流成像，直至对一组结构光中的所有图案完成成像。例如，投影装置投射一组三幅结构光，例如图4所示，而图像传感器中包括两个子图像传感器。此时，第一子图像传感器对第一幅图案进行成像，第二子图像传感器对第二幅图案进行成像，第一子图像传感器再对第一幅图案进行成像。换句话说，在一组 结构光中包含的图案数不大于图像传感器所包含的子图像传感器数的情况下，可由图像传感器中包含的不同图像传感器分别对不同的图案进行成像。而在一组结构光中包含的图案数大于图像传感器所包含的子图像传感器数的情况下，图像传感器所包含的子图像传感器数需要进行二次成像，即，同一个子图像传感器在不同的时刻对不同的图案进行成像。
在某些实施例中，投射的图案可以是离散光斑图案，例如，由LCOS变换开启像素得到的不同的离散光斑图案。用于合成一幅深度图像的一组离散光斑图案优选具有投射到同一区域内互不重叠的光斑分布，以便共同获取同一拍摄对象的尽可能多的深度学习。
应该理解的是，相比于空间调制的离散光斑图案，利用条纹光图案成像可以看作是时间调制方案，即，以不同时刻投射的不同条纹来合成一幅深度数据图，具有足够高的QE(量子效率)，能够在微秒级将图像传感器的曝光像素电荷充满。为此，尤其可以结合卷帘型图像传感器，实现高抗干扰系数的深度测量装置。
在某些实施例中，所述图像传感器可以是如上所述的单个图像传感器，即实现为单目系统。在其他实施例中，所述图像传感器可以是两个图像传感器，即实现为双目系统。如下将结合图8和图9分别对同一个图像传感器包括多个子图像传感器的单目和双目实现进行说明。
图8示出了根据本发明一个实施例的深度数据测量头的组成示意图。如图所示，基于单目原理的深度数据测量头800包括投影装置810和图像传感器820。
在此，投影装置810可以是类似于图3所示的310的包括LCOS的装置，用于向拍摄区域扫描投射不同图案的一组结构光，并且该组结构光中包含至少两个不同图案的结构光。而图像传感器820则用于对所述拍摄区域进行拍摄以获得在所述一组结构光照射下的一组图像帧，以用于所述拍摄区域的单次深度数据计算。
例如，投影装置810依次投射如图4所示的三个图案。这三个图案作为一组，并由图像传感器分别对其中的每一个图案进行成像，由此得到包括3帧的一组图像帧。根据单目成像原理，这3帧图像可以各自与其对应的参考图像帧进行比对，并共同用于针对拍摄区域的一次深度数据的计 算，即，能够计算出一帧的深度图像。
不同于常规测量头中图像传感器仅包括一块感光单元，并且一块感光单元进行三次成像来获取3帧一组的图像帧，图8所示的图像传感器820包括至少共用部分光路的至少两个子图像传感器，所述至少两个子图像传感器分别用于对所述投影装置相继投射的不同图案的结构光进行成像。
图8示出了图像传感器820包括两个子图像传感器(感光单元)的例子。如图所示，图像传感器820则包括子图像传感器823和824。在这其中，子图像传感器823和824共用光路直到分束装置822的分束面，并且与上述分束区域相距的距离相等，或者至少是距离大致相等。换句话说，本发明引入了彼此同轴的传感器结构。在此，可以使用子图像传感器823对例如图1中三个图案中的第一种图案的结构光进行成像。随后，可以使用子图像传感器824对例如图1中三个图案中的第二种图案的结构光进行成像。换句话说，此时可以看作具有相同光程(或是完全等效光路的)子图像传感器824在原地，代替823进行了后一幅图案结构光的成像。由此，相邻两帧的成像间隔就可以不依赖于每个图像传感器的帧间隔，而以更小的间隔进行成像。
类似地，图9示出了根据本发明一个实施例的深度数据测量头的组成示意图。相比于示意性示出的投射装置，图中更为详尽地给出了图像传感器的一个组成实例。
如图9所示，基于双目原理的深度数据测量头900包括投影装置910以及具有预定相对位置关系的第一图像传感器920和第二图像传感器930。虽然图中为了方便说明而没有示出，测量头900还可以包括用于包围上述装置的壳体，并且图9所示的连接结构940可以看作是固定上述装置并连接至壳体的机构。在某些实施例中，连接结构940可以是其上包括控制电路的电路板。应该理解的是，在其他实现中，上述装置910-930可以以其他方式连接至壳体，并进行相应的数据传输和指令接收操作。
在此，投影装置910用于向拍摄区域扫描投射不同图案的一组结构光，并且该组结构光中包含至少两个不同图案的结构光。而具有预定相对位置关系的第一图像传感器920和第二图像传感器930则用于对所述拍摄区域进行拍摄以获得在所述一组结构光照射下的一组图像帧对，以用于所述拍 摄区域的单次深度数据计算。
例如，投影装置910依次投射如图4所示的三个图案(在其他实施例中，也可以是三个离散光斑图案)。这三个图案作为一组，并由图像传感器分别对其中的每一个图案进行成像，由此得到包括三对(6帧)的一组图像帧对。这6帧图像共同用于针对拍摄区域的一次深度数据的计算，即，能够计算出一帧的深度图像。
不同于常规双目测量头中，第一和第二图像传感器各自仅包括一块感光单元，并且每一块感光单元分别进行三次成像来获取三对(6帧)的一组图像帧对，图9所示的第一和第二图像传感器各自包括至少共用部分光路的至少两个子图像传感器，所述至少两个子图像传感器分别用于对所述投影装置相继投射的不同图案的结构光进行成像。
图9示出了第一和第二图像传感器各自包括两个子图像传感器(感光单元)的例子。如图所示，第一图像传感器920包括子图像传感器923和924，第二图像传感器930则包括子图像传感器933和934。在这其中，子图像传感器923和924共用光路直到分束装置922的分束面，并且与上述分束区域相距的距离相等。同样地，子图像传感器933和934共用光路直到分束装置932的分束面，并且与上述分束区域相距的距离相等。换句话说，本发明引入了彼此同轴的多组双目结构。在此，可以将子图像传感器923和933看作是第一组图像传感器(第一组双目)，用于对例如图4中三个图案中的第一种图案的结构光进行成像。随后，可以将看作是第二组图像传感器(第二组双目)的子图像传感器924和934用于对例如图4中三个图案中的第二种图案的结构光进行成像。换句话说，此时可以看作分别与923和933同轴的子图像传感器924和934在原地，代替923和933进行了后一幅图案结构光的成像。由此，相邻两帧的成像间隔就可以不依赖于每个图像传感器的帧间隔，而以更小的间隔进行成像。
为此，测量头800或900还可以包括：同步装置，用于在所述投影装置以小于所述子图像传感器的帧成像间隔的第一间隔投射至少两个不同图案的结构光的同时，使得图像传感器820或是第一和第二图像传感器920和930各自包括至少两个子图像传感器同步地以所述第一间隔相继分别对所述至少两个不同图案的结构光进行成像。相应地，每一个子图像传 感器仍然以不小于所述子图像传感器的帧成像间隔的第二间隔进行自身的下一帧成像(例如，就以本身的帧间隔成像)，并且上述成像操作能够同步装置的同步下与所述投影装置的投射同步。
图10示出了同轴两组成像和单组成像的对比时序图。这里为了方便说明，可以设每一个感光单元(子图像传感器)的帧率为100帧/s，则其帧间隔为10ms，并且可以设每个感光单元所需的曝光时间为1ms。
如果图像传感器820或是第一和第二图像传感器920和930是仅包括单个感光单元的常规图像传感器，在要利用例如图4所示的三幅图案进行深度数据计算时，则如图10下部所示，需要在第0、第10和第20毫秒处进行三次成像。为此，合成每一幅深度数据图像需要拍摄对象持续21ms保持不动(因此更难以拍摄处于运动中的对象)，并且帧率也从100帧/s降至33.3帧/s。
相比之下，如果图像传感器是包括两个感光单元，在要利用例如三幅图案进行深度数据计算时，则如图10上部所示，第一组感光单元在第0毫秒处进行针对图案1的成像，紧接着第二组感光单元在第1毫秒处就进行针对图案2的成像，随后在间隔10ms之后，第一组感光单元在第10毫秒处进行针对图案3的成像，这样就完成一副深度数据图像所需的三次成像。随后，在第11毫秒，第二组感光单元就能开始下一轮针对图案1的成像。在第20毫秒，第一组感光单元进行针对图案2的成像。在第21毫秒，第二组感光单元再进行针对图案3的成像。这样，不同组感光单元成像的间隔仅需间隔成像所需时间(例如，1ms)，同一组感光单元的再次成像间隔则仍然遵循帧率对应的帧间隔时间(例如，10ms)。此时，通过引入两组同轴双目，合成每一幅深度数据图像仅需要拍摄对象持续11ms保持不动(因此更易于拍摄处于运动中的对象)，并且帧率能保持在接近66.6帧/s。
虽然结合图8-图10描述具有两组同轴(同光轴)感光单元的例子，但在其他实施例中，第一和第二图像传感器各自还可以包括更多个感光单元。图11示出了同轴三组双目成像的时序图。此时，第一和第二图像传感器各自可以包括同轴的三个感光单元(子图像传感器)。为此，如图11 所示，第一组感光单元在第0毫秒处进行针对图案1的成像，紧接着第二组感光单元在第1毫秒处就进行针对图案2的成像，紧接着第三组感光单元在第2毫秒处就进行针对图案3的成像。随后，在第10毫秒开始下一轮的三组成像，在第20毫秒开始再下一轮的三组成像，并以此类推。此时，通过引入三组同轴双目，仅需3ms就可获取合成一幅深度数据图像所需的三组(6帧)图像，即拍摄对象只需要持续3ms保持不动，因此大大提升了针对运动对象的拍摄水平，并且帧率能保持在接近100帧/s(在此例中，拍摄100帧需要1003ms，即1.003秒)。
由此，应该理解的是，仅通过引入额外的一组同轴双目结构(或单目结构)，就可以将基于多帧合成的深度数据帧率提升一倍，并缩短每一帧的成像时间。理论上，可以布置与投射装置投射图像数量相同组数的同轴双目结构，由此使得每一深度帧的成帧时间与传感器的帧间隔，仅与曝光时间的倍数相关(在帧间隔大于曝光时间x同轴结构组数的情况下)。例如，在基于四幅图案合成深度帧的情况下，如果是使用如图3所示的两组同轴双目，则获取四帧的成像时间微涨至12ms，但帧率则跌至接近50帧/s。但如果使用四组同轴双目，则获取四帧的成像时间仅为4ms，并且帧率仍然保持为接近100帧/s。但过多的引入同轴结构会增加图像传感器的构造难度，为此需要在成本、可行性和成像速度上进行折中。
为了实现同一图像传感器内不同感光单元的同轴配置，需要对光路进行设计。
在图9的例子中，示出了基于分束实现的同轴布置(图8的例子也具有类似结构)。此时，以第一图像传感器920为例，可以包括：镜片单元921，用于接收入射的返回结构光；分束装置922，用于将入射的返回结构光分成至少第一光束和第二光束；第一子图像传感器923，用于对第一光束进行成像；第二子图像传感器924，用于对对应于不同图案的返回结构光的第二光束进行成像。
在一个实施例中，分束装置922是光学棱镜，例如四方棱镜或三棱镜。由此，入射光中经反射的红外光到达第二子图像传感器924，入射光中未经反射的可见光则可进行直线传播至第一子图像传感器923。
如图所示，采用棱镜形式的分束装置922可以将入射光分成传播方向互相垂直的两束光束。相应地，第一子图像传感器923和第二子图像传感器924也可以垂直布置，以便各自以垂直角度接收入射的可见光和红外光光束。
为了消除视差并实现像素级或是接近像素级对齐，需要入射光中的成分具有相同的光程。为此，在使用四分棱镜作为分束装置922的情况下，可以将第一子图像传感器923和第二子图像传感器924布置在与分束装置922的分束区域相距相等的距离处。而在使用三棱镜作为分束装置922的情况下，则可以根据空气与棱镜材料的折射率之比，灵活调整两个感光单元与分束装置922，尤其是与分束区域的距离。
第一子图像传感器923和第二子图像传感器924之间的像素级对齐或近似对齐可以通过使得入射光共享大部分光路并具有相同的光程来理论实现。但在图像传感器的实际制造过程中，会因为第一子图像传感器923和第二子图像传感器924的实际布置无法呈现理想的垂直和等距状况而造成两者成像之间的偏差。这时，可以对制造好的图像传感器进行强制软件矫正。例如，通过引入标定靶并使得第一子图像传感器923和第二子图像传感器924的成像都与标定靶对齐，从而实现真正的像素级矫正。
在一个实施例中，光束在入射第一子图像传感器923和第二子图像传感器924之前，还可以经过滤光片，以进一步滤除其他波长的光的影响。在一个实施例中，投射装置可以投射红外激光，因此图像传感器中布置的滤光片可以是相应的红外光透射单元，用于透过特定频率范围红外光，例如本发明中使用波长为780-1100nm的红外光。在其他实施例中，投射装置也可以投射可见光，例如投射红色激光或是蓝色激光，例如635nm的红光或者450nm的蓝光。例如，相比于QE仅为20％的830nm红外光，635nm红光的QE高达90％～95％。虽然环境光中可能也包括红光或是蓝光，但是由于曝光时间短且激光瞬时光强大，因此也能够在对应的投射红光或是蓝光的滤光片的帮助下进行高信噪比的成像。在投射装置投射可见光，例如红光的情况下，第一和第二子图像传感器也可以实现为相应地可见光传感器。
优选地，在分束装置是四方棱镜的情况下，滤光片的一侧可以直接与四方棱镜物理接触，另一侧与感光单元物理接触，而感光单元和四方棱镜则卡接在壳体内，由此确保各器件相对位置的高度不变性。
在某些实施例中，尤其是在第一和第二子图像传感器是用于接收投射的红外图案的红外光传感器的情况下，图像传感器中还可以如上所述布置额外的可见光感光单元(图中未示出)用来捕获被测物体的图像信息，从而使得图像传感器捕获的图像中既包含被测物体的图像信息又包含深度信息。可见光感应单元可以是灰度传感器，或是彩色传感器。其中灰度传感器仅捕获亮度信息，彩色传感器则可用于捕获被测物体的色彩信息，此时可见光感应单元可由三原色感应单元组成，其中三原色可以是红绿蓝三原色(RGB)也可以是青红黄三原色(CMY)。
应该理解的是，虽然基于图9具体描述的第一图像传感器920的结构，但第二图像传感器930也可以具有相同的结构。另外，应该理解的是，可以将923和933看作是第一组双目，924和934看作是第二组双目，但也可以将923和934看作第一组双目，924和933看所第二组双目，只要在相应的图案入射后接通成像即可。
在如图9所示利用分束实现光路共享的情况下，由于每一个感光单元获取的光亮会减少，为此可以通过增加投射亮度或是扩大入射光圈的方法来确保成像的敏感性或是有效距离范围。
为此，作为替换，还可以基于光路转换来实现光路共享。此时，图像传感器可以各自包括：镜片单元，用于接收入射的返回结构光；光路转换装置，用于将入射的返回结构光输送至至少第一子路径和第二子路径；第一子图像传感器，用于在第一子路径上对返回结构光进行成像；第二子图像传感器，用于在第二子路径上对对应于不同图案的返回结构光进行成像。在一个实施例中，光路转换装置可以是转镜，其可以在例如第0毫秒将入射光反射至感光单元923，在第1毫秒将入射光反射至感光单元924等等。在其他实施例中，光路转换装置也可以是基于其他机械、化学或电学原理进行光路转换的装置。
如上所述的分束装置或是光路转换装置都可以看作是来改变光路以 将入射的返回结构光输送到第一子图像传感器和第一子图像传感器的光路变换装置。在其他实施例中，还可以利用诸如光纤引导装置之类的其他光路变换装置。
如上同轴布置多个子图像传感器的方案，可以实现为单目或双目的条纹光投射方案，或是双目的离散光斑投射方案。在这其中，线型光的条纹图案投射尤其适合与卷帘式曝光相结合，由此实现高抗干扰系数的深度数据测量，尤其适于在室外，在阳光下对目标对象进行测量，例如可以实现为汽车定损仪等等。散斑测量方案则适用于对连续平面的深度测量，例如，用于上下料抓取或是船厂的焊缝检测等。在某些实施例中，还可以在同轴布置的多个子图像传感器中包括可见光传感器，以例如在对应像素未被主动投射的红外光照射时开启，以获取被摄区域的可见光二维图像。
由上可知，本发明的结构光投射装置和深度数据测量设备能够与共有光路的多对双目传感器相配合，从而进一步缩短帧间隔，提升深度融合数据的质量。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。

Claims (17)

  1. 一种深度数据测量设备，包括：
    投射装置，用于向拍摄对象投射结构光；
    成像装置，用于对所述拍摄对象进行拍摄以获得在所述结构光照射下的二维图像帧，
    其中，所述投射装置包括：
    激光发生器，用于生成激光；
    硅基液晶(LCOS)器件，用于获取所述激光并生成用于进行投射的结构光。
  2. 如权利要求1所述的设备，其中，所述激光发生器包括：
    垂直腔面发射激光器(VCSEL)，用于生成所述激光。
  3. 如权利要求2所述的设备，其中，所述VCSEL生成偏振光，并且所述LCOS器件通过调整每个像素对应液晶的相位差来控制光的反射。
  4. 如权利要求2所述的设备，其中，所述VCSEL包括由多个发光单元组成的发光阵列，并且所述VCSEL在发射激光时根据投射的结构光图案，关闭特定行、列或是发光单元。
  5. 如权利要求1所述的设备，其中，所述成像装置还包括：
    与所述投射装置相对距离固定的一个图像传感器，其中，该图像传感器拍摄获得的所述结构光的二维图像帧被用于与参考结构光图像帧相比较，以求取所述拍摄对象的深度数据，或者
    与所述投射装置相对距离固定的第一和第二图像传感器，用于对所述拍摄对象进行拍摄以获得在所述结构光照射下的第一和第二二维图像帧，其中，基于所述第一和第二二维图像帧以及所述第一和第二图像传感器之间的预定相对位置关系求取所述拍摄对象的深度数据。
  6. 如权利要求1所述的设备，其中，所述投射装置投射的结构光为红外结构光，并且所述深度数据测量设备还包括：
    可见光传感器，用于对所述拍摄对象进行拍摄以获得在可见光照射下的二维图像帧。
  7. 如权利要求1所述的设备，其中，所述LCOS器件用于：
    投射呈二维分布的经编码的离散光斑，
    并且，所述成像装置用于同步拍摄投射的所述呈二维分布的结构光以获取所述二维图像帧。
  8. 如权利要求1所述的设备，其中，所述LCOS器件用于：
    分别投射具有不同条纹编码的一组结构光，
    并且，所述成像装置用于拍摄投射的每一种结构光以获取一组二维图像帧，该组二维图像帧共同用于求取一次所述拍摄对象的深度数据。
  9. 如权利要求8所述的设备，其中，所述LCOS器件用于：
    扫描投射所述条纹编码，并且
    所述成像装置包括：
    同步开启与当前扫描位置相对应的条纹方向上的像素列进行成像的卷帘传感器。
  10. 如权利要求8所述的设备，其中，所述激光发生器是包括由多个发光单元组成的发光阵列的VCSEL，并且用于：
    部分点亮所述VCSEL的发光单元列，
    所述成像装置包括：
    同步开启与当前点亮的发光单元列照亮位置相对应的条纹方向上的像素列进行成像的卷帘传感器。
  11. 如权利要求8所述的设备，其中，所述投影装置用于：
    在一个成像周期内分多个时段投射条纹编码图案，其中每个时段投射 所述图案的一部分，并且所述多个时段投射的图案部分能够合并成一幅完整的条纹编码图案，
    所述成像装置用于：
    在每个时段中，开启投射图案对应部分的像素列用于对投射图案部分进行成像，并且开启其他像素列用于对环境光进行成像。
  12. 如权利要求1所述的设备，其中，所述投影装置用于：
    向拍摄区域投射不同图案的一组结构光，所述一组结构光中包含至少两个不同图案的结构光，
    所述成像装置包括的图像传感器用于：
    对所述拍摄对象进行拍摄以获得在所述一组结构光照射下的一组图像帧，以用于所述拍摄区域的单次深度数据计算，
    其中，所述图像传感器包括至少共用部分光路的至少两个子图像传感器，所述至少两个子图像传感器分别用于对所述投影装置相继投射的不同图案的结构光进行成像。
  13. 如权利要求12所述的设备，还包括：
    同步装置，用于在所述投影装置以小于所述子图像传感器的帧成像间隔的第一间隔投射至少两个不同图案的结构光的同时，使得所述至少两个子图像传感器同步地以所述第一间隔相继分别对所述至少两个不同图案的结构光进行成像。
  14. 如权利要求13所述的设备，其中，所述同步装置用于：
    使得每个子图像传感器以不小于所述子图像传感器的帧成像间隔的第二间隔进行自身的下一帧成像，并与所述投影装置的投射同步。
  15. 如权利要求12所述的深度数据测量头，其中，所述图像传感器包括：
    镜片单元，用于接收入射的返回结构光；
    光路变换装置，用于改变光路以将入射的返回结构光输送到第一子图 像传感器和第一子图像传感器；
    第一子图像传感器和第二子图像传感器，用于在不同时刻对不同的图案进行成像。
  16. 一种结构光投射装置，包括：
    垂直腔面发射激光器(VCSEL)，用于生成所述激光。
    硅基液晶(LCOS)器件，用于获取所述激光并生成用于进行投射的结构光。
  17. 如权利要求16所述的装置，还包括：
    扩散片，布置在所述激光的传播光路上，以将所述VCSEL生成的激光转换为面光源；
    整形光学组件，用于将所述扩散片产生的面光源提供给所述LCOS器件；以及
    透镜组，用于向外投射由所述LCOS器件生成的结构光。
PCT/CN2021/107785 2020-07-22 2021-07-22 深度数据测量设备和结构光投射装置 WO2022017441A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US18/016,670 US20230296372A1 (en) 2020-07-22 2021-07-22 Depth data measuring device and structured light projection unit
JP2023504300A JP7508150B2 (ja) 2020-07-22 2021-07-22 深度データ測定機器及び構造化光投影ユニット
EP21846474.1A EP4187197A4 (en) 2020-07-22 2021-07-22 DEPTH DATA MEASUREMENT DEVICE AND STRUCTURED LIGHT PROJECTION DEVICE

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202010712929.2 2020-07-22
CN202010712929.2A CN111721239A (zh) 2020-07-22 2020-07-22 深度数据测量设备和结构光投射装置
CN202110423508.2 2021-04-20
CN202110423509.7A CN115218820A (zh) 2021-04-20 2021-04-20 结构光投射装置、深度数据测量头、计算设备和测量方法
CN202110423508.2A CN115218812A (zh) 2021-04-20 2021-04-20 深度数据测量头、计算装置及其对应方法
CN202110423509.7 2021-04-20

Publications (1)

Publication Number Publication Date
WO2022017441A1 true WO2022017441A1 (zh) 2022-01-27

Family

ID=79728500

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/107785 WO2022017441A1 (zh) 2020-07-22 2021-07-22 深度数据测量设备和结构光投射装置

Country Status (4)

Country Link
US (1) US20230296372A1 (zh)
EP (1) EP4187197A4 (zh)
JP (1) JP7508150B2 (zh)
WO (1) WO2022017441A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363525A (zh) * 2022-03-17 2022-04-15 杭州灵西机器人智能科技有限公司 Hdr快速自动成像方法、结构光相机、电子设备和存储介质

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117528063A (zh) * 2023-11-21 2024-02-06 深圳市三劲科技有限公司 一种手机摄像头景深测量的光学装置及方法

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102621546A (zh) * 2012-03-28 2012-08-01 浙江大学 基于关联成像的三维信息获取方法
CN103868472A (zh) * 2013-12-23 2014-06-18 黑龙江科技大学 一种用于高反射率零件的面结构光三维测量装置与方法
CN104197861A (zh) * 2014-08-25 2014-12-10 深圳大学 基于结构光灰度向量的三维数字成像方法
US20150070472A1 (en) * 2013-09-11 2015-03-12 National Taiwan University Measuring apparatus for three-dimensional profilometry and method thereof
CN204902786U (zh) * 2015-05-26 2015-12-23 东莞市盟拓光电科技有限公司 可降低被测物表面反光的三维测量系统
CN110501836A (zh) * 2018-05-18 2019-11-26 扬明光学股份有限公司 图案产生装置及其制造方法
CN111721238A (zh) * 2020-07-22 2020-09-29 上海图漾信息科技有限公司 深度数据测量设备和目标对象数据采集方法
CN111721239A (zh) * 2020-07-22 2020-09-29 上海图漾信息科技有限公司 深度数据测量设备和结构光投射装置
CN212779132U (zh) * 2020-07-22 2021-03-23 上海图漾信息科技有限公司 深度数据测量设备和结构光投射装置
CN212779131U (zh) * 2020-07-22 2021-03-23 上海图漾信息科技有限公司 深度数据测量设备

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4188515B2 (ja) 1999-10-04 2008-11-26 株式会社ミツトヨ 光学式形状測定装置
WO2009040822A2 (en) * 2007-09-25 2009-04-02 Explay Ltd. Micro-projector
US8717416B2 (en) * 2008-09-30 2014-05-06 Texas Instruments Incorporated 3D camera using flash with structured light
EP3637045B1 (en) 2017-06-07 2023-05-24 Sony Semiconductor Solutions Corporation Information processing device and method
US11605172B2 (en) * 2017-12-08 2023-03-14 Arizona Board Of Regents On Behalf Of The University Of Arizona Digital fringe projection and multi-spectral polarization imaging for rapid 3D reconstruction
CN108650447B (zh) * 2018-07-06 2024-03-05 上海图漾信息科技有限公司 图像传感器、深度数据测量头及测量系统

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102621546A (zh) * 2012-03-28 2012-08-01 浙江大学 基于关联成像的三维信息获取方法
US20150070472A1 (en) * 2013-09-11 2015-03-12 National Taiwan University Measuring apparatus for three-dimensional profilometry and method thereof
CN103868472A (zh) * 2013-12-23 2014-06-18 黑龙江科技大学 一种用于高反射率零件的面结构光三维测量装置与方法
CN104197861A (zh) * 2014-08-25 2014-12-10 深圳大学 基于结构光灰度向量的三维数字成像方法
CN204902786U (zh) * 2015-05-26 2015-12-23 东莞市盟拓光电科技有限公司 可降低被测物表面反光的三维测量系统
CN110501836A (zh) * 2018-05-18 2019-11-26 扬明光学股份有限公司 图案产生装置及其制造方法
CN111721238A (zh) * 2020-07-22 2020-09-29 上海图漾信息科技有限公司 深度数据测量设备和目标对象数据采集方法
CN111721239A (zh) * 2020-07-22 2020-09-29 上海图漾信息科技有限公司 深度数据测量设备和结构光投射装置
CN212779132U (zh) * 2020-07-22 2021-03-23 上海图漾信息科技有限公司 深度数据测量设备和结构光投射装置
CN212779131U (zh) * 2020-07-22 2021-03-23 上海图漾信息科技有限公司 深度数据测量设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4187197A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363525A (zh) * 2022-03-17 2022-04-15 杭州灵西机器人智能科技有限公司 Hdr快速自动成像方法、结构光相机、电子设备和存储介质
CN114363525B (zh) * 2022-03-17 2022-06-28 杭州灵西机器人智能科技有限公司 Hdr快速自动成像方法、结构光相机、电子设备和存储介质

Also Published As

Publication number Publication date
JP2023535916A (ja) 2023-08-22
JP7508150B2 (ja) 2024-07-01
US20230296372A1 (en) 2023-09-21
EP4187197A1 (en) 2023-05-31
EP4187197A4 (en) 2023-12-20

Similar Documents

Publication Publication Date Title
JP7146004B2 (ja) 同期スピニングｌｉｄａｒおよびローリングシャッターカメラシステム
US6600168B1 (en) High speed laser three-dimensional imager
WO2022017441A1 (zh) 深度数据测量设备和结构光投射装置
US7078720B2 (en) Range finder for measuring three-dimensional geometry of object and method thereof
CN107783353B (zh) 用于捕捉立体影像的装置及系统
CN111721239A (zh) 深度数据测量设备和结构光投射装置
US20040125205A1 (en) System and a method for high speed three-dimensional imaging
CN207991480U (zh) 微型结构光三维测量装置
CN111692987B (zh) 深度数据测量头、测量装置和测量方法
US10091496B2 (en) Systems, devices, and methods for calibrating a light field projection system
CN113534596B (zh) Rgbd立体相机及成像方法
WO2023193408A1 (zh) 激光雷达及激光雷达控制方法
US11326874B2 (en) Structured light projection optical system for obtaining 3D data of object surface
JP2010085472A (ja) 画像投影・撮像装置
CN108616698B (zh) 成像装置
CN212779132U (zh) 深度数据测量设备和结构光投射装置
US20240167811A1 (en) Depth data measuring head, computing device and measurement method
CN216246133U (zh) 结构光投射装置、深度数据测量头和计算设备
WO2022222497A1 (zh) 深度数据测量头、计算装置及其对应方法
CN216283296U (zh) 深度数据测量头和深度数据计算设备
CN117369197B (zh) 3d结构光模组、成像系统及获得目标物体深度图的方法
CN219417971U (zh) 基于双目视差具有深度信息采集功能的投影光机及投影仪
CN117804342A (zh) 一种三维扫描仪和三维扫描方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21846474

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2023504300

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2021846474

Country of ref document: EP

Effective date: 20230222