CN111721239A

CN111721239A - Depth data measuring device and structured light projection apparatus

Info

Publication number: CN111721239A
Application number: CN202010712929.2A
Authority: CN
Inventors: 王敏捷; 梁雨时
Original assignee: Shanghai Tuyang Information Technology Co ltd
Current assignee: Shanghai Tuyang Information Technology Co ltd
Priority date: 2020-07-22
Filing date: 2020-07-22
Publication date: 2020-09-29

Abstract

A depth data measuring apparatus and a structured light projecting device included therein are disclosed. The apparatus comprises: a projection device for projecting the structured light to a photographic subject; an imaging device for photographing the photographic subject to obtain a two-dimensional image frame under the structured light irradiation, wherein the structured light projecting device includes: a laser generator for generating laser light; a Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection. The invention uses LCOS to carry out fine projection of structured light, and improves the imaging precision of depth data. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure can be employed in combination to achieve low power consumption and miniaturization of the projection apparatus.

Description

Depth data measuring device and structured light projection apparatus

Technical Field

The invention relates to the field of three-dimensional imaging, in particular to a depth data measuring device and a structured light projection device.

Background

The depth camera is an acquisition device for acquiring depth information of a target object, and is widely applied to the fields of three-dimensional scanning, three-dimensional modeling and the like, for example, more and more smart phones are equipped with depth cameras for face recognition.

Although three-dimensional imaging is a hot point of research in the field for many years, the existing depth camera still has the problems of high power consumption, large volume, poor anti-interference capability, incapability of realizing fine real-time imaging and the like.

For this reason, there is a need for an improved depth data measuring device.

Disclosure of Invention

It is an object of the present disclosure to provide an improved depth data measuring device using LCOS for fine projection of structured light, thereby improving the imaging accuracy of the depth data. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure can be employed to achieve low power consumption and miniaturization of the projection apparatus.

According to a first aspect of the present disclosure, there is provided a depth data measuring apparatus comprising: a projection device for projecting the structured light to a photographic subject; an imaging device for photographing the photographic subject to obtain a two-dimensional image frame under the structured light irradiation, wherein the structured light projecting device includes: a laser generator for generating laser light; a Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection. Thereby, the projection pattern control with pixel level accuracy is performed by the LCOS. Further, each pixel of the LCOS device may be controlled, for example, by a processing device, to open and close to produce a different projected structured light pattern. Thereby expanding the application field of the device.

Optionally, the laser generator comprises: a Vertical Cavity Surface Emitting Laser (VCSEL) to generate the laser light. Thus, the vertical emission performance of the VCSEL can be utilized to further reduce the volume, power consumption, and heat generation.

Alternatively, the projection device may include: a diffusion sheet disposed on a propagation path of the laser light to convert the laser light generated by the VCSEL into a surface light source. The projection device may further include: and the shaping optical assembly is used for providing the surface light source generated by the diffusion sheet to the LCOS device.

Alternatively, the characteristics of the VCSEL may be utilized to generate polarized light, and the LCOS device controls the reflection of light by adjusting the phase difference of the liquid crystal corresponding to each pixel.

Alternatively, the VCSELs may be a light emitting array comprising a plurality of light emitting cells, and the VCSELs turn off specific rows, columns or light emitting cells according to a projected structured light pattern. In other words, the VCSEL itself can emit various light emission patterns

Optionally, the projection device may further include: a lens group to project structured light generated by the LCOS device.

Optionally, the device may be a monocular imaging device, and then the imaging apparatus further comprises: and the image sensor is fixed in relative distance with the projection device, wherein the two-dimensional image frame of the structured light obtained by the image sensor is used for being compared with the reference structured light image frame to obtain the depth data of the shooting object.

Alternatively, the apparatus may be a binocular imaging apparatus, and then, the imaging apparatus may further include: and first and second image sensors fixed in relative distance from the projection device, for photographing the photographic subject to obtain first and second two-dimensional image frames under the structured light irradiation, wherein the depth data of the photographic subject is found based on the first and second two-dimensional image frames and a predetermined relative positional relationship between the first and second image sensors.

Optionally, the structured light projected by the projection device is infrared structured light, and the imaging device further includes: and the visible light sensor is used for shooting the shooting object to obtain a two-dimensional image frame under the irradiation of visible light. Thereby providing color information of the photographic subject.

Optionally, the apparatus may further include: and the processing device is connected with the projection device and the imaging device and is used for controlling the projection of the projection device and the imaging of the imaging device. Further, the processing device is configured to: and obtaining the depth data of the shooting object by utilizing the two-dimensional image frame shot by the imaging device.

In various implementations, the LCOS device may be used to: projecting the encoded discrete light spots in a two-dimensional planar distribution, and the imaging device is configured to synchronously capture the projected structured light in the two-dimensional planar distribution to acquire the two-dimensional image frames. The LCOS device may also be used to: a set of structured light with different fringe codes is projected respectively, and the imaging device is used for synchronously shooting each projected structured light to acquire a set of two-dimensional image frames which are used for solving the depth data of the shooting object once. In particular, the LCOS device may be used to: scan projecting the fringe code, and the imaging device comprises: and synchronously opening the rolling curtain sensor for imaging the pixel columns in the stripe direction corresponding to the current scanning position.

The cameras can be independently arranged, and for this purpose, the equipment can also comprise: and the shell is used for accommodating the projection device and the imaging device and fixing the relative positions of the projection device and the imaging device. Further, the apparatus may further include: and the signal transmission device penetrates through the shell and is connected with the projection device and the imaging device, and is used for transmitting control signals for the projection device and the imaging device inwards and transmitting the two-dimensional image frames outwards.

According to a second aspect of the present disclosure, there is provided a structured light projection device comprising: a Vertical Cavity Surface Emitting Laser (VCSEL) to generate the laser light. A Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection. Further, the apparatus may further include: a diffusion sheet disposed on a propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source; a shaping optical assembly for providing the surface light source produced by the diffuser to the LCOS device; and a lens group for projecting structured light generated by the LCOS device outward. And thus may be used for structured light projection for various types of depth data computing devices.

Therefore, the depth data measuring device of the invention uses LCOS to perform fine projection of structured light, thereby improving the imaging precision of depth data, and is particularly suitable for depth data measurement of tiny objects or details. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure may be employed to achieve low power consumption and miniaturization of the projection apparatus, and the VCSEL may have an array structure and may partially emit light to further reduce power consumption and device heat generation.

Drawings

The above and other objects, features and advantages of the present disclosure will become more apparent by describing in greater detail exemplary embodiments thereof with reference to the attached drawings, in which like reference numerals generally represent like parts throughout.

Fig. 1 shows a schematic view of an example of an object to be measured.

Fig. 2 shows a schematic diagram of discrete spots projected by a laser beam onto the surface of an object to be measured.

Fig. 3 shows a schematic configuration of a depth data measuring apparatus according to an embodiment of the present invention.

Fig. 4 shows the principle of depth imaging with fringe-coded structured light.

Fig. 5 shows a schematic composition diagram of a depth data measuring device according to an embodiment of the present invention.

Fig. 6 shows a schematic composition diagram of a depth data measuring device according to an embodiment of the present invention.

Fig. 7 illustrates a light emitting path of the projection apparatus shown in fig. 3.

Detailed Description

Preferred embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While the preferred embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

The three-dimensional measurement method based on structured light detection can be used for carrying out three-dimensional measurement on the surface of an object in real time.

The three-dimensional measuring method based on structured light detection is a method capable of carrying out real-time three-dimensional detection on the surface of a moving object. Briefly, the measuring method comprises the steps of firstly projecting a two-dimensional laser texture pattern with coded information, such as a discretized speckle pattern, onto the surface of a natural body, continuously acquiring laser textures by another image acquisition device with a relatively fixed position, comparing the acquired laser texture pattern with a reference surface texture pattern with known depth distances stored in a memory in advance, calculating the depth distances of various laser texture sequence segments projected onto the surface of the natural body according to the difference between the acquired texture pattern and the known reference texture pattern, and further measuring to obtain three-dimensional data of the surface of an object to be measured. The three-dimensional measurement method based on the structured light detection adopts a parallel image processing method, so that the moving object can be detected in real time, the method has the advantage of being capable of quickly and accurately performing three-dimensional measurement, and is particularly suitable for use environments with high requirements on real-time measurement.

Fig. 1 shows a schematic view of an example of an object to be measured. The figure schematically shows a human hand as the object to be measured. Fig. 2 shows a schematic diagram of discrete spots projected by a laser beam onto the surface of an object to be measured. In a monocular imaging scene, the shot discrete spot image shown in fig. 2 may be compared with a reference standard image, so as to calculate the depth data of each discrete spot, and thus integrate the depth data of the whole object to be measured. As can be seen from fig. 2, since there is a certain distance between the discrete laser spots, more spot information cannot be emitted for the narrow position of the projection surface, so that part of the real depth information is easily lost.

In the prior art, a structured light projection device capable of performing fine projection is lacked, so that the depth data of a fine object cannot be measured with high precision.

To this end, the present invention provides an improved depth data measuring device that uses LCOS for fine projection of structured light, thereby improving the imaging accuracy of the depth data. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure can be employed to achieve low power consumption and miniaturization of the projection apparatus.

Fig. 3 shows a schematic configuration of a depth data measuring apparatus according to an embodiment of the present invention. As shown, the depth data measuring apparatus 300 may include a projection device 310 and an imaging device 320.

The projection device 310 is used for projecting the structured light to the photographic subject. The imaging device 320 is used for shooting the shooting object to obtain a two-dimensional image frame under the structured light irradiation.

To illustrate the internal structure of the projection device 310, fig. 3 does not show the housing and/or the fixing member of the projection device 310, which may be used to fix the relative positions of the illustrated devices and may serve to protect the devices from external contamination and external impact.

As shown, the projection apparatus for projecting structured light mainly includes two devices: a laser generator 311 and a Liquid Crystal On Silicon (LCOS) device 312.

Here, the laser generator 311 is used to generate laser light. A Liquid Crystal On Silicon (LCOS) device may then be used as a projection pattern generator for capturing the laser light and generating structured light for projection. Thus, the LCOS is used to control the projection pattern with extremely high precision. Further, each pixel of the LCOS device may be controlled to open and close by a processing device, e.g., internal or external to the device, to produce different projected structured light patterns. Thereby expanding the application scene of the device.

Lcos (liquid Crystal on silicon), which is a liquid Crystal with silicon or liquid Crystal on silicon, is a matrix liquid Crystal display device based on a reflective mode and having a very small size. The matrix is fabricated on a silicon chip using CMOS technology.

In particular, LCOS may employ a CMOS integrated circuit chip coated with liquid crystal silicon as the substrate for a reflective LCD. The CMOS substrate is formed by using an advanced process to grind and flatten, plating aluminum as a reflector, then attaching the CMOS substrate to a glass substrate containing a transparent electrode, and injecting liquid crystal for packaging. The LCOS places the control circuit behind the display device, which can increase the transmittance, thereby achieving greater light output and higher resolution.

The LCOS can be regarded as one of the LCDs, the conventional LCD is fabricated on a glass substrate, the LCOS is fabricated on a silicon wafer, and the light utilization efficiency can reach more than 40% due to the reflective projection. The LCOS panel is similar to TFT LCD in structure, and has partitions between the upper and lower substrates to separate them, and liquid crystal is filled between the substrates to form a light valve, and the liquid crystal molecules are driven to rotate by the switch of the circuit to determine the brightness and darkness of the projection. The upper substrate of the LCOS panel may be ITO conductive glass, and the lower substrate may be a CMOS substrate coated with liquid crystal silicon. Since the lower substrate is made of single crystal silicon, it has a good electron mobility, and the single crystal silicon can be formed into a thin line, thereby achieving a high resolution. The pixel pitch (i.e., the horizontal distance between two same color pixels) of existing LCOS devices may be very small, e.g., 8 to 20 microns (10-6).

In the present invention, since the laser generator preferably projects infrared light, for example, 940nm infrared light, the LCOS device used in the present invention is used to generate a projection for a pattern at one wavelength (i.e., only "monochrome"), unlike the LCOS panels commonly used in the prior art to display three colors of RGB. Thus, the LCOS device of the present invention can have a smaller pixel pitch, thereby enabling the projection of extremely fine structured light patterns.

In one embodiment, the laser generator 311 may comprise a Vertical Cavity Surface Emitting Laser (VCSEL) or be implemented inter alia. A VCSEL can be used to generate the laser. Thus, the vertical emission performance of the VCSEL can be utilized to further reduce the volume, power consumption, and heat generation.

Further, as shown in fig. 3, the projection device 310 may further include: a diffuser (diffuser)313 disposed on a propagation optical path of the laser light to convert the laser light generated by the VCSEL into a surface light source. Thereby providing the LCOS device 312 with its desired background light. Further, the projection device may further include: and a shaping optical assembly 314 for shaping (e.g., shaping to conform to the shape of an LCOS device) the surface light source produced by the diffuser to provide to the LCOS device.

In addition, the projection device 310 may further include: a lens group to project structured light generated by the LCOS device.

It should be understood that while the diffuser 313, the shaping optical assembly 314, and the lens assembly 315 for projection are shown, in other embodiments one or more of the components described above may be omitted (e.g., such that the exit shape of the VCSEL 311 directly conforms to the desired cross-sectional shape of the LCOS to omit the shaping optical assembly 314), or other components may be substituted or added. All such conventional optical modifications are intended to be within the scope of the present invention.

Further, based on the principle that the LCOS reflects polarized light, the VCSEL 311 may directly generate polarized light, and the LCOS device controls the reflection of light by adjusting the phase difference of the liquid crystal corresponding to each pixel. Since the LCOS312 projects polarized light through the lens group 315, the adverse effect of specular reflection on the imaging quality of the imaging device 320 can be reduced, thereby improving the imaging quality. Further, the apparatus may also be used for high precision flaw detection of reflective surfaces (e.g., glass surfaces).

In addition, although the LCOS312 is itself a pixel matrix composed of a plurality of pixels, and the projection pattern can be precisely controlled by controlling the "on-off" of each pixel (e.g., controlling the angle of the liquid crystal in the pixel with the incident polarized light). On the other hand, however, the VCSEL 311 may also include a matrix structure, for example, including a light emitting array composed of a plurality of light emitting cells. To this end, in some embodiments, the VCSELs 311 may also turn off certain rows, columns, or light emitting units based on the projected structured light pattern when emitting laser light. In other words, although the VCESL 311 is used as the surface light source of the LCOS312, the light emitting pattern of the VCESL 311 has a certain correlation with the pattern of the surface light source received by the LCOS312, and can be precisely fine-tuned by the LCOS 312.

For example, in some cases, the projection device 310 may project a stripe pattern as structured light and finely image. According to the structured light measurement principle, whether the scanning angle alpha can be accurately determined is the key of the whole fringe pattern measurement system, the determined scanning angle can be realized by LCOS in the invention, and the significance of image coding and decoding is to determine the scanning angle of the coded structured light, namely the surface structured light system. Fig. 4 shows the principle of depth imaging with fringe-coded structured light. For ease of understanding, the encoding principle of the stripe-structured light is briefly illustrated in the figures as a two-gray-scale three-bit binary time encoding. The projection device can sequentially project three patterns as shown in the figure to the measured object in the shooting area, and the three patterns divide the projection space into 8 areas by light and dark gray scales respectively. Each region corresponds to a respective projection angle, wherein it can be assumed that bright regions correspond to code "1" and dark regions correspond to code "0". And combining the code values of one point in the three code patterns on the scene in the projection space according to the projection sequence to obtain the region code value of the point, thereby determining the region where the point is located and then decoding to obtain the scanning angle of the point.

In projecting the leftmost pattern of FIG. 4, in one embodiment, VCESL 311 can be fully illuminated and projected by LCOS312 by turning off the pixel columns on the left corresponding to 0-3. In another embodiment, VCESL 311 may be partially illuminated, for example, to illuminate the portion corresponding to the right (typically not exactly 4-7 but may be a larger range of 3-7 portions), thereby ensuring that the columns of pixels of LCOS312 corresponding to 4-7 receive sufficient backlight and are projected by LCOS312 by turning off the columns of pixels corresponding to 0-3 on the left.

Thus, by turning off some of the light emitting cells of the VCSEL in the projection, the power consumption of the VCSEL can be further reduced, thereby reducing the amount of heat generated by the device and obtaining more rest time for each light emitting cell of the VCSEL. Therefore, the method is particularly suitable for being used in a heat sensitive scene, and the service life of the VCSEL can be prolonged.

As shown in fig. 3, the depth data measuring device of the present invention may be a monocular device, i.e., comprising only one camera to capture structured light. To this end, the imaging device 320 may be implemented as an image sensor with a fixed relative distance from the projection device, wherein the two-dimensional image frame of the structured light captured by the image sensor is used for comparing with the reference structured light image frame to find the depth data of the photographic subject.

Alternatively, the depth data measuring apparatus of the present invention may be a binocular apparatus, that is, including two cameras to synchronously photograph the structured light, and find the depth data using the parallax in the two images. To this end, the image forming apparatus further includes: and first and second image sensors fixed in relative distance from the projection device, for photographing the photographic subject to obtain first and second two-dimensional image frames under the structured light irradiation, wherein the depth data of the photographic subject is found based on the first and second two-dimensional image frames and a predetermined relative positional relationship between the first and second image sensors.

In a binocular imaging system, the above-described decoding process of the fringe coding, such as shown in fig. 4, can be simplified by directly matching the coded values of the respective points in the first and second image sensors. To improve the matching accuracy, the number of projected patterns in the temporal coding may be increased, for example, a two-gray scale five-bit binary temporal coding. In the application scenario of binocular imaging, this means that, for example, each pixel in each of the left and right image frames contains 5 or 0 or 1 region code values, thereby enabling left and right image matching with higher accuracy (e.g., pixel level). In the case of a constant projection rate of the projection device, five coding patterns are equivalent to achieving higher accuracy of image matching at a higher temporal cost than the three coding patterns of fig. 4. This is still quite desirable in cases where the projection device is inherently very high in projection rate (e.g., fast switching of LCOS projection patterns).

As mentioned above, the structured light projected by the projection device is preferably infrared structured light, thereby avoiding interference of visible light. At this time, the depth data measuring apparatus of the present invention may further include: and the visible light sensor is used for shooting the shooting object to obtain a two-dimensional image frame under the irradiation of visible light. For example, RGB sensors may be included to obtain color two-dimensional information of the subject to be photographed for combination with the derived depth information, e.g., to obtain 3-dimensional information, or as a supplement or correction to depth learning.

At this point, the LCOS device may be used to: a set of structured lights (e.g., three or more sets of stripe patterns as shown in fig. 4) with different stripe codes are projected respectively, and the imaging device is configured to synchronously capture each projected structured light to acquire a set of two-dimensional image frames, which are collectively used to obtain depth data of the photographic subject once.

In some cases, the LCOS device may project one complete pattern at a time. In other cases, LCOS devices may be used to: scan projecting the fringe code, and the imaging device comprises: and synchronously opening the rolling curtain sensor for imaging the pixel columns in the stripe direction corresponding to the current scanning position. For example, the VCSELs may illuminate some number of their columns in turn and cooperate with the alternating reflection of the LCOS (i.e., the LCOS projects a structured light pattern that illuminates the several columns in turn) while synchronizing with the turning on of the pixel columns of the rolling shutter sensor. Thereby, the heat dissipation of the VCSEL is further reduced and the interference of ambient light on the structured light imaging is avoided.

Fig. 5 shows a schematic composition diagram of a depth data measuring device according to an embodiment of the present invention. As shown in fig. 5, the depth data measuring head 500 includes a projection device 510 and two image sensors 520_1 and 520_ 2.

The projection device 510 is used for scanning and projecting structured light with stripe codes to a shooting area. For example, within 3 consecutive image frame projection periods, the projection device 510 may successively project three patterns as shown in fig. 4, the imaging results of which may be used for the generation of depth data. 520_1 and 520_2, which may be referred to as first and second image sensors, respectively, have a predetermined relative positional relationship for photographing a photographing region to obtain first and second two-dimensional image frames, respectively, under structured light illumination. For example, in the case where the projection device 510 projects three patterns as shown in fig. 1, the first and second image sensors 520_1 and 520_2 may respectively image a photographing region (e.g., an imaging plane and a region in a certain range before and after the imaging plane in fig. 5) on which the three patterns are projected in three synchronized image frame imaging periods.

As shown in fig. 5, the projection device 510 may project linear light extending in the x direction in the z direction (i.e., toward the photographing region). In different embodiments, the projection of the line light may be already shaped (i.e., the emergent light itself is the line light), or may be a light spot moving in the x direction (i.e., the scanned line light). Accordingly, the LCOS in the projection device 510 may reflect one or more columns of pixels (line light), or a block of pixels (light spots) made up of one or more pixels. The projected line light can be continuously moved in the y-direction to cover the entire imaging area. The lower part of fig. 5 gives a more understandable illustration of the scanning of the line light for a perspective view of the shooting area.

In the embodiment of the invention, the direction of the light ray exiting from the measuring head is approximately defined as the z direction, the vertical direction of the shooting plane is the x direction, and the horizontal direction is the y direction. The stripe-structured light projected by the projection device may be a result of a movement of a line-shaped light extending in the x direction in the y direction. Although in other embodiments, the synchronization and imaging process may be performed with respect to the stripe structure light obtained by moving the linear light extending in the horizontal y direction in the x direction, in the embodiment of the present invention, the vertical stripe light is preferably used for the description.

Further, the measuring head 500 further comprises a synchronization device 550, which may be realized, for example, by a processing device as described below. The synchronization device 550 is connected to the projection device 510 (including both VCSEL and LCOS) and the first and second image sensors 520_1 and 520_2, respectively, to achieve precise synchronization therebetween. Specifically, the synchronization device 550 may synchronously turn on pixel columns in the stripe direction corresponding to the current scanning position in the first and second image sensors 520_1 and 520_2 for imaging based on the scanning position of the projection device 510. As shown in fig. 5, the current streak is being scanned to the center area of the photographing region. For this, in the image sensors 520_1 and 520_2, a pixel column (for example, 3 adjacent pixel columns) located in the central region is turned on for imaging. As the stripes move in the y-direction (as indicated by the arrows in the lower perspective view of fig. 5), the imaging enabled pixel columns in image sensors 520_1 and 520_2 also move in synchronization accordingly (as indicated by the arrows above the matrix in the upper left block diagram of fig. 5). Thus, the range of the pixel columns imaged at each time can be controlled by using the one-dimensional characteristics of the fringe image, thereby reducing the adverse effect of ambient light on the measurement result. To further reduce the influence of ambient light, the projection device is particularly suitable for projecting light that is not easily confused with ambient light, such as infrared light. In addition, since the correspondence relationship between the pixel columns and the scanning light is influenced by many factors, such as the width, power, speed of the projection light, and the photosensitive efficiency of the image sensor, the pixel column range (and the corresponding number) that is turned on at each time in synchronization can be determined based on the calibration operation, for example.

In other embodiments, the LCOS device may also be used to project the encoded discrete light spots in a two-dimensional planar distribution, and the imaging device is used to synchronously capture the projected structured light in the two-dimensional planar distribution to acquire the two-dimensional image frames. For example, an LCOS device can project discrete spots as shown in fig. 2 (but with much higher precision, the object being photographed is also typically much smaller).

In different embodiments, the device may be implemented as a measuring head for implementing the shooting function only, or may contain processing and computing means. In addition, where processing and computing equipment is included, the measurement head and the processing and computing means may be housed in the same housing or separately connected via signal transmission means, depending on the application.

Although not shown in fig. 3, the depth data measuring apparatus of the present invention may further include: and a processing device (control function) connected to the projection device and the imaging device, for controlling the projection by the projection device and the imaging by the imaging device. For example, the processing means may be for: controlling the LCOS device pixels to open and close to generate different projected structured light patterns.

Additionally, the processing apparatus may further comprise a computing function and be configured to: and obtaining the depth data of the shooting object by utilizing the two-dimensional image frame shot by the imaging device.

Further, the depth data measuring apparatus of the present invention may further include: and the shell is used for accommodating the projection device and the imaging device and fixing the relative positions of the projection device and the imaging device. The fixture 330 shown in fig. 3 may be part of the housing.

In certain embodiments, processing means for control and/or calculation may be included inside the housing. In some cases, however, it is desirable to have a separate camera and processing. At this time, the apparatus may include: and the signal transmission device penetrates through the shell and is connected with the projection device and the imaging device, and is used for transmitting control signals for the projection device and the imaging device inwards and transmitting the two-dimensional image frames outwards. When the depth data measuring apparatus of the present invention includes the processing device, the signal transmission device may be a signal connection line with the processing device, such as an optical fiber or a coaxial cable. The signal transmission means may be a connection interface with the processing means when the device itself does not comprise processing functionality.

As shown, the depth data measuring apparatus includes a separate measuring head 600, a signal transmission device 640, and a processor 650. A perspective view of the measuring head 600 is schematically shown, together with a cable schematic of the signal transmission means (transmission cable) 640 and a symbolic schematic of the processor 650. It should be understood that in various implementations, the processor 650 may be enclosed by a separate processor housing, or plugged into another device, such as a computing motherboard of an acquisition device described below, or otherwise secured, as the present disclosure is not limited thereto.

The measuring head performs the active projection of structured light and the binocular measurement function for structured light. The measurement head 600 may include a structured light projection device 610, first and second image sensors 620_1 and 620_2 having a predetermined relative positional relationship, and a housing 630.

Structured light projecting device 610 can be used to project structured light toward a photographic subject and includes a VCSEL in combination with LCOS structure as previously described. The first and second image sensors 620_1 and 620_2 are used to photograph the photographic subject to obtain first and second two-dimensional image frames, respectively, under the structured light illumination. The housing 630 is used to house the structured light projection device and the first and second image sensors and to fix the relative positions of the structured light projection device and the first and second image sensors.

A signal transmission device 640 may be connected to the structured light projection device 111 and the first and second image sensors through the housing 630 for transmitting control signals for the structured light projection device 610 and the first and second image sensors to the inside (inside the housing) and transmitting first and second two-dimensional image frames taken by the image sensors to the outside (outside the housing).

The processor 650 is connected to the signal transmission device 640 and located outside the housing 630, and is configured to send the control signal through the signal transmission device and calculate motion data of the photographic subject based on the continuously acquired first and second two-dimensional image frames and the predetermined relative position relationship between the first and second image sensors.

Thus, the depth data measuring apparatus of the present invention enables a compact, lightweight, and low heat dissipation arrangement for the measurement head by separating the measurement head from a processor (e.g., processing circuitry), thereby facilitating installation in an imaging space of, for example, a medical imaging apparatus.

Here, the signal transmission device 640 may include a coaxial cable, whereby the control signal and the image data are directly transmitted by an electric signal. Further, in a high magnetic field environment such as MRI acquisition, in order to avoid using an iron-nickel material, an optical fiber may be used as the signal transmission device 640. At this time, the structured light projection device, the image sensor, and the processor may each include an optical-to-electrical converter for converting an optical signal transmitted by the optical fiber into an electrical signal or converting a signal to be transmitted into an optical signal.

In another embodiment, the invention may also be embodied as a structured light projection device. The apparatus may include: a Vertical Cavity Surface Emitting Laser (VCSEL) for generating the laser light; and a Liquid Crystal On Silicon (LCOS) device for capturing the laser light and generating structured light for projection. Further, the apparatus may further include: a diffusion sheet disposed on a propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source; a shaping optical assembly for providing the surface light source produced by the diffuser to the LCOS device; and a lens group for projecting structured light generated by the LCOS device outward. The structured light projection device described above can be used in conjunction with various imaging devices to enable depth data measurement and calculation for various scenes.

It should be understood that the present invention, since it employs LCOS for projection using the principle of reflection, the laser generator and the projection lens group can be disposed on a folded optical path, thereby contributing to the compactness and miniaturization of the apparatus. Fig. 7 illustrates a light emitting path of the projection apparatus shown in fig. 3. As shown, laser light emitted from a laser generator 711, such as a VCSEL, is sent to the LCOS 712 via a diffuser 713 and a shaping component 714, and is reflected by associated liquid crystals inside the LCOS 712 and then projected by a lens assembly 715.

The depth data measuring apparatus and the structured light projecting device constituting the apparatus according to the present invention have been described in detail hereinabove with reference to the accompanying drawings. The invention uses LCOS to carry out fine projection of structured light, thereby improving the imaging precision of depth data, and is particularly suitable for depth data measurement of tiny objects or details. LCOS can also transform various projection codes including speckle or fringe to meet various imaging scenarios. Further, a VCSEL structure may be employed to achieve low power consumption and miniaturization of the projection apparatus, and the VCSEL may have an array structure and may partially emit light to further reduce power consumption and device heat generation.

Having described embodiments of the present invention, the foregoing description is intended to be exemplary, not exhaustive, and not limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen in order to best explain the principles of the embodiments, the practical application, or improvements made to the technology in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A depth data measuring apparatus comprising:

a projection device for projecting the structured light to a photographic subject;

an imaging device for photographing the photographic subject to obtain a two-dimensional image frame under the structured light irradiation,

wherein the projection device comprises:

a laser generator for generating laser light;

a Liquid Crystal On Silicon (LCOS) device to acquire the laser light and generate structured light for projection.

2. The apparatus of claim 1, wherein the laser generator comprises:

a Vertical Cavity Surface Emitting Laser (VCSEL) to generate the laser light.

3. The apparatus of claim 2, wherein the projection device comprises:

a diffusion sheet disposed on a propagation path of the laser light to convert the laser light generated by the VCSEL into a surface light source.

4. The apparatus of claim 3, wherein the projection device further comprises:

and the shaping optical assembly is used for providing the surface light source generated by the diffusion sheet to the LCOS device.

5. The apparatus of claim 2, wherein the VCSEL generates polarized light and the LCOS device controls the reflection of the light by adjusting a phase difference of a corresponding liquid crystal for each pixel.

6. The apparatus of claim 2, wherein the VCSEL includes a light emitting array comprised of a plurality of light emitting cells, and the VCSEL turns off a specific row, column or light emitting cell according to a projected structured light pattern when emitting laser light.

7. The apparatus of claim 1, wherein the projection device further comprises:

a lens group to project structured light generated by the LCOS device.

8. The apparatus of claim 1, wherein the imaging device further comprises:

and the image sensor is fixed in relative distance with the projection device, wherein the two-dimensional image frame of the structured light obtained by the image sensor is used for being compared with the reference structured light image frame to obtain the depth data of the shooting object.

9. The apparatus of claim 1, wherein the imaging device further comprises:

and first and second image sensors fixed in relative distance from the projection device, for photographing the photographic subject to obtain first and second two-dimensional image frames under the structured light irradiation, wherein the depth data of the photographic subject is found based on the first and second two-dimensional image frames and a predetermined relative positional relationship between the first and second image sensors.

10. The apparatus of claim 1, wherein the structured light projected by the projection device is infrared structured light, and the depth data measuring apparatus further comprises:

and the visible light sensor is used for shooting the shooting object to obtain a two-dimensional image frame under the irradiation of visible light.

11. The apparatus of claim 1, further comprising:

and the processing device is connected with the projection device and the imaging device and is used for controlling the projection of the projection device and the imaging of the imaging device.

12. The apparatus of claim 10, wherein the processing device is to:

and obtaining the depth data of the shooting object by utilizing the two-dimensional image frame shot by the imaging device.

13. The apparatus of claim 10, wherein the processing device is to:

controlling the LCOS device pixels to open and close to generate different projected structured light patterns.

14. The apparatus of claim 1, wherein the LCOS device is to:

projecting the encoded discrete spots in a two-dimensional planar distribution,

and the imaging device is used for synchronously shooting the projected structured light distributed in a two-dimensional plane to acquire the two-dimensional image frame.

15. The apparatus of claim 1, wherein the LCOS device is to:

a set of structured light with different fringe codes is projected separately,

and the imaging device is used for synchronously shooting each kind of projected structured light to acquire a group of two-dimensional image frames, and the group of two-dimensional image frames are commonly used for solving the depth data of the shooting object once.

16. The apparatus of claim 15, wherein the LCOS device is to:

scan projecting the stripe code, an

The image forming apparatus includes:

and synchronously opening the rolling curtain sensor for imaging the pixel columns in the stripe direction corresponding to the current scanning position.

17. The apparatus of claim 1, further comprising:

and the shell is used for accommodating the projection device and the imaging device and fixing the relative positions of the projection device and the imaging device.

18. The apparatus of claim 17, further comprising:

and the signal transmission device penetrates through the shell and is connected with the projection device and the imaging device, and is used for transmitting control signals for the projection device and the imaging device inwards and transmitting the two-dimensional image frames outwards.

19. A structured light projection device, comprising:

a Vertical Cavity Surface Emitting Laser (VCSEL) to generate the laser light.

20. The apparatus of claim 19, further comprising:

a diffusion sheet disposed on a propagation light path of the laser light to convert the laser light generated by the VCSEL into a surface light source;

a shaping optical assembly for providing the surface light source produced by the diffuser to the LCOS device; and

a lens group for projecting structured light generated by the LCOS device outward.