WO2019179123A1

WO2019179123A1 - Tof camera and design method for diffractive optical element

Info

Publication number: WO2019179123A1
Application number: PCT/CN2018/113875
Authority: WO
Inventors: 余新; 胡飞; 鲁宁; 李屹
Original assignee: 深圳光峰科技股份有限公司
Priority date: 2018-03-19
Filing date: 2018-11-05
Publication date: 2019-09-26
Also published as: CN110285788A; CN110285788B

Abstract

A ToF camera (100) comprises a dot-matrix light source (107), a light receiving lens system (102), and a sensor (101). The dot-matrix light source is used for emitting illuminating beams arranged in a dot matrix to a target object (103); the light receiving lens system is used for receiving illuminating beams arranged in a dot matrix reflected after the emitted illuminating beams are irradiated to the target object so as to image the target object on the sensor; the sensor is used for sensing said illuminating beams received by the light receiving lens system for the imaging onto the sensor so as to obtain depth image information of the target object, wherein the sensor comprises multiple pixels (101a), the pixels have one-to-one correspondence to the illuminating beams arranged in a dot matrix, and each pixel is used for sensing a corresponding illuminating beam. Also disclosed are a projection system having the ToF camera, a robot, a driving system, an electronic device, and a design method for a diffractive optical element for the dot-matrix light source of the ToF camera.

Description

ToF camera and design method of diffractive optical element

Technical field

The present invention relates to a ToF camera, a projection system having the ToF camera, a robot, a driving system and an electronic device, and a method of designing a diffractive optical element for a dot matrix light source of the ToF camera.

Background technique

It is often necessary to measure the depth information of a target object in applications such as gesture recognition and 3D imaging of driving systems (such as automated driving systems), projection systems, or electronic devices. Commonly used depth measurement methods include triangulation, structured light measurement, and ToF ranging. These different ranging techniques each have advantages and disadvantages. The triangulation method is insensitive to ambient light and has a simple hardware structure. However, since multiple cameras are used, there is a problem of synchronization between cameras, resulting in high image analysis cost and difficulty in miniaturization of the product. The structured light measurement method has high spatial resolution and low computational cost, but the measurement result is greatly affected by ambient light and the measurement high-speed moving object has large error. Compared with the triangulation method and the structured light measurement method, the ToF ranging method has a simple structure, a small product volume, and a good result of measuring a high-speed moving object, and the ambient light has little influence on the measurement result, and the ranging principle of the ToF ranging method is adopted. Figure is shown in Figure 1. Considering the accuracy of the measurement results, the triangulation method generally measures medium and long distances, and the structured light measurement method and the ToF distance measurement method measure short distances.

Specifically, the ToF ranging method is divided into two categories, one is to measure the echo time of the pulse signal to measure the distance, and the general single-point range finder and the scanning laser radar adopt this method. Due to the concentration of light energy, rangefinders and lidars using this method measure from a few meters to several kilometers. The other type is the phase detection method, which measures the distance using the phase shift of the modulated signal echoes loaded on the continuous optical signal. The distance measured by this method is limited by the frequency of the modulated signal, typically from a few hundred kilohertz to several tens of megahertz. The effective measurement distance decreases as the frequency of the modulated signal increases. The ToF camera adopts the latter ranging principle, and its structure is as shown in Fig. 2. The system is composed of a light source 101, an object 102 to be measured, a lens 103, a ToF depth sensor 104, and a hardware control processing circuit 105. The amplitude of the light emitted by the light source 101 is modulated by a sine or square wave and is emitted to the object 102 to be measured. The lens 103 collects the reflected signal light and transmits it to the ToF depth image sensor 104. The ToF depth image sensor 104 is a phase locked CCD or CMOS photosensitive array. Figure 3 is a schematic view showing the structure of a phase-locked CCD. The area shown as Light Opening is the effective photosensitive area, and other incident light does not affect the result. This special structure of phase-locked CCD or CMOS devices dynamically controls the electric field distribution in the device, causing electrons in the hole-electron pair excited by the photons to move to specific regions of the device and accumulate in the electron traps in the region. Due to limitations of CCD or CMOS devices, each frame of image requires a long time to integrate, so high-speed sampling of high-speed modulated optical signals (several hundred kilohertz to several tens of megahertz) cannot be achieved. However, a phase-locked CCD or CMOS device can quickly switch the electric field distribution on the device at a frequency synchronized with the modulation frequency, so that the energy in the same phase range of the modulated signal accumulates in the same electronic trap, thereby achieving different phase time points for the modulated signal. Sampling.

The calculation principle of the distance of the measured object is shown in Fig. 4. The phase difference between the reflected light and the signal light emitted by the laser

Meets the following formula 1.

Where A is the amplitude of the received signal and B is the DC component of the received signal, which is proportional to the number of received signal photons and the number of ambient/noise photons, respectively. A and B respectively meet the following formula 2 and formula 3:

Further, the distance of the measured object conforms to the following formula 4:

Since the error sources of ToF camera ranging are mainly ambient light, quantum noise and device noise. The ambient illuminance does not change with distance, so the measurement result is independent of the distance, but is related to the angle of light reception. Quantum noise, also known as shot noise, is determined by the quantum effect in the photoelectric effect and does not change with changes in the environment. The device noise includes reset noise, flicker noise, current noise of the amplifier, and dark current noise. The noise intensity increases with increasing temperature. The size of the measurement error limits the distance of the ToF camera's ranging. The process of reducing the error is to increase the distance measured by the ToF camera. Since quantum noise does not vary with the device and the environment, quantum noise determines the maximum accuracy of the ideal range.

Since the ToF camera ranging is the phase difference between the measured incident light signal and the reflected light signal to calculate the distance, the error ΔL meets the following formula 5:

Ideally, the ambient light is much smaller than the signal light and can be approximated as B=A/2. Substituting it into Equation 5 yields the following Equation 6:

According to Equation 6, it can be inferred that the larger the A, the smaller the measurement error. However, since the number of electrons that can be accumulated by each pixel on the ToF depth image sensor is limited, when the cumulative number of pixels of the pixel is saturated, the increase of A does not reduce the error and wastes the power of the light source. Therefore, the value of A is limited and cannot be infinite. When the pixel saturation electron number is 100000, the error can be controlled to 0.04% of the maximum ranging.

The method of further reducing the measurement error under the condition that the value of A is limited is to reduce the influence of ambient light on the measurement result. Because ambient illuminance does not change with distance, its effect is independent of distance and light angle. Therefore, it is possible to reduce the angle of view of the lens by increasing the focal length of the light-receiving lens, thereby reducing the influence of ambient light.

Under different lens focal length conditions, the variation of the ambient light power entering the lens with the distance is shown in Fig. 5(a). Under different environmental photoelectron numbers, the variation of simulation measurement accuracy with distance is shown in Fig. 5(b). The simulation results show that when the ambient illuminance is constant, the angle of view changes from 50 degrees to 15 degrees, and the ambient light is 5%, which can increase the measurement distance by four times without affecting the accuracy. At the same time, the less the number of electrons generated by ambient light, the higher the measurement accuracy.

Further, the expression of the influence of device noise on the measurement result (ie, error ΔL) is Equation 7:

Among them, Npseudo is the noise electron introduced by the device.

If the laser used is a non-ideal demodulation and light source, and A and B are respectively represented by the number of free electrons introduced by them, the following Equation 8, Equation 9, and Formula 10 can be known.

B _eff =N _ambient +N _pseudo +PE _opt (Equation 8);

A=C _mod C _demod PE _opt (Equation 9);

In the specific measurement application, the device parameters and environmental influences are basically constant, and the influence of the device noise can be reduced by the following three methods, thereby reducing the error ΔL during the measurement process:

1) Cooling down can reduce device noise;

2) Increasing the number of received electrons can reduce the effects of device noise. There are two methods. The first method is to increase the aperture. If you do not change the number of photo-converted electrons, you need to double the aperture for every doubling of the distance. The disadvantage of this method is that the structure volume increases and the cost increases. . The second method is to increase the optical power, which requires four times the optical power for every doubling of the distance.

3) Since remotely adapted light sources can cause near-field device saturation, the use of dynamic optical power can reduce such errors.

Assuming the device and source parameters are as follows: source power 700mW, beam divergence angle 50 degrees, pixel photosensitive area 12.5×14.5μm2, laser wavelength 630nm, quantum efficiency 65%, integral sampling time 25ms, lens efficiency 0.35, lens aperture 2.6mm, simulate the number of incident photons and the relationship between error changes and environmental variables. Figure 6 shows the relationship between the reflectivity of an object and the number of incident photons. It can be seen that there is a linear relationship between the reflectivity of the object and the number of photons received by the sensor. On the other hand, in order to suppress the influence of ambient light, the light-receiving angle of the lens affects the accuracy of the measurement. Simulation experiments show that when the lens light-receiving angle is reduced from 50 degrees to 15 degrees, the number of photons generated by ambient light will become original. 5%, which will greatly improve the accuracy of the measurement, resulting in a farther measurement distance. The measurement result in the long distance range after the error is reduced by changing the lens angle to 15° is shown in Fig. 7. The simulation parameter is F=1.5MHz, and the reflectivity of the measured object is 0.5.

Figure 1(a) shows the variation of the number of electrons generated by signal light and ambient light in a pixel with distance under different light source power conditions; Figure 2(b) shows the variation of measurement error with distance under different light source optical power conditions. curve. The simulation results show that long-distance measurement requires very large light source power to ensure measurement accuracy. The difference frequency method of two high-frequency light sources can increase the maximum non-confusion ranging of the phase method. However, theoretically analyzing the surface, the method of difference frequency does not increase the accuracy of the measurement.

Considering that the effective photosensitive area of the pixel structure on the ToF depth image sensor is very small relative to the total pixel area, generally between 6% and 15% of the pixel area, the method of further increasing the measurement accuracy is to improve the light source signal efficiency of the light source. A method for improving the light efficiency from the sensor structure is to add a corresponding microlens to each pixel to further concentrate the incident light into the photosensitive region. The problem with this method is that it increases the intensity of the signal light and also increases the intensity of the ambient light, which is disadvantageous for the accuracy improvement in a strong light environment.

Summary of the invention

In view of the above technical problems, it is necessary to provide a design method of a ToF camera, a projection system having the ToF camera, a robot, a driving system, an electronic device and an electronic device, and a diffractive optical element for a dot matrix light source of the ToF camera.

A ToF camera comprising a dot matrix light source, a light receiving lens system and a sensor, the dot matrix light source for emitting a lattice array of illumination beams to a target object, the light collection lens system for receiving the points Array of illumination beams illuminating the target object and reflecting the array of illumination beams arranged to image the target object on the sensor, the sensor for sensing the image of the collection lens system An illumination beam arranged in the lattice on the sensor to obtain depth image information of the target object, wherein the sensor comprises a plurality of pixels, and the pixels are aligned with the illumination beam arranged by the lattice Correspondingly, each pixel is used to sense a corresponding illumination beam.

A projection system comprising a ToF camera comprising a lattice light source, a light receiving lens system and a sensor, the dot matrix light source for emitting a lattice array of illumination beams to a target object, the light collection a lens system for receiving an illumination beam of the lattice array of illuminated illumination beams that are illuminated to the target object to image the target object on the sensor, the sensor for sensing the location The light-receiving lens system images the illumination beam of the lattice arrangement on the sensor to obtain depth image information of the target object, wherein the sensor includes a plurality of pixels, the pixel and the The illumination beams arranged in a lattice are in one-to-one correspondence, and each pixel is used to sense a corresponding illumination beam.

A robot comprising a ToF camera, the ToF camera comprising a lattice light source, a light receiving lens system and a sensor, the dot matrix light source for emitting a lattice array of illumination beams to a target object, the light receiving lens The system is configured to receive an illumination beam of the lattice array of illuminated illumination beams that are illuminated to the target object to image the target object on the sensor, the sensor for sensing the a light receiving lens system imaging the illumination beam of the lattice arrangement on the sensor to obtain depth image information of the target object, wherein the sensor comprises a plurality of pixels, the pixel The illumination beams arranged in a lattice are in one-to-one correspondence, and each pixel is used to sense a corresponding illumination beam.

In one embodiment, the robot is an automated guided robot.

A driving system comprising a driving device and a ToF camera, the ToF camera comprising a dot matrix light source, a light receiving lens system and a sensor, wherein the dot matrix light source is used to emit a lattice array of illumination beams to a target object, The light-receiving lens system is configured to receive an illumination beam of a dot matrix arranged by the illumination beam of the lattice arrangement to be irradiated to a target object to image the target object on the sensor, the sensor is used for Sensing the illumination lens of the lattice array imaged onto the sensor to obtain depth image information of the target object, wherein the sensor includes a plurality of pixels, The pixels are in one-to-one correspondence with the illumination beams arranged in the lattice, and each pixel is used to sense a corresponding illumination beam.

In one embodiment, the driving system is an automated driving system.

An electronic device comprising a ToF camera comprising a dot matrix light source, a light receiving lens system and a sensor, the dot matrix light source for emitting a lattice array of illumination beams to a target object, the light collection a lens system for receiving an illumination beam of the lattice array of illuminated illumination beams that are illuminated to the target object to image the target object on the sensor, the sensor for sensing the location The light-receiving lens system images the illumination beam of the lattice arrangement on the sensor to obtain depth image information of the target object, wherein the sensor includes a plurality of pixels, the pixel and the The illumination beams arranged in a lattice are in one-to-one correspondence, and each pixel is used to sense a corresponding illumination beam.

In one embodiment, the electronic device is a cell phone, a computer, or an unmanned aerial vehicle.

A method for designing a diffractive optical element for a lattice source of a ToF camera, which comprises the following steps:

Calculating an effective photosensitive area distribution of the sensor of the ToF camera;

Setting a typical plane distance of the far field plane;

Calculating a projection pattern distribution of the far field plane on the sensor;

A phase map of the diffractive optical element is calculated.

Compared with the prior art, the ToF camera of the present invention uses a dot matrix light source and makes the pixels correspond to the illumination beams arranged in a lattice, so that each pixel of the sensor can sense the corresponding illumination beam, thereby providing reliable The sensing result improves the reliability of the ToF camera; meanwhile, at the same power, the number of photons sensed per pixel is increased relative to the source using uniform illumination, and the ambient light sensed by each pixel The generated photons have substantially no change compared with the light source with uniform illumination, which not only improves the light efficiency and the maximum ranging distance of the ToF camera, but also improves the signal-to-noise ratio of the received sensing signal, and has better measurement accuracy.

DRAWINGS

Figure 1 is a schematic diagram of the ranging of the ToF ranging method.

Figure 2 is a principle of ranging used in a ToF camera.

FIG. 3 is a schematic structural view of a phase locked CCD.

Figure 4 is a schematic diagram of the calculation of the distance of the measured object.

Fig. 5(a) is a schematic diagram showing the variation of ambient light power with distance simulated into the lens under different lens focal lengths.

Fig. 5(b) is a schematic diagram showing the variation of simulation measurement accuracy with distance under different environmental photoelectron numbers.

Figure 6 is a schematic diagram showing the relationship between the reflectance of an object and the number of incident photons.

Fig. 3(a) is a graph showing the relationship between the number of electrons generated by signal light and ambient light in a pixel with distance under different light source power conditions.

Fig. 4(b) is a graph showing the variation of measurement error with distance under different light source optical power conditions.

FIG. 8 is a schematic structural diagram of a ToF camera according to a preferred embodiment of the present invention.

Fig. 9 is a view showing the optical path of the modulating element shown in Fig. 8 for diffracting the light source light emitted from the light source.

Figure 10 is a schematic view showing the design method of the modulation element shown in Figure 8.

Fig. 11(a) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as a function of measurement distance when using 630 nm wavelength signal light.

Fig. 11(b) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as a function of measurement distance when using 1550 nm wavelength signal light.

Fig. 5(a) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as a function of measurement distance when using 630 nm wavelength signal light.

Fig. 6(b) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as a function of measurement distance when using 1550 nm wavelength signal light.

Figure 13(a) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as a function of measured distance.

Fig. 13(b) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as measured distance after the effective pixels are reduced to 1/4 of the original.

Main component symbol description

ToF camera 100

Dot matrix light source 107

Receiving lens system 102

Target object 103

Far field plane 103a

First area 103b

Sensor 101

Light source 104

Collimating lens 106

Diffractive optical element 105

Pixel 101a

The invention will be further illustrated by the following detailed description in conjunction with the accompanying drawings.

detailed description

The invention aims at the characteristics of the sensor structure of the ToF ranging technology, and proposes a method of using a dot matrix illumination source instead of a uniform illumination source, so that the ToF ranging method is improved in light efficiency, and the measurement distance of the ToF camera under the same optical power condition is improved.

Specifically, please refer to FIG. 8. FIG. 8 is a schematic structural diagram of a ToF camera 100 according to a preferred embodiment of the present invention. The ToF camera 100 includes a dot matrix light source 107, a light collecting lens system 102, and a sensor 101. The dot matrix light source 107 is configured to emit a lattice-arranged illumination beam to the target object 103, and the light-receiving lens system 102 is configured to receive the lattice arrangement of the lattice-arranged illumination beam that is irradiated to the target object 103 and reflected. Illuminating a light beam to image the target object on the sensor 101, the sensor 101 for sensing the lattice arrangement of the light receiving lens system 102 onto the sensor 101 The illumination beam is obtained to obtain depth image information of the target object 103, wherein the sensor 101 includes a plurality of pixels 101a, the pixels 101a are in one-to-one correspondence with the illumination beams arranged in the lattice, and each pixel 101a It is used to sense a corresponding illumination beam.

Specifically, the lattice source 107 includes a light source 104, a collimating lens 106, and a modulating element 105. The light source 104 is an inventive diode or laser source for emitting source light, and the modulating element 105 is operative to modulate the source light into the array of illumination beams. The collimating lens 106 is disposed between the light source 104 and the modulating element 105 for collimating the light source light emitted by the light source 104 to the modulating element 105.

In the present embodiment, the modulation element 105 is a diffractive optical element. Please refer to FIG. 9. FIG. 9 is a schematic diagram of an optical path of the modulation element 105 for diffracting the light source light emitted by the light source 104. The modulating element 105 can include a plurality of diffractive elements, each diffractive unit for converting received source light into an illumination beam that is reflected by the target object 103 and projected through the collection lens system 102 to the A corresponding pixel 101a on the sensor 101.

Further, in this embodiment, the sensor 101 is a ToF chip, which may be a phase locked CCD or a CMOS photosensitive array; the light receiving lens system 102 may include a light receiving lens, and the light receiving lens system 102 An optical center may be in line with a center of the target object 103 and a center of the sensor 101; in an embodiment, the target object 103 may be located at the light source 104 via the light receiving lens system The plane formed by the far field of 102, or the target object 103 is a far field plane 103a located in the far field, for example, to more clearly describe the optical path principle and diffraction of the ToF camera 100. The design principle of optical components.

Specifically, the pixel 101a includes an effective photosensitive area 101b, and the sensor further includes a non-photosensitive area 101c. The non-photosensitive area 101c may be located between the effective photosensitive areas 101b of two adjacent pixels 101a. An illumination beam corresponding to the pixel 101a covers the effective photosensitive area 101b, that is, the spot size of the illumination beam is consistent with the effective photosensitive area, so that the spot of the illumination beam is just covered to the effective Photosensitive area 101b.

The function of the light receiving lens system 102 is to change the spatial angular distribution of the illumination beam of the far field plane 103a reflecting the lattice light source 107 to the spatial position distribution on the sensor 101. Any point on the far field plane 103a can be mapped to a corresponding point on the sensor 101. Through the light receiving lens system 102, any effective photosensitive area 101b on the sensor 101 uniquely corresponds to an area on the far field plane 103a (as the first area 103b on the far field plane 103a indicates corresponding region). Thus, the effective photosensitive area of the sensor 101 obtains a light field signal formed by the far field plane illuminated by the lattice source 107, as can be understood. The light field signal is a beam signal of the illumination light beam arranged by the dot matrix incident on the sensor 101 via the far field plane 103a and the light receiving lens system 102, and the ToF camera according to the sensing The beam signal sensed by the device 101 can obtain depth image information of the far field plane 103a.

As shown in FIG. 8, since the distance between the light source 104 and the sensor 101 is negligible, the spatial angle corresponding to the center of each diffraction unit 105a can be regarded as equal to the spatial angle corresponding to the center of the corresponding pixel 101a. The spatial angle corresponding to the center of each pixel (i, j) includes an angle θ ^c _i,j between the vertical direction and the optical axis corresponding to the pixel (i, j) and a horizontal direction corresponding to the pixel (i, j) Angle with the optical axis

among them,

Where i and j respectively represent the number of horizontal lines and the number of vertical columns of the pixel (i, j) on the pixel lattice on the sensor 101, that is, the pixel (i, j) is The pixels in the i-th row and the j-th column on the sensor 101, FOV _V is the field of view angle in the vertical direction, FOV _h is the field of view angle in the horizontal direction, and N _v is the number of pixels in the vertical direction, N _h The number of pixels in the horizontal direction.

Further, in the pixel (i, j), the effective photosensitive area corresponds to a half angle of a spatial angle

versus

Meet the following formula:

Where d _h and d _v are the dimensions of the effective photosensitive area 101b in the vertical and horizontal directions, and d ₁ is the distance between the sensor 101 and the optical center of the light receiving lens system 102.

According to the above analysis, the illumination beam arranged by the lattice source 104 on the far field plane 103a and the projection pattern of the sensor 101 passing through the light receiving lens system 102 on the far field plane 103a The present invention further provides a method of designing a diffractive optical element (ie, the modulating element 105) for the lattice source 104 of the ToF camera 100, that is, suitable for the sensor 101 and the light-receiving lens. The phase distribution of the diffractive optical elements of system 102. The projection pattern of the sensor 101 on the far field plane 103a may be equivalent to a spatial solid angle distribution of the effective photosensitive area 101b due to the distance between the light source 104 and the sensor 101. It can be omitted, so that the spatial angle corresponding to the center of each diffraction unit 105a can be regarded as equal to the spatial angle corresponding to the center of the corresponding pixel 101a. Therefore, in the design method, the spatial angle corresponding to the center of each pixel 101a can be obtained by calculation. The center of the diffraction unit 105a corresponds to a spatial angle, such as to obtain a phase distribution of the diffractive optical element, wherein a spatial angular distribution corresponding to the center of each pixel 101a is defined by the size of the sensor 101 and the effective photosensitive area 101b. And the focal length of the light receiving lens system 102 is determined.

Specifically, according to the above principle, as shown in FIG. 10, the design method includes the following steps S1-S4.

In step S1, the effective photosensitive area 101b distribution of the sensor 101 of the ToF camera is calculated. It can be understood that the parameters of the sensor 101 can be known. Specifically, the effective photosensitive area map of the sensor 101 can be determined according to the arrangement of the pixels 101a of the sensor 101 and the position, size and shape of the effective photosensitive area 101b in the pixel 101a.

In step S2, the plane typical distance of the far field plane 103a is set. It can be understood that the plane typical distance is the distance d2 from the far field plane 103a to the light receiving lens system 102, and can also be regarded as the detection distance of the ToF camera 100 to the far field plane 103a, and the ToF Camera 100 capabilities are related.

Step S3, calculating a projection pattern distribution of the far field plane 103a on the sensor 101. Calculating the effective photosensitive area of the pixel 101a according to the focal length of the light receiving lens system 102, the size of the sensor 101, the pitch of the pixel 101a (such as the width of the non-photosensitive area 101c), and the effective photosensitive area 101b. The spatial angular distribution corresponding to 101b (shown as θ in FIG. 8, as shown in Equations 11-14, and will not be described herein) and the spatial angular interval of the pixel 101a (θ1 shown in FIG. 8).

In step S4, a phase map of the diffractive optical element is calculated. Specifically, the diffractive optical design and the phase hologram can be utilized according to the spatial angular distribution of the center of each pixel 101a of the sensor 101, the optical axis of the light source 104, and the design angle θ2 between the optical axes of the light receiving lens system 102. a method of generating a phase delay profile of the diffractive optical element for designing the diffractive optical element. In addition, since the distance measured between the light source 104 and the sensor 101 is negligible, θ2 may be approximately 0 in the design, that is, the spatial angular distribution projected by the sensor 101 (ie, each The spatial angle corresponding to the center of the pixel can be directly equivalent to the spatial angular distribution of the light emitted by the diffractive optical element (as shown in FIG. 8 , as shown in Equation 11-14, which will not be described herein).

Compared with the prior art, the ToF camera 100 of the present invention can use the dot matrix light source 107 and make the pixels 101a correspond to the illumination beams arranged in a lattice one by one, so that each pixel 101a of the sensor 101 can sense the corresponding illumination. The beam, which in turn provides reliable sensing results, increases the reliability of the ToF camera 100; at the same time, at the same power, the number of photons sensed per pixel 101a is increased relative to the source using uniform illumination, and each The photon generated by the ambient light sensed by the pixel 101a has substantially no change compared with the light source with uniform illumination, which not only improves the light efficiency and the maximum ranging distance of the ToF camera 100, but also improves the signal noise of the receiving sensing signal. Compared with, it has better measurement accuracy.

The present invention also provides a projection system for projecting a display image, the projection system having a ToF camera, the projection system having a gesture recognition function, the projection system being implemented according to a sensing signal of the ToF camera The gesture recognition function, the ToF camera adopts the ToF camera 100 of the above embodiment.

The present invention also provides a robot, which may be an automatic guided robot, the robot including a ToF camera, which uses the ToF camera to sense external objects, and the ToF camera adopts the above embodiment. ToF camera 100.

The present invention also provides a driving system, which may be an automatic driving system including a driving device (such as a self-driving car) and a ToF camera, the ToF camera being mountable on the driving device, the ToF camera The ToF camera of the above embodiment is employed.

In addition, it can be understood that the ToF camera 100 and the ToF camera of the modified embodiment in the above embodiments of the present invention can also be used in electronic devices such as mobile phones, computers, unmanned aerial vehicles, and the like, and are not limited to the above-mentioned projection systems, robots, and Driving system.

Further, it has been experimentally verified that, assuming that the illumination efficiency of the light source is 70%, the dot matrix is used when the power of the uniform illumination and the dot illumination (i.e., the illumination of the array of illumination beams provided by the lattice source) is the same. The number of photons detected by illumination is increased by a factor of 10 relative to uniform illumination. Specifically, in order to verify the effect of using a lattice source, a ToF camera using uniform illumination and a lattice illumination source was simulated to observe a curve of measurement accuracy as a function of measurement distance at different powers. A 65×24 resolution sensor 101 is used in the simulation, the target object has a reflectivity of 35%, and two simulations are performed for beams of different wavelengths of 630 nm and 1550 nm.

The simulation result is shown in FIG. 11. When the resolution of the sensor 101 (ie, the ToF chip) is 65*24, that is, when 65*24 dot matrix pixels are arranged, FIG. 11(a) is when the 630 nm wavelength signal light is used. , the measurement accuracy of uniform illumination and dot matrix illumination at different powers as a function of measurement distance; Figure 11 (b) shows the measurement accuracy of uniform illumination and dot matrix illumination at different powers using 1550 nm wavelength signal light. A graph of changes with measured distance. The comparison shows that the optical power of the dot matrix illumination source is only one tenth of the uniform illumination source with the same accuracy as the uniform illumination source. At the same time, the accuracy of the 1550 nm wavelength source is high, which is caused by the difference in the transmittance spectrum of the electromagnetic waves of different wavelengths.

In addition, the resolution of the ToF camera 100 also affects the measurement error. To verify the effect of the chip resolution, the above simulation is repeated using another 320×240 resolution sensor. The simulation results are shown in FIG. . Figure 7(a) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as measured distance using 630 nm wavelength signal light; Figure 8(b) shows the different powers when using 1550 nm wavelength signal light. A graph of the measurement accuracy of uniform illumination and dot matrix illumination as a function of measured distance.

Comparing the simulation results can lead to the conclusion:

1. As the number of pixels increases, the required optical power also increases proportionally;

2. To achieve high-resolution ToF camera long-distance ranging, the method of increasing optical power and point source cannot meet the requirements of the sensor under the 100m range;

3. Accuracy within 1m at a distance of 100m at 65×24 resolution.

Summarizing the above analysis results, we can draw the following two conclusions:

First of all, compared to the scanning laser radar, the ToF camera has a high scanning speed and a high precision in a short range, and can be applied to an AGV robot.

The characteristics of AGV robot application are: slow moving speed, small moving range (depot, factory area, etc.), 5MHZ modulation, 30m ranging range can meet the ranging requirements. To verify the ranging accuracy of the ToF camera for these characteristics, the measurement distance in the simulation is within 30 meters. In addition, other simulation parameters are: Cmod=1, (λ)=0.65, klens=0.35, Tint=33ms, λ = 630 nm, D = 2.6 mm, ρ = 0.2, Aimage = 4.6 mm2, Apixel = 3.75 μm 2 , τ = 0.8, Npix = 320 × 280, lensAngle = 50°, and the simulation results are shown in Fig. 13. Fig. 13(a) is a graph showing the measurement accuracy of uniform illumination and dot illumination at different powers as measured distance; Fig. 13(b) shows the uniformity at different powers after the effective pixels are reduced by 1/4 of the original power. A graph of the measurement accuracy of illumination and dot matrix illumination as a function of measured distance.

It can be observed from the simulation results shown in Fig. 14 that the 7W dot matrix illumination can achieve an accuracy of 3% at 30m, and the number of sampling points is 2.3M/s (320×240*30ps). By reducing the number of pixels on the CDD sensor to 1/4 of the original (reducing the number of dot matrix points, each pixel can allocate more energy), the 7W dot matrix illumination can achieve an accuracy of 1% at 30m.

In summary, although the principle of ToF camera limits its application in long-distance ranging, from the simulation results, it can achieve 1% to 3% accuracy within 30m after applying dot matrix illumination, enough to meet low speed. The requirements of AGV, the specific parameters are: 3%, with TI's ToF chip can achieve 2Mpps rate; 1%, 700kpps rate; its performance has exceeded most of the market AGV laser radar.

Secondly, the ToF camera is used for long-distance ranging to meet the error requirement, and the optical power required is very high, resulting in a weak cost and volume advantage. Therefore, the ToF camera can be applied to a driving system such as an automatic driving system. However, compared to AGV robots, the ToF camera is more suitable for AGV robots.

Claims

A ToF camera, comprising: a dot matrix light source, a light receiving lens system and a sensor, wherein the dot matrix light source is used to emit a lattice array of illumination beams to a target object, the light receiving lens The system is configured to receive an illumination beam of the lattice array of illuminated illumination beams that are illuminated to the target object to image the target object on the sensor, the sensor for sensing the a light receiving lens system imaging the illumination beam of the lattice arrangement on the sensor to obtain depth image information of the target object, wherein the sensor comprises a plurality of pixels, the pixel The illumination beams arranged in a lattice are in one-to-one correspondence, and each pixel is used to sense a corresponding illumination beam.
A ToF camera according to claim 1, wherein said lattice source comprises a light source and a modulating element, said light source emitting light source, said modulating element for modulating said source light into said lattice arrangement Illumination beam.
A ToF camera according to claim 2, wherein said modulating element is a diffractive optical element comprising a plurality of diffractive elements, each diffracting unit converting the received source light into an illumination beam and passing said illumination beam The light receiving lens system projects onto a corresponding one of the pixels on the sensor.
A ToF camera according to claim 3, wherein the spatial angle corresponding to the center of each diffraction unit is equal to the spatial angle corresponding to the center of the corresponding pixel.
The ToF camera according to claim 2, wherein the light source is the inventive diode or the laser light source.
A ToF camera according to claim 2, wherein said lattice source further comprises a collimating lens disposed between said light source and said modulating element for emitting said light source The source light is collimated and provided to the modulating element.
A ToF camera according to any of claims 1-6, wherein each pixel comprises an effective photosensitive area, and the illumination beam sensed by each pixel covers the effective photosensitive area of the pixel.
The ToF camera according to claim 7, wherein the spatial angle corresponding to the center of each pixel (i, j) comprises an angle θ c between the vertical direction corresponding to the pixel (i, j) and the optical axis. The angle between the horizontal direction and the optical axis corresponding to i, j and pixel (i, j)
among them,

Where i and j respectively represent the number of horizontal lines and vertical columns of the pixel on the pixel lattice on the sensor, FOV V is the vertical field of view angle, and FOV h is the horizontal direction Field angle, N v is the number of pixels in the vertical direction, and N h is the number of pixels in the horizontal direction.
A ToF camera according to claim 8, wherein the effective photosensitive area of each pixel (i, j) corresponds to a spatial angle half width θ Δ i,j and
Meet the following formula:

Where d h and d v are the dimensions of the effective photosensitive area in the vertical and horizontal directions, and d 1 is the distance between the sensor and the optical center of the light-receiving lens system.
A projection system, characterized in that the projection system comprises a ToF camera, the ToF camera employing the ToF camera of any one of claims 1-9.
A robot, characterized in that the robot comprises a ToF camera, and the ToF camera employs the ToF camera according to any one of claims 1-9.
A driving system, characterized in that the driving system comprises a driving device and a ToF camera, and the ToF camera employs the ToF camera according to any one of claims 1-9.
An electronic device, comprising: a ToF camera, the ToF camera employing the ToF camera of any one of claims 1-9.
A method for designing a diffractive optical element for a lattice source of a ToF camera, characterized in that the method comprises the following steps:

Calculating an effective photosensitive area distribution of the sensor of the ToF camera;

Setting a typical plane distance of the far field plane;

Calculating a projection pattern distribution of the far field plane on the sensor;

A phase map of the diffractive optical element is calculated.
A design method according to claim 14, wherein said diffractive optical element comprises a plurality of diffraction units, each of which converts the received source light into an illumination beam and passes said illumination beam through said light collection The lens system is projected onto a corresponding one of the pixels on the sensor.
The design method according to claim 15, wherein the phase map of the diffractive optical element includes a spatial angle corresponding to the center of each diffraction unit, and a spatial angle corresponding to the center of each diffraction unit is equal to a corresponding pixel center. The spatial angle, the spatial angle corresponding to the center of each pixel (i, j) includes the angle θ c i,j between the vertical direction corresponding to the pixel (i, j) and the optical axis , and the pixel (i, j) The angle between the horizontal direction and the optical axis
And respectively meet the following formula:

Where i and j respectively represent the number of horizontal lines and vertical columns of the pixel on the pixel lattice on the sensor, FOV V is the vertical field of view angle, and FOV h is the horizontal direction Field angle, N v is the number of pixels in the vertical direction, and N h is the number of pixels in the horizontal direction.
The design method according to claim 16, wherein each pixel comprises an effective photosensitive area, and an illumination beam sensed by each pixel is imaged on an effective photosensitive area of the pixel, and a spatial angle corresponding to the effective photosensitive area Half width θ Δ i,j and
Meet the following formula:

Where d h and d v are the dimensions of the effective photosensitive area in the vertical and horizontal directions, and d 1 is the distance between the sensor and the optical center of the light-receiving lens system.