WO2022257558A1

WO2022257558A1 - Time-of-flight module, terminal and depth detection method

Info

Publication number: WO2022257558A1
Application number: PCT/CN2022/083585
Authority: WO
Inventors: 戴阳
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-06-10
Filing date: 2022-03-29
Publication date: 2022-12-15
Also published as: CN113419252A

Abstract

A time-of-flight module (100), a terminal (1000) and a depth detection method. The time-of-flight module (100) comprises a light source (20), a lens (30) and an image sensor (40). When a detection pixel (42) of the image sensor (40) receives light reflected by the lens (30), a timer (401) starts timing; when a photosensitive pixel (41) of the image sensor (40) receives light reflected by a target object, the timer (401) stops timing so as to obtain the time of reception; and the image sensor (40) generates depth information of the target object according to the time of reception.

Description

Time-of-flight module, terminal and depth detection method

priority information

This application claims priority and rights to the patent application No. 202110646449.5 filed with the State Intellectual Property Office of China on June 10, 2021, and is hereby incorporated by reference in its entirety.

technical field

The present application relates to the field of ranging, and more specifically, relates to a time-of-flight module, a terminal and a depth detection method.

Background technique

Time of flight (ToF) is a ranging technology that calculates the distance between the target object and the sensor by measuring the time difference between the transmitted signal and the signal reflected by the target object.

Contents of the invention

Embodiments of the present application provide a time-of-flight module, a terminal, and a depth detection method.

The time-of-flight module of the embodiment of the present application includes a light source, a lens and an image sensor. The light source is used for emitting light. The lens is located on the side where the light source emits light. The image sensor includes a photosensitive pixel, a detection pixel and a timer. When the detection pixel receives the light reflected by the lens, the timer starts counting; when the photosensitive pixel receives the light reflected by the target object, the timer counts. The timer stops timing to obtain the receiving time; the image sensor generates the depth information of the target object according to the receiving time.

The terminal in the embodiment of the present application includes a casing and a time-of-flight module, and the time-of-flight module is arranged on the casing. The time-of-flight module includes a light source, a lens and an image sensor. The light source is used for emitting light. The lens is located on the side where the light source emits light. The image sensor includes a photosensitive pixel, a detection pixel and a timer. When the detection pixel receives the light reflected by the lens, the timer starts counting; when the photosensitive pixel receives the light reflected by the target object, the timer counts. The timer stops timing to obtain the receiving time; the image sensor generates the depth information of the target object according to the receiving time. The depth detection method in the embodiment of the present application is applied to a time-of-flight module, and the time-of-flight module includes a light source, a lens, and an image sensor. The light source is used for emitting light. The lens is located on the side where the light source emits light. The image sensor includes a photosensitive pixel, a detection pixel and a timer. When the detection pixel receives the light reflected by the lens, the timer starts counting; when the photosensitive pixel receives the light reflected by the target object, the timer counts. The timer stops timing to obtain the receiving time; the image sensor generates the depth information of the target object according to the receiving time.

The depth detection method in the embodiment of the present application is applied to a time-of-flight module, the time-of-flight module includes a light source, a lens and an image sensor, the lens is located on the side where the light source emits light, and the depth detection method includes: controlling The light source emits light; when the detection pixel of the image sensor receives the light reflected by the lens, the timer of the image sensor starts timing; when the photosensitive pixel of the image sensor receives the light reflected by the target object When the light is detected, the timer stops counting to generate a receiving time; according to the receiving time, the depth information of the target object is calculated.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and understandable from the description of the embodiments in conjunction with the following drawings, wherein:

FIG. 1 is a schematic structural diagram of a time-of-flight module in some embodiments of the present application;

Fig. 2 is a schematic diagram of the scene of the time-of-flight module of the prior art solution;

FIG. 3 is a schematic structural diagram of a terminal in some embodiments of the present application;

Figure 4 and Figure 5 are schematic diagrams of scenes of the time-of-flight module in some embodiments of the present application;

6 is a schematic diagram of a lens of a time-of-flight module according to some embodiments of the present application;

FIG. 7 is a schematic structural diagram of an image sensor of a time-of-flight module according to some embodiments of the present application.

8 to 9 are schematic plan views of image sensors in some embodiments of the present application;

FIG. 10 is a schematic diagram of a scene of an image sensor in some embodiments of the present application;

11 to 16 are schematic plan views of image sensors in some embodiments of the present application;

Fig. 17 is a schematic flowchart of a depth detection method in some embodiments of the present application.

Detailed ways

Embodiments of the present application are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary, and are intended to explain the present application, and should not be construed as limiting the present application.

In some implementations, the detection pixels are used to receive the light reflected by the lens to generate a trigger signal, and the timer starts counting when receiving the trigger signal; the photosensitive pixels are used to receive the target object the reflected light to generate a receiving signal, and the timer stops counting when receiving the receiving signal to generate a receiving time.

In some embodiments, the curvature of the lens is greater than a predetermined curvature.

In some embodiments, the time-of-flight module further includes a casing, the casing includes a substrate, a top board, a side board, and a spacer board, and the substrate, the top board, and the side boards enclose a receiving space, The light source and the image sensor are disposed on the substrate, the spacer plate is disposed on the top plate and located in the accommodation space, the spacer plate separates the light source and the image sensor, and the spacer plate A gap is formed between the substrate and the substrate, and the light reflected by the lens enters the detection pixel from the gap.

In some implementations, the detection pixels are located on the side of the image sensor close to the light source, and the width of the gap is based on the light reflected by the lens, which enters the detection pixels in the gap. The maximum height is determined.

In some embodiments, the image sensor further includes a first reflective member, the first reflective member is arranged on the side of the detection pixel close to the top plate and away from the light source, and the first reflective member is used for for reflecting the light reflected by the lens.

In some embodiments, the image sensor further includes a second reflective member, the second reflective member is arranged on the side of the detection pixel close to the top plate and close to the light source, and the second reflective member uses to reflect the light reflected by the first reflective member.

In some embodiments, the image sensor further includes a third reflective element and a fourth reflective element. The third reflective member and the fourth reflective member are arranged opposite to each other, and form a closed space with the first reflective member and the second reflective member.

In some embodiments, the photosensitive pixels and the detection pixels are arranged in a matrix, the detection pixels include pixels in a predetermined column in the matrix close to the light source, and the direction of the columns of the matrix is perpendicular to the image The arrangement direction of the sensor and the light source.

In some embodiments, both the photosensitive pixels and the detection pixels are single photon avalanche diodes.

In some embodiments, the light source comprises a VCSEL.

The time-of-flight module, terminal and depth detection method of the embodiment of the present application control the light source to emit light and reflect the light to the detection pixel through the lens. The time taken to stop counting when the light emitted by the target object is reached is the receiving time, and the image sensor can generate the depth information of the target object according to the receiving time. Among them, since the timer starts timing is based on the trigger signal generated by the detection pixel, and the moment when the timer receives the trigger signal (considering the extremely fast speed of light, the time for the lens to reflect light to the detection pixel can be ignored or deducted after pre-measurement) Therefore, the time to start timing is the actual light-emitting time of the light source, and the receiving time has nothing to do with the temperature drift of the light source, thereby eliminating the problem of inaccurate timing starting point caused by temperature drift, so as to ensure the accuracy of timing, thereby ensuring the image sensor to generate The accuracy of the depth information of the target object. Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Please refer to FIG. 1 , the embodiment of the present application provides a time-of-flight module 100 . The TOF module 100 includes a light source 20 , a lens 30 and an image sensor 40 . The light source 20 is used for emitting light. The lens 30 is located on the side where the light source 20 emits light. The image sensor 40 includes a photosensitive pixel 41 and a detection pixel 42. The detection pixel 42 is used to receive light reflected by the lens 30 to generate a trigger signal, and transmit the trigger signal to the photosensitive pixel 41; the photosensitive pixel 41 starts counting when receiving the trigger signal The image sensor 40 generates the depth information of the target object according to the receiving time of the photosensitive pixel 41 receiving the light reflected by the target object (shown in FIG. 2 ).

At present, when using the time-of-flight technology to calculate the distance between the target object and the image sensor 40, the emitted light and the light reflected by the target object are often measured by directly measuring the time-of-flight (direct Time of Flight, dToF) module. The time difference is used to calculate the distance between the target object and the image sensor 40 through the time difference.

As shown in Figure 2, the dToF module consists of three main components: drive control chip, photosensitive pixel and light emitting device. When the dToF module controls the driving control chip to send light-emitting instructions to the light-emitting device, and the photosensitive pixel receives the light reflected by the target object, it will experience a total of 4 moments, namely T0 moment, T1 moment, T2 moment and T3 moment. Among them, the time T0 is the moment when the dToF module controls the drive control chip to send the light-emitting command to the light-emitting device, the time T1 is the time when the drive control chip sends the light-emitting command, and the time T2 is the time when the light-emitting device responds to the light-emitting command to emit light , the moment T3 is the moment when the photosensitive pixel receives the light reflected back by the target object.

Specifically, when the dToF module calculates the time difference between the light emitted by the light-emitting device and the light emitted by the target object, theoretically speaking, the timer in the dToF module starts counting at T2 and ends at T3, then the best Accurate flight time. However, during operation, it takes a certain amount of time for the light-emitting device to receive the control signal from the drive control chip and actually emit light in response to the control signal, so that the actual light-emitting time T2 of the light-emitting device often lags behind T1. Moreover, in the actual working scene of the dToF module, the temperature of the light-emitting device will change, resulting in a change in the response speed of the light-emitting device from receiving the control signal to start emitting light to actually emitting light. For example, when the temperature is higher, the response speed of the light-emitting device is slower, which makes it difficult to define the time T2 (the actual light-emitting time of the light-emitting device). Therefore, the dToF module cannot use T2 time as the starting time to get the most accurate flight time.

Therefore, the timer will often use the T0 time as the time to start counting. However, the time T2 is difficult to define, and there is no fixed time difference between the time T0 and the time T2. Therefore, when calculating the flight time, the time error of the light-emitting device responding to the light-emitting command cannot be eliminated. It will cause the dToF module to fail to find an accurate timing starting point when measuring the flight time, resulting in inaccurate distance measurement, so that the image sensor 40 generates inaccurate depth information of the target object.

When using the direct Time of Flight (dToF) technology for distance measurement, since the light-emitting device in the dToF module will cause unstable response speed due to temperature changes, it is difficult to define the time to start timing. Ensure the accuracy of the final generated depth map.

The time-of-flight module 100 of the embodiment of the present application controls the light source 20 to emit light, and reflects the light to the detection pixel 42 through the lens 30. At this time, the timer 401 starts counting, and the timer 401 counts from the start to the photosensitive pixel 41. The time taken to stop counting when receiving the light emitted by the target object is the receiving time, and the image sensor 40 can generate the depth information of the target object according to the receiving time. Wherein, because the timer 401 starts counting time is to generate a trigger signal according to the detection pixel 41, and the moment when the timer 401 receives the trigger signal (considering that the speed of light is extremely fast, the time for the lens 30 to reflect the light to the detection pixel can be ignored or in advance deduction after calculation), therefore, the time to start counting is the actual light emitting time of the light source 20, and the receiving time has nothing to do with the temperature drift of the light source 20, thereby eliminating the inaccurate problem of the starting point of timing caused by temperature drift, so as to ensure the accuracy of timing , so as to ensure the accuracy of the depth information of the target object generated by the image sensor 40 .

Please make further explanation below in conjunction with accompanying drawing.

Referring to FIG. 3 , an embodiment of the present application provides a terminal 1000 . The terminal 1000 includes a time-of-flight module 100 and a housing 200 . The time-of-flight module 100 is disposed on the casing 200 .

The terminal 1000 can be, but not limited to, VR glasses, AR glasses, mobile phones, tablet computers, notebook computers, smart watches, game consoles, head-mounted display devices, laser rulers, etc., and these electronic devices often have a time-of-flight module 100 to realize A function to generate depth information of a target object.

Specifically, the time-of-flight module 100 can be a dTOF module, and the dTOF module can calculate the difference between the target object and the time-of-flight module 100 according to the emission time of the light emitted by the light source 20 and the time when the light is reflected back to the image sensor 40 by the target object. The distance between them is the depth information of the target object.

The casing 200 includes a side wall 201 and a back plate 202 at the bottom, and the side wall 201 and the back plate 202 form a receiving space 203 for accommodating components of the terminal 1000 . It can be understood that the time-of-flight module 100 is set in the storage space 203, and when the time-of-flight module 100 needs to generate the depth information of the target object, it can transmit the light source to the target object through the back plate 202, thereby Get the depth information of the target object.

Wherein, the material of the casing 200 may be metal, glass, plastic, etc., or the material of the casing 200 may be a mixture of metal, glass, and plastic. For example, the material of the side wall 201 is metal, and the material of the back plate 202 is glass. For another example, the material of the side wall 201 and part of the back plate 202 is metal, and the material of the other part of the back plate 202 is glass.

Please refer to FIG. 1 , the time-of-flight module 100 includes a casing 10 , a light source 20 , a lens 30 and an image sensor 40 . The light source 20 , the lens 30 and the image sensor 40 are disposed in the casing 10 . Wherein, the light source 20 is used to emit light, the lens 30 is used to reflect the light emitted by the light source 20 , and the image sensor 40 includes photosensitive pixels 41 and detection pixels 42 .

The housing 10 includes a substrate 11 , a top plate 12 , a side plate 13 and a partition plate 14 . Both ends of the side plate 13 are respectively connected to the substrate 11 and the top plate 12 , and one end of the spacer plate 14 is connected to the top plate 12 .

The substrate 11 is arranged in the casing 200, the substrate 11, the top plate 12 and the side plate 13 are surrounded by a housing space 15, the light source 20, the lens 30 and the image sensor 40 are accommodated in the housing space 15, and the light source 20 and the image sensor 40 is directly disposed on the substrate 11, and the terminal 1000 can power on the light source 20 and the image sensor 40 through the substrate 11 to ensure that the light source 20 and the image sensor 40 can work normally.

The spacer 14 is arranged in the accommodation space 15, and the spacer 14 is used for separating the light source 20 and the image sensor 40. When the light emitted by the light source 20 is reflected by the lens 30, the spacer 14 is used for blocking part of the light, so as to prevent the light from entering the containing space. When there is a part of the housing space 15 for the image sensor 40 , crosstalk light is formed, thereby reducing the interference of the crosstalk light on the image sensor 40 .

A gap 16 is formed between the spacer plate 14 and the substrate 11. When the light source 20 emits light and reflects the light through the lens 30, the light reflected by the lens 30 will be incident on the detection pixel 42 through the gap 16, so that the detection pixel 42 generates trigger signal. Wherein, the width of the gap 16 is determined at the maximum height of the gap 16 according to the light reflected by the lens 30 to the detection pixel 42 .

Specifically, the detection pixel 42 is located on the side of the image sensor 40 close to the light source 20. When the light emitted by the light source 20 is reflected to the detection pixel 42 through the lens 30, if there is no spacer 14, the light reflected by the lens 30 will fall on the On the photosensitive pixels 41 and the detection pixels 42 , but the light is not reflected by the target object, when the light is reflected on the photosensitive pixels 41 , it will interfere with the image sensor 40 . Therefore, in order to ensure that the light reflected by the lens 30 is reflected to the detection pixels 42 and not to the photosensitive pixels 41 , a spacer 14 is required to form a gap 16 between the spacer 14 and the substrate 11 .

As shown in FIG. 4 , the width of the gap 16 is related to the maximum height of the light reflected by the lens 30 at the gap 16 . When the light L is reflected to the position where the detection pixel 42 is closest to the photosensitive pixel 41, the distance H between the light L at the position of the spacer plate 14 and the substrate 11 should be such that the spacer plate 14 allows the light reflected by the lens 30 to pass through the gap 16 The maximum height of the light above the light L will theoretically be reflected to the photosensitive pixel 41 (such as light M, light X), and the spacer 14 needs to block the light higher than the height of the light L to ensure that this part of the light will not be reflected To the photosensitive pixel 41, therefore, the height of the gap 16 should be the distance H between the light L at the position of the spacer plate 14 and the substrate 11, so as to ensure that the light reflected by the lens 30 will only be reflected on the detection pixel 42, so as to reduce the Interference of crosstalk light on the image sensor 40 . At present, in the time-of-flight module 100, in order to reduce the impact of the image sensor 40 from crosstalk light, a spacer 14 is often provided between the light source 20 and the image sensor 40, and the spacer 14 is directly connected to the substrate 11 to prevent Turn off the light source 20 and the image sensor 40. However, in order to ensure the normal operation of the light source 20 and the image sensor 40, the side of the image sensor 40 close to the light source 20 needs to be connected to the circuit board of the substrate 11 with a connection line, and the connection line is located between the image sensor 40 and the light source 20. between.

In the time-of-flight module 100 of the embodiment of the present application, a gap 16 is formed between the spacer plate 14 and the substrate 11, and the connection line connecting the image sensor 40 to the circuit board of the substrate 11 can be arranged at the position of the gap 16 Therefore, there is no need to place the connection line, and the distance between the image sensor 40 and the light source 20 can be reduced, thereby reducing the required size of the time-of-flight module 100 .

The light source 20 may be a vertical-cavity surface-emitting laser (Vertical-Cavity Surface-Emitting Laser, VCSEL), an edge-emitting semiconductor laser (edge-emitting semiconductor lasers, EEL), and a light emitting diode (Light Emitting Diode, LED) and other light sources 20. These light sources 20 may be point light sources 20 composed of a single laser or diode, or array light sources 20 composed of multiple lasers or diodes. The light source 20 can emit laser light to the target object under the control of the time-of-flight module 100 for dTOF ranging. The light source 20 in the embodiment of the present application is a VCSEL, and it can be understood that the light source 20 is not limited to the VCSEL.

Please refer to FIG. 1 , the lens 30 is located on the side where the light source 20 emits light. In the time-of-flight module 100 of the embodiment of the present application, the curvature of the lens 30 needs to be greater than a preset curvature. Wherein, the preset curvature is 45 degrees.

When the light source 20 emits light to the lens 30, at this time, because the light enters the lens 30 from the air, that is, the light enters another medium from one medium, the light will be refracted by the lens 30, and the light will also form on the surface of the lens 30. reflection. The total energy of the light is fixed. When the light is reflected and refracted at the same time, the energy of the light will be divided into two parts, and the energy of the reflected light and the energy of the refracted light are interlinked to ensure that the total energy of the light does not change. Change. For example, when the energy of the reflected ray increases, the energy of the refracted ray decreases correspondingly, which is the same as the increase in the energy of the reflected ray.

When the lens 30 is a high-curvature lens, then when the light source 20 emits light to the surface of the lens 30, the incident angle of the light is larger than that of the low-curvature lens, resulting in stronger reflection of the light on the surface of the lens 30, and the light The greater the energy of the reflected light, the smaller the energy of the refracted light refracted by the lens 30 .

Please refer to FIG. 5 , when the light enters the lens 30 , since the lens 30 is a high-curvature lens, the energy of the refracted light is smaller than that of the light refracted by the low-curvature lens. The time-of-flight module 100 is installed in the terminal 1000. Since the material of the casing 200 of the terminal 1000 covering the time-of-flight module 100 is glass, the light incident on the casing 200 after being refracted by the lens 30 will be When the reflection is formed inside the casing 200, due to the weak energy of the light, the light is also weak when it is reflected by the casing 200 and then enters the time-of-flight module 100. The energy of the crosstalk light Z formed can reduce the impact of the crosstalk light Z caused by the reflection of the casing 200 on the time-of-flight module 100 .

In the time-of-flight module 100 of the embodiment of the present application, the curvature of the lens 30 needs to be greater than a predetermined curvature, wherein the predetermined curvature is 45 degrees. As shown in Figure 6, the curvature of the lens 30 is specifically expressed as the angle β between the line connecting the center point O of the lens 30 and the edge point P of the curved edge of the lens 30 and the optical axis K of the lens 30, the lens 30 The curvature must be greater than the preset curvature, that is, the included angle β must be greater than 45 degrees.

Please combine FIG. 1 and FIG. 7 , the image sensor 40 includes photosensitive pixels 41 , detection pixels 42 and a timer 401 . The timer 401 starts counting when the detection pixel 42 receives the light reflected by the lens 30 , and stops counting when the photosensitive pixel 41 receives the light reflected by the target object, so as to generate the receiving time.

Both photosensitive pixels 41 and detection pixels 42 are single photon avalanche diodes. The timer 401 is a timing circuit, for example, a time-to-digital converter (Time To Digital Converter, TDC) circuit. The photosensitive pixel 41, the detection pixel 42 and the timer 401 are connected through a circuit. After the photosensitive pixel 41 or the detection pixel 42 receives light, it can convert photons into electrons, and amplify it into a voltage change signal, so as to transmit it to the timer through the circuit 401, so as to trigger the timer 401 to perform the work of starting timing or stopping timing. The timer 401 can convert the time signal into a digital signal through the signal transmitted by the photosensitive pixel 41 and the detection pixel 42, so as to obtain the receiving time. For example, when the timer 401 starts counting, the time signal at the start time is converted into a digital signal, and when the timer 401 stops counting, the time signal at the time stop time is converted into a digital signal. The time difference between the counting time and the stop counting time, that is, the receiving time.

Specifically, when the detection pixel 42 receives the light reflected by the lens 30, it will generate a trigger signal, and the timer 401 can receive the trigger signal to start timing (starting from 0), and the photosensitive pixel 41 will receive the light reflected by the target object. , a receiving signal will be generated. Similarly, when the timer 401 receives the receiving signal, it will stop counting. Since the start counting starts counting from 0, the time for stopping counting is the time taken from the start counting to the stop counting. The timer 401 can generate the receiving time, and the image sensor 40 can generate the depth information of the target object according to the receiving time.

In one embodiment, the photosensitive pixel 41 and the detection pixel 42 can share a timer 401, that is, the circuits of the photosensitive pixel 41 and the detection pixel 42 are connected to a timer 401 at the same time, and the detection pixel 42 generates a trigger signal and the photosensitive pixel 41 generates a trigger signal. When receiving a signal, both the trigger signal and the received signal will be received by the same timer 401, and the timer 401 can perform the actions of starting and stopping timing respectively, so as to obtain the receiving time. In another embodiment, the photosensitive pixel 41 and the detection pixel 42 can be respectively connected to a timer 401 , that is, the image sensor 40 includes two timers 401 that act on the photosensitive pixel 41 and the detection pixel 42 respectively. When the detection pixel 42 generates a trigger signal, the timer 401 acting on the detection pixel 42 starts counting, and through the circuit, the timer 401 acting on the photosensitive pixel 41 starts counting, and the photosensitive pixel 41 acts on when generating the receiving signal. The timer 401 in the detection pixel 42 stops counting, so as to obtain the receiving time.

Specifically, the photosensitive pixels 41 and the detection pixels 42 are arranged in a matrix, and the detection pixels 42 include a predetermined column of pixels close to the light source 20 in the matrix. Wherein, the direction of the columns of the matrix is perpendicular to the arrangement direction of the image sensor 40 and the light source 20 .

As shown in FIG. 7 , the image sensors 40 are arranged in 5 rows and 6 columns, the column direction of the matrix is the A direction, the height direction of the image sensors 40 is the B direction, and the arrangement direction of the image sensors 40 and the light source 20 is the C direction. Wherein, the A direction is perpendicular to the B direction and the C direction, and the B direction is perpendicular to the C direction.

Wherein, the column of the image sensor 40 close to the light source 20 , that is, the sixth column is the detection pixels 42 . The other columns are photosensitive pixels 41, and the photosensitive pixels 41 are provided with a microlens array 43 (Micro Lens Array, MLA), and the microlens array 43 includes a plurality of microlenses 44 to ensure that the light reflected by the target object is After being refracted by 44 , it can be better incident into the photosensitive pixel 41 to improve the photosensitive effect of the photosensitive pixel 41 .

In one embodiment, after the light source 20 emits light, the light emitted to the edge of the lens 30 will be reflected by the lens 30 to be reflected to the detection pixel 42 , and at this time, the detection pixel 42 can generate a trigger signal. The light emitted to the middle position of the lens 30 can be refracted by the lens 30 to reach the target object, and reflected by the target object to the photosensitive pixel 41. At this time, the photosensitive pixel 41 can generate a receiving signal, and the timer 401 can receive a trigger Signal and receive signal to perform the work of starting and stopping timing, respectively, so as to determine the receiving time. During the whole process, since the trigger signal is generated when the light is emitted and reflected by the lens 30 onto the detection pixel 42, no matter whether the response speed of the light source 20 changes due to the influence of temperature, the timer 401 starts counting and stops Timing will not affect the receiving time, and the speed of light is relatively fast, and the time for the light to be reflected by the lens 30 to the detection pixel 42 is negligible, thereby ensuring that the image sensor 40 can accurately obtain the moment when the light source 20 emits light. In order to ensure the accuracy of the depth information of the target object generated by the image sensor 40 .

Please refer to FIG. 8 and FIG. 9 , in some implementations, the detection pixel 42 of the image sensor 40 may also be provided with a microlens 44 , and the microlens 44 is offset on the side of the detection pixel 42 close to the top plate 12 .

Specifically, the light reflected by the lens 30 to the detection pixel 42 needs to be refracted by the microlens 44 on the detection pixel 42, so that more light can enter the detection pixel 42, thereby improving the sensitivity of the detection pixel 42. strength.

More specifically, when the detection pixel 42 generates a trigger signal, the light needs to be incident on the inside of the detection pixel 42, for example, when the light is incident on the inside of the detection pixel 42 at 3 microns to 5 microns, then the detection pixel 42 can generate the trigger signal. As shown in FIG. 8 , when the detection pixel 42 is not provided with a microlens 44, it can be seen that theoretically, the light E will directly incident on the edge position of the detection pixel 42, and since the angle between the light E and the detection pixel 42 is relatively small is small, the light E cannot be well incident into the detection pixel 42 , resulting in a weak photosensitive intensity of the detection pixel 42 . When the detection pixel 42 is provided with a microlens 44 , the light E can be refracted by the microlens 44 to change the angle at which the light E enters the detection pixel 42 , thereby increasing the photosensitive intensity of the detection pixel 42 .

It should be noted that the time-of-flight module 100 in the method of this application can adjust the height and offset distance of the microlens 44 on the detection pixel 42 so that the microlens 44 on the detection pixel 42 can refract the lens 30 at the same time. After reflection, the light incident on the photosensitive pixel 41 and the detection pixel 42 refracts the light that should have entered the photosensitive pixel 41 to the detection pixel 42. On the one hand, the light entering the photosensitive pixel 41 is reduced to reduce the interference of crosstalk light , so as to improve the accuracy of the target object generated by the image sensor 40. On the other hand, the light incident on the photosensitive pixel 41 is refracted to the detection pixel 42, which improves the photosensitive intensity of the detection pixel 42, thereby improving the accuracy of generating the trigger signal.

From this, it can be seen that the microlens 44 on the detection pixel 42 is used to refract the light reflected by the lens 30 to the detection pixel 42, so that the light enters the detection pixel 42 at a large angle, so that the light can better enter the detection pixel 42 interior. In addition, the offset direction and offset distance of the microlens 44 on the detection pixel 42 need to be determined according to the angle of the light reflected by the lens 30 to the detection pixel 42 .

As shown in Figure 10(a), the incident angle of the light T1 reflected by the lens is θ1 (the angle between the light and the detection pixel 42). It can be seen that if the detection pixel 42 is not provided with a microlens 44, the light T1 will be directly incident on the photosensitive pixel 41, and when the detection pixel 42 is provided with a microlens 44, the light T1 is refracted at point Y1 of the microlens 44 on the detection pixel 42, but the light refracted by the microlens 44 on the detection pixel 42 T1 will still be incident on the photosensitive pixel 41 , at this time, it is necessary to shift the microlens 44 on the detection pixel 42 to the side close to the light source 20 to change the refracted position of the light T1 on the microlens 44 . As shown in Figure 10(b), after the microlens 44 on the detection pixel 42 is shifted to the side close to the light source 20, the position where the light T1 is refracted at the microlens 44 changes from Y1 point to Y2 point, at this time, It can be found that after the light T1 is refracted by the microlens 44 , the light T1 can be refracted to the detection pixel 42 . From this, it can be concluded that when the microlens 44 on the detection pixel 42 is shifted to the side close to the light source 20 , the incident position of light can be made closer to the detection pixel 42 .

Please combine Figure 10(a) and Figure 10(c), it can be seen that the incident angles of light T1 and light T2 are θ1 and θ2 respectively, and θ1 is greater than θ2, no matter whether microlens 44 is set on detection pixel 42, light T1 Both the light ray T2 and the light ray T2 will finally be incident on the photosensitive pixel 41 , but the incident position of the light ray T2 is farther away from the detection pixel 42 than the incident position of the light ray T1 . Then, the offset of the microlens 44 on the detection pixel 42 needs to be larger than that of the microlens 44 in FIG. d) as shown).

Therefore, it can be concluded that when the incident angle of the light is small, in order to ensure that the final incident position of the light is on the detection pixel 42 , it is necessary to shift the microlens 44 on the detection pixel 42 to the side close to the light source. And the smaller the incident angle of the light is, the larger the offset of the microlens 44 is.

Please refer to FIG. 11 and FIG. 12 , in some embodiments, the time-of-flight module 100 further includes a first reflective member 50 , and the first reflective member 50 is disposed on a side of the detection pixel 42 close to the top plate 12 and away from the light source 20 .

Specifically, the first reflector 50 is used to reflect the light reflected by the lens 30, so as to ensure that the light reflected by the lens 30 will not enter the photosensitive pixel 41, thereby ensuring that no crosstalk light will be formed inside the image sensor 40, so as to ensure the image The accuracy with which the sensor 40 generates depth information of the target object.

Wherein, the height of the first reflective member 50 is determined according to the maximum height of the light reflected by the lens 30 at the position of the first reflective member 50 . As shown in FIG. 11 , if the ray G is the ray with the highest height reflected by the lens 30 into the image sensor 40, then when the first reflective member 50 can reflect the ray G, it means that the ray reflected by the lens 30 is incident on the image sensor 40. When inside the image sensor 40, the light incident on the photosensitive pixel 41 will be reflected by the first reflective member 50, therefore, the height of the first reflective member 50 needs to be greater than the height when the light G is reflected to the position of the first reflective member 50, Therefore, it is ensured that the light reflected by the lens 30 will not enter the photosensitive pixel 41 , so as to ensure the accuracy of the depth information of the target object generated by the image sensor 40 .

Please refer to FIG. 13 and FIG. 14 , in some embodiments, the time-of-flight module 100 can also include the first reflective member 50 and the second reflective member 60 at the same time. The second reflector 60 is disposed on a side of the detection pixel 42 close to the top plate 12 and close to the light source 20 .

Specifically, the second reflective member 60 is used to reflect the light reflected by the first reflective member 50. When the light reflected by the lens 30 enters the first reflective member 50, the light will be reflected by the first reflective member 50 to the second The second reflective member 60 is reflected to the detection pixel 42 by the second reflective member 60 to generate a trigger signal. The detection pixel 42 can pass through the first reflector 50 and the second reflector 60 to receive the light that will theoretically be reflected by the lens 30 to the position of the photosensitive pixel 41, so as to increase the photosensitive intensity of the detection pixel 42, thereby increasing the generation of the trigger signal. accuracy. Wherein, the height of the second reflective member 60 is determined according to the maximum height of the light reflected by the first reflective member 50 at the position of the second reflective member 60 .

As shown in FIG. 13 , if the light R is reflected by the first reflective member 50 , it is reflected to the highest position of the second reflective member 60 . Then, when the second reflective member 60 can reflect light R, it means that the light reflected by the first reflective member 50 will be reflected by the second reflective member 60 to the detection pixel 42. Therefore, the height of the second reflective member 60 needs to be greater than The height when the light R is reflected to the position of the second reflective member 60, so as to ensure that the light reflected by the first reflective member 50 is reflected to the detection pixel 42, thereby increasing the photosensitive intensity of the detection pixel 42, so as to improve the generation of the trigger signal. accuracy.

It should be noted that the height of the second reflector 60 needs to be further adjusted according to the minimum height of the light reflected by the lens 30 when it is reflected to the position of the second reflector 60, and the height of the second reflector 60 needs to be less than The minimum height is to prevent the second reflector 60 from blocking the light reflected by the lens 30 from directly entering the detection pixel 42 .

Please refer to FIG. 15 , in some embodiments, a third reflective member 70 and a fourth reflective member 80 may also be disposed on the detection pixel 42 . Wherein, the third reflective member 70 and the fourth reflective member 80 are arranged opposite to each other, and form a closed space 90 surrounded by the first reflective member 50 and the second reflective member 60 .

Specifically, it is not guaranteed that all the light reflected by the lens 30 will be reflected on the first reflective member 50. As shown in FIG. , at this time, if the included angle α between the light U and the detection pixel 42 is small, it can be seen that theoretically, if the fourth reflective member 80 is not provided, the light U will not be incident into the detection pixel 42 in the end. Therefore, the light in this direction and at a smaller angle with the detection pixel 42 can be incident into the detection pixel 42, and the fourth reflective member 80 can be provided to block this type of light. At this time, the light will be captured by the fourth The reflection of the reflector 80 is the same as the principle of the first reflector 50 and the second reflector 60, and the third reflector 70 can be provided to ensure that the light reflected by the fourth reflector 80 will be blocked and reflected by the third reflector 70 to the detection pixel 42 so as to ensure that the incident light from any direction can pass through the cooperation of the first reflective member 50 , the second reflective member 60 , the third reflective member 70 and the fourth reflective member 80 to be incident on the detection pixel 42 , so as to increase the photosensitive intensity of the detection pixel 42, so as to improve the accuracy of generating the trigger signal.

In the time-of-flight module 100 of the embodiment of the present application, when the light source 20 emits light, the lens 30 will reflect the light to the detection pixel 42. When the detection pixel 42 receives the light, the timer 401 starts timing, and the When 41 receives the light reflected by the target object, the timing is stopped to obtain the receiving time, and the image sensor 40 can receive the time, that is, the flight time of the light, so as to generate the depth information of the target object.

Please refer to Figure 17, the embodiment of the present application provides a depth detection method, the depth detection method includes steps:

01: Control the light source 20 to emit light;

02: When the detection pixel 42 of the image sensor 40 receives the light reflected by the lens 30, the timer 401 of the image sensor 40 starts timing;

03: When the photosensitive pixel 41 of the image sensor 40 receives the light reflected by the target object, the timer 401 stops timing to generate the receiving time;

04: Calculate the depth information of the target object according to the receiving time.

Wherein, the depth detection method in the embodiment of the present application can be applied to the time-of-flight module 100 in the manner described in the present application.

Specifically, in the image sensor 40 , the circuits of the photosensitive pixel 41 and the detection pixel 42 are connected with the circuit of the timer 401 . Thus, when the detection pixel 42 of the image sensor 40 receives the light reflected by the lens 30, the timer 401 starts timing (the timer 401 starts counting from 0), and when the photosensitive pixel 41 receives the light reflected by the target object, The timer 401 stops counting. At this time, the time taken from the start of counting to the stop of counting is the receiving time, that is to say, the receiving time is the time of flight from when the light source emits light to when the light reaches the target object and is reflected to the photosensitive pixel 41 , when the flight time is known, the distance between the image sensor 40 and the target object can be calculated according to the following formula, that is, the depth information of the target object can be calculated.

d=(Δt*c)/2

Wherein, d is the distance between the image sensor 40 and the target object, Δt is the time difference calculated from the time when the timing starts to the time when the timing is stopped, that is to say, Δt is the receiving time, that is, the flight time of light, and c is the speed of light.

The depth detection method of the embodiment of the present application controls the light source 20 to emit light, and reflects the light to the detection pixel 42 through the lens 30. At this time, the timer 401 starts counting, and the timer 401 counts from the start until the photosensitive pixel 41 receives The time taken to stop counting the light emitted by the target object is the receiving time, and the image sensor 40 can generate the depth information of the target object according to the receiving time. Wherein, because the timer 401 starts counting time is to generate a trigger signal according to the detection pixel 41, and the moment when the timer 401 receives the trigger signal (considering that the speed of light is extremely fast, the time for the lens 30 to reflect the light to the detection pixel can be ignored or in advance deduction after calculation), therefore, the time to start counting is the actual light emitting time of the light source 20, and the receiving time has nothing to do with the temperature drift of the light source 20, thereby eliminating the inaccurate problem of the starting point of timing caused by temperature drift, so as to ensure the accuracy of timing , so as to ensure the accuracy of the depth information of the target object generated by the image sensor 40 .

In the description of this specification, descriptions with reference to the terms "certain embodiments", "in one example", "exemplarily" and the like mean that specific features, structures, materials or characteristics described in connection with the embodiments or examples are included in the In at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the present application includes additional implementations in which functions may be performed out of the order shown or discussed, including in substantially simultaneous fashion or in reverse order depending on the functions involved, which shall It should be understood by those skilled in the art to which the embodiments of the present application belong.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of said features. In the description of the present application, "plurality" means at least two, such as two, three, unless otherwise specifically defined.

Although the implementation of the present application has been shown and described above, it can be understood that the above-mentioned implementation is exemplary and should not be construed as limiting the application, and those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims

A time-of-flight module, comprising:

a light source for emitting light;

a lens, the lens is located on the side where the light source emits light;

The image sensor includes a photosensitive pixel, a detection pixel and a timer. When the detection pixel receives the light reflected by the lens, the timer starts counting; when the photosensitive pixel receives the light reflected by the target object, the timer starts counting. The timer is stopped to obtain the receiving time; the image sensor generates the depth information of the target object according to the receiving time.
The time-of-flight module according to claim 1, wherein the detection pixels are used to receive the light reflected by the lens to generate a trigger signal, and the timer starts counting when receiving the trigger signal; The photosensitive pixels are used to receive light reflected by the target object to generate a receiving signal, and the timer stops timing when receiving the receiving signal to generate a receiving time.
The time-of-flight module according to claim 1, wherein the curvature of the lens is greater than a preset curvature.
The time-of-flight module according to claim 1, further comprising a housing, the housing comprising a substrate, a top plate, a side plate and a spacer plate, the substrate, the top plate and the side plates enclose The accommodating space, the light source and the image sensor are arranged on the substrate, the spacer plate is arranged on the top plate and located in the accommodating space, the spacer plate separates the light source and the image sensor, the A gap is formed between the spacer plate and the substrate, and the light reflected by the lens enters the detection pixel from the gap.
The time-of-flight module according to claim 4, wherein the detection pixels are located on the side of the image sensor close to the light source, and the width of the gap is determined according to the light reflected by the lens. The maximum height of the shot into the detection pixel is determined.
The time-of-flight module according to claim 1, wherein the image sensor further includes a microlens, and the microlens is offset and disposed on a side of the detection pixel close to the top plate.
The time-of-flight module according to claim 1, wherein the image sensor further comprises a first reflective member, the first reflective member is arranged on a side of the detection pixel close to the top plate and away from the light source, The first reflector is used to reflect the light reflected by the lens.
The time-of-flight module according to claim 7, wherein the image sensor further comprises a second reflective member, the second reflective member is arranged on a side of the detection pixel close to the top plate and close to the light source, The second reflector is used to reflect the light reflected by the first reflector.
The time-of-flight module according to claim 8, wherein the image sensor further comprises a third reflective member and a fourth reflective member. The third reflective member and the fourth reflective member are arranged opposite to each other and form a closed space with the first reflective member and the second reflective member.
The time-of-flight module according to claim 1, wherein the photosensitive pixels and the detection pixels are arranged in a matrix, the detection pixels include pixels in a predetermined column in the matrix close to the light source, and the pixels of the matrix The direction of the columns is perpendicular to the arrangement direction of the image sensors and the light sources.
The time-of-flight module according to claim 1, wherein the photosensitive pixels and the detection pixels are single photon avalanche diodes.
The time-of-flight module of claim 1, wherein the light source comprises a VCSEL.
A terminal, including

chassis;

A time-of-flight module, the time-of-flight module is arranged on the casing, and the time-of-flight module includes:

a light source for emitting light;

a lens, the lens is located on the side where the light source emits light;

The image sensor includes a photosensitive pixel, a detection pixel and a timer. When the detection pixel receives the light reflected by the lens, the timer starts counting; when the photosensitive pixel receives the light reflected by the target object, the timer starts counting. The timer is stopped to obtain the receiving time; the image sensor generates the depth information of the target object according to the receiving time.
The terminal according to claim 13, wherein the detection pixel is used to receive the light reflected by the lens to generate a trigger signal, and the timer starts counting when receiving the trigger signal; The light reflected by the target object is received to generate a receiving signal, and the timer stops timing when receiving the receiving signal to generate a receiving time.
The terminal according to claim 13, wherein the curvature of the lens is greater than a preset curvature.
The terminal according to claim 13, wherein the time-of-flight module further comprises a casing, the casing includes a substrate, a top plate, a side plate and a spacer plate, the substrate, the top plate and the side plates The plate encloses a storage space, the light source and the image sensor are arranged on the substrate, the spacer plate is set on the top plate and is located in the storage space, and the spacer plate separates the light source and the image sensor In the sensor, a gap is formed between the spacer plate and the substrate, and the light reflected by the lens enters the detection pixel from the gap.
The terminal according to claim 16, wherein the detection pixels are located on the side of the image sensor close to the light source, and the width of the gap is based on the light reflected by the lens, which is incident on the gap. The maximum height of the detected pixels is determined.
The terminal according to claim 13, wherein the image sensor further includes a microlens, and the microlens is offset and disposed on a side of the detection pixel close to the top plate.
The terminal according to claim 13, wherein the image sensor further includes a first reflective member, the first reflective member is disposed on a side of the detection pixel close to the top plate and away from the light source, the first reflective member A reflector is used to reflect the light reflected by the lens.
The terminal according to claim 19, wherein the image sensor further includes a second reflective member, the second reflective member is arranged on a side of the detection pixel close to the top plate and close to the light source, the first reflective member The two reflective elements are used to reflect the light reflected by the first reflective element.
The terminal according to claim 20, wherein the image sensor further comprises a third reflective member and a fourth reflective member. The third reflective member and the fourth reflective member are arranged opposite to each other, and form a closed space with the first reflective member and the second reflective member.
The terminal according to claim 13, wherein the photosensitive pixels and the detection pixels are arranged in a matrix, the detection pixels include pixels in a predetermined column close to the light source in the matrix, and the direction of the columns of the matrix perpendicular to the arrangement direction of the image sensor and the light source.
The terminal according to claim 13, wherein the photosensitive pixels and the detection pixels are single photon avalanche diodes.
The terminal of claim 13, wherein the light source comprises a VCSEL.
A depth detection method, which is applied to a time-of-flight module, the time-of-flight module includes a light source, a lens, and an image sensor, the lens is located on the side where the light source emits light, and the depth detection method includes:

controlling the light source to emit light;

When the detection pixel of the image sensor receives the light reflected by the lens, the timer of the image sensor starts timing;

When the photosensitive pixel of the image sensor receives the light reflected by the target object, the timer stops counting to generate a receiving time;

Calculate the depth information of the target object according to the receiving time.