CN110603457A

CN110603457A - Image sensing system and electronic device

Info

Publication number: CN110603457A
Application number: CN201880000680.3A
Authority: CN
Inventors: 杨孟达
Original assignee: Shenzhen Goodix Technology Co Ltd
Current assignee: Shenzhen Goodix Technology Co Ltd
Priority date: 2018-04-12
Filing date: 2018-04-12
Publication date: 2019-12-20
Also published as: WO2019196049A1

Abstract

An image sensing system (10) includes a plurality of time-of-flight ranging pixel cells (11), each including a first pixel circuit (PX1) including a first photosensitive element (PD1) and a first transfer gate (TG1) that receives a first transfer signal (TX1) and outputs a first pixel output signal (Pout 1); and a second pixel circuit (PX2) including a second photosensitive element (PD2) and a second transfer gate (PD2) that receives a second transfer signal (TX2) and outputs a second pixel output signal (Pout 2); a control unit (14) for generating a first transmission signal and a second transmission signal; and a depth calculation unit (16) for calculating a time-of-flight depth value from the first pixel output signal and the second pixel output signal.

Description

Image sensing system and electronic device

Technical Field

The present disclosure relates to image sensing systems and electronic devices, and more particularly, to an image sensing system and an electronic device capable of generating a time-of-flight depth image and a general image simultaneously.

Background

With the rapid development of science and technology, the acquisition of three-dimensional information of objects has wide application prospects in various application fields, such as production automation, human-computer interaction, medical diagnosis, reverse engineering, digital modeling and the like. The structured light three-dimensional measurement method is widely applied as a non-contact three-dimensional information acquisition technology due to the advantages of simplicity in implementation, high speed, high precision and the like.

Time of Flight (ToF) ranging is a commonly used three-dimensional depth measurement method. However, the time-of-flight depth image required for time-of-flight ranging is different from the requirement of an image generated by a general camera (referred to as a general image for short), and the general image requires high resolution, while the time-of-flight depth image requires good sensitivity. In the prior art, two different photosensitive pixel arrays are required to obtain a normal image and a flight depth image, which results in an increase in production cost.

Accordingly, there is a need for improvement in the art.

Disclosure of Invention

It is therefore an object of some embodiments of the present invention to provide an image sensing system and an electronic device capable of generating a depth image in flight and a general image simultaneously, so as to overcome the disadvantages of the prior art.

In order to solve the above technical problem, an embodiment of the present invention provides an image sensing system, including a plurality of time-of-flight ranging pixel units, each of the time-of-flight ranging pixel units includes a first pixel circuit including a first photosensitive element and a first transmission gate, the first transmission gate is coupled to the first photosensitive element, the first transmission gate receives a first transmission signal and is turned on for a first on-time, and the first pixel circuit outputs a first pixel output signal for the first on-time; and a second pixel circuit including a second photosensitive element and a second transfer gate, the second transfer gate being coupled to the second photosensitive element, the second transfer gate receiving a second transfer signal and being turned on for a second on-time, the second pixel circuit outputting a second pixel output signal for the second on-time; a control unit, coupled to the first transmission gate and the second transmission gate of each time-of-flight ranging pixel unit, for generating the first transmission signal to the first transmission gate and the second transmission signal to the second transmission gate; and a depth calculation unit, coupled to the plurality of time-of-flight ranging pixel units, for calculating a time-of-flight depth value corresponding to each of the time-of-flight ranging pixel units according to the first pixel output signal and the second pixel output signal; and a light emitting unit for emitting light during the first on-time.

For example, the first and second pixel circuits of the time-of-flight ranging pixel units are arranged in a first array, the time-of-flight ranging pixel units are arranged in a second array, the first array generates a first image, the second array generates a second image, and a first resolution of the first image is greater than a second resolution of the second image.

For example, the first pixel circuit includes a first output transistor coupled to the first transmission gate; and a first reading transistor coupled to the first output transistor for outputting the first pixel output signal; the second pixel circuit comprises a second output transistor coupled to the second transmission gate; and a second reading transistor coupled to the second output transistor for outputting the second pixel output signal.

For example, the first on-time and the second on-time are separated by a time interval, and the depth calculating unit calculates the time-of-flight depth value as D_ToF(Pout2)/(Pout1+ Pout2) × (c × T); wherein D is_ToFRepresenting the value of the time-of-flight depth, Pout1 representing the first pixel output signal, Pout2 representing the second pixel output signal, c representing the speed of light, and T representing the length of time of the first on-time or the second on-time.

For example, each time-of-flight ranging pixel unit further includes a third pixel circuit including a third photosensitive element and a third transfer gate, the third transfer gate being coupled to the third photosensitive element, the third transfer gate receiving a third transfer signal and being turned on for a third on-time when the light-emitting unit does not emit light, the third pixel circuit outputting a third pixel output signal for the third on-time; wherein the control unit generates the third transmission signal to the third transmission gate; wherein the depth calculation unit calculates the depth value corresponding to the time of flight according to the first pixel output signal, the second pixel output signal, and the third pixel output signal.

For example, the third pixel circuit includes a third output transistor coupled to the third transmission gate; and a third read transistor coupled to the third output transistor for outputting the third pixel output signal.

For example, the first on-time and the second on-time are separated by a time interval, and the depth calculating unit calculates the time-of-flight depth value as D_ToF(Pout2-Pout3)/(Pout1+ Pout 2-2 Pout3) × (c × T); wherein D is_ToFRepresenting the time-of-flight depth value, Pout1 representing the first pixel output signal, Pout2 representing the second pixel output signal, Pout3 representing the third pixel output signal, c representing the speed of light, and T representing the length of time of the first on-time or the second on-time.

In order to solve the above technical problem, an embodiment of the present invention provides an electronic device, including an image sensing system, where the image sensing system includes a plurality of time-of-flight ranging pixel units, each of the time-of-flight ranging pixel units includes a first pixel circuit including a first photosensitive element and a first transmission gate, the first transmission gate is coupled to the first photosensitive element, the first transmission gate receives a first transmission signal and is turned on for a first on-time, and the first pixel circuit outputs a first pixel output signal for the first on-time; and a second pixel circuit including a second photosensitive element and a second transfer gate, the second transfer gate being coupled to the second photosensitive element, the second transfer gate receiving a second transfer signal and being turned on for a second on-time, the second pixel circuit outputting a second pixel output signal for the second on-time; a control unit, coupled to the first transmission gate and the second transmission gate of each time-of-flight ranging pixel unit, for generating the first transmission signal to the first transmission gate and the second transmission signal to the second transmission gate; a depth calculation unit, coupled to the plurality of time-of-flight ranging pixel units, for calculating a time-of-flight depth value corresponding to each of the time-of-flight ranging pixel units according to the first pixel output signal and the second pixel output signal; and a light emitting unit for emitting light during the first on-time.

The method can simultaneously meet the high resolution requirement of the common image and the sensitivity requirement of the flight depth image, only a single group of photosensitive pixel arrays are used for simultaneously generating the common image and the flight depth image, and the method has the advantages of saving the production cost and effectively utilizing the pixel arrays.

Drawings

Fig. 1 is a schematic diagram of an image sensing system according to an embodiment of the present disclosure;

FIG. 2 is a diagram of a first pixel circuit and a second pixel circuit according to an embodiment of the present disclosure;

FIG. 3 is a waveform diagram of a plurality of signals according to an embodiment of the present application;

FIG. 4 is a schematic view of an image sensing system according to an embodiment of the present disclosure;

FIG. 5 is a diagram of a third pixel circuit according to an embodiment of the present application;

FIG. 6 is a schematic view of an electronic device according to an embodiment of the present disclosure;

FIG. 7 is a schematic view of an image sensing system according to an embodiment of the present disclosure;

FIG. 8 is a schematic view of an image sensing system according to an embodiment of the present disclosure;

FIG. 9 is a waveform diagram of a plurality of signals according to an embodiment of the present application;

FIG. 10 is a waveform diagram of a plurality of signals according to an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

Referring to fig. 1 and fig. 2 together, fig. 1 is a schematic diagram of an image sensing system 10 according to an embodiment of the present disclosure, and fig. 2 is a schematic diagram of a first pixel circuit PX1 and a second pixel circuit PX2 according to an embodiment of the present disclosure. The image sensing system 10 can be used to generate Time of Flight (ToF) depth images as well as general images, which can be color or black and white images generated by general cameras. The image sensing system 10 includes a light emitting unit 12, a plurality of time-of-flight ranging pixel units 11, a control unit 14, and a depth calculating unit 16. The Light Emitting unit 12 is used for performing the time-of-flight distance measurement, and may be a Light-Emitting Diode (LED), such as an Infrared (IR) LED, and the Light Emitting unit 12 receives the Light Emitting signal LD and emits incident Light. The time-of-flight ranging pixel unit 11 includes a first pixel circuit PX1 and a second pixel circuit PX2, each of the first pixel circuit PX1 and the second pixel circuit PX2 may output a pixel value to form a general image, and on the other hand, when the image sensing system 10 performs time-of-flight ranging, the plurality of time-of-flight ranging pixel units 11 are configured to receive the reflected light corresponding to the light emitting units 12, and the depth calculating unit 16 may generate a time-of-flight depth image according to the outputs of the plurality of time-of-flight ranging pixel units 11.

As shown in fig. 2, the first pixel circuit PX1 includes a Photo sensing element PD1, a transmission gate TG1, an output transistor DV1, a read transistor RD1, a reset transistor RT1, and an Anti-Blooming (Anti-Blooming) transistor AB1, and the second pixel circuit PX2 includes a Photo sensing element PD2, a transmission gate TG2, an output transistor DV2, a read transistor RD2, a reset transistor RT2, and an Anti-Blooming transistor AB2, wherein the Photo sensing elements PD1, PD2 may be Photo diodes (Photo diodes). The circuit structures of the first pixel circuit PX1 and the second pixel circuit PX2 are shown in fig. 2a and fig. 2b, respectively. The transmission gate TG1 of the first pixel circuit PX1 receives the first transmission signal TX1, and the transmission gate TG2 of the second pixel circuit PX2 receives the second transmission signal TX 2. The read transistor RD1 outputs a first pixel output signal Pout1, and the read transistor RD2 outputs a second pixel output signal Pout 2. In addition, the reset transistors RT1, RT2 receive the reset signal Rst, the anti-blooming transistors AB1, AB2 receive the anti-blooming signals TX1 ', TX 2', and the read transistors RD1, RD2 receive the read signal RS.

When the image sensing system 10 performs the time-of-flight ranging, the control unit 14 generates the first transmission signal TX1 to the transmission gate TG1 of the first pixel circuit PX1 and generates the second transmission signal TX2 to the transmission gate TG2 of the second transmission signal TX2, and the depth calculating unit 16 receives the first pixel output signal Pout1 output from the reading transistor RD1 of the first pixel circuit PX1 and the second pixel output signal Pout2 output from the reading transistor RD2 of the second pixel circuit PX2, and calculates the time-of-flight depth value D corresponding to the time-of-flight ranging pixel unit 11 according to the first pixel output signal Pout1 and the second pixel output signal Pout2_ToF。

Referring to fig. 3, fig. 3 is a waveform diagram illustrating the light-emitting signal LD, the first transmission signal TX1 and the second transmission signal TX2 when the image sensing system 10 operates in the time-of-flight ranging mode. As shown in fig. 3, the light emitting unit 12 receives the light emitting signal LD to emit light in the first on-time T1, and the electrical signal converted from the reflected light passing through the light sensing element can be represented as the reflected signal RX in fig. 3. The transmission gate TG1 of the first pixel circuit PX1 is turned on for a first on-time T1, and the transmission gate TG2 of the second pixel circuit PX2 is turned on for a second on-time T2. In addition, the photo sensing element PD1 receives the reflected light during the first on time T1 and the reading transistor RD1 outputs a first pixel output signal Pout1, and the photo sensing element PD2 receives the reflected light during the second on time T2 and the reading transistor RD2 outputs a second pixel output signal Pout 2. In one embodiment, the first on-time T1 is separated from the second on-time T2 by a time interval Δ T.

The on-times T1 and T2 of the transmission gates TG1 and TG2 partially overlap with the corresponding reflected light (as shown by the hatching in fig. 3), wherein the overlapping portion of the on-time T1 and the reflected light time is related to the first pixel output signal Pout1, and the overlapping portion of the on-time T2 and the reflected light time is related to the second pixel output signal Pout 2. The time-of-flight ranging method can be used for the reflected light at the on-time T1 and the guide according to the time length of the reflected light at the on-time T2Calculating a depth-of-flight value D by a proportion of the length of time that the time T2 occurs_ToFIn other words, the depth calculation unit 16 may calculate the in-flight depth value D_ToFIs D_ToF(Pout2)/(Pout1+ Pout2) × (c × T), where c represents the speed of light and T represents the time length of the on-time T1 or T2. In this way, the image sensing system 10 can generate the in-flight depth image.

On the other hand, since the plurality of first pixel circuits PX1 and the plurality of second pixel circuits PX2 of the image sensing system 10 are also arranged in an array, and the pixel output signals Pout1 and Pout2 can be regarded as the pixel values of the pixel circuits PX1 and PX2, the array formed by the plurality of first pixel circuits PX1 and the plurality of second pixel circuits PX2 can also be used to form a general image. The plurality of first pixel circuits PX1 and the plurality of second pixel circuits PX2 may be arranged in a first array M1, and the plurality of time-of-flight ranging pixel units 11 may be arranged in a second array M2. Taking the embodiment shown in fig. 1 as an example, the Size (Size) of the first array M1 is 8 × 8, and the Size of the second array M2 is 4 × 8. The first array M1 may be used to generate an 8 x 8 generic image (which may correspond to the first image in the claims), while the second array M2 may be used to generate a 4 x 8 in-flight depth image (which may correspond to the second image in the claims). Therefore, the first Resolution (Resolution) of the general image generated by the first array M1 is greater than the second Resolution of the in-flight depth image generated by the second array M2.

The requirement of the general image is different from that of the time-of-flight depth image, the general image requires high resolution, and the time-of-flight depth image requires better sensitivity (i.e. the photosensitive element requires larger photosensitive area). Therefore, the conventional technique uses two different sets of photosensitive pixel arrays to obtain a general image and a time-of-flight depth image, which results in an increase in production cost. In contrast, the present application uses the first array M1 formed by arranging the plurality of first pixel circuits PX1 and the plurality of second pixel circuits PX2 to generate the general image, and uses the second array M2 formed by arranging the plurality of time-of-flight ranging pixel units 11 to generate the time-of-flight depth image, so that the present application can simultaneously satisfy the high resolution requirement of the general image and the sensitivity requirement of the time-of-flight depth image (wherein the photosensitive area of the time-of-flight ranging pixel units 11 is the sum of the photosensitive elements PD1 and PD2), in other words, the present application only needs one set of photosensitive pixel arrays, and can simultaneously generate the general image and the time-of-flight depth image, thereby saving the production cost and effectively utilizing the pixel arrays.

It should be noted that the above-mentioned embodiments are provided to illustrate the inventive concept of the present application, and those skilled in the art can make various modifications without departing from the scope of the present invention. For example, please refer to fig. 4 and fig. 5 together, in which fig. 4 is a schematic diagram of an image sensing system 40 according to an embodiment of the present application, and fig. 5 is a schematic diagram of a third pixel circuit PX3 according to an embodiment of the present application. The image sensor system 40 is similar to the image sensor system 10, and different from the image sensor system 10, the time-of-flight ranging pixel unit 41 (compared to the time-of-flight ranging pixel unit 11) included in the image sensor system 40 further includes a third pixel circuit PX3, the third pixel circuit PX3 is configured to receive the background light, and the output third pixel output signal Pout3 is configured to remove the background light component from the first pixel output signal Pout1 and the second pixel output signal Pout 2. The third pixel circuit PX3 includes a photo sensor PD3, a transfer gate TG3, an output transistor DV3, a read transistor RD3, a reset transistor RT3, and an anti-blooming transistor AB3, and its circuit structure is shown in fig. 5. Similarly, the reset transistor RT3 receives the reset signal Rst, the anti-blooming transistor AB3 receives the anti-blooming signal TX 3', and the read transistor RD3 receives the read signal RS.

The transmission gate TG3 receives a third transmission signal TX3, wherein a waveform diagram of the third transmission signal TX3 is also shown in fig. 3, and the transmission gate TG3 is turned on for a third on-time T3 when the light emitting unit 12 does not emit light. At the third on-time T3 shown in fig. 3, the photo sensor PD3 receives the background light and the readout transistor RD3 outputs the third pixel output signal Pout 3. When the image sensing system 40 performs the time-of-flight ranging, the control unit 44 generates transmission signals TX1, TX2, TX3 to transmission gates TG1, TG2, TG3 of the time-of-flight ranging pixel unit 41, the depth calculating unit 46 receives pixel output signals Pout1, Pout2, Pout3 output by the pixel circuits PX1, PX2, PX3, and calculates a time-of-flight depth value D corresponding to the time-of-flight ranging pixel unit 41 based on the pixel output signals Pout1, Pout2, Pout3_ToFWherein the depth calculation unit 16 may calculate the depth-of-flight value D_ToFIs D_ToF＝(Pout2-Pout3)/(Pout1+Pout2–2*Pout3)*(c*T)。

Thus, taking the embodiment shown in fig. 4 as an example, the size of the array M1 'of the plurality of pixel circuits PX1, PX2, PX3 is 9 × 8, which can be used to generate a general image, and the size of the array M1' of the plurality of time-of-flight ranging pixel units 41 is 3 × 8, which can be used to generate a time-of-flight depth image. Similarly, it is within the scope of the present application that the resolution of the general image generated by array M1 'is greater than the resolution of the in-flight depth image generated by array M2'.

In addition, the pixel circuits in the time-of-flight ranging pixel unit can be arranged according to actual needs, and are not limited to be arranged in the same row (or the same column) as in fig. 1 or fig. 4. For example, referring to fig. 7 and 8, fig. 7 and 8 are schematic diagrams of

image sensing systems

70 and 80, respectively, according to an embodiment of the present disclosure. The

image sensing systems

70, 80 are similar to the image sensing systems 10, 40, respectively, and like elements are labeled with like reference numerals. In fig. 1, the pixel circuits PX1/PX2 in the time-of-flight ranging pixel unit 11 are arranged in the same row, and in fig. 7, the pixel circuits PX1 and PX2 in the time-of-flight ranging pixel unit 71 are arranged alternately, which also meets the requirements of the present application and belongs to the scope of the present application. In addition, in fig. 4, the pixel circuits PX1, PX2, and PX3 are arranged in a 3 × 1 array in the time-of-flight ranging pixel unit 40, and in fig. 8, the pixel circuits PX1, PX2, and PX3 in the time-of-flight ranging pixel unit 81 are arranged in a mutually perpendicular manner, which also meets the requirements of the present application and belongs to the scope of the present application.

Referring to fig. 9 and 10, fig. 9 is a waveform diagram of the light-emitting signal LD, the transmission signals TX1, TX2 and TX3 when the image sensing system 10/40 operates in the invisible (e.g., infrared) grayscale image capture mode, and fig. 10 is a waveform diagram of the reset signal Rst, the light-emitting signal LD, the transmission signals TX1 to TX3 and the anti-blooming signals TX1 'to TX 3' when the image sensing system 10/40 operates in the normal camera mode. Similarly, the reflection signal RX in fig. 9 corresponds to the incident light emitted by the light emitting unit 12, and as shown in fig. 9, the conducting intervals T1, T2 and T3 of the transmission gates TG1, TG2 and TG3 need to cover the arrival time of the reflected light.

In addition, in the normal camera mode, the light emitting unit 12 does not emit light, and after the reset transistors RT1, RT2, RT2 are reset, the light sensing elements PD1, PD2, PD3 start exposure, and the anti-blooming transistors AB1, AB2, AB3 are turned off. After a period of exposure, the transmission gates TG1, TG2, TG3 are turned on to extract the photoelectrons stored in the photosensitive elements PD1, PD2, PD 3. At reading and time t_RDThen, the read transistors RD1, RD2, RD3 are turned on, and the anti-blooming transistors AB1, AB2, AB3 are also turned on.

In addition, the image sensing system of the present application can be disposed in an electronic device such as a mobile phone or a tablet computer. Referring to fig. 6, fig. 6 is a schematic view of an electronic device 6 according to an embodiment of the disclosure. The electronic device 6 includes an image sensing system 60, and the image sensing system 60 can be implemented by the image sensing system 10 or the image sensing system 40.

In summary, the present application can simultaneously satisfy the high resolution requirement of the general image and the sensitivity requirement of the time-of-flight depth image, and only a single set of photosensitive pixel arrays is used to simultaneously generate the general image and the time-of-flight depth image. Compared with the prior art, the pixel array has the advantages of saving production cost and effectively utilizing the pixel array.

The above description is only exemplary of the present application and should not be taken as limiting the present application, and any modifications, equivalents, improvements, etc. made within the spirit and principle of the present application should be included in the scope of the present application.

Claims

An image sensing system, comprising:

a plurality of time-of-flight ranging pixel units, each time-of-flight ranging pixel unit comprising:

a first pixel circuit including a first photosensitive element and a first transfer gate, the first transfer gate being coupled to the first photosensitive element, the first transfer gate receiving a first transfer signal and being turned on for a first on-time, the first pixel circuit outputting a first pixel output signal for the first on-time; and

a second pixel circuit including a second photosensitive element and a second transfer gate, the second transfer gate being coupled to the second photosensitive element, the second transfer gate receiving a second transfer signal and being turned on for a second on-time, the second pixel circuit outputting a second pixel output signal for the second on-time;

a control unit, coupled to the first transmission gate and the second transmission gate of each time-of-flight ranging pixel unit, for generating the first transmission signal to the first transmission gate and the second transmission signal to the second transmission gate;

a depth calculation unit, coupled to the plurality of time-of-flight ranging pixel units, for calculating a time-of-flight depth value corresponding to each of the time-of-flight ranging pixel units according to the first pixel output signal and the second pixel output signal; and

and the light emitting unit is used for emitting light in the first conduction time.
The image sensing system of claim 1, wherein the first and second pluralities of pixel circuits of the time-of-flight ranging pixel units are arranged in a first array, the time-of-flight ranging pixel units are arranged in a second array, the first array producing a first image and the second array producing a second image, the first image having a first resolution greater than a second resolution of the second image.
The image sensing system of claim 1, wherein the first pixel circuit comprises:

a first output transistor coupled to the first transmission gate; and

a first reading transistor coupled to the first output transistor for outputting the first pixel output signal;

the second pixel circuit includes:

a second output transistor coupled to the second transmission gate; and

a second reading transistor coupled to the second output transistor for outputting the second pixel output signal.
The image sensing system of claim 1, wherein the first on-time and the second on-time are separated by a time interval, and the depth calculation unit calculates the depth-of-flight value as

D_ToF＝(Pout2)/(Pout1+Pout2)*(c*T)；

Wherein D is_ToFRepresenting the value of the time-of-flight depth, Pout1 representing the first pixel output signal, Pout2 representing the second pixel output signal, c representing the speed of light, and T representing the length of time of the first on-time or the second on-time.
The image sensing system of claim 1, wherein each time-of-flight ranging pixel cell further comprises:

a third pixel circuit including a third photosensitive element and a third transfer gate, the third transfer gate being coupled to the third photosensitive element, the third transfer gate receiving a third transfer signal and being turned on for a third on-time when the light emitting unit does not emit light, the third pixel circuit outputting a third pixel output signal for the third on-time;

wherein the control unit generates the third transmission signal to the third transmission gate;

wherein the depth calculation unit calculates the depth value corresponding to the time of flight according to the first pixel output signal, the second pixel output signal, and the third pixel output signal.
The image sensing system of claim 5, wherein the third pixel circuit comprises:

a third output transistor coupled to the third transmission gate; and

a third reading transistor coupled to the third output transistor for outputting the third pixel output signal.
The image sensing system of claim 5, wherein the first on-time and the second on-time are separated by a time interval, and the depth calculation unit calculates the depth-of-flight value as

D_ToF＝(Pout2-Pout3)/(Pout1+Pout2–2*Pout3)*(c*T)；

Wherein D is_ToFRepresenting the time-of-flight depth value, Pout1 representing the first pixel output signal, Pout2 representing the second pixel output signal, Pout3 representing the third pixel output signal, c representing the speed of light, and T representing the length of time of the first on-time or the second on-time.
An electronic device comprising the image sensing system of any one of claims 1-7.