JP7358771B2 - 3D imaging unit, camera, and 3D image generation method - Google Patents

Description

The present invention relates to a 3D imaging unit that captures three-dimensional (3D) images including distance, and in particular, a CMOS image sensor (CIS) in which pixels are arranged in a matrix, a 3D imaging unit using this CIS, and this 3D imaging unit. The present invention relates to a camera and a 3D image generation method using .

In recent years, solid-state imaging devices that capture distance images (3D images) have been actively developed. Conventionally, CCD image sensors (CCD) have been the mainstream in 3D solid-state imaging devices, but CIS is now becoming mainstream.

Among various distance image acquisition methods, the time-of-flight (TOF) type is a method that has high distance accuracy, a wide measurable distance range, and relatively easy distance calculation. TOF type distance imaging elements mainly include continuous wave modulation (CW) type and optical pulse synchronization type. Both the CW type and the optical pulse type have a structure in which a plurality of charge distribution mechanisms are added to a photodetector such as a photoelectric conversion unit consisting of a photodiode (PD), and the on/off of the plurality of charge distribution mechanisms is It performs lock-in drive that repeatedly turns off.

TOF type distance imaging elements generally use infrared light as the signal light, and use a bandpass filter to remove as much light as possible other than the signal light to remove environmental light (background light). It is true. However, it is very difficult to remove all ambient light. In a three-branch lock-in pixel called a 3-tap lock-in pixel, one of the three charge distribution mechanisms, which distributes charge first, is used exclusively for removing ambient light. Due to its high removal ability, it is suitable for use in environments where the influence of ambient light is large, such as outdoors or brightly lit rooms.

In this way, the three-branch lock-in pixel makes it possible to realize a distance imaging element with high resistance to ambient light and high precision. However, the conventional three-branch type lock-in pixel has a large number of elements in the pixel and has a complicated internal structure, so it has been difficult to increase the number of pixels and increase the resolution, that is, to miniaturize the pixels.

International Publication No. 2007/026779 pamphlet

In view of the above problems, the present invention realizes a function equivalent to a pixel structure more complicated than a conventional three-branch lock-in pixel, or even a four-branch lock-in pixel, with a simple structure, and can achieve high resolution. The present invention aims to provide a highly accurate 3D imaging unit, a camera using this 3D imaging unit, and a 3D image generation method.

A first aspect of the present invention includes (a) a pixel array section in which a plurality of pixels that receive pulsed reflected light from an object and perform photoelectric conversion are arranged in a matrix; and (b) in a first frame. , the pixels are driven to generate a first analog signal at the timing of the first acquisition achievement period defined in synchronization with the pulse of the projection light that generates the reflected light, and the second analog signal is generated in a time domain different from the first frame. In the frame, a driving unit that drives a pixel to generate a second analog signal at the timing of a second acquisition period defined for a pulse of projection light with a length different from the first acquisition period; ) In each of the first and second frames, first and second analog signals from a plurality of pixels arranged in a specific row of the matrix are independently read out and converted into first and second digital data. a column processing circuit; (d) a distributor that discriminates and outputs first and second digital data; and (e) first and second digital data that independently store the discriminated first and second digital data. 2 memory, and (f) a distance calculation unit that inputs the first and second digital data read from the first and second memories and calculates the distance to the target object. This is the summary.

A second aspect of the present invention provides (a) a first and a second device that receives pulsed reflected light from a target object, performs photoelectric conversion on the received pulsed light, and transfers the photoelectrically converted signal charges along mutually different charge transfer paths. (b) A pixel array section in which a plurality of pixels are arranged in a matrix, each having a charge distribution mechanism around the photoelectric conversion section; drive the first charge distribution mechanism to generate the first analog signal at the timing of the first acquisition achievement period defined by the first acquisition achievement period; drive the second charge distribution mechanism to generate the second analog signal at the timing of the second acquisition achievement period, and in the second frame, drive the second charge distribution mechanism at the timing of the first acquisition achievement period without emitting pulses of projection light. a drive unit that drives the first charge distribution mechanism to generate a third analog signal, and drives the second charge distribution mechanism to generate a fourth analog signal at the timing of the second acquisition achievement period; (c) a column processing circuit that independently reads first to fourth analog signals from a plurality of pixels arranged in a specific row of the matrix and converts them into first to fourth digital data; (d) (e) a distributor that discriminates and outputs the first to fourth digital data; (e) first to fourth memories that independently store the discriminated first to fourth digital data; and (f) The gist of the present invention is that it is a 3D imaging unit that includes a distance calculation section that inputs data read out from the first to fourth memories and calculates the distance to an object.

A third aspect of the present invention includes (a) a light emitting unit that emits light pulses to a target object, (b) an imaging optical system that adjusts the optical path of pulsed reflected light from the target object, and (c) an imaging optical system. (d) A pixel array section in which a plurality of pixels that receive reflected light adjusted by an optical system and photoelectrically convert it are arranged in a matrix; drive the pixels to generate the first analog signal at the timing of the first acquisition achievement period defined by (f) a drive section that drives the pixels to generate the second analog signal at the timing of the second acquisition achievement period defined for the pulse of the projection light; (g) a column processing circuit that independently reads first and second analog signals from a plurality of pixels arranged in a specific row and converts them into first and second digital data; (h) first and second memories that independently store the discriminated first and second digital data; (i) first and second memories; The gist is that the camera is equipped with a distance calculation unit that inputs first and second digital data read from the camera and calculates the distance to the target object.

A fourth aspect of the present invention includes (a) a light emitting unit that emits light pulses to a target object, (b) an imaging optical system that adjusts the optical path of pulsed reflected light from the target object, and (c) an imaging optical system. First and second charge distribution mechanisms are provided around the photoelectric conversion section, respectively, for receiving the reflected light adjusted by the optical system, photoelectrically converting it, and transferring the photoelectrically converted signal charges along different charge transfer paths. (d) in the first frame, at the timing of the first acquisition achievement period defined in synchronization with the pulse of the projection light that generates the reflected light; A second charge distribution mechanism is driven to generate a first analog signal, and a second charge distribution mechanism is driven to generate a first analog signal, and a second charge distribution mechanism is generated at a timing of a second acquisition achievement period defined for a pulse of projection light with a length different from the first acquisition achievement period. The dividing mechanism is driven to generate a second analog signal, and in the second frame, without emitting pulses of projection light, the first charge distribution mechanism is driven at the timing of the first acquisition achievement period to generate a third analog signal. (e) a drive section that generates an analog signal of , and drives a second charge distribution mechanism to generate a fourth analog signal at the timing of the second acquisition achievement period; (f) a column processing circuit that independently reads the first to fourth analog signals from the pixels and converts them into first to fourth digital data, and (f) distinguishes the first to fourth digital data from each other and outputs them. (g) first to fourth memories that independently store the discriminated first to fourth digital data, and (h) data read from the first to fourth memories. The gist is that the camera is equipped with a distance calculation unit that inputs input data and calculates the distance to the target object.

A fifth aspect of the present invention includes (a) using a pixel array section in which a plurality of pixels are arranged in a matrix to receive and photoelectrically convert pulsed reflected light from an object; (c) generating a first analog signal by driving a pixel at the timing of a first acquisition achievement period defined in synchronization with a pulse of projection light that generates reflected light in one frame; (c) a first frame; In a second frame in a time domain different from the first acquisition period, the pixel is driven at the timing of a second acquisition period defined for the pulse of the projection light with a length different from the first acquisition period to generate a second analog signal. (d) in each of the first and second frames, independently reading first and second analog signals from a plurality of pixels arranged in a specific row of the matrix; (e) discriminating the first and second digital data from each other and outputting the same; (f) converting the discriminated first and second digital data into the first digital data independently; and (g) inputting the first and second digital data read from the first and second memories and calculating the distance to the target object. The gist is that it is a generation method.

A sixth aspect of the present invention provides (a) first and second charge distribution mechanisms that transfer signal charges along different charge transfer paths, respectively, in the periphery of a photoelectric conversion unit that generates signal charges. (b) using a pixel array section in which a plurality of pixels are arranged in a matrix to receive and photoelectrically convert pulsed reflected light from an object; and (b) converting projection light to generate reflected light in the first frame. A first charge distribution mechanism is driven to generate a first analog signal at the timing of a first acquisition achievement period defined in synchronization with the pulse, and a pulse of projection light is generated with a length different from the first acquisition achievement period. (c) a state in which pulses of projection light are not emitted in the second frame; Then, the first charge distribution mechanism is driven at the timing of the first acquisition achievement period to generate the third analog signal, and the second charge distribution mechanism is driven at the timing of the second acquisition achievement period to generate the fourth analog signal. (d) independently reading first to fourth analog signals from a plurality of pixels arranged in a specific row of the matrix and converting them into first to fourth digital data; (e) discriminating the first to fourth digital data from each other and outputting the same; and (f) storing the discriminated first to fourth digital data independently in the first to fourth memories. and (g) inputting the data read from the first to fourth memories and calculating the distance to the target object.

According to the present invention, a high-precision 3D imaging unit that achieves functions equivalent to a conventional three-branch lock-in pixel or other complex pixel structure with a simple structure, and is capable of achieving high resolution. The camera used and the 3D image generation method can be provided.

FIG. 2 is a logical block diagram schematically explaining an example of a main part of the 3D imaging unit according to the first embodiment of the present invention. FIG. 2 is a schematic block diagram illustrating the internal structure of a light receiving section and a storage section included in the elements of the 3D imaging unit according to the first embodiment. FIG. 2 is a logical block diagram illustrating the internal structure of a control unit included in the peripheral circuit of the solid-state imaging device, which is an element of the 3D imaging unit according to the first embodiment, as a hardware resource. FIG. 2 is a plan view schematically illustrating the structure of a one-branch pixel of a solid-state imaging device that is an element of a 3D imaging unit according to a first embodiment. FIG. 3 is a circuit diagram showing an example of a one-branch type pixel of a solid-state imaging device that is an element of the 3D imaging unit according to the first embodiment. FIG. 3 is a timing diagram illustrating a 3D image generation method according to the first embodiment. 2 is a graph showing changes in photoelectric conversion ratio with respect to wavelength in silicon (Si) epitaxial growth layers of different thicknesses. FIG. 3 is a schematic diagram showing the aspect ratio of a pixel when Si epitaxial growth layers of different thicknesses are used. FIG. 2 is a schematic diagram showing an example of a cross-sectional view of the 3D imaging unit according to the first embodiment. FIG. 7 is another schematic diagram showing an example of another cross-sectional view of the 3D imaging unit according to the first embodiment. FIG. 7 is yet another schematic diagram showing yet another example of a cross-sectional view of the 3D imaging unit according to the first embodiment. FIG. 7 is a timing diagram illustrating a 3D image generation method according to a second embodiment of the present invention. It is a schematic diagram explaining the 3D image generation method based on the 1st modification of 2nd Embodiment. It is a schematic diagram explaining the 3D image generation method based on the 2nd modification of 2nd Embodiment. It is a schematic diagram explaining the 3D image generation method based on the 3rd modification of 2nd Embodiment. FIG. 7 is a plan view schematically illustrating the structure of a two-branch pixel of a solid-state imaging device that is an element of a 3D imaging unit according to a third embodiment of the present invention. FIG. 7 is a circuit diagram showing an example of a pixel of a solid-state imaging device that is an element of a 3D imaging unit according to a third embodiment. FIG. 7 is a timing diagram illustrating a 3D image generation method according to a third embodiment. FIG. 3 is a plan view for comparing the areas of pixels of solid-state imaging devices that constitute a 3D imaging unit according to the first and third embodiments of the present invention and a conventional technique. It is a schematic diagram explaining the 3D image generation method based on the 1st modification of 3rd Embodiment. It is a schematic diagram explaining the 3D image generation method based on the 2nd modification of 3rd Embodiment. It is a schematic diagram explaining the 3D image generation method based on the 3rd modification of 3rd Embodiment. 7 is a flowchart illustrating an outline of an adjustment operation performed by a peripheral circuit of a 3D imaging unit according to another embodiment. FIG. 7 is a timing diagram illustrating a 3D image generation method according to another embodiment. FIG. 7 is a timing diagram illustrating a 3D image generation method according to still another embodiment. 1 is a block diagram illustrating an outline of the structure of a camera, which is an example of a field of application of a 3D imaging unit according to first to third embodiments of the present invention and modifications thereof; FIG. FIG. 2 is a plan view schematically illustrating the structure of a pixel of a solid-state imaging device according to a comparative example. FIG. 7 is a timing diagram illustrating an operation when driving a solid-state imaging device according to a comparative example. 1 is a logical block diagram illustrating an outline of an example of a main part of a conventional solid-state imaging device for comparison. FIG. 2 is a schematic block diagram illustrating the internal structure of a light receiving section and a storage section included in a conventional solid-state imaging device. FIG. 2 is a plan view schematically illustrating the structure of a three-branch pixel of a conventional solid-state imaging device. FIG. 2 is a circuit diagram showing an example of a pixel of a conventional solid-state imaging device. FIG. 2 is a timing diagram illustrating an operation when driving a conventional solid-state imaging device. FIG. 2 is a timing diagram illustrating an outline of an operation when driving a conventional solid-state imaging device. FIG. 2 is a timing diagram illustrating an operation when driving a conventional solid-state imaging device.

Next, first to third embodiments of the present invention will be described with reference to the drawings. In the description of the drawings related to the first to third embodiments and their modifications, the same or similar parts are given the same or similar symbols. However, it should be noted that the drawings are schematic and that the relationship between the thickness and the planar dimension, the ratio of the thickness of each member, etc. may differ from the actual one. Therefore, the specific thickness and dimensions should be determined with reference to the following explanation. Furthermore, it goes without saying that the drawings include portions with different dimensional relationships and ratios.

Further, the first to third embodiments and their modifications illustrate devices and methods for embodying the technical idea of the present invention, and the technical idea of the present invention The following does not specify the structure or arrangement of circuit blocks, or the layout on a semiconductor chip. The technical idea of the present invention can be modified in various ways within the technical scope defined by the claims.

In the following description of the first to third embodiments and their modifications, the first conductivity type is p-type and the second conductivity type is n-type. It will be easily understood that even if the 2-conductivity type is changed to the p-type, the same effect can be obtained by reversing the electrical polarity. In this case, it goes without saying that the high level and low level of the pulse waveform may also need to be appropriately inverted according to the technical common sense of those skilled in the art. For example, for convenience of explanation, FIG. 2 below shows a 3D imaging unit based on a 3D imaging device in which a plurality of pixels (ranging elements) are arranged in a two-dimensional matrix in a pixel array section, but this is merely an example. It's nothing more than that. A line sensor layout in which distance measuring elements are one-dimensionally arranged as pixels in a pixel array section may also be used. Alternatively, the distance sensor may have a simple structure in which only a single distance measuring element is arranged in the pixel array section.

(First embodiment)
As shown in FIG. 1, the 3D imaging unit according to the first embodiment of the present invention has a two-dimensional image in which a light receiving section 18 and peripheral circuit sections such as a driving section 15 arranged around the light receiving section 18 are integrated. The element is a sensor (solid-state imaging device). As shown in FIG. 2, the light receiving section 18 constituting the solid-state imaging device has a large number of pixels X _ij (i=1 to n, j=1 to m; m and n are each 2 or more) in a two-dimensional matrix. ) are arranged, forming a rectangular pixel array. The driving unit 15 is located on the upper side of the light receiving unit 18 forming the pixel array, and the horizontal scanning circuit 22 (not shown in FIG. 1) is located on the lower _side of the pixel rows X _{11 to} X _1m ; ;...; Provided along the X _n1 to X _nm direction. On the left side of the light receiving section 18, there are pixel columns X ₁₁ to X _n1 ; _{X 12} to X _n2 ;...; _{X 1j to} X _nj ;...; A vertical scanning circuit 21, which is omitted, is provided. The driving unit 15 is connected to a light emitting unit 19 that repeatedly projects light necessary for each pixel X _ij to measure distance as a distance measuring element as a pulse signal.

In FIG. 2, noise processing circuits NC ₁ to NC _m for processing signals from the light receiving section 18 are provided between the lower side of the light receiving section 18 forming the pixel array and the horizontal scanning circuit 22. The noise processing circuit NC ₁ includes a correlated double sampling circuit (CDS) 23 and an analog-to-digital conversion circuit (ADC) 24. The output signal from the light receiving section 18 is subjected to noise suppression by a correlated double sampling circuit 23 and then converted into digital data by an analog-digital conversion circuit 24. Although not shown, the noise processing circuits NC ₂ to NC _m are also composed of a correlated double sampling circuit (CDS) 23 and an analog-to-digital conversion circuit (ADC) 24, similar to the noise processing circuit NC ₁ . There is.

A control signal for controlling the drive unit 15 is transmitted from the control unit 14 to the drive unit 15 . The control unit 14 includes a program storage device 17 that stores a program that instructs a series of operations in the control unit 14, and a data storage device 13 that stores data, threshold values, etc. necessary for logical operations in the control unit 14. Connected. The control section 14 is further connected to an output section 16 that outputs the results of logical operations in the control section 14 . A distance calculation section 12 is connected to the data storage device 13, which inputs the output signal from the light receiving section 18 via the storage section 11 and performs calculation processing necessary for forming a distance image. The distance calculating section 12 calculates an estimated distance L between the object 10 and the light receiving section 18 shown in FIG.

The driving section 15, the horizontal scanning circuit 22, and the vertical scanning circuit 21 sequentially scan the pixels X _ij in the light receiving section 18, and read out pixel signals and perform an electronic shutter operation. That is, in the 3D imaging unit according to the first embodiment _{of the} present invention, the light receiving unit 18 is scanned in the vertical direction in units of each pixel _row X ₁₁ to X _{1m ;} By _doing so, _the pixel signals _of each pixel row X _{11 ~} X _1m ;X ₂₁ ~ _{X 2m ;} ...; The pixel signal is read out by vertical output signal lines provided every _1j to _Xnj ;...; _X1m to _Xnm .

The main parts are shown in FIG. 2, including a light receiving section 18 forming a pixel array, a driving section 15 located around the light receiving section 18 (see FIG. 1), a horizontal scanning circuit 22, a vertical scanning circuit 21, and a noise processing circuit. NC ₁ to NC _m are monolithically integrated on the same semiconductor chip, and the distance calculation section 12, control section 14, program storage device 17, data storage device 13, storage section 11, output section 16, etc. are integrated on another semiconductor chip. Even in the structure, the light receiving section 18, the driving section 15, the horizontal scanning circuit 22, the vertical scanning circuit 21, the noise processing circuits NC ₁ to NC _m , the distance calculation section 12, the control section 14, the program storage device 17, the data storage device 13, A structure in which the storage section 11 and the output section 16 are integrated on the same semiconductor chip may be used.

Specifically, the light receiving section 18, the driving section 15, the horizontal scanning circuit 22, the vertical scanning circuit 21, and the noise processing circuits _NC1 to _NCm forming a pixel array are integrated on a first semiconductor chip, and the distance calculating section 12, The control section 14, program storage device 17, data storage device 13, storage section 11, output section 16, etc. are integrated into a second semiconductor chip, and the first semiconductor chip and the second semiconductor chip are stacked and integrated. A similar structure is acceptable. Alternatively, the light receiving section 18 is configured on the first semiconductor chip, the driving section 15, the horizontal scanning circuit 22, the vertical scanning circuit 21, and the noise processing circuits NC ₁ to NC _m are integrated on the second semiconductor chip, and the distance calculating section 12 , a structure in which the control section 14, program storage device 17, data storage device 13, storage section 11, output section 16, etc. are integrated in a third semiconductor chip, and the first to third semiconductor chips are stacked and integrated. But it doesn't matter.

In any event, these integrated structures are merely exemplary and are not limited to the topology or layout shown in FIG. At least some of the circuits of the distance calculating section 12, the control section 14, the program storage device 17, the data storage device 13, the storage section 11, and the output section 16 located within the area surrounded by the broken line in FIG. It may also be mounted on a board or the like. For example, a configuration may be adopted in which the central processing control section 361 in the camera configuration shown in FIG. 26 is made to share the functions of the control section 14 in FIG.

_Signal reading _from each pixel X ₁₁ to X _1m ; X ₂₁ to X _2m ;...; However, the first _drive signal G1 for transferring signal charge from each photoelectric _conversion section of each pixel X ₁₁ to X _1m ; X ₂₁ to X _2m ;...; Since _the signals are _{simultaneously} applied to the pixels X ₁₁ to X _1m ; X ₂₁ to X _2m ;...; Therefore, signal readout from the pixel portion is performed with a readout period provided after the processing by the noise processing circuits NC ₁ to NC _m is completed. The storage unit 11 includes a distributor (demultiplexer) 113, a first memory 111, and a second memory 112. Digital data output from one signal line is discriminated by a distributor (demultiplexer) and stored in the first memory 111 or the second memory 112. The data stored in the first memory 111 and the second memory 112 are transferred to the distance calculating section 12, and the estimated distance L is calculated.

The control unit 14 includes a time setting logic circuit 141, a time setting value output control circuit 142, a distance image output control circuit 143, a setting value determination circuit 144, and a sequence control circuit, as shown in a block diagram of a logical configuration in FIG. 145 as the hardware qualification. The time setting logic circuit 141 has a light projection time T ₀ , a first acquisition achievement period τ _AA1 , a second acquisition achievement period τ _AA2 , a first charge accumulation time T _a1 , and a second charge accumulation time T _a2 shown in FIG. 6 which will be described later. , or in accordance with the output signal of the set value determination circuit 144, the time setting logic circuit 141 sets the values of the light projection time T ₀ and the first acquisition achievement period τ _AA1 shown in FIG. This is a logic circuit that changes as appropriate.

The time setting value output control circuit 142 controls the repetition cycle time T _C , the light projection time T ₀ , the first acquisition achievement period τ _AA1 , the second acquisition achievement period τ _AA2 , and the first charge set or changed by the time setting logic circuit 141 . This is a logic circuit that outputs an accumulation time T _a1 , a second charge accumulation time T _a2 , a charge transfer time T _{on ,} etc. to the driving section 15 as a control signal. The charge transfer time T _on defined as the pulse width of the first drive signal G1 is set at different timings with an offset time in between, as shown in FIG.

The distance image output control circuit 143 is a logic circuit that synthesizes distance image data and outputs it to the output unit 16.

The sequence control circuit 145 relies on clock signals to operate each of the time setting logic circuit 141, time setting value output control circuit 142, distance image output control circuit 143, setting value determination circuit 144, program storage device 17, and data storage device 13. This is a logic circuit that performs sequential sequence control. Each of the time setting logic circuit 141, time setting value output control circuit 142, distance image output control circuit 143, setting value determination circuit 144, and sequence control circuit 145 can transmit and receive information via the bus 146.

In the computer system illustrated in FIG. 1, the data storage device 13 can be any combination appropriately selected from a group including a plurality of registers, a plurality of cache memories, a main storage device, and an auxiliary storage device. be. Further, the cache memory may be a combination of a primary cache memory and a secondary cache memory, or may have a hierarchy including a tertiary cache memory. Although not shown, if the data storage device 13 includes a plurality of registers, the bus 146 may be extended to the program storage device 17, the data storage device 13, etc.

The control unit 14 shown in FIG. 1 can configure a computer system using a microprocessor (MPU) or the like implemented as a microchip. In addition, the control unit 14 that makes up the computer system includes a digital signal processor (DSP) with enhanced arithmetic functions and specialized for signal processing, and a microcontroller (microcontroller) equipped with memory and peripheral circuits for the purpose of controlling embedded devices. ) etc. may be used. Alternatively, the main CPU of a current general-purpose computer may be used as the control unit 14.

Furthermore, part or all of the configuration of the control unit 14 may be configured with a programmable logic device (PLD) such as a field programmable gate array (FPGA). When part or all of the control unit 14 is configured by a PLD, the data storage device 13 can be configured as a memory element such as a memory block included in a part of the logical blocks that configure the PLD. Furthermore, the control unit 14 may have a structure in which a CPU core-like array and a PLD-like programmable core are mounted on the same chip. This CPU core-like array includes a hard macro CPU pre-installed inside the PLD and a soft macro CPU configured using logic blocks of the PLD. In other words, a configuration in which software processing and hardware processing are mixed within the PLD may be used.

_Signal generation in each _of the pixels X ₁₁ to X _1m ; X ₂₁ to X _2m ;...; An example of a specific planar structure of the portion 181 is shown in FIG. Further, an equivalent circuit diagram corresponding to the planar structure shown in FIG. 4 is shown in FIG. An n ⁺ type charge storage region 43a is arranged as a floating drain region FD1 on the right side of the lower end of the photoelectric conversion section 41 made of a photodiode (PD). Further, an n ⁺ type charge discharge region 43b is arranged as a drain region for discharge at the center of the left end of the photoelectric conversion section 41 and spaced apart from the charge accumulation region 43a so as to surround the region of the photoelectric conversion section 41.

The transfer gate electrode 42a of the transfer transistor G1 constituting one charge distribution mechanism surrounds the photoelectric conversion section 41 whose region is defined by blocking light except for the opening with a light-shielding film (not shown); A discharge gate electrode 42b of a discharge transistor GD constituting one charge discharge mechanism is arranged. A conductive thin film such as polycrystalline silicon is used for the transfer gate electrode 42a and the discharge gate electrode 42b so as to form an insulated gate transistor structure, respectively.

A charge storage region 43a is provided as a drain region of a transfer transistor G1 having an insulated gate transistor (MOS transistor) structure, and a charge discharge region 43b is provided as a drain region of a discharge transistor GD. As shown in FIG. 5, an auxiliary capacitor C1 is connected to the floating drain region FD1. Floating drain region FD1 in FIG. 5 corresponds to charge storage region 43a in FIG. 4. For example, a depletion type MOS structure can be adopted as the auxiliary capacitor C1. Corresponding to the equivalent circuit shown in FIG. 5, above the photoelectric conversion section 41 in the planar pattern of FIG. The structure is illustrated. The auxiliary capacitance C1 of the adjacent pixel shown in the upper part of FIG. 4 is provided on the n ⁺ type diffusion region 49 and the n ⁻ type diffusion region 49 connected to the n ⁺ type along the vertical direction of the substrate. The gate insulating film (not shown) and the gate electrode 47 provided on the gate insulating film constitute the gate structure of the MOSFET, thereby realizing an n ^- channel depletion type MOS structure. For the gate electrode 47, a conductive thin film such as polycrystalline silicon can be used.

Similarly, the auxiliary capacitor C1, which is circuit-connected to the photoelectric conversion unit (PD) 41 in the center of FIG. A depletion type MOS structure is formed by an n ⁺ type diffusion region, a gate insulating film provided on an n ⁻ type diffusion region connected to the n ⁺ type, and a gate electrode 47 provided on the gate insulating film. Can be configured. The white rectangle shown inside the corner of the charge storage region 43a located above the plane pattern in FIG. 4 schematically shows the contact hole of the charge storage region 43a of another photoelectric conversion section adjacent to the upper side. . The white rectangle shown on the right side of the upper part of the gate electrode 47 located above the plane pattern in FIG. 4 schematically shows the contact hole of the gate electrode 47 of another photoelectric conversion section adjacent to the upper side. . In FIG. 4, the vertical thick solid line connecting the upper and lower two white squares is schematically shown, but the charge storage region 43a and the gate electrode 47 of the auxiliary capacitor C1 are connected via surface wiring etc. be done.

Here, the auxiliary capacitor C1 may be added as a MOS capacitive element as shown in FIG. good. However, since the 3D imaging unit according to the first embodiment has the purpose of miniaturizing pixels, we will exemplify a condition in which MOS capacitance is intentionally added and the layout becomes more difficult, even if the pixels are miniaturized. Explain that the element capacitance does not decrease. That is, according to the 3D imaging unit according to the first embodiment, even when the planar pattern of the pixel structure is miniaturized, it is possible to arrange the auxiliary capacitor C1 of a desired planar pattern size within the pixel. In the planar pattern of FIG. 4, an open rectangle shown inside the diffusion region 49, which is a portion protruding above the gate electrode 47 from the right side of the upper side of the gate electrode 47, indicates a contact hole of the diffusion region 49. The diffusion region 49 is connected to the ground potential (GND) through a surface wiring extending above the contact hole of the diffusion region 49, as shown schematically by solid lines bent at right angles at three places.

Focusing on the photoelectric conversion section 41, a reset transistor RT having a reset gate electrode 44 is configured using a charge storage region 43a located at the lower right of the photoelectric conversion section 41 as a source region. In the upper part of FIG. 4, a charge storage region 43a of another photoelectric conversion section adjacent to the upper side of the photoelectric conversion section 41 is shown, and the charge storage region 43a is used as a source region and the other photoelectric conversion section has a reset gate electrode 44. A reset transistor RT is configured. Each of the reset gate electrodes 44 can be made of a conductive thin film such as polycrystalline silicon. Reset transistor RT has a reset drain (RD) region 50 that faces charge storage region 43a via reset gate electrode 44.

The reset drain region 50 forms a common region with the amplification drain region of the amplification transistor SF1. The amplification transistor SF1 includes an amplification drain region forming a common region with the reset drain region 50, an amplification source region 51 opposite to the amplification drain region, and a gate insulating region over a channel region between the amplification drain region and the amplification source region 51. It is composed of an amplification gate electrode 47 provided through a film. As can be seen from the upper pattern in FIG. 4, the amplification gate electrode 47 is composed of the same conductive thin film as the gate electrode 47 of the auxiliary capacitor C1, and therefore is given the same reference numeral in FIG. Since the gate electrode 47 of the auxiliary capacitor C1 is electrically connected to the charge storage region 43a, the amplification gate electrode 47 of the amplification transistor SF1 is also electrically connected to the charge storage region 43a. Focusing on the photoelectric conversion section 41 shown in the center of FIG. It is connected.

A single white rectangle shown at the center of each of the two patterns of the reset drain region 50 shown at the top and bottom of FIG. 4 schematically represents a contact hole. As schematically shown by the thick solid line, the reset drain region 50 is connected to the power supply potential (V _DD ) via the surface wiring extending above the contact hole of the reset drain region 50. The charge discharge region 43b is connected to a power supply potential (V _DD ) via a surface wiring or the like running over a contact hole schematically shown as a single white rectangle. The amplification source region 51 of the amplification transistor SF1 is configured as a common region with the selection drain electrode of the selection transistor SL1 for pixel selection. The selection transistor SL1 includes a selection drain region forming a common region with the amplification source region 51, a selection source region 52 opposite to the selection drain region, and a gate insulating region over the channel region between the selection drain region and the selection source region 52. It is composed of a selection gate electrode 46 provided through a film.

In the planar pattern of FIG. 4, a contact hole is schematically shown as a single white rectangle inside the selection source region 52 of the selection transistor SL1. The selection source region 52 of the selection transistor SL1 is connected to the first vertical output signal line Sig1 via a contact hole. A horizontal line selection control signal S is applied to the selection gate electrode 46 of the selection transistor SL1 from the vertical scanning circuit 21 shown in FIG. By setting the selection control signal S to a high level, the selection transistor SL1 becomes conductive, and the first vertical output signal line Sig1 is brought to a potential corresponding to the potential of the charge storage region 43a amplified by the source follower type amplification transistor SF1. becomes.

Photoelectrons generated in the light receiving region of the photoelectric conversion section 41 move to the charge storage region 43a when the transfer gate electrode 42a is turned on. Pulsed light is emitted from approximately the same position as the image sensor shown in FIG. 1, and reflected light from the object 10 is received by the image sensor. As the projection light, for example, a light emitting diode (LED) or a semiconductor laser (LD) is used to project extremely short pulsed light of, for example, ns order to fs order. The received light enters the image sensor after a delay time Td from the time when the projection light is emitted, depending on the distance between the object 10 and the image sensor. Note that the area surrounded by the broken line in FIG. 4 is a boundary line that determines a pixel boundary.

FIG. 6 is a timing diagram illustrating the 3D image generation method according to the first embodiment. FIG. 6(a) shows that the operations of the first on-frame (ON1 frame) and the second on-frame (ON2 frame) are alternately repeated, and FIG. 6(b) is a timing diagram of the first on-frame. , FIG. 6(c) is a timing diagram of the second on-frame. As shown in FIG. 6(b), in the first on-frame, the projection light is emitted at the timing of the light projection time _T0 . The pulse of the first drive signal G1 in the first on-frame is also synchronized with the light projection time T ₀ to define the first acquisition achievement period τ _AA1 . As shown in FIG. 6A, the first acquisition achievement period τ _AA1 can be set to the same value as the first charge accumulation time _Ta1 (τ _AA1 = _Ta1 ). The projection light is periodically emitted with a repetition period time (cycle time) _TC .

By applying the first drive signal G1 with a pulse width T _on to the transfer gate electrode 42a at the timing of this first acquisition achievement period τ _AA1 , charge is accumulated in the charge storage region 43a in accordance with the pulse of the first drive signal G1. The number of photoelectrons decreases compared to the number of photoelectrons generated by the received light due to the delay in the received light. As shown in FIG. 6C, in the second on-frame as well, projection light is emitted at the timing of the light projection time _T0 , similarly to the first on-frame. The timing of the pulses of the first drive signal G1 in the second on-frame is determined to define a second acquisition achievement period τ _AA2 synchronized with the optical projection time T ₀ , and the first drive signal G1 with a pulse width T _on is It is applied to the transfer gate electrode 42a. The second acquisition achievement period τ _AA2 is a time measured from the rise of the projection light, and is a time different from the second charge accumulation time T _a2 . As shown in FIG. 6(c), the second charge accumulation time T _a2 is defined by being measured from the fall of the pulse applied to the discharge gate electrode 42b of the discharge transistor GD. The photoelectrons acquired during the second acquisition achievement period τ _AA2 are accumulated in the charge accumulation region 43a. By applying the first drive signal G1 to the transfer gate electrode 42a at the timing shown in _FIG . The number of photoelectrons generated by the received light increases by the same amount as the amount of photoelectrons generated by the received light that decreased during the first on-frame due to the delay in the received light. That is, since the amount of accumulated charge corresponding to each frame differs depending on the distance to the subject, the distance to the subject can be determined from the amount of accumulated charge in each frame.

If the amount of charge acquired in the first acquisition achievement period τ _AA1 in the first on-frame is Q1 ₁ and the amount of charge acquired in the second acquisition achievement period τ _AA2 in the second on-frame is Q1 ₂ , the subject distance is

L=cT ₀ /2×Q1 ₂ /(Q1 ₁ +Q1 ₂ )...(1)

is required. Here, c is the speed of light.

The conventional three-branch lock-in pixel shown in FIG. 31 has three distribution gates, but the 3D imaging unit according to the first embodiment uses only one transfer transistor G1 as a distribution gate. have. Therefore, in order to repeat the cycle operation of the first on-frame and the second on-frame, a structure is required that quickly discharges the charges generated and collected in the photodiode outside of the storage time. In order to perform this discharge operation, the discharge gate electrode 42b of the discharge transistor GD whose drain is connected to the power source may be driven to turn on the discharge transistor GD.

As described above, the 3D imaging unit according to the first embodiment has a simple structure of a one-branch lock-in pixel that has one transfer transistor G1 and one discharge transistor GD for one photodetector PD. As a result, a highly accurate, fine pixel distance sensor can be realized. As already explained, in the 3D imaging unit according to the first embodiment, one photodetection section PD is connected to the floating drain region FD1 and the auxiliary capacitor C1 via one transfer transistor G1, and one A charge discharge region 43b functioning as a drain of a discharge transistor GD constituting one charge discharge mechanism is connected to a power source _VDD , and charges are discharged from the photodetector PD. Furthermore, since it has a simple structure of a one-branch type lock-in pixel, the pixel size can be further miniaturized, and the resolution of the captured image is improved.

The control section 14 of the 3D imaging unit according to the first embodiment shown in FIG. 1 operates as follows. The time setting logic circuit 141 of the control unit 14 determines the timing of applying the pulse signal of the first drive signal G1 based on the rising timing of the output pulse of the projection light, and sets the first acquisition achievement period τ _AA1 . . The time setting value output control circuit 142 outputs the set first acquisition achievement period τ _AA1 to the drive section 15 shown in FIG. 1 as a control signal. The light emitting section 19 emits pulsed light in response to a control signal applied from the time set value output control circuit 142 of the control section 14 through the driving section 15 . The pulsed light reflected from the object 10 is applied to each pixel X _ij of the pixel array section constituted by the light receiving section 18 shown in FIG. 1 through a lens 20, a BPF (band pass filter), and the like.

In the first on-frame, the drive section 15 drives each pixel X _ij of the pixel array section as a lock-in pixel at the drive timing shown in FIG. 6(b), and the signal charge of the photoelectric conversion section 41 of each pixel X _ij is is accumulated in the charge accumulation region 43a of the pixel X _ij . Thereafter, the vertical scanning circuit 21 selects a row of the pixel array section (X ₁₁ to X _1m ; X ₂₁ to X _2m ; . . . ; X _n1 to X _nm ) of the light receiving section 18 . The signal potential and reset potential of the pixels (X _i1 to X _im : i=1 to n) in the row selected in the first on-frame pass through one signal line, and noise is suppressed by the correlated double sampling circuit 23. Thereafter, it is converted into digital data by an analog-to-digital conversion circuit 24. The correlated double sampling circuit 23 and the analog-to-digital conversion circuit 24 independently read signals based on the signal charges accumulated in the plurality of charge accumulation regions 43a arranged in a specific row of the matrix, and perform noise suppression and analog-to-digital conversion. A column processing circuit (23, 24) for digital conversion is configured. The first on-frame data converted into digital data by the column processing circuits (23, 24) is column-selected by the horizontal scanning circuit 22 and transferred to the storage unit 11. In the storage unit 11, the transferred first on-frame digital data is discriminated and stored in one of the plurality of memories (111, 112) by a distributor (demultiplexer) 113 synchronized with the drive unit 15. . For example, signal data obtained in the first on-frame is stored in the first memory 111, and signal data obtained in the second on-frame is stored in the second memory 112.

When a predetermined repetition period time (cycle time) T _C elapses and the first on-frame operation shown in FIG. 6(a) ends, the second on-frame operation shown in FIG. 6(b) starts. is started. The time setting logic circuit 141 of the control unit 14 sets the second acquisition achievement period τ _AA2 so that the timing of the received light arriving later than the timing of the projection light can be adjusted. That is, the time setting logic circuit 141 sets the second acquisition achievement period τ _AA2 which is longer than the first acquisition achievement period τ AA1 in synchronization with the emission timing of _the projection light. The time setting value output control circuit 142 outputs the set second acquisition achievement period τ _AA2 to the drive section 15 as a control signal. The light emitting section 19 emits pulsed light in response to a control signal applied from the time set value output control circuit 142 of the control section 14 through the driving section 15 . The pulsed light reflected from the object 10 is irradiated onto each pixel X _ij of the pixel array section constituted by the light receiving section 18 .

In the second on-frame, lock-in pixel driving is performed at the drive timing shown in FIG. 6(c), and charges are accumulated in the charge accumulation region 43a of each pixel X _ij . Thereafter, _the vertical scanning circuit 21 selects the row _of the pixel array section (X ₁₁ to X _1m ; X ₂₁ to X _2m ; ...; is transferred to the storage unit 11. In the storage unit 11, the transferred second on-frame digital data is transferred by a distributor (demultiplexer) 113 to the first memory 111 and the second memory 112, whichever memory was not used during the first on-frame. is memorized.

The data stored in the first memory 111 and the second memory of the storage section 11 is transferred to the distance calculation section 12, and the distance is calculated using equation (1). The data may be transferred from the storage unit 11 to the distance calculation unit 12 when all of the data of the first on-frame and the second on-frame are stored in the memory of the storage unit 11, or when the data of the second on-frame is transferred. A method may also be used in which the data is sequentially performed at the time when the data starts to be stored in the memory of the storage unit 11. Further, the order of the first on-frame and the second on-frame may be either the first on-frame or the second on-frame.

Here, let us compare FIG. 4 with FIG. 31, which shows a conventional three-branch lock-in pixel structure. FIG. 32 is a circuit diagram of a conventional three-branch lock-in pixel corresponding to FIG. 5. A pixel of the solid-state imaging device that constitutes the 3D imaging unit according to the first embodiment has a transfer transistor G1 that constitutes one charge distribution mechanism and a discharge transistor GD that constitutes one charge discharge mechanism. , a pixel of a conventional solid-state imaging device has three charge distribution mechanisms, a first transfer transistor G1, a second transfer transistor G2, and a third transfer transistor G3, and a discharge transistor GD, which constitutes one charge discharge mechanism. ing. In FIG. 31, a plane pattern is shown in which each of the first transfer transistor G1, second transfer transistor G2, and third transfer transistor G3 has a first transfer gate electrode 92a, a second transfer gate electrode 92b, and a third transfer gate electrode 92c. is exemplified. Further, a planar pattern is shown in which the discharge transistor GD has a discharge gate electrode 92d.

4 and 5 show a simple structure in which a charge storage region 43a, an auxiliary capacitor C1, and an amplification transistor SF1 are connected to the transfer transistor G1 of the 3D imaging unit according to the first embodiment via wiring or the like. In FIGS. 31 and 32, the first transfer transistor G1, the second transfer transistor G2, and the third transfer transistor G3, which are three charge distribution mechanisms, have the same structure as the transfer transistor G1 shown in FIGS. 4 and 5, respectively. , first charge storage region 93a, second charge storage region 93b, third charge storage region 93c, first auxiliary capacitor C1, second auxiliary capacitor C2, third auxiliary capacitor C3, amplification transistor SF1, second amplification transistor SF2, The third amplification transistor SF3 has a complicated structure connected via wiring and the like.

The pixel plan view shown in FIG. 4 and the pixel plan view shown in FIG. 31 are configured with the same layout rule, and the area of the pixel plan view shown in FIG. 4 is four times the area of the pixel plan view shown in FIG. 31. The size is 1. Note that the area surrounded by the broken line in FIG. 31 is a boundary line that determines a pixel boundary.

Comparing the internal structures of the conventional solid-state imaging device shown in FIG. 1 and FIG. 29, it is found that the storage section 11 shown in FIG. 1 is omitted in the conventional solid-state imaging device. Furthermore, _when comparing FIG. 2 and _FIG . 30, in FIG. 2, _the vertical _scanning circuit 21 scans the pixel array section (X ₁₁ to X _1m ; The signal potential and reset potential of the selected row pass through one signal line, are converted into digital data, and then stored in the memory of the storage unit 11. On the other hand, in FIG. 30, the signal of the selected row of the pixel array section (X ₁₁ to X _1m ; X ₂₁ to X _2m ; ...; X _n1 to X _nm ) of the light receiving section 71 is The potential and the reset potential pass through either of the two signal lines, are converted into digital data, and then transferred to the distance calculation unit 72.

Photoelectrons generated in the light receiving region of the photoelectric conversion section 91 of the conventional solid-state imaging device move to the first charge accumulation region 93a when the first charge distribution mechanism 92a is turned on. Similarly, when the second charge distribution mechanism 92b of the conventional solid-state imaging device is brought into a conductive state, it moves to the second charge accumulation region 93b, and when the third charge distribution mechanism 92c is rendered conductive, it moves to the third charge accumulation region 93c. Move to. In the conventional solid-state imaging device, pulsed light is emitted from almost the same position as the image sensor shown in FIG. ) or a semiconductor laser (LD) to project extremely short pulsed light of, for example, ns order to fs order, as in the case of the 3D imaging unit according to the first embodiment.

FIG. 33 is a timing diagram illustrating an operation when driving a conventional solid-state imaging device. The received light enters the image sensor after a delay time T _d from the time when the projection light is emitted, depending on the distance between the object 10 and the image sensor. If the emission time _T0 of the projection light is synchronized with the on/off voltage pulses of the first drive signal G1, the second drive signal G2, and the third drive signal G3, the first The received light is transferred to the charge accumulation region 93a, to the second charge accumulation region 93b in response to the pulse of the second drive signal G2, and to the third charge accumulation region 93c in response to the pulse of the third drive signal third transfer transistor G3. Depending on the delay, that is, the distance to the subject, a difference occurs in the amount of charge accumulation corresponding to each gate, and the distance to the subject is determined.

Furthermore, it is desirable that the projection light used in the conventional solid-state imaging device be synchronized with the second charge accumulation time G2, as shown in FIG. By synchronizing with the charge accumulation time G2, only electrons unnecessary for distance measurement, such as environmental light other than received light and dark current, are accumulated in the first charge accumulation region 93a. Here, electrons unnecessary for distance measurement, such as environmental light and dark current, are similarly accumulated in the second charge accumulation region 93b and third charge accumulation region 93c, but are unnecessary in the first charge accumulation region 93a. By accumulating only the electrons, it becomes a simple offset component, and according to the following equation (2), it becomes possible to accurately measure the distance to the object excluding the influence of environmental light and the like.

In a conventional solid-state imaging device, the amount of charge accumulated in the first charge accumulation region 93a is Q2 ₁ , the amount of charge accumulated in the second charge accumulation region 93b is Q2 ₂ , and the amount of charge accumulated in the third charge accumulation region 93c If Q2 _{is 3} , then

L=cT ₀ /2×Q3'/(Q2'+Q3')...(2)

is required. Here, Q2'=Q2 ₂ -Q2 ₁ and Q3'=Q2 ₃ -Q2 ₁ .

Here, when distance measurement is performed to a somewhat distant subject, photoelectrons that should be accumulated in the third charge accumulation region 93c are accumulated in the first charge accumulation region 93a in the next cycle. Since the charge accumulated in the first charge accumulation region 93a is an offset component due to environmental light or the like, when a received light component is added to the first charge accumulation region 93a, the distance accuracy is greatly degraded. Therefore, in the conventional solid-state imaging device as well, if the discharge transistor GD whose drain is connected to the power supply is turned on after the accumulation time of the third charge accumulation region 93c, the delayed received light component is discharged and the next A delayed received light component does not enter the periodic first charge accumulation region 93a, and distance accuracy can be maintained favorably.

The light receiving section of the 3D imaging unit is composed of, for example, a pixel array of n rows and m columns (n×m). By sequentially reading out all the signals from this pixel array of n rows and m columns, one two-dimensional distance image and one two-dimensional image are created. If this one image is defined as one frame, a two-dimensional distance moving image and a two-dimensional moving image are created by consecutive frames. Here, as shown in FIG. 34, one frame time (T _f ) of the conventional solid-state imaging device is composed of a light irradiation period (T _L ) and a signal readout period (T _R ). In the conventional 3D image generation method, the drive timing is within one cycle (repetition period) within the light irradiation period (T _L ).

TOF lock-in pixels measure distance using the speed of light, so the charge accumulation time per time is extremely short. FIG. 35 shows an example of drive timing when measuring a maximum distance of about 5.1 m using a conventional solid-state imaging device. As shown in FIG. 35, the charge accumulation time per exposure is as short as 34 ns, and almost no photoelectrons are generated by light reception during one exposure. Therefore, in conventional solid-state imaging devices, the signal amount is increased by increasing the total accumulation time by repeating the same cycle a huge number of times, for example, from thousands to hundreds of thousands of times. Here is an example. Note that in the TOF lock-in pixel, the charge accumulation time is generally global shutter drive in which all pixels are driven simultaneously. Increasing the number of cycles increases the total exposure time and therefore increases the signal amount. However, there is a trade-off in that increasing the exposure time reduces video performance (ie, frame rate).

FIG. 35 is an example of a conventional solid-state imaging device at 30 FPS (frames per second) at which smooth moving images can be obtained. At 30 FPS, one frame time (T _f ) is approximately 33 ms. If the time required for reading (T _R ) is about 6 ms, the light irradiation time (T _L ) is about 27 ms. In the conventional solid-state imaging device, the cycle time (T _C ) is 300 ns, so the maximum number of cycles that can be repeated during the light irradiation time (T _L ) is 90,000 times.

FIG. 7 shows basic characteristics for determining the thickness of the silicon (Si) epitaxial growth layer necessary for the solid-state imaging device constituting the 3D imaging unit according to the first embodiment, and shows the wavelength of the photoelectric conversion rate. FIG. A TOF distance imaging device such as the 3D imaging unit according to the first embodiment generally uses infrared light as signal light, and typical infrared light wavelengths are 850 nm and 940 nm. As shown in FIG. 7, when the thickness of the Si epitaxial growth layer is 10 μm, 46% of the incident photons of infrared light with a wavelength of 850 nm are converted into photoelectrons, and when the thickness of the Si epitaxial growth layer is 20 μm, When the thickness of the Si epitaxial growth layer is 30 μm, 87% of the incident photons are converted into photoelectrons.

In addition, when the thickness of the Si epitaxial growth layer is 10 μm, 27% of the incident photons of infrared light with a wavelength of 940 nm are converted into photoelectrons, and when the thickness of the Si epitaxial growth layer is 20 μm, the incident photons are converted into photoelectrons. 47% of photons are converted to photoelectrons, and when the thickness of the Si epitaxial growth layer is 30 μm, 62% of incident photons are converted to photoelectrons. The higher the photoelectric conversion ratio, the more desirable it is, so for example, if we aim for around 50% or more, the thickness of the Si epitaxial growth layer will need to be 10 μm or more for infrared light with a wavelength of 850 nm, and for infrared light with a wavelength of 940 nm. In this case, the thickness of the Si epitaxial growth layer needs to be 20 μm or more.

FIG. 8 shows the aspect ratio defined by the thickness (numerator) of the Si epitaxial growth layer relative to the pixel dimension (denominator) when the thickness of the Si epitaxial growth layer is 10 μm, 20 μm, and 30 μm. That is, FIG. 8A shows the aspect ratio defined based on the pixel dimension corresponding to the pixel area of the solid-state imaging device that is an element of the 3D imaging unit according to the first embodiment. FIG. 8(b) shows the aspect ratio defined based on the pixel dimension corresponding to the pixel area in the case of a bifurcated lock-in pixel, which is an element of a 3D imaging unit according to a third embodiment, which will be described later. 8(c) shows the aspect ratio in the case of the pixel area of a conventional three-branch type lock-in pixel. As can be seen from FIG. 19, which will be described later, the pixel area in FIG. 8(c) is the largest, and the pixel area decreases in the order of (b) and (a). Even for fine pixels, the required thickness of the Si epitaxial growth layer is determined by the wavelength of the light, so it cannot be made thin. As a result, as shown in Figure 8, the smaller the pixel area, the larger the aspect ratio. Become.

FIG. 9 schematically shows a cross-sectional structure of a pixel of a solid-state imaging device that constitutes a 3D imaging unit according to the first embodiment. As shown in FIG. 8, the smaller the pixel area, the larger the aspect ratio. As a typical example, the thickness of the Si epitaxial growth layer is 20 μm, that is, the aspect ratio is 6.3. As the aspect ratio increases, crosstalk between pixels increases, and even if the pixels are made finer and the number of pixels is increased, the desired resolution cannot be obtained. FIG. 9(a) shows the case of the front-side illumination (FSI) type, and FIG. 9(b) shows the case of the back-side illumination (BSI) type. The 3D imaging unit according to the first embodiment of the front surface illumination (FSI) type shown in FIG. It is composed of a layer 212 and a p ⁺ substrate 211.

As a structure for preventing crosstalk when the aspect ratio becomes large, the element isolation region 213 in FIG. 9 exemplifies a deep trench isolation (DTI) structure using a deep trench. In order to reduce crosstalk, it is desirable that at least the photoelectrons generated by photoelectric conversion are collected in the photodiode of the self-pixel without crosstalk, and an element isolation structure using the DTI structure shown in FIG. 9 is effective. It is. Further, the 3D imaging unit according to the first embodiment of the backside illumination (BSI) type shown in FIG. , a p-type epitaxial growth layer 212, a p ⁺ -type substrate, or a p ⁺ -type surface pinning layer 222.

Here, if the thickness of the Si epitaxial growth layer is 20 μm, a trench with a depth of about 15 μm or more is desirable, and if the thickness of the Si epitaxial growth layer is 30 μm, a trench with a depth of about 25 μm or more is desirable. That is, it is desirable that the trench has a depth equal to or greater than the thickness of the Si epitaxial growth layer minus 5 μm. For example, it is desirable to have an element isolation region with a width of 0.6 μm and a depth of 15 μm, that is, an aspect ratio of about 25 or more, that is, a width narrower than 0.6 μm. The aspect ratio of about 25 is smaller than the element isolation region of a visible light BSI type fine pixel with an aspect ratio of about 40, is comparable to a DRAM using a trench capacitor, and can be manufactured using existing technology.

If the deep trench used for the element isolation region 213 is doped with a p-type impurity, for example by ion implantation, dark current generated at the interface can be suppressed. For deeper trenches, for example, after forming a thin thermal oxide film of about 5 nm, an oxide film may be buried, or non-doped polysilicon with good embedding properties may be buried, or polysilicon doped with impurities may be buried to form a trench DRAM. It is also possible to use a MOS capacitor as shown in FIG.

FIG. 10 shows a schematic diagram of another cross-sectional structure of the solid-state imaging device that constitutes the 3D imaging unit according to the first embodiment. FIG. 10(a) shows the case of the front-side illumination (FSI) type, and FIG. 10(b) shows the case of the back-side illumination (BSI) type. The 3D imaging unit according to the first embodiment of the front surface illumination (FSI) type shown in FIG. It is composed of an epitaxial growth layer 212 and a p ⁺ substrate 211. Further, the 3D imaging unit according to the first embodiment of the backside illumination (BSI) type shown in FIG. , a wiring layer 216, a p-type epitaxial growth layer 212, a p ⁺ -type substrate, or a p ⁺ -type surface pinning layer 222.

As shown in the explanation of the cross-sectional structure in FIG. 9, even if an element isolation structure is provided to suppress crosstalk of photoelectrons generated by photoelectric conversion, crosstalk due to high angle incident light cannot be suppressed. Crosstalk caused by incident light can be suppressed to some extent by microlenses, but it cannot be suppressed sufficiently with a structure with such a large aspect ratio. In order to suppress crosstalk of incident light, it is best to guide the incident light so that it irradiates as perpendicularly to the Si substrate as possible, and a collimator using a waveguide as shown in FIG. 10 is effective. For example, silicon nitride (Si _x N _y ) having a refractive index of about 2 may be used for the waveguide, and it can be manufactured using existing technology. Note that although a combination of an element isolation region and a waveguide is used in FIG. 10, a structure in which only a waveguide is formed may be used.

FIG. 11 shows a schematic diagram of still another cross-sectional structure of the solid-state imaging device that constitutes the 3D imaging unit according to the first embodiment. FIG. 11(a) shows the case of the front-side illumination (FSI) type, and FIG. 11(b) shows the case of the back-side illumination (BSI) type. The 3D imaging unit according to the first embodiment of the front surface illumination (FSI) type shown in FIG. , a p-type first epitaxial growth layer 231 , and a p ⁺ substrate 211 . Further, the 3D imaging unit according to the first embodiment of the backside illumination (BSI) type shown in FIG. , a wiring layer 216 , a p-type second epitaxial growth layer 232 , a p-type first epitaxial growth layer 231 , and a p ⁺ -type substrate or a p ⁺ -type surface pinning layer 222 .

As described using FIG. 9, even if an element isolation structure is provided to suppress crosstalk of photoelectrons generated by photoelectric conversion, crosstalk due to high angle incident light cannot be suppressed. As explained using FIG. 10, it is effective to suppress the crosstalk of the incident light by using a collimator structure using a waveguide in order to guide the incident light so as to irradiate the Si substrate as perpendicularly as possible. However, in the 3D imaging unit according to the first embodiment, since an electric field cannot be applied to a region far from the photodiode, the generated photoelectrons undergo a random walk due to diffusion until they reach the photodiode that can be used as a signal. This takes a long time, resulting in a decrease in sensitivity and distance accuracy. In order to suppress random walks due to diffusion, it is best to apply an electric field to the Si substrate, and a structure in which the substrate has a concentration difference as shown in FIG. 11 is effective.

For example, the p-type first epitaxial growth layer 231 is epitaxially grown with 1×10 ¹⁵ /cm ³ of Si, and the p-type second epitaxial growth layers 212 and 232 are made with a 1×10 By epitaxially growing Si with an impurity density of ¹³ / ^cm3 , it is possible to apply an electric field of over 0.1V, suppress random walk, shorten the time it takes to reach a photodiode that can be used as a signal, and improve sensitivity. It is possible to prevent a decrease in distance accuracy and a decrease in distance accuracy.

The laminated wafer of the first epitaxial growth layer 231 and the second epitaxial growth layer 212 shown in FIG. 11(a) and the laminated wafer of the first epitaxial growth layer 231 and the second epitaxial growth layer 232 shown in FIG. 11(b) are It can be manufactured using technology. Furthermore, random walk may be suppressed by applying a voltage to the p ⁺ substrate, that is, by applying a back bias. Although FIG. 11 shows a two-layer epitaxial growth substrate, there is no need to limit the number of layers to two, and three or more layers may be used. Furthermore, in FIG. 11, a combination of an element isolation region, a waveguide, and a laminated substrate is used, but a structure in which only a laminated substrate is formed, a combination of a laminated substrate and an element isolation region, or a laminated substrate and a laminated substrate may be used. A combination of wave paths may also be used.

In recent years, particularly in two-dimensional distance sensors, the number of pixels and resolution has increased, that is, the pixels have become smaller. Conventional three-branch lock-in pixels have a large number of elements, and as they become smaller, the sensitivity decreases and the elements cannot be arranged. In contrast, the 3D imaging unit according to the first embodiment has a simple structure of a one-branch lock-in pixel that has one transfer transistor G1 and one discharge transistor GD for one photodetector PD. As a result, a highly accurate, fine pixel distance sensor can be realized. In particular, since the structure is a simple one-branch lock-in pixel, the pixel area can be reduced and the resolution of the captured image can be improved.

(Second embodiment)
The structure of the 3D imaging unit according to the second embodiment is almost the same as the structure of the 3D imaging unit according to the first embodiment shown in FIGS. 1 to 5. When driving the solid-state imaging device that is an element of the 3D imaging unit according to the first embodiment, as shown in FIG. 6(a), the first on-frame of the first frame and the second on-frame of the second frame are The actions were repeated alternately. The signal data obtained in each of the first on-frame and the second on-frame were stored in the first memory 111 and second memory 112 of the storage unit 11 shown in FIG. On the other hand, in the 3D imaging unit according to the second embodiment, as shown in FIG. 12(a), the first frame (first on-frame), the second frame (second on-frame), and the third frame (Off-frame) operations are repeated in sequence.

Similarly to the 3D imaging unit according to the first embodiment, the "first on-frame" and the "second on-frame" each irradiate projection light and accumulate photoelectrons generated by the received light in the charge accumulation region 43a. This is a frame that The third frame "off frame" is a frame in which photoelectrons are accumulated in the charge accumulation region 43a without irradiating projection light. Therefore, in the 3D imaging unit according to the second embodiment, in addition to the first memory 111 and the second memory 112 of the storage unit 11 of the 3D imaging unit according to the first embodiment, The structure differs from that shown in FIG. 2 in that it includes a third memory for storing signal data.

FIG. 12 is a timing diagram illustrating the 3D image generation method according to the second embodiment. FIG. 12A shows that the first on-frame, second on-frame, and off-frame operations are repeated in order. 12(b) is a timing diagram of the first on-frame, FIG. 12(c) is a timing diagram of the second on-frame, and FIG. 12(d) is a timing diagram of the off-frame. ) is the same diagram as FIGS. 6(b) and 6(c). The operation of the 3D imaging unit according to the second embodiment of the first on-frame and second on-frame is similar to the operation of the 3D imaging unit according to the first embodiment. As shown in FIG. 12(b), in the first on-frame, the projection light is emitted at the timing of the light projection time _T0 . The timing of the pulse of the first drive signal G1 having a pulse width T _on is synchronized with the optical projection time T ₀ and operates as defined by the first acquisition achievement period τ _AA1 . As shown in FIG. 12(b), the first acquisition achievement period τ _AA1 can be set to the same value as the first charge accumulation time _Ta1 (τ _AA1 = _Ta1 ). As in the first embodiment, the projection light is periodically emitted at a repetition period time (cycle time) _TC .

As shown in FIG. 12(c), in the second on-frame, projection light is emitted at the timing of the light projection time _T0 , similarly to the first on-frame. The timing of the pulses of the first drive signal G1 in the second on-frame is determined to define a second acquisition achievement period τ _AA2 synchronized with the optical projection time T ₀ , and the first drive signal G1 with a pulse width T _on is It is applied to the transfer gate electrode 42a. The second acquisition achievement period τ _AA2 is a time measured from the rise of the projection light, and is a time different from the second charge accumulation time T _a2 . As shown in FIG. 12(c), the second charge accumulation time T _a2 is defined based on the fall of the pulse applied to the discharge gate electrode 42b of the discharge transistor GD. The photoelectrons acquired during the second acquisition achievement period τ _AA2 are accumulated in the charge accumulation region 43a. By applying the first drive signal G1 to the transfer gate electrode 42a at the timing shown in _FIG . The number of photoelectrons generated by the received light increases by the same amount as the amount of photoelectrons generated by the received light that decreased during the first on-frame due to the delay in the received light.

As shown in FIG. 12(d), in the off-frame, which is the third frame, the same operation as in the first on-frame is performed without emitting the projection light. That is, in the over-frame, the pulse of the first drive signal G1 is operated during the first acquisition achievement period τ _AA1 without emitting the projection light. Since the O-frame does not emit projection light, only electrons unnecessary for distance measurement, such as environmental light other than the received light and dark current, are accumulated in the charge accumulation region 43a. Note that the first on-frame, second on-frame, and off-frame may be in any order.

Electrons unnecessary for distance measurement, such as environmental light other than the received light and dark current, are similarly accumulated in the charge accumulation regions 43a of the first on-frame and second on-frame. With respect to the signal data obtained in each frame, the signal data obtained in the off-frame becomes a simple offset component due to the accumulation of unnecessary electrons, and according to the following equation (3), the signal data obtained in the off-frame This makes it possible to accurately measure the distance to the object, excluding the effects of such factors.

Q2 1 is the amount of charge acquired in the first on-frame and accumulated in the charge accumulation region 43a, Q2 ₂ is the amount of charge acquired in the second on-frame and accumulated in the charge accumulation region 43a, and Q2 ₂ is the amount of charge acquired in the off-frame and accumulated in the charge accumulation region 43a. If the amount of charge accumulated in 43a is Q2 _OFF , then

L=cT ₀ /2×Q2 ₂ '/(Q2 ₁ '+Q2 ₂ ')...(3)

is required. Here, Q2 ₁ ′=Q2 ₁ −Q2 _OFF and Q2 ₂ ′=Q2 ₂ −Q2 _OFF .

The conventional solid-state imaging device shown in FIG. 31 has a plurality of distribution gates, but the second embodiment has only a transfer transistor G1 serving as one distribution gate. In order to repeat the cyclic operation of the first on-frame, second on-frame, and off-frame, it is necessary to quickly discharge the charges generated and collected in the photoelectric conversion unit other than the storage time. In order to perform this drain operation, the drain transistor GD connected to the power source may be turned on.

In this way, also in the 3D imaging unit according to the second embodiment, the floating drain region FD1 and the auxiliary capacitor C1 are connected from one photodetecting section PD via the transfer transistor G1 serving as one distribution gate. A highly accurate, fine pixel distance sensor can be realized by a one-branch lock-in pixel having a structure connected to a power supply V _DD through one discharge transistor GD.

Although FIG. 12D shows a case where the same operation as the first on-frame is performed in the third off-frame, this is merely an example. The off-frame operation in which projection light is not emitted may be the same as the first on-frame operation or the same operation as the second on-frame operation. Furthermore, as the number of frames increases, the video characteristics deteriorate. Since the ambient light component is proportional to the accumulation time, off-frames can be corrected by, for example, reducing the number of repetitions, and in that case, simply adding a proportionality constant. In this way, speeding up means may be implemented by shortening the off-frame time.

If the off-frame operation is the same as the first on-frame or second on-frame operation, the ambient light component can be removed as an offset component using the same configuration output section, which is necessary when offsetting different outputs within one frame. This eliminates the need for gain correction, making processing easier.

-First modification of the second embodiment-
FIG. 13 is a diagram illustrating a 3D image generation method according to the first modification of the second embodiment. The pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the first modification of the second embodiment is the same as the pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the first embodiment shown in FIG. This is a one-branch type lock-in pixel in which one transfer transistor G1 is connected to a photoelectric conversion section 41 consisting of a photodiode (PD) whose portion other than the opening is shielded from light. The transfer transistor G1 has a floating drain region FD1, and an auxiliary capacitor C1 and a source follower type amplification transistor SF1 are connected to the floating drain region FD1 via wiring or the like.

The 3D image generation method according to the first modification of the second embodiment has the same basic technical idea as the second embodiment shown in FIG. The projection is performed in synchronization with the timing of the first acquisition achievement period of one frame, and the transfer transistor G1 is also synchronized with the timing of the first acquisition achievement period of the first frame. In the second frame, the projection light is projected in synchronization with the timing of the first acquisition achievement period of the first frame, but the transfer transistor G1 is projected in synchronization with the timing of the second acquisition achievement period set in the second frame. , the projection light is not emitted in the third frame. Note that the order of the first frame, second frame, and third frame may be any order as in the first embodiment.

The difference between the first modification of the second embodiment and the second embodiment lies in the method of excluding the offset component of the distance image, that is, the negative effects of environmental light and the like, from the images obtained in each frame. More specifically, in the first modification of the second embodiment, the transfer transistor G1 is set during the first acquisition achievement period set to no projection light (first off-frame) and with projection light (first on-frame). The transfer transistor G1 is synchronized to the second acquisition achievement period set with projection light (second on-frame), and a set of 4 frames with no projection light (second off-frame) is set, and environmental light, etc. The purpose of this embodiment is to use two frames, one before and one after, instead of using one frame as in the second embodiment.

In that case, as shown in FIG. 13, the off-frame is used for both the first and second on-frame set before the off-frame and the first and second on-frame set after the off-frame. Ru. At this time, for example, as shown in FIG. 13, the average value of the off-frame before the first and second on-frame sets and the off-frame after the first and second on-frame sets is calculated based on the environment. If used to exclude the harmful effects of light, etc., the temporal fluctuations can be detected more accurately, and the accuracy of distance measurement can be improved when the environmental light is strong or changes. Here, for the operation of the frame in which the projection light is not emitted, various time reduction methods may be used as in the second embodiment.

-Second modification of second embodiment-
FIG. 14 is a diagram illustrating a 3D image generation method according to a second modification of the second embodiment. The pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the second modification of the second embodiment is the same one-branch type as the pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the second embodiment. It is a lock-in pixel and has one charge storage region (floating drain region) and one charge discharge region (drain). In the conventional solid-state imaging device, data of three distribution gates, that is, a distance image excluding adverse effects such as environmental light, was obtained within one frame, but in the 3D imaging unit according to the second embodiment, A case has been described in which the functions of a conventional solid-state imaging device are realized using three frames.

When driving the solid-state imaging devices that are elements of the 3D imaging unit according to the second embodiment, unless the respective frame rates are changed, the total frame rate will be reduced to ⅓, and the dynamic resolution will be reduced. In the second modification of the second embodiment, to solve the above problem, as shown in FIG. 14(a), for one off-frame without projection light, two sets of on-frame with projection light are provided. By driving continuously and using one off-frame as a frame for excluding the negative effects of the following two on-frames such as environmental light, the frame rate is 6/5 times faster than that of the second embodiment. , the functions of the prior art can be realized while operating at a frame rate 2/5 times slower than that of the prior art. That is, it is possible to suppress a decrease in dynamic resolution.

As shown in FIG. 14(b), the number of consecutive sets of the first on-frame and second on-frame with projection light may be three (3 sets). When the first on-frame and second on-frame sets are driven in three consecutive sets, the frame rate is 9/7 times faster than that of the second embodiment, and the frame rate is 3/7 times slower than that of the conventional technology. The functions of the conventional technology can be realized while operating the system. That is, J may be a natural number of 2 or more, and driving may be performed in which J consecutive projection lights are present. This technology allows the functions of the conventional technology to be realized while operating at a frame rate as slow as J/(2J+1) times. That is, it is possible to suppress a decrease in dynamic resolution. Note that, similarly to the second embodiment, the off-frame may be placed before or after the on-frame. Further, as in the second embodiment, various time reduction methods may be used for off-frame operations.

-Third modification of the second embodiment-
FIG. 15 is a diagram illustrating a 3D image generation method according to a third modification of the second embodiment. The third modified example of the second embodiment is a combination of the first modified example of the second embodiment and the second modified example of the second embodiment, and has a higher dynamic resolution than the second embodiment. While suppressing the decrease, it is possible to improve the accuracy of distance measurement when the environmental light is strong or when there is a change, compared to the second embodiment and the second modification of the second embodiment.

(Third embodiment)
The structure of the 3D imaging unit according to the third embodiment is almost the same as the structure of the 3D imaging unit according to the first embodiment, but as shown in FIG. The structure differs from that shown in FIG. 4 in that it includes a transfer transistor G1 and a second transfer transistor G2. In FIG. 16, a planar pattern is illustrated in which the first transfer transistor G1 and the second transfer transistor G2 each have a first transfer gate electrode 152a and a second transfer gate electrode 152b. A pixel of a solid-state imaging device constituting a 3D imaging unit according to the third embodiment has a discharge transistor GD constituting one charge discharging mechanism in addition to two charge distribution mechanisms. Drain transistor GD is shown in FIG. 16 as having a drain gate electrode 152c.

When driving the solid-state imaging device that is an element of the 3D imaging unit according to the first embodiment, as shown in FIG. 6(a), a first on-frame (ON1 frame) and a second on-frame (ON2 frame) Repeat these actions alternately. The signal data obtained in each of the first on-frame and the second on-frame are stored in the first memory 111 and the second memory 112 of the storage unit 11 shown in FIG. In the 3D imaging unit according to the third embodiment, on-frame and off-frame operations are alternately repeated, as shown in FIG. 18(a).

As shown in FIGS. 18(b) and 18(c), data corresponding to the pulse of the first drive signal G1 and data corresponding to the pulse of the second drive signal G2 are obtained in each of the on-frame and off-frame. All the stored data is stored in the memory of the storage unit 11. Therefore, the 3D imaging unit according to the third embodiment has a first memory 111 and a second memory for storing signal data obtained on-frame in the storage section 11 of the 3D imaging unit according to the first embodiment. The 3D imaging unit according to the first embodiment is different from the 3D imaging unit according to the first embodiment in that, in addition to the memory 112, it includes a third memory and a fourth memory for storing signal data obtained off-frame.

Although the pixel plan view of the solid-state imaging device constituting the 3D imaging unit according to the third embodiment shown in FIG. 16 and the pixel plan view according to the conventional technology shown in FIG. 31 are drawn using the same layout rule, The area of the pixel plan view shown in FIG. 16 is half the area of the pixel plan view shown in FIG. 31. As shown in FIGS. 16 and 17, in the 3D imaging unit according to the third embodiment, the first transfer transistor G1 has the same function as that of the 3D imaging unit according to the first embodiment shown in FIGS. Similar to the transfer transistor G1, the first charge storage region 153a, the first auxiliary capacitor C1, and the first amplification transistor SF1 are connected via wiring or the like. Furthermore, in the 3D imaging unit according to the third embodiment, the second charge storage region 153b, the second auxiliary capacitor C2, and the second amplification transistor SF2 are also connected to the second transfer transistor G2 via wiring or the like. There is.

The first auxiliary capacitor C1 and the second auxiliary capacitor C2 of the 3D imaging unit according to the third embodiment may be added as MOS capacitive elements. However, since parasitic capacitances such as pn junction capacitance and inter-wiring capacitance can essentially be used instead, there is no need to intentionally add any elements. Further, the first auxiliary capacitor C1 and the second auxiliary capacitor C2 are laid out to be MOS capacitors having approximately the same size as the auxiliary capacitor (first auxiliary capacitor) C1 of the 3D imaging unit according to the first embodiment.

When a gate voltage that turns on the first transfer transistor G1 is applied to the first transfer gate electrode 152a, the photoelectrons generated in the light receiving region of the photoelectric conversion unit 151 move to the first charge storage region 153a. On the other hand, when a gate voltage that turns on the second transfer transistor G2 is applied to the second transfer gate electrode 152b, photoelectrons generated in the light receiving region of the photoelectric conversion section 151 move to the second charge storage region 153b. The pulsed light is emitted from almost the same position as the 3D imaging unit according to the third embodiment, the reflected light from the object is received by the imaging element, and the projection light is emitted from, for example, a light emitting diode (LED) or a semiconductor laser ( This is similar to the 3D imaging unit according to the first embodiment in that an extremely short pulsed light of, for example, ns order to fs order is projected using an LD (LD).

FIG. 18 is a timing diagram illustrating a 3D image generation method according to the third embodiment. FIG. 18(a) shows that the operations of the first frame (on-frame) and the second frame (off-frame) are alternately repeated, and FIG. 18(b) is a timing diagram of the on-frame. (c) is an off-frame timing diagram. As shown in FIG. 18(b), in the on-frame, the projection light is emitted at the timing of the light projection time _T0 , and the timing of the pulse of the first drive signal G1 with the pulse width T _on is the first acquisition achievement period τ _AA1 is applied to the first transfer gate electrode 152a. The first acquisition achievement period τ _AA1 can be set to the same value as the first charge accumulation time _Ta1 (τ _AA1 = _Ta1 ). The photoelectrons acquired during the first acquisition achievement period τ _AA1 are accumulated in the first charge accumulation region 153a.

Further, as shown in FIG. 18(b), in the on-frame, the timing of the pulse of the second drive signal G2 is determined to define a second acquisition achievement period τ _AA2 synchronized with the optical projection time T ₀ . Then, the second drive signal G2 with a pulse width T _on is applied to the second transfer gate electrode 152b at the timing of the second acquisition achievement period τ _AA2 . The second acquisition achievement period τ _AA2 is the time measured from the rise of the projection light. The photoelectrons acquired during the second acquisition achievement period τ _AA2 are accumulated in the second charge accumulation region 153b. The photoelectrons accumulated in the second charge accumulation region 153b at the timing shown in FIG. 18(b) cannot be acquired during the first acquisition achievement period τ _AA1 out of the photoelectrons generated by the received light due to the delay in the received light. increases by the same amount as the minute.

The timing of the pulses of the second drive signal G2 is synchronized with the second acquisition achievement period τ _AA2 that is long enough to include the time when the received light has completely arrived. By setting the timing shown in FIG. 18(b), to the first charge accumulation region 153a in accordance with the pulse of the first drive signal G1, and to the second charge accumulation region 153b in accordance with the pulse of the second drive signal G2, Depending on the delay of the received light, that is, the distance to the subject, a difference occurs in the amount of accumulated charge corresponding to each gate, and the distance to the subject is determined.

As shown in FIG. 18C, in the off-frame, which is the second frame, the same operation as in the first on-frame is performed without emitting the projection light. That is, in the O-frame, the pulse of the first drive signal G1 is operated during the first acquisition achievement period τ _AA1 and the pulse of the second drive signal G2 is operated during the second acquisition achievement period τ _AA2 without emitting the projection light. let Since the O-frame does not emit projection light, only electrons unnecessary for distance measurement, such as environmental light other than the received light and dark current, are accumulated in the first charge accumulation region 153a and the second charge accumulation region 153b.

Note that the first frame and the second frame may be performed in the opposite order, that is, the first frame may be an off-frame operation that does not emit projection light, and the second frame may be an on-frame operation that emit projection light. . In the 3D imaging unit according to the third embodiment, environmental light other than received light and dark current are accumulated in the first charge accumulation region 153a and the second charge accumulation region 153b during off-frames, and are unnecessary for distance measurement. Since electrons are also accumulated in the on-frame first charge accumulation region 153a and second charge accumulation region 153b, the off-frame output becomes a simple offset component with respect to each on-frame output, and is expressed as follows. By using equation (4), it becomes possible to accurately measure the distance to the object excluding the influence of environmental light and the like.

Q3 1 is the amount of charge acquired in the on-frame and accumulated in the first charge accumulation region 153a, Q3 ₂ is the amount of charge acquired in the off-frame and accumulated in the first _charge accumulation region 153a. If the amount of charge obtained is Q3 _1OFF and the amount of charge acquired in the off-frame and accumulated in the second charge accumulation region 153b is Q3 _2OFF , then

L=cT ₀ /2×Q3 ₂ '/(Q3 ₁ '+Q3 ₂ ')...(4)

is required. Here, Q3 ₁ ′=Q3 ₁ −Q3 _1OFF and Q3 ₂ ′=Q3 ₂ −Q3 _2OFF .

When the subject is on-frame and a little far away, photoelectrons that should have entered the second charge accumulation region 153b end up entering the first charge accumulation region 153a at the next repetition cycle time _TC . Since the first charge accumulation region 153a is also a signal component, if a delayed received light component enters the first charge accumulation region 153a beyond the repetition cycle time _TC , the distance accuracy will be significantly degraded. Therefore, if the discharge transistor GD whose drain (charge discharge region) is connected to the power source is turned on after the accumulation time of the second charge accumulation region 153b, the delayed received light component is discharged, and then the next repetition cycle time The delayed received light component does not enter the first charge storage region 153a with a period of T _C , and good distance accuracy can be maintained.

In this way, the two-branch type lock-in has a simple structure in which the first transfer transistor G1, the second transfer transistor G2, and one discharge transistor GD as two distribution gates are connected to one photodetector PD. By reducing the pixel area, a highly accurate, fine pixel distance sensor can be realized. In the 3D imaging unit according to the third embodiment, two first floating drain regions FD1 and two second floating drain regions are connected from one photodetecting section PD via first transfer transistor G1 and second transfer transistor G2, respectively. FD2 and two first auxiliary capacitors C1 and second auxiliary capacitors C2 are connected, and a drain (charge discharge region) connected to a power supply V _DD is connected from one photodetector PD through one discharge transistor GD. It has a two-branch lock-in pixel structure, which reduces the pixel area and provides highly accurate ranging and high-resolution images.

In the 3D imaging unit according to the third embodiment, the off-frame operation in which projection light is not emitted is preferably the same as the on-frame operation, but as the number of frames increases, the moving image characteristics deteriorate. Since the ambient light component is proportional to the accumulation time, for example, in off-frames, the number of repetitions is reduced, or only either the first transfer gate electrode 152a or the second transfer gate electrode 152b is driven, etc. Correction is possible by simply adding a proportionality constant. In addition to this, various speed-up measures may be taken, such as shortening the off-frame time.

Further, in the 3D imaging unit according to the third embodiment, if the off-frame operation is the same as the on-frame operation, the amount of charge accumulated in the first charge accumulation region 153a in the on-frame and off-frame is different from that of the on-frame operation. Since the ambient light component can be removed as an offset component depending on the amount of charge accumulated in the second charge accumulation region 153b, unlike the prior art, two different first floating drain regions FD1 and second floating drain regions within one frame can be removed. Gain correction required when offsetting between FD2s can be eliminated, and processing becomes lighter.

FIG. 19 is a diagram comparing pixel plan views of solid-state imaging devices according to the first embodiment, the third embodiment, and the conventional technology, with FIG. 19(a) showing the first embodiment, and FIG. ) is a pixel plan view of the third embodiment, and (c) is a pixel plan view of a conventional solid-state imaging device. The length of one side of each pixel is 3.2 μm in FIG. 19(a), 4.5 μm in FIG. 19(b), and 6.4 μm in FIG. 19(c). The solid-state imaging device, which is an element of the 3D imaging unit according to the first embodiment, has the same layout rule as the conventional solid-state imaging device, and the pixel area is one-fourth of that of the conventional solid-state imaging device. Four times as many pixels can be integrated. The solid-state imaging device, which is an element of the imaging unit according to the third embodiment, has the same layout rule as the conventional solid-state imaging device, and has a pixel area that is half the size of the conventional solid-state imaging device. Double the number of pixels can be integrated.

As the aspect ratio increases, crosstalk between pixels increases, and even if the pixels are made finer and the number of pixels is increased, the desired resolution cannot be obtained. It is desirable that the aspect ratio be 1 or less, and the range that crosstalk can be tolerated without taking special measures, specifically process-related measures, varies depending on the required specifications and is difficult to make a general decision. The aspect ratio is about 2. In other words, in FIG. 8, only the conventional technology using infrared light with a wavelength of 850 nm and a thickness of 10 μm has crosstalk within an allowable range without taking any special measures, specifically, process measures. Under other conditions, there is a problem that crosstalk cannot be tolerated. Even in the solid-state imaging device that is an element of the 3D imaging unit according to the third embodiment, if the element isolation region 213 is configured by a measure such as a DTI structure using a deep trench as shown in FIGS. , crosstalk can be prevented even if the pixel area becomes smaller.

-First modification of third embodiment-
FIG. 20 is a diagram illustrating a 3D image generation method according to the first modification of the third embodiment. The pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the first modification of the third embodiment is the same as the 3D imaging unit according to the third embodiment shown in FIG. 16, except for the opening. This is a two-branch type lock-in pixel in which a first transfer transistor G1 and a second transfer transistor G2 are connected as two distribution gates to a light-shielded photoelectric conversion section. The first transfer transistor G1 has a first floating drain region FD1, and the first auxiliary capacitor C1 and the gate electrode of the first amplification transistor SF1 are connected to the first floating drain region FD1 via wiring or the like. The second transfer transistor G2 has a second floating drain region FD2, and a second auxiliary capacitor C2 and a second amplification transistor SF2 are connected to the second floating drain region FD2 via wiring or the like.

The 3D image generation method according to the first modification of the third embodiment is also the same as the 3D imaging unit according to the third embodiment, and in the first frame (on-frame), the projection light is The projection light is not emitted in the second frame (off frame) in synchronization with the first acquisition achievement period τ _AA1 and the second acquisition achievement period τ _AA2 . Note that, as in the first embodiment, even if the first frame and the second frame are in the reverse order, that is, the first frame is an operation that does not emit projection light, and the second frame is an operation that causes projection light to be emitted. I don't mind.

The difference between the 3D imaging unit according to the first modification of the third embodiment and the 3D imaging unit according to the third embodiment is that the offset component of the distance image, that is, the environmental light There are ways to exclude such negative effects. More specifically, in the first modification of the third embodiment, a set of three frames includes one without projection light, one with projection light, and one without projection light. In addition, instead of using one frame as in the third embodiment as a frame for excluding the adverse effects of environmental light, two off-frames without projection light are used before and after an on-frame with projection light.

At this time, as shown in FIG. 20, the off-frame without projection light is used for both the preceding and succeeding on-frames with projection light. At this time, for example, as shown in FIG. 20, the average value of the two off-frames before and after the on-frame is used to exclude the adverse effects of ambient light and the like. If the off-frame average value is used, the temporal variation can be detected more accurately, and the accuracy of distance measurement can be improved when the ambient light is strong or changes. In the off-frame operation in which projection light is not emitted in the 3D imaging unit according to the first modification of the third embodiment, various time reduction methods may be performed as in the third embodiment.

-Second modification of third embodiment-
FIG. 21 is a diagram illustrating a 3D image generation method according to a second modification of the third embodiment. The pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the second modification of the third embodiment is the same two-branch type as the pixel structure of the solid-state imaging device constituting the 3D imaging unit according to the third embodiment. It is a lock-in pixel and has two charge storage regions (floating drain region) and one charge discharge region (drain). In the conventional solid-state imaging device, data of three distribution gates, that is, a distance image excluding negative effects such as environmental light, was obtained within one frame, but in the 3D imaging unit according to the third embodiment, Functions equivalent to those of the conventional technology are achieved using two frames. In the driving in the third embodiment, if each frame rate is not changed, the total frame rate (number of frames per unit time) will be reduced to 1/2, and the dynamic resolution will be reduced.

In the second modification of the third embodiment, in order to solve the above problem, two on-frames are continuously driven for one off-frame, and one off-frame is driven as shown in FIG. 21(a). By using the subsequent two on-frames as a frame for excluding the negative effects of environmental light, etc., the frame rate is 4/3 times faster than that of the third embodiment, and the frame rate is 2/3 times slower than that of the conventional technology. It is possible to realize the functions of the conventional example while operating the system. That is, it is possible to suppress a decrease in dynamic resolution.

As shown in FIG. 21(b), the number of consecutive on-frames may be three. In the case of driving with three consecutive on-frames, the functions of the conventional example can be realized while operating at a frame rate as fast as 3/2 times that of the first embodiment and as slow as 3/4 times that of the prior art. In other words, it may be driven with J (a natural number of 2 or more) consecutive on-frames, and in that case, the frame rate is 2J/(J+1) times faster than that of the third embodiment, and the frame rate is J/(J+1) times faster than that of the conventional example. The functions of the conventional example can be realized while operating at a frame rate as low as (J+1) times. That is, it is possible to suppress a decrease in dynamic resolution. Note that, as in the third embodiment, the off-frame may be placed before or after the on-frame. Further, as in the third embodiment, various time reduction methods may be used for off-frame operations.

-Third modification of third embodiment-
FIG. 22 is a diagram illustrating a 3D image generation method according to a third modification of the third embodiment. The third modified example of the third embodiment is a combination of the first modified example of the third embodiment and the second modified example of the third embodiment, and has a higher dynamic resolution than the third embodiment. While suppressing the decrease, it is possible to improve the accuracy of distance measurement when the environmental light is strong or when there is a change, compared to the third and second modified examples of the third embodiment.

(Other embodiments)
As mentioned above, the present invention has been described using the first to third embodiments and their modifications, but it should be understood that the statements and drawings that form part of this disclosure limit the present invention. do not have. Various alternative embodiments, implementations, and operational techniques will be apparent to those skilled in the art from this disclosure. Driving the solid-state imaging device, which is an element of the 3D imaging unit according to the second and third modified examples of the second embodiment and the second and third modified examples of the third embodiment, is the basis of the present invention. Solid-state imaging that is an element of a 3D imaging unit according to the first embodiment, the second embodiment, the first modification of the second embodiment, the third embodiment, and the first modification of the third embodiment Although this drive improves the dynamic resolution of the device, its distance accuracy is somewhat inferior to changes in environmental light and the like.

That is, normally, the 3D imaging unit according to the first embodiment, the second embodiment, the first modification of the second embodiment, the third embodiment, and the first modification of the third embodiment When performing basic driving of the solid-state imaging device that is an element and wishing to increase the dynamic resolution, 3D imaging according to the second and third modified examples of the second embodiment and the second and third modified examples of the third embodiment is used. It is sufficient to drive the solid-state imaging device, which is an element of the unit, to increase the dynamic resolution. These can be realized, for example, by the photographer viewing the environment and making a selection using the mode switch of a camera equipped with an image sensor as shown in FIG.

For example, an alternative embodiment is also possible in which the selection between the basic drive and the drive for increasing the dynamic resolution is performed automatically, rather than being selected by the photographer. FIG. 23 shows a flowchart outlining the flow of an example of 3D image generation operation according to another embodiment. In step S311 of FIG. 23, "basic drive" is set. Here, "basic drive" means the third embodiment or the first modification of the third embodiment in the case of a two-branch lock-in pixel, and the "basic drive" in the case of a one-branch lock-in pixel. Examples include the first embodiment, the second embodiment, and a first modification of the second embodiment.

In step S312, ``distance measurement'' is performed using the set ``basic drive'', followed by ``storing an ambient light image'' in step S313, and ``outputting a distance image'' in step S314. Here, in the example of the flowchart in FIG. 23, "memorize the ambient light image" is used to make a judgment based on the ambient light output, but it is not limited to only the ambient light image, but also includes distance images, images of each tap, etc. You can memorize it.

Next, in step S315, it is checked whether "the difference with the ambient light image of the previous frame is compared and the difference is smaller than a predetermined threshold value." Check the difference, and if the difference is small (Yes), it is determined that there is little change in the environment, and the number of frames for acquiring the environmental light image is reduced.If the difference is large, the "set to high speed drive" in step S316 is changed. If (No), it is determined that the change in the environment is large, and the frame for acquiring the ambient light image and the frame for acquiring the distance image including the ambient light are paired, and the "set to basic drive" is changed in step S311, Perform the next imaging. Here, in step S315, since there is no stored image immediately after the rise, you may leave the answer "No" and capture the next frame, or you may compare it with the ambient light image stored at the previous fall. good.

In the flowchart of FIG. 23, "high-speed drive" refers to the second modification or the third modification of the third embodiment in the case of a two-branch lock-in pixel, and in the case of a one-branch lock-in pixel. , a second modification example and a third modification example of the second embodiment. Further, when changing to "set to high speed drive", the number of consecutive on-frames may be determined in advance and fixed, or the number of consecutive on-frames may be gradually increased. As a result of the above operation, the basic drive is normally performed, and when you want to increase the dynamic resolution, the drive selection to increase the dynamic resolution is not done manually by the photographer, but is automatically selected by the camera equipped with an image sensor. be able to.

As a comparative example, FIG. 27 shows a planar pattern of a lock-in pixel with seven branches and one discharge section. In FIG. 27, a perforated photoelectric conversion section 321 whose portion other than the opening is shielded from light by a light-shielding film (not shown) has eight MOS gates, a first transfer transistor G1, a second transfer transistor G2, and a third transfer transistor G3. , the fourth transfer transistor G4, the fifth transfer transistor G5, the sixth transfer transistor G6, and the seventh transfer transistor G7 are connected to the drain transistor GD. The first transfer transistor G1 to the seventh transfer transistor G7 of the eight MOS gates include a first floating drain region FD1 to a seventh floating drain region FD7, an auxiliary capacitor (not shown), and a source follower type amplification. Transistors (not shown) are connected via wiring or the like. The drain (charge discharge region) of the discharge transistor GD constituting the remaining charge discharge mechanism is connected to the power supply V _DD .

The photoelectrons generated and collected in the photoelectric conversion section are sent to one of the distribution gates of the first transfer transistor G1 to the seventh transfer transistor G7 connected to the photoelectric conversion section, for example, the first transfer transistor G1, at a high level. If given, photoelectrons move to the first floating drain region FD1, and similarly the second transfer transistor G2, the third transfer transistor G3, the fourth transfer transistor G4, the fifth transfer transistor G5, the sixth transfer transistor G6, When a high level is applied to the seventh transfer transistor G7, the photoelectric conversion unit supplies the second floating drain region FD2, the third floating drain region FD3, the fourth floating drain region FD4, the fifth floating drain region FD5, and the sixth floating drain region FD6. , photoelectrons move to the seventh floating drain region FD7.

As shown in FIG. 28, when the subject is relatively close, there is a delay time T _dn from the time when the projection light is emitted, and when the subject is relatively far away, there is a delay T _df . The emission time (T ₀ ) of the projected light and the on/off voltage pulses corresponding to the first to seventh transfer transistors G1 to G7 are synchronized. For example, when the subject is relatively close, the reflected light is transmitted to the second floating drain region FD2 in accordance with the pulse of the second transfer transistor G2, and to the third floating drain region FD3 in accordance with the pulse of the third transfer transistor G3. The delay (T _dn ) causes a difference in the charge accumulated in each floating drain region, and the object distance is determined. Similarly, when the subject is relatively far away, the reflection is reflected to the fifth floating drain region FD5 in accordance with the pulse of the fifth transfer transistor G5, and to the sixth floating drain region FD6 in accordance with the pulse of the sixth transfer transistor G6. The light delay (T _df ) causes a difference in the charge accumulated in each floating drain region, which determines the object distance.

As shown in FIG. 28, when the number of distribution gates is increased, if the charge accumulation time (T _a ) is the same cycle, the increased number of distribution gates makes it possible to measure a longer distance. When measuring, by setting the T _a time short, the resolution of the measurement distance can be increased by the increase in the number of distributed gates. FIG. 24 shows a driving method that uses the two-branch lock-in pixel shown in FIG. 16 to provide the same function as the comparative example shown in FIG. 28.

(1) to (4) shown in the upper row of each of FIGS. 24(a) and (b) indicate the positions of the first on-frame 1 to the fourth on-frame located below. Numbers (1) to (4) written in the vertical direction of the timing diagram in FIG. 24(c) correspond to first on-frame 1 to fourth on-frame. FIG. 24(a) is an example in which white off-frames and shaded on-frames are alternately and continuously driven, and the off-frames are used as frames for excluding the negative effects of on-frames such as environmental light. In FIG. 24(b), four shaded first on-frames 1 to 4 are continuously driven for one white off-frame, and one off-frame is driven sequentially to the four subsequent first on-frames. It is used as a frame for excluding the negative effects of environmental light and the like in on-frames 1 to 4. As shown in Figure 24, by combining the four first on-frames 1 to 4 on-frames, a function equivalent to an eight-branch lock-in pixel with eight floating drain regions and one charge discharge part (drain) can be obtained. realizable. Furthermore, using this method, in principle, the number of branches (=number of taps) can be increased infinitely.

FIG. 25 shows a 3D image generation method according to another embodiment that uses the one-branch lock-in pixel shown in FIG. 4 to provide the same function as the comparative example shown in FIG. 28. In FIGS. 25(a) and 25(b), the first on-frame (ON1) to fourth on-frame (ON4) shown as shaded cells and the fifth on-frame (ON5) to fourth on-frame shown as hatched cells The 7th on-frame (ON7) corresponds to the first on-frame (ON1) to the seventh on-frame (ON7) in FIG. 25(c). FIG. 25A shows an example in which the white off-frame and the first on-frame (ON1) to the seventh on-frame (ON7) are alternately and consecutively driven.

FIG. 25(b) shows that for one off-frame, consecutive cells from the first on-frame (ON1) to the seventh on-frame (ON7) are sequentially driven in tandem, and one off-frame is driven sequentially. It is used as a frame for excluding the adverse effects of environmental light and the like on the two consecutive cells of the first on-frame (ON1) to the seventh on-frame (ON7). As shown in FIG. 25, by combining eight sets of frames, a function equivalent to a seven-branch lock-in pixel having seven floating drain regions and one charge discharge part (drain) can be realized. Furthermore, if this method is used, in principle, the number of frames can be infinitely increased and the equivalent number of taps can be increased using a single-branch lock-in pixel.

That is, by setting P to be a positive integer of 4 or more and having P memories in parallel between the distributor 113 and the distance calculating section 12, the number of frames can be set to P. In this case, the drive unit 15 shown in FIG. 1 drives the charge distribution mechanism at different acquisition achievement period timings in (P-1) frames to generate (P-1) analog signals. Then, in one frame, without emitting pulses of projection light, the charge distribution mechanism is driven at the same timing as the acquisition period of any one of the (P-1) frames. After the charge distribution mechanism is driven, the charge discharging mechanism is driven. The column processing circuits (23, 24) shown in FIG. The P pieces of memory store the discriminated P pieces of digital data, and the distance calculation unit 12 calculates the distance to the target object using the P pieces of digital data read from the P pieces of memory. can be used as correction data.

The structure of the charge distribution mechanism and the charge discharge mechanism of the 3D imaging unit according to the first to third embodiments of the present invention and their modifications is the electrode structure of a lateral electric field controlled charge modulation element (LEFM) and an insulated gate transistor. The structure of the transfer transistor is not limited to the above-described structure as long as it has a function of transporting and transferring signal charges. The photoelectric conversion section of the 3D imaging unit according to the first to third embodiments of the present invention and their modifications is a photoelectric conversion section using a pn junction type photodiode or a MOS structure with a transparent electrode as a gate electrode. The present invention is not limited to the photogate structure having the following structure, and any other structure having a similar photoelectric conversion function may be used. Thus, it goes without saying that the present invention includes various embodiments not described here. Therefore, the technical scope of the present invention is determined only by the matters specifying the invention in the claims that are reasonable from the above description.

The 3D imaging units according to the first to third embodiments of the present invention and their modifications can be used as 3D imaging devices in the technical field, such as cameras exemplarily shown in FIG. 26, for example. As exemplarily shown in FIG. 26, a camera such as a video camera that can be used in the technical field includes an imaging optical system 2 that adjusts the optical path from an object 10 (see FIG. 1), and an imaging optical system 2 that adjusts the optical path from an object 10 (see FIG. 1). An imaging chip 368 that uses the elements of the 3D imaging unit according to the first to third embodiments and their modifications to capture an image of the object 10 incident along the optical axis of the image sensor, and an autofocus (AF) A distance sensor (distance measuring element) 374 is provided, which uses as a main part the pixels of a solid-state imaging device constituting the 3D imaging unit according to the first to third embodiments and their modifications.

The imaging chip 368 illustrated in FIG _. 26 includes the light receiving section 18 shown in FIG _. are integrated on a second semiconductor chip, and the distance calculation section 12, control section 14, program storage device 17, data storage device 13, storage section 11, output section 16, etc. are integrated on a third semiconductor chip, and the first to A chip having a structure in which third semiconductor chips are sequentially stacked and integrated may also be used. Further, the light receiving section 18 shown in FIG. 1 is configured on a first semiconductor chip, and the driving section 15, horizontal scanning circuit 22, vertical scanning circuit 21, and noise processing circuits NC ₁ to NC _m are integrated on a second semiconductor chip. , a chip having a structure in which the first and second semiconductor chips are stacked and integrated is adopted as the imaging chip 368, and includes the distance calculation section 12, the control section 14, the program storage device 17, the data storage device 13, and the storage section 11. The output unit 16 and the like may be made to function in other configurations of the camera exemplarily shown in FIG. 26.

A camera that may utilize the present invention converts image data output from the imaging chip 368 using the elements of the 3D imaging unit according to the first to third embodiments and their modifications into digital data. An A/D conversion circuit 365, a memory (semiconductor storage device) 364 that stores image data converted into digital data by the A/D conversion circuit 365, and a central processing control unit (CPU) 361 that receives image data from the memory 364. and an image processing unit 362 that receives image data via a central processing control unit 361 and processes the image data. An adjustment data storage device 363 that stores adjustment data of the imaging chip 368 and the distance sensor (distance measuring element) 374 is connected to the image processing unit 362.

Note that FIG. 26 is only an example, and an adjustment data storage device for storing adjustment data sent from the central processing control unit 361 is connected to the semiconductor chip on which the imaging chip 368 or the distance sensor (distance measuring element) 374 is mounted. A structure may also be used in which adjustment data is supplied to a drive circuit on a semiconductor chip. Cameras that may utilize the present invention further include a drive unit 366 connected to the central processing control unit 361, a memory card interface 378 such as a media controller, an operation unit 377, and a liquid crystal display (LCD) drive circuit 375. , motor drivers 373a, 373b, 373c, and a strobe control circuit 380. A display section 376 made of an LCD is connected to the LCD drive circuit 375, and a strobe device 387 is connected to the strobe control circuit 380. The strobe device 387 can constitute the light emitting section 1 shown in FIG.

The central processing control section 361 of the camera illustrated in FIG. 26 includes an image processing section 362 connected to the central processing control section 361, a driving section 366, a memory 364, a memory card interface 378, an operation section 377, an LCD drive circuit 375, It outputs commands and electrical signals that control the operations and processes of the distance sensor (distance measuring element) 374, motor drivers 373a, 373b, 373c, and strobe control device. Although not shown, the central processing control unit 361 includes an image processing unit 362, a drive unit 366, a memory 364, a memory card interface 378, an operation unit 377, an LCD drive circuit 375, and a distance sensor (distance measuring element). 374, motor drivers 373a, 373b, 373c, various logic circuits such as a WB adjustment command output circuit that performs auto white balance (AWB) adjustment, in addition to command output circuits that execute the respective operations of the strobe control device. It is incorporated as a standard hardware resource.

As shown in FIG. 26, the photographing lens 370 constituting the imaging optical system (369, 370) includes, for example, a main lens 370a, a zoom lens 370b adjacent to the main lens 370a, a focus lens 370c adjacent to the zoom lens 370b, etc. can be provided. In the structure illustrated in FIG. 26, a zoom motor 371a is connected to the zoom lens 370b, and a focus motor 371b is connected to the focus lens 370c. A diaphragm 369 that constitutes an imaging optical system (369, 370) is arranged between the focus lens 370c and the imaging chip 368. For example, an iris motor 371c that drives the aperture blades is connected to an aperture 369 made up of five aperture blades. The zoom motor 371a, the focus motor 371b, and the iris motor 371c are composed of stepping motors, and their operation is controlled by drive pulses sent from motor drivers 373a, 373b, and 373c connected to the central processing control unit 361. Imaging preparation processing is performed based on the signal from 377.

The zoom motor 371a moves the zoom lens 370b to the wide side or the telephoto side in 20 to 50 steps, for example, and zooms the photographic lens 370. The focus motor 371b moves the focus lens 370c according to the distance from the object 10 and the magnification of the zoom lens 370b, and adjusts the focus of the photographing lens 370 so that the imaging conditions of the camera are optimized. The iris motor 371c operates the diaphragm blades of the diaphragm 369 to change the aperture area of the diaphragm 369, and adjusts the exposure of the photographing lens 370 in five steps, for example, from aperture value F2.8 to F16 in steps of 1 AV.

The photographing lens 370 is not limited to the configuration illustrated in FIG. 26, and may be an interchangeable lens that can be attached to and detached from the camera, for example. The photographing lens 370 is composed of a plurality of optical lens groups such as a main lens 370a, a zoom lens 370b, and a focus lens 370c, so that the light beam from the object 10 is directed to the surface of the imaging chip 368 arranged near the focal plane of the object 10. to form an image.

The imaging chip 368 according to the first to third embodiments and their modifications is mounted on a chip mounting substrate (package substrate) 367 made of glass or ceramic. A timing generator (TG) 372 is connected to the imaging chip 368 , and the timing generator 372 is connected to the central processing control section 361 via a drive section 366 . The timing generator 372 generates a timing signal (clock pulse) in response to a signal sent from the central processing control unit 361 via the drive unit 366, and the timing signal is sent to the semiconductor chip constituting the imaging chip 368 via the chip mounting board 367. The signal is sent to pixels in each row as an electronic shutter signal from a drive circuit provided as a peripheral circuit.

That is, the central processing control section 361 controls the timing generator 372 via the drive section 366 to control the shutter speed of the electronic shutter of the imaging chip 368. Note that the timing generator 372 may be monolithically integrated as a peripheral circuit on a semiconductor chip that constitutes the imaging chip 368.

The imaging signal output from the central pixel array section of the semiconductor chip constituting the imaging chip 368 is input to a correlated double sampling circuit (CDS) provided as a peripheral circuit on the periphery of the semiconductor chip. The image pickup chip 368 outputs R, G, and B image data that accurately corresponds to the amount of accumulated charge in each pixel. Image data output from the imaging chip 368 is amplified by an amplifier (not shown), and converted into digital data by an A/D conversion circuit 365. The imaging chip 368 is timing controlled by the driving unit 366, converts the image of the object 10 formed on the light receiving surface of the imaging chip 368 into an image signal, and outputs the image signal to the A/D conversion circuit 365.

Although not shown, the image processing unit 362 of the camera illustrated in FIG. A logical operation circuit for automatic exposure (AE) detection that integrates G signals weighted differently between the area and the peripheral area and outputs the integrated value. A shooting Ev value calculation circuit that calculates the brightness (shooting Ev value) of the object 10, and various image processing and calculations associated with image processing, such as a gradation conversion processing circuit, a white balance correction processing circuit, a γ correction processing circuit, etc. It is also possible to provide various logic circuits (hardware modules) that perform the following on image data as hardware resources in the logic configuration.

The image processing unit 362 of the camera illustrated in FIG. 26 can be realized if there is an image processing engine or the like. Furthermore, if the calculation load is high for feature value generation and identification processing, it may be implemented in hardware. For example, it is also possible to configure the image processing unit 362 in a computer system using an MPU or the like implemented as a microchip. Further, as the image processing unit 362 constituting the computer system, a DSP with enhanced arithmetic operation functions and specialized for signal processing, a microcomputer equipped with memory and peripheral circuits and intended for embedded device control, etc. may be used. Alternatively, the main CPU of a current general-purpose computer may be used for the image processing section 362. Furthermore, part or all of the image processing unit 362 may be configured with a PLD such as an FPGA.

10...Object, 11...Storage unit, 12,72...Distance calculation unit, 13,73...Data storage device, 14,74...Control unit, 15,75...Drive unit, 16,76... ...output section, 17,77...program storage device, 18,71...light receiving section, 19...light emitting section, 20...lens, 21,81...vertical scanning circuit, 22,82...horizontal scanning circuit, 23... Correlated double sampling circuit 23... Analog-digital conversion circuit, 111... First memory, 112... Second memory, 113... Distributor (demultiplexer), 181... Signal generation section, 141... ... Time setting logic circuit, 142 ... Time setting value output control circuit, 143 ... Distance image output control circuit, 144 ... Setting value judgment circuit, 145 ... Sequence control circuit, 146 ... Bus, 41, 91, 151 , 214, 321...Photoelectric conversion unit, 42a, 92a, 152a...Transfer gate electrode, 42b, 92d, 152c...Ejection gate electrode, 43a, 153a...Charge storage region (first charge storage region), 43b... ... Charge discharge region, 153b ... Second charge storage region, 44, 94a ... Reset gate electrode, 46 ... Selection gate electrode, 47 ... Gate electrode (amplification gate electrode of amplification transistor), 48 ... Virtual line, 49... Diffusion region, 50... Reset drain region, 51... Amplification source region, 52... Selected source region, 92b, 152b... Second transfer gate electrode, 92c... Third transfer gate electrode, 93a... 1st charge accumulation region, 93b...2nd charge accumulation region, 93c...3rd charge accumulation region, 94b...2nd reset gate electrode, 94c...3rd reset gate electrode, 211...p ⁺ substrate, 212 ... Epitaxial growth layer (second epitaxial growth layer), 213 ... Element isolation region, 215 ... Microlens, 216 ... Wiring layer, 217 ... Metal light shielding film, 221 ... Waveguide, 222 ... P ⁺ type substrate or p ⁺ type surface pinning layer, 231... first epitaxial growth layer, 232... second epitaxial growth layer, 361... central processing control section, 362... image processing section, 363... adjustment data storage device, 364... Memory, 365...A/D conversion circuit, 366...Drive unit, 367...Chip mounting board, 368...Solid-state imaging device, 369...Aperture, 370...Photographing lens, 370a...Main lens, 370b... ...Zoom lens, 370c...Focus lens, 371a...Zoom motor, 371b...Focus motor, 371c...Iris motor, 372...Timing generator (TG), 373a, 373b, 373c...Motor driver, 374... Distance sensor, 375...LCD drive circuit, 376...Display section, 377...Operation section, 378...Memory card interface, 379...Memory card, 380...Strobe control circuit, 387...Strobe device

Claims (12)

a pixel array section in which a plurality of pixels are arranged in a matrix to receive pulsed reflected light from an object and convert it into electricity;
In a first frame, the pixel is driven to generate a first analog signal at the timing of a first acquisition achievement period defined in synchronization with the pulse of the projection light that generates the reflected light, and the pixel is driven to generate a first analog signal. drives the pixel in a second frame in a different time domain at the timing of a second acquisition achievement period, which is defined for the pulse of the projection light and has a different length from the first acquisition achievement period, to obtain a second acquisition period. a drive unit that generates an analog signal;
In each of the first and second frames, the first and second analog signals from the plurality of pixels arranged in a specific row of the matrix are independently read out, and first and second digital data are read out. A column processing circuit that converts
a distributor that discriminates and outputs the first and second digital data;
first and second memories that independently store the discriminated first and second digital data;
A 3D imaging unit comprising: a distance calculation unit that inputs the first and second digital data read from the first and second memories and calculates a distance to the target object. Each of the pixels is
a photoelectric conversion unit that photoelectrically converts the reflected light;
a charge distribution mechanism disposed around the photoelectric conversion unit and transferring signal charges generated in the photoelectric conversion unit along a charge transfer path;
a charge discharging mechanism disposed around the photoelectric conversion unit apart from the charge distribution mechanism, and discharging charges other than the signal charges from the photoelectric conversion unit ;
3D according to claim 1, further comprising a charge accumulation region that is arranged around the photoelectric conversion unit in contact with the charge distribution mechanism and that accumulates the signal charge transferred by the charge distribution mechanism. Imaging unit. In the first frame, the drive unit drives the charge distribution mechanism at the timing of the first acquisition achievement period, drives the charge discharge mechanism after driving the charge distribution mechanism, and drives the charge discharge mechanism at the timing of the first acquisition achievement period. 3D imaging according to claim 2, wherein in the frame, the charge distribution mechanism is driven at the timing of the second acquisition achievement period, and the charge discharge mechanism is driven after the drive of the charge distribution mechanism is completed. unit. further comprising a third memory between the distributor and the distance calculation unit in parallel with the first and second memories;
The drive unit drives the charge distribution mechanism at the timing of the first acquisition achievement period in a third frame in a time domain different from the first and second frames, while not emitting pulses of the projection light. to generate a third analog signal, and after driving the charge distribution mechanism, drive the charge discharge mechanism;
The column processing circuit independently reads out the third analog signals from the plurality of pixels arranged in a specific row of the matrix in the third frame, and converts them into third digital data,
the distributor discriminates and outputs the first to third digital data;
the third memory stores the discriminated third digital data;
4. The 3D imaging unit according to claim 3, wherein the distance calculation section uses the third digital data read from the third memory as correction data for calculating the distance to the object. P memories are provided in parallel between the distributor and the distance calculation unit, where P is a positive integer of 4 or more,
The drive unit drives the charge distribution mechanism at timings of different acquisition achievement periods in (P-1) frames to generate (P-1) analog signals, and generates (P-1) analog signals in one frame. , in a state where the pulse of the projection light is not emitted, drive the charge distribution mechanism at the same timing as the acquisition period of any one of the (P-1) frames to generate one analog signal; after driving the charge distribution mechanism, driving the charge discharging mechanism;
The column processing circuit reads P analog signals independently and converts them into P digital data,
the distributor discriminates and outputs P pieces of digital data;
the P memories store the discriminated P pieces of digital data;
5. The 3D imaging unit according to claim 4, wherein the distance calculation section uses the P pieces of digital data read from the P pieces of memory as correction data for calculating the distance to the object. First and second charge distribution mechanisms are installed around the photoelectric conversion unit to receive pulsed reflected light from a target object, photoelectrically convert it, and transfer the photoelectrically converted signal charges along different charge transfer paths. a pixel array section in which a plurality of pixels are arranged in a matrix, each having a pixel array section;
In a first frame, drive the first charge distribution mechanism at the timing of a first acquisition achievement period defined in synchronization with a pulse of projection light that generates the reflected light to generate a first analog signal; The second charge distribution mechanism is driven to generate a second analog signal at the timing of a second acquisition achievement period that is defined for the pulse of the projection light and has a different length from the first acquisition achievement period. , in a second frame in a time domain different from the first frame, in a state where the pulse of the projection light is not emitted, the first charge distribution mechanism is driven at the timing of the first acquisition achievement period to generate a third charge distribution mechanism; a driving unit that generates an analog signal and drives the second charge distribution mechanism at the timing of the second acquisition achievement period to generate a fourth analog signal;
a column processing circuit that independently reads out the first to fourth analog signals from the plurality of pixels arranged in a specific row of the matrix and converts them into first to fourth digital data;
a distributor that discriminates and outputs the first to fourth digital data;
first to fourth memories each independently storing the discriminated first to fourth digital data;
A 3D imaging unit comprising: a distance calculation section that inputs data read out from the first to fourth memories and calculates a distance to the object. Each of the pixels is
a charge discharging mechanism disposed around the photoelectric conversion unit apart from the first and second charge distribution mechanisms, and discharging charges other than the signal charges from the photoelectric conversion unit ;
a first charge accumulation region disposed around the photoelectric conversion unit in contact with the first charge distribution mechanism and accumulating the signal charge transferred by the first charge distribution mechanism;
It is characterized by further comprising a second charge accumulation region disposed around the photoelectric conversion unit in contact with the second charge distribution mechanism and for accumulating the signal charges transferred by the second charge distribution mechanism. The 3D imaging unit according to claim 6. The drive unit drives the charge discharging mechanism after both the first and second charge distributing mechanisms have been driven in the first frame, and the drive unit drives the charge discharging mechanism in the second frame. 8. The 3D imaging unit according to claim 7, wherein the charge discharging mechanism is driven after both of the separation mechanisms have been driven. a light emitting unit that emits a light pulse to a target object;
an imaging optical system that adjusts the optical path of the pulsed reflected light from the target object;
a pixel array section in which a plurality of pixels are arranged in a matrix to receive and photoelectrically convert the reflected light adjusted by the imaging optical system;
In a first frame, the pixel is driven to generate a first analog signal at the timing of a first acquisition achievement period defined in synchronization with the pulse of the projection light that generates the reflected light, and the pixel is driven to generate a first analog signal. drives the pixel in a second frame in a different time domain at the timing of a second acquisition achievement period, which is defined for the pulse of the projection light and has a different length from the first acquisition achievement period, to obtain a second acquisition period. a drive unit that generates an analog signal;
In each of the first and second frames, the first and second analog signals from the plurality of pixels arranged in a specific row of the matrix are independently read out, and first and second digital data are read out. A column processing circuit that converts
a distributor that discriminates and outputs the first and second digital data;
first and second memories that independently store the discriminated first and second digital data;
A camera comprising: a distance calculation section that inputs the first and second digital data read from the first and second memories and calculates a distance to the target object. a light emitting unit that emits a light pulse to a target object;
an imaging optical system that adjusts the optical path of the pulsed reflected light from the target object;
A photoelectric conversion unit includes first and second charge distribution mechanisms that receive and photoelectrically convert the reflected light adjusted by the imaging optical system and transfer the photoelectrically converted signal charges along mutually different charge transfer paths. a pixel array section in which a plurality of pixels are arranged in a matrix, each having a plurality of pixels around the periphery of the pixel array section;
In a first frame, drive the first charge distribution mechanism at the timing of a first acquisition achievement period defined in synchronization with a pulse of projection light that generates the reflected light to generate a first analog signal; The second charge distribution mechanism is driven to generate a second analog signal at the timing of a second acquisition achievement period that is defined for the pulse of the projection light and has a different length from the first acquisition achievement period. , in a second frame in a time domain different from the first frame, in a state where the pulse of the projection light is not emitted, the first charge distribution mechanism is driven at the timing of the first acquisition achievement period to generate a third charge distribution mechanism; a driving unit that generates an analog signal and drives the second charge distribution mechanism at the timing of the second acquisition achievement period to generate a fourth analog signal;
a column processing circuit that independently reads out the first to fourth analog signals from the plurality of pixels arranged in a specific row of the matrix and converts them into first to fourth digital data;
a distributor that discriminates and outputs the first to fourth digital data;
first to fourth memories each independently storing the discriminated first to fourth digital data;
A camera characterized by comprising: a distance calculation section that inputs data read out from the first to fourth memories and calculates a distance to the object. Using a pixel array section in which a plurality of pixels are arranged in a matrix, receiving pulsed reflected light from an object and photoelectrically converting it;
In a first frame, driving the pixel at a timing of a first acquisition achievement period defined in synchronization with a pulse of projection light that generates the reflected light to generate a first analog signal;
In a second frame in a time domain different from the first frame, driving the pixel at the timing of a second acquisition achievement period that is defined for the pulse of the projection light and has a length different from the first acquisition achievement period. generating a second analog signal;
In each of the first and second frames, the first and second analog signals from the plurality of pixels arranged in a specific row of the matrix are independently read out, and first and second digital data are read out. and the step of converting it to
discriminating the first and second digital data from each other and outputting the same;
storing the discriminated first and second digital data independently in first and second memories, respectively;
A 3D image generation method, comprising: inputting the first and second digital data read from the first and second memories and calculating a distance to the target object. A pixel array in which a plurality of pixels are arranged in a matrix, each of which has first and second charge distribution mechanisms that transfer signal charges along different charge transfer paths, respectively, around a photoelectric conversion section that generates the signal charges. a step of receiving pulsed reflected light from a target object and photoelectrically converting it using a device;
In a first frame, drive the first charge distribution mechanism at the timing of a first acquisition achievement period defined in synchronization with a pulse of projection light that generates the reflected light to generate a first analog signal; The second charge distribution mechanism is driven to generate a second analog signal at the timing of a second acquisition achievement period that is defined for the pulse of the projection light and has a different length from the first acquisition achievement period. step and
In a second frame in a time domain different from the first frame, in a state in which the pulse of the projection light is not emitted, the first charge distribution mechanism is driven at the timing of the first acquisition achievement period to generate a third analog signal. generating a signal and driving the second charge distribution mechanism at the timing of the second acquisition achievement period to generate a fourth analog signal;
independently reading out the first to fourth analog signals from the plurality of pixels arranged in a specific row of the matrix and converting them into first to fourth digital data;
discriminating the first to fourth digital data from each other and outputting the same;
storing the discriminated first to fourth digital data independently in first to fourth memories;
A 3D image generation method, comprising the step of inputting data read from the first to fourth memories and calculating a distance to the target object.

Images