JP2021082940A - Imaging device and imaging apparatus - Google Patents

Abstract

To provide an imaging device and an imaging apparatus that can reduce work and cost of manufacturing in performing pixel shifting processing among imaging bodies, and that can reduce size of the imaging apparatus.SOLUTION: An imaging apparatus comprises: a light receiving section (photoelectric conversion section) 12 that generates and stores charges according to incident light; and a signal processing section 13 that reads the stored charges obtained by the light receiving section 12 to output. The light receiving section 12 has two light receiving layers 12a, b laminated in a light incident direction, each of the light receiving layers 12a, b has pixels arranged in a two-dimensional array in the same pitch, and these two layers light receiving layers 12a, b are laminated in each of a longitudinal direction and a lateral direction of a pixel array by shifting 1/2 pixel pitch to each other. Predetermined signal processing is performed in each of signal processing sections 13a, b for an image signal that is received and photoelectric-converted in each of light receiving layers 12a, b, pixel interpolation operation is performed to each other in an image signal interpolation operation section 50 to be synthesized, and a high resolution image signal is generated.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup device and an image pickup device used for imaging various objects to be imaged, for example, an image pickup device and an image pickup device having a large number of pixels that can be adapted to super high-definition and integral photography (IP). is there.

In recent years, there has been a remarkable demand for higher resolution of the image sensor, and so far, an image sensor having 33 million pixels (7,680 × 4,320 pixels) corresponding to, for example, Super Hi-Vision has been realized. However, in the future, further higher resolution is desired, and a dramatic improvement in the level is strongly desired.

As described above, as a technique for obtaining an image having the same resolution as an image obtained by an image sensor having more pixels by using an image sensor having a large number of pixels, for example, a four-plate image pickup method (dual green method) is called. The technology to be used is known.
This technology uses, for example, a 4-split Phillips-type prism, and uses an image sensor for receiving a green signal (G channel) on each of the two emission end faces of the four light emission ends, and uses the other two light emission ends. Of these, an image sensor for receiving a red signal (R channel) is attached to one of the emission end faces, and an image sensor for receiving a blue signal (B channel) is attached to the other emission end face (non-described later). See Patent Document 1).

In this case, the image sensors for the two G channels are adjusted in position so as to be spatially shifted by half a pixel in an oblique direction with respect to the optical image, and attached to the light emitting end face of the prism, and each image sensor is attached. By mutually interpolating the images obtained in the above, the resolution can be doubled the resolution of each image sensor.
In this way, the resolution of the system can be effectively improved with an image sensor having a small number of pixels.

That is, in the conventional technique of the four-plate image pickup method, the two image pickup elements for the G channel attached to the end face of the prism are spatially shifted by 1/2 pixel in the oblique direction with respect to the optical image. By attaching to different prism end faces and performing pixel shifting processing, it is possible to double the resolution of the image sensor in the horizontal and vertical directions (see Non-Patent Document 1).

M. Sugawara et al .: “Four-chip CCD Camera for HDTV,” SPE Proc., 2137, pp.122-129 (1994)

However, according to the above-mentioned conventional technique, if the accuracy of the pixel shift position is poor, the spatial resolution of the composite image also deteriorates. Therefore, it is necessary to attach the image sensor and the prism with high accuracy, which requires labor and cost for manufacturing. There was a problem. Further, since it is necessary to use a prism made of glass and to arrange an image pickup element at each light emitting end of the prism, there is also a problem that the image pickup device becomes large.

The present invention has been made in view of such circumstances, and when an image sensor having a predetermined number of pixels is used to obtain an image having a resolution equivalent to that of an image sensor having more pixels than this, the present invention is made. It is an object of the present invention to provide an image pickup device and an image pickup device capable of reducing the manufacturing labor and cost when performing pixel shift processing between image pickup elements and making the image pickup device compact.

In order to achieve the above object, the image pickup device of the present invention is used.
It is provided with a photoelectric conversion unit that generates and stores electric charges according to incident light and a signal processing unit that reads out and outputs the accumulated charges obtained by the photoelectric conversion unit.
The photoelectric conversion unit is formed by stacking photoelectric conversion layers composed of predetermined n layers (n is a natural number of 2 or more) in the light incident direction, and each of these predetermined n layers is two-dimensional at the same pixel pitch. The pixels are arranged in an array, and at least a layer other than the layer in which the incident light is last incident is formed so as to be able to transmit a part of the light among the predetermined n layers.
The predetermined n layers are arranged so that the vertical and horizontal directions of the pixel arrangement are aligned with each other, and the h / n pixel pitch and the k / n pixel pitch are staggered from each other in the vertical and horizontal directions. It is characterized by.
However, the h and the k are integers of 1 or more.

Further, the photoelectric conversion layer is two layers, and these two layers can be arranged so as to be arranged so as to be shifted by 1/2 pixel pitch from each other in each of the vertical and horizontal directions.
Further, the photoelectric conversion unit and the signal processing unit may be arranged side by side in the direction perpendicular to the light incident direction.
Further, it is preferable that each of the predetermined n layers is configured to absorb the same amount of light as each other.

Further, the imaging device of the present invention
Equipped with any of the above-mentioned image sensors
An image pickup lens that forms an image of an imaged object image information carried by the incident light on the image pickup element, and
It is characterized by including an image signal interpolation calculation unit that performs an operation so that image signals from the signal processing unit are interpolated with each other, corresponding to each of the predetermined n layers of the photoelectric conversion unit.

In the image pickup device and the image pickup apparatus of the present invention, the photoelectric conversion unit is formed by stacking photoelectric conversion layers composed of a plurality of predetermined layers in the light incident direction, and each of the predetermined plurality of layers has the same pixel pitch. Pixels are arranged in a two-dimensional array, and among the predetermined plurality of layers, the layer on the light incident side is configured to photoelectrically convert a part of the incident light and transmit the remaining light to the lower layer. Therefore, a part of the light that reaches the image pickup element is photoelectrically converted in the upper layer (the layer on the light incident side) of the predetermined plurality of layers of the photoelectric conversion layer, and the rest is photoelectrically converted in the lower layer (the light incident side). Is photoelectrically converted in the opposite layer), and so on, the light absorbed in each photoelectric conversion layer is photoelectrically converted.

Then, in these predetermined plurality of layers, the vertical and horizontal directions of the pixel arrangement are aligned, and in each of the vertical and horizontal directions, h / n pixel pitch and k / n pixel pitch are provided (n is the number of photoelectric conversion layers, respectively. Since h and k are arranged in a staggered manner (an integer of 1 or more), the image signals obtained by photoelectric conversion between the layers of the predetermined plurality of layers are interpolated between the pixels. After that, by performing pixel interpolation processing on these image signals, a high-resolution image can be obtained.

Therefore, according to the image pickup device and the image pickup apparatus of the present invention, predetermined plurality of layers (n layers) of photoelectric conversion layers stacked on each other are arranged in each of the vertical and horizontal directions by h / n pixel pitch and k / n pixels. By interpolating by shifting the pitch by pitch, it is possible to improve the resolution of the acquired image without using a large prism as in the prior art and arranging the image sensor on each of the prism light emitting end faces. As a result, it is possible to reduce the labor and cost required for manufacturing when performing the pixel shift processing between the image pickup devices, and to make the image pickup device compact.

It is a schematic diagram which shows the schematic structure of the image pickup device and the image pickup apparatus which concerns on embodiment of this invention. It is a conceptual diagram which shows the state of the pixel shift of the light-receiving layer of two layers in the photoelectric conversion part of the image pickup device which concerns on the embodiment shown in FIG. It is the schematic which shows the positional relationship between the image pickup element and the image pickup lens in the image pickup apparatus which concerns on embodiment shown in FIG.

Hereinafter, the image pickup device according to the embodiment of the present invention will be conceptually described.
That is, the configuration of the image pickup device 10 according to the present embodiment is as follows.
First, as shown in FIG. 1, a light receiving unit (photoelectric conversion unit) 12 that generates and stores electric charges according to incident light and a signal processing unit 13 that reads out and outputs the stored charges obtained by the light receiving unit 12 A prepared configuration is assumed. The light receiving portion 12 is generally formed by laminating a light receiving layer (photoelectric conversion layer) on a support substrate (silicon wafer: not shown).

Further, the light receiving unit 12 is formed by stacking a plurality of light receiving layers 12a and b in the light incident direction, and each of the light receiving layers 12a and b has a configuration in which pixels are arranged in a vertical and horizontal array.
Further, the light receiving layer (upper layer) 12a arranged on the light incident side and the light receiving layer (lower layer) 12b arranged on the side opposite to the light incident side have pixel arrangements in the vertical direction (Y-axis direction) and the horizontal direction. In addition to being aligned in all of the (X-axis directions), they are arranged so as to overlap each other in a positional relationship shifted from each other by a 1/2 pixel pitch in both of these two directions.

Further, the signal processing unit 13 is arranged adjacent to each of the light receiving layers 12a and 12 in the lateral direction (in the direction perpendicular to the incident direction, side by side). That is, an upper layer signal processing unit 13a is provided corresponding to the light receiving layer (upper layer) 12a, and a lower layer signal processing unit 13b is provided corresponding to the light receiving layer (lower layer) 12b. The pixel signals from the pixels 12a1 to a4 are sent to the upper layer signal processing unit 13a via the signal line 14, while the pixel signals from the pixels 12b1 to b5 of the light receiving layer 12b are sent to the signal line 14 respectively. Is sent to the lower layer signal processing unit 13b via the above.

The image sensor 10 in which the light receiving layers 12a and 12b are laminated and the electrical wiring is connected to the signal processing unit is housed in a so-called IC package. At this time, each connection terminal of the image sensor is connected to the terminal on the IC package side by wire bonding.

Further, the image pickup device 100 according to the embodiment of the present invention includes the above-mentioned image pickup element 10, the image signal interpolation calculation unit 50, and the image pickup lens 30 shown in FIG.
The image signal interpolation calculation unit 50 shown in FIG. 1 inputs an image signal output from the upper layer signal processing unit 13a and an image signal output from the lower layer signal processing unit 13b, and interpolates both image signals with each other. Performs arithmetic processing to generate and output a high-resolution image signal.

Further, the image pickup lens 30 has a convex lens function capable of forming an image of an image to be imaged carried by a light flux incident on the image pickup apparatus 100 on the image pickup device 10. The image pickup lens 30 may be an aspherical single lens or a composite lens system including a plurality of lenses.

FIG. 2 shows a state in which the light receiving layer (upper layer) 12a and the light receiving layer (lower layer) 12b constituting the light receiving unit 12 are arranged so as to be shifted by 1/2 pixel pitch from each other in the vertical and horizontal pixel arrangement directions. It is shown roughly. That is, the corresponding pixels 12an and bn of both the light receiving layers 12a and b are shifted by 1/2 pixel pitch from each other in both the vertical and horizontal directions.
As a result, the gaps between the pixels are interpolated with each other, so that a high-resolution image can be obtained.

The plurality of light receiving layers 12a and b are laminated on a support substrate (not shown) without using an adhesive or the like, and are referred to as light receiving layers 12a and b and signal processing units 13a and b (hereinafter referred to as light receiving layers 12a and b). It can be easily formed by using a technique of a technique (a technique including so-called direct bonding, hybrid bonding, and room temperature bonding) in which image pickup substrate portions on which (sometimes) are mounted are directly bonded to each other.

Further, when joining the imaging substrate portions, an alignment mark is formed at a corresponding position of each imaging substrate portion on which the light receiving layers 12a and b are mounted, and the alignment marks are aligned with each other to match the imaging substrate portions. The positioning of the overlap when joining the portions can be easily improved. Therefore, the amount of pixel misalignment can be easily adjusted by shifting the positions of the two light receiving layers by 1/2 pixel pitch from each other in the vertical and horizontal directions with respect to the alignment mark.

The alignment marks are formed at the corresponding diagonal corners of the four corners of each imaging substrate, for example, and when viewed from above by an infrared camera, the alignment marks (for example, one of which is an annular mark) are formed. The other side is loosely fitted in the ring (it is a circular mark having a size to be fitted with a margin), but the alignment is performed while confirming that they overlap in a predetermined state.

When each image pickup board portion on which the light receiving layers 12a, b, etc. are mounted is laminated on the support substrate, each image pickup substrate portion having the alignment mark formed is placed on the outside of each support substrate, and the light receiving layers are connected to each other. Are opposed to each other and joined. After this, one support substrate is removed.

The materials used for the light receiving layers 12a and 12b are not only photodiodes and embedded photodiodes using silicon, which are the same materials as those used for commonly used CMOS image sensors, but also silicon substrates and transistor layers. A photodiode formed of an SOI substrate or an FDSOI substrate having an embedded oxide film between the two can be used. In particular, it is known that the FDSOI substrate has a thin transistor layer of about 50 nm. By using this, a part of the light, which is a feature of this embodiment, is received and photoelectrically converted, and the remaining light is the lower layer. It is possible to realize a method of transmitting light to.

Further, as a material for the photoelectric conversion layer other than silicon, selenium or an organic photoelectric conversion material previously disclosed to the JPO by the applicant of the present application can also be applied. In particular, when an organic photoelectric conversion material is used, organic photoelectric conversion layers having sensitivity to red, blue, and green light can be laminated, and the corresponding color light can be efficiently spectrally absorbed in each layer.
As the material of the signal processing units 13a and 13b, a transparent material can be used.

Although FIG. 2 schematically shows how three pixels are arranged in each of the vertical and horizontal directions, in reality, there are thousands of pixels in the vertical and horizontal directions (in the case of Super Hi-Vision, the width is 7). , 680 pixels x 4,320 pixels vertically) are arranged.

Further, FIG. 3 shows a state in which the subject image information carried by the light passing through the image pickup lens 30 is formed in the image pickup apparatus 100. A part of the light transmitted through the image pickup lens 30 and incident on the image pickup element 10 is received by the light receiving layer (upper layer) 12a, and the remaining light is received by the light receiving layer (lower layer) 12b.

It is preferable that the light receiving amounts of these two light receiving layers 12a and b are set to be substantially equal. For example, if the light transmittance of the light receiving layer 12a is 50% and the light transmittance of the light receiving layer 12b is 0%, they are the same. A certain amount of light can be efficiently distributed to the two light receiving layers 12a and b. For this purpose, it is important to set the thickness of the light receiving layer 12b to be significantly larger than the thickness of the light receiving layer 12a.

Further, for example, if the light transmittance of the light receiving layer 12a is 2/3 and the light transmittance of the light receiving layer 12b is 1/2, the amount of light subjected to photoelectric conversion in each of the light receiving layers 12a and b is about the same. Can be. The light transmittance of each of the light receiving layers 12a and 12b can be adjusted on the assumption that it is inversely proportional to the layer thickness.

The image pickup device and the image pickup device of the present invention are not limited to those of the above-described embodiment, and can be changed to various other aspects.
For example, in the above embodiment, the case where pixel interpolation is performed between two light receiving layers is described, but three or more light receiving layers may be laminated.

Generally, when n layers of light receiving layers are laminated so that pixel interpolation can be performed on each other, the light receiving layers are arranged vertically and horizontally by h / n pixel pitch and k / n pixel pitch (k / n pixel pitch). However, it is effective to perform pixel interpolation by configuring the h and the k to be arranged so as to be staggered from each other (the h and the k are integers of 1 or more).

That is, for example, when there are three light receiving layers, when pixel interpolation is performed between these light receiving layers, the three laminated light receiving layers are placed in both the vertical and horizontal directions with a 1/3 pixel pitch of each other, or Try to shift each other by 2/3 pixel pitch. As a result, three times the resolution can be obtained.

Further, in the above embodiment, the case where the present invention is used for a normal image sensor is described. For example, in the specification of Japanese Patent Application No. 2019-40858 already disclosed by the applicant to the Japan Patent Office. It is also possible to adapt to integral photography (IP) as described.

Further, four light receiving layers made of the above-mentioned organic photoelectric conversion material are laminated, and two light receiving layers thereof are used as a light receiving layer for receiving a green signal (G channel), and among the other two light receiving layers, One is a light receiving layer for receiving a red signal (R channel), the other is a light receiving layer for receiving a blue signal (B channel), and the two light receiving layers for receiving a green signal (G channel) are described above. By adopting the method of shifting the pitch by 1/2 pixel as described in the above-described embodiment, a high-quality image similar to that of the 4-panel imaging method (dual green method) can be obtained in an extremely compact and easy-to-manufacture manner. It is possible to do.

Further, when the light receiving layer is made of the organic photoelectric conversion material, the same method as that for G light can be adopted for other colored lights of R light and B light, and the combination thereof is appropriately selected. When trying to obtain twice the resolution for each of the R, G, and B colored lights, a total of six light receiving layers are laminated.
Further, if the photoelectric conversion unit and the signal processing unit are configured to be laminated on each other, the lateral spread of the element can be suppressed, so that the image pickup device can be further made compact.

10 Image sensor 12 Light receiving part (photoelectric conversion part)
12a Light receiving layer (upper layer)
12b Light receiving layer (lower layer)
12a1 to a4, 12an, 12b1 to b5, 12bn pixels 13 Signal processing unit 13a Upper layer signal processing unit 13b Lower layer signal processing unit 30 Imaging lens 50 Image signal interpolation calculation unit 100 Imaging device

Claims (5)

It is provided with a photoelectric conversion unit that generates and stores electric charges according to incident light and a signal processing unit that reads out and outputs the accumulated charges obtained by the photoelectric conversion unit.
The photoelectric conversion unit is formed by stacking photoelectric conversion layers composed of predetermined n layers (n is a natural number of 2 or more) in the light incident direction, and each of these predetermined n layers is two-dimensional at the same pixel pitch. The pixels are arranged in an array, and at least a layer other than the layer in which the incident light is last incident is formed so as to be able to transmit a part of the light among the predetermined n layers.
The predetermined n layers are arranged so that the vertical and horizontal directions of the pixel arrangement are aligned with each other, and the h / n pixel pitch and the k / n pixel pitch are staggered from each other in the vertical and horizontal directions. An image sensor characterized by.
However, the h and the k are integers of 1 or more. The image pickup device according to claim 1, wherein the photoelectric conversion layer is two layers, and these two layers are arranged so as to be arranged so as to be arranged so as to be arranged so as to be shifted by 1/2 pixel pitch from each other in the vertical and horizontal directions. The image pickup device according to claim 1 or 2, wherein the photoelectric conversion unit and the signal processing unit are arranged side by side in a direction perpendicular to a light incident direction. The image pickup device according to any one of claims 1 to 3, wherein each of the predetermined n layers is configured to absorb the same amount of light as each other. The image sensor according to any one of claims 1 to 4 is provided.
An image pickup lens that forms an image of an imaged object image information carried by the incident light on the image pickup element, and
An image pickup apparatus including an image signal interpolation calculation unit that performs an operation so as to interpolate image signals from the signal processing unit, corresponding to each of the predetermined n layers of the photoelectric conversion unit.

Images