TW202335486A - Solid-state imaging device and electronic apparatus - Google Patents

Abstract

A solid-state imaging device according to an embodiment comprises a pixel array unit in which a plurality of pixels, including a plurality of first pixels each performing photoelectric conversion of light of a first wavelength component, are arrayed in a matrix, each of the pixels including: a photoelectric conversion unit that performs photoelectric conversion of entry light; a transfer transistor that controls the transfer of charge generated in the photoelectric conversion unit; a floating diffusion region in which the charge transferred from the photoelectric conversion unit via the transfer transistor is stored; and an amplifying transistor that causes a voltage signal corresponding to the charge stored in the floating diffusion region to appear in a signal line. The plurality of first pixels are arrayed in a first diagonal direction in the pixel array unit. Of the plurality of first pixels arrayed in the first diagonal direction, at least two share one floating diffusion region.

Description

Solid-state imaging devices and electronic equipment

The present disclosure relates to a solid-state imaging device and electronic equipment.

In recent years, there have been image sensors (also known as image sensors) in which a plurality of pixels arranged in a matrix receive the wavelength components of each of the three primary colors of light: red (R), green (G), and blue (B) and generate a color image. called a solid-state imaging device). In such an image sensor that can capture color images, generally speaking, red (R), green (G), and blue (B) wavelength components are selected by repeatedly arranging 4 pixel units of 2×2. The so-called Bayer array color filter is a color filter that transmits light naturally. In addition, in recent years, various color filter arrangements have appeared, such as a so-called quadruple Bayer array color filter in which each color filter constituting the Bayer array is further divided into 2×2/4. [Prior technical literature] [Patent Document]

[Patent Document 1] Japanese Patent Application Publication No. 2010-193200 [Patent Document 2] Japanese Patent Application Publication No. 2013-143730 [Patent Document 3] Japanese Patent Application Publication No. 2016-9872

[Problem to be solved by the invention]

In addition, in recent years, the industry has developed image sensors equipped with a so-called merge mode that changes the resolution. However, in previous image sensors, when the resolution is changed, there is a possibility of image quality deterioration. For example, in an image sensor that uses a Bayer arrangement for color filters, the image quality is high when imaging is performed in full-pixel mode without binning, but when the resolution is lowered due to binning, it may Alias, etc. are generated, and the image quality is reduced beyond the reduction in resolution. Also, for example, in an image sensor using a quadruple Bayer arrangement for color filters, the image quality is high when reading out at low resolution, but the image quality when imaging is performed in full-pixel mode is sometimes poor. There is a possibility of image quality degradation when Bayer arrangement is used.

Therefore, this disclosure proposes a solid-state imaging device and electronic equipment that can suppress degradation of image quality. [Technical means to solve problems]

In order to solve the above-mentioned problems, a solid-state imaging device according to one aspect of the present disclosure includes a pixel array section composed of a plurality of pixels arranged in a matrix. The plurality of pixels each include a plurality of photoelectric converters that photoelectrically convert light of the first wavelength component. The first pixel; and the aforementioned pixels respectively include: a photoelectric conversion portion that photoelectrically converts incident light; a transfer transistor that controls the transfer of charges generated in the aforementioned photoelectric conversion portion; and a floating diffusion region that accumulates from the aforementioned photoelectric conversion portion The aforementioned charge is transferred via the aforementioned transfer transistor; and an amplification transistor causes a voltage signal corresponding to the aforementioned charge accumulated in the aforementioned floating diffusion region to appear on a signal line; and the aforementioned plurality of first pixels are arranged in the aforementioned pixel array portion. In the first oblique direction; at least two first pixels among the plurality of first pixels arranged in the first oblique direction share one floating diffusion area.

Hereinafter, one embodiment of the present disclosure will be described in detail based on the drawings. In addition, in the following embodiments, the same parts are assigned the same reference numerals, and repeated descriptions are omitted.

In addition, this disclosure will be explained in accordance with the order of items shown below. 1. First embodiment 1.1 Example of the structure of electronic equipment (camera device) 1.2 Structure example of image sensor 1.3 Example of pixel composition 1.3.1 Example of pixel configuration with FD common structure 1.4 Basic function example of unit pixel 1.5 Example of multilayer structure of image sensor 1.6 Basic structure example of pixel 1.7 Reasons for reduced image quality 1.8 Suppression of image quality degradation 1.9 Pixel arrangement rearrangement mosaic 1.10 Correction of characteristic differences between pixels 1.11 Application of global shutter method 1.12 Application to another color filter arrangement 1.13 Summary 2. Second embodiment 2.1 Example of color filter arrangement and pixel common structure 2.2 Variation examples of color filter arrangement and pixel common structure 2.2.1 The first variation 2.2.2 Second variation 2.2.3 The third variation 3. Third embodiment 3.1 Variation examples 4. Application examples for smartphones 5. Application examples for moving objects 6. Application examples of endoscopic surgery system

1. First embodiment First, the first embodiment of the present disclosure will be described in detail with reference to the drawings. In addition, in this embodiment, the case where the technology of this embodiment is applied to a CMOS (Complementary Metal-Oxide-Semiconductor) type solid-state imaging device (hereinafter also referred to as an image sensor) is exemplified. However, It is not limited to this. For example, it can be a CCD (Charge Coupled Device) type image sensor or a TOF (Time of Flight) sensor or an EVS (Event-based Vision Sensor). The technology of this embodiment is applied to various sensors equipped with photoelectric conversion elements, such as detectors.

1.1 Example of the structure of electronic equipment (camera device) FIG. 1 is a block diagram showing a schematic configuration example of an electronic device (camera device) equipped with the image sensor according to the first embodiment. As shown in FIG. 1 , the imaging device 1 includes, for example, an imaging lens 11 , an image sensor 10 , a memory unit 14 , and a processor 13 .

The imaging lens 11 is an example of an optical system that collects incident light and forms the image on the light-receiving surface of the image sensor 10 . The light-receiving surface may be a surface of the image sensor 10 on which the photoelectric conversion elements are arranged. The image sensor 10 photoelectrically converts the incident light and generates image data. In addition, the image sensor 10 performs specific signal processing such as noise removal or white balance adjustment on the generated image data.

The memory unit 14 is composed of, for example, flash memory, DRAM (Dynamic Random Access Memory, dynamic random access memory), or SRAM (Static Random Access Memory, static random access memory), etc., and records data from the image sensor 10 Input image data, etc.

The processor 13 is composed of a CPU (Central Processing Unit), for example, and may include an application processor that executes an operating system or various application software, a GPU (Graphics Processing Unit), or baseband processing. Devices etc. The processor 13 performs various processing as needed on the image data input from the image sensor 10 or the image data read from the memory unit 14, or performs display to the user, or performs display to the user through a specific network. Send externally.

1.2 Structure example of image sensor FIG. 2 is a block diagram showing an example of the schematic configuration of the CMOS image sensor according to the first embodiment. Here, the CMOS image sensor is an image sensor produced using a CMOS process or partially using it. For example, the image sensor 10 of this embodiment is composed of a back-illuminated image sensor.

The image sensor 10 of this embodiment has, for example, a stacked structure in which a light-receiving chip 41 (substrate) on which the pixel array portion 21 is arranged and a circuit chip 42 (substrate) on which peripheral circuits are arranged are laminated (for example, see FIG. 5 ). . The peripheral circuit may include, for example, a vertical drive circuit 22, a row processing circuit 23, a horizontal drive circuit 24, and a system control unit 25.

The image sensor 10 further includes a signal processing unit 26 and a data storage unit 27 . The signal processing unit 26 and the data storage unit 27 may be provided on the same semiconductor chip as the peripheral circuit, or may be provided on separate semiconductor chips.

The pixel array section 21 has a structure in which pixels 30 having photoelectric conversion elements that generate and accumulate charges corresponding to the amount of light received are arranged in a two-dimensional grid in the column direction and the row direction, that is, in a matrix form. Here, the column direction means the arrangement direction of the pixels in the pixel column (horizontal direction in the figure), and the row direction means the arrangement direction of the pixels in the pixel row (the longitudinal direction in the figure). The specific circuit structure of the pixel 30 and the details of the pixel structure will be described later.

In the pixel array section 21, for a matrix-like pixel arrangement, the pixel driving lines LD are wired along the column direction for each pixel column, and the vertical signal lines VSL are wired along the row direction for each pixel row. The pixel driving line LD transmits a driving signal for driving when reading a signal from the pixel. In FIG. 2 , the pixel driving lines LD are shown as one wiring line, but are not limited to one wiring line. One end of the pixel driving line LD is connected to an output terminal corresponding to each column of the vertical driving circuit 22 .

The vertical driving circuit 22 is composed of a shift register, an address decoder, etc., and all pixels drive each pixel of the pixel array portion 21 simultaneously or in column units. That is, the vertical driving circuit 22 and the system control unit 25 that controls the vertical driving circuit 22 together form a driving unit that controls the operation of each pixel of the pixel array unit 21 . Although the specific structure of the vertical drive circuit 22 is not shown in the figure, generally speaking, it has two scanning systems: a read scanning system and an exclusion scanning system.

In order to read signals from the pixels 30 , the readout scanning system sequentially selects and scans the pixels 30 of the pixel array portion 21 in column units. The signal read from the pixel 30 is an analog signal. The exclusion scanning system performs exclusion scanning on the readout column that is readout by the readout scanning system by an exposure time portion earlier than the readout scan.

Through the elimination scanning performed by the elimination scanning system, unnecessary charges are eliminated from the photoelectric conversion elements of the pixels 30 in the readout row, thereby resetting the photoelectric conversion elements. Furthermore, by eliminating (resetting) unnecessary charges with the elimination scanning system, a so-called electronic shutter operation is performed. Here, the electronic shutter operation means an operation of discarding the photocharge of the electrical conversion element and restarting the exposure (that is, starting the accumulation of electric charge).

The signal read out by the readout operation performed by the readout scanning system corresponds to the amount of light received after the immediately preceding readout operation or the electronic shutter operation. Furthermore, the period from the readout timing of the immediately preceding readout operation or the exclusion timing of the electronic shutter operation to the readout timing of the current readout operation becomes the charge accumulation period of the pixel 30 (also referred to as the exposure period).

The signal output from each pixel 30 of the pixel column selected for scanning by the vertical drive circuit 22 is input to the row processing circuit 23 via the vertical signal line VSL for each pixel column. The row processing circuit 23 performs specific signal processing on the signal output from each pixel in the selected column through the vertical signal line VSL for each pixel row of the pixel array section 21, and temporarily retains the signal-processed pixel signal.

Specifically, the line processing circuit 23 performs at least noise removal processing such as CDS (Correlated Double Sampling) processing or DDS (Double Data Sampling) processing as signal processing. For example, CDS removal processing is used to remove fixed pattern noise inherent to the pixel, such as reset noise and threshold value deviations of the amplification transistors in the pixel. In addition, the row processing circuit 23 also has, for example, an AD (analog-to-digital) conversion function to convert an analog pixel signal that can be read from a photoelectric conversion element into a digital signal and output it.

The horizontal driving circuit 24 is composed of a shift register, an address decoder, etc., and sequentially selects the readout circuit (hereinafter also referred to as a pixel circuit) corresponding to the pixel row of the row processing circuit 23. Through the selective scanning performed by the horizontal driving circuit 24, the pixel signals processed by each pixel circuit in the row processing circuit 23 are sequentially output.

The system control unit 25 is composed of a timing generator that generates various timing signals, and performs drive control of the vertical drive circuit 22 , the row processing circuit 23 , the horizontal drive circuit 24 , etc. based on the various timings generated by the timing generator.

The signal processing unit 26 at least has an arithmetic processing function, and performs various signal processing such as arithmetic processing on the pixel signal output from the self-processing circuit 23 . The data storage unit 27 temporarily stores data required for signal processing in the signal processing unit 26 .

In addition, the image data output from the signal processing unit 26 may, for example, perform specific processing in the processor 13 of the camera device 1 equipped with the image sensor 10 or the like, or may be sent to the outside via a specific network.

1.3 Example of pixel composition FIG. 3 is a circuit diagram showing an example of a schematic configuration of a pixel in this embodiment. As shown in FIG. 3 , the pixel 30 includes, for example, a photoelectric conversion part PD, a transfer transistor 31 , a floating diffusion region FD, a reset transistor 32 , an amplification transistor 33 and a selection transistor 34 .

In this description, the reset transistor 32, the amplification transistor 33 and the selection transistor 34 are also collectively referred to as a pixel circuit. The pixel circuit may include at least one of the floating diffusion region FD and the transfer transistor 31 .

The photoelectric conversion part PD photoelectrically converts the incident light. The transfer transistor 31 transfers charges generated in the photoelectric conversion part PD. The floating diffusion region FD accumulates charges transferred by the transfer transistor 31 . The amplifying transistor 33 causes a pixel signal of a voltage corresponding to the charge accumulated in the floating diffusion region FD to appear on the vertical signal line VSL. The reset transistor 32 is suitable for discharging the charges accumulated in the floating diffusion region FD and the photoelectric conversion part PD. The selection transistor 34 selects the pixel 30 to be read out.

The anode of the photoelectric conversion part PD is grounded, and the cathode is connected to the source of the transfer transistor 31 . The connection node between the drain electrode of the transfer transistor 31 and the source electrode of the reset transistor 32 and the gate electrode of the amplification transistor 33 form a floating diffusion region FD.

The reset transistor 32 is connected between the floating diffusion region FD and the vertical reset input line VRD. The drain of the reset transistor 32 is connected to the vertical reset input line VRD, and the source of the amplifying transistor 33 is connected to the vertical current supply line VCOM. The drain of the amplification transistor 33 is connected to the source of the selection transistor 34 . The drain of the selection transistor 34 is connected to the vertical signal line VSL.

The gate of the transmission transistor 31 is connected via the transmission transistor drive line LD31, the gate of the reset transistor 32 is via the reset transistor drive line LD32, and the gate of the selection transistor 34 is connected via the selection transistor drive line LD34. Pulse signals as drive signals are supplied to the vertical drive circuits 22 respectively.

In such a structure, the potential of the capacitor formed by the floating diffusion region FD is determined by the electric charge accumulated therein and the capacitance of the floating diffusion region FD. In addition to the ground capacitance, the capacitance of the floating diffusion region FD is also determined by the diffusion layer capacitance of the drain of the transfer transistor 31 , the source diffusion layer capacitance of the reset transistor 32 , the gate capacitance of the amplification transistor 33 , etc.

1.3.1 Example of pixel configuration with FD common structure FIG. 4 is a circuit diagram showing an example of the schematic configuration of a pixel having a common structure of FD (Floating Diffusion) in this embodiment. As shown in FIG. 4 , the pixel 30A has the same structure as the pixel 30 described above using FIG. 3 . It is provided with one floating diffusion region FD and is connected to each other through respective transfer transistors 31 - 1 - 31 - 4 . The structure of a plurality of (four in this example) photoelectric conversion parts PD1 to PD4. In addition, a pixel circuit common to the pixels 30A sharing the floating diffusion area FD is connected to the floating diffusion area FD.

In such a configuration, the transfer transistors 31L and 31R are respectively connected to different transfer transistor driving lines LD31L and LD31R at their gates, and are configured to be driven independently.

1.4 Basic function example of unit pixel Next, the basic functions of the pixel 30 are explained. The reset transistor 32 turns on/off the discharge of electric charge accumulated in the floating diffusion region FD in accordance with the reset signal RST supplied from the vertical drive circuit 22 . At this time, by turning on the transfer transistor 31, the charge accumulated in the photoelectric conversion part PD can also be discharged.

When a high-level reset signal RST is input to the gate of the reset transistor 32 , the floating diffusion region FD is clamped to the voltage applied through the vertical reset input line VRD. Thereby, the charges accumulated in the floating diffusion region FD are discharged (reset). At this time, by inputting the high-level transmission signal TRG to the gate of the transmission transistor 31, the charge accumulated in the photoelectric conversion part PD is also discharged (reset).

In addition, when a low-level reset signal RST is input to the gate of the reset transistor 32 , the floating diffusion region FD is electrically disconnected from the vertical reset input line VRD and becomes a floating state.

The photoelectric conversion part PD photoelectrically converts incident light and generates charges corresponding to the amount of light. The generated charge is accumulated on the cathode side of the photoelectric conversion part PD. The transfer transistor 31 turns on/off the transfer of charges from the photoelectric conversion part PD to the floating diffusion region FD in accordance with the transfer control signal TRG supplied from the vertical drive circuit 22 . For example, when a high-level transfer control signal TRG is input to the gate of the transfer transistor 31, the charges accumulated in the photoelectric conversion part PD are transferred to the floating diffusion region FD. On the other hand, when the low-level transfer control signal TRG is supplied to the gate of the transfer transistor 31, the transfer of charges from the photoelectric conversion part PD is stopped. In addition, while the transfer transistor 31 stops the transfer of charges to the floating diffusion region FD, the charges resulting from photoelectric conversion are accumulated in the photoelectric conversion part PD.

The floating diffusion region FD has a function of accumulating charges transferred from the photoelectric conversion part PD via the transfer transistor 31 and converting them into voltages. Therefore, in the floating state in which the reset transistor 32 is turned off, the potential of the floating diffusion region FD is modulated corresponding to the amount of charge accumulated therein.

The amplification transistor 33 functions as an amplifier that uses the potential change of the floating diffusion region FD connected to its gate as an input signal, and the output voltage signal is output to the vertical signal line VSL as a pixel signal via the selection transistor 34 .

The selection transistor 34 turns on/off the voltage signal from the amplification transistor 33 to the output of the vertical signal line VSL in accordance with the selection control signal SEL supplied from the vertical drive circuit 22 . For example, when a high-level selection control signal SEL is input to the gate of the selection transistor 34, the voltage signal from the amplification transistor 33 is output to the vertical signal line VSL. When a low-level selection control signal SEL is input, the voltage signal is stopped. The signal is output to the vertical signal line VSL. Thereby, in the vertical signal line VSL connected to a plurality of pixels, only the output of the selected pixel 30 can be taken out.

In this way, the pixel 30 is driven according to the transfer control signal TRG, the reset signal RST, the switching control signal FDG, and the selection control signal SEL supplied from the vertical driving circuit 22 .

1.5 Example of multilayer structure of image sensor FIG. 5 is a diagram showing an example of a multilayer structure of the image sensor according to this embodiment. As shown in FIG. 5 , the image sensor 10 has a structure in which a light-receiving chip 41 and a circuit chip 42 are stacked up and down. The light-receiving wafer 41 has a structure in which the light-receiving wafer 41 and the circuit chip 42 are laminated. The light-receiving wafer 41 is, for example, a semiconductor chip including the pixel array portion 21 in which the photoelectric conversion portions PD are arranged. The circuit chip 42 is, for example, a semiconductor wafer in which pixel circuits are arranged.

For the bonding of the light-receiving chip 41 and the circuit chip 42, for example, so-called direct bonding in which the bonding surfaces of the two are planarized and the two are bonded together by electron force can be used. However, it is not limited to this. For example, so-called copper-copper (Cu-Cu) bonding in which electrode pads made of copper (Cu) formed on mutual bonding surfaces are bonded to each other, or other bump bonding can also be used. .

In addition, the light-receiving chip 41 and the circuit chip 42 are electrically connected through a connection portion such as a TSV (Through-Silicon Via), which is a through contact that penetrates the semiconductor substrate. For the connection using TSV, for example, the so-called double TSV method in which the TSV provided on the light-receiving chip 41 and the TSV provided from the light-receiving chip 41 to the circuit chip 42 are connected on the surface of the chip, or the self-light-receiving chip can be used. The TSV 41 is connected to the circuit chip 42, so-called shared TSV method, etc. are used to connect the two.

However, when the light-receiving chip 41 and the circuit chip 42 are bonded using Cu-Cu (copper-copper) bonding or bump bonding, they are electrically connected through the Cu-Cu (copper-copper) bonding portion or the bump bonding portion. connection.

1.6 Basic structure example of pixel Next, a basic structural example of the pixel in the first embodiment will be described with reference to FIG. 6 and the pixel 30 illustrated in FIG. 3 . In addition, the basic structural example of the pixel 30A illustrated in FIG. 4 may be the same. FIG. 6 is a cross-sectional view showing an example of the basic cross-sectional structure of the pixel in the first embodiment. In addition, an example of the cross-sectional structure of the light-receiving chip 41 provided with the photoelectric conversion part PD of the pixel 30 is shown in FIG. 6 .

As shown in FIG. 6 , in the image sensor 10 , the photoelectric conversion part PD receives the incident light L1 incident from the back surface (upper surface in the figure) side of the semiconductor substrate 58 . A planarizing film 53, a color filter 52, and a crystal-mounted lens 51 are provided above the photoelectric conversion part PD, and the incident light L1 incident from the light-receiving surface 57 into the semiconductor substrate 58 is photoelectrically converted by passing through each part in sequence.

For the semiconductor substrate 58, for example, a semiconductor substrate containing a Group IV semiconductor containing at least one of carbon (C), silicon (Si), germanium (Ge), and tin (Sn), or a semiconductor substrate containing boron (B), A semiconductor substrate of at least two III-V group semiconductors selected from the group consisting of aluminum (A1), gallium (Ga), indium (In), nitrogen (N), phosphorus (P), arsenic (As) and antimony (Sb). However, it is not limited to these, and various semiconductor substrates can be used.

For example, the photoelectric conversion part PD may have a structure in which the N-type semiconductor region 59 is formed as a charge accumulation region that accumulates charges (electrons). In the photoelectric conversion part PD, the N-type semiconductor region 59 is provided in a region surrounded by the P-type semiconductor regions 56 and 64 of the semiconductor substrate 58 . On the front surface (lower surface) side of the semiconductor substrate 58 of the N-type semiconductor region 59, a P-type semiconductor region 64 having a higher impurity concentration than the back surface (upper surface) side is provided. That is, the photoelectric conversion part PD has a HAD (Hole-Accumulation Diode) structure, and P-type semiconductor regions 56 and 64 are provided at each interface between the upper surface side and the lower surface side of the N-type semiconductor region 59 , to suppress the generation of dark current.

A pixel separation part 60 that electrically separates the plurality of pixels 30 is provided inside the semiconductor substrate 58 , and a photoelectric conversion part PD is provided in a region defined by the pixel separation part 60 . In the figure, when the image sensor 10 is viewed from the upper surface side, the pixel separation part 60 is provided in a grid shape, for example, interposed between a plurality of pixels 30 , and the photoelectric conversion part PD is disposed in the pixel separation part 60 . Within the area designated by Section 60.

In each photoelectric conversion part PD, the anode is grounded. In the image sensor 10, the signal charges (for example, electrons) accumulated in the photoelectric conversion part PD are read out through the transfer transistor 31 (not shown) (see FIG. 3) and the like, and It is output as an electrical signal to a vertical signal line VSL (not shown) (see FIG. 3 ).

The wiring layer 65 is provided on the surface (lower surface) of the semiconductor substrate 58 that is opposite to the back surface (upper surface) on which the light-shielding film 54 , the planarizing film 53 , the color filter 52 , the crystal-mounted lens 51 and other components are provided.

The wiring layer 65 is composed of wiring 66, an insulating layer 67, and through-electrodes (not shown). The electrical signal from the light-receiving chip 41 is transmitted to the circuit chip 42 via the wiring 66 and the through-electrode (not shown). Similarly, the substrate potential of the light-receiving chip 41 is also applied from the circuit chip 42 through the wiring 66 and the through-electrode (not shown).

For example, the circuit chip 42 illustrated in FIG. 5 is bonded to the surface of the wiring layer 65 that is opposite to the side where the photoelectric conversion part PD is provided.

The light-shielding film 54 is provided on the back surface (upper surface in the figure) of the semiconductor substrate 58 to block part of the incident light L1 from above the semiconductor substrate 58 toward the back surface of the semiconductor substrate 58 .

The light-shielding film 54 is provided above the pixel separation portion 60 provided inside the semiconductor substrate 58 . Here, the light-shielding film 54 is provided to protrude in a convex shape on the back surface (upper surface) of the semiconductor substrate 58 via an insulating film 55 such as a silicon oxide film. In contrast, the light-shielding film 54 is not provided in an opening above the photoelectric conversion portion PD provided inside the semiconductor substrate 58 so that the incident light L1 is incident on the photoelectric conversion portion PD.

That is, in the figure, when the image sensor 10 is viewed from the upper surface side, the planar shape of the light-shielding film 54 is a grid shape, forming openings for the incident light L1 to pass toward the light-receiving surface 57 .

The light-shielding film 54 is formed of a light-shielding material that blocks light. For example, the light-shielding film 54 is formed by sequentially stacking a titanium (Ti) film and a tungsten (W) film. In addition, the light-shielding film 54 can be formed, for example, by sequentially stacking a titanium nitride (TiN) film and a tungsten (W) film.

The light-shielding film 54 is covered with a planarizing film 53 . The planarizing film 53 is formed using an insulating material that allows light to pass through. Examples of the insulating material include silicon oxide (SiO ₂ ).

The pixel separation part 60 has, for example, a groove part 61, a fixed charge film 62, and an insulating film 63. It is provided on the back surface (upper surface) side of the semiconductor substrate 58 and covers the groove part 61 that divides the plurality of pixels 30.

Specifically, the fixed charge film 62 is provided to cover the inner surface of the groove portion 61 formed on the back surface (upper surface) side of the semiconductor substrate 58 with a certain thickness. Furthermore, the insulating film 63 is provided (filled) so as to be embedded in the groove portion 61 covered with the fixed charge film 62 .

Here, the fixed charge film 62 is formed using a high dielectric material having a negative fixed charge to form a positive charge (hole) accumulation region in the interface portion with the semiconductor substrate 58 to suppress the generation of dark current. The fixed charge film 62 has negative fixed charges, and the negative fixed charges apply an electric field at the interface with the semiconductor substrate 58 to form a positive charge (hole) accumulation region.

The fixed charge film 62 may be formed of, for example, a hafnium oxide film (HfO ₂ film). again. In addition, the fixed charge film 62 may be formed of at least one oxide containing, for example, hafnium, zirconium, aluminum, tantalum, titanium, magnesium, yttrium, lanthanoid series elements, or the like.

In addition, the pixel separation unit 60 is not limited to the above-mentioned configuration, and various changes are possible. For example, by using a reflective film that reflects light, such as a tungsten (W) film, instead of the insulating film 63 , the pixel separation portion 60 can be formed into a light reflective structure. Thereby, since the pixel separation part 60 can reflect the incident light L1 entering the photoelectric conversion part PD, the optical path length of the incident light L1 in the photoelectric conversion part PD can be increased. In addition, by using the pixel separation part 60 as a light reflective structure, the leakage of light to adjacent pixels can be reduced, so that the image quality, distance measurement accuracy, etc. can be further improved. In addition, when a metal material such as tungsten (W) is used as the material of the reflective film, an insulating film such as a silicon oxide film may be provided in the groove portion 61 instead of the fixed charge film 62 .

In addition, the structure of forming the pixel separation part 60 into a light reflective structure is not limited to the use of an irradiation film. For example, a material having a higher refractive index or a lower refractive index than the semiconductor substrate 58 may be embedded in the groove part 61 . to achieve.

Furthermore, FIG. 6 illustrates a so-called RDTI (Reverse Deep Trench Isolation) structure in which a pixel isolation portion 60 is provided in a groove portion 61 formed from the back surface (upper surface) side of the semiconductor substrate 58 The pixel isolation part 60 is not limited thereto. For example, a so-called DTI (Deep Trench Isolation) in which the pixel isolation part 60 is provided in a groove formed from the surface (lower surface) side of the semiconductor substrate 58 may be used. The pixel isolation portion 60 has various structures, such as a so-called FTI (Full Trench Isolation) structure, in which the pixel isolation portion 60 is provided in a groove portion formed to penetrate the front and back of the semiconductor substrate 58 .

1.7 Reasons for reduced image quality Next, several color filter arrangements are exemplified and explained with respect to the factors that cause image quality degradation when the resolution is changed.

FIG. 7 is a top view showing an example of a pixel arrangement using a Bayer arrangement as a color filter arrangement. As shown in FIG. 7 , the Bayer arrangement has a structure in which basic units 50A are regularly repeated in the matrix direction. The basic units 50A are arranged in a 2×2 matrix with a ratio of R:G:B=1:2:1 and have the following configuration: A pixel 30 having a color filter 52 that mainly transmits a red (R) wavelength component (also referred to as an R pixel 30R), and a color filter 52 that mainly transmits a green (G) wavelength component. The pixel 30 (also referred to as the G pixel 30G), and the pixel 30 (also referred to as the B pixel 30B) provided with the color filter 52 that mainly emits a wavelength component of blue (B).

FIG. 8 is a diagram showing an example of a planar arrangement in a case where a quadruple Bayer arrangement is used as a color filter arrangement. As shown in FIG. 8 , the quadruple Bayer array pixel array portion 21B has a structure in which a basic unit 50B is repeated in the matrix direction. The basic unit 50B is a matrix arrangement of the same color in which each pixel 30 of the Bayer array is divided into 2×2. Composed of 4 pixels and 30.

In addition, the color filter arrangement that can be used in this embodiment is not limited to the illustrated Bayer arrangement and the quadruple Bayer arrangement, and various color filter arrangements can be used.

In previous image sensors using color filter arrangements such as these, there was a possibility that image quality would be degraded when the resolution was changed. For example, when a Bayer arrangement is used for color filters, the image quality is high when shooting in full-pixel mode without binning, but when the resolution is reduced by binning, aliasing, etc. may occur. , the image quality is reduced beyond the resolution reduction. On the other hand, for example, when a quadruple Bayer arrangement is used for color filters, the image quality is higher when reading out at low resolution, but the image quality when imaging is performed in full-pixel mode is higher than when using Bayer. There is a possibility that the image quality will be degraded due to the arrangement.

In addition, in high-resolution image sensors, if readout is performed in full-pixel mode, problems such as a decrease in frame rate or an increase in power consumption may occur. In order to cope with such a problem, image sensors are sometimes equipped with a binning mode that reduces the resolution by adding pixels and reads it out, for example, during monitoring before photography or animation photography.

FIG. 9 is a diagram illustrating pixel addition of a conventional image sensor using a Bayer arrangement. Here, the pixel addition has the following constraints, for example, pixels of the same color must be added, and pixels in the same basic unit 50A or adjacent basic units 50A must be added in order to suppress degradation of image quality. For this reason, as shown in Figure 9(A), in previous image sensors using the Bayer arrangement, generally speaking, pixels of the same color arranged one pixel apart in the vertical direction (also called the row direction) are added together. .

However, in the case where pixels of the same color arranged one pixel apart in the vertical direction are added, as shown in FIG. 9(B) , since the spatial coordinates of the read pixels are the average of the added pixels 30, Gaps are generated between pixels of the same color in the vertical direction (row direction), and the spatial frequencies of pixels of the same color become non-uniform in the vertical and horizontal directions. As a result, aliasing occurs in the output image, resulting in problems such as reduced image quality. In addition, in images whose image quality is reduced due to aliasing, etc., problems such as reduced accuracy of recognition processing may also occur.

1.8 Suppression of image quality degradation Therefore, in this embodiment, by appropriately arranging the combination of the added pixels 30, the spatial frequency of the added pixels is suppressed from becoming non-uniform. FIG. 10 is a diagram illustrating a combination example of added pixels in this embodiment. As shown in FIG. 10(A) , in this embodiment, at least G pixels 30G among R pixels 30R, G pixels 30G, and B pixels 30B are arranged adjacent to G in the oblique direction in the basic unit 50A. Pixel 30G adds up. Thereby, as shown in FIG. 10(B) , since the spatial frequency of the pixels added and read out from the G pixel 30G can be made uniform in the vertical and horizontal directions, it is possible to suppress deterioration in image quality due to aliasing, etc. reduce.

In addition, in the process of demosaic or rearrangement of mosaic, the process of interpolating G pixels and generating full RGB pixels is usually performed. However, as in this embodiment, by appropriately arranging the combination of the added pixels, the vertical direction and The spatial frequencies of G pixels in the horizontal direction become uniform, and the above-mentioned processing can be omitted. In addition, in the Bayer arrangement, a reduced image with excellent characteristics can also be obtained.

In addition, FIG. 10 illustrates the case of adding 2 pixels, but it is not limited thereto. In the case of adding 3 or more pixels, for at least one color among the R pixel 30R, the G pixel 30G and the B pixel 30B, For the component pixels, the combination of the added pixels can also be adjusted so that the added spatial frequencies in the vertical and horizontal directions are uniform or approximately uniform.

However, in order to perform pixel addition, the floating diffusion area FD must be shared among the pixels 30 to be added. For example, as illustrated in FIG. 11 , when a total of eight pixels 30 constituting two basic units 50A arranged in the vertical direction share one floating diffusion area FD, the method of adding pixels illustrated in FIG. 10 can be used. However, as shown in the example of FIG. 12 , when a total of four pixels 30 constituting one basic unit 50A share one floating diffusion area FD, it is impossible to arrange the R pixels 30R with one pixel apart in the vertical direction. and the B pixel 30B are respectively combined as additional pixels.

Therefore, in this embodiment, as illustrated in FIG. 13 , three pixels 30 are arranged in the vertical direction, and one pixel 30 adjacent to the pixel 30 located at one end of the three pixels 30 in the horizontal direction is configured. A total of four pixels 30 (hereinafter also referred to as four pixels 30 arranged in an L shape) have one floating diffusion area FD. In the following description, a set of pixels 30 sharing the same floating diffusion area FD is also called a common unit.

In such a shared structure, there are two R pixels 30R and two G pixels 30G arranged in an L shape, a common unit 50ar sharing a floating diffusion area FD, and two B pixels arranged in an L shape. 30B and the two G pixels 30G share a common unit 50ab of one floating diffusion area FD.

In this way, by configuring the common units 50ar and 50ab by three pixels 30 arranged in the vertical direction and one pixel 30 adjacent in the horizontal direction with respect to the pixel 30 located at one end of the three pixels 30, it is possible to realize The combination of same-color pixels (R pixels 30R or B pixels 30B) arranged one pixel apart in the vertical direction, and the combination of same-color pixels (G pixels 30G) adjacent in the diagonal direction can be performed as shown in FIG. 10 The combination of added pixels.

Furthermore, as shown in the following formula (1), the capacitance (FD capacitance) C _FD of the floating diffusion region FD is proportional to the wiring length (FD wiring length) constituting the floating diffusion region FD, and as shown in the following formula ( 2) As shown, the conversion efficiency μ when the floating diffusion region FD converts charges into voltage is inversely proportional to the FD capacitance C _FD . In addition, in formula (2), q is the electric quantity. FD capacitor C _FD FD wiring length (1) Conversion efficiency μ=q/C _FD (2)

Therefore, the shorter the FD wiring length is, the more the conversion efficiency μ can be improved. However, as illustrated in FIG. 13 , by setting the floating diffusion region FD at the inner corner of the L-shaped pixel array, in other words, By arranging the floating diffusion region FD between the addition pixels arranged in the oblique direction (in this example, between the G pixels 30G), it is possible to connect the transfer transistor 31 from the floating diffusion region FD to each pixel 30 The wiring length is shortened to the same extent as the common FD common structure of 2×2 pixels illustrated in FIG. 12 . Thereby, a high conversion efficiency μ can be maintained, and thus the deterioration of image quality caused by a decrease in conversion efficiency μ can be suppressed.

1.9 Pixel arrangement rearrangement mosaic However, as shown in FIG. 14(A) , in the case where the four pixels 30 arranged in an L shape share the floating diffusion area FD, the same driving method as that used for the pixel array portion 21A applying the Bayer arrangement is used. To read out image data, four pixels 30 arranged in an L shape are read out as the basic unit of a 2×2 Bayer array, as shown in FIG. 14(B) .

To this end, in this embodiment, as shown in FIG. 14(C), in the read image data, the R pixel 30R of #4 of the shared unit 50ar including the two R pixels 30R and the Rearrangement and mosaic processing of replacing B pixel 30B of #1 of unit 50ab shared by two B pixels 30B. With this, the arrangement of the R pixels, G pixels, and B pixels of the image data can be set as a Bayer arrangement, so that the processing of the general Bayer arrangement can be used in the later stage.

FIG. 15 is a block diagram showing a structure for executing the rearrangement mosaic processing of this embodiment. As shown in FIG. 15 , the RAW image data read from the pixel array section 21 and AD-converted in the AD conversion circuit 23 a of the row processing circuit 23 is input to the rearrangement mosaic processing section 102 for each column. At this time, the column data including the pixels to be replaced are temporarily stored in the column memories 101-1, ..., 101-n, and then input to the rearrangement mosaic processing unit 102. In addition, the column data may be image data of one column in the column direction.

The rearrangement mosaic processing unit 102 directly outputs the column data that does not include pixels to be replaced. On the other hand, for the column data including the pixels to be replaced, the pixel values are read from the column memories 101-1, ..., 101-n in the correct order and output.

This is explained using the example of Figure 14 . In addition, for the convenience of explanation, in the RAW image data, the column data containing the pixel values of #1 and #2 in the common unit 50ar is set as the first column data, and the pixels containing #3 and #4 in the common unit 50ar are The value column data is set as the 2nd column data, the column data containing the pixel values of #1 and #2 in the shared unit 50ab is set as the 3rd column data, and the column data containing the pixel values #3 and #4 in the shared unit 50ab is set The column data is set to column 1 data.

In FIG. 14 , for example, in the RAW image data, the B pixel 30B (#1) that should be located at the lower right of the basic unit 50A is replaced with the upper left of the basic unit 50A that is located on the lower side in the vertical direction with respect to the basic unit 50A. The case of the square R pixel 30R (#4), in other words, the case where the R pixel #4 of the second column data is replaced with the B pixel #1 of the third column data (see FIG. 14(B) ).

In the case of the example shown in FIG. 14 , first, the data in the first column read from the pixel array unit 21A and AD-converted pass through the rearrangement mosaic processing unit 102 until output. Next, the second column data including the pixels to be replaced is temporarily stored in the column memory 101-1, and similarly the third column data including the pixels to be replaced is temporarily stored in the column memory 101-2. If so, the rearrangement mosaic processing unit 102 reads out the column data of the two columns in the correct pixel order from the second column data and the third column data stored in the column memories 101-1 and 101-2, and sequentially Output the read column data. Then, the rearrangement mosaic processing unit 102 directly outputs the subsequently input data in the fourth column. By this, the pixel arrangement of the output image data can be set to the correct Bayer arrangement.

In addition, the remosaic processing can be executed in the row processing circuit 23, the signal processing unit 26, or outside the image sensor 10, such as the processor 13. However, when the pixel array portion 21A is driven so that the pixel arrangement of the image data read out from the pixel array portion 21A is a normal Bayer arrangement, the above-mentioned rearrangement mosaic processing can be omitted.

1.10 Correction of characteristic differences between pixels As illustrated in FIG. 13 , when the floating diffusion area FD is arranged at the inner corner of the L-shaped pixel array, for one pixel among the four pixels 30 (for example, the R in the lower left corner of the common unit 50ar The distance between the pixel 30R) and the floating diffusion area FD is different from the distance between the pixel 30R and the other pixels 30 . Therefore, even if the pixels 30 of the same color are included in the same shared unit 50ar or 50ab, differences in characteristics may occur between the pixels. Therefore, in this embodiment, a process for correcting the characteristic difference between pixels of the same color generated in the above manner can be implemented.

For the correction process of the characteristic difference between pixels of the same color, consider the process of correcting the sensitivity difference between pixels, and the process of correcting the color mixing difference caused by the influence from surrounding pixels, etc. Hereinafter, an example of the correction process will be described using FIG. 16 . In addition, for the sake of clarity, in the description using FIG. 16 , the R pixel 30R is called pixels R ₀ and R ₁ , the G pixel 30G is called pixels G ₀ and G ₁ , and the B pixel 30B is called pixels B 0 and B ₀ . _B1 .

The process of correcting the sensitivity difference between pixels may be, for example, a process of correcting the pixel values of the pixels 30 of the same color and different phases by multiplying them by a gain. For example, for the pixel R ₀ , the pixel value (RAW data) read from the pixel R ₀ can be set to R ₀ , the corrected pixel value R ₀ can be set to R ₀ ', and the correction to the pixel value R 0 can be set to R ₀ . The gain coefficient is set to g _R0 . For pixel R ₁ , the pixel value (RAW data) read from the pixel R ₁ is set to R ₁ . The corrected pixel value R ₁ is set to R ₁ '. For the pixel value The correction gain coefficient of R ₁ is set to g _R1 , and the correction processing shown in the following equations (3) and (4) is performed. In addition, this correction process may be the same for pixels B ₀ , B ₁ , G ₀ , and G ₁ .

The process of correcting the color mixture difference caused by the influence from surrounding pixels may be, for example, a process of correcting using the pixel values of the surrounding pixels 30 . For example, the pixel value (RAW data) read from the pixel G ₀ can be set to G ₀ , the corrected pixel value G ₀ can be set to G ₀ ', and the pixel value (RAW data) read from the pixel G ₁ can be set to G 0 . Set as G ₁ , set the corrected pixel value G ₁ as G ₁ ', set the color mixing correction coefficient between pixel G ₀ and pixel R ₀ as C _G0R0 , set the color mixing between pixel G ₀ and pixel B ₁ Set the correction coefficient to C _G0B1 , set the color mixing correction coefficient between pixel G ₁ and pixel R ₀ to C _G1R0 , and set the color mixing correction coefficient between pixel G ₁ and pixel B ₀ to C _G1B0 . The execution is as follows: (5) and the correction processing shown in equation (6).

By performing the above correction process, the characteristic difference between pixels of the same color can be reduced, so that the degradation of image quality can be further suppressed. In addition, this correction process may be performed by the row processing circuit 23 or the signal processing unit 26 .

1.11 Application of global shutter method In the above, as an example of the driving method of the image sensor 10, the so-called rolling shutter method can be used to read pixel values (column data) for each column in the horizontal direction. However, it is not limited to this. For example, all pixels can be used. 30 simultaneous drives of the so-called global shutter method.

FIG. 17 is a circuit diagram showing an example of the schematic configuration of a pixel 100 using the VDGS (Voltage Domain Global Shutter) method, which is one of the global shutter methods. As shown in FIG. 3 , the pixel 100 includes a front-end circuit 110 , capacitive elements 121 and 122 , a selection circuit 130 , a back-end reset transistor 141 , and a back-end circuit 150 . In this structure, the rear-end reset transistor 141 and the rear-end circuit 150 can be shared among a plurality of pixels 100 .

The front-stage circuit 110 includes a photoelectric conversion part PD, a transfer transistor 112, an FD (Floating Diffusion, floating diffusion) reset transistor 113, a floating diffusion region FD, a front-stage amplification transistor 115, and a current source transistor 116.

The photoelectric conversion part PD generates electric charges through photoelectric conversion. The transfer transistor 112 transfers charges from the photoelectric conversion part PD to the floating diffusion region FD in accordance with the transfer signal trg from the vertical scanning circuit 211.

The FD reset transistor 113 is initialized by extracting charges from the floating diffusion region FD according to the FD reset signal rst from the vertical scanning circuit 211 . The floating diffusion region FD accumulates electric charges and generates a voltage corresponding to the amount of electric charges. The front-stage amplifying transistor 115 amplifies the voltage level of the floating diffusion region FD and outputs it to the front-stage node 120 .

In addition, the sources of the FD reset transistor 113 and the front-stage amplification transistor 115 are connected to the power supply voltage VDD. The current source transistor 116 is connected to the drain of the front-stage amplification transistor 115 . The current source transistor 116 supplies the current id1 according to the control of the vertical scanning circuit 211 .

One end of each of the capacitive elements 121 and 122 is commonly connected to the previous node 120 , and the other end of each is connected to the selection circuit 130 .

The selection circuit 130 includes a selection transistor 131 and a selection transistor 132 . The selection transistor 131 switches the path between the capacitive element 121 and the subsequent node 140 in accordance with the selection signal ϕr from the vertical scanning circuit 211. The selection transistor 132 switches the path between the capacitive element 122 and the subsequent node 140 according to the selection signal ϕs from the vertical scanning circuit 211.

The rear-stage reset transistor 141 initializes the level of the rear-stage node 140 to a specific potential Vreg according to the rear-stage reset signal rstb from the vertical scanning circuit 211. As for the potential Vreg, a potential different from the power supply potential VDD is set (for example, a potential lower than VDD).

The rear-stage circuit 150 includes a rear-stage amplification transistor 151 and a rear-stage selection transistor 152 . The rear-stage amplification transistor 151 amplifies the level of the rear-stage node 140 . The rear-stage selection transistor 152 outputs a signal of a level amplified by the rear-stage amplification transistor 151 as a pixel signal to the vertical signal line VSL according to the rear-stage selection signal selb from the vertical scanning circuit 211.

In addition, nMOS (n-channel Metal Oxide Semiconductor, n-channel metal oxide semiconductor) transistors, for example, are used as various transistors (transfer transistor 112 and so on) in the pixel 100 .

The vertical scanning circuit 211 supplies the high-level FD reset signal rst and the transmission signal trg to all pixels at the beginning of exposure. Thereby, the photoelectric conversion part PD is initialized. Hereinafter, this control is called "PD reset".

Then, the vertical scanning circuit 211 sets the rear-stage reset signal rstb and the selection signal ϕr to a high level for all pixels just before the exposure is completed, and supplies the high-level FD reset signal rst during the entire pulse period. Thereby, the floating diffusion area FD is initialized, and a level corresponding to the current level of the floating diffusion area FD is held in the capacitive element 121 . This control is referred to as "FD reset" below. This control is referred to as "FD reset" below.

Hereinafter, the level of the floating diffusion region FD when FD is reset and the level corresponding to this level (the holding level of the capacitive element 121 or the level of the vertical signal line VSL) are collectively referred to as "P phase" or "P phase". Reset level".

At the end of the exposure, the vertical scanning circuit 211 sets the rear-stage reset signal rstb and the selection signal ϕs to a high level for all pixels, and supplies a high-level transmission signal trg during the entire pulse period. Thereby, signal charges corresponding to the exposure amount are transferred to the floating diffusion area FD, and a level corresponding to the level of the floating diffusion area FD at that time is held in the capacitive element 122 .

Hereinafter, the level of the floating diffusion region FD during signal charge transfer and the level corresponding to this level (the holding level of the capacitive element 122 or the level of the vertical signal line VSL) are collectively referred to as "D phase" or "D phase". signal level".

Exposure control that starts and ends exposure for all pixels at the same time is called the global shutter method. Through this exposure control, the front-end circuit 110 of all pixels sequentially generates reset levels and signal levels. The reset level is held by the capacitive element 121 , and the signal level is held by the capacitive element 122 .

After the exposure is completed, the vertical scanning circuit 211 selects the column in sequence and outputs the reset level and signal level of the column in sequence. When outputting the reset level, the vertical scanning circuit 211 sets the FD reset signal rst of the selected column and the subsequent selection signal selb to a high level, and supplies a high level selection signal ϕr throughout the specific period. Thereby, the capacitive element 121 is connected to the subsequent node 140, and the reset level is read.

After reading the reset level, the vertical scanning circuit 211 supplies the high-level rear segment reset signal rstb during the entire pulse period without setting the FD reset signal rst and the rear segment selection signal selb of the selected column to a high level. . Thereby, the level of the subsequent node 140 is initialized. At this time, both the selection transistor 131 and the selection transistor 132 are in an off state, and the capacitive elements 121 and 122 are disconnected from the subsequent node 140 .

After the initialization of the back-stage node 140, the vertical scanning circuit 211 supplies the high-level selection signal ϕs throughout the specific period while setting the FD reset signal rst and the back-stage selection signal selb of the selected column to a high level. Thereby, the capacitive element 122 is connected to the subsequent node 140, and the signal level is read out.

Through the above readout control, the selection circuit 130 of the selected column sequentially performs the control of connecting the capacitive element 121 to the subsequent node 140, the control of disconnecting the capacitive elements 121 and 122 from the subsequent node 140, and the control of connecting the capacitor 121 to the subsequent node 140. The component 122 is connected to the control of the subsequent node 140 . In addition, when the capacitive elements 121 and 122 are disconnected from the rear node 140, the selected row rear reset transistor 141 initializes the level of the rear node 140. In addition, the subsequent stage circuit 150 of the selected column sequentially reads out the reset level and the signal level from the capacitive elements 121 and 122 through the subsequent node 140 and outputs them to the vertical signal line VSL.

In the above structure, for example, it is possible to arrange the light-receiving circuit including the photoelectric conversion part PD, the transfer transistor 112, the FD reset transistor 113, and the front-stage amplification transistor 115 on the light-receiving chip 41, and other circuits Structure 302 is disposed on circuit chip 42 .

In addition, the rear-stage reset transistor 141 and the rear-stage circuit 150 are circuit structures that do not affect the conversion efficiency μ of the floating diffusion region FD. Therefore, in this embodiment, the rear-stage reset transistor 141 and the rear-stage circuit 150 are shared among the plurality of pixels constituting the basic unit, and the wiring length of the shared portion is changed. Thereby, the conversion efficiency μ of each pixel 100 can be maintained, and a pixel sharing structure can be achieved.

In addition, in the front-end circuit 110, the wiring length from the drain of the transfer transistor 112 to the gate of the front-end amplification transistor 115 and the source of the FD reset transistor 113 may affect the conversion efficiency μ of the floating diffusion region FD. , so it is preferable to have the same wiring length among the pixels 100 sharing the same floating diffusion region FD.

In this way, in the structure in which pixels are shared by the circuit structure 302 arranged on the circuit chip 42, the pixels are shared from the Bayer arrangement of the basic units 50A as shown in FIG. 18 to the L-shaped arrangement as shown in FIG. 19 The design change of the common unit 50ar (or the common unit 50ab) that constitutes the common structure of the pixels, such as the design change from the circuit structure 302 laid out in a rectangular area to the circuit structure 302a laid out in an L-shaped area, can mainly be achieved by This is achieved by changing the wiring lengths of the rear-stage reset transistor 141 and the rear-stage circuit 150 without affecting the conversion efficiency μ. Therefore, the conversion efficiency μ of each pixel 100 can be maintained and is achieved by a very easy operation.

1.12 Application to another color filter arrangement The pixel arrangement shape of the common unit illustrated above is not limited to the L shape, and the optimal arrangement shape can be adopted by considering the combination with the color filter arrangement used. Therefore, here, an example of a pixel arrangement shape using an RGBW arrangement (W is white) and changing the Bayer arrangement will be described.

FIG. 20 is a diagram showing an example of a common unit applicable to an image sensor using an RGBW arrangement (W is white). As shown in FIG. 20(A) , in addition to the R pixel 30R, the G pixel 30G, and the B pixel 30B, a W pixel 30W is also arranged, and the W pixel 30W is provided with a color having wide transmission characteristics in the visible light range. The filter 52 can also suppress image quality by appropriately arranging a combination of addition pixels for at least one of RGBW (except G in FIG. 20 ) so that the spatial frequencies in the vertical and horizontal directions become uniform. decrease. For example, in the example shown in FIG. 20(B) , the spatial frequencies in the vertical direction and the horizontal direction of each of the W pixels, R pixels, and B pixels can be set to be uniform.

1.13 Summary As described above, according to this embodiment, the spatial frequencies in the vertical direction and the horizontal direction can be equalized for at least one color component in the image data read out from the pixel array unit 21, so that the occurrence of aliasing, etc. can be suppressed. Suppress image quality degradation.

2. Second embodiment Next, the second embodiment of the present disclosure will be described in detail with reference to the drawings. In addition, the same structures, operations, and effects as those in the above-mentioned embodiments are cited, and repeated descriptions are omitted.

In the above-described first embodiment, a case has been described in which at least one color in the image data is obtained by appropriately arranging the pixel arrangement of the common units sharing the floating diffusion area FD, that is, by adding a combination of pixels. components to uniformize the spatial frequencies in the vertical and horizontal directions and suppress the degradation of image quality. In this regard, in the second embodiment, the pixel driving lines LD, the pixel sharing structure (that is, the pixel arrangement of the shared units), the color filter arrangement, and the like are appropriately arranged. This makes it possible to obtain image data that is better for post-processing in which the spatial frequencies in the vertical direction and the horizontal direction are uniformized for at least one color component in the image data, so that degradation of image quality can be suppressed.

In this embodiment, the electronic equipment and the image sensor can be the same as the electronic equipment and the image sensor illustrated in the first embodiment. However, in this embodiment, the color filter arrangement and pixel common structure are replaced with those illustrated below

2.1 Example of color filter arrangement and pixel common structure FIG. 21 is a top view showing an example of pixel arrangement using the color filter arrangement and pixel common structure of this embodiment. As shown in FIG. 21 , in this embodiment, the pixel array unit 221 has a pixel arrangement structure in which R pixels 30R, G pixels 30G, and B pixels 30B are arranged in an oblique direction.

In such a structure, the common unit is composed of a 2×2 common unit 50r and a 2×2 common unit 50b as shown in FIG. 22(A) , and the 2×2 common unit 50r includes a Two R pixels 30R in the right direction and two G pixels 30G arranged in the right diagonally upward direction. The 2×2 shared unit 50b includes two B pixels 30B arranged in the right diagonally downward direction and two B pixels 30B arranged in the right diagonally upward direction. 2G pixels 30G.

In this way, by setting the pixels of the same color located at the opposite corners of the 2×2 pixel arrangement as a combination of the added pixels, as shown in Figure 22(B), the added R pixels, B pixels and G pixels can be The spatial frequency is uniformized in both the vertical and horizontal directions, thus reducing image quality degradation caused by aliasing and the like.

Furthermore, according to this embodiment, since the added spatial coordinates of R pixels or B pixels and G pixels are consistent, the accuracy of pixel interpolation of R pixels and B pixels can be easily improved.

Furthermore, according to this embodiment, there is also an advantage that the common units 50r and 50b can be configured without changing the wiring structure of the floating diffusion region FD.

In addition, in the case of full pixel readout, the rearrangement mosaic processing of the pixel arrangement can be performed by using the method illustrated in Figures 14 and 15 to convert the R pixels, G pixels and B pixels in the image data. The arrangement is set to Bayer arrangement. However, it is not limited to this, and demosaic processing corresponding to the pixel arrangement in this example can be performed.

According to this embodiment, it is possible to obtain image data with better brightness resolution than in the first embodiment using the quadruple Bayer arrangement illustrated in FIG. 8 . Furthermore, according to this embodiment, the image quality when pixel addition is completed can be improved compared with the image quality when the Bayer arrangement or the quadruple Bayer arrangement is used. In this way, the degradation of image quality can be suppressed in both full pixel mode and merge mode.

Since other structures, operations, and effects are the same as those in the above-described embodiment, detailed descriptions are omitted here.

2.2 Variation examples of color filter arrangement and pixel common structure The color filter arrangement and pixel sharing structure of the second embodiment are highly compatible with the structure in which pixels arranged in the diagonal direction are used as additive pixels. Therefore, various colors in which pixels of the same color are arranged in the diagonal direction can be used. Filter arrangement. Therefore, several examples of modifications of the color filter arrangement and pixel sharing structure of the second embodiment described above using FIGS. 21 and 22 will be described below. In addition, the configuration, operations, and effects not specifically mentioned in the following description may be the same as those in the above-mentioned embodiment or other modified examples.

2.2.1 The first variation FIG. 23 is a top view showing a pixel arrangement example using the color filter arrangement and the pixel common structure of the first variation. As shown in FIG. 23(A) , the pixel array section 221A of the first modification example has a pixel arrangement structure in which in addition to the R pixels 30R, G pixels 30G, and B pixels 30B, W pixels 30W are also arranged in the oblique direction.

In such a configuration, the common unit is composed of a 2×2 common unit 50rw, a 2×2 common unit 50gw, and a 2×2 common unit 50bw, as shown in FIG. 23(A) , and the 2×2 The shared unit 50rw includes two R pixels 30R arranged in the diagonally lower right direction and two W pixels 30W arranged in the upper right direction. The 2×2 shared unit 50gw includes two G pixels arranged in the diagonally lower right direction. 30G and two W pixels 30W arranged in the diagonally upward direction to the right. The 2×2 shared unit 50bw includes two B pixels 30B arranged in the diagonally downward direction to the right and two W pixels 30W arranged in the diagonally upward direction to the right.

In this way, in the case where in addition to the R pixels 30R, G pixels 30G and B pixels 30B, the W pixels 30W are also provided, by setting the same color pixels located at the opposite corners of the 2×2 pixel arrangement as a combination of additive pixels, Alternatively, as shown in FIG. 23(B) , the added spatial frequencies of each of the R pixels, B pixels, G pixels, and W pixels can be normalized in both the vertical and horizontal directions. Thereby, like the above-mentioned embodiment, it is possible to suppress deterioration in image quality due to aliasing or the like.

2.2.2 Second variation FIG. 24 is a top view showing a pixel arrangement example using a color filter arrangement and a pixel sharing structure according to the second variation. As shown in FIG. 24(A) , the pixel array section 221B of the second modification example has a pixel arrangement structure in which the W pixels 30W are arranged in the same configuration as the pixel array section 221A described as the first modification example. It is replaced with an IR pixel 30IR provided with a color filter 52 having transmission characteristics for infrared light (including near-infrared light).

Therefore, the pixel array section 221B of this modification example is composed of a 2×2 shared unit 50rir, a 2×2 shared unit 50gir, and a 2×2 shared unit 50bir, as shown in FIG. 24(A) , and the 2×2 shared unit 50bir The common unit 50rir of 2 includes two R pixels 30R arranged in the diagonally lower right direction and two IR pixels 30IR arranged in the upper right direction. The common unit 50gir of 2×2 includes two R pixels 30R arranged in the diagonally lower right direction. G pixel 30G and two IR pixels 30IR arranged in the right diagonally upward direction. The 2×2 shared unit 50bir includes two B pixels 30B arranged in the right diagonally downward direction and two IR pixels arranged in the right diagonally upward direction. 30IR.

In this way, in the case where in addition to the R pixels 30R, G pixels 30G and B pixels 30B, the IR pixel 30IR is also provided, by setting the same color pixels located at the diagonal corners of the 2×2 pixel arrangement as a combination of additive pixels, Alternatively, as shown in FIG. 24(B) , the added spatial frequencies of each of the R pixels, B pixels, G pixels, and IR pixels can be normalized in both the vertical and horizontal directions. Thereby, like the above-mentioned embodiment, it is possible to suppress deterioration in image quality due to aliasing or the like.

2.2.3 The third variation As in the above-mentioned first variation and second variation, in the case where in addition to the R pixel 30R, G pixel 30G, and B pixel 30B, the W pixel 30W and the IR pixel 30IR are also included in the pixel array portion 21, by at least By setting the vertical and horizontal spatial frequencies of the added G pixels 30G, W pixels 30W, or IR pixels 30IR to be uniform, degradation of image quality can be suppressed. Therefore, for example, in the case where the common unit illustrated in FIGS. 25 to 28 has m×n (m and n are integers of 3 or more) pixel arrangements (the examples in FIGS. 25 to 28 are 3×3 pixel arrangements) ), the calculation is not limited to pixel addition of 2 pixels of the same color, and various changes such as pixel addition of 4 pixels of the same color or 5 pixels of the same color can be performed.

In addition, the pixel array portion 221C illustrated in FIG. 25 is composed of a 3×3 shared unit 50r4 and a 3×3 shared unit 50b4, and the 3×3 shared unit 50r4 includes 5 G pixels located at the four corners and the center respectively. 30G, and four R pixels 30R located in the center of each of the four sides. The 3×3 total unit 50b4 includes five G pixels 30G located at the four corners and the center, and four R pixels 30R located in the center of each of the four sides. Bpixel 30B.

The pixel array portion 221D illustrated in FIG. 26 is composed of a 3×3 shared unit 50r5 and a 3×3 shared unit 50b5, and the 3×3 shared unit 50r5 includes five R pixels 30R located at the four corners and the center respectively. and 4 G pixels 30G located in the center of each of the four sides. The 3×3 shared unit 50b5 includes 5 B pixels 30B located at the four corners and the center respectively, and 4 G pixels located in the center of each of the four sides. 30G.

The pixel array portion 221E illustrated in FIG. 27 is composed of a 3×3 shared unit 50r4w and a 3×3 shared unit 50b4, and the 3×3 shared unit 50r4w includes five W pixels 30W located at the four corners and the center respectively. and 4 R pixels 30R located at the center of each of the four sides. The 3×3 shared unit 50b4 includes 5 G pixels 30G located at the four corners and the center respectively, and 4 B pixels located at the center of each of the four sides. 30B.

The pixel array portion 221F illustrated in FIG. 28 is composed of a 3×3 shared unit 50r5w, a 3×3 shared unit 50g5w, and a 3×3 shared unit 50b5w. The 3×3 shared unit 50r5w includes four corners respectively. There are 5 R pixels 30R in the center and 4 W pixels 30W located in the center of each of the four sides. The 3×3 total unit 50g5w includes 5 G pixels 30G located in the four corners and the center, and 5 G pixels 30G located in the four sides. There are 4 W pixels 30W in the center. The 3×3 shared unit 50b5w includes 5 B pixels 30B located at the four corners and the center respectively, and 4 W pixels 30W located in the center of each of the four sides.

In this way, when the total unit has m×n (m, n is an integer of 3 or more) pixel arrangements, by setting the vertical and horizontal spatial frequencies of the added pixels for at least one wavelength component. It is uniform, and like the above-mentioned embodiment, it is also possible to suppress deterioration in image quality due to aliasing or the like.

3. Third embodiment Next, the third embodiment of the present disclosure will be described in detail with reference to the drawings. In addition, the same structures, operations, and effects as those in the above-mentioned embodiments are cited, and repeated descriptions are omitted.

As shown in FIG. 29 , for example, in an image sensor using a rolling shutter method, shutter control is performed for each column. Specifically, the reset (reset signal) and pixel signal of the floating diffusion region FD (and photoelectric conversion part PD) are performed by pixel driving lines (also called horizontal signal lines) LD wired in each column along the column direction. The timing control of readout (selection signal), etc. accumulates charges in the floating diffusion region FD of the pixel 30.

In such a configuration, when HDR (High Dynamic Range) photography is performed by changing the shutter time for each pixel 30, for example, it is necessary to shorten the charge accumulation time as shown in FIG. 30. The pixels (hereinafter also referred to as short-storage pixels) 30S are arranged in odd-numbered rows (or even-numbered rows), and the pixels 30L with long charge storage time (hereinafter also referred to as long-storage pixels) are arranged in even-numbered rows (or odd-numbered rows). Each column changes the control, or as illustrated in FIG. 31 , the number of horizontal signal lines is increased by setting separate pixel driving lines LD (for example, reset lines) in the short storage pixels 30S and the long storage pixels 30L.

Here, when the control is changed for each column, the wiring structure and the like can be simplified, but the resolution is different in the horizontal direction and the vertical direction, so there is a risk that aliasing etc. will occur and the image quality will be reduced. On the other hand, when horizontal signal lines are added, the horizontal and vertical directions can be sampled uniformly, so the degradation of image quality can be suppressed. However, due to the complexity of the wiring structure, etc., it is difficult to mount it on a high-resolution system. image sensor problem.

Therefore, in this embodiment, as illustrated in FIG. 32 or FIG. 33 , the previous structure of using one pixel driving line LD to control one column of pixels 30 is replaced with using one pixel driving line LD to control the distributed arrangement of the pixels 30 . Change the format of 2 rows of long storage pixels 30L or short storage pixels 30S. Specifically, the short storage pixels 30S and the long storage pixels 30L with different charge accumulation times are arranged in the oblique direction (that is, in an alternating manner in the column direction and row direction), and a horizontal signal line is connected to The pixels 30 (short storage pixels 30S or long storage pixels 30L) with the same charge storage time are divided into two columns and arranged.

Thereby, the increase in the number of horizontal signal lines can be suppressed, and the pixels 30 with different charge storage times (short storage pixels 30S or long storage pixels 30L) can be evenly arranged throughout the entire pixel array portion 321 without missing a beat. The vertical and horizontal spatial frequencies of each of the short storage pixel 30S and the long storage pixel 30L are set to be uniform. As a result, it is possible to obtain an HDR image in which degradation of image quality is suppressed.

However, as shown in FIG. 34(A) , if the short storage pixels 30S or the long storage pixels 30L are alternately arranged in two upper and lower columns so as not to be adjacent to each other, the same driving method as that used for the pixel array unit 21A applying the Bayer arrangement is used. When reading is performed using the same driving method, as shown in FIG. 34(B) , the pixels 30 with the same charge accumulation time are alternately read out as image data of one column. Therefore, the short storage time of the pixel array portion 321 The actual pixel arrangement of the pixels 30S and the long storage pixels 30L is different from the pixel arrangement of the short storage pixels 30S and the long storage pixels 30L in the read image data.

Therefore, in this embodiment, as shown in FIG. 34(C) , in the read image data, a rearrangement mosaic process of replacing the short storage pixels 30S and the long storage pixels 30L in every other row can be performed. In the example shown in FIG. 34 , the rearrangement mosaic process of replacing the long storage pixel 30L of #2 with the short storage pixel 30S of #4 in the even-numbered column from the left can be performed. Thereby, the actual pixel arrangement of the short storage pixels 30S and the long storage pixels 30L in the pixel array part 321 can be consistent with the pixel arrangement of the short storage pixels 30S and the long storage pixels 30L in the read image data. Therefore, The vertical and horizontal spatial frequencies of each of the short storage pixel 30S and the long storage pixel 30L can be made uniform, thereby obtaining an HDR image that suppresses degradation in image quality.

In addition, the rearrangement mosaic process of this embodiment can be realized using the structure similar to the structure for executing the rearrangement mosaic process demonstrated using FIG. 15 in 1st Embodiment, for example. For example, in the example shown in FIG. 34 , first, the first column data read from the pixel array unit 321 and AD-converted are temporarily stored in the column memory 101 - 1 , and similarly, the second column data are temporarily stored in the column memory. Body 101-2. In this case, the rearrangement mosaic processing unit 102 alternately reads the pixel value of the short storage pixel 30S and the long storage pixel 30L from the second column data and the third column data stored in the column memories 101-1 and 101-2. The pixel value is used to generate 1 column of image data and output it. Thereafter, by repeating the same operation two columns at a time, output image data having a pixel arrangement consistent with the actual pixel arrangement in the pixel array unit 321 is output.

In addition, the remosaic processing can be executed in the row processing circuit 23, the signal processing unit 26, or outside the image sensor 10, such as the processor 13. However, when the pixel array unit 321 is driven in such a manner that the pixel arrangement in the image data read out from the pixel array unit 321 is consistent with the actual pixel arrangement of the pixel array unit 321, the above-mentioned rearrangement mosaic processing may be omitted.

Since other structures, operations, and effects are the same as those in the above-described embodiment, detailed descriptions are omitted here.

3.1 Variation examples As in the above-mentioned third embodiment, the pixels 30S having different charge accumulation times are arranged in the diagonal direction in a non-adjacent manner throughout the upper and lower plurality of columns. The pixel array shown has a structure in which monochromatic pixels with different charge accumulation times (in this example, W pixels 30WS and 30WL with different charge accumulation times) are alternately arranged up, down, left, and right (see, for example, FIG. 32 or FIG. 33 ). For example, the portion 321A is also applicable to a structure in which pixels of the same color arranged in the diagonal direction are added together as shown in FIGS. 36 to 39 (refer to the above-mentioned second embodiment).

In addition, the pixel array section 321B illustrated in FIG. 36 is equipped with R pixels 30R, G pixels 30G, and B pixels 30B, which are respectively arranged in the oblique direction, similarly to the pixel array section 221 explained using FIG. 21 in the second embodiment. The pixel configuration structure.

The pixel array section 321C illustrated in FIG. 37 has a pixel arrangement structure using the quadruple Bayer arrangement described using FIG. 8 in the first embodiment as a basic unit.

The pixel array section 321D illustrated in FIG. 38 is equipped with, in addition to the R pixel 30R, the G pixel 30G, and the B pixel 30B, W The pixels 30W are also arranged in a pixel arrangement structure in an oblique direction.

The pixel array section 321E illustrated in FIG. 39 is equipped with an IR pixel in addition to the R pixel 30R, the G pixel 30G, and the B pixel 30B, similarly to the pixel array section 221B explained using FIG. 24 as a second modification example of the second embodiment. The pixels 30IR are also arranged in a pixel arrangement structure in an oblique direction.

In this way, the structure in which one pixel drive line LD is connected to the pixels 30 in a plurality of columns is also applicable to a pixel arrangement structure in which pixels of the same color to be added are arranged in a diagonal direction throughout a plurality of columns. Furthermore, the configuration in which one pixel driving line LD is connected to the pixels 30 in a plurality of columns can be applied to a pixel arrangement structure in which pixels of the same color to be added are arranged in a diagonal direction throughout a plurality of columns, so that the pixel pitch can be reduced, so that the pixel pitch can be reduced. Get higher quality images with high resolution. In addition, the pixel arrangement structure in which one pixel driving line LD is connected to a plurality of columns of pixels 30 is not limited to the above example, and can be variously modified.

4. Application examples for smartphones The technology disclosed herein (the technology) can further be applied to a variety of products. For example, the technology disclosed herein can be applied to smartphones and the like. To this end, a structural example of a smartphone 900 as an electronic device to which the present technology is applied will be described with reference to FIG. 40 . FIG. 40 is a block diagram showing an example of a schematic functional configuration of a smartphone 900 to which the technology of the present disclosure (the present technology) can be applied.

As shown in Figure 40, a smartphone 900 includes a CPU (Central Processing Unit) 901, a ROM (Read Only Memory) 902, and a RAM (Random Access Memory) 903 . In addition, the smart phone 900 includes a storage device 904, a communication module 905, and a sensor module 907. Furthermore, the smartphone 900 includes the camera device 1 , a display device 910 , a speaker 911 , a microphone 912 , an input device 913 , and a bus 914 . In addition, the smartphone 900 may replace the CPU 901 or have a processing circuit such as a DSP (Digital Signal Processor) together with the CPU 901 .

The CPU 901 functions as a computing processing device and a control device, and controls all operations or part thereof in the smartphone 900 according to various programs recorded in the ROM 902, RAM 903, or storage device 904, etc. The ROM 902 stores programs and operation parameters used by the CPU 901. The RAM 903 temporarily stores programs used in the execution of the CPU 901, parameters appropriately changed during its execution, and the like. The CPU 901, ROM 902, and RAM 903 are connected to each other through a bus 914. In addition, the storage device 904 is a data storage device configured as an example of the memory unit of the smartphone 900 . The storage device 904 is composed of, for example, a magnetic memory device such as an HDD (Hard Disk Drive), a semiconductor memory device, an optical memory device, or the like. The storage device 904 stores programs and various data executed by the CPU 901, as well as various data obtained from the outside.

The communication module 905 is, for example, a communication interface composed of communication devices used to connect to the communication network 906 . The communication module 905 may be, for example, a communication card for wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), WUSB (Wireless USB, Wireless USB), etc. In addition, the communication module 905 can be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line, asymmetric digital subscriber line), or a modem for various communications, etc. For example, the communication module 905 uses specific protocols such as TCP (Transmission Control Protocol)/IP (Internet Protocol) to send and receive signals with the Internet or other communication machines. In addition, the communication network 906 connected to the communication module 905 is a network connected by wire or wireless, such as the Internet, home LAN, infrared communication or satellite communication, etc.

The sensor module 907 includes, for example, motion sensors (such as acceleration sensors, gyroscope sensors, geomagnetic sensors, etc.), biological information sensors (such as pulse sensors, blood pressure sensors, Various sensors such as fingerprint sensors, etc.), or position sensors (such as GNSS (Global Navigation Satellite System) receivers, etc.).

The camera device 1 is installed on the front of the smartphone 900 and can capture objects located on the back or front of the smartphone 900 . Specifically, the imaging device 1 may include an imaging element (not shown) such as a CMOS (Complementary MOS, complementary MOS) image sensor to which the technology of the present disclosure (this technology) can be applied, and photoelectric conversion of the imaging element. The latter signal is formed by applying a signal processing circuit (not shown) for imaging signal processing. Furthermore, the imaging device 1 may further include an optical system mechanism (not shown) composed of an imaging lens, a zoom lens, a focus lens, and the like, and a drive system mechanism (not shown) that controls the operation of the optical system mechanism. Furthermore, the above-mentioned imaging element collects the incident light from the object into an optical image, and the above-mentioned signal processing circuit performs photoelectric conversion on the formed optical image in units of pixels, reads out the signal of each pixel as an imaging signal, and performs image processing. image processing to obtain camera images.

The display device 910 is disposed on the front of the smart phone 900, and may be, for example, an LCD (Liquid Crystal Display), an organic EL (Electro Luminescence) display, or other display device. The display device 910 can display the operation screen, the photographed image obtained by the above-mentioned imaging device 1, and the like.

The speaker 911 may, for example, output a call sound to the user, or a sound accompanying the image content displayed on the display device 910 , or the like.

The microphone 912 may, for example, collect the user's call sound, the sound including instructions for activating functions of the smartphone 900 , or the sounds of the surrounding environment of the smartphone 900 .

The input device 913 is, for example, a button, keyboard, touch panel, mouse, or other device operated by the user. The input device 913 includes an input control circuit that generates an input signal based on information input by the user and outputs it to the CPU 901 . By operating the input device 913, the user can input various data and instruct processing actions to the smartphone 900.

The above shows an example of the configuration of the smartphone 900 . Each of the above-mentioned components may be constructed using general-purpose components, or may be constructed using hardware specialized for the functions of each component. The above-mentioned structure can be appropriately changed according to the technical level at the time of implementation.

5. Application examples for moving objects The technology disclosed herein (the technology) can be applied to a variety of products. For example, the technology disclosed in the present disclosure can be implemented as a device mounted on any type of mobile object such as cars, electric cars, hybrid cars, motorcycles, bicycles, personal mobility devices, airplanes, drones, ships, and robots.

41 is a block diagram showing a schematic configuration example of a vehicle control system as an example of a mobile body control system to which the technology of the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in FIG. 41 , the vehicle control system 12000 includes a drive system control unit 12010 , a vehicle body system control unit 12020 , an exterior information detection unit 12030 , an interior information detection unit 12040 , and an integrated control unit 12050 . In addition, as the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio and video output unit 12052, and an in-vehicle network I/F (Interface) 12053 are shown.

The drive system control unit 12010 controls the actions of devices associated with the vehicle's drive system according to various programs. For example, the drive system control unit 12010 serves as a driving force generating device for generating the driving force of the vehicle such as an internal combustion engine or a driving motor, a driving force transmission mechanism for transmitting the driving force to the wheels, a steering mechanism for adjusting the rudder angle of the vehicle, And the control device of the braking device that generates the braking force of the vehicle functions.

The vehicle body system control unit 12020 controls the operations of various devices equipped on the vehicle body according to various programs. For example, the vehicle body system control unit 12020 functions as a control device for a keyless entry system, a smart key system, an electric window device, or various lights such as headlights, taillights, brake lights, direction lights, or fog lights. In this case, radio waves or signals from various switches emitted from a portable device that replaces the key can be input to the vehicle body system control unit 12020. The vehicle body system control unit 12020 accepts the input of such radio waves or signals and controls the door lock device, electric window device, lights, etc. of the vehicle.

The vehicle exterior information detection unit 12030 detects information outside the vehicle equipped with the vehicle control system 12000. For example, the camera unit 12031 is connected to the vehicle exterior information detection unit 12030. The vehicle exterior information detection unit 12030 causes the camera unit 12031 to capture images of the exterior of the vehicle and receives the captured images. The off-vehicle information detection unit 12030 can perform object detection processing or distance detection processing of people, vehicles, obstacles, signs, or text on the road based on the received images.

The imaging unit 12031 is a photo sensor that receives light and outputs an electrical signal corresponding to the received amount of light. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. In addition, the light received by the imaging unit 12031 may be visible light or invisible light such as infrared rays.

The in-vehicle information detection unit 12040 detects the information in the vehicle. For example, the in-vehicle information detection unit 12040 is connected to a driver status detection unit 12041 that detects the driver's status. The driver's state detection unit 12041 includes, for example, a camera that takes pictures of the driver. The in-vehicle information detection unit 12040 can calculate the driver's fatigue or concentration based on the detection information input from the driver's state detection unit 12041, and can also determine driving Whether the person dozed off.

The microcomputer 12051 can calculate the control target value of the driving force generating device, the steering mechanism or the braking device based on the information inside and outside the vehicle obtained by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, and output a control command to the drive system control unit 12010 . For example, the microcomputer 12051 can implement ADAS (Advanced Driver Assistance Systems, advanced driving) including vehicle collision avoidance or impact mitigation, following driving based on vehicle distance, vehicle speed maintenance, vehicle collision warning, or vehicle lane departure warning, etc. The function of the auxiliary system is coordinated control for the purpose.

In addition, the microcomputer 12051 controls the driving force generating device, the steering mechanism, the braking device, etc. based on the information around the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, thereby making it possible to perform operations that do not rely on the driver. Coordinated control for the purpose of autonomous driving and autonomous driving.

In addition, the microcomputer 12051 can output control instructions to the vehicle body system control unit 12020 based on the information outside the vehicle obtained by the vehicle outside information detection unit 12030. For example, the microcomputer 12051 can perform coordinated control for the purpose of anti-glare, such as controlling the headlights based on the position of the vehicle ahead or the oncoming vehicle detected by the vehicle exterior information detection unit 12030, and switching the high beam to low beam. .

The sound and image output unit 12052 sends an output signal of at least one of sound and image to an output device capable of visually or audibly notifying information to passengers of the vehicle or outside the vehicle. In the example of FIG. 41 , an audio speaker 12061, a display unit 12062, and an instrument panel 12063 are illustrated as output devices. The display portion 12062 may include, for example, at least one of a vehicle-mounted display and a head-up display.

FIG. 42 is a diagram showing an example of the installation position of the imaging unit 12031.

In FIG. 42 , imaging units 12101, 12102, 12103, 12104, and 12105 are provided as the imaging unit 12031.

The camera units 12101, 12102, 12103, 12104, and 12105 are, for example, disposed at positions such as the front bumper, rear mirror, rear bumper, tailgate, and upper part of the windshield in the cabin of the vehicle 12100. The camera unit 12101 provided on the front bumper and the camera unit 12105 provided on the upper part of the windshield in the vehicle cabin mainly acquire images of the front of the vehicle 12100. The camera units 12102 and 12103 provided in the rear view mirror mainly acquire images of the side of the vehicle 12100. The camera unit 12104 provided in the rear bumper or tailgate mainly acquires images of the rear of the vehicle 12100 . The camera unit 12105 provided on the upper part of the windshield in the car is mainly used to detect vehicles or pedestrians in front, obstacles, traffic signals, traffic signs or lane lines, etc.

Furthermore, FIG. 42 shows an example of the imaging range of the imaging units 12101 to 12104. The camera range 12111 represents the camera range of the camera unit 12101 provided on the front bumper, the camera ranges 12112 and 12113 represent the camera ranges of the camera units 12102 and 12103 respectively provided on the rear view mirror, and the camera range 12114 represents the camera range provided on the rear bumper or tailgate. The camera range of the door camera unit 12104. For example, by overlapping the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of imaging elements, or may be an imaging element having pixels for phase difference detection.

For example, the microcomputer 12051 obtains the distance to each three-dimensional object within the imaging range 12111 to 12114 and the temporal change of the distance (relative speed to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. , and in particular, the nearest three-dimensional object located on the traveling path of the vehicle 12100 and traveling in approximately the same direction as the vehicle 12100 at a specific speed (for example, 0 km/h or more) can be captured as the preceding vehicle. Furthermore, the microcomputer 12051 can set the distance between vehicles ahead that should be ensured in advance, and perform automatic braking control (including stop following control), automatic acceleration control (including following start control), etc. In this way, coordinated control can be carried out for the purpose of autonomous driving that does not rely on the driver's operation.

For example, the microcomputer 12051 can classify the three-dimensional object data related to the three-dimensional object into other three-dimensional objects such as motorcycles, ordinary vehicles, large vehicles, pedestrians, telephone poles, etc. based on the distance information obtained from the camera units 12101 to 12104. to automatically avoid obstacles. For example, the microcomputer 12051 can identify obstacles around the vehicle 12100 as obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Furthermore, the microcomputer 12051 determines the collision risk indicating the risk of collision with each obstacle, and when encountering a situation where the collision risk is above a set value and a collision is likely to occur, it outputs an output to the driver through the audio speaker 12061 or the display unit 12062 Alarm, or forced deceleration or avoidance steering through the drive system control unit 12010, and driving support for avoiding collisions can be performed.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can identify pedestrians by determining whether there are pedestrians in the captured images of the imaging units 12101 to 12104. Such identification of pedestrians is performed by, for example, a program that captures feature points of the image captured by the imaging units 12101 to 12104 of the infrared camera, and performs pattern matching processing on a series of feature points representing the outline of the object to determine whether the pedestrian is a pedestrian. proceed with the procedure. When the microcomputer 12051 determines that there is a pedestrian in the image captured by the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 displays a square outline for emphasis overlaid on the recognized pedestrian. Control display unit 12062. In addition, the audio image output unit 12052 may control the display unit 12062 to display an icon or the like showing pedestrians at a desired position.

The above has described an example of a mobile body control system to which the technology of the present disclosure is applicable. The technology of the present disclosure can be applied to, for example, the imaging unit 12031 in the structure described above. By applying the technology of the present invention to the imaging unit 12031, a photographic image that is easier to observe can be obtained, thereby reducing driver fatigue.

6. Application examples of endoscopic surgery system The technology disclosed herein (the technology) can be applied to a variety of products. For example, the technology of the present invention may be applied to endoscopic surgical systems.

FIG. 43 is a diagram showing an example of the schematic configuration of an endoscopic surgery system to which the technology of the present disclosure (the present technology) can be applied.

In FIG. 43 , the operator (physician) 11131 uses the endoscopic surgery system 11000 to perform surgery on the patient 11132 on the hospital bed 11133 . As shown in the figure, the endoscopic surgery system 11000 includes: an endoscope 11100, a tracheostomy tube 11111 or an energy treatment device 11112 and other surgical instruments 11110, a support arm device 11120 that supports the endoscope 11100, and a device for endoscopic surgery. Trolley 11200 with various devices for microscopic surgery.

The endoscope 11100 includes: a lens barrel 11101, a region of a specific length from the front end of which is inserted into the body cavity of the patient 11132; and a camera head 11102, which is connected to the base end of the lens barrel 11101. In the illustrated example, the endoscope 11100 is configured as a so-called rigid endoscope having a rigid barrel 11101. However, the endoscope 11100 may also be configured as a so-called flexible endoscope having a soft barrel 11101.

The front end of the lens barrel 11101 is provided with an opening in which an objective lens is embedded. A light source device 11203 is connected to the endoscope 11100. The light generated by the light source device 11203 is guided to the front end of the lens barrel by a light guide extending inside the lens barrel 11101, and is directed to the body cavity of the patient 11132 through the objective lens. Observe subject irradiation. Furthermore, the endoscope 11100 can be a straight-view mirror, a strabismus mirror or a side-view mirror.

An optical system and an imaging element are provided inside the camera head 11102, and the reflected light (observation light) from the observation object is collected by the optical system on the imaging element. The imaging element photoelectrically converts the observation light to generate an electrical signal corresponding to the observation light, that is, an image signal corresponding to the observation image. This image signal is sent to the camera control unit (CCU: Camera Control Unit) 11201 as RAW data.

The CCU 11201 includes a CPU (Central Processing Unit, central processing unit) or a GPU (Graphics Processing Unit, graphics processing unit), etc., and collectively controls the operations of the endoscope 11100 and the display device 11202. Furthermore, the CCU 11201 receives an image signal from the camera head 11102 and performs various image processing on the image signal for displaying an image based on the image signal, such as development processing (demosaic processing).

The display device 11202 displays an image based on an image signal subjected to image processing by the CCU 11201 under control from the CCU 11201 .

The light source device 11203 includes, for example, a light source such as an LED (Light Emitting Diode), and supplies irradiation light when photographing a surgical site or the like to the endoscope 11100 .

The input device 11204 is an input interface for the endoscopic surgery system 11000. The user can input various information or instructions to the endoscopic surgery system 11000 through the input device 11204. For example, the user inputs an instruction to change the imaging conditions of the endoscope 11100 (type of irradiation light, magnification, focal length, etc.).

The treatment tool control device 11205 controls the driving of the energy treatment tool 11112 used for cauterization, incision of tissue, sealing of blood vessels, etc. For the purpose of ensuring the field of view of the endoscope 11100 and ensuring the operating space of the operator, the insufficiency device 11206 inflates the body cavity of the patient 11132, and sends gas into the body cavity through the insufficiency tube 11111. A device for printing in a form. The recorder 11207 is a device that can record various information related to surgery. Printer 11208 can convert various information related to surgery into text, images, charts, etc.

In addition, the light source device 11203 that provides illumination light to the endoscope 11100 when photographing the surgical site may include, for example, a white light source composed of an LED, a laser light source, or a combination thereof. When a white light source is constituted by a combination of RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high precision, so the white balance of the captured image can be adjusted in the light source device 11203. Furthermore, in this case, by irradiating the observation object with laser light from each of the RGB laser light sources in a time-sharing manner, and controlling the drive of the imaging element of the camera head 11102 in synchronization with the irradiation timing, it is also possible to capture and photograph in a time-sharing manner. RGB corresponding images. According to this method, a color image can be obtained even if the imaging element is not provided with a color filter.

In addition, the light source device 11203 can control the driving in such a manner that the intensity of the light output is changed every specific time. By controlling the drive of the imaging element of the camera head 11102 in synchronization with the timing of changes in the intensity of light, acquiring images in a time-sharing manner, and synthesizing the images, a high dynamic range without underexposure and overexposure can be generated. image.

In addition, the light source device 11203 may be configured to supply light of a specific wavelength band corresponding to special light observation. In special light observation, for example, by utilizing the wavelength dependence of light absorption by biological tissues, light with a narrower frequency band is irradiated compared to the irradiation light (that is, white light) during normal observation, and high Endoscopic narrow-band imaging (Narrow Band Imaging) is a technique that captures specific tissues such as blood vessels on the mucosal surface with contrast. Alternatively, in special light observation, fluorescence observation using fluorescence generated by irradiation with excitation light to obtain an image can be performed. In fluorescence observation, the biological tissue can be irradiated with excitation light to observe the fluorescence from the biological tissue (autofluorescence observation), or a reagent such as indocyanine green (ICG) can be locally injected into the biological tissue and the The biological tissue is irradiated with excitation light corresponding to the fluorescence wavelength of the reagent to obtain a fluorescence image or the like. The light source device 11203 may be configured to provide narrow-band light and/or excitation light corresponding to such special light observation.

FIG. 44 is a block diagram showing an example of the functional configuration of the camera head 11102 and CCU 11201 shown in FIG. 43 .

The camera head 11102 includes a lens unit 11401, an imaging unit 11402, a drive unit 11403, a communication unit 11404, and a camera head control unit 11405. The CCU 11201 includes a communication unit 11411, an image processing unit 11412, and a control unit 11413. The camera head 11102 and the CCU 11201 are communicatively connected by a transmission cable 11400.

The lens unit 11401 is an optical system provided at a connection portion with the lens barrel 11101. The observation light captured from the front end of the lens barrel 11101 is guided to the camera head 11102 and then incident on the lens unit 11401. The lens unit 11401 is configured by combining a plurality of lenses including a zoom lens and a focus lens.

The imaging element constituting the imaging unit 11402 may be one (so-called single-board type) or a plurality of imaging elements (so-called multi-board type). When the imaging unit 11402 has a multi-plate structure, for example, a color image can be obtained by using each imaging element to generate an image signal corresponding to each of RGB and combining them. Alternatively, the imaging unit 11402 may be configured to include a pair of imaging elements for respectively acquiring right-eye and left-eye image signals corresponding to 3D (Dimensional) display. By performing 3D display, the operator 11131 can more accurately grasp the depth of the biological tissue at the surgical site. In addition, when the imaging unit 11402 is configured of a multi-plate type, a plurality of lens units 11401 may be provided corresponding to each imaging element.

In addition, the imaging unit 11402 does not necessarily need to be provided in the camera head 11102. For example, the imaging unit 11402 may be provided inside the lens barrel 11101 directly behind the objective lens.

The driving part 11403 is composed of an actuator, and moves the zoom lens and the focus lens of the lens unit 11401 by a specific distance along the optical axis under control from the camera head control part 11405. Thereby, the magnification and focus of the captured image captured by the imaging unit 11402 can be appropriately adjusted.

The communication unit 11404 is composed of a communication device for sending and receiving various information to and from the CCU 11201. The communication unit 11404 sends the image signal obtained from the camera unit 11402 as RAW data to the CCU 11201 via the transmission cable 11400.

In addition, the communication unit 11404 receives a control signal for controlling the driving of the camera head 11102 from the CCU 11201 and supplies it to the camera head control unit 11405. The control signal includes, for example, information that specifies the frame rate of the captured image, information that specifies the exposure value when capturing, and/or information that specifies the magnification and focus of the captured image, and the imaging conditions. Related information.

In addition, the above-mentioned imaging conditions such as frame rate, exposure value, magnification, and focus can be appropriately specified by the user, or can be automatically set by the control unit 11413 of the CCU 11201 based on the acquired image signal. If it is the latter, the endoscope 11100 needs to be equipped with the so-called AE (Auto Exposure, automatic exposure) function, AF (Auto Focus, automatic focus) function and AWB (Auto White Balance, automatic white balance) function.

The camera head control unit 11405 controls the driving of the camera head 11102 based on the control signal received from the CCU 11201 via the communication unit 11404.

The communication unit 11411 is composed of a communication device for sending and receiving various information to and from the camera head 11102 . The communication unit 11411 receives the image signal sent from the camera head 11102 via the transmission cable 11400.

Furthermore, the communication unit 11411 sends a control signal for controlling the drive of the camera head 11102 to the camera head 11102 . Image signals or control signals can be sent through electrical communication or optical communication.

The image processing unit 11412 performs various image processing on the image signal sent as RAW data from the own camera 11102 .

The control unit 11413 performs various controls related to the imaging of the surgical site and the like by the endoscope 11100 and the display of the captured image obtained by imaging the surgical site and the like. For example, the control unit 11413 generates a control signal for controlling the driving of the camera head 11102 .

Furthermore, the control unit 11413 causes the display device 11202 to display the captured image showing the surgical site or the like based on the image signal processed by the image processing unit 11412 . At this time, the control unit 11413 can use various image recognition technologies to identify various objects in the captured image. For example, by detecting the shape or color of the edges of objects included in the captured image, the control unit 11413 can identify surgical instruments such as forceps, specific biological parts, bleeding, mist when using the energy treatment tool 11112, etc. . When causing the display device 11202 to display the captured image, the control unit 11413 may use the recognition result to display various surgical support information overlayed on the image of the surgical site. By overlaying and displaying the surgical support information, the surgeon 11131 is reminded, thereby reducing the burden on the surgeon 11131 and allowing the surgeon 11131 to perform the surgery accurately.

The transmission cable 11400 connecting the camera head 11102 and the CCU 11201 may be an electrical signal cable corresponding to the communication of electrical signals, an optical fiber corresponding to the optical communication, or a composite cable thereof.

Here, in the example shown in the figure, the transmission cable 11400 can be used for wired communication, but the communication between the camera head 11102 and the CCU 11201 can also be performed wirelessly.

The above has described an example of the endoscopic surgery system to which the technology of the present disclosure can be applied. The technology of the present disclosure can be applied to the imaging unit 11402 of the camera head 11102 in the structure described above. By applying the technology of the present disclosure to the connector 11102, a clearer image of the surgical site can be obtained, so the operator can accurately confirm the surgical site.

In addition, here, as an example, the endoscopic surgery system is described, but the technology of the present disclosure can also be applied to, for example, a microscope surgery system.

The embodiments of the present disclosure have been described above. However, the technical scope of the present disclosure is not limited to the above-described embodiment itself, and various changes can be made within the scope that does not deviate from the gist of the present disclosure. In addition, the constituent elements of different embodiments and modifications can be appropriately combined.

In addition, the effects of each embodiment described in this specification are only illustrative in the final analysis, and are not limiting effects, and may have other effects.

In addition, this technology may also adopt the following configuration. (1) A solid-state imaging device including a pixel array section composed of a plurality of pixels arranged in a matrix, each of the plurality of pixels including a plurality of first pixels that photoelectrically convert light of a first wavelength component, and The aforementioned pixels include: a photoelectric conversion part that photoelectrically converts incident light; A transfer transistor that controls the transfer of charges generated in the aforementioned photoelectric conversion part; a floating diffusion region that accumulates the aforementioned charge transferred from the aforementioned photoelectric conversion portion through the aforementioned transfer transistor; and an amplifying transistor that causes a voltage signal corresponding to the electric charge accumulated in the floating diffusion region to appear on the signal line; The plurality of first pixels are arranged in the first oblique direction in the pixel array portion; At least two first pixels among the plurality of first pixels arranged in the first oblique direction share one floating diffusion area. (2) The solid-state imaging device of (1) above, wherein the pixel array section includes a plurality of combinations of at least two first pixels that share one floating diffusion area; and The combination of the at least two first pixels is regularly arranged in the pixel array portion. (3) The solid-state imaging device according to the above (1) or (2), wherein the floating diffusion area is arranged between the at least two first pixels sharing the floating diffusion area. (4) The solid-state imaging device according to any one of the above (1) to (3), wherein the plurality of pixels respectively include a plurality of second pixels that photoelectrically convert light of a second wavelength component that is different from the first wavelength component; and The plurality of second pixels are arranged in the row direction in the pixel array section; Among the plurality of second pixels, at least two second pixels arranged every other pixel in the row direction share one floating diffusion area. (5) The solid-state imaging device of (4) above, wherein the at least two first pixels share the one floating diffusion area, and the floating diffusion area is the same as the at least two second pixels sharing the one floating diffusion area. (6) The solid-state imaging device of (4) or (5) above further includes a processing unit that rearranges the pixel arrangement of the image data read from the pixel array unit. (7) The solid-state imaging device according to any one of the above (1) to (6), further including a correction unit that corrects the image value read from each of the plurality of pixels. (8) The solid-state imaging device according to any one of the above (1) to (7) further includes a drive circuit that drives the plurality of pixels to start exposure of the plurality of pixels simultaneously. (9) The solid-state imaging device according to any one of the above (1) to (8), wherein the pixel array section includes a color filter array with a Bayer array as a repeating unit. (10) The solid-state imaging device according to any one of the above (1) to (8), wherein the pixel array section includes a color filter array with a quadruple Bayer array as a repeating unit. (11) The solid-state imaging device according to any one of the above (1) to (8), wherein the pixel array section includes a color filter array with an RGBW array as a repeating unit. (12) The solid-state imaging device according to any one of the above (1) to (3), wherein the plurality of pixels respectively include a plurality of second pixels that photoelectrically convert light of a second wavelength component that is different from the first wavelength component; and The plurality of second pixels are arranged in a second oblique direction intersecting the first oblique direction in the pixel array portion; At least two of the plurality of second pixels arranged in the second oblique direction share one floating diffusion area. (13) The solid-state imaging device of (12) above, wherein the at least two first pixels share the one floating diffusion area, and the floating diffusion area is the same as the at least two second pixels sharing the one floating diffusion area. (14) The solid-state imaging device of (12) or (13) above, wherein the plurality of pixels respectively include a plurality of third pixels that photoelectrically convert light of a third wavelength component that is different from the aforementioned first wavelength component and the aforementioned second wavelength component. pixels; and The plurality of third pixels are arranged in the row direction and the column direction in the pixel array section; Among the plurality of third pixels, at least two third pixels arranged every other pixel in the row direction and column direction share one floating diffusion area. (15) The solid-state imaging device of (12) or (13) above, wherein the plurality of pixels include: the plurality of first pixels, the plurality of second pixels, and the plurality of first wavelength components and the aforementioned second wavelength components respectively. A plurality of third pixels for photoelectric conversion of light with different third wavelength components; and The pixel array section includes a first unit in which one or more first pixels and one or more second pixels are arranged in a matrix, and one or more first pixels and one or more third pixel matrices The second unit of the pattern arrangement is a repeating unit of color filter arrangement. (16) The solid-state imaging device according to any one of the above (1) to (15), further includes: A driving circuit that drives the aforementioned plurality of pixels; and A plurality of driving lines extending from the aforementioned driving circuit and connected to the aforementioned plurality of pixels; and The at least two first pixels arranged in the first oblique direction and sharing the one floating diffusion area are connected to the same driving line. (17) An electronic machine having: The solid-state imaging device according to any one of the above (1) to (16); and A processor that performs specific processing on image data output from the solid-state imaging device. (18) A solid-state camera device having: A pixel array section, which has a plurality of pixels arranged in a matrix; A driving circuit that drives the aforementioned plurality of pixels; and A plurality of driving lines extending from the aforementioned driving circuit and connected to the aforementioned plurality of pixels; and The aforementioned pixels include: a photoelectric conversion part that photoelectrically converts incident light; A transfer transistor that controls the transfer of charges generated in the aforementioned photoelectric conversion part; a floating diffusion region that accumulates the aforementioned charge transferred from the aforementioned photoelectric conversion portion through the aforementioned transfer transistor; and an amplifying transistor that causes a voltage signal corresponding to the electric charge accumulated in the floating diffusion region to appear on the signal line; The aforementioned plurality of pixels includes a plurality of first pixels and a plurality of second pixels; The first pixel and the second pixel are respectively arranged in the oblique direction in the pixel array portion; The first pixels arranged in two adjacent columns in the pixel array section are connected to the same first driving line; The second pixels arranged in two adjacent columns in the pixel array section are connected to the same second driving line that is different from the first driving line. (19) The solid-state imaging device of (18) above, wherein the drive circuit sets the charge accumulation time of the first pixel before the first drive line is connected to the first period, and sets the charge accumulation time of the second pixel before the second drive line is connected to the first period. The charge accumulation time is a second period that is different from the first period. (20) The solid-state imaging device of (18) above, wherein the first pixel photoelectrically converts the light of the first wavelength component; and The second pixel photoelectrically converts light with a second wavelength component that is different from the first wavelength component. (twenty one) The solid-state imaging device according to any one of the above (18) to (20), wherein each of the two first pixels arranged in the oblique direction among the first pixels arranged in the two columns has one floating diffusion in total. area; and Each of the two second pixels arranged in the oblique direction among the second pixels arranged in the two columns has one floating diffusion area. (twenty two) The solid-state imaging device of (21) above, wherein the two first pixels share the one floating diffusion area, and the two second pixels share the same floating diffusion area. (twenty three) The solid-state imaging device according to any one of (18) to (22) above, further including a processing unit that rearranges the pixel arrangement of the image data read from the pixel array unit. (twenty four) The solid-state imaging device according to any one of (18) to (23) above, further including a correction unit that corrects the image value read from each of the plurality of pixels. (25) An electronic machine having: The solid-state imaging device according to any one of the above (18) to (24); and A processor that performs specific processing on image data output from the solid-state imaging device.

1: Electronic equipment (camera device) 10: Solid-state imaging device (image sensor) 11: Imaging lens 13: Processor 14: Memory unit 21, 21A, 21B, 221, 221A, 221B, 221C, 221D, 221E, 221F, 321, 321A, 321B, 321C, 321D, 321E: Pixel array section 22: Vertical drive circuit 23: Line processing circuit 23a: AD conversion circuit 24: Horizontal drive circuit 25: System control section 26: Signal processing section 27: Data storage section 30 ,30A,100,B ₀ ,B ₁ ,G ₀ ,G ₁ ,R _0, R ₁ :pixel 30B:B pixel 30G:G pixel 30IR:IR pixel 30L:long storage pixel 30R:R pixel 30S:short storage pixel /Pixels 30W, 30WL, 30WS: W pixels 31, 31-1~31-4, 112: Transmission transistor 32: Reset transistor 33: Amplification transistor 34, 131, 132: Selection transistor 41: Light-receiving chip 42: Circuit chip 50A, 50B : Basic unit 50ab, 50ar, 50arw, 50abw, 50b, 50b4, 50b5, 50b5w, 50bir, 50bw, 50g5w, 50gir, 50gw, 50r, 50r4, 50r4w, 50r5, 50r5w, 50rir, 50rw: Total unit 51: Crystal-loaded lens 52: Color filter 53: Planarizing film 54: Light-shielding film 55, 63: Insulating film 56, 64: P-type semiconductor region 57: Light-receiving surface 58: Semiconductor substrate 59: N-type semiconductor region 60: Pixel separation part 61: Groove 62: Fixed charge film 65: Wiring layer 66: Wiring 67: Insulating layer 101-1~101-n: Column memory 102: Rearrangement mosaic processing unit 110: Front-end circuit 113: FD reset transistor 115: Front-end Amplification transistor 116: current source transistor 120: front-stage node 121, 122: capacitor element 130: selection circuit 140: rear-stage node 141: rear-stage reset transistor 150: rear-stage circuit 151: rear-stage amplification transistor 152: rear-stage selection transistor 302, 302a :Circuit composition 900:Smartphone 901:CPU 902:ROM 903:RAM 904:Storage device 905:Communication module 906:Communication network 907:Sensor module 910:Display device 911:Speaker 912:Microphone 913:Input Device 914: Bus 11000: Endoscopic surgery system 11100: Endoscope 11101: Lens barrel 11102: Camera head 11110: Surgical instrument 11111: Verestroperitoneum tube 11112: Energy disposal instrument 11120: Support arm device 11131: Operator (doctor) ) 11132: Patient 11133: Hospital bed 11200: Trolley 11201: Camera control unit/CCU 11202: Display device 11203: Light source device 11204: Input device 11205: Treatment device control device 11206: Insemination device 11207: Recorder 11208: Printer 11400 :Transmission cable 11401: Lens unit 11402, 12031, 12101, 12102, 12103, 12104, 12105: Camera section 11403: Drive section 11404: Communication section 11405: Camera head control section 11411: Communication section 11412: Image processing section 11413: Control unit 12000: Vehicle control system 12001: Communication network 12010: Drive system control unit 12020: Vehicle body system control unit 12030: Outside vehicle information detection unit 12040: Inside vehicle information detection unit 12041: Driver status detection unit 12050: Integrated control unit 12051 : Microcomputer 12052: Sound and image output unit 12053: Vehicle network I/F 12061: Audio speaker 12062: Display unit 12063: Instrument panel 12100: Vehicle 12111, 12112, 12113, 12114: Camera range B: Blue FD: Floating diffusion Area G: Green id1: Current L1: Incident light LD: Pixel drive line/horizontal signal line LD31: Transfer transistor drive line LD32: Reset transistor drive line LD34: Select transistor drive line PD, PD1~PD4: Photoelectric conversion Part R: red RST: reset signal rst: FD reset signal rstb: rear section reset signal SEL: selection control signal selb: rear section selection signal TRG: transmission signal/transmission control signal trg: transmission signal VDD: power supply voltage VRD: vertical Reset input line Vreg: potential VSL: vertical signal line VCOM: vertical current supply line W: white ϕr, ϕs: selection signal

FIG. 1 is a block diagram showing a schematic configuration example of an electronic device equipped with the solid-state imaging device according to the first embodiment of the present disclosure. FIG. 2 is a block diagram showing a schematic configuration example of the CMOS solid-state imaging device according to the first embodiment of the present disclosure. FIG. 3 is a circuit diagram showing an example of a schematic configuration of a pixel according to the first embodiment of the present disclosure. FIG. 4 is a circuit diagram showing a schematic configuration example of a pixel having the FD common structure according to the first embodiment of the present disclosure. FIG. 5 is a diagram showing an example of a multilayer structure of the image sensor according to the first embodiment of the present disclosure. FIG. 6 is a cross-sectional view showing an example of a basic cross-sectional structure of a pixel according to the first embodiment of the present disclosure. 7 is a top view showing a pixel arrangement example of the first embodiment of the present disclosure in which a Bayer arrangement is adopted as a color filter arrangement. FIG. 8 is a top view showing a pixel arrangement example of the first embodiment of the present disclosure in which a quadruple Bayer arrangement is used as the color filter arrangement. 9(A) and (B) are diagrams for explaining pixel addition of a conventional image sensor using the Bayer arrangement. FIGS. 10(A) and 10(B) are diagrams illustrating an example of a combination of addition pixels according to the first embodiment of the present disclosure. FIG. 11 is a diagram illustrating a pixel addition calculation example in the case of a shared structure of 8 pixels. FIG. 12 is a diagram illustrating a pixel addition calculation example in the case of a 4-pixel shared structure. FIG. 13 is a diagram illustrating a configuration example of a common unit according to the first embodiment of the present disclosure. FIGS. 14(A) to 14(C) are diagrams for explaining the rearrangement mosaic processing according to the first embodiment of the present disclosure. FIG. 15 is a block diagram showing a structure for executing the rearrangement mosaic processing according to the first embodiment of the present disclosure. FIG. 16 is a diagram for explaining the correction process in the first embodiment of the present disclosure. FIG. 17 is a circuit diagram showing an example of the schematic configuration of a pixel using the VDGS (Voltage Domain Global Shutter) method, which is one of the global shutter methods according to the first embodiment of the present disclosure. FIG. 18 is a diagram showing an example of a circuit configuration in a case where pixels are shared by basic units in a Bayer arrangement. FIG. 19 is a diagram showing an example of a circuit configuration in the case of sharing pixels constituting the first embodiment of the present disclosure. 20(A) and (B) are diagrams showing an example of a common unit applicable to an image sensor using an RGBW arrangement (W is white) according to the first embodiment of the present disclosure. 21 is a top view showing a pixel arrangement example using a color filter arrangement and a pixel sharing structure according to the second embodiment of the present disclosure. 22(A) and 22(B) are diagrams illustrating a combination example of addition pixels according to the second embodiment of the present disclosure. 23 (A) and (B) are plan views showing a pixel arrangement example using a color filter arrangement and a pixel sharing structure according to the first variation of the second embodiment of the present disclosure. 24(A) and (B) are plan views showing a pixel arrangement example using a color filter arrangement and a pixel sharing structure according to the second variation of the second embodiment of the present disclosure. FIG. 25 is a top view (part 1) showing a pixel arrangement example of a color filter arrangement and a pixel sharing structure according to the third variation of the second embodiment of the present disclosure. FIG. 26 is a top view (Part 2) showing a pixel arrangement example of a color filter arrangement and a pixel sharing structure using a third variation of the second embodiment of the present disclosure. FIG. 27 is a top view (part 3) showing a pixel arrangement example of a color filter arrangement and a pixel sharing structure using a third variation of the second embodiment of the present disclosure. FIG. 28 is a top view (Part 4) showing a pixel arrangement example of a color filter arrangement and a pixel sharing structure using a third variation of the second embodiment of the present disclosure. FIG. 29 is a diagram illustrating shutter control of an image sensor using a rolling shutter method. Figure 30 is a diagram showing a connection example of pixel driving lines of an image sensor using a rolling shutter method. FIG. 31 is a diagram showing another connection example of pixel driving lines of an image sensor using a rolling shutter method. FIG. 32 is a diagram showing a connection example of pixel driving lines according to the third embodiment of the present disclosure. FIG. 33 is a diagram showing another connection example of pixel driving lines according to the third embodiment of the present disclosure. 34(A) to 34(C) are block diagrams showing a structure for executing the rearrangement mosaic processing according to the third embodiment of the present disclosure. FIG. 35 is a top view showing a pixel arrangement example according to the third embodiment of the present disclosure. FIG. 36 is a top view (part 1) showing another pixel arrangement example according to the third embodiment of the present disclosure. FIG. 37 is a top view (part 2) showing another pixel arrangement example according to the third embodiment of the present disclosure. FIG. 38 is a top view (Part 3) showing yet another pixel arrangement example according to the third embodiment of the present disclosure. FIG. 39 is a top view (Part 4) showing yet another pixel arrangement example according to the third embodiment of the present disclosure. FIG. 40 is a block diagram showing an example of a schematic functional configuration of a smartphone. FIG. 41 is a block diagram showing an example of the schematic configuration of the vehicle control system. FIG. 42 is an explanatory diagram showing an example of the installation positions of the vehicle exterior information detection unit and the camera unit. FIG. 43 is a diagram showing an example of the schematic configuration of the endoscopic surgery system. Figure 44 is a block diagram showing an example of the functional configuration of the camera head and CCU.

21A: Pixel array section

30B:B pixel

30G:G pixels

30R:R pixels

50ab,50ar:shared units

FD: floating diffusion area

VSL: vertical signal line

Claims (17)

A solid-state imaging device including a pixel array section composed of a plurality of pixels arranged in a matrix, each of the plurality of pixels including a plurality of first pixels that photoelectrically convert light of a first wavelength component, and The aforementioned pixels include: a photoelectric conversion part that photoelectrically converts incident light; A transfer transistor that controls the transfer of charges generated in the aforementioned photoelectric conversion part; a floating diffusion region that accumulates the aforementioned charge transferred from the aforementioned photoelectric conversion portion through the aforementioned transfer transistor; and an amplification transistor that causes a voltage signal corresponding to the electric charge accumulated in the floating diffusion region to appear on the signal line; and The plurality of first pixels are arranged in the first oblique direction in the pixel array portion; At least two first pixels among the plurality of first pixels arranged in the first oblique direction have a total of one floating diffusion area. The solid-state imaging device of claim 1, wherein the pixel array section includes a plurality of combinations of at least two first pixels that share one floating diffusion area; and The combination of the at least two first pixels is regularly arranged in the pixel array portion. The solid-state imaging device of claim 1, wherein the floating diffusion area is arranged between the at least two first pixels that share the floating diffusion area. The solid-state imaging device of claim 1, wherein the plurality of pixels each include a plurality of second pixels that photoelectrically convert light of a second wavelength component that is different from the first wavelength component; and The plurality of second pixels are arranged in the row direction in the pixel array section; Among the plurality of second pixels, at least two second pixels arranged every other pixel in the row direction have a total of one floating diffusion area. The solid-state imaging device of Claim 4, wherein the at least two first pixels share the one floating diffusion area, and the floating diffusion area is the same as the at least two second pixels sharing the one floating diffusion area. The solid-state imaging device of claim 4 further includes a processing unit that rearranges the pixel arrangement of the image data read from the pixel array unit. The solid-state imaging device of claim 1 further includes a correction unit that corrects the image value read from each of the plurality of pixels. The solid-state imaging device of claim 1 further includes a drive circuit that drives the plurality of pixels to simultaneously start exposure of the plurality of pixels. The solid-state imaging device according to claim 1, wherein the pixel array section includes a color filter array with a Bayer array as a repeating unit. The solid-state imaging device according to claim 1, wherein the pixel array section includes a color filter array with a repeating unit of a quadruple Bayer array. The solid-state imaging device according to claim 1, wherein the pixel array section includes a color filter array with an RGBW array as a repeating unit. The solid-state imaging device of claim 1, wherein the plurality of pixels each include a plurality of second pixels that photoelectrically convert light of a second wavelength component that is different from the first wavelength component; and The plurality of second pixels are arranged in a second oblique direction intersecting the first oblique direction in the pixel array portion; At least two of the plurality of second pixels arranged in the second oblique direction have a total of one floating diffusion area. The solid-state imaging device of claim 12, wherein the at least two first pixels share the one floating diffusion area, and the floating diffusion area is the same as the at least two second pixels share the one floating diffusion area. The solid-state imaging device of claim 12, wherein the plurality of pixels respectively include a plurality of third pixels that photoelectrically convert light of a third wavelength component that is different from the aforementioned first wavelength component and the aforementioned second wavelength component; and The plurality of third pixels are arranged in the row direction and the column direction in the pixel array portion; Among the plurality of third pixels, at least two third pixels arranged every other pixel in the row direction and column direction have a total of one floating diffusion area. The solid-state imaging device of claim 12, wherein the plurality of pixels include: the plurality of first pixels, the plurality of second pixels, and a third wavelength that is different from the first wavelength component and the second wavelength component respectively. A plurality of third pixels that perform photoelectric conversion of component light; and The pixel array section includes a first unit in which one or more first pixels and one or more second pixels are arranged in a matrix, and one or more first pixels and one or more third pixels. The second unit of the matrix arrangement is a repeating unit of color filter arrangement. The solid-state imaging device of claim 1 further includes: A driving circuit that drives the aforementioned plurality of pixels; and A plurality of driving lines extending from the aforementioned driving circuit and connected to the aforementioned plurality of pixels; and The at least two first pixels arranged in the first oblique direction and sharing the one floating diffusion area are connected to the same driving line. An electronic machine containing: The solid-state imaging device of claim 1; and A processor that performs specific processing on image data output from the solid-state imaging device.