JP2014175992A - Solid state imaging device and imaging apparatus using the same - Google Patents

Abstract

PROBLEM TO BE SOLVED: To achieve at least one of gradual expansion of a dynamic range, natural pixel addition using photoelectric conversion parts without waste and more suitable pixel interpolation.SOLUTION: A solid state imaging device includes: a plurality of first photoelectric conversion parts PDH (PDHr, PDHg, PDHb) arrayed like a square lattice so as to be arranged in a first direction and a second direction orthogonal to the first direction; and second photoelectric conversion parts PDL (PDLr, PDLg, PDLb) configured so that when setting respective positions shifted from positions of the plurality of first photoelectric conversion parts PDH in the first and second directions by a half of a pitch P of the plurality of first photoelectric conversion parts PDH (PDHr, PDHg, PDHb) in the first and second directions as reference positions A, four second photoelectric conversion parts PDL are arranged near each reference position A and a light receiving area of the second photoelectric conversion part PDL is smaller than that of the first photoelectric conversion part PDH.

Description

  The present invention relates to a solid-state imaging device and an imaging apparatus using the same.

  2 (a) and 2 (b) of Patent Document 1 below show a plurality of first photoelectric conversions arranged in a square lattice so as to be aligned in the first direction and the second direction orthogonal thereto. Part (light receiving element) and the first direction of the plurality of first photoelectric conversion units and the second direction from the position of the plurality of first photoelectric conversion units to the first direction and the second direction A solid-state imaging device comprising: a second photoelectric conversion unit (light receiving element) that is arranged one by one at each position shifted by half the pitch in the second direction and has a light receiving area smaller than that of the first photoelectric conversion unit. An element is disclosed.

JP 2000-125209 A

  However, the conventional solid-state imaging device cannot perform dynamic range expansion step by step, cannot perform natural pixel addition using a photoelectric conversion unit without waste, cannot perform appropriate pixel interpolation, etc. Inconvenience occurs.

  The present invention has been made in view of such circumstances, and can perform dynamic range expansion step by step, can perform natural pixel addition using a photoelectric conversion unit without waste, and more appropriately. An object of the present invention is to provide a solid-state imaging device capable of performing simple pixel interpolation and at least one of them, and an imaging device using the same.

  The following aspects are presented as means for solving the problems. The solid-state imaging device according to the first aspect includes a plurality of first photoelectric conversion units arranged in a square lattice so as to be arranged in a first direction and a second direction orthogonal thereto, and the plurality of first photoelectric conversion elements. Each position shifted from the position of the photoelectric conversion unit by half the pitch of the first direction and the second direction of the plurality of first photoelectric conversion units in the first direction and the second direction. Of the first photoelectric conversion unit, the second photoelectric conversion unit being arranged in the vicinity of each reference position and having a light receiving area smaller than that of the first photoelectric conversion unit, and the plurality of first photoelectric conversion units A plurality of first color filters that are arranged in correspondence with each other and have a color array having a repetition period of 2 rows and 2 columns, and the four second photoelectric conversion units arranged in the vicinity of each of the reference positions. 4 corresponding to each of the repetition cycles of the 2 rows and 2 columns. Each said first color filter is obtained with a four second color filters of the same color, the.

  The solid-state imaging device according to the second aspect is configured such that in the first aspect, the signals of the first photoelectric conversion units and the signals of the second photoelectric conversion units can be individually read out. is there.

  In the first or second aspect, the solid-state imaging device according to the third aspect is one for each of the first photoelectric conversion units in the vicinity of the four reference positions adjacent to the first photoelectric conversion unit. The four second photoelectric conversion units arranged one by one, wherein the second color filter having the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit is provided The four second photoelectric conversion units and the signals of two or more photoelectric conversion units among the first photoelectric conversion units can be added and read out.

  In the solid-state imaging device according to the fourth aspect, in any one of the first to third aspects, each of the first photoelectric conversion units is in the vicinity of the four reference positions adjacent to the first photoelectric conversion unit. Each of the four second photoelectric conversion units arranged in the first direction passes through the center of the first photoelectric conversion unit and extends in the first direction, and passes through the center in the second direction. The four second photoelectric elements are arranged at positions that are line-symmetric with respect to the extended second line, and are arranged in the vicinity of the four reference positions close to the first photoelectric conversion unit. The colors of the second color filters provided corresponding to the conversion units are arranged at positions that are line-symmetric with respect to the first line and the second line, respectively.

  The imaging device according to the fifth aspect includes, for the solid-state imaging device according to any one of the first to fourth aspects, and each of the first photoelectric conversion units, the four references that are close to the first photoelectric conversion unit. Four second photoelectric conversion units arranged one by one in the vicinity of the position, wherein the first color filter has the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit. At least one photoelectric conversion unit is selected according to a preset shooting mode from the group of four second photoelectric conversion units provided with two color filters and the first photoelectric conversion unit. A selection unit; and a processing unit that reads a signal from the photoelectric conversion unit of each group selected by the selection unit, performs a process according to a preset shooting mode, and outputs the signal as a pixel signal. .

  The imaging device according to the sixth aspect includes the solid-state imaging device according to any one of the first to fourth aspects and the four reference standards adjacent to the first photoelectric conversion unit, for each of the first photoelectric conversion units. Four second photoelectric conversion units arranged one by one in the vicinity of the position, wherein the first color filter has the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit. A signal obtained by adding signals of one or more second photoelectric conversion units selected according to a command from the four second photoelectric conversion units provided with two color filters, and the first And a means for performing processing for obtaining a pixel signal having a wider dynamic range than a pixel signal based only on the signal of the first photoelectric conversion unit based on the signal of the photoelectric conversion unit of the first photoelectric conversion unit.

  An imaging device according to a seventh aspect includes a solid-state imaging device according to any one of the first to fourth aspects, and (i) for each of the first photoelectric conversion units, according to a signal of the first photoelectric conversion unit. (Ii) each of the first photoelectric conversion units is arranged in the vicinity of one of the four reference positions close to the first photoelectric conversion unit. One of the second photoelectric conversion units, wherein the second color filter having the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit is provided 1 A process according to which a signal corresponding to the signal of each of the two second photoelectric conversion units is used as a pixel signal; (iii) for each of the first photoelectric conversion units, the four reference positions adjacent to the first photoelectric conversion unit One of each is arranged near the reference position. Two second photoelectric conversion units, the second color filter having the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit, are provided. A process in which a signal corresponding to a signal obtained by adding the signals of the two second photoelectric conversion units is used as a pixel signal; (iv) each of the first photoelectric conversion units is close to the first photoelectric conversion unit Three of the second photoelectric conversion units arranged one by one in the vicinity of three of the four reference positions, the first photoelectric conversion unit provided corresponding to the first photoelectric conversion unit A process of converting a signal corresponding to a signal obtained by adding the signals of the three second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter into a pixel signal (v ) For each of the first photoelectric converters, the first photoelectric converter Four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the conversion unit, the first photoelectric conversion units provided corresponding to the first photoelectric conversion units A process of converting a signal corresponding to a signal obtained by adding the signals of the four second photoelectric conversion units provided with the second color filter of the same color as the color filter into a pixel signal; and (vi) For each of the first photoelectric conversion units, there are four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, Signals of the four second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter provided corresponding to the photoelectric conversion unit of the first color filter, A signal corresponding to the signal obtained by adding the signal from the photoelectric converter Processing the pixel signal, it is obtained and a means for performing selectively in accordance with a command of three or more processing among the.

  In the seventh aspect, the imaging device according to the eighth aspect is arranged in the vicinity of the four reference positions in the vicinity of the first photoelectric conversion unit, with respect to each of the first photoelectric conversion units. Four of the second photoelectric conversion units, wherein the second color filter of the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit is provided 4 There is provided means for performing processing for converting a signal corresponding to a signal obtained by adding the signals of the two second photoelectric conversion units and the signal of the first photoelectric conversion unit into a pixel signal.

  An imaging device according to a ninth aspect includes, for the solid-state imaging device according to any one of the first to fourth aspects, and each of the first photoelectric conversion units, the four references that are close to the first photoelectric conversion unit. Four second photoelectric conversion units arranged one by one in the vicinity of the position, wherein the first color filter has the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit. Means for performing processing that uses a signal corresponding to a signal obtained by adding the signal of the four second photoelectric conversion units provided with the two color filters and the signal of the first photoelectric conversion unit as a pixel signal And.

  An imaging apparatus according to a tenth aspect includes the solid-state imaging device according to any one of the first to fourth aspects and at least a part of the first photoelectric conversion units among the plurality of first photoelectric conversion units. Four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, provided corresponding to the first photoelectric conversion unit. In addition, based on a signal corresponding to a signal obtained by adding the signals of the four second photoelectric conversion units provided with the second color filter of the predetermined color different from the color of the first color filter, Interpolation processing means for performing a process of interpolating a signal corresponding to a signal of the first photoelectric conversion unit when it is assumed that the first color filter of the predetermined color is provided corresponding to one photoelectric conversion unit; , With.

  According to the present invention, dynamic range expansion can be performed in stages, natural pixel addition using a photoelectric conversion unit can be performed without waste, and more appropriate pixel interpolation can be performed. A solid-state imaging device capable of realizing at least one of them and an imaging apparatus using the same can be provided.

1 is a schematic block diagram schematically showing an electronic camera according to a first embodiment of the present invention. It is a circuit diagram which shows schematic structure of the solid-state image sensor in FIG. FIG. 3 is a circuit diagram illustrating a pixel block in FIG. 2. It is a schematic plan view which shows typically arrangement | positioning of the photoelectric conversion part of the solid-state image sensor in FIG. It is a figure which shows typically a mode that the signal of one high sensitivity photodiode for every color in FIG. 4 and the four low sensitivity photodiodes of the circumference | surroundings is added. It is a figure which shows typically the relationship between incident light quantity and an output. It is a figure which shows typically the mode of the interpolation of the signal of the red photodiode in FIG. It is a figure which shows typically the mode of the interpolation of the signal of the green photodiode in FIG. It is a figure which shows typically the mode of the interpolation of the signal of the blue photodiode in FIG. It is a schematic plan view which shows typically arrangement | positioning of the photoelectric conversion part of the solid-state image sensor by the 2nd Embodiment of this invention. It is a figure which shows typically a mode that the signal of one high sensitivity photodiode for every color in FIG. 10 and the four low sensitivity photodiodes of the circumference | surroundings is added. It is a figure which shows typically the mode of the interpolation of the signal of the red photodiode in FIG. It is a figure which shows typically the mode of the interpolation of the signal of the blue photodiode in FIG. It is a schematic plan view which shows typically arrangement | positioning of the photoelectric conversion part of the solid-state image sensor by the 3rd Embodiment of this invention.

  Hereinafter, a solid-state imaging device and an imaging apparatus according to the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a schematic block diagram schematically showing an electronic camera 1 as an imaging apparatus according to the first embodiment of the present invention.

  The electronic camera 1 according to the present embodiment is configured as, for example, a single-lens reflex digital camera. However, the imaging apparatus according to the present invention is not limited to this, and is mounted on another electronic camera such as a compact camera or a mobile phone. The present invention can be applied to various imaging devices such as an electronic camera and an electronic camera such as a video camera that captures moving images.

  A photographing lens 2 is attached to the electronic camera 1. The photographing lens 2 is driven by a lens control unit 3 for focus and diaphragm. In the image space of the photographic lens 2, the imaging surface of the solid-state imaging device 4 is arranged.

  The solid-state imaging device 4 is driven by a command from the imaging control unit 5 and outputs a digital image signal. The digital signal processing unit 6 performs image processing such as digital amplification, color interpolation processing, and white balance processing on the digital image signal output from the solid-state imaging device 4. The memory 7 is connected to the bus 8. The bus 8 is also connected with a lens control unit 3, an imaging control unit 5, a CPU 9, a display unit 10 such as a liquid crystal display panel, a recording unit 11, an image compression unit 12 and an image processing unit 13. An operation unit 14 such as a release button or a shooting mode selection dial is connected to the CPU 9. A recording medium 11a is detachably attached to the recording unit 11.

  In the present embodiment, when a release button (not shown) of the operation unit 14 is half-pressed, the CPU 9 in the electronic camera 1 sets the defocus amount based on a detection signal from a focus detection sensor (not shown). The lens control unit 3 adjusts the photographing lens 2 so that the lens is calculated and is brought into focus according to the defocus amount. Further, the CPU 9 causes the lens control unit 3 to adjust the photographing lens 2 so that the aperture is instructed in advance by the operation unit 14. Then, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14, whereby a digital image signal is read from the solid-state imaging device 4. This image signal is processed by the digital signal processing unit 6 and then temporarily stored in the memory 7. Thereafter, the CPU 9 performs desired processing on the image signal in the memory 7 by the image processing unit 13 and the image compression unit 12 as necessary based on the command of the operation unit 14, and the recording unit 11 performs processing after processing. A signal is output and recorded on the recording medium 11a.

  In the present embodiment, the operation unit 14 includes a shooting mode selection dial (not shown) in addition to the release button described above. The operation unit 14 also includes buttons necessary for operating the camera such as a power button and cursor keys. The user operates the electronic camera 1 using the operation unit 14 including these, and the operation information is output to the CPU 9. The CPU 9 controls the operation of the entire electronic camera 1 in accordance with operation information input from the operation unit 14. In the present embodiment, first to eleventh shooting modes to be described later can be selected by operating the shooting mode selection dial.

  FIG. 2 is a circuit diagram showing a schematic configuration of the solid-state imaging device 4 in FIG. In the present embodiment, the solid-state imaging device 4 is configured as a CMOS solid-state imaging device, but may be configured as another XY address-type solid-state imaging device or a CCD-type solid-state imaging device.

  As shown in FIG. 2, the solid-state imaging device 4 includes a pixel unit 22 composed of pixel blocks 21 of n rows and m columns, a vertical scanning circuit 23, and control lines 31 to 37 provided for each row of the pixel blocks 21. A plurality of (m) vertical signal lines 24 that are provided for each column of the pixel blocks 21 and receive signals from the pixel blocks 21 in the corresponding columns, and a constant current source 25 provided in each vertical signal line 24; A CDS circuit (correlated double sampling circuit) 26 and an A / D converter 27 provided corresponding to each vertical signal line 24 and a horizontal readout circuit 28 are provided. Note that the rows and columns of the pixel blocks 21 are represented by the rows and columns of photodiodes PDH of the high-sensitivity pixels 21H described later that are included in each pixel block 21 one by one. 2 and FIG. 3 described later, n indicates a row of the pixel block 21.

  FIG. 3 is a circuit diagram showing the pixel block 21 in FIG. 2 and 3, each pixel block 21 includes one high-sensitivity pixel (relatively high sensitivity pixel) 21H and four low-sensitivity pixels (relatively low sensitivity pixels) 21L1 to 21L4. And have.

  The high-sensitivity pixel 21H includes a high-sensitivity photodiode PDH as a first photoelectric conversion unit that generates and accumulates charges according to incident light, and a charge-voltage conversion unit that receives the charges and converts the charges into voltages. A floating capacitor FD, an amplification transistor AMP as an amplifier that outputs a signal according to the potential of the floating capacitor FD, a transfer transistor TXH that transfers charges from the photodiode PDH to the floating capacitor FD, and a floating capacitor FD 3 has a reset transistor RES for resetting the potential and a selection transistor SEL for selecting a readout row, which are connected as shown in FIG. In FIG. 3, Vdd is a power supply potential.

  The low-sensitivity pixel 21L1 is a low-sensitivity photodiode PDL1 as a second photoelectric conversion unit that generates and accumulates charge according to incident light, and a charge-voltage conversion unit that receives the charge and converts the charge into a voltage. Floating capacitor FD, amplification transistor AMP as an amplifier that outputs a signal corresponding to the potential of floating capacitor FD, transfer transistor TXL1 that transfers charges from photodiode PDH to floating capacitor FD, and floating capacitor FD 3 has a reset transistor RES for resetting the potential and a selection transistor SEL for selecting a readout row, which are connected as shown in FIG. In the low sensitivity pixels 21L2 to 21L4, in the low sensitivity pixel 21L1, the photodiode PDL1 as the second photoelectric conversion unit is replaced with the photodiodes PDL2 to PDL4 as the second photoelectric conversion unit, and the transfer transistor TXL1 is the transfer transistor. It has a configuration replaced by TXL2 to TXL4.

  Thus, when attention is paid to each of the pixels 21H and 21L1 to 21L4, it has the same configuration as a general CMOS image sensor. In the present embodiment, for each pixel block 21, the pixels 21H, 21L1 to L4 belonging to the pixel block 21 share one set of floating capacitance unit FD, amplification transistor AMP, reset transistor RES, and selection transistor SEL. Yes. However, it is not always necessary to adopt such a shared configuration, and those sets may be provided for each pixel.

  The gate of the transfer transistor TXH is connected to the control line 32 that supplies the control signal φTXH from the vertical scanning circuit 23 to the transfer transistor TXH for each row of the pixel block 21. The gate of the transfer transistor TXL1 is connected to the control line 34 that supplies the control signal φTXL1 from the vertical scanning circuit 23 to the transfer transistor TXL1 for each row of the pixel block 21. The gate of the transfer transistor TXL2 is connected to the control line 35 for supplying the control signal φTXL2 from the vertical scanning circuit 23 to the transfer transistor TXL2 for each row of the pixel block 21. The gate of the transfer transistor TXL3 is connected to the control line 36 that supplies the control signal φTXL3 from the vertical scanning circuit 23 to the transfer transistor TXL3 for each row of the pixel block 21. The gate of the transfer transistor TXL4 is connected to the control line 37 for supplying the control signal φTXL4 from the vertical scanning circuit 23 to the transfer transistor TXL4 for each row of the pixel block 21.

  The gate of the reset transistor RES is connected to the control line 31 that supplies the control signal φRES from the vertical scanning circuit 23 to the reset transistor RES for each row of the pixel block 21. The gate of the selection transistor SEL is connected to a control line 33 that supplies a control signal φSEL from the vertical scanning circuit 23 to the selection transistor SEL for each row of the pixel block 21.

  The photodiodes PDH, PDL1 to PDL4 generate charges according to the amount of incident light (subject light). The transfer transistors TXH, TXL1 to TXL4 are turned on during the high level period of the corresponding control signals φTXH, φTXL1 to φTXL4, and transfer the charges accumulated in the corresponding photodiodes PDH, PDL1 to PDL5 to the floating capacitor FD. The reset transistor RES is turned on during the high level period of the control signal φRES and resets the floating capacitor FD.

  The amplification transistor AMP outputs a voltage to the vertical signal line 24 via the selection transistor SEL according to the voltage value of the floating capacitance unit FD. The selection transistor SEL is turned on during the high level period of the control signal φSEL, and connects the source of the amplification transistor AMP to the vertical signal line 24. In the present embodiment, the transistors AMP, TXH, TXL1 to TXL4, RES, and SEL of the pixel block 21 are all nMOS transistors.

  The vertical scanning circuit 23 constitutes a control unit that outputs control signals φSEL, φRES, φTXH, φTXL1 to φTXL4 for each row of the pixel block 21 under the control of the imaging control unit 5 in FIG.

  In the present embodiment, after a mechanical shutter (not shown) is opened for a predetermined exposure period and charges are accumulated in the charge accumulation layers of the photodiodes PDH and PDL1 to PDL4 of the pixels 21H and 21L1 to 21L4, the pixels Each row of the block 21 is sequentially selected, and the same operation is sequentially performed for each row of the pixel block 21.

  When a row of the pixel block 21 is selected, the control signals φTXL1 to φTXL4 of the row are set to a low level, the transfer transistors TXL1 to TXL4 are kept off, and the control signals φSEL, φRES, and φTXH of the row are selected. Thus, by performing readout control similar to that of a general CMOS solid-state imaging device, the signal of the photodiode PDH of the high-sensitivity pixel 21H in the row can be read out from the pixel to the vertical signal line 24. When a certain row of the pixel block 21 is selected, the control signals φTXH, φTXL2 to φTXL4 of the row are set to the low level, and the transfer transistors TXH, TXL2 to TXL4 are kept off, and the row is controlled. By performing readout control similar to that of a general CMOS type solid-state imaging device by the signals φSEL, φRES, and φTXL1, the signal of the photodiode PDL1 of the low sensitivity pixel 21L1 in the row can be read out from the pixel to the vertical signal line 24. it can. Similarly, when a row of the pixel block 21 is selected, the control signals φTXH, φTXL1, φTXL3, φTXL4 of the row are set to a low level, and the transfer transistors TXH, TXL1, TXL3, and TXL4 are kept off. By performing read control similar to that of a general CMOS type solid-state imaging device by the control signals φSEL, φRES, φTXL2 of the row, the signal of the photodiode PDL2 of the low sensitivity pixel 21L2 of the row is transferred from the pixel to the vertical signal line. 24 can be read out. Similarly, when a certain row of the pixel block 21 is selected, the signal of the photodiode PDL3 of the low-sensitivity pixel 21L2 of the row and the signal of the photodiode PDL4 of the low-sensitivity pixel 21L2 of the row are also transmitted to the vertical signal line. 24 can be read out. By sequentially performing such an operation when a certain row of the pixel block 21 is selected, the signals of the photodiodes PDH and PDL1 to PDL4 can be individually read out to the vertical signal line 24. Of course, when a row of the pixel block 21 is selected by the control signals φSEL, φRES, φTXH, φTXL1 to φTXL4, only necessary signals among the signals of the photodiodes PDH, PDL1 to PDL4 can be read out.

  When a row of the pixel block 21 is selected, the transfer signals TXH and TXL1 are transferred when the charges of the photodiodes in the row are transferred to the floating capacitor FD by the control signals φSEL, φRES, φTXH, φTXL1 to φTXL4. Two or more transfer transistors of .about.TX4 are simultaneously turned on, and other points are controlled by the same readout control as that of a general CMOS type solid-state imaging device, so that one of the photodiodes PDH, PDL1 to PDL4. An addition signal obtained by adding the signals of two or more arbitrary photodiodes (in this example, a signal obtained by adding the charges of the two or more photodiodes by the floating capacitor FD) can be read out to the vertical signal line 24.

  As described above, in this embodiment, the photodiodes of the pixels 21H, 21L1 to 21L4 are controlled by the control signals φSEL, φRES, φTXH, φTXL1 to φTXL4 output from the vertical scanning circuit 23 under the control of the imaging control unit 5 in FIG. The signals of PDH and PDL1 to PDL4 can be individually read out to the vertical signal line 24, or the signals of any two or more of the photodiodes PDH and PDL1 to PDL4 of the pixels 21H and 21L1 to 21L4 are added. A signal can also be read out to the vertical signal line 24.

  The signal read out to the vertical signal line 24 as described above is subjected to a predetermined noise removal process by the CDS circuit 26 for each column of the pixel block 21, and then the A / D converter 27. It is converted into a digital signal, and the digital signal is held in the A / D converter 27. The digital image signal held in each A / D converter 27 is horizontally scanned by the horizontal readout circuit 28 in each horizontal scanning period, converted into a predetermined signal format as necessary, and externally (in FIG. 1). To the digital signal processor 6). The solid-state imaging device 4 does not necessarily include the A / D converter 27 and may be configured so that an analog image signal is output from the horizontal readout circuit 28.

  FIG. 4 schematically shows the arrangement of photodiodes PDH (PDHr, PDHg, PDHb) and PDL (PDLr, PDLg, PDLb) as photoelectric conversion units of the solid-state imaging device 4 in FIG. 1 when viewed from the light incident side. It is a schematic plan view shown. In this embodiment, the left-right direction (first direction) in FIG. 4 matches the row direction, and the up-down direction (second direction orthogonal to the first direction) in FIG. 4 matches the column direction. ing.

  A color filter (not shown) is provided on the light incident side corresponding to each of the photodiodes PDH, PDL1 to PDL4. In FIG. 4, a photodiode PDH provided with a red color filter is denoted by reference symbol PDHr, a photodiode PDH provided with a green color filter is denoted by reference symbol PDHg, and a photodiode PDH provided with a blue color filter. Is given the symbol PDHb. In FIG. 4, photodiodes PDL1 to PDL4 provided with a red color filter are denoted by reference symbol PDLr, and photodiodes PDL1 to PDL4 provided with a green color filter are denoted by reference symbol PDLg. Among the photodiodes PDL1 to PDL4, the photodiode provided with the blue color filter is denoted by the symbol PDLb. When the photodiodes PDL1 to PDL4 and the photodiodes PDLr, PDLg, and PDLb are described without being distinguished from each other, they may be described with reference numerals PDL.

  As shown in FIG. 4, the light receiving areas of the photodiodes PDL are the same. The light receiving areas of the photodiodes PDH are the same. The light receiving area of the photodiode PDL of the low sensitivity pixels 21L1 to 21L4 is smaller than the light receiving area of the photodiode PDH of the high sensitivity pixel 21H. Thereby, the sensitivity of the photodiode PDL is relatively low, and the sensitivity of the photodiode PDH is relatively high.

  The photodiodes PDH of the high sensitivity pixel 21H are arranged in a square lattice so as to be aligned in the row direction and the column direction. The pitch in the row direction and the pitch in the column direction of the photodiodes PDH are the same P.

  In the present embodiment, a plurality of types of first color filters that transmit light of different color components are arranged in two rows and two columns on the light incident side of the photodiode PDH as each first photoelectric conversion unit. Are provided in a color array having a repetition period of. In the present embodiment, as shown in FIG. 4, a Bayer arrangement is adopted as the color arrangement, and the first color filters (not shown) of red, green, and blue correspond to the photodiodes PDH according to the Bayer arrangement. Are arranged. That is, in the pixel portion 22, the photodiode PDHr provided with the red first color filter and the photodiode PDHg provided with the green first color filter in the odd rows (or even rows) of the photodiode PDH. Are alternately arranged, and the photodiode PDHg provided with the first green color filter and the photodiode PDHb provided with the blue first color filter are alternately arranged in the even-numbered (or odd-numbered) rows of the photodiode PDH. Are lined up.

  The photodiode PDL as the second photoelectric conversion unit has each position shifted from the position of the photodiode PDH as the first photoelectric conversion unit by a half of the pitch P in the row direction and the column direction as the reference position A. Four each are arranged near the reference position A. On the light incident side of the four photodiodes PDL arranged in the vicinity of each reference position A, the same color as that of the four first color filters corresponding to the repetition period of 2 rows and 2 columns of the first color filter, respectively. A second color filter (not shown) is provided corresponding to each photodiode PDL. In the present embodiment, since the Bayer arrangement is adopted as the color arrangement of the photodiode PDH, the four photodiodes PDL provided in the vicinity of each reference position A are provided with the red second color filter. It consists of one photodiode PDLr, two photodiodes PDLg each provided with a green second color filter, and one photodiode PDLb provided with a blue second color filter.

  In the present embodiment, each photodiode PDL is configured to form a rhombus obtained by rotating a square by 45 ° in FIG. 4, and four photodiodes PDL are located above, below, right and left of the reference position A in the vicinity of each reference position A. As a result, the four photodiodes PDL as a whole form a rhombus obtained by rotating the square in FIG. 4 by 45 °. Each photodiode PDH is configured to form a regular octagon as shown in FIG. Thus, the photodiodes PDH and PDL can be arranged densely. However, the shape of the photodiodes PDH and PDL and the arrangement of the four photodiodes PDL in the vicinity of each reference position A are not limited to the example shown in FIG.

  In the present embodiment, in the vicinity of each reference position A, two photodiodes PDLg provided with a second green color filter are provided on the left and right sides of the reference position A. In the vicinity of each reference position A, a photo provided with a red second color filter at a position closer to the photodiode PDHr provided with the red first color filter, above and below the reference position A. A diode PDLr is provided. In the vicinity of each reference position A, a photo provided with a blue second color filter at a position closer to the photodiode PDHb provided with the blue first color filter, above and below the reference position A. A diode PDLb is provided.

  In the present embodiment, the photodiode PDH and PDL and the color arrangement of the first and second color filters described above are arranged in the vicinity of four reference positions A close to the photodiode PDH for each photodiode PDH. Each of the four photodiodes PDL arranged includes a first line (not shown) extending in the row direction through the center of the photodiode PDH and a second line (not shown) extending in the column direction through the center. 4) and four reference positions A close to the photodiode PDH (reference position A on the upper left side in FIG. 4 with respect to the photodiode PDH, diagonally). Each of four fonts arranged near the upper right reference position A, the lower left reference position A, and the lower right reference position A). The color of the second color filter provided corresponding to the diode PDL are disposed at positions respectively become symmetrical with respect to the first line and the second line.

  In the present embodiment, one photodiode PDH and four photodiodes PDL1 to PDL4 belonging to each pixel block 21 are determined as follows.

  Each pixel block 21 has one photodiode PDH, a photodiode PDHr provided with a red first color filter, a photodiode PDHg provided with a green first color filter, One of the photodiodes PDHb provided with a blue first color filter is provided. As described above, the rows and columns of the pixel blocks 21 are represented by the rows and columns of the photodiodes PDH included in each pixel block 21.

  Although not shown in the drawing, the four photodiodes PDL1 to PDL4 belonging to each pixel block 21 having the photodiode PDHr provided with the red first color filter are four adjacent to the photodiode PDHr. The four photodiodes PDLr are arranged in the vicinity of the two reference positions A and provided with the same second red color filter. In addition, four photodiodes PDL1 to PDL4 belonging to each pixel block 21 having the photodiode PDHg provided with the first green color filter are each in the vicinity of four reference positions A adjacent to the photodiode PDHg. The four photodiodes PDLg are arranged one by one (one of the two closer to the photodiode PDHg) and provided with the same second green color filter. Further, the four photodiodes PDL1 to PDL4 belonging to each pixel block 21 having the photodiode PDHb provided with the blue first color filter are each in the vicinity of the four reference positions A adjacent to the photodiode PDHb. The four photodiodes PDLb are provided one by one and provided with the same blue second color filter.

  FIG. 6 is a diagram schematically illustrating the relationship between the amount of incident light and the output. In FIG. 6, L is an output corresponding to the signal of one photodiode PDL, 2L is an output corresponding to a signal obtained by adding the signals of two photodiodes PDL, and 3L is a signal obtained by adding the signals of three photodiodes PDL. 4L is an output corresponding to a signal obtained by adding the signals of four photodiodes PDL, H is an output corresponding to a signal of one photodiode PDH, H + 4L is a signal of one photodiode PDH and four photodiodes Outputs corresponding to signals obtained by adding PDL signals are shown. However, these outputs indicate the levels quantized by the A / D converter 27 in FIG. Here, it is assumed that the A / D converter 27 quantizes the analog signal with 8-bit gradation (256 gradations). In the following description, it is assumed that the light receiving area of the photodiode PDH is α times (α> 4) the light receiving area of the photodiode PDL.

  Since a photodiode with a large light receiving area is more sensitive than a photodiode with a small light receiving area, it is saturated with a small amount of incident light. That is, in the case of a high-luminance subject, the large light receiving area photodiode is saturated, but the small light receiving area photodiode is difficult to be saturated. If not saturated, the output 2L is twice the output L, the output 3L is three times the output L, the output 4L is four times the output L, and the output H is α of the output L for the same incident light amount. The output H + 4L is (α + 4) times the output L.

  Therefore, processing for obtaining a pixel signal having a wider dynamic range than the pixel signal based only on the output H based on the output H and an output with lower sensitivity (output L, 2L, 3L, or 4L) (hereinafter referred to as dynamic Called “range expansion processing”). The dynamic range of the pixel signal based only on the output H is a range from the light amount zero to the light amount M2. On the other hand, as the dynamic range expansion processing based on the output H and the output 4L, for example, the output H is used as a pixel signal in the range from the light amount zero to the light amount M2, and the output 4L is used in the range from the light amount M2 to the light amount M3. A process of using a value multiplied by (α / 4) as a pixel signal is performed. As a result, the dynamic range is extended to the range from the light amount zero to the light amount M3, and the signal of the photodiode PDH having a relatively large light receiving area is used for the dark portion, so that the SN ratio is improved, and the subject is a high brightness subject. However, since a signal of a photodiode having a relatively small light receiving area (corresponding to four photodiodes PDL) is used, whiteout can be prevented.

  As the dynamic range expansion process based on the output H and the output 3L, for example, the output H is used as a pixel signal in the range from the light amount zero to the light amount M2, and the output 3L is (α in the range from the light amount M2 to the light amount M4. / 3) Processing using the multiplied value as the pixel signal is performed. As the dynamic range expansion processing based on the output H and the output 2L, for example, the output H is used as a pixel signal in the range from the light amount zero to the light amount M2, and the output 2L is (α / 2) in the range from the light amount M2 to the light amount M5. ) Processing using the multiplied value as the pixel signal is performed. As the dynamic range expansion processing based on the output H and the output L, for example, the output H is used as a pixel signal in the range from the light amount zero to the light amount M2, and the output L is multiplied by α in the range from the light amount M2 to the light amount M6. Is used as a pixel signal.

  These dynamic range expansion processes also provide the same advantages as the dynamic range expansion process based on the output H and the output 4L. However, using a higher-sensitivity output together with the output H can prevent overexposure even with a higher-brightness subject by extending the range of the dynamic range, while being brighter than 255 gradations. The gradation characteristics of the part become rough. On the other hand, using the output with lower sensitivity together with the output H can smooth the gradation characteristics of the bright part of 255 gradations or more, while lowering the expansion amount of the range of the dynamic range. If the subject has a brightness, whiteout cannot be prevented. Therefore, by switching the output used together with the output H to two or more of the outputs 4L, 3L, 2L, and L, the smoothness of the gradation characteristics of the bright part which is a trade-off and the amount of expansion of the dynamic range The balance can be switched between two or more stages.

  Further, as can be understood from FIG. 6, processing using the output L as the pixel signal, processing using the output 2L as the pixel signal, processing using the output 3L as the pixel signal, processing using the output 4L as the pixel signal, pixel By switching to three or more processes among a process using the output H as a signal and a process using the output H + L as a pixel signal, the photographing sensitivity can be switched to three or more stages.

  In the present embodiment, when the first shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 passes through the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. By controlling the solid-state imaging device 4, a signal (corresponding to the output H in FIG. 6) corresponding to the signal of the photodiode PDH is selected and read from each pixel block 21 of the solid-state imaging device 4. An image signal having the pixel signal as a pixel signal is subjected to processing such as digital amplification by the digital signal processing unit 6 and then stored in the memory 7 as a captured image. The first shooting mode is a mode in which imaging is performed using only the photodiode PDH of the high sensitivity pixel 21H.

  When the second shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, the signal corresponding to the signal of the photodiode PDH (corresponding to the output H in FIG. 6) and the signal obtained by adding the signals of the four photodiodes PDL are received from each pixel block 21 of the solid-state imaging device 4. Each signal (corresponding to the output 4L in FIG. 6) is selected and read out, processed by the digital signal processing unit 6 such as digital amplification, and then stored in the memory 7. Thereafter, the CPU 9 performs a process corresponding to the dynamic range expansion process based on the output H and the output 4L described above for each signal of each pixel block 21 based on these signals stored in the memory 7. In this way, an image signal obtained by using a signal obtained corresponding to each pixel block 21 as a pixel signal is acquired and stored in the memory 7 as a captured image. The second shooting mode is a mode for obtaining a captured image with an expanded dynamic range by performing a process corresponding to the dynamic range extending process based on the output H and the output 4L.

  FIG. 5 is a diagram schematically showing a state in which signals of one high-sensitivity photodiode PDH for each color in FIG. 4 and four low-sensitivity photodiodes PDL around it are added. FIG. 5A shows a state in which signals of one high-sensitivity photodiode PDHr included in one red pixel block 21 and four low-sensitivity photodiodes PDLr around it are added. FIG. 5B shows a state in which signals from one high-sensitivity photodiode PDHg included in one green pixel block 21 and four low-sensitivity photodiodes PDLg around it are added. FIG. 5C shows a state in which signals of one high sensitivity photodiode PDHb and four surrounding low sensitivity photodiodes PDLb included in one blue pixel block 21 are added.

  In the second imaging mode, as described above, a signal corresponding to the signal obtained by adding the signals of the four photodiodes PDL is read from each pixel block 21. In the second imaging mode, the signal of one photodiode PDH is not added, but as can be seen from FIG. 5, in each pixel block 21, the center of gravity of the four photodiodes PDL to which the signal is added is one photodiode. Since it coincides with the center of gravity of PDH, the position indicated by the addition signal obtained by adding the signals of the four photodiodes PDL coincides with the position indicated by the signal of the photodiode PDH with high accuracy. Therefore, there is no signal displacement between the dark part and the bright part, and the dynamic range can be expanded more appropriately.

  When the third shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 through the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, a signal corresponding to the signal of the photodiode PDH (corresponding to the output H in FIG. 6) and a signal obtained by adding the signals of the three photodiodes PDL from each pixel block 21 of the solid-state imaging device 4 Each signal (corresponding to the output 3L in FIG. 6) is selected and read out, processed by the digital signal processing unit 6 such as digital amplification, and then stored in the memory 7. Thereafter, the CPU 9 performs a process corresponding to the dynamic range expansion process based on the output H and the output 3L described above for each signal of each pixel block 21 based on these signals stored in the memory 7. In this way, an image signal obtained by using a signal obtained corresponding to each pixel block 21 as a pixel signal is acquired and stored in the memory 7 as a captured image. The third shooting mode is a mode for obtaining a captured image having an extended dynamic range by performing a process corresponding to the dynamic range extending process based on the output H and the output 3L.

  When the fourth shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 through the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, a signal corresponding to the signal of the photodiode PDH (corresponding to the output H in FIG. 6) and a signal obtained by adding the signals of the two photodiodes PDL from each pixel block 21 of the solid-state imaging device 4 Each signal (corresponding to the output 2L in FIG. 6) is selected and read out, processed by the digital signal processing unit 6 such as digital amplification, and then stored in the memory 7. Thereafter, the CPU 9 performs a process corresponding to the dynamic range expansion process based on the output H and the output 2L described above for each signal of each pixel block 21 based on these signals stored in the memory 7. In this way, an image signal obtained by using a signal obtained corresponding to each pixel block 21 as a pixel signal is acquired and stored in the memory 7 as a captured image. The fourth shooting mode is a mode for obtaining a captured image having an extended dynamic range by performing a process corresponding to the dynamic range extending process based on the output H and the output 2L.

  When the fifth shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. By doing this, from each pixel block 21 of the solid-state imaging device 4, a signal corresponding to the signal of the photodiode PDH (corresponding to the output H in FIG. 6) and a signal corresponding to the signal of one photodiode PDL (FIG. 6 corresponding to the output L in FIG. 6 is selected and read out, processed by the digital signal processing unit 6 such as digital amplification, and then stored in the memory 7. Thereafter, the CPU 9 performs, on the basis of these signals stored in the memory 7, a process corresponding to the dynamic range expansion process based on the output H and the output L described above for each signal of each pixel block 21. In this way, an image signal obtained by using a signal obtained corresponding to each pixel block 21 as a pixel signal is acquired and stored in the memory 7 as a captured image. The fifth shooting mode is a mode for obtaining a captured image having an extended dynamic range by performing a process corresponding to the dynamic range extending process based on the output H and the output L described above.

  When the sixth shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. Thus, from each pixel block 21 of the solid-state imaging device 4, a signal corresponding to the signal obtained by adding the signal corresponding to the signal of the photodiode PDH (corresponding to the output H in FIG. 6) and the signal of the four photodiodes PDL. (Corresponding to the output H + 4L in FIG. 6) is selected and read out, and after being subjected to processing such as digital amplification by the digital signal processing unit 6, it is stored in the memory 7 as a captured image. The sixth shooting mode is a mode in which imaging is performed with higher sensitivity than the first shooting mode.

  In the sixth shooting mode, as described above, a signal corresponding to a signal obtained by adding the signal of one photodiode PDH and the signal of four photodiodes PDL is read from each pixel block 21. As can be seen from FIG. 5, in each pixel block 21, the centroids of the four photodiodes PDL to which the signals are added coincide with the centroids of the one photodiode PDH, so that the addition signal obtained by adding the signals of the four photodiodes PDL. The position indicated by is coincident with the position indicated by the signal of the photodiode PDH with high accuracy. Therefore, the center of gravity of the signal obtained by adding all these signals also coincides with the center of gravity of one photodiode PDH. For this reason, in the sixth shooting mode, more natural pixel addition can be performed as compared with the case of adding signals whose center of gravity is shifted. Further, in the sixth shooting mode, since all the signals of the photodiodes PDL and PDH are used, high-sensitivity imaging can be performed using the signals of the photodiodes PDL and PDH without waste.

  When the seventh shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, a signal corresponding to the signal obtained by adding the signals of the four photodiodes PDL (corresponding to the output 4L in FIG. 6) is selected and read out from each pixel block 21 of the solid-state image pickup device 4, and the digital signal After processing such as digital amplification by the processing unit 6, the captured image is stored in the memory 7. The seventh shooting mode is a mode in which imaging is performed with a sensitivity lower than that of the first shooting mode.

  In this seventh imaging mode, the signal of one photodiode PDH is not added, but as can be seen from FIG. 5, in each pixel block 21, the center of gravity of the four photodiodes PDL to which the signal is added is one photodiode. Since it coincides with the center of gravity of PDH, the position indicated by the addition signal obtained by adding the signals of the four photodiodes PDL coincides with the position indicated by the signal of the photodiode PDH with high accuracy.

  When the eighth shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, a signal corresponding to the signal obtained by adding the signals of the three photodiodes PDL (corresponding to the output 3L in FIG. 6) is selected and read out from each pixel block 21 of the solid-state imaging device 4, and is read as a digital signal. After processing such as digital amplification by the processing unit 6, the captured image is stored in the memory 7. The eighth shooting mode is a mode in which imaging is performed with a sensitivity lower than that of the seventh shooting mode.

  When the ninth shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, a signal corresponding to the signal obtained by adding the signals of the two photodiodes PDL (corresponding to the output 2L in FIG. 6) is selected and read out from each pixel block 21 of the solid-state image pickup device 4 to obtain a digital signal After processing such as digital amplification by the processing unit 6, the captured image is stored in the memory 7. The ninth shooting mode is a mode in which imaging is performed with a sensitivity lower than that of the eighth shooting mode.

  When the tenth shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. As a result, a signal (corresponding to the output 2L in FIG. 6) corresponding to the signal of one photodiode PDL is selected and read from each pixel block 21 of the solid-state imaging device 4, and is read by the digital signal processing unit 6. After processing such as digital amplification, the captured image is stored in the memory 7. The tenth shooting mode is a mode in which imaging is performed with a sensitivity lower than that of the ninth shooting mode.

  When the eleventh shooting mode is selected by the shooting mode selection dial of the operation unit 14, the CPU 9 controls the solid-state imaging device 4 via the imaging control unit 5 in synchronization with the full pressing operation of the release button of the operation unit 14. Thus, from each pixel block 21 of the solid-state imaging device 4, a signal corresponding to the signal of the photodiode PDH, a signal corresponding to the signal of the photodiode PDL1, a signal corresponding to the signal of the photodiode PDL2, and a signal of the photodiode PDL3 And a signal corresponding to the signal of the photodiode PDL4 are respectively selected and read out individually, processed by the digital signal processing unit 6 such as digital amplification, and stored in the memory 7.

  Thereafter, the CPU 9 causes the image processing unit 13 to perform an interpolation process described below based on these signals stored in the memory 7.

  FIG. 7 is a diagram schematically showing the photodiodes PDHr and PDLr provided with the red color filter in FIG. 4 and the virtual photodiode PDHr ′ interpolated based on the signal of the photodiode PDLr. . It is assumed that the virtual photodiode PDHr ′ is provided with a red first color filter instead of the blue first color filter in the photodiode PDHb (see FIG. 4) provided with the blue first color filter. This corresponds to the photodiode PDHb.

  In the present embodiment, as shown in FIGS. 4 and 7, the image processing unit 13 has, for each photodiode PDHb provided with the blue first color filter, four reference positions close to the photodiode PDHb. Four photodiodes PDLr arranged one by one in the vicinity of A, each of which corresponds to a signal obtained by adding the signals of the four photodiodes PDLr provided with a second color filter of red different from blue Based on this, a signal corresponding to the signal of the virtual photodiode PDHr ′ is interpolated. In the present embodiment, specifically, signals (digital values) corresponding to the signals of the four photodiodes PDHr read into the memory 7 are added, and a red first color filter is provided. In order to match the level of the signal of the photodiode PDHr, the value obtained by multiplying the added value by α (α is the value described above) is obtained as a signal corresponding to the signal of the virtual photodiode PDHr ′. At this time, in this embodiment, since the centroids of the four photodiodes PDLr to which the signals are added coincide with the centroids of the virtual photodiode PDHr ′, highly accurate interpolation can be realized.

  FIG. 8 is a diagram schematically showing the photodiodes PDHg and PDLg provided with the green color filter in FIG. 4 and a virtual photodiode PDHg ′ interpolated based on the signal of the photodiode PDLg. . The virtual photodiode PDHg ′ includes a green first photodiode in place of the red or blue first color filter in the photodiode PDHr or PDHb (see FIG. 4) provided with the red or blue first color filter. This corresponds to the photodiode PDHr or PDHb when it is assumed that a color filter is provided.

  In the present embodiment, as shown in FIGS. 4 and 8, the image processing unit 13 applies each photodiode PDHr or PDHb provided with a red or blue first color filter to the photodiode PDHr or PDHb. Four photodiodes PDLg arranged one by one in the vicinity of four adjacent reference positions A each including four photodiodes PDLg provided with a second color filter of green different from red and blue Based on the signal, the signal corresponding to the signal of the virtual photodiode PDHg ′ is interpolated based on the signal. In the present embodiment, specifically, signals (digital values) corresponding to the signals of the four photodiodes PDHg read out in the memory 7 are added, and a first green color filter is provided. In order to match the level of the signal of the photodiode PDHg, a value obtained by multiplying the added value by α is obtained as a signal corresponding to the signal of the virtual photodiode PDHg ′. At this time, in this embodiment, since the centroids of the four photodiodes PDLg to which the signals are added coincide with the centroids of the virtual photodiode PDHg ′, highly accurate interpolation can be realized.

  FIG. 9 is a diagram schematically showing the photodiodes PDHb and PDLb provided with the blue color filter in FIG. 4 and a virtual photodiode PDHb ′ interpolated based on the signal of the photodiode PDLb. . It is assumed that the virtual photodiode PDHb ′ is provided with a blue first color filter instead of the red first color filter in the photodiode PDHr (see FIG. 4) provided with the red first color filter. This corresponds to the photodiode PDHr.

  In the present embodiment, as shown in FIGS. 4 and 9, the image processing unit 13 includes, for each photodiode PDHr provided with the first red color filter, four reference positions close to the photodiode PDHr. Four photodiodes PDLb arranged one by one in the vicinity of A, each of which corresponds to a signal obtained by adding the signals of four photodiodes PDLb provided with a second color filter of blue different from red Based on this, a signal corresponding to the signal of the virtual photodiode PDHb ′ is interpolated. In the present embodiment, specifically, signals (digital values) corresponding to the signals of the four photodiodes PDHb read out in the memory 7 are added, and a blue first color filter is provided. In order to match the level of the signal of the photodiode PDHb, a value obtained by multiplying the added value by α (α is the value described above) is obtained as a signal corresponding to the signal of the virtual photodiode PDHb ′. At this time, in this embodiment, since the centroids of the four photodiodes PDLb to which the signals are added coincide with the centroids of the virtual photodiode PDHb, highly accurate interpolation can be realized.

  Then, the CPU 9 uses signals corresponding to the signals of the virtual photodiodes PDHr ′, PDHg, and PDHb ′ obtained by the above-described interpolation processing by the image processing unit 13 as the signals of the photodiodes PDHr, PDHg, and PDHb in the memory 7. Along with a signal corresponding to the signal, the image is stored in the memory 7 as a captured image.

  The eleventh shooting mode is a mode in which imaging is performed at a higher resolution (in this embodiment, twice the resolution) than the first shooting mode.

  According to the present embodiment, since the first to fifth photographing modes can be performed, the dynamic range can be expanded stepwise. In addition, according to the present embodiment, the first, sixth to tenth shooting modes can be performed, so that the imaging sensitivity can be changed stepwise. In addition, according to the present embodiment, since the sixth shooting mode can be performed, natural pixel addition using the photodiodes PDH and PDL without waste can be performed. Furthermore, according to the present embodiment, since the eleventh shooting mode can be performed, high-resolution shooting by more appropriate pixel interpolation can be performed.

[Second Embodiment]
FIG. 10 is a schematic view of the arrangement of the photodiodes PDH (PDHr, PDHg, PDHb) and PDL (PDLr, PDLg, PDLb) as photoelectric conversion units of the solid-state imaging device according to the second embodiment when viewed from the light incident side. FIG. 4 schematically shows a plan view corresponding to FIG. 10, elements that are the same as or correspond to those in FIG. 4 are given the same reference numerals, and redundant descriptions thereof are omitted.

  The solid-state imaging device according to the present embodiment differs from the solid-state imaging device 4 in the first embodiment in that the arrangement of the photodiodes PDLr and the photodiodes PDHb is upside down in the vicinity of each reference position A. It is only a point. The solid-state imaging device according to the present embodiment can be used in place of the solid-state imaging device 4 in the first embodiment.

  Also in this embodiment, the same advantages as those in the first embodiment can be obtained.

  FIG. 11 is a diagram schematically showing a state in which signals of one high-sensitivity photodiode PDH for each color in FIG. 10 and four low-sensitivity photodiodes PDL around it are added, corresponding to FIG. Yes. 11, elements that are the same as or correspond to those in FIG. 5 are given the same reference numerals, and redundant descriptions thereof are omitted.

  In FIG. 5, the four photodiodes PDLr that add signals are relatively close to the photodiode PDHr, and the four photodiodes PDLb that add signals are relatively close to the photodiode PDHb. In FIG. 11, the four photodiodes PDLr that add signals are relatively far from the photodiode PDHr, and the four photodiodes PDLb that add signals are relatively far from the photodiode PDHb. Therefore, in the first embodiment, the signal obtained by adding the four photodiodes PDLr more accurately indicates the signal at the center of gravity and the four photodiodes PDLb than the present embodiment. The signal obtained by adding is more accurately indicative of the signal at the center of gravity. Therefore, the first embodiment is more advantageous than the present embodiment in this respect.

  FIG. 12 is a diagram schematically showing the photodiodes PDHr and PDLr provided with the red color filter in FIG. 10 and a virtual photodiode PDHr ′ interpolated based on the signal of the photodiode PDLr. This corresponds to FIG. FIG. 13 is a diagram schematically showing the photodiodes PDHb and PDLb provided with the blue color filter in FIG. 10 and a virtual photodiode PDHb ′ interpolated based on the signal of the photodiode PDLb. 9 corresponds to FIG. 12 and 13, elements that are the same as or correspond to those in FIGS. 7 and 9 are given the same reference numerals, and redundant descriptions thereof are omitted.

  7 and 9, the four photodiodes PDLr that add signals are relatively far from the virtual photodiode PDHr ′, and the four photodiodes PDLb that add signals are relatively far from each other. On the other hand, in FIG. 12 and FIG. 13, the four photodiodes PDLr that add signals are relatively close to the virtual photodiode PDHr ′, and the four photodiodes PDLb that add signals are relatively close to each other. It is in. Therefore, the signal of the virtual photodiode PDHr 'and the signal of the virtual photodiode PDHb' can be interpolated with higher accuracy in the present embodiment than in the first embodiment. Therefore, the present embodiment is more advantageous than the first embodiment in this respect.

[Third Embodiment]
FIG. 14 is a schematic view of the arrangement of photodiodes PDH (PDHr, PDHg, PDHb) and PDL (PDLr, PDLg, PDLb) as photoelectric conversion units of the solid-state imaging device according to the third embodiment when viewed from the light incident side. FIG. 4 schematically shows a plan view corresponding to FIG. 14, elements that are the same as or correspond to those in FIG. 4 are given the same reference numerals, and redundant descriptions thereof are omitted.

  The solid-state imaging device according to the present embodiment is different from the solid-state imaging device 4 in the first embodiment in that the photodiode PDLr is arranged above the reference position A and the photo is near any reference position A. This is only the point where the diode PDHb is disposed below the reference position A. The solid-state imaging device according to the present embodiment can be used in place of the solid-state imaging device 4 in the first embodiment.

  Also in this embodiment, the same advantages as those in the first embodiment can be obtained.

  However, in this embodiment, the center of gravity of the four photodiodes PDLr to which signals are added is shifted from the center of gravity of the photodiode PDHr as in FIG. 5A, and the signals are added in the same manner as in FIG. The center of gravity of the four photodiodes PDLb deviates from the center of gravity of the photodiode PDHb. However, since the photodiodes PDLr and PDLb are arranged in the vicinity of the reference position A, the amount of deviation thereof is small, so that there is no particular problem.

  Further, in the present embodiment, the center of gravity of the four photodiodes PDLr to which signals are added is shifted from the center of gravity of the virtual photodiode PDHr ′ as in FIG. 7, and the signals are added in the same manner as in FIG. The centroids of the two photodiodes PDLb deviate from the centroid of the virtual photodiode PDHb ′. However, since the photodiodes PDLr and PDLb are arranged in the vicinity of the reference position A, the amount of deviation thereof is small, so that there is no particular problem.

  Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments.

  In the first embodiment, as described above, the left-right direction (first direction) in FIG. 4 matches the row direction, and the up-down direction in FIG. 4 (second direction orthogonal to the first direction). (Direction) coincides with the column direction, and the same applies to the other embodiments. However, the arrangement may be rotated by 45 ° so that, for example, the horizontal and vertical directions in FIG. 4 form 45 ° with respect to the row and column directions.

  In addition, as described above, the solid-state imaging device of each of the embodiments employs a Bayer array as a color array of color filters. However, in the present invention, the color arrangement of the color filter is not limited to this, and other color arrangements having a repetition period of 2 rows and 2 columns (for example, complementary color arrangement using magenta, green, cyan, and yellow) are used. It may be adopted.

  Further, in each of the above embodiments, each of the shooting modes is selected in response to a selection command from the shooting mode selection dial of the operation unit 14. However, in the present invention, the photographing mode may be automatically selected according to detection signals from various sensors.

  Furthermore, in each of the embodiments described above, signal addition is performed within the pixel block 21 of the solid-state imaging device except for the interpolation processing. However, outside of the interpolation processing, the signal addition is performed outside the solid-state imaging device. Signal addition may be performed. In this case, the combination of photodiodes in each pixel block 21 may be changed so that the signal addition of the interpolation processing is performed in the pixel block 21.

  In each of the above embodiments, the first to eleventh shooting modes are performed. However, in the present invention, only a part of the shooting modes may be performed. .

1 Electronic camera PDH (PDHr, PDHg, PDHb) High sensitivity photodiode PDL (PDL1 to PDL4, PDLr, PDLg, PDLb) Low sensitivity photodiode 21H High sensitivity pixel 21L Low sensitivity pixel

Claims (10)

A plurality of first photoelectric conversion units arranged in a square lattice so as to be arranged in a first direction and a second direction orthogonal thereto;
From the positions of the plurality of first photoelectric conversion units, the pitches of the first direction and the second direction of the plurality of first photoelectric conversion units are changed in the first direction and the second direction. Each of the positions shifted by half is used as a reference position, and each of the second photoelectric conversion units is arranged in the vicinity of each of the four reference positions and has a light receiving area smaller than that of the first photoelectric conversion unit;
A plurality of first color filters which are provided corresponding to each of the plurality of first photoelectric conversion units and form a color array having a repetition period of 2 rows and 2 columns;
The same color as each of the four first color filters provided corresponding to each of the four second photoelectric conversion units disposed in the vicinity of each reference position and corresponding to the repetition cycle of the 2 rows and 2 columns. Four second color filters of
A solid-state imaging device comprising:   2. The solid-state imaging device according to claim 1, wherein the signal of each first photoelectric conversion unit and the signal of each second photoelectric conversion unit can be individually read out.   For each of the first photoelectric conversion units, there are four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, The four second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter provided corresponding to the photoelectric conversion unit, and the first photoelectric conversion unit 3. The solid-state imaging device according to claim 1, wherein the signals of two or more photoelectric conversion units among the conversion units are configured to be read out. 4.   For each of the first photoelectric conversion units, each of the four second photoelectric conversion units arranged in the vicinity of the four reference positions adjacent to the first photoelectric conversion unit is the first photoelectric conversion unit. The first line extending in the first direction through the center of the first line and the second line extending in the second direction through the center are arranged at positions that are line symmetric with respect to the first line. The color of the second color filter provided corresponding to each of the four second photoelectric conversion units respectively disposed in the vicinity of the four reference positions adjacent to one photoelectric conversion unit is the first color filter. 4. The solid-state imaging device according to claim 1, wherein the solid-state imaging element is disposed in a line-symmetrical position with respect to the second line and the second line. 5. A solid-state imaging device according to any one of claims 1 to 4,
For each of the first photoelectric conversion units, there are four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, The four second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter provided corresponding to the photoelectric conversion unit, and the first photoelectric conversion unit A selection unit that selects at least one photoelectric conversion unit according to a preset shooting mode from the group consisting of:
A processing unit that reads a signal from the photoelectric conversion unit of each group selected by the selection unit, performs a process according to a preset shooting mode, and outputs the signal as a pixel signal;
An imaging apparatus comprising: A solid-state imaging device according to any one of claims 1 to 4,
For each of the first photoelectric conversion units, there are four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, Selected according to a command from the four second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter provided corresponding to the photoelectric conversion unit Based on the signal obtained by adding the signals of the one or more second photoelectric conversion units and the signal of the first photoelectric conversion unit, the pixel signal based only on the signal of the first photoelectric conversion unit Means for performing processing for obtaining a pixel signal having a wider dynamic range,
An imaging apparatus comprising: A solid-state imaging device according to any one of claims 1 to 4,
(I) For each of the first photoelectric conversion units, processing that uses a signal corresponding to the signal of the first photoelectric conversion unit as a pixel signal; (ii) for each of the first photoelectric conversion units, One second photoelectric conversion unit arranged in the vicinity of one of the four reference positions close to the photoelectric conversion unit, provided corresponding to the first photoelectric conversion unit. (Iii) processing for converting a signal corresponding to a signal of the second photoelectric conversion unit provided with the second color filter of the same color as the color of the first color filter into a pixel signal; For each of the first photoelectric conversion units, two second photoelectric conversion units arranged one by one in the vicinity of two of the four reference positions in the vicinity of the first photoelectric conversion unit. Before the first photoelectric conversion unit is provided. Processing for converting a signal corresponding to a signal obtained by adding the signals of the two second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter into a pixel signal (iv ) For each of the first photoelectric converters, three second photoelectric elements arranged one by one in the vicinity of the three reference positions among the four reference positions adjacent to the first photoelectric converter. Three second photoelectric conversion units, each of which is provided with the second color filter having the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit. A process according to which a signal corresponding to the signal obtained by adding the signals of the units is added as a pixel signal; (v) each of the first photoelectric conversion units in the vicinity of the four reference positions adjacent to the first photoelectric conversion unit. Four said second ones arranged one by one Four second photoelectric elements, each of which is provided with the second color filter having the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit. A process according to which a signal corresponding to a signal obtained by adding the signals of the conversion unit is a pixel signal; and (vi) for each of the first photoelectric conversion units, the four reference positions adjacent to the first photoelectric conversion unit Four second photoelectric conversion units arranged one by one near each of the second photoelectric conversion units, the second color having the same color as the color of the first color filter provided corresponding to the first photoelectric conversion unit A process of converting a signal corresponding to a signal obtained by adding the signals of the four second photoelectric conversion units provided with the color filter and the signal of the first photoelectric conversion unit into a pixel signal, Means for selectively performing three or more processes in response to a command;
An imaging apparatus comprising:   For each of the first photoelectric conversion units, there are four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, Signals of the four second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter provided corresponding to the photoelectric conversion unit of the first color filter, 8. The image pickup apparatus according to claim 7, further comprising means for performing a process of converting a signal corresponding to a signal obtained by adding the signal of the photoelectric conversion unit into a pixel signal. A solid-state imaging device according to any one of claims 1 to 4,
For each of the first photoelectric conversion units, there are four second photoelectric conversion units arranged one by one near each of the four reference positions adjacent to the first photoelectric conversion unit, Signals of the four second photoelectric conversion units provided with the second color filter of the same color as the color of the first color filter provided corresponding to the photoelectric conversion unit of the first color filter, Means for processing a signal corresponding to a signal obtained by adding the signal of the photoelectric conversion unit as a pixel signal;
An imaging apparatus comprising: A solid-state imaging device according to any one of claims 1 to 4,
For the first photoelectric conversion units of at least a part of the plurality of first photoelectric conversion units, four ones are arranged near each of the four reference positions adjacent to the first photoelectric conversion units. The second photoelectric conversion unit, wherein the second color filter having a predetermined color different from the color of the first color filter provided corresponding to the first photoelectric conversion unit is provided 4 When it is assumed that the first color filter of the predetermined color is provided corresponding to the first photoelectric conversion unit based on a signal corresponding to a signal obtained by adding the signals of the two second photoelectric conversion units Interpolation processing means for performing a process of interpolating a signal corresponding to the signal of the first photoelectric conversion unit,
An imaging apparatus comprising:

Images