JP2012195675A - Imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enhance the sensitivity of an imaging device while minimizing deterioration in resolution without mixing colors of imaging signals in a Bayer array.SOLUTION: Weighted addition of only the pixel values of first type pixels that are positioned in an area of 5×5 pixels with a target pixel as the center and detect light of a first color component is performed to generate a first color component (R) value, and similarly weighted addition of only the pixel values of pixels that are positioned in an area of 5×5 pixels with a target pixel as the center and detect light of a third color component (B) is performed to generate a third color component (B) value. Weighted addition of only the pixel values of pixels that are positioned in an area of 3×3 pixels with a target pixel as the center and detect light of a second color component (G) is performed to generate a second color component (G) value.

Description

  The present invention relates to an imaging apparatus capable of obtaining a captured image with higher sensitivity in a low-illuminance imaging environment.

  As a conventional imaging device, there is one configured to increase the sensitivity and improve the S / N of a solid-state imaging device by executing a function of adding all digital signals up to N pixels before (for example, Patent Documents). 1).

JP 2000-184274 A (page 4, paragraph 0010)

  Since the conventional image pickup apparatus adds image pickup signals from consecutive adjacent pixels, there is a problem that when a color image pickup element is used, colors are mixed and a color signal cannot be reproduced.

  The present invention has been made to solve the above-described problems. An image pickup apparatus capable of increasing the sensitivity of a color signal without mixing colors even when pixels of an image pickup signal read from a color image pickup element are added. The purpose is to obtain.

The imaging apparatus of the present invention
A first type of pixel that detects light of the first color component and generates a corresponding imaging signal; a second type of pixel that detects light of the second color component and generates a corresponding imaging signal; A third type of pixel that detects light of the third color component and generates a corresponding imaging signal; and a fourth type of pixel that detects light of the second color component and generates a corresponding imaging signal; The image pickup unit regularly arranged in the screen with a combination of two horizontal pixels and two vertical pixels as a basic unit, and the image pickup signal output from the image pickup unit is delayed for a predetermined time, and the target pixel and its pixel A pixel extraction unit that extracts and simultaneously outputs signals representing pixel values of surrounding pixels;
An addition coefficient generation unit for generating an addition coefficient;
Using the first type pixel located in the region of 5 horizontal pixels and 5 vertical pixels centered on the target pixel as a reference pixel, only the pixel value of the reference pixel is weighted and added by the addition coefficient, and the first pixel of the target pixel is obtained. A first color component generation unit for generating color component values of
Using the second type and fourth type pixels located in the region of three horizontal pixels and three vertical pixels centered on the target pixel as reference pixels, only the pixel value of the reference pixel is weighted and added by an addition coefficient, A second color component generation unit that generates a second color component value of the pixel;
Using the third type pixel located in the region of 5 horizontal pixels and 5 vertical pixels centered on the target pixel as a reference pixel, only the pixel value of the reference pixel is weighted and added by the addition coefficient, And a third color component generation unit for generating the color component value.

  According to the present invention, it is possible to obtain an effect that the sensitivity can be improved by, for example, 4 times even in an image signal read from a color image sensor, and the subject can be visually recognized even in an extremely dark low-light environment.

It is a block block diagram which shows the imaging device of Embodiment 1 of this invention. FIG. 2 is a block configuration diagram illustrating an example of a pixel addition circuit in FIG. 1. It is a figure which shows the spatial arrangement | positioning of the pixel added by the pixel addition circuit of FIG. It is a block block diagram which shows an example of the pixel extraction part 6E of FIG. 2 is a block configuration diagram illustrating an example of an R pixel addition unit 6R and an R signal coefficient setting circuit 95 of the pixel addition circuit of FIG. FIG. 2 is a block configuration diagram illustrating an example of a G pixel addition unit 6G and a G signal coefficient setting circuit 195 of the pixel addition circuit of FIG. 2 is a block configuration diagram illustrating an example of a B pixel addition unit 6B and a B signal coefficient setting circuit 125 of the pixel addition circuit of FIG. (A)-(d) is a figure which shows the array information of the color filter of the pixel added by R pixel addition part 6R, G pixel addition part 6G, and B pixel addition part 6E of FIGS. (A)-(d) is a figure which shows the arrangement | sequence of the reference pixel used in order to produce | generate R signal in R pixel addition part 6R of FIG. (A)-(d) is a figure which shows the arrangement | sequence of the reference pixel used in order to produce | generate G signal in G pixel addition part 6G of FIG. (A)-(d) is a figure which shows the arrangement | sequence of the reference pixel used in order to produce | generate B signal in B pixel addition part 6B of FIG. (A)-(c) is a figure which shows the relationship between the sensitization magnification L and the weighting coefficient Kw. It is a block block diagram which shows another structure of the imaging device of Embodiment 1 of this invention. It is a block block diagram which shows the imaging device of Embodiment 2 of this invention. (A) to (e) are the subject illuminance, the lens aperture, the amplification gain of the programmable gain amplification circuit, the sensitization magnification of the pixel addition circuit, the exposure time of the imaging unit, and the amplitude of the output signal of the pixel addition circuit. It is a figure which shows a relationship. It is a block block diagram which shows the imaging device of Embodiment 3 of this invention.

  Hereinafter, the light of the first, second, and third color components is red, green, and blue light, respectively, and pixels that detect the light of these color components and output corresponding signals are arranged in a Bayer array. A case where the image pickup unit is used will be described in detail.

Embodiment 1 FIG.
FIG. 1 shows an imaging apparatus according to Embodiment 1 of the present invention. In FIG. 1, a lens 1 focuses a subject image on an imaging surface of a CCD imaging device 2.

  As shown in FIGS. 8A to 11D, which will be described later, the CCD image pickup device (CCD) 2 has an R pixel that detects red light (first color component light), green light (first light). The G pixel for detecting the second color component light) and the B pixel for detecting the blue light (third color component light) are arranged in a Bayer array. , Blue light (light of the first color component, light of the second color component, light of the third color component) is detected, that is, photoelectrically converted, and the charge generated by the photoelectric conversion passes through the inside of the device. It is transferred and output as an electrical signal (imaging signal).

  The imaging signal includes an R signal (a signal representing a first color component value) that is a pixel signal from the R pixel, a G signal (a signal representing a second color component value) that is a pixel signal from the G pixel, and A B signal (a signal representing a third color component value) that is a pixel signal from the B pixel is included.

  The R pixel, the G pixel, and the B pixel are, for example, a photoelectric conversion element that includes a color filter that transmits only red light, a photoelectric conversion element that includes a color filter that transmits only green light, and only blue light. It is composed of a photoelectric conversion element provided with a color filter to be transmitted.

Noise or the like is removed from the imaging signal output from the CCD imaging device 2 by a correlated double sampling processing circuit (CDS) 3.
The programmable gain amplifier (PGA) 4 amplifies the output signal of the correlated double sampling processing circuit 3 with a gain controlled by the control signal output from the control circuit 12 and outputs the amplified signal.
The A / D conversion circuit (ADC) 5 converts the output signal of the programmable gain amplifier circuit 4 into a digital signal.

  The pixel addition circuit 6 receives the imaging signal output from the A / D conversion circuit 5 and sequentially selects each pixel position as a target pixel position. Among the target pixel and the surrounding pixels, the pixel of the same color (same Signals of pixels that detect light of color components) are selectively added. By performing such processing for all pixel positions in the imaging screen, R signals, G signals, and B signals are generated for all pixel positions.

  The video signal processing circuit 7 performs color synchronization processing, gradation correction processing, noise reduction processing, contour correction processing, white balance adjustment processing, signal amplitude adjustment processing, color correction processing, and the like on the output signal of the pixel addition circuit 6. The added video signal is output from the video signal output terminal 8.

  The synchronization signal generation circuit 11 generates a vertical synchronization signal and a horizontal synchronization signal and supplies them to the pixel addition circuit 6, the video signal processing circuit 7 and the timing generation circuit 10.

  The timing generation circuit 10 generates a drive timing signal DRT for the CCD image pickup device 2 and supplies it to the drive circuit 9. The drive circuit 9 generates a drive signal DRS for the CCD image sensor 2 based on the drive timing signal DRT output from the timing generation circuit 10. The CCD image pickup device 2 performs photoelectric conversion and charge transfer based on the drive signal DRS output from the drive circuit 9.

  The control circuit 12 controls the aperture of the lens 1, controls the timing of charge readout from the photoelectric conversion element of the CCD image sensor 2 generated by the timing generation circuit 10, and the control of the forced discharge timing of the charge (accordingly, the charge accumulation time, that is, the exposure time). Control), control of the amplification gain of the programmable gain amplifier circuit 4, and control of the pixel addition processing of the pixel addition circuit 6.

  As shown in FIG. 2, the pixel addition circuit 6 includes a pixel extraction unit 6E, an R pixel addition unit 6R, a G pixel addition unit 6G, a B pixel addition unit 6B, an R signal coefficient setting circuit 95, and a G signal coefficient. A setting circuit 195 and a B signal coefficient setting circuit 295 are included. The R pixel addition unit 6R constitutes a first color component generation unit that generates a signal that represents the first color component value, and the G pixel addition unit 6G generates a signal that represents the second color component value. 2 color component generation units, and the B pixel addition unit 6B configures a third color component generation unit that generates a signal representing the third color component value.

  As described above, the A / D conversion circuit 5 outputs imaging signals from the Bayer-arranged R, G, and B pixels, and the pixel addition circuit 6 sequentially selects each pixel position in the imaging screen as the target pixel. Add as

In the Bayer array, a basic unit of four pixels, which is composed of one R pixel, one B pixel, and two G pixels, that is, two horizontal pixels and two vertical pixels is repeated.
In this basic unit, two G pixels are arranged on one diagonal line of the basic unit of four pixels, and an R pixel and a B pixel are arranged on the other diagonal line.
Of the two G pixels, a pixel on the same horizontal line (row) as the B pixel is denoted by symbol GB, and a pixel on the same horizontal line (row) as the R pixel is denoted by symbol GR. In addition, the R pixel may be represented by a symbol RR and the B pixel may be represented by a symbol BB.

Hereinafter, the R pixel is referred to as a first type pixel, the GB pixel is referred to as a second type pixel, the B pixel is referred to as a third type pixel, and the BR pixel is referred to as a fourth type pixel. Further, both the GB pixel and the GR pixel may be collectively referred to as a G pixel.
Further, the target pixel is indicated by P33, and as shown in FIG. 8C, when the target pixel is an R pixel, it is indicated by reference numeral RR33, and as shown in FIG. 8A, the target pixel is a GB pixel. When the target pixel is a GR pixel, as shown in FIG. 8B, when the target pixel is a B pixel, as shown in FIG. 8B. This is indicated by symbol BR33.

  The pixels on the imaging screen of the imaging device 2 are aligned in the horizontal direction (row direction) H and the vertical direction (column direction) V as shown in FIG. 3, and are arranged in a matrix as a whole. The position of each pixel on the screen is represented by a combination (h, v) of the horizontal coordinate h and the vertical coordinate v, and the pixel at the coordinate (h, v) is represented by the symbol Phv (h, v are 1, 2, 3,...). It is represented by The value of h differs by 1 between pixels adjacent in the horizontal direction, and the value of v differs by 1 between pixels adjacent in the vertical direction. That is, the distance (pixel pitch) between adjacent pixels is represented by 1.

FIG. 3 shows the arrangement of pixels in a horizontal 5 pixel area and a vertical 5 pixel area (hereinafter sometimes simply referred to as a “5 × 5 pixel area or block”) centered on the target pixel P33. .
Pixels (P22, P32, P34, P23, P43, P24, P34, P44) in which the larger one of the difference between the horizontal coordinate values and the vertical coordinate value with respect to the target pixel P33 is 1, that is, the target pixel A pixel other than the value pixel that is located in an area of 3 horizontal pixels and 3 vertical pixels (3 × 3 pixel area) centering on the pixel may be referred to as a pixel adjacent to the target pixel or simply an adjacent pixel.

  The pixel extraction unit 6E extracts pixel values of pixels in the 5 × 5 pixel area shown in FIG. 3 from the output of the A / D conversion circuit 5. The extracted pixel value is supplied to the R pixel adding unit 6R, the G pixel adding unit 6G, and the B pixel adding unit 6B.

The pixel extraction unit 6E is configured as shown in FIG.
In FIG. 4, “1L-DL” indicates a one-line delay circuit, and “1D-DL” indicates a one-pixel delay circuit.
The pixel extraction unit 6E is configured by connecting one-line delay circuits 22 to 25 and one-pixel delay circuits 32 to 35, 37 to 40, 42 to 45, 47 to 50, and 52 to 55 as illustrated. And simultaneously output pixel signals indicating the pixel values of the pixel of interest and surrounding pixels.
For example, when the target pixel is P33 in FIG. 3, pixel signals representing the pixel values of the target pixel P33 and the surrounding pixels P11 to P55 are output simultaneously.

  The pixel signal Pc output from the A / D conversion circuit 5 is applied to the input terminal 101 of the pixel addition circuit 6 as the pixel signal P55 and supplied to the pixel extraction unit 6E of the pixel addition circuit 6.

The pixel signal P55 is sequentially delayed by one line by the one-line delay circuits 22 to 25, and pixel signals P54, P53, P52, and P51 delayed by 1, 2, 3, and 4 lines, respectively, are output with respect to the pixel signal P55. The
The pixel signal P55 is also sequentially delayed by one pixel by the one-pixel delay circuits 32 to 35, and pixel signals P45, P35, P25, and P15 delayed by 1, 2, 3, and 4 pixels, respectively, are output with respect to the pixel signal P55. The

The pixel signal P54 output from the one-line delay circuit 22 is sequentially delayed by one pixel by the one-pixel delay circuits 37 to 40, and the pixel signal P44 is delayed by 1, 2, 3, and 4 pixels, respectively, with respect to the pixel signal P54. P34, P24, and P14 are output.
The pixel signal P53 output from the one-line delay circuit 23 is sequentially delayed by one pixel by the one-pixel delay circuits 42 to 45, and the pixel signal P43 is delayed by 1, 2, 3, and 4 pixels, respectively, with respect to the pixel signal P53. P33, P23, and P13 are output.

The pixel signal P52 output from the one-line delay circuit 24 is sequentially delayed by one pixel by the one-pixel delay circuits 47 to 50, and the pixel signal P42 is delayed by 1, 2, 3, and 4 pixels, respectively, with respect to the pixel signal P52. P32, P22, and P12 are output.
The pixel signal P51 output from the one-line delay circuit 25 is sequentially delayed by one pixel by the one-pixel delay circuits 52 to 55, and the pixel signal P41 delayed by 1, 2, 3, and 4 pixels, respectively, with respect to the pixel signal P51. P31, P21, and P11 are output.

The pixel signals P55 to P11 represent the pixel values of the pixels P55 to P11 in FIG. 3, respectively, and are simultaneously output from the pixel extraction unit 6E at the timing when the pixel signal P55 is input.
The pixel signals P55 to P11 are respectively connected to the R pixel addition unit 6R, the G pixel addition unit 6G, and the B pixel addition unit 6B via the connection terminals 111 to 115, 121 to 125, 131 to 135, 141 to 145, and 151 to 155. To be supplied.

  5 shows the R pixel addition unit 6R and the R signal coefficient setting circuit 95 of the pixel addition circuit 6, and FIG. 6 shows the G pixel addition unit 6G and the G signal coefficient setting circuit 195 of the pixel addition circuit 6. These show the B pixel addition unit 6B and the B signal coefficient setting circuit 295 of the pixel addition circuit 6.

  The pixel values of the pixels P11 to P55 extracted by the pixel extraction unit 6E are the pixel values of the pixels in the 5 × 5 pixel region, and the multiplication circuits 61 to 85 of the R pixel addition unit 6R shown in FIG. 6 is supplied to the multiplication circuits 161 to 185 of the G pixel addition unit 6G and the multiplication circuits 261 to 285 of the B pixel addition unit 6B shown in FIG.

  The multiplication circuits 61 to 85, 161 to 185, and 261 to 285 are provided corresponding to the respective pixels in the 5 × 5 pixel region, and the pixel values and R signal coefficient settings of the corresponding pixels are provided. Multiplication is performed with the addition coefficient supplied from the circuit 95, the G signal coefficient setting circuit 195, and the B signal coefficient setting circuit 295.

  As will be described later, only the pixels in the region of 3 horizontal pixels and 3 vertical pixels (region of 3 × 3 pixels) may be supplied to the G pixel addition unit 6G.

  Each of the R pixel addition unit 6R, the G pixel addition unit 6G, and the B addition unit 6B performs weighted addition using the pixel values of pixels of the same color in the reference area including the target pixel and the surrounding pixels as a reference pixel, and performs addition. Output as pixel value. That is, the R pixel addition unit 6R performs weighted addition using the pixel value of the red pixel (R pixel) in the reference region as a reference pixel, and outputs it as a red addition pixel value R. The G pixel addition unit 6G The pixel value of the green pixel (G pixel) in the region is weighted and added as a reference pixel, and is output as a green added pixel value G. The B pixel addition unit 6B is a blue pixel (B pixel) in the reference region. Are added as a reference pixel and output as a blue added pixel value B.

  The R pixel addition unit 6R and the B pixel addition unit 6B exclude pixels at the four corners of the 5 × 5 pixel region (differences in the coordinate value in the horizontal direction and the pixel in the vertical direction with respect to the target pixel). The G pixel adding unit 6G uses a 3 × 3 pixel area as a reference area.

  In the selection of the reference pixel, six or less pixels including the target pixel are selected as reference pixels in the order of the distance from the target pixel. However, in this case, the selected pixel is centered on the target pixel. This means that they are arranged symmetrically in the horizontal direction and in the vertical direction (line symmetry about the horizontal line passing through the target pixel and line symmetry about the vertical line passing through the target pixel).

As described in detail below, the control circuit 12 supplies the sensitization magnification L to the R signal coefficient setting circuit 95, the G signal coefficient setting circuit 195, and the B signal coefficient setting circuit 295 via the control terminal 103,
The R signal coefficient setting circuit 95, the G signal coefficient setting circuit 195, and the B signal coefficient setting circuit 295 generate an addition coefficient based on the supplied sensitization magnification L.
The sensitization magnification L is a value represented by a number from 1 to 4, for example, which may include a decimal part.

  The addition coefficient for the reference pixel is given by, for example, a product of a normalization coefficient Kn that is inversely proportional to the number of reference pixels and a weighting coefficient Kw determined for each reference pixel.

  For example, the weighting factor Kw for each pixel depends on the distance from the target pixel of the pixel and the sensitization magnification L. When the sensitization magnification L is relatively small, the weighting factor Kw increases as the distance increases. The smaller the sensitization magnification L is, the smaller the dependence of the weighting factor Kw on the distance is set. When the sensitization magnification is the maximum value, the addition coefficients for all the reference pixels may be equal to each other.

  The addition coefficient for pixels other than the reference pixel is fixed to zero. However, in the present embodiment, a multiplication circuit is provided for all the pixels in the 5 × 5 pixel region, and the addition coefficient for pixels other than the reference pixel is set to zero so as to be excluded from the weighted addition. . This is because the pixel addition units 6R, 6G, and 6B have the same configuration.

  In FIG. 5, the control circuit 12 supplies the sensitization magnification L from 1 to 4 times to the R signal coefficient setting circuit 95 via the control terminal 103. The synchronization signal generation circuit 11 supplies the horizontal synchronization signal HD and the vertical synchronization signal VD to the R signal coefficient setting circuit 95 via the synchronization signal input terminal 104.

  The R signal coefficient setting circuit 95 determines the pixel position of the target pixel P33 based on the horizontal synchronization signal HD and the vertical synchronization signal VD, specifies the pixel position on the color filter array of the target pixel, and increases the specified pixel position. Based on the sensitivity L, 25 addition coefficients KR11 to KR55 are set independently to each of the multiplication circuits 61 to 85 provided corresponding to each pixel position. The addition coefficient for pixels other than the reference pixel is fixed to zero.

The multiplication circuits 61 to 85 multiply the pixel values P11 to P55 input from the pixel extraction unit 6E via the connection terminals 111 to 155, respectively, and the addition coefficients KR11 to KR55 set by the R coefficient setting circuit 95.
The addition circuit 90 adds the output values of the multiplication circuits 61 to 85 and outputs the addition result as an addition pixel signal R from the R signal output terminal 106.

In FIG. 6, the control circuit 12 supplies the sensitization magnification L to the G signal coefficient setting circuit 195 via the control terminal 103. The synchronization signal generation circuit 11 supplies the horizontal synchronization signal HD and the vertical synchronization signal VD to the G signal coefficient setting circuit 195 via the synchronization signal input terminal 104.
The G signal coefficient setting circuit 195 determines the pixel position of the target pixel P33 based on the horizontal synchronization signal HD and the vertical synchronization signal VD, specifies the pixel position on the color filter array of the target pixel, and increases the specified pixel position. Based on the sensitivity L, 25 addition coefficients are set independently for each of the multiplication circuits 161 to 185.

The multiplication circuits 161 to 185 multiply the pixel values P11 to P55 input from the pixel extraction unit 6E via the connection terminals 111 to 155, respectively, and the addition coefficients KG11 to KG55 set by the G coefficient setting circuit 195.
The addition circuit 190 adds the output values of the multiplication circuits 161 to 185 and outputs the addition result as an addition pixel signal G from the G signal output terminal 107.

In FIG. 7, the control circuit 12 supplies the sensitization magnification L to the B signal coefficient setting circuit 295 via the control terminal 103. The synchronization signal generation circuit 11 supplies the horizontal synchronization signal HD and the vertical synchronization signal VD to the B signal coefficient setting circuit 295 via the synchronization signal input terminal 104.
The B signal coefficient setting circuit 295 determines the pixel position of the target pixel P33 on the basis of the horizontal synchronization signal HD and the vertical synchronization signal VD, specifies the pixel position on the color filter array of the target pixel, and increases the specified pixel position. Based on the sensitivity L, 25 addition coefficients are set independently for each of the multiplication circuits 261 to 285.

  The multiplication circuits 261 to 285 multiply the pixel values P11 to P55 input from the pixel extraction unit 6E via the connection terminals 111 to 155, respectively, and the addition coefficients KB11 to KB55 set by the B coefficient setting circuit 295. The addition circuit 290 adds the output values of the multiplication circuits 261 to 285 and outputs the addition result as the addition pixel signal B from the B signal output terminal 108.

FIGS. 8A to 8D show the arrangement of pixels in a 5 × 5 pixel area centered on the target pixel in consideration of the color filter arrangement. The R pixel, G pixel, and B pixel are arranged in a checkered pattern. The target pixel to be processed may be an R pixel, a GB pixel, a B pixel, or a GR pixel, and is specified from the horizontal pixel position and the vertical pixel position.
FIG. 8A shows the peripheral pixel arrangement when the target pixel is GB33, FIG. 8B shows the peripheral pixel arrangement when the target pixel is BB33, and shows the peripheral pixel arrangement when the target pixel is RR33. FIG. 8D shows a peripheral pixel arrangement when the pixel of interest is GR33.
If the color filter of the pixel of interest changes, the color filter arrangement of the peripheral pixels also changes. In addition, even in the same G pixel, the color filter arrangement of the peripheral pixels is different between the GR pixel and the GB pixel.

In the pixel arrangement of FIG. 8A, RR12, RR32, RR52, RR14, RR34, RR54, RR16, RR36, and RR56 indicate R pixels.
GR22, GR42, GR62, GR24, GR44, GR64, GR26, GR46, and GR66 indicate G pixels arranged on the same horizontal line (same row) as the R pixels. There are a case where it is expressed as a G pixel based on the characteristics of the color filter, and a case where it is expressed as a GR pixel in order to specify the pixel position of the G pixel arranged on the same horizontal line as the R pixel.
BB21, BB41, BB61, BB23, BB43, BB63, BB25, BB45, and BB65 denote B pixels.
GB11, GB31, GB51, GB13, GB33, GB53, GB15, GB35, and GB55 indicate G pixels arranged on the same horizontal line as the B pixel.
There are cases where it is expressed as a G pixel based on the characteristics of the color filter, and cases where it is expressed as a GB pixel in order to specify the pixel position of the G pixel arranged on the same horizontal line as the B pixel.

  In the pixel arrangement of FIG. 8B, RR02, RR22, RR42, RR04, RR24, RR44, RR06, RR26, and RR46 indicate R pixels. GR12, GR32, GR52, GR14, GR34, GR54, GR16, GR36, and GR56 indicate G pixels arranged on the same horizontal line as the R pixel. BB11, BB31, BB51, BB13, BB33, BB53, BB15, BB35, and BB55 indicate B pixels. GB01, GB21, GB41, GB03, GB23, GB43, GB05, GB25, and GB45 indicate G pixels arranged on the same horizontal line as the B pixel.

  In the pixel arrangement of FIG. 8C, RR11, RR31, RR51, RR13, RR33, RR53, RR15, RR35, and RR55 indicate R pixels. GR21, GR41, GR61, GR23, GR43, GR63, GR25, GR45, and GR65 indicate G pixels arranged on the same horizontal line as the R pixel. BB20, BB40, BB60, BB22, BB42, BB62, BB24, BB44, and BB64 indicate B pixels. GB10, GB30, GB50, GB12, GB32, GB52, GB14, GB34, and GB54 denote G pixels arranged on the same horizontal line as the B pixel.

  In the pixel arrangement of FIG. 8D, RR01, RR21, RR41, RR03, RR23, RR43, RR05, RR25, and RR45 indicate R pixels. GR11, GR31, GR51, GR13, GR33, GR53, GR15, GR35, and GR55 indicate G pixels arranged on the same horizontal line as the R pixel. BB10, BB30, BB50, BB12, BB32, BB52, BB14, BB34, and BB54 denote B pixels. GB00, GB20, GB40, GB02, GB22, GB42, GB04, GB24, and GB44 indicate G pixels arranged on the same horizontal line as the B pixel.

  FIGS. 9A to 9D show an array of reference pixels used for generating an R signal in the R pixel addition unit 6R of the pixel addition circuit of FIG. As shown in the figure, an R pixel (a pixel for detecting R component light, for example, a pixel provided with an R filter) located in a reference area Ar obtained by removing four corner pixels from a 5 × 5 pixel area serves as a reference pixel. Used. The reference pixel is used in weighted addition in pixel addition, but depending on the value of sensitization magnification L described later, the addition coefficient is “0”. That is, when the sensitization magnification is small, the addition coefficient of the pixel relatively far from the target pixel is “0”, and the pixel value of the target pixel is obtained only from the pixel relatively close to the target pixel.

FIG. 9A shows the reference area Ar and the reference pixel when the target pixel is GB33 (second type pixel). As shown, there are six R pixels in the reference area Ar, and all of them are used as reference pixels. That is, the pixel RR32 positioned one line before the target pixel, the pixel RR12 positioned two pixels before the target pixel, the pixel RR52 positioned two pixels before the target pixel, and one line of the target pixel A pixel RR34 located behind, a pixel RR14 located two pixels before the line of interest, and a pixel RR54 located two pixels after the line of interest
Are used as reference pixels.
At this time, the R signal (first component value) is obtained by the following equation.
R = KR12 × RR12 + KR32 × RR32 + KR52 × RR52
+ KR14xRR14 + KR34xRR34 + KR54xRR54
The R signal coefficient setting circuit 95 sets a coefficient that is not used in the above expression among the addition coefficients KR11 to KR55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The R signal coefficient setting circuit 95 calculates the addition coefficient used in the above equation according to the following equation.
KR12 = (L−1) × 4/18
KR32 = (L + 8) / 18
KR52 = (L−1) × 4/18
KR14 = (L−1) × 4/18
KR34 = (L + 8) / 18
KR54 = (L−1) × 4/18
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/6 that is inversely proportional to the number of pixels to be added is the normalization coefficient Kn.
For example, the equation for calculating KR32 (addition coefficient for pixels in a 3 × 3 pixel region) is:
KR32 = (L + 8) / 12 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L + 8) / 12 is the weighting coefficient Kw.
Similarly, the equation for calculating KR12 (addition coefficient for pixels outside the 3 × 3 pixel area) is:
KR12 = (L−1) / 3 × 4/6
Among them, 4/6 is a normalization coefficient Kn, and the remaining (L-1) / 3 is a weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the R pixel addition unit 6R performs the following calculation.
R = 0 x RR12 + 9/18 x RR32 + 0 x RR52
+ 0 x RR14 + 9/18 x RR34 + 0 x RR54
When the sensitization magnification L is 2 times, the R pixel addition unit 6R performs the following calculation.
R = 4/18 × RR12 + 10/18 × RR32 + 4/18 × RR52
+ 4/18 × RR14 + 10/18 × RR34 + 4/18 × RR54
When the sensitization magnification L is 3, the R pixel addition unit 6R performs the following calculation.
R = 8/18 × RR12 + 11/18 × RR32 + 8/18 × RR52
+ 8/18 × RR14 + 11/18 × RR34 + 8/18 × RR54
When the sensitization magnification L is 4, the R pixel adding unit 6R performs the following calculation.
R = 12/18 x RR12 + 12/18 x RR32 + 12/18 x RR52
+ 12/18 × RR14 + 12/18 × RR34 + 12/18 × RR54

FIG. 9B shows the reference area Ar and the reference pixel when the target pixel is BB33 (third type pixel). As shown in the figure, there are four R pixels in the reference area Ar, and all of them are used as reference pixels. That is, a pixel RR22 positioned one pixel before the target pixel, a pixel RR42 positioned one pixel before the target pixel, a pixel RR42 positioned one pixel before the target pixel, and a pixel RR42 positioned one pixel before the target pixel, and A pixel RR44 located one pixel after the target pixel is used as a reference pixel.
At this time, the R signal (first component value) is obtained by the following equation.
R = KR22 × RR22 + KR42 × RR42
+ KR24xRR24 + KR44xRR44
The R signal coefficient setting circuit 95 sets a coefficient that is not used in the above expression among the addition coefficients KR11 to KR55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The R signal coefficient setting circuit 95 calculates the addition coefficient used in the above equation according to the following equation.
KR22 = L / 4
KR42 = L / 4
KR24 = L / 4
KR44 = L / 4
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/4 that is inversely proportional to the number of pixels to be added is the normalization coefficient.
For example, the equation for obtaining KR22 is
KR22 = L / 4 × 4/4
Of these, 4/4 is the normalization coefficient Kn, and the remaining L / 4 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the R pixel addition unit 6R performs the following calculation.
R = 1/4 x RR22 + 1/4 x RR42
+ 1/4 x RR24 + 1/4 x RR44
When the sensitization magnification L is 2 times, the R pixel addition unit 6R performs the following calculation.
R = 2/4 × RR22 + 2/4 × RR42
+ 2/4 x RR24 + 2/4 x RR44
When the sensitization magnification L is 3, the R pixel addition unit 6R performs the following calculation.
R = 3/4 × RR22 + 3/4 × RR42
+ 3/4 x RR24 + 3/4 x RR44
When the sensitization magnification L is 4, the R pixel adding unit 6R performs the following calculation.
R = 4/4 × RR22 + 4/4 × RR42
+ 4/4 x RR24 + 4/4 x RR44

FIG. 9C shows the reference area Ar and the reference pixel when the target pixel is RR33 (first type pixel). As illustrated, there are five R pixels, all of which are used as reference pixels. That is, the target pixel RR33, the reference pixel RR31 positioned two lines before the target pixel, the reference pixel RR13 positioned two pixels before the target pixel, the reference pixel RR53 positioned two pixels after the target pixel, and two lines of the target pixel The reference pixel RR35 located later is used as the reference pixel.
At this time, the R signal is obtained by the following equation.
R = KR31 × RR31
+ KR13 x RR13 + KR33 x RR33 + KR53 x RR53
+ KR35 x RR35
The R signal coefficient setting circuit 95 sets a coefficient that is not used in the above expression among the addition coefficients KR11 to KR55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The R signal coefficient setting circuit 95 calculates the addition coefficient used in the above equation according to the following equation.
KR31 = (L−1) × 4/15
KR13 = (L−1) × 4/15
KR33 = (16-L) / 15
KR53 = (L−1) × 4/15
KR35 = (L−1) × 4/15
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/5 that is inversely proportional to the number of pixels to be added is the normalization coefficient.
For example, the equation for calculating KR33 (addition coefficient for the target pixel) is
KR33 = (16−L) / 12 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (16-L) / 12 is the weight coefficient Kw.
Similarly, the equation for calculating KR31 (addition coefficient for pixels outside the 3 × 3 pixel region) is:
KR31 = (L−1) / 3 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the R pixel addition unit 6R performs the following calculation.
R = 0 x RR31
+ 0 × RR13 + 15/15 × RR33 + 0 × RR53
+ 0 × RR35
When the sensitization magnification L is 2 times, the R pixel addition unit 6R performs the following calculation.
R = 4/15 × RR31
+ 4/15 x RR13 + 14/15 x RR33 + 4/15 x RR53
+ 4/15 x RR35
When the sensitization magnification L is 3, the R pixel addition unit 6R performs the following calculation.
R = 8/15 × RR31
+ 8/15 x RR13 + 13/15 x RR33 + 8/15 x RR53
+ 8/15 x RR35
When the sensitization magnification L is 4, the R pixel adding unit 6R performs the following calculation.
R = 12/15 x RR31
+ 12/15 × RR13 + 12/15 × RR33 + 12/15 × RR53
+ 12/15 × RR35

FIG. 9D shows the reference region Ar and the reference pixel when the target pixel is GR33 (fourth type pixel). As shown, there are six R pixels in the reference area Ar, and all of them are used as reference pixels. That is, a pixel RR21 positioned one pixel before the target pixel two pixels before, a pixel RR41 positioned one pixel before the target pixel two pixels before, a pixel RR23 positioned one pixel before the target pixel, and one pixel of the target pixel The pixel RR43 located behind, the pixel RR25 located one pixel before two lines after the target pixel, and the pixel RR45 located one pixel after two lines after the target pixel are used as reference pixels.
At this time, the R signal is obtained by the following equation.
R = KR21 × RR21 + KR41 × RR41
+ KR23 × RR23 + KR43 × RR43
+ KR25 × RR25 + KR45 × RR45
The R signal coefficient setting circuit 95 sets a coefficient that is not used in the above expression among the addition coefficients KR11 to KR55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The R signal coefficient setting circuit 95 calculates the addition coefficient used in the above equation according to the following equation.
KR21 = (L−1) × 4/18
KR41 = (L−1) × 4/18
KR23 = (L + 8) / 18
KR43 = (L + 8) / 18
KR25 = (L−1) × 4/18
KR45 = (L−1) × 4/18
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/6, which is inversely proportional to the number of pixels to be added, is the normalization coefficient.
For example, the equation for calculating KR23 (addition coefficient for pixels in a 3 × 3 pixel region) is:
KR23 = (L + 8) / 12 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L + 8) / 12 is the weighting coefficient Kw.
Similarly, the equation for calculating KR21 (addition coefficient for pixels outside the 3 × 3 pixel region) is:
KR21 = (L−1) / 3 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the R pixel addition unit 6R performs the following calculation.
R = 0 x RR21 + 0 x RR41
+ 9/18 x RR23 + 9/18 x RR43
+ 0 × RR25 + 0 × RR45
When the sensitization magnification L is 2 times, the R pixel addition unit 6R performs the following calculation.
R = 4/18 × RR21 + 4/18 × RR41
+ 10/18 × RR23 + 10/18 × RR43
+ 4/18 x RR25 + 4/18 x RR45
When the sensitization magnification L is 3, the R pixel addition unit 6R performs the following calculation.
R = 8/18 × RR21 + 8/18 × RR41
+1 1/18 x RR23 + 11/18 x RR43
+ 8/18 x RR25 + 8/18 x RR45
When the sensitization magnification L is 4, the R pixel adding unit 6R performs the following calculation.
R = 12/18 x RR21 + 12/18 x RR41
+12/18 x RR23 +12/18 x RR43
+12/18 x RR25 +12/18 x RR45

  FIGS. 10A to 10D show an array of reference pixels used to generate a G signal in the G pixel addition unit 6G of the pixel addition circuit of FIG. As shown in the figure, a G pixel (a pixel for detecting light of a G component, for example, a pixel provided with a G filter) located in a 3 × 3 pixel reference region Ag is used as a reference pixel. The reference pixel is used in weighted addition in pixel addition, but depending on the value of sensitization magnification L described later, the addition coefficient is “0”. That is, when the sensitization magnification is small, the addition coefficient of the pixel relatively far from the target pixel is “0”, and the pixel value of the target pixel is obtained only from the pixel relatively close to the target pixel.

FIG. 10A shows the reference region Ag and the reference pixel when the target pixel is GB33 (second type pixel). As illustrated, there are five G pixels (GB pixels or GR pixels) in the reference region Ag, and all of them are used as reference pixels. That is, the target pixel GB33, the pixel GR22 positioned one pixel before the target pixel, the pixel GR42 positioned one pixel before the target pixel, the pixel GR42 positioned one pixel before the target pixel, and one pixel before the target pixel one line before And the pixel GR44 located one pixel after one line after the target pixel is used as a reference pixel.
At this time, the G signal is obtained by the following equation.
G = KG22 × GR22 + KG42 × GR42
+ KG33 x GB33
+ KG24 × GR24 + KG44 × GR44
The G signal coefficient setting circuit 195 sets a coefficient that is not used in the above expression among the addition coefficients KG11 to KG55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The G signal coefficient setting circuit 195 calculates the addition coefficient used in the above expression according to the following expression.
KG22 = (L−1) × 4/15
KG42 = (L−1) × 4/15
KG33 = (16−L) / 15
KG24 = (L−1) × 4/15
KG44 = (L−1) × 4/15
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/5 that is inversely proportional to the number of pixels to be added is the normalization coefficient.
For example, the equation for calculating KG33 (addition coefficient for the target pixel) is
KG33 = (16−L) / 12 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (16-L) / 12 is the weight coefficient Kw.
Similarly, the equation for obtaining KG22 (addition coefficient for pixels other than the target pixel) is
KG22 = (L−1) / 3 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the G pixel addition unit 6G performs the following calculation.
G = 0 x GR22 + 0 x GR42
+ 15/15 × GB33
+ 0 × GR24 + 0 × GR44
When the sensitization magnification L is 2, the G pixel addition unit 6G performs the following calculation.
G = 4/15 × GR22 + 4/15 × GR42
+ 14/15 × GB33
+ 4/15 × GR24 + 4/15 × GR44
When the sensitization magnification L is 3, the G pixel addition unit 6G performs the following calculation.
G = 8/15 × GR22 + 8/15 × GR42
+ 13/15 × GB33
+ 8/15 × GR24 + 8/15 × GR44
When the sensitization magnification L is 4, the G pixel addition unit 6G performs the following calculation.
G = 12/15 × GR22 + 12/15 × GR42
+ 12/15 × GB33
+ 12/15 × GR24 + 12/15 × GR44

FIG. 10B shows the reference region and the reference pixel when the target pixel is BB33 (third type pixel). As illustrated, there are four G pixels (GB pixels or GR pixels) in the reference area Ag, and all of them are used as reference pixels. That is, the pixel GR32 located one line before the target pixel, the pixel GB23 located one pixel before the target pixel, the pixel GB43 located one pixel after the target pixel, and the pixel GR34 located one line after the target pixel are referenced. Used as a pixel.
At this time, the G signal is obtained by the following equation.
G = KG32 × GR32 + KG34 × GR34
+ KG23 x GB23 + KG43 x GB43
The G signal coefficient setting circuit 195 sets a coefficient that is not used in the above expression among the addition coefficients KG11 to KG55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The G signal coefficient setting circuit 195 calculates the addition coefficient used in the above expression according to the following expression.
KG32 = L / 4
KG34 = L / 4
KG23 = L / 4
KG43 = L / 4
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/4 that is inversely proportional to the number of pixels to be added is the normalization coefficient.
For example, the equation for calculating KG32 is
KG32 = L / 4 × 4/4
Of these, 4/4 is the normalization coefficient Kn, and the remaining L / 4 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the G pixel addition unit 6G performs the following calculation.
G = 1/4 x GR32 + 1/4 x GR34
+ 1/4 x GB23 + 1/4 x GB43
When the sensitization magnification L is 2, the G pixel addition unit 6G performs the following calculation.
G = 2/4 × GR32 + 2/4 × GR34
+ 2/4 x GB23 + 2/4 x GB43
When the sensitization magnification L is 3, the G pixel addition unit 6G performs the following calculation.
G = 3/4 x GR32 + 3/4 x GR34
+ 3/4 x GB23 + 3/4 x GB43
When the sensitization magnification L is 4, the G pixel addition unit 6G performs the following calculation.
G = 4/4 × GR32 + 4/4 × GR34
+ 4/4 x GB23 + 4/4 x GB43

FIG. 10C shows the reference region Ag and the reference pixel when the target pixel is RR33 (first type pixel). As shown in the figure, there are four G pixels (GB pixels or GR pixels) in the reference area AG, and all of them are used as reference pixels. That is, the pixel GB32 positioned one line before the target pixel, the pixel GR23 positioned one pixel before the target pixel, the pixel GR43 positioned one pixel after the target pixel, and the pixel GB34 positioned one line after the target pixel are referenced. Used as a pixel.
At this time, the G signal is obtained by the following equation.
G = KG32 × GB32 + KG34 × GB34
+ KG23 x GR23 + KG43 x GR43
The G signal coefficient setting circuit 195 sets a coefficient that is not used in the above expression among the addition coefficients KG11 to KG55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The G signal coefficient setting circuit 195 calculates the addition coefficient used in the above expression according to the following expression.
KG32 = L / 4
KG34 = L / 4
KG23 = L / 4
KG43 = L / 4
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/4 that is inversely proportional to the number of pixels to be added is the normalization coefficient. For example, the equation for obtaining KG32 is:
KG32 = L / 4 × 4/4
Of these, 4/4 is the normalization coefficient Kn, and the remaining L / 4 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the G pixel addition unit 6G performs the following calculation.
G = 1/4 x GB32 + 1/4 x GB34
+ 1/4 x GR23 + 1/4 x GR43
When the sensitization magnification L is 2, the G pixel addition unit 6G performs the following calculation.
G = 2/4 × GB32 + 2/4 × GB34
+ 2/4 x GR23 + 2/4 x GR43
When the sensitization magnification L is 3, the G pixel addition unit 6G performs the following calculation.
G = 3/4 × GB32 + 3/4 × GB34
+ 3/4 x GR23 + 3/4 x GR43
When the sensitization magnification L is 4, the G pixel addition unit 6G performs the following calculation.
G = 4/4 × GB32 + 4/4 × GB34
+ 4/4 x GR23 + 4/4 x GR43

FIG. 10D shows the reference region Ag and the reference pixel when the target pixel is GR33 (fourth type pixel). As illustrated, there are five G pixels (GB pixels or GR pixels) in the reference region Ag, and all of them are used as reference pixels. That is, the target pixel GR33, the pixel GB22 positioned one pixel before the target pixel, the pixel GB42 positioned one pixel before the target pixel, the pixel GB42 positioned one line before the target pixel, and one pixel before the target pixel one pixel before And the pixel GB44 located one pixel after one line after the target pixel is used as a reference pixel.
At this time, the G signal is obtained by the following equation.
G = KG22 × GB22 + KG42 × GB42
+ KG33 x GR33
+ KG24 × GB24 + KG44 × GB44
The G signal coefficient setting circuit 195 sets a coefficient that is not used in the above expression among the addition coefficients KG11 to KG55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The G signal coefficient setting circuit 195 calculates the addition coefficient used in the above expression according to the following expression.
KG22 = (L−1) × 4/15
KG42 = (L−1) × 4/15
KG33 = (16−L) / 15
KG24 = (L−1) × 4/15
KG44 = (L−1) × 4/15
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/5 that is inversely proportional to the number of pixels to be added is the normalization coefficient.
For example, the equation for calculating KG33 (addition coefficient for the target pixel) is
KG33 = (16−L) / 12 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (16-L) / 12 is the weight coefficient Kw.
Similarly, the equation for obtaining KG22 (addition coefficient for pixels other than the target pixel) is
KG22 = (L−1) / 3 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the G pixel addition unit 6G performs the following calculation.
G = 0 × GB22 + 0 × GB42
+ 15/15 × GR33
+ 0 × GB24 + 0 × GB44
When the sensitization magnification L is 2, the G pixel addition unit 6G performs the following calculation.
G = 4/15 × GB22 + 4/15 × GB42
+ 14/15 × GR33
+ 4/15 × GB24 + 4/15 × GB44
When the sensitization magnification L is 3, the G pixel addition unit 6G performs the following calculation.
G = 8/15 × GB22 + 8/15 × GB42
+ 13/15 × GR33
+ 8/15 × GB24 + 8/15 × GB44
When the sensitization magnification L is 4, the G pixel addition unit 6G performs the following calculation.
G = 12/15 × GB22 + 12/15 × GB42
+ 12/15 × GR33
+ 12/15 × GB24 + 12/15 × GB44

  FIGS. 11A to 11D show an array of reference pixels used for generating a B signal in the B pixel addition unit 6B of the pixel addition circuit of FIG. As shown in the figure, a B pixel (a pixel for detecting light of a B component, for example, a pixel provided with a B filter) located in a reference area Ab obtained by removing four corner pixels from a 5 × 5 pixel area is used as a reference pixel. Used. The reference pixel is used in weighted addition in pixel addition, but depending on the value of sensitization magnification L described later, the addition coefficient is “0”. That is, when the sensitization magnification is small, the addition coefficient of the pixel relatively far from the target pixel is “0”, and the pixel value of the target pixel is obtained only from the pixel relatively close to the target pixel.

FIG. 11A shows the reference region Ab and the reference pixel when the target pixel is GB33 (second type pixel). As shown in the figure, there are six B pixels in the reference area Ab, and all of them are used as reference pixels. That is, the pixel BB21 positioned one pixel before the target pixel two pixels before, the pixel BB41 positioned one pixel before the target pixel two pixels, the pixel BB23 positioned one pixel before the target pixel, and the target pixel 1 The pixel BB43 positioned after the pixel, the pixel BB25 positioned one pixel before the target pixel two lines, and the pixel BB45 positioned one pixel after the target pixel two lines are used as reference pixels.
At this time, the B signal is obtained by the following equation.
B = KB21 × BB21 + KB41 × BB41
+ KB23 x BB23 + KB43 x BB43
+ KB25 x BB25 + KB45 x BB45
The B signal coefficient setting circuit 295 sets a coefficient that is not used in the above expression among the addition coefficients KB11 to KB55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The B signal coefficient setting circuit 295 calculates the addition coefficient used in the above equation according to the following equation.
KB21 = (L−1) × 4/18
KB41 = (L−1) × 4/18
KB23 = (L + 8) / 18
KB43 = (L + 8) / 18
KB25 = (L−1) × 4/18
KB45 = (L−1) × 4/18
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/6, which is inversely proportional to the number of pixels to be added, is the normalization coefficient.
For example, the equation for calculating KB23 (addition coefficient for pixels in a 3 × 3 pixel region) is:
KB23 = (L + 8) / 12 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L + 8) / 12 is the weighting coefficient Kw.
Similarly, the equation for calculating KB21 (addition coefficient for pixels outside the 3 × 3 pixel area) is:
KB21 = (L−1) / 3 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the B pixel addition unit 6B performs the following calculation.
B = 0 x BB21 + 0 x BB41
+ 9/18 x BB23 + 9/18 x BB43
+ 0 × BB25 + 0 × BB45
When the sensitization magnification L is 2, the B pixel addition unit 6B performs the following calculation.
B = 4/18 × BB21 + 4/18 × BB41
+ 10/18 × BB23 + 10/18 × BB43
+ 4/18 x BB25 + 4/18 x BB45
When the sensitization magnification L is 3, the B pixel addition unit 6B performs the following calculation.
B = 8/18 × BB21 + 8/18 × BB41
+1 1/18 × BB23 + 1/118 × BB43
+ 8/18 x BB25 + 8/18 x BB45
When the sensitization magnification L is 4, the B pixel addition unit 6B performs the following calculation.
B = 12/18 x BB21 + 12/18 x BB41
+12/18 x BB23 +12/18 x BB43
+ 12/18 × BB25 + 12/18 × BB45

FIG. 11B shows the reference region Ab and the reference pixel when the target pixel is BB33 (third type pixel). As shown in the figure, there are five B pixels in the reference area Ab, and all of them are used as reference pixels. That is, the target pixel BB33, the pixel BB31 positioned two lines before the target pixel, the pixel BB13 positioned two pixels before the target pixel, the pixel BB53 positioned two pixels after the target pixel, and the second line after the target pixel Pixel BB35 is used as a reference pixel.
At this time, the B signal is obtained by the following equation.
B = KB31 × BB31
+ KB13 × BB13 + KB33 × BB33 + KB53 × BB53
+ KB35 x BB35
The B signal coefficient setting circuit 295 sets a coefficient that is not used in the above expression among the addition coefficients KB11 to KB55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The B signal coefficient setting circuit 295 calculates the addition coefficient used in the above equation according to the following equation.
KB31 = (L−1) × 4/15
KB13 = (L−1) × 4/15
KB33 = (16-L) / 15
KB53 = (L−1) × 4/15
KB35 = (L−1) × 4/15
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/5 that is inversely proportional to the number of pixels to be added is the normalization coefficient.
For example, the equation for calculating KB33 (addition coefficient for the target pixel) is:
KB33 = (16−L) / 12 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (16-L) / 12 is the weight coefficient Kw.
Similarly, the equation for calculating KB31 (addition coefficient for pixels outside the 3 × 3 pixel area) is:
KB31 = (L−1) / 3 × 4/5
4/5 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the B pixel addition unit 6B performs the following calculation.
B = 0 x BB31
+ 0 × BB13 + 15/15 × BB33 + 0 × BB53
+0 xBB35
When the sensitization magnification L is 2, the B pixel addition unit 6B performs the following calculation.
B = 4/15 × BB31
+ 4/15 × BB13 + 14/15 × BB33 + 4/15 × BB53
+ 4/15 × BB35
When the sensitization magnification L is 3, the B pixel addition unit 6B performs the following calculation.
B = 8/15 × BB31
+ 8/15 x BB13 + 13/15 x BB33 + 8/15 x BB53
+ 8/15 × BB35
When the sensitization magnification L is 4, the B pixel addition unit 6B performs the following calculation.
B = 12/15 x BB31
+ 12/15 × BB13 + 12/15 × BB33 + 12/15 × BB53
+ 12/15 × BB35

FIG. 11C shows the reference region Ab and the reference pixel when the target pixel is RR33 (first type pixel). As shown in the figure, there are four B pixels in the reference area Ab, and all of them are used as reference pixels. That is, a pixel BB22 positioned one pixel before the target pixel, a pixel BB42 positioned one pixel before the target pixel, a pixel BB24 positioned one pixel before the target pixel, a pixel BB24 positioned one pixel before the target pixel, and A pixel BB44 located one pixel after one line after the target pixel is used as a reference pixel.
At this time, the B signal is obtained by the following equation.
B = KB22 × BB22 + KB42 × BB42
+ KB24 × BB24 + KB44 × BB44
The B signal coefficient setting circuit 295 sets a coefficient that is not used in the above expression among the addition coefficients KB11 to KB55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The B signal coefficient setting circuit 295 calculates the addition coefficient used in the above equation according to the following equation.
KB22 = L / 4
KB42 = L / 4
KB24 = L / 4
KB44 = L / 4
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/4 that is inversely proportional to the number of pixels to be added is the normalization coefficient. For example, the equation for determining KB22 is
KB22 = L / 4 × 4/4
Of these, 4/4 is the normalization coefficient Kn, and the remaining L / 4 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the B pixel addition unit 6B performs the following calculation.
B = ¼ × BB22 + 1/4 × BB42
+ 1/4 x BB24 + 1/4 x BB44
When the sensitization magnification L is 2, the B pixel addition unit 6B performs the following calculation.
B = 2/4 × BB22 + 2/4 × BB42
+ 2/4 xBB24 + 2/4 xBB44
When the sensitization magnification L is 3, the B pixel addition unit 6B performs the following calculation.
B = 3/4 × BB22 + 3/4 × BB42
+ 3/4 x BB24 + 3/4 x BB44
When the sensitization magnification L is 4, the B pixel addition unit 6B performs the following calculation.
B = 4/4 × BB22 + 4/4 × BB42
+ 4/4 × BB24 + 4/4 × BB44

FIG. 11D shows the reference region Ab and the reference pixel when the target pixel is GR33 (fourth type pixel). As shown in the figure, there are six B pixels in the reference area Ab, and all of them are used as reference pixels. That is, the pixel BB32 positioned one line before the target pixel, the pixel BB12 positioned two pixels before the target pixel, the pixel BB52 positioned two pixels before the target pixel, and one line of the target pixel The pixel BB34 located after, the pixel BB14 located two pixels before one line after the target pixel, and the pixel BB54 located two pixels after one line after the target pixel are used as reference pixels.
At this time, the B signal is obtained by the following equation.
B = KB12 × BB12 + KB32 × BB32 + KB52 × BB52
+ KB14 × BB14 + KB34 × BB34 + KB54 × BB54
The B signal coefficient setting circuit 295 sets a coefficient that is not used in the above expression among the addition coefficients KB11 to KB55, that is, a coefficient for the pixel at the reference pixel position to “0”.

The B signal coefficient setting circuit 295 calculates the addition coefficient used in the above equation according to the following equation.
KB12 = (L−1) × 4/18
KB32 = (L + 8) / 18
KB52 = (L−1) × 4/18
KB14 = (L−1) × 4/18
KB34 = (L + 8) / 18
KB54 = (L−1) × 4/18
Here, L is a sensitization magnification set by the control circuit 12.

Each of the above coefficients is given by the product of the normalization coefficient Kn and the weight coefficient Kw. In the above example, 4/6, which is inversely proportional to the number of pixels to be added, is the normalization coefficient.
For example, the equation for determining KB32 (addition coefficient for pixels in a 3 × 3 pixel region) is:
KB32 = (L + 8) / 12 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L + 8) / 12 is the weighting coefficient Kw.
Similarly, the equation for calculating KB12 (addition coefficient for pixels outside the 3 × 3 pixel area) is:
KB12 = (L−1) / 3 × 4/6
4/6 is the normalization coefficient Kn, and the remaining (L-1) / 3 is the weighting coefficient Kw. The same applies to other coefficients.

When the sensitization magnification L is 1, the R pixel addition unit 6R performs the following calculation.
B = 0 x BB12 + 9/18 x BB32 + 0 x BB52
+ 0 × BB14 + 9/18 × BB34 + 0 × BB54
When the sensitization magnification L is 2 times, the R pixel addition unit 6R performs the following calculation.
B = 4/18 × BB12 + 10/18 × BB32 + 4/18 × BB52
+ 4/18 × BB14 + 10/18 × BB34 + 4/18 × BB54
When the sensitization magnification L is 3, the R pixel addition unit 6R performs the following calculation.
B = 8/18 × BB12 + 11/18 × BB32 + 8/18 × BB52
+ 8/18 × BB14 + 11/18 × BB34 + 8/18 × BB54
When the sensitization magnification L is 4, the R pixel adding unit 6R performs the following calculation.
B = 12/18 x BB12 + 12/18 x BB32 + 12/18 x BB52
+ 12/18 × BB14 + 12/18 × BB34 + 12/18 × BB54

In general, the addition coefficient Ka (KR21, KR22,..., KG22, KG23,..., KB21, KB22,...) For the reference pixel is given by the product of the weighting coefficient Kw and the normalization coefficient Kn.
The normalization coefficient Kn is determined in advance according to the color component value to be obtained by addition, and is inversely proportional to the number of reference pixels. In the above example, if the number of reference pixels is Np (= 4, 5 or 6), the normalization coefficient Kn = 4 / Np (= 4/4, 4/5 or 4/6).
On the other hand, the weighting factor Kw is determined for each pixel to be added, and the weighting factor Kw for each pixel depends on the distance from the target pixel of the pixel and the sensitization magnification, and the sensitization magnification L Is relatively small, the weight coefficient Kw is set to be smaller as the distance is larger, and the dependency of the weight coefficient Kw on the distance is smaller as the sensitization magnification L is larger.
The sum Ska of all the reference pixels of the addition coefficient is equal to the sensitization magnification L.

When the number of reference pixels is 4 (FIG. 9B, FIG. 10B, FIG. 10C, FIG. 11C)
When the number of reference pixels is 5 (FIG. 9C, FIG. 10A, FIG. 10D, FIG. 11B),
When the number of reference pixels is 6 (FIG. 9A, FIG. 9D, FIG. 11A, FIG. 11D),
FIG. 12A, FIG. 12B, and FIG. 12C show the relationship between the sensitization magnification L and the weighting coefficient Kw.
In FIG. 12A, a symbol Kw4 indicates a weighting factor Kw common to all reference pixels.
In FIG. 12B, a symbol Kw50 indicates a weighting factor for the target pixel, and a symbol Kw51 indicates a weighting factor for pixels other than the target pixel.
In FIG. 12C, the symbol Kw61 indicates the weighting factor for the reference pixel in the 3 × 3 pixel region, and the symbol Kw62 indicates the weighting factor for the pixel outside the 3 × 3 pixel region.

  In the above embodiment, the control circuit 12 outputs the sensitization magnification L, and the coefficient setting circuits 95, 195, 295 generate the addition coefficient Ka based on the sensitization magnification L. The coefficient setting circuits 95, 195, and 295 constitute an addition coefficient generation unit that generates the addition coefficient Ka.

  As described in the above example, pixel addition is performed using a highly correlated pixel that is closest to the pixel of interest, so that it is possible to improve sensitivity while minimizing degradation of image resolution. In particular, the G pixel contributing to the resolution uses a pixel adjacent to the target pixel (a pixel in a 3 × 3 pixel region) as a reference pixel.

  As explained in the above example, pixel addition is performed using the pixel value of the pixel located in the vicinity of the target pixel, and therefore a pixel value with a relatively high correlation, thus improving sensitivity while suppressing image resolution degradation. can do. In particular, for G pixels that have a large influence on the luminance component and thus contribute greatly to the improvement in resolution, pixel addition is performed using only adjacent pixels (pixels in a 3 × 3 pixel region) as reference pixels. Degradation of resolution due to addition can be minimized.

  As described in the above example, by performing pixel addition immediately after being output from the image sensor (that is, before processing by the video signal processing circuit 7), pixel addition is not affected by video signal processing. High sensitivity signal can be generated. In the pixel addition performed in the subsequent stage of the video signal processing, since the color synchronization processing and the filter processing are performed using the pixels located in the vicinity, the horizontal resolution and the vertical resolution are reduced more than expected. When pixel addition is performed using an adjacent pixel (a pixel having the larger difference between the horizontal coordinate value and the vertical coordinate value being 1), the pixel adjacent to that pixel is a color It may be generated using pixels adjacent to adjacent pixels in the synchronization process, and as a result, pixel addition is performed with pixels whose pixel distance in the horizontal direction or vertical direction is 2 pixel pitches. become. Compared with this, pixel addition using a pixel signal before video signal processing has an effect of obtaining a higher resolution image compared to pixel addition using a pixel signal after video signal processing.

  In addition, since non-linear filter processing and gradation conversion processing are performed in the video signal processing, the signal amplitude may be lost when a low amplitude signal is input. For this reason, even if two pixels are added to the output signal of the video signal processing, there is a possibility that the image signal is not doubled. In the example described above, the pixels are added before the video signal processing. Therefore, when two pixels are added, there is an effect that the image signal is doubled.

As described in the above example, when the target pixel is an R pixel or a B pixel, if the sensitization magnification L is 1, the pixel addition is performed using only the target pixel itself or its neighboring pixels. When the sensitization magnification L is 4 times, pixel addition using peripheral pixels arranged around adjacent pixels is also performed. At the sensitization magnification L in the meantime, as the sensitization magnification increases, For example, the increase or decrease of the addition coefficient is proportional to the increase of the sensitization magnification so that the addition coefficient changes continuously.
On the other hand, when the target pixel is a G pixel, pixel addition is performed using only the target pixel or only the target pixel and its adjacent pixels, regardless of the sensitization magnification of 1 to 4 times.

  Since it comprised as mentioned above, in standard illumination intensity (within the range beyond the high illumination side reference value), the sensitization magnification L can be set to 1 and the image without the fall of a resolution can be output. In the low illuminance range (the range below the reference value on the low illuminance side), the sensitization magnification L is set to 4 times, and an image can be output with a sensitivity of 4 times with a reduction in resolution minimized. Depending on the sensitivity and noise level of the image sensor mounted on the camera, for example, the high illuminance side reference value may be set to 1 lx and the low illuminance side reference value may be set to 0.25 lx. In addition, in the medium illuminance range between the sensitization magnification L = 1 times at standard illuminance and the sensitization magnification L = 4 times at low illuminance, the sensitization magnification L can be set seamlessly. Even when an image obtained from the imaging apparatus is viewed as a moving image, there is an effect that the image does not change suddenly in the process of illuminance being dark and does not give an uncomfortable feeling. The brightness of the output signal does not change suddenly, and the person watching the screen does not feel uncomfortable or feel uncomfortable.

In short, the combination of the control circuit 12 and the R signal coefficient setting circuit 95 determines the addition coefficient used in the R pixel addition unit (first color component generation unit) 6R.
When the subject illuminance is within a high illuminance range equal to or higher than a predetermined high illuminance side reference value and the target pixel is an R pixel (first type pixel) as shown in FIG. The addition coefficient for the pixel is zero,
When the subject illuminance is within a high illuminance range equal to or greater than a predetermined high illuminance side reference value and the target pixel is other than the R pixel as shown in FIGS. 9A, 9B, 9D (GB pixel, In the case of a B pixel or a GR pixel, that is, a second type, a third type, or a fourth type pixel), reference pixels that are not included in the 3 × 3 pixel region centered on the target pixel The addition coefficient of
When the subject illuminance is within the low illuminance range below the predetermined low illuminance side reference value, the addition coefficient for all of the reference pixels is set to a non-zero value equal to each other,
When the subject illuminance is within the medium illuminance range between the low illuminance side reference value and the high illuminance side reference value, as the subject illuminance increases, the value used in the low illuminance range is used in the high illuminance range. The above addition coefficient is continuously changed to a value.

The combination of the control circuit 12 and the G signal coefficient setting circuit 195 determines the addition coefficient used in the G pixel addition unit (second color component generation unit) 6G.
When the subject illuminance is in a high illuminance range that is equal to or higher than a predetermined high illuminance side reference value, as shown in FIGS. 10A and 10D, the target pixel is a GB pixel or a GR pixel (second type or fourth type). Pixel), that is, a pixel for detecting light of the G component, the addition coefficient for reference pixels other than the target pixel is set to zero,
The subject illuminance is within a low illuminance range equal to or lower than a predetermined low illuminance side reference value, and as shown in FIGS. 10A and 10D, the target pixel is a GB pixel or a GR pixel (second type or fourth type). Pixel), that is, a pixel for detecting light of the G component, the addition coefficients for all the reference pixels are set to be equal to each other,
The subject illuminance is within a high illuminance range equal to or greater than a predetermined reference value on the high illuminance side, and as shown in FIGS. 10C and 10B, the target pixel is an R pixel or B pixel (first or third type). Pixel), that is, a pixel other than a pixel that detects light of the G component, the addition coefficients for all the reference pixels are set to the same first value,
The subject illuminance is within a low illuminance range equal to or lower than a predetermined low illuminance side reference value, and as shown in FIGS. 10C and 10B, the target pixel is an R pixel or B pixel (first type or third type). Pixel), that is, a pixel other than a pixel that detects light of the G component, the addition coefficients for all the reference pixels are set to be equal to each other, a second value that is greater than the first value,
If the subject illuminance is within the middle illuminance range between the low illuminance side reference value and the high illuminance side reference value,
As the subject illuminance increases, the addition coefficient is continuously changed from the value used in the low illuminance range to the value used in the high illuminance range.

Further, the combination of the control circuit 12 and the B signal coefficient setting circuit 195 determines the addition coefficient used in the B pixel addition unit (third color component generation unit) 6B.
When the subject illuminance is within a high illuminance range equal to or greater than a predetermined reference value on the high illuminance side and the target pixel is a B pixel (third type pixel) as shown in FIG. The addition coefficient for the pixel is zero,
The illuminance of the subject is within a high illuminance range that is equal to or greater than a predetermined high illuminance side reference value, and the pixel of interest is a pixel other than the B pixel (R pixel, GB) as shown in FIGS. Pixel, GR pixel, that is, first type, second type, and fourth type pixel), the addition coefficient for the reference pixel not included in the 3 × 3 pixel region centered on the target pixel is set to zero,
When the subject illuminance is within the low illuminance range below the predetermined low illuminance side reference value, the addition coefficient for all of the reference pixels is set to a non-zero value equal to each other,
When the subject illuminance is within the medium illuminance range between the low illuminance side reference value and the high illuminance side reference value, as the subject illuminance increases, the value used in the low illuminance range is used in the high illuminance range. The above addition coefficient is continuously changed to a value.

  In the above example, the case where the sensitization magnification L up to 4 times is set has been described, but a value larger than 4 times may be set. In the above equation for generating the addition coefficient, the sensitization magnification L is set in a range from 1 to 4 times, and when it is 4 times, the weights of all reference pixels are equal. The generation formula of the addition coefficient of the sensitization magnification L larger than 4 is a generation formula that can set the sensitization magnification L with the weights of the reference pixels being equal. Depending on the color filter arrangement pattern of the peripheral pixels, the addition coefficient generation formula is L / 4 in the case of 4-pixel addition, and the addition coefficient generation formula is L / 5 in the case of 5-pixel addition. Then, the generation formula of the addition coefficient is L / 6. Setting the sensitization magnification L larger than 4 times needs to be used with attention to gradation drop.

  9A to 9D and 11A to 11D, the reference signal used for generating the R signal in the R pixel adding unit 6R and the B signal in the B pixel adding unit 6B are generated. The reference area used for this purpose is a part obtained by removing the four corner pixels from the 5 × 5 pixel area, but the whole 5 × 5 pixels may be used as the reference area without removing the four corner pixels. In the above example, six or less pixels including the target pixel are selected as reference pixels in the order of the distance from the target pixel. However, more pixels may be selected as reference pixels. .

  As described in the above example, since pixel values of pixels that detect light of the same color component (pixel values from pixels having the same color color t) are added, a highly sensitive color without color mixing An image is obtained.

  When a captured image at low illuminance is amplified by an analog amplifier, noise becomes larger than a signal. Further, gradation reduction occurs when amplified by a digital amplifier. As described in the above example, the present invention achieves higher sensitivity by pixel addition of pixels in the vicinity region, so noise is smaller than that of the signal. For example, when two pixels are added, the signal component is doubled, the noise component is square root doubled, and a relatively pure signal component is increased. In addition, since the pixels located in the vicinity have a high correlation as a property of the image, a highly effective sensitivity improvement is realized by adding a plurality of pixels located in the vicinity of the target pixel.

In the above example, the configuration using the CCD image sensor 2 as shown in FIG. 1 as one configuration example of the solid-state image sensor has been described. However, the two-dimensional image sensor is not limited to the CCD image sensor, Any thing is good. Further, not limited to the interline transfer CCD, a frame transfer CCD or a frame interline transfer CCD may be used.
FIG. 13 shows a configuration using the CMOS image sensor 20. The CMOS image sensor may be a device with a single imaging function or a device with integrated peripheral functions.
FIG. 13 shows a case of a CMOS image sensor in which peripheral functions are integrated.
The functions of the CCD image pickup device 2, the correlated double sampling processing circuit 3, the programmable gain amplification circuit 4, the A / D conversion circuit 5, and the timing generation circuit 10 in FIG. 1 are included in the CMOS image pickup device 20.

Embodiment 2. FIG.
FIG. 14 shows an imaging apparatus according to Embodiment 2 of the present invention. In FIG. 14, the detection circuit 13 is added, and the control circuit 12b is provided instead of the control circuit 12 of FIG. Play.

  The detection circuit 13 detects the magnitude of the signal supplied from the pixel addition circuit 6, obtains a signal amplitude level, for example, an average level detection value ASA, and outputs it as illuminance information.

In the above detection, the detection circuit 13 obtains the calculated value ASA of the average level of the signal amplitude by dividing the sum of the pixel values of all effective pixels by the total number of effective pixels.
Such calculation of the average level is performed, for example, every vertical scanning cycle, and is executed by, for example, integration processing and division processing. The “calculated value” of the average level of the signal amplitude obtained as described above may be referred to as “detected value” of the average level of the signal amplitude.

  Note that the division by the total number of effective pixels in the calculation of the average level of the signal amplitude described above may be realized by a bit shift process of a digital value when the number of pixels is given by 2 to the nth power (n is an integer). . Further, since the total number of effective pixels is a constant in the same system, division of the total number of effective pixels may be omitted.

The control circuit 12b is similar to the control circuit 12 of the first embodiment, but has an additional function as follows. That is, the control circuit 12b controls the aperture of the lens 1 based on the detection value ASA of the average level of the signal amplitude supplied from the detection circuit 13, and from the photoelectric conversion element of the CCD image pickup device 2 generated by the timing generation circuit 10. The charge read timing and charge forced discharge timing control (accordingly, charge accumulation time, ie, exposure time control), amplification gain control of the programmable gain amplifier circuit 4, and pixel addition processing of the pixel adder circuit 6 are controlled.
Further, the video signal processing circuit 7 calculates the level of noise included in the output of the pixel addition circuit 6 every vertical scanning period, and supplies it to the control circuit 12b.

  In the above example, the calculation of the average level of the signal amplitude and the calculation of the noise level are performed every vertical scanning period. However, the signal processing time in the detection circuit 13 and the video signal processing circuit 7 and Considering the transmission time from the detection circuit 13 and the video signal processing circuit 7 to the control circuit 12b, it may be performed only once in several vertical scans.

  In the above example, the detection circuit 13 calculates the average level of the signal amplitude, but there may be a case where peak detection of the signal amplitude is performed. The output of the detection circuit 13 is generated so as to increase the visibility of the subject of interest. For example, when it is not desired to saturate a highlight portion to white, peak detection is performed. When a halftone is clearly visible even if the highlight portion is saturated to white, average detection is performed.

  As will be described in detail below, the sensitivity control by the pixel addition circuit 6 can also be controlled as part of the exposure control, so that there is an effect that an image can be obtained in which the subject can always be viewed under optimum conditions even if the illuminance environment changes. In addition, the pixel addition circuit 6 adjusts the signal amplitude by changing the number of pixels to be added (the number of pixels having an addition coefficient other than zero among the reference pixels) and the addition coefficient.

  The control circuit 12b performs automatic exposure control so that the detection value ASA of the average level of the signal amplitude obtained from the detection circuit 13 is constant. When the signal amplitude is large in imaging in a bright environment, the control circuit 12b controls the aperture of the lens 1 to reduce the amount of light incident on the CCD image sensor 2, or the forced charge discharge timing by the timing generation circuit 10. In this adjustment, the exposure time is reduced by controlling to forcibly discharge the charge accumulated in the photoelectric conversion element of the CCD image pickup device 2.

  When the signal amplitude becomes smaller due to imaging in a dark environment, the control circuit 12b amplifies the imaging signal by controlling to increase the amplification gain of the programmable gain amplifier circuit 4. However, when the amplification gain is too large, noise becomes conspicuous and an image with poor visibility is obtained. As another method, the control circuit 12b can extend the exposure time by controlling the charge reading from the photoelectric conversion element of the CCD image pickup device 2 to be thinned out in units of vertical scanning periods. However, if the exposure time is too long, the moving subject becomes an afterimage, resulting in an image with poor visibility. Furthermore, a circuit for performing interpolation (frame interpolation) of missing images in units of vertical scanning cycles is required.

Similar to the first embodiment, the control circuit 12b of the present embodiment sets the sensitization magnification L to the pixel addition circuit 6 from 1 to 4. The sensitization magnification L is set (adjusted) based on the illuminance information from the detection circuit 13 and the exposure parameters. As described in the first embodiment, the addition coefficients KR, KG, and KB are adjusted by changing the sensitization magnification L. Therefore, the addition coefficient is adjusted based on the illuminance information.
By adding 4 to 6 pixels located in the vicinity region of the target pixel, it is possible to realize a fourfold increase in sensitivity, and to greatly improve the visibility even in an extremely dark environment. Further, since the decrease in frame frequency can be prevented or suppressed, there is no deterioration in dynamic resolution, and deterioration in horizontal resolution and vertical resolution can be minimized.

Hereinafter, an example of a procedure for sensitivity adjustment when the subject illuminance changes will be described. First, description will be made assuming that the exposure time is maintained at a constant value (standard exposure time) Tr.
When the illuminance of the subject gradually decreases and the detected value ASA of the average level of the signal amplitude decreases, the aperture of the lens 1 is controlled in the open direction (range Sa in FIG. 15A), and the average level of the signal amplitude To maintain. Here, “maintaining the average level of the signal amplitude” means maintaining the average level of the signal amplitude of the output of the pixel addition circuit 6, and hence the average level of the signal amplitude expressed by the output of the detection circuit 13. means.

  After the aperture of the lens 1 is opened (fully opened), control is performed to increase the amplification gain of the programmable gain amplifier circuit 4 (range Sb in FIG. 15B), and the average level of the signal amplitude is maintained. After the amplification gain of the programmable gain amplifier circuit 4 increases and becomes larger than the predetermined upper limit value UGL of the amplification gain, the sensitization magnification L of the pixel addition circuit 6 is controlled to increase (FIG. 15 (c) ) Range Sc), the average level ASA of the signal amplitude is maintained.

  The average level ASA can be maintained by controlling the sensitization magnification L until the sensitization magnification L reaches the maximum value (L = 4). When the subject illuminance further decreases, the average level ASA starts to decrease.

  At the standard exposure time Tr, when the lens diaphragm is opened, the amplification gain is maximized, and the sensitization magnification L is 1, the output of the pixel addition circuit 6 (output of the pixel addition units 6R, 6G, 6B) is a predetermined level. The illuminance HL becomes a reference value on the high illuminance side, and the illuminance is 1/4 of the high illuminance reference value HL, that is, at the standard exposure time Tr, the lens aperture is opened, the amplification gain is maximized, and the sensitization magnification L is 4. The illuminance LL at which the output of the pixel addition circuit 6 is at a predetermined level is set as the low illuminance side reference value.

  When the subject illuminance gradually becomes brighter and exceeds the low illuminance side reference value LL and the detection value ASA of the average level of the signal amplitude is about to increase, the pixel addition circuit 6 is controlled to reduce the sensitization magnification L ( The range Sc) of FIG. 15 (c) maintains the average level of the signal amplitude. When the sensitization magnification L is reduced to 1, the control is performed so as to reduce the amplification gain of the programmable gain amplifier circuit 4, and the average level of the signal amplitude is maintained. After the amplification gain of the programmable gain amplifier circuit 4 decreases (range Sb in FIG. 15B) and becomes smaller than the predetermined lower limit value LGL, the aperture of the lens 1 is controlled in the light shielding direction (FIG. 15 ( The average level of the signal amplitude is maintained in the range Sa) of a) (FIG. 15 (e)). As the illuminance further increases, the average level ASA increases.

As a result of the above control, the average level ASA of the signal amplitude can be kept constant in the range from the lower limit LL to the upper limit UL as shown by the solid line in FIG.
Although the exposure time is constant, the exposure time may be controlled according to the illuminance of the subject. For example, the exposure time may be increased when the signal amplitude does not reach a sufficient value even when the illuminance decreases and the sensitization magnification L reaches the maximum value (range in FIG. 15D). Sd). On the contrary, even if the illuminance increases and the aperture is maximized (the F value is maximized), if the signal amplitude is too large, the exposure time may be shortened (see FIG. 15D). Range Se).
When such exposure time control is applied, the average level ASA of the signal amplitude can be kept constant in the range from the lower limit LLe to the upper limit ULe, as shown by a dotted line in FIG.

  The predetermined upper limit value UGL of the amplification gain is determined based on a noise level detection value ANL detected by the video signal processing circuit 7. (Considering that it is necessary to increase the amplification gain when the illuminance of the subject decreases and the S / N of the output of the image sensor 2 decreases accordingly) The detected value ANL of the noise level is the average level of the signal amplitude. The amplification gain of the programmable gain amplifier circuit 4 when the predetermined noise ratio (first predetermined noise ratio, that is, the allowable upper limit value) NPR1 with respect to the detection value ASA is set as the predetermined upper limit value UGL. The first predetermined noise ratio NPR1 is set to 1/50, for example.

  The calculated noise level ANL is obtained by extracting noise components by noise reduction processing and dividing the sum of absolute values of noise components in the entire effective pixel range by the total number of effective pixels. By the noise reduction process, a noise reduction signal NRS in which the noise of the input signal is reduced is obtained. A noise component can be extracted by subtracting the noise reduction signal NRS from the input signal (a signal before the noise reduction process is performed in the video signal processing circuit 7). The “calculated value” of the noise level obtained as described above may be referred to as “detected value”.

  Since the allowable noise level for visual recognition of the subject varies depending on the application, the first predetermined noise ratio NPR1 varies depending on the application of the imaging apparatus, such as whether S / N is important or image resolution is important. The control circuit 12b dynamically observes the gain set in the programmable gain amplification circuit 4 and the detection value ANL of the noise level supplied from the detection circuit 13 to the control circuit 12b, and dynamically increases the predetermined upper limit value UGL of the amplification gain. The programmable gain amplifying circuit 4 and the pixel adding circuit 6 may be controlled, and the noise level detection value ANL is compared with the detection value ASA of the average level of the signal amplitude before the imaging apparatus is shipped from the factory. A storage unit (volatile memory backed up by a non-volatile memory or battery) that measures the amplification gain reaching a predetermined noise ratio NPR1 of 1 and retains the stored contents even when the imaging device is turned off as the predetermined upper limit value UGL 16), and the control circuit 12b refers to the predetermined upper limit value UGL of the above-mentioned amplification gain. Amplifying circuit 4 and may control the pixel addition circuit 6.

  The predetermined lower limit value LGL of the amplification gain is determined based on a noise level detection value ANL supplied from the video signal processing circuit 7 to the control circuit 12b. The amplification gain of the programmable gain amplifying circuit 4 when the detection value ANL of the noise level falls below a predetermined noise ratio (second predetermined noise ratio) NPR2 with respect to the detection value ASA of the average level of the signal amplitude is the predetermined gain. The lower limit is LGL. The second predetermined noise ratio is determined based on the first predetermined noise ratio NPR1 and the sensitivity magnification (4 times) by the pixel addition circuit 6. For example, the second predetermined noise ratio NPR2 is defined as 1/200 (= {(1/50) × (1/4)}).

  Since the allowable noise level for visual recognition of the subject differs depending on the application, the second predetermined noise ratio NPR2 varies depending on the application of the imaging device, such as whether S / N is important or image resolution is important. The control circuit 12b dynamically observes the gain set in the programmable gain amplifier circuit 4 and the detection value ANL of the noise level supplied from the detection circuit 13 to the control circuit 12b, and dynamically determines the predetermined lower limit value LGL of the amplification gain. The programmable gain amplifying circuit 4 and the pixel adding circuit 6 may be controlled, and the noise level detection value ANL is compared with the detection value ASA of the average level of the signal amplitude before the imaging apparatus is shipped from the factory. 2, the amplification gain reaching a predetermined noise ratio NPR2 of 2 is measured, and the predetermined lower limit value LGL is written to the storage unit 16 that can retain the stored contents even when the power of the imaging apparatus is turned off. The programmable gain amplifier circuit 4 and the pixel addition circuit 6 may be controlled with reference to the lower limit value LGL.

  The control circuit 12b controls the aperture of the lens 1, the exposure time of the CCD image pickup device 2, the amplification gain of the programmable gain amplification circuit 4, and the signal amplitude adjustment function by pixel addition in the pixel addition circuit 6 to control the average level of the signal amplitude (pixel The average level of the signal amplitude of the output of the adder circuit 6 is maintained.

Since the control as described above is performed, the illuminance of the subject is determined based on the amplitude ASA of the signal obtained by the detection circuit 13 and the exposure control parameters (aperture, amplification gain, sensitization sensitivity, exposure time) at that time. Can be requested.
In other words, the converted value obtained by performing the reverse calculation based on the exposure control parameter with respect to the output of the detection circuit is a value corresponding to the illuminance of the subject. In other words, it is determined whether the illuminance obtained by the reverse calculation is higher than the high illuminance side reference value, lower luminance side luminance value or less, or in the range between these reference values, and pixel addition is performed according to the determination result (Sensitization sensitivity) can be controlled.

  Since it is configured as described above, it is possible to output an image with optimal brightness with good visibility by sequentially switching aperture control, amplification gain control, pixel addition control, and exposure time control during exposure control. There is an effect that can be done.

  Further, the pixel addition circuit is configured so that the sensitization magnification can be set not by the number of pixels to be added but by the addition coefficient, and the sensitization magnification L can be set not only by an integer but also by a value that can include a decimal. In the control, the pixel addition control can be seamlessly switched using the addition coefficient also with a value after the decimal point, and there is an effect that it is possible to output an easy-to-view image without a sudden change in the brightness of the image in the process of changing the illuminance.

Embodiment 3 FIG.
FIG. 16 shows an imaging apparatus according to Embodiment 3 of the present invention. In FIG. 16, except for the point that the photometric unit 14 is added and the point that the control circuit 12 c is provided instead of the control circuit 12, the same effect is obtained as in the first embodiment.

  The photometry unit 14 measures the illuminance of the subject in the direction in which the light enters the lens 1. Mounting and positioning of the illuminance sensor (not shown) of the photometry unit 14 is determined based on the optical axis of the lens, and the illuminance of the subject imaged by the lens 1 is measured.

  The control circuit 12c is similar to the control circuit 12 of the first embodiment, but has an additional function as follows. That is, the control circuit 12 c controls the aperture of the lens 1 based on the illuminance value supplied from the photometry unit 14, the charge readout timing from the photoelectric conversion element of the CCD image sensor 2 generated by the timing generation circuit 10, and the charge forcing. Control of the discharge timing (accordingly, control of charge accumulation time, that is, exposure time), control of amplification gain of the programmable gain amplifier circuit 4, and control of pixel addition processing of the pixel adder circuit 6 are performed.

The control circuit 12c determines the aperture of the lens 1, the exposure time of the CCD image pickup device 2, the amplification gain of the programmable gain amplification circuit 4, and the sensitization magnification of the pixel addition circuit 6 according to the set value table held in the storage unit 16. Set up.
In the setting value table, the aperture of the lens 1, the exposure time of the CCD image pickup device 2, the amplification gain of the programmable gain amplification circuit 4, and the sensitization magnification of the pixel addition circuit 6 are registered for each illuminance value.

  When the illuminance is bright, the exposure time of the image sensor 2 is set to the standard exposure time Tr based on the frame rate, the amplification gain of the programmable gain amplification circuit 4 is set to 1 time, and the sensitization magnification L of the pixel addition circuit 6 is set to 1 time. Then, the aperture of the lens 1 is reduced (range Sa in FIG. 15A). When the lens 1 reaches the maximum aperture and the illuminance further increases, the exposure time of the image sensor 2 is controlled to be shorter than the standard exposure time Tr (range Se in FIG. 15D).

  When the illuminance decreases, the exposure time of the image sensor 2 is set to the standard exposure time Tr based on the frame rate, the amplification gain of the programmable gain amplification circuit 4 is set to 1 time, and the sensitization magnification of the pixel addition circuit 6 is set to 1 time. The aperture of the lens 1 is opened (range Sa in FIG. 15A). When the lens 1 is opened and the illuminance is further darkened, the amplification gain of the programmable gain amplifier circuit 4 is increased from 1 (range Sb in FIG. 15B). The amplification gain of the programmable gain amplifying circuit 4 is the above upper limit value (the value of the amplification gain when the noise level included in the output of the pixel adding circuit 6 reaches the first predetermined ratio, that is, the noise When the level becomes a maximum gain value that satisfies the constraint that the first predetermined ratio (the upper limit value of the allowable range) is not exceeded, and the illuminance further decreases, the sensitization magnification of the pixel addition circuit 6 is reduced from 1 ×. It is increased (range Sc in FIG. 15C). When it gets darker, the exposure time is lengthened (range Sd in FIG. 15D).

  Since it is configured as described above, an aperture image, amplification gain control, pixel addition control, and exposure time control can be sequentially switched during exposure control to output an image with optimal brightness with good visibility. There is an effect that can be done.

  In the pixel addition circuit, the sensitization magnification can be set not by the number of pixels to be added but by the addition coefficient, and the sensitization magnification L can be set by a value including not only an integer but also a decimal. Among them, the pixel addition control is also seamlessly switched using the addition coefficient with a value after the decimal point, and there is an effect that an easy-to-view image can be output without a sudden change in the brightness of the image in the process of illuminance change.

  In the second and third embodiments, it has been described that the exposure time is lengthened when the signal amplitude is not sufficient even when the sensitization magnification is maximized. This is a result of giving priority to keeping the frame rate unchanged. It is. When the resolution is more important than the frame rate, the control to increase the exposure time is performed first, and the sensitization magnification is increased when the signal amplitude is not sufficient even if the exposure time is increased (for example, up to a predetermined value). Alternatively, the control for increasing the sensitization magnification and the control for extending the exposure time may be performed in parallel.

  DESCRIPTION OF SYMBOLS 1 Lens, 2 CCD image pick-up element, 3 Correlation double sampling processing circuit, 4 Programmable gain amplifier circuit, 5 A / D conversion circuit, 6 pixel addition circuit, 6E pixel extraction part, 6R R pixel addition part, 6G G pixel addition part 6B B pixel addition unit, 7 video signal processing circuit, 8 video signal output terminal, 9 drive circuit, 10 timing generation circuit, 11 synchronization signal generation circuit, 12, 12b, 12c control circuit, 13 detection circuit, 14 photometry unit, 20 CMOS image sensor, 101 input terminal, 103 control terminal, 104 sync signal input terminal, 22-25 1 line delay circuit, 32-55 1 pixel delay circuit, 61-85 multiplier circuit, 90 adder circuit, 95 R signal coefficient setting Circuit, 106 R signal output terminal, 161-185 multiplication circuit, 19 Adder circuit, 195 G signal coefficient setting circuit, 107 G signal output terminal, 261 to 285 multiplication circuit, 290 adder circuits, 295 B signal coefficient setting circuit, 108 B signal output terminal.

Claims (19)

A first type of pixel that detects light of the first color component and generates a corresponding imaging signal; a second type of pixel that detects light of the second color component and generates a corresponding imaging signal; A third type of pixel that detects light of the third color component and generates a corresponding imaging signal; and a fourth type of pixel that detects light of the second color component and generates a corresponding imaging signal; The image pickup unit regularly arranged in the screen with a combination of two horizontal pixels and two vertical pixels as a basic unit, and the image pickup signal output from the image pickup unit is delayed for a predetermined time, and the target pixel and its pixel A pixel extraction unit that extracts and simultaneously outputs signals representing pixel values of surrounding pixels;
An addition coefficient generation unit for generating an addition coefficient;
Using the first type pixel located in the region of 5 horizontal pixels and 5 vertical pixels centered on the target pixel as a reference pixel, only the pixel value of the reference pixel is weighted and added by the addition coefficient, and the first pixel of the target pixel is obtained. A first color component generation unit for generating color component values of
Using the second type and fourth type pixels located in the region of three horizontal pixels and three vertical pixels centered on the target pixel as reference pixels, only the pixel value of the reference pixel is weighted and added by an addition coefficient, A second color component generation unit that generates a second color component value of the pixel;
Using the third type pixel located in the region of 5 horizontal pixels and 5 vertical pixels centered on the target pixel as a reference pixel, only the pixel value of the reference pixel is weighted and added by the addition coefficient, An image pickup apparatus comprising: a third color component generation unit that generates a color component value of:   The imaging apparatus according to claim 1, wherein the first color component is red, the second color component is green, and the third color component is blue, and the pixels are arranged in a Bayer array.   The first type pixel is a pixel provided with a first color filter that transmits light of the first color component, and the second type pixel and the fourth type pixel are the second color. It is a pixel provided with a second color filter that transmits component light, and the third type pixel is a pixel provided with a third color filter that transmits light of the third color component. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized.   The first color component generation unit performs weighted addition using all pixel values of the first type of pixels other than the pixels located at the four corners in the horizontal 5 pixel and vertical 5 pixel regions, The third color component generation unit performs weighted addition using all the pixel values of the third type pixels other than the pixels located at the four corners of the horizontal 5 pixel and vertical 5 pixel regions. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. The addition coefficient used when obtaining the first color component value includes a normalization coefficient that is inversely proportional to the number of reference pixels used when obtaining the first color component value, and a weighting coefficient for each reference pixel. Given by product,
The addition coefficient used when obtaining the second color component value includes a normalization coefficient that is inversely proportional to the number of reference pixels used when obtaining the second color component value, and a weighting coefficient for each reference pixel. Given by product,
The addition coefficient used when obtaining the third color component value is a normalization coefficient that is inversely proportional to the number of reference pixels used when obtaining the third color component value, and a weighting coefficient for each reference pixel. The imaging device according to claim 1, wherein the imaging device is given by a product. The weighting factor for each reference pixel depends on the distance of the pixel from the target pixel and the sensitization magnification,
When the sensitization magnification is relatively small, the weighting factor is smaller as the distance is larger,
The imaging apparatus according to claim 5, wherein as the sensitization magnification increases, the dependency of the weighting factor on the distance decreases.   The imaging apparatus according to claim 6, wherein the addition coefficient generation unit increases the sensitization magnification as the illuminance of the subject is lower. An illuminance information generation unit that generates illuminance information representing the illuminance of the subject is provided,
The imaging apparatus according to claim 7, wherein the addition coefficient generation unit determines the sensitization magnification based on the illuminance information. The imaging apparatus according to claim 8, wherein the illuminance information generation unit includes a photometry unit that measures the illuminance based on light from a subject and generates illuminance information.   The illuminance information generation unit is configured to generate the illuminance based on signal amplitudes and exposure control parameters of the first, second, and third color component values generated by the first, second, and third color component generation units. The imaging apparatus according to claim 8, wherein information is generated. The first color component generation unit includes:
When the target pixel is the first type pixel, the target pixel, the reference pixel located two lines before the target pixel, the reference pixel located two pixels before the target pixel, and the reference pixel located two pixels after the target pixel , And the result of weighted addition of pixel values of reference pixels located two lines after the target pixel is output as the first color component value of the target pixel;
When the target pixel is the second type pixel, a reference pixel located one line before the target pixel, a reference pixel positioned two pixels before the target pixel, and two pixels one line before the target pixel A reference pixel located behind, a reference pixel located one line after the target pixel, a reference pixel located two pixels before the target pixel one line, and a reference pixel located two pixels after the target pixel one line after The result of weighted addition of the values is output as the first color component value of the target pixel,
When the target pixel is the third type pixel, the reference pixel located one pixel before the target pixel, one reference pixel before the target pixel, the reference pixel positioned one pixel before the target pixel, and one line after the target pixel A result of weighted addition of pixel values of a reference pixel located one pixel before and a reference pixel located one pixel after one line after the pixel of interest is output as the first color component value of the pixel of interest,
When the pixel of interest is the fourth type pixel, a reference pixel located one pixel before two lines before the pixel of interest, a reference pixel located one pixel after two lines before the pixel of interest, and one pixel before the pixel of interest , A reference pixel located one pixel after the pixel of interest, a reference pixel located one pixel after the second line of the pixel of interest, and a pixel of a reference pixel located one pixel after the second line of the pixel of interest The image pickup apparatus according to any one of claims 1 to 10, wherein a result of weighted addition of values is output as a first color component value of a target pixel. The second color component generation unit
When the target pixel is the first type or the third type pixel, the reference pixel positioned one line before the target pixel, the reference pixel positioned one pixel before the target pixel, and the reference positioned one pixel after the target pixel The result of weighted addition of the pixel value of the pixel and the reference pixel located one line after the target pixel is output as the second color component value of the target pixel;
When the target pixel is the second type or the fourth type pixel, the target pixel, a reference pixel positioned one pixel before the target pixel, and a reference pixel positioned one pixel before the target pixel one line before The second color component value of the target pixel is obtained by weighted addition of the pixel values of the reference pixel positioned one pixel before the target pixel one pixel before and the reference pixel positioned one pixel after the target pixel one line after The image pickup apparatus according to claim 1, wherein the image pickup apparatus outputs the image as The third color component generation unit
When the target pixel is the first type pixel, the reference pixel located one pixel before the target pixel, one reference pixel before the target pixel, the reference pixel one pixel before the target pixel, and one line after the target pixel A result of weighted addition of pixel values of a reference pixel located one pixel before and a reference pixel located one pixel after one line after the target pixel is output as a third color component value of the target pixel;
When the target pixel is the second type pixel, a reference pixel located one pixel before two lines before the target pixel, a reference pixel located one pixel before two lines before the target pixel, and one pixel before the target pixel , A reference pixel located one pixel after the pixel of interest, a reference pixel located one pixel after the second line of the pixel of interest, and a pixel of a reference pixel located one pixel after the second line of the pixel of interest The result of weighted addition of the values is output as the third color component value of the target pixel,
When the target pixel is the third type pixel, the target pixel, a reference pixel located two lines before the target pixel, a reference pixel located two pixels before the target pixel, and a reference pixel located two pixels after the target pixel , And the result of weighted addition of the pixel values of the reference pixels located two lines after the target pixel is output as the third color component value of the target pixel;
When the pixel of interest is the fourth type of pixel, a reference pixel located one line before the pixel of interest, a reference pixel located two pixels before the line of interest, and two pixels one line before the pixel of interest A reference pixel located behind, a reference pixel located one line after the target pixel, a reference pixel located two pixels before the target pixel one line, and a reference pixel located two pixels after the target pixel one line after The image pickup apparatus according to claim 1, wherein the result of weighted addition of the values is output as a third color component value of the target pixel. The addition coefficient generation unit determines an addition coefficient used in the first color component generation unit.
When the subject illuminance is within a high illuminance range equal to or higher than a predetermined high illuminance side reference value and the target pixel is the first type pixel, the addition coefficient for the reference pixels other than the target pixel is set to zero,
When the subject illuminance is within a high illuminance range equal to or greater than a predetermined reference value on the high illuminance side and the target pixel is the second type, third type, or fourth type pixel, the horizontal 3 centered on the target pixel The addition coefficient for reference pixels not included in the pixel, vertical 3 pixel region is zero,
When the subject illuminance is within a low illuminance range equal to or lower than a predetermined low illuminance side reference value, the addition coefficients for all of the reference pixels are equal to each other, a value other than zero,
When the subject illuminance is within a medium illuminance range between the low illuminance side reference value and the high illuminance side reference value, as the subject illuminance increases, from the value used in the low illuminance range, The imaging apparatus according to claim 11, wherein the addition coefficient is continuously changed to a value to be used. The addition coefficient generation unit determines an addition coefficient used in the second color component generation unit.
When the subject illuminance is in a high illuminance range equal to or higher than a predetermined high illuminance side reference value, and the target pixel is the second type or fourth type pixel, the addition coefficient for the reference pixels other than the target pixel is set to zero. ,
When the subject illuminance is within a low illuminance range equal to or lower than a predetermined low illuminance side reference value and the target pixel is the second type or fourth type pixel, the addition coefficients for all the reference pixels are set to be equal to each other. ,
When the subject illuminance is within a high illuminance range equal to or higher than a predetermined high illuminance side reference value and the target pixel is the first type or the third type pixel, the first addition coefficient for all reference pixels is equal to each other. Value of
When the subject illuminance is within a low illuminance range equal to or lower than a predetermined low illuminance side reference value and the target pixel is the first type or third type pixel, the addition coefficients for all reference pixels are equal to each other, A second value greater than the first value,
When the subject illuminance is within a medium illuminance range between the low illuminance side reference value and the high illuminance side reference value, as the subject illuminance increases, from the value used in the low illuminance range, The imaging apparatus according to claim 12, wherein the addition coefficient is continuously changed to a value to be used. The addition coefficient generation unit determines an addition coefficient used in the third color component generation unit.
When the subject illuminance is within a high illuminance range equal to or higher than a predetermined high illuminance side reference value and the target pixel is the third type pixel, the addition coefficient for the reference pixel other than the target pixel is set to zero,
When the subject illuminance is within a high illuminance range equal to or greater than a predetermined reference value on the high illuminance side and the target pixel is the first type, second type, or fourth type pixel, three horizontal pixels centered on the target pixel , The addition coefficient for reference pixels not included in the vertical 3 pixel region is zero,
When the subject illuminance is within a low illuminance range equal to or less than a predetermined low illuminance side reference value, the addition coefficient for all of the reference pixels is a value other than zero equal to each other,
When the subject illuminance is within a medium illuminance range between the low illuminance side reference value and the high illuminance side reference value, as the subject illuminance increases, from the value used in the low illuminance range, The imaging apparatus according to claim 13, wherein the addition coefficient is continuously changed to a value to be used.   The imaging apparatus according to claim 5, wherein the predetermined normalization coefficient is given by 4 / Np, where Np is the number of the reference pixels. A control circuit that controls the lens aperture and exposure time of the imaging unit; and a signal amplification unit that amplifies the imaging signal output from the imaging unit;
The high illuminance side reference value is that the lens diaphragm is opened, the exposure time is the standard exposure time, the amplification gain of the signal amplification unit, and the level of noise included in the pixel addition circuit does not exceed a predetermined value. Illuminance at which the output of the first to third color component generation units is a predetermined level when the maximum gain value satisfying the constraint condition and the sensitization magnification of the first to third color component generation units is set to 1. The imaging device according to claim 14, 15 or 16.   The imaging device according to claim 14, 15 or 16, wherein the low illuminance side reference value is 1/4 times the high illuminance side reference value.

Images