JP5780786B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus capable of obtaining a captured image with higher sensitivity in a low-illuminance imaging environment.

  As a conventional imaging device, a device configured to increase the sensitivity of a solid-state imaging device and improve the S / N by executing a function of adding all digital signals up to N pixels before is known (for example, , See Patent Document 1).

JP 2000-184274 A (page 4, paragraph 0010)

  Since the conventional image pickup apparatus adds image pickup signals from consecutive adjacent pixels, there is a problem that when a color image pickup element is used, colors are mixed and a color signal cannot be reproduced.

  The present invention has been made to solve the above-described problems. An image pickup apparatus capable of increasing the sensitivity of a color signal without mixing colors even when pixels of an image pickup signal read from a color image pickup element are added. The purpose is to obtain.

The imaging apparatus of the present invention
An imaging unit having a plurality of types of pixels that detect light of a plurality of different color components and output corresponding imaging signals;
A noise reduction circuit for reducing noise of an imaging signal output from the imaging unit;
Based on the imaging signal in which noise has been reduced by the noise reduction circuit, for each target pixel, a region having a high correlation among the regions formed by pixels that detect light of the same color component around each target pixel. An area selection circuit to select;
The imaging signals of the target pixel outputted from the imaging unit, and a selection adding circuit for outputting the addition result as an addition pixel signal by adding the image pickup signal of a pixel included in the realm selected by the area selecting circuit Prepared,
The plurality of color components includes first, second and third color components;
The plurality of types of pixels are
A first type of pixel that detects light of the first color component and generates a corresponding imaging signal; a second type of pixel that detects light of the second color component and generates a corresponding imaging signal; A third type of pixel that detects light of the third color component and generates a corresponding imaging signal; and a fourth type of pixel that detects light of the second color component and generates a corresponding imaging signal; Including
The first to fourth types of pixels are regularly arranged in the screen with a combination of two horizontal pixels and two vertical pixels as a basic unit,
The second type and fourth type pixels are arranged side by side in the direction of one diagonal line in the basic unit, and the first type and third type pixels are arranged side by side in the direction of the other diagonal line. ,
The region selection circuit includes:
A pixel extraction unit that simultaneously extracts signals representing pixel values of a pixel of interest and its surrounding pixels by delaying an imaging signal output from the noise reduction circuit by a predetermined time;
Of the pixels extracted by the pixel extraction unit, forming a plurality of combinations of a target pixel and a plurality of pixels that detect light of the same color component as the target pixel, and among the plurality of combinations, characterized in that it comprises a maximum value and the minimum value of the difference between the correlation determination unit determines that constitutes the high have area of said correlation combining the minimum pixel value of pixels constituting the combination.

  According to the present invention, it is possible to obtain an effect that the sensitivity can be improved by, for example, 4 times even in an image signal read from a color image sensor, and the subject can be visually recognized even in an extremely dark low-light environment.

It is a block block diagram which shows the imaging device of Embodiment 1 of this invention. FIG. 2 is a block configuration diagram illustrating an example of a pixel addition circuit in FIG. 1. It is a figure which shows pixel space arrangement | positioning centering on the attention pixel output from an image pick-up element. FIG. 3 is a block configuration diagram illustrating an example of a noise reduction circuit in FIG. 2. It is a block block diagram which shows an example of the pixel extraction part of FIG. It is a block block diagram which shows an example of the weighted addition part of FIG. FIG. 4 is a diagram illustrating color arrangement information when a pixel of interest is a G pixel in the pixel space arrangement of FIG. 3. FIG. 4 is a diagram illustrating calculation target pixels of the noise reduction circuit when the target pixel is a G pixel in the pixel space arrangement of FIG. 3. FIG. 4 is a diagram illustrating color arrangement information when a pixel of interest is an R pixel in the pixel space arrangement of FIG. 3. FIG. 4 is a diagram illustrating calculation target pixels of the noise reduction circuit when the target pixel is an R pixel in the pixel space arrangement of FIG. 3. FIG. 4 is a diagram illustrating color arrangement information when a pixel of interest is a B pixel in the pixel space arrangement of FIG. 3. FIG. 4 is a diagram illustrating calculation target pixels of the noise reduction circuit when the target pixel is a B pixel in the pixel space arrangement of FIG. 3. FIG. 3 is a block configuration diagram illustrating a part of an example of a region selection circuit in FIG. 2. It is a block block diagram which shows the structural example of 50 A of pixel extraction parts of FIG. It is a block block diagram which shows the structural example of the correlation determination part 50B of FIG. 13, and the pixel designation | designated circuit 590. FIG. (A)-(d) is a figure which shows the 1st to 4th addition pattern when the attention pixel of a pixel addition circuit is a G pixel. (A)-(d) is a figure which shows the 5th-8th addition pattern when the attention pixel of a pixel addition circuit is G pixel. (A)-(d) is a figure which shows the 9th to 12th addition pattern when the pixel of interest of the pixel addition circuit is a G pixel. (A)-(d) is a figure which shows the 1st-4th addition pattern when the attention pixel of a pixel addition circuit is R pixel. (A)-(d) is a figure which shows the 1st to 4th addition pattern when the attention pixel of a pixel addition circuit is a B pixel. FIG. 3 is a block configuration diagram illustrating an example of a selective addition circuit in FIG. 2. It is a block block diagram which shows another structure of the imaging device of Embodiment 1 of this invention. It is a block block diagram which shows the imaging device of Embodiment 2 of this invention. (A) to (e) are the subject illuminance, the lens aperture, the amplification gain of the programmable gain amplification circuit, the sensitization magnification of the pixel addition circuit, the exposure time of the imaging unit, and the amplitude of the output signal of the pixel addition circuit. It is a figure which shows a relationship. It is a block block diagram which shows the imaging device of Embodiment 3 of this invention.

  Hereinafter, the light of the first, second, and third color components is red, green, and blue light, respectively, and pixels that detect the light of these color components and output corresponding signals are arranged in a Bayer array. A case where the image pickup unit is used will be described in detail.

Embodiment 1 FIG.
FIG. 1 shows an imaging apparatus according to Embodiment 1 of the present invention. In FIG. 1, a lens 1 focuses a subject image on an imaging surface of a CCD imaging device 2.

  As shown in FIGS. 8 to 11 described later, the CCD image pickup device (CCD) 2 has an R pixel for detecting red light (first color component light), green light (second color component light). ) For detecting G) and B pixels for detecting blue light (light of the third color component) are arranged in a Bayer array, whereby red light, green light, blue light (first light) 1 color component light, 2nd color component light, 3rd color component light) is detected, that is, photoelectrically converted, and the electric charge generated by the photoelectric conversion is transferred inside the element, and an electric signal It is output as (imaging signal).

  The imaging signal includes an R signal (a signal representing a first color component value) that is a pixel signal from the R pixel, a G signal (a signal representing a second color component value) that is a pixel signal from the G pixel, and A B signal (a signal representing a third color component value) that is a pixel signal from the B pixel is included.

  The R pixel, the G pixel, and the B pixel are, for example, a photoelectric conversion element that includes a color filter that transmits only red light, a photoelectric conversion element that includes a color filter that transmits only green light, and only blue light. It is composed of a photoelectric conversion element provided with a color filter to be transmitted.

Noise or the like is removed from the imaging signal output from the CCD imaging device 2 by a correlated double sampling processing circuit (CDS) 3.
The programmable gain amplifier (PGA) 4 amplifies the output signal of the correlated double sampling processing circuit 3 with a gain controlled by the control signal output from the control circuit 12 and outputs the amplified signal.
The A / D conversion circuit (ADC) 5 converts the output signal of the programmable gain amplifier circuit 4 into a digital signal.

  The pixel addition circuit 6 receives the imaging signal Pc output from the A / D conversion circuit 5, sequentially selects each pixel position as a target pixel position, and has the same color as the target pixel among the target pixel and surrounding pixels. Signals of pixels (pixels that detect light of the same color component) are selectively added. Such processing is performed for all the pixel positions in the imaging screen.

  The video signal processing circuit 7 performs color synchronization processing, gradation correction processing, noise reduction processing, contour correction processing, white balance adjustment processing, signal amplitude adjustment processing, color correction processing, and the like on the output signal of the pixel addition circuit 6. The added video signal is output from the video signal output terminal 8.

  The synchronization signal generation circuit 11 generates a vertical synchronization signal and a horizontal synchronization signal and supplies them to the pixel addition circuit 6, the video signal processing circuit 7 and the timing generation circuit 10.

  The timing generation circuit 10 generates a drive timing signal DRT for the CCD image pickup device 2 and supplies it to the drive circuit 9. The drive circuit 9 generates a drive signal DRS for the CCD image sensor 2 based on the drive timing signal DRT output from the timing generation circuit 10. The CCD image pickup device 2 performs photoelectric conversion and charge transfer based on the drive signal DRS output from the drive circuit 9.

  The control circuit 12 controls the aperture of the lens 1, controls the timing of charge readout from the photoelectric conversion element of the CCD image sensor 2 generated by the timing generation circuit 10, and the control of the forced discharge timing of the charge (therefore, the charge accumulation time, that is, the exposure time Control), control of the amplification gain of the programmable gain amplifier circuit 4, and control of the pixel addition processing of the pixel addition circuit 6.

  As illustrated in FIG. 2, the pixel addition circuit 6 includes a noise reduction circuit 40, a region selection circuit 50, a delay circuit 19, and a selection addition circuit 30.

  As described above, the A / D conversion circuit 5 outputs the imaging signals Pc from the R, G, and B pixels arranged in the Bayer array, and the pixel addition circuit 6 sequentially pays attention to each pixel position in the imaging screen. Addition is performed as a pixel.

  The pixels on the imaging screen of the imaging device 2 are aligned in the horizontal direction (row direction) H and the vertical direction (column direction) V as shown in FIG. 3, and are arranged in a matrix as a whole. The position of each pixel on the screen is represented by a combination (h, v) of the horizontal coordinate h and the vertical coordinate v, and the pixel at the coordinate (h, v) is represented by the symbol Phv. The value of h differs by 1 between pixels adjacent in the horizontal direction, and the value of v differs by 1 between pixels adjacent in the vertical direction. That is, the distance (pixel pitch) between adjacent pixels is represented by 1.

  FIG. 3 shows 5 ranges (5 × 5 pixel range) of 5 horizontal pixels and 5 vertical pixels centering on the target pixel P33 and the arrangement of pixels located in the periphery thereof.

  The imaging signal Pc from the A / D conversion circuit 5 is applied to the input terminal 15 and supplied to the delay circuit 19 and the noise reduction circuit 40.

  The noise reduction circuit 40 receives the imaging signal Pc output from the imaging unit 2 and supplied to the pixel addition circuit 6 via the correlated double sampling processing circuit 3, the programmable gain amplification circuit 4, and the A / D conversion circuit 5. For example, imaging is performed by performing two-dimensional filtering, for example, low-pass filtering, on an imaging signal from a pixel that generates an imaging signal having the same color component as that of the pixel of interest among surrounding pixels. Reduce signal noise.

  Based on the imaging signal Pf whose noise has been reduced by the noise reduction circuit 40, the area selection circuit 50 is configured to detect, for each target pixel, a region formed by pixels that detect light of the same color component around each target pixel. Among these, a region having a high correlation is selected, and the pixel position of the pixel in the selected region is notified to the selection adding circuit 30.

The delay circuit 19 delays the imaging signal Pc for a predetermined time and outputs a delayed imaging signal Pd.
Select adder circuit 30 receives an imaging signal Pd which is delayed, the imaging signal for each pixel of interest, the addition result by adding the image pickup signal of a pixel included in the realm selected by region selection circuit 50 (the pixel area) Is output as the addition pixel signal Pe. The added pixel signal Pe is supplied to the video signal processing circuit 7 via the output terminal 16.

  The output of the delay circuit 19 (delayed imaging signal Pd) is sent to the selective addition circuit 30 in synchronization with the timing of notification from the area selection circuit 50 to the selective addition circuit 30 of the pixel position of the pixel in the selected area. Need to be supplied. The delay circuit 19 is provided for that purpose, and the delay time is determined based on the processing delay time in the noise reduction circuit 40 and the region selection circuit 50.

  As shown in FIG. 4, the noise reduction circuit 40 includes a pixel extraction unit 40A, a weighted addition unit 40B, and a coefficient setting unit 40C.

  The pixel extraction unit 40A is configured, for example, as shown in FIG. In FIG. 5, “1L-DL” indicates a one-line delay circuit, and “1D-DL” indicates a one-pixel delay circuit. The pixel extraction unit 40A is configured by connecting one-line delay circuits 422 to 425 and one-pixel delay circuits 432 to 435, 437 to 440, 442 to 445, 447 to 450, and 452 to 455 as illustrated. And simultaneously output pixel signals indicating the pixel values of the pixel of interest and surrounding pixels. For example, when the target pixel is P33 in FIG. 3, pixel signals representing the pixel values of the target pixel P33 and the surrounding pixels P11 to P55 are output simultaneously.

  The pixel signal Pc output from the A / D conversion circuit 5 is applied to the input terminal 401 of the pixel addition circuit 6 and supplied to the pixel extraction unit 40A of the noise reduction circuit 40.

The input pixel signal Pc is sequentially delayed by one line by the one-line delay circuits 422 to 425 as the pixel signal P55, and the pixel signal P54 delayed by 1, 2, 3, and 4 lines, respectively, with respect to the pixel signal P55. P53, P52, and P51 are output.
The pixel signal P55 is also sequentially delayed by one pixel by the one-pixel delay circuits 432 to 435, and pixel signals P45, P35, P25, and P15 delayed by 1, 2, 3, and 4 pixels, respectively, are output with respect to the pixel signal P55. The

The pixel signal P54 output from the one-line delay circuit 422 is sequentially delayed by one pixel by the one-pixel delay circuits 437 to 440, and pixel signals P44 that are respectively delayed by 1, 2, 3, and 4 pixels with respect to the pixel signal P54. P34, P24, and P14 are output.
The pixel signal P53 output from the one-line delay circuit 423 is sequentially delayed by one pixel by the one-pixel delay circuits 442 to 445, and is delayed by 1, 2, 3, and 4 pixels from the pixel signal P53, respectively. P33, P23, and P13 are output.

The pixel signal P52 output from the one-line delay circuit 424 is sequentially delayed by one pixel by the one-pixel delay circuits 447 to 450, and is delayed by 1, 2, 3, and 4 pixels, respectively. P32, P22, and P12 are output.
The pixel signal P51 output from the one-line delay circuit 425 is sequentially delayed by one pixel by the one-pixel delay circuits 452 to 455, and the pixel signal P41 delayed by 1, 2, 3, and 4 pixels, respectively, with respect to the pixel signal P51. P31, P21, and P11 are output.

The pixel signals P55 to P11 represent the pixel values of the pixels P55 to P11 in FIG. 3, respectively, and are simultaneously output from the pixel extraction unit 40A at the timing when the pixel signal P55 is input.
Pixel signals P55 to P11 are supplied to multiplication circuits 461 to 485, respectively.

  The weighted addition circuit 40B of the noise reduction circuit 40 is configured by connecting multiplication circuits 461 to 485 and an addition circuit 490 as shown in FIG. 6, for example, and is extracted by the pixel extraction unit 40A. Two-dimensional filtering, for example, low-pass filtering, is performed by multiplying the pixel value of each pixel by a coefficient and adding the result.

The filter characteristic FCH of the weighted addition circuit 40B is determined by the control circuit 12, and a coefficient representing the filter characteristic is applied to the control terminal 417 of the noise reduction circuit 40 via the control terminal 17 of the pixel addition circuit 6, and the coefficient setting circuit. 40C, and the weighted addition circuit 40B performs weighted addition using the coefficient set in the coefficient setting circuit 40C.
The horizontal synchronization signal HD and the vertical synchronization signal VD output from the synchronization signal generation circuit 11 are applied to the control terminal 418 of the noise reduction circuit 40 via the synchronization signal input terminal 18 of the pixel addition circuit 6, and then to the coefficient setting circuit 40C. Supplied.

The coefficient setting circuit 40C determines the pixel position of the target pixel P33 based on the horizontal synchronization signal HD and the vertical synchronization signal VD, and specifies the pixel position on the color filter array of the target pixel.
The coefficient setting circuit 40C also specifies whether the pixel of interest is an R pixel, a G pixel, or a B pixel based on the horizontal synchronization signal HD and the vertical synchronization signal VD.
Then, 25 filter coefficients K11 to K55 are supplied independently to each of the multiplication circuits 461 to 485 of the weighted addition unit 40B based on the specified pixel position and the filter coefficient.

  The multiplication circuits 461 to 485 multiply the input pixel values P11 to P55 by the filter coefficients K11 to K55. The addition circuit 490 adds the output values of the multiplication circuits 461 to 485, and outputs the addition result from the output terminal 402 as a pixel value Pf with reduced noise.

  7, 9 and 11 show an arrangement of R pixels, G pixels and B pixels. In this arrangement, R pixels, G pixels, and B pixels are arranged in a checkered pattern, and an arrangement of four pixels of two horizontal pixels and two vertical pixels is repeated as a basic unit. In this basic unit, two G pixels are arranged on one diagonal line, and R pixels and B pixels are arranged on the other diagonal line.

FIG. 7 shows a color filter arrangement of a 5 × 5 pixel region and surrounding pixels when the color filter of the target pixel is a G pixel in the pixel space arrangement of FIG. 3.
R12, R32, R52, R14, R34, and R54 indicate R pixels. G11, G31, G51, G22, G42, G13, G33, G53, G24, G44, G15, G35, and G55 indicate G pixels. B21, B41, B23, B43, B25, and B45 indicate B pixels.

  For example, the color filter array is configured by repeating an array of four pixels of R32, G33, B43, and G42, that is, four pixels of two horizontal pixels and two vertical pixels.

  FIG. 7 illustrates an example in which the neighboring pixels (pixels before and after one pixel) on the same line of G33 are B pixels, but the neighboring pixels on the same line are R pixels as in G22. There is also an array pattern. In this case, the color arrangement is such that the R pixel and the B pixel are interchanged. However, since the pixel to be added is only the G pixel when the target pixel is the G pixel, only a slight correction is made for one color arrangement. Is true.

FIG. 8 shows an array of calculation target pixels when the target pixel is a G pixel in the noise reduction circuit 40 of the pixel addition circuit 6. At this time, the output of the noise reduction circuit 40 is obtained by the following equation.
G = K22 × G22 + K42 × G42
+ K33 x G33
+ K24 x G24 + K44 x G44
The coefficient setting circuit 40C sets a coefficient that is not used in the above expression among the filter coefficients K11 to 55 to “0”.

The coefficient setting circuit 40 </ b> C sets a filter coefficient that further reduces noise based on the setting from the control circuit 12. For example, the following filter coefficients are set.
K22 = 1/8
K24 = 1/8
K33 = 4/8
K42 = 1/8
K44 = 1/8
When the signal amplitude is small at low illuminance, the filter coefficient is set to, for example, an 8-fold coefficient, so that each coefficient is 1 or more in order to prevent even a slight signal component from being lost. To do.

Alternatively, the coefficient setting circuit 40C sets, for example, the following filter coefficients based on the setting from the control circuit 12.
K22 = 1/5
K24 = 1/5
K33 = 1/5
K42 = 1/5
K44 = 1/5
When the signal amplitude is small at low illuminance, the filter coefficient is set to, for example, a factor of 5 each, and thereby each coefficient is set to 1 or more in order to prevent even a slight signal component from being lost. To do.

In the above example, the noise reduction circuit 40 is a noise reduction filter that targets the surrounding five pixels. However, in the case of an imaging signal at low illuminance with a lot of noise, noise reduction for the surrounding nine pixels is performed. It is better to configure it as a filter. At this time, the output of the noise reduction circuit 40 is obtained by the following equation.
G = K31 × G31
+ K22 x G22 + K42 x G42
+ K13 x G13 + K33 x G33 + K53 x G53
+ K24 x G24 + K44 x G44
+ K35 x G35

FIG. 9 shows a color filter arrangement of a 5 × 5 pixel region and surrounding pixels when the color filter of the target pixel is an R pixel in the pixel space arrangement of FIG. 3.
R11, R31, R51, R13, R33, R53, R15, R35, and R55 indicate R pixels. G21, G41, G12, G32, G52, G23, G43, G14, G34, G54, G25, and G45 denote G pixels. B22, B42, B24, and B44 indicate B pixels.

  For example, the color filter array is configured by repeating an arrangement of four pixels of R33, G34, B44, and G43, which is a four-pixel arrangement of two horizontal pixels and two vertical pixels.

FIG. 10 shows an array of calculation target pixels when the target pixel is an R pixel in the noise reduction circuit 40 of the pixel addition circuit 6. At this time, the output of the noise reduction circuit 40 is obtained by the following equation.
R = K31 × R31
+ K13 x R13 + K33 x R33 + K53 x R53
+ K35 x R35
The coefficient setting circuit 40C sets a coefficient that is not used in the above expression among the filter coefficients K11 to 55 to “0”.

The coefficient setting circuit 40 </ b> C sets a filter coefficient that further reduces noise based on the setting from the control circuit 12. For example, the following filter coefficients are set.
K31 = 1/8
K13 = 1/8
K33 = 4/8
K53 = 1/8
K35 = 1/8
When the signal amplitude is small at low illuminance, the filter coefficient is set to, for example, an 8-fold coefficient, so that each coefficient is 1 or more in order to prevent even a slight signal component from being lost. To do.

Alternatively, the coefficient setting circuit 40C sets, for example, the following filter coefficients based on the setting from the control circuit 12.
K31 = 1/5
K13 = 1/5
K33 = 1/5
K53 = 1/5
K35 = 1/5
When the signal amplitude is small at low illuminance, each of the filter coefficients is set to, for example, a factor of five, so that the filter coefficient is 1 or more in order to prevent even a weak signal component from being lost.

In the above example, the noise reduction circuit 40 is a noise reduction filter that targets the surrounding five pixels. However, in the case of an imaging signal at low illuminance with a lot of noise, noise reduction for the surrounding nine pixels is performed. It is better to configure it as a filter. At this time, the output of the noise reduction circuit 40 is obtained by the following equation.
R = K11 × R11 + K31 × R31 + K51 × R51
+ K13 x R13 + K33 x R33 + K53 x R53
+ K15 x R15 + K35 x R35 + K55 x R55

FIG. 11 shows a color filter arrangement of a 5 × 5 pixel region and surrounding pixels when the color filter of the target pixel is a B pixel in the pixel space arrangement of FIG.
R22, R42, R24, and R44 indicate R pixels. G21, G41, G12, G32, G52, G23, G43, G14, G34, G54, G25, and G45 denote G pixels. B11, B31, B51, B13, B33, B53, B15, B35, and B55 indicate B pixels.

  For example, the color filter array is configured by repeating an array of four pixels of R22, G23, B33, and G32, that is, four horizontal pixels and two vertical pixels.

FIG. 12 shows an array of calculation target pixels when the target pixel is a B pixel in the noise reduction circuit 40 of the pixel addition circuit 6. At this time, the output of the noise reduction circuit 40 is obtained by the following equation.
B = K31 × B31
+ K13 x B13 + K33 x B33 + K53 x B53
+ K35 x B35
The coefficient setting circuit 40C sets a coefficient that is not used in the above expression among the filter coefficients K11 to 55 to “0”.

The coefficient setting circuit 40 </ b> C sets a filter coefficient that further reduces noise based on the setting from the control circuit 12. For example, the following filter coefficients are set.
K31 = 1/8
K13 = 1/8
K33 = 4/8
K53 = 1/8
K35 = 1/8
When the signal amplitude is small at low illuminance, the filter coefficient is set to, for example, an 8-fold coefficient, so that each coefficient is 1 or more in order to prevent even a slight signal component from being lost. To do.

Alternatively, the coefficient setting circuit 40C sets, for example, the following filter coefficients based on the setting from the control circuit 12.
K31 = 1/5
K13 = 1/5
K33 = 1/5
K53 = 1/5
K35 = 1/5
When the signal amplitude is small at low illuminance, each of the filter coefficients is set to, for example, a factor of five, so that the filter coefficient is 1 or more in order to prevent even a weak signal component from being lost.

In the above example, the noise reduction circuit 40 is a noise reduction filter that targets the surrounding five pixels. However, in the case of an imaging signal at low illuminance with a lot of noise, noise reduction for the surrounding nine pixels is performed. It is better to configure it as a filter. At this time, the output of the noise reduction circuit 40 is obtained by the following equation.
B = K11 × B11 + K31 × B31 + K51 × B51
+ K13 x B13 + K33 x B33 + K53 x B53
+ K15 x B15 + K35 x B35 + K55 x B55

  In the above example, the noise reduction circuit 40 is configured as a low-pass filter based on, for example, the assumption that noise is distributed in a high frequency range, but is a noise reduction filter according to the characteristics of the noise included in the imaging signal. Better.

  In the above example, the noise reduction circuit 40 is provided to prevent erroneous determination due to noise and to specify a pixel with higher correlation, so that the image quality can be improved without lowering the resolution of the pixel added image.

Next, the operation of the region selection circuit 50 will be described with reference to FIGS.
Based on the imaging signal Rf in which noise is reduced by the noise reduction circuit 40, the area selection circuit 50 is configured to detect, for each target pixel, an area formed by pixels that detect light of the same color component around each target pixel. Among them, a region having a high correlation is selected.

More specifically, the region selection circuit 50 delays the imaging signal output from the noise reduction circuit 40 by a predetermined time, and outputs an imaging signal having the same color component as that of the pixel of interest, which is positioned around the pixel of interest. Each pixel value of a pixel (reference pixel) to be extracted is simultaneously extracted, and each pixel value is composed of a combination of a pixel of interest and a plurality of pixels at a specific relative position with respect to the pixel of interest among surrounding reference pixels multiple realm calculated as each for differences a variation of the maximum and minimum pixel values of (the pixel area), the correlation among the plurality of the pixel regions the variation width is determined minimum pixel region The selective addition circuit 30 is notified as a high pixel region. That is, the information Sp indicating the highly correlated pixel region is supplied to the selective adding circuit 30.

  As shown in FIG. 13, the region selection circuit 50 includes a pixel extraction unit 50A and a correlation determination unit 50B.

The pixel extraction unit 50A delays the imaging signal output from the noise reduction circuit 40 by a predetermined time, and simultaneously extracts signals representing pixel values of the target pixel and the surrounding pixels.
The correlation determination unit 50B forms a plurality of combinations of the pixel of interest extracted from the pixel extraction unit 50A and a plurality of pixels that detect light of the same color component as the pixel of interest, Among the combinations, the combination having the smallest difference between the maximum value and the minimum value of the pixels constituting the combination is determined to constitute the highly correlated pixel region.

  The pixel extraction unit 50A is configured, for example, as shown in FIG. In FIG. 14, “2L-DL” indicates a 2-line delay circuit, “1L-DL” indicates a 1-line delay circuit, “4D-DL” indicates a 4-pixel delay circuit, and “2D-DL” indicates 2 A pixel delay circuit is indicated, and “1D-DL” indicates a one-pixel delay circuit.

  The pixel extraction unit 50A includes two-line delay circuits 511 and 512, one-line delay circuits 522 to 525, four-pixel delay circuits 530 and 531, two-pixel delay circuits 532 to 537, one-pixel delay circuits 542 to 545, 547 to 550, 552 to 555, 557 to 560, and 562 to 565 are connected as shown in the figure, and simultaneously output pixel signals indicating pixel values of the pixel of interest and surrounding pixels.

  For example, when the target pixel is P33 in FIG. 3, pixel signals representing pixel values of the target pixel P33 and surrounding pixels P11 to P55, PL3, P3T, P3B, and PR1 to PR5 are output simultaneously. The same reference numerals P11 to P55 as those used in the description of the pixel extraction unit 40A in FIG. 5 are used. In FIG. 5, the pixel signal Pc is delayed, whereas in FIG. The difference is that Pf is delayed.

The pixel signal Pf output from the noise reduction circuit 40 is applied to the input terminal 501 of the region selection circuit 50.
The pixel signal Pf applied to the input terminal 501 is sequentially delayed by the two-line delay circuit 511, the one-line delay circuits 522 to 525, and the two-line delay circuit 512 as the pixel signal PRB. Pixel signals PR5, PR4, PR3, PR2, PR1, and PRT delayed by 2, 3, 4, 5, 6, and 8 lines are output.
The pixel signal PRB is also delayed by the 4-pixel delay circuit 530 and output as the pixel signal P3B.

The pixel signal PR5 output from the two-line delay circuit 511 is delayed by two pixels by the two-pixel delay circuit 532, and further sequentially delayed by one pixel by the one-pixel delay circuits 542 to 545. Pixel signals P55, P45, P35, P25, and P15 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PR4 output from the 1-line delay circuit 522 is delayed by 2 pixels by the 2-pixel delay circuit 533, and further sequentially delayed by 1 pixel by the 1-pixel delay circuits 547 to 550, so that the pixel signal PR4 is 2 respectively. Pixel signals P54, P44, P34, P24, and P14 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PR3 output from the 1-line delay circuit 523 is delayed by 2 pixels by the 2-pixel delay circuit 534, further delayed by 1 pixel by the 1-pixel delay circuits 552 to 555, and further by 2 by the 2-pixel delay circuit 535. Pixel signals P53, P43, P33, P23, P13, and PL3 delayed by 2, 3, 4, 5, 6, and 8 pixels with respect to the pixel signal PR3 are output.

The pixel signal PR2 output from the one-line delay circuit 524 is delayed by two pixels by the two-pixel delay circuit 536, and further sequentially delayed by one pixel by the one-pixel delay circuits 557 to 560. Pixel signals P52, P42, P32, P22, and P12 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PR1 output from the one-line delay circuit 525 is delayed by two pixels by the two-pixel delay circuit 537, and further sequentially delayed by one pixel by the one-pixel delay circuits 562 to 565. Pixel signals P51, P41, P31, P21, and P11 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PRT output from the 2-line delay circuit 512 is delayed by the 4-pixel delay circuit 531 and output as the pixel signal P3T.

The pixel signals P55 to P11, P3B, PR3, PL3, and P3T represent the pixel values of the pixels P55 to P11, P3B, PR3, PL3, and P3T in FIG. Output simultaneously from the pixel extraction unit 50A.
Pixel signals P55 to P11, P3B, PR3, PL3, and P3T are supplied to a pixel selection circuit 570 in the correlation determination unit 50B.

  For example, as illustrated in FIG. 15, the correlation determination unit 50 </ b> B includes a pixel selection circuit 570, change width calculation circuits 571 to 582, a minimum value calculation circuit 585, and a pixel designation circuit 590.

  In FIG. 15, the horizontal synchronization signal HD and the vertical synchronization signal VD output from the synchronization signal generation circuit 11 are applied to the synchronization signal input terminal 518 of the region selection circuit 50 through the synchronization signal input terminal 18 of the pixel addition circuit 6. , The pixel selection circuit 570, the minimum value calculation circuit 585, and the pixel designation circuit 590.

The pixel selection circuit 570 determines the pixel position of the target pixel P33 based on the horizontal synchronization signal HD and the vertical synchronization signal VD, and specifies the pixel position on the color filter array of the target pixel. The pixel selection circuit 570 also specifies whether the target pixel is an R pixel, a G pixel, or a B pixel.
Then, the pixel signals P55 to P11, P3B, PR3, PL3, and P3T supplied from the pixel extraction unit 50A are received, and a plurality of pixels that are at a specific relative position with respect to the target pixel and the target pixel among the surrounding reference pixels. A combination of pixel values of the pixels is generated. A plurality of such combinations are generated for each pixel of interest.

  If a pixel having a high correlation with the pixel of interest can be correctly selected as a pixel used for pixel addition in the selective addition circuit 30, resolution degradation of the image after pixel addition can be reduced. Therefore, a plurality of combinations of the target pixel and peripheral pixels at a specific relative position with respect to the target pixel are formed, the correlation height is obtained for each combination, and pixel addition is performed using the pixels belonging to the combination with the highest correlation. I do.

  In order to obtain the level of correlation for each combination, the difference between the maximum value and the minimum value of the pixels belonging to each combination is obtained as the change width, and the combination with the smallest change width is determined as the combination with the highest correlation. . Since the pixels belonging to the combination having the highest correlation are used for pixel addition, the combination is called an addition pattern.

  FIGS. 16A to 18D show addition patterns of 4-pixel addition when the color filter of the pixel of interest is G pixels in the pixel space arrangement of FIG. In the case of 4-pixel addition of G pixels, an optimal addition pattern having the highest correlation is obtained from 12 addition patterns.

  FIG. 16A shows the target pixel and its surrounding pixels, that is, the target pixel G33, the pixel G31 two lines before the target pixel, the pixel G22 one pixel before the target pixel, and the target pixel. An upper block pattern GP1 constituted by a combination of pixels G42 after one pixel before one line is shown.

  FIG. 16B shows the target pixel and its neighboring pixels on the right side, that is, the target pixel G33, the pixel G53 two pixels after the target pixel, the pixel G42 one pixel before the target pixel one line, and the target pixel A right block pattern GP2 configured by a combination of the pixel G44 after one line after one line is shown.

  FIG. 16C shows the target pixel and its peripheral pixels, that is, the target pixel G33, the pixel G13 two pixels before the target pixel, the pixel G22 one pixel before the target pixel, and the target pixel. A right block pattern GP2 configured by a combination of pixels G24 one pixel before one line is shown.

  FIG. 16D shows the target pixel and its lower peripheral pixels, that is, the target pixel G33, the pixel G35 two lines after the target pixel, the pixel G24 one pixel before the first line of the target pixel, and the target pixel. The upper block pattern GP4 configured by the combination of the pixel G44 one pixel after the first line is shown.

  FIG. 17A shows the target pixel and the surrounding pixels above it, that is, the target pixel G33, the pixel G3T four lines before the target pixel, the pixel G31 two lines before the target pixel, and two lines after the target pixel. An upper vertical line pattern GP constituted by a combination of pixels G35 is shown.

  FIG. 17B shows the target pixel and its lower peripheral pixels, that is, the target pixel G33, the pixel G3B four lines after the target pixel, the pixel G35 two lines after the target pixel, and two lines before the target pixel. A lower vertical line pattern GP6 constituted by a combination of the pixels G31 is shown.

  FIG. 17C shows the target pixel and its peripheral pixels on the left side, that is, the target pixel G33, the pixel GL3 four pixels before the target pixel, the pixel G13 two pixels before the target pixel, and two pixels after the target pixel. A left lateral line pattern GP7 configured by a combination of pixels G53 is shown.

  FIG. 17D shows the target pixel and its peripheral pixels on the right side, that is, the target pixel G33, the pixel GR3 four pixels after the target pixel, the pixel G53 two pixels after the target pixel, and two pixels before the target pixel. A right lateral line pattern GP8 configured by a combination of pixels G13 is shown.

  FIG. 18A shows the target pixel and the upper left peripheral pixel, that is, the target pixel G33, the pixel G11 two pixels before the target pixel two lines before, and the pixel G22 one pixel before the target pixel one line before , And a left upper diagonal line pattern GP9 composed of a combination of a pixel G44 one pixel after one line of the target pixel.

  FIG. 18B shows the target pixel and its lower right peripheral pixel, that is, the target pixel G33, the pixel G55 two pixels after the second line of the target pixel, and the pixel one pixel after the first line of the target pixel. A lower right diagonal line pattern GP10 including a combination of G44, a pixel one pixel before the pixel of interest and one pixel before G22, and G22 is shown.

  FIG. 18C shows the target pixel and the peripheral pixels on the upper right side thereof, that is, the target pixel G33, the pixel G51 two pixels before the target pixel two lines before, and the pixel G42 one pixel after the first line of the target pixel. And an oblique line pattern GP11 on the upper right side constituted by a combination of the pixel G24 one pixel before and one pixel after the target pixel.

  FIG. 18D shows the target pixel and the lower left peripheral pixel, that is, the target pixel G33, the pixel G15 two pixels before the second line of the target pixel, and the pixel G24 one pixel before the one line after the target pixel. , And a diagonal line pattern GP12 on the lower left side constituted by a combination of a pixel G42 that is one pixel before and one pixel before the target pixel.

  The pixel selection circuit 570 calculates the first to twelfth change widths using the pixel values of the pixels constituting the patterns GP1 to GP12 as the pixel values constituting the first to twelfth addition patterns AP1 to AP12, respectively. Supply to circuits 571-582.

  19A to 19D show addition patterns of 4-pixel addition when the color filter of the pixel of interest is an R pixel in the pixel space arrangement of FIG. In the case of four-pixel addition of R pixels, an optimum addition pattern having the highest correlation is obtained from the four addition patterns.

  FIG. 19A shows the target pixel and its upper left peripheral pixel, that is, the target pixel R33, the pixel R31 two lines before the target pixel, the pixel R11 two pixels before the target pixel two lines, and the target pixel This shows an upper left block pattern RP1 composed of a combination of the pixel R13 two pixels before.

  FIG. 19B shows the target pixel and the peripheral pixels on the upper right side thereof, that is, the target pixel R33, the pixel R31 two lines before the target pixel, the pixel R51 two pixels after the two lines before the target pixel, and the target pixel. An upper right side block pattern RP2 composed of a combination of the pixel R53 after the second pixel is shown.

  FIG. 19C shows the target pixel and the lower left peripheral pixel, that is, the target pixel R33, the pixel R13 two pixels before the target pixel, the pixel R35 two lines after the target pixel, and two lines after the target pixel. The lower left side block pattern RP3 comprised by the combination of the pixel R15 of the 2 previous pixels is shown.

  FIG. 19D shows the target pixel and its lower right peripheral pixels, that is, the target pixel R33, a pixel R53 two pixels after the target pixel, a pixel R35 two lines after the target pixel, and two lines of the target pixel. The lower right block pattern RP4 configured by the combination of the pixel R55 after the subsequent two pixels is shown.

  The pixel selection circuit 570 supplies the patterns PR1 to PR4 to the first to fourth change width calculation circuits 571 to 574 as first to fourth addition patterns AP1 to AP4, respectively.

  FIGS. 20A to 20D show addition patterns of 4-pixel addition when the color filter of the target pixel is B pixel in the pixel space arrangement of FIG. In the case of four-pixel addition of B pixels, an optimum addition pattern having the highest correlation is obtained from the four addition patterns. An addition pattern in the case of adding four B pixels is shown in FIG.

  FIG. 20A illustrates the target pixel and the upper left peripheral pixel, that is, the target pixel B33, the pixel B31 two lines before the target pixel, the pixel B11 two pixels before the target pixel two lines, and the target pixel. The upper left block pattern BP1 including the combination of the pixel B13 two pixels before is shown.

  FIG. 20B shows the target pixel and the peripheral pixels on the upper right side thereof, that is, the target pixel B33, the pixel B31 two lines before the target pixel, the pixel B51 two pixels before the second line of the target pixel, and the target pixel. An upper right side block pattern BP2 constituted by a combination of the pixel B53 after two pixels is shown.

  FIG. 20C shows the target pixel and the lower left peripheral pixel, that is, the target pixel B33, the pixel B13 two pixels before the target pixel, the pixel B35 two lines after the target pixel, and two lines after the target pixel. The lower left side block pattern BP3 comprised by the combination of pixel B15 of 2 pixels before is shown.

  FIG. 20D shows the target pixel and its lower right peripheral pixels, that is, the target pixel B33, a pixel B53 two pixels after the target pixel, a pixel B35 two lines after the target pixel, and two lines of the target pixel. A lower right block pattern BP4 constituted by a combination of the pixel B55 after the subsequent two pixels is shown.

  The pixel selection circuit 570 supplies the patterns PB1 to PB4 to the first to fourth change width calculation circuits 571 to 574 as the first to fourth addition patterns AP1 to AP4, respectively.

  The first to twelfth change width calculation circuits 571 to 582 respectively receive the pixel values constituting the input first to twelfth addition patterns AP1 to AP12, that is, attention among the target pixel and the surrounding reference pixels. The difference between the maximum pixel value and the minimum pixel value is calculated as a change width for each of a plurality of pixel regions (addition patterns) configured by a combination of pixel values of a plurality of pixels at specific relative positions with respect to the pixel.

  That is, the change width calculation circuits 571 to 582 compare the input pixel values of the four pixels to obtain the maximum pixel value and the minimum pixel value. Next, the difference between the maximum pixel value and the minimum pixel value is obtained and supplied to the minimum value calculation circuit 585 as the change width of the addition pattern (pixel area).

When the target pixel is a G pixel,
The change width calculation circuit 571 calculates a change width between pixels of the upper block pattern GP1 input as the first addition pattern AP1, and outputs the change width as the first change width WP1.
The change width calculation circuit 572 calculates the change width between the pixels of the right block pattern GP2 input as the second addition pattern AP2, and outputs the change width as the second change width WP2.
The change width calculation circuit 573 calculates the change width between the pixels of the left block pattern GP3 input as the third addition pattern AP3, and outputs it as the third change width WP3.
The change width calculation circuit 574 calculates the change width between the pixels of the lower block pattern GP4 input as the fourth addition pattern AP4, and outputs it as the fourth change width WP4.
The change width calculation circuit 575 calculates a change width between pixels of the upper vertical line pattern GP5 input as the fifth addition pattern AP5, and outputs the change width as a fifth change width WP5.
The change width calculation circuit 576 calculates the change width between the pixels of the lower vertical line pattern GP6 input as the sixth addition pattern AP6, and outputs it as the sixth change width WP6.
The change width calculation circuit 577 calculates the change width between the pixels of the left horizontal line pattern GP7 input as the seventh addition pattern AP7, and outputs it as the seventh change width WP7.
The change width calculation circuit 578 calculates the change width between the pixels of the right lateral line pattern GP8 input as the eighth addition pattern AP8, and outputs it as the eighth change width WP8.
The change width calculation circuit 579 calculates the change width between the pixels of the upper left diagonal line pattern GP9 input as the ninth addition pattern AP9, and outputs the change width as the ninth change width WP9.
The change width calculation circuit 580 calculates the change width between the pixels of the lower right oblique line pattern GP10 input as the tenth addition pattern AP10, and outputs it as the tenth change width WP10.
The change width calculation circuit 581 calculates the change width between the pixels of the upper right diagonal line pattern GP11 input as the eleventh addition pattern AP11, and outputs it as the eleventh change width WP11.
The change width calculation circuit 582 calculates a change width between pixels of the lower left oblique line pattern GP12 input as the twelfth addition pattern AP12, and outputs the change width as the twelfth change width WP12.

When the target pixel is an R pixel or a B pixel,
The change width calculation circuit 571 calculates the change width between the pixels of the upper left block pattern RP1 or BP1 input as the first addition pattern AP1, and outputs the change width as the first change width WP1.
The change width calculation circuit 572 calculates the change width between the pixels of the upper right block pattern RP2 or BP2 input as the second addition pattern AP2, and outputs it as the second change width WP2.
The change width calculation circuit 573 calculates the change width between the pixels of the lower left block pattern RP3 or BP3 input as the third addition pattern AP3, and outputs it as the third change width WP3.
The change width calculation circuit 574 calculates the change width between the pixels of the lower right block pattern RP4 or BP4 input as the fourth addition pattern AP4, and outputs it as the fourth change width WP4.
The change width calculation circuits 575 to 582 do not perform a change width calculation operation.

An operation when the target pixel of the minimum value calculation circuit 585 is a G pixel will be described.
The minimum value calculation circuit 585 receives the first to twelfth change widths WP1 to WP12 output from the change width calculation circuits 571 to 582, obtains the minimum one of them, and determines the addition pattern APM having the minimum change width as a pixel. This is notified to the designation circuit 590.
The pixel designation circuit 590 outputs information Sp indicating the pixel position (pixel area constituting the pixel) of the pixel constituting the addition pattern (selected addition pattern) APM notified from the minimum value calculation circuit 585 to the output terminal 502. To the selective addition circuit 30.

  By such processing, the correlation area detection unit 595 configured by the minimum value calculation circuit 585 and the pixel designation circuit 590 causes the first to twelfth change widths WP1 to WP12 output from the change width calculation circuits 571 to 582. In response, the pixel region including the pixels constituting the addition pattern having the smallest change width is determined as a highly correlated pixel region, and information Sp for identifying the region is supplied to the selective addition circuit 30.

When the constituent pixels of the upper block pattern GP1 have the smallest change width, the pixel position information Sp of the pixels G31, G22, G42, and G33 is supplied to the selective addition circuit 30.
When the constituent pixels of the right block pattern GP2 have the smallest change width, the pixel position information Sp of the pixels G42, G33, G53, and G44 is supplied to the selective addition circuit 30.
When the constituent pixels of the left block pattern GP3 have the smallest change width, the pixel position information Sp of the pixels G22, G13, G33, and G24 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower block pattern GP4 have the smallest change width, the pixel position information Sp of the pixels G33, G24, G44, and G35 is supplied to the selective addition circuit 30.

When the constituent pixels of the upper vertical line pattern GP5 have the smallest change width, the pixel position information Sp of the pixels G3T, G31, G33, and G35 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower vertical line pattern GP6 have the smallest change width, the pixel position information Sp of the pixels G31, G33, G35, and G3B is supplied to the selective addition circuit 30.
When the constituent pixels of the left lateral line pattern GP7 have the smallest change width, the pixel position information Sp of the pixels GL3, G13, G33, and G53 is supplied to the selective addition circuit 30.
When the constituent pixels of the right lateral line pattern GP8 have the smallest change width, the pixel position information Sp of the pixels G13, G33, G53, and GR3 is supplied to the selective addition circuit 30.

When the constituent pixels of the upper left diagonal line pattern GP9 have the smallest change width, the pixel position information Sp of the pixels G11, G22, G33, and G44 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower right diagonal line pattern GP10 have the smallest change width, the pixel position information Sp of the pixels G22, G33, G44, and G55 is supplied to the selective addition circuit 30.
When the constituent pixels of the upper right diagonal line pattern GP11 have the smallest change width, the pixel position information Sp of the pixels G51, G42, G33, and G24 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower left diagonal line pattern GP12 have the smallest change width, the pixel position information Sp of the pixels G42, G33, G24, and G15 is supplied to the selective addition circuit 30.

An operation when the target pixel of the minimum value calculation circuit 585 is an R pixel will be described.
The minimum value calculation circuit 585 receives the first to fourth change widths WP1 to WP4 output from the change width calculation circuits 571 to 574, obtains the minimum one of them, and determines the addition pattern APM having the minimum change width as a pixel. This is notified to the designation circuit 590.
The pixel designation circuit 590 outputs information Sp indicating the pixel position (pixel area constituting the pixel) of the pixel constituting the addition pattern (selected addition pattern) APM notified from the minimum value calculation circuit 585 to the output terminal 502. To the selective addition circuit 30.

  Through such processing, the correlation area detection unit 595 receives the first to fourth change widths WP1 to WP4 output from the change width calculation circuits 571 to 574, and receives the change patterns from the pixels constituting the addition pattern with the minimum change width. Is determined as a highly correlated pixel area, and information Sp for identifying the area is supplied to the selective addition circuit 30.

When the constituent pixels of the upper left block pattern RP1 have the smallest change width, the pixel position information Sp of the pixels R11, R31, R13, and R33 is supplied to the selective addition circuit 30.
When the constituent pixels of the upper right block pattern RP2 have the smallest change width, the pixel position information Sp of the pixels R31, R51, R33, and R53 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower left block pattern RP3 have the smallest change width, the pixel position information Sp of the pixels R13, R33, R15, and R35 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower right block pattern RP4 have the smallest change width, the pixel position information Sp of the pixels R33, R53, R35, and R55 is supplied to the selective addition circuit 30.

An operation when the target pixel of the minimum value calculation circuit 585 is a B pixel will be described.
The minimum value calculation circuit 585 receives the first to fourth change widths WP1 to WP4 output from the change width calculation circuits 571 to 574, obtains the minimum one of them, and determines the addition pattern APM having the minimum change width as a pixel. This is notified to the designation circuit 590.
The pixel designation circuit 590 outputs information Sp indicating the pixel position (pixel area constituting the pixel) of the pixel constituting the addition pattern (selected addition pattern) APM notified from the minimum value calculation circuit 585 to the output terminal 502. To the selective addition circuit 30.

  Through such processing, the correlation area detection unit 595 receives the first to fourth change widths WP1 to WP4 output from the change width calculation circuits 571 to 574, and receives the change patterns from the pixels constituting the addition pattern with the minimum change width. Is determined as a highly correlated pixel area, and information Sp for identifying the area is supplied to the selective addition circuit 30.

When the constituent pixels of the upper left block pattern BP1 have the smallest change width, the pixel position information Sp of the pixels B11, B31, B13, and B33 is supplied to the selective addition circuit 30.
When the constituent pixels of the upper right block pattern BP2 have the smallest change width, the pixel position information Sp of the pixels B31, B51, B33, and B53 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower left block pattern BP3 have the smallest change width, the pixel position information Sp of the pixels B13, B33, B15, and B35 is supplied to the selective addition circuit 30.
When the constituent pixels of the lower right block pattern BP4 have the smallest change width, the pixel position information Sp of the pixels B33, B53, B35, and B55 is supplied to the selective addition circuit 30.

  As described above, an optimum addition pattern having the highest correlation is obtained from 12 addition patterns in the case of G pixels, and from 4 addition patterns in the case of R pixels and B pixels. A pixel having a high correlation with the target pixel can be correctly selected as the pixel, and resolution degradation of the image after pixel addition can be reduced.

Next, the selective addition circuit 30 will be described with reference to FIG.
The selective addition circuit 30 is output from the imaging unit 2 and supplied via the correlated double sampling processing circuit 3, the programmable gain amplification circuit 4, the A / D conversion circuit 5, and further via the delay circuit 19 in the pixel addition circuit 6. The image pickup signal Pd of each pixel of interest is received, among them, the image pickup signals of the pixels included in the pixel region selected by the region selection circuit 50 are added, and the addition result is output as the addition pixel signal Pe.

The selective addition circuit 30 is configured as shown in FIG. 21, for example.
In FIG. 21, “2L-DL” indicates a 2-line delay circuit, “1L-DL” indicates a 1-line delay circuit, “4D-DL” indicates a 4-pixel delay circuit, and “2D-DL” indicates 2 A pixel delay circuit is indicated, and “1D-DL” indicates a one-pixel delay circuit.

  2-line delay circuits 311, 331, 1-line delay circuits 322-325, 4-pixel delay circuits 330, 331, 2-pixel delay circuits 332-337, 1-pixel delay circuits 342-345, 347-350, 352 355, 357 to 360, and 362 to 365 are connected as shown in the figure, and simultaneously output pixel signals indicating pixel values of the pixel of interest and surrounding pixels.

  For example, when the target pixel is P33 in FIG. 3, pixel signals representing pixel values of the target pixel P33 and surrounding pixels P11 to P55, PL3, P3T, P3B, and PR1 to PR5 are output simultaneously. Note that the same symbols P11 to P55, PL3, P3T, P3B, PRT, PR1 to PR5, and PRB are used as in the description of the pixel extraction unit 40A in FIG. 5 and the pixel extraction unit 50A in FIG. In FIG. 5, the pixel signal Pc is delayed, and in FIG. 14, the pixel signal Pf is delayed, whereas in FIG. 21, the pixel signal Pd is delayed.

The imaging signal Pd output from the delay circuit 19 is applied to the input terminal 301 of the selective addition circuit 30.
The input pixel signal Pd is sequentially delayed by the two-line delay circuit 311, the one-line delay circuits 322 to 325, and the two-line delay circuit 312 as the pixel signal PRB. Pixel signals PR5, PR4, PR3, PR2, PR1, and PRT delayed by 4, 5, 6, and 8 lines are output.
The pixel signal PRB is also delayed by the 4-pixel delay circuit 330 and output as the pixel signal P3B.

The pixel signal PR5 output from the two-line delay circuit 311 is delayed by two pixels by the two-pixel delay circuit 332, and further sequentially delayed by one pixel by the one-pixel delay circuits 342 to 345. Pixel signals P55, P45, P35, P25, and P15 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PR4 output from the 1-line delay circuit 322 is delayed by 2 pixels by the 2-pixel delay circuit 333, and further sequentially delayed by 1 pixel by the 1-pixel delay circuits 347 to 350. Pixel signals P54, P44, P34, P24, and P14 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PR3 output from the 1-line delay circuit 323 is delayed by 2 pixels by the 2-pixel delay circuit 334, further delayed by 1 pixel by the 1-pixel delay circuits 352 to 355, and further by 2 by the 2-pixel delay circuit 335. Pixel signals P43, P33, P23, P13, and PL3 that are delayed by 2, 3, 4, 8, and 8 pixels with respect to the pixel signal PR3 are output.

The pixel signal PR2 output from the one-line delay circuit 324 is delayed by two pixels by the two-pixel delay circuit 336, and further sequentially delayed by one pixel by the one-pixel delay circuits 357 to 360. Pixel signals P52, P42, P32, P22, and P12 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PR1 output from the 1-line delay circuit 325 is delayed by 2 pixels by the 2-pixel delay circuit 337, and further sequentially delayed by 1 pixel by the 1-pixel delay circuits 362 to 365. Pixel signals P51, P41, P31, P21, and P11 delayed by 3, 4, 5, and 6 pixels are output.
The pixel signal PRT output from the 2-line delay circuit 312 is delayed by the 4-pixel delay circuit 331 and output as the pixel signal P3T.

  The pixel signals P55 to P11, P3B, PR3, PL3, and P3T represent the pixel values of the pixels P55 to P11, P3B, PR3, PL3, and P3T in FIG. Simultaneously output and supplied to the pixel selection circuit 370.

In FIG. 21, the horizontal synchronizing signal HD and the vertical synchronizing signal VD output from the synchronizing signal generating circuit 11 are applied to the synchronizing signal input terminal 318 of the selective adding circuit 30 through the synchronizing signal input terminal 18 of the pixel adding circuit 6. , And supplied to the pixel selection circuit 370.
Information Sp indicating the pixel position of the pixel belonging to the addition pattern notified from the pixel specifying circuit 590 of the area selection circuit 50 is supplied to the pixel selection circuit 370 via the pixel position input terminal 319 of the selection addition circuit 30.

The pixel selection circuit 370 determines the pixel position of the target pixel P33 based on the horizontal synchronization signal HD and the vertical synchronization signal VD, and specifies the pixel position on the color filter array of the target pixel.
The pixel selection circuit 370 also specifies whether the target pixel is an R pixel, a G pixel, or a B pixel. Then, the pixel positions of the pixels constituting the addition pattern (the addition pattern selected as having the highest correlation with the target pixel) indicated by the information Sp from the pixel specifying circuit 590 are also specified.

  The pixel selection circuit 370 supplies the pixel values Ps1 to Ps4 of the four pixels constituting the selected addition pattern to the addition circuit 390. The sensitization magnification L of 1 to 4 times output from the control circuit 12 is applied to the control terminal 317 of the selective addition circuit 30 via the control terminal 17 of the pixel addition circuit 6 and supplied to the addition circuit 390.

The addition circuit 390 adds the pixel values of the four pixels supplied from the pixel selection circuit 370 and supplies the addition result to the video signal processing circuit 7 via the output terminal 302 as an addition pixel signal Pe. In this addition, an addition coefficient is applied to the pixel value before the addition so that the addition result has a predetermined sensitization magnification L. That is, weighted addition using the addition coefficient is performed.
For example, if the pixel values of four pixels are Ps1, Ps2, Ps3, Ps4 and the sensitization magnification is L,
Pe = (Ps1 + Ps2 + Ps3 + Ps4) × L / 4
The added pixel value Pe is obtained by the calculation represented by In the following description, the added pixel value for the G pixel is indicated by Ge, the added pixel value for the R pixel is indicated by Re, and the added pixel value for the B pixel is indicated by Be.

The sensitization magnification L is determined, for example, in relation to the subject illuminance. For example, when the subject illuminance is equal to or higher than a first predetermined value (high illuminance side reference value), the sensitization magnification L is set to 1, and a second predetermined value (low illuminance) lower than the first predetermined value. (Side reference value) or less, the sensitization magnification is set to 4, and in the range lower than the high illuminance side reference value and higher than the low illuminance side reference (medium illuminance range), the sensitization magnification is gradually increased as the illuminance decreases. .
In the weighted addition, the addition coefficient multiplied by the pixel value of each pixel depends on the sensitization magnification.
A different addition coefficient may be used for each pixel. For example, when the sensitization magnification is 1, the addition coefficient for the pixel of interest is 1, and the addition coefficient for the other pixels is 0. When the sensitization magnification is 4, for example, the addition coefficient for all pixels is The same value may be used.
When the sensitization magnification is between 1 and 4, the addition coefficient may be continuously changed from the value when the sensitization magnification is 1 to the value when the sensitization magnification is the maximum value.

An operation when the pixel of interest is a G pixel will be described.
When the upper block pattern GP1 as the first addition pattern AP1 is selected, the adder circuit 390 performs the following calculation.
Ge = (G31 + G22 + G42 + G33) × L / 4
When the right block pattern GP2 as the second addition pattern AP2 is selected, the adder circuit 390 performs the following calculation.
Ge = (G42 + G33 + G53 + G44) × L / 4
When the left block pattern GP3 as the third addition pattern AP3 is selected, the adder circuit 390 performs the following calculation.
Ge = (G22 + G13 + G33 + G24) × L / 4
When the lower block pattern GP4 is selected as the fourth addition pattern AP4, the addition circuit 390 performs the following calculation.
Ge = (G33 + G24 + G44 + G35) × L / 4

When the upper vertical line pattern GP5 as the fifth addition pattern AP5 is selected, the addition circuit 390 performs the following calculation.
Ge = (G3T + G31 + G33 + G35) × L / 4
When the lower vertical line pattern GP6 is selected as the sixth addition pattern AP6, the addition circuit 390 performs the following calculation.
Ge = (G31 + G33 + G35 + G3B) × L / 4
When the left lateral line pattern GP7 as the seventh addition pattern AP7 is selected, the adder circuit 390 performs the following calculation.
Ge = (GL3 + G13 + G33 + G53) × L / 4
When the right lateral line pattern GP8 as the eighth addition pattern AP8 is selected, the addition circuit 390 performs the following calculation.
Ge = (G13 + G33 + G53 + GR3) × L / 4

When the upper left diagonal line pattern GP9 as the ninth addition pattern AP9 is selected, the addition circuit 390 performs the following calculation.
Ge = (G11 + G22 + G33 + G44) × L / 4
When the lower right diagonal line pattern GP10 as the tenth addition pattern AP10 is selected, the addition circuit 390 performs the following calculation.
Ge = (G22 + G33 + G44 + G55) × L / 4
When the upper right diagonal line pattern GP11 as the eleventh addition pattern AP11 is selected, the addition circuit 390 performs the following calculation.
Ge = (G51 + G42 + G33 + G24) × L / 4
When the lower left diagonal line pattern GP12 as the twelfth addition pattern AP12 is selected, the addition circuit 390 performs the following calculation.
G = (G42 + G33 + G24 + G15) × L / 4

An operation when the target pixel is an R pixel will be described.
When the upper left block pattern RP1 as the first addition pattern AP1 is selected, the adder circuit 390 performs the following calculation.
Re = (R11 + R31 + R13 + R33) × L / 4
When the upper right block pattern RP2 as the second addition pattern AP2 is selected, the adder circuit 390 performs the following calculation.
Re = (R31 + R51 + R33 + R53) × L / 4
When the lower left block pattern RP3 as the third addition pattern AP3 is selected, the adder circuit 390 performs the following calculation.
Re = (R13 + R33 + R15 + R35) × L / 4
When the lower right block pattern RP4 as the fourth addition pattern AP4 is selected, the adder circuit 390 performs the following calculation.
Re = (R33 + R53 + R35 + R55) × L / 4

An operation when the pixel of interest is a B pixel will be described.
When the upper left block pattern BP1 as the first addition pattern AP1 is selected, the adder circuit 390 performs the following calculation.
Be = (B11 + B31 + B13 + B33) × L / 4
When the upper right block pattern BP2 as the second addition pattern AP2 is selected, the adder circuit 390 performs the following calculation.
Be = (B31 + B51 + B33 + B53) × L / 4
When the lower left block pattern BP3 as the third addition pattern AP3 is selected, the adder circuit 390 performs the following calculation.
Be = (B13 + B33 + B15 + B35) × L / 4
When the lower right block pattern BP4 as the fourth addition pattern AP4 is selected, the adder circuit 390 performs the following calculation.
Be = (B33 + B53 + B35 + B55) × L / 4

  In the above example, for the G pixel, pixel addition is configured to use an addition pattern that assumes an image including a high-resolution subject, such as a vertical, horizontal, diagonal line pattern, or block pattern. Can be added. Even when pixels are added to a scene including a high-resolution subject, there is an effect of preventing the high-resolution portion from blurring.

In the above example, only the block pattern is used for the R pixel and the B pixel. However, as the addition pattern of the R pixel and the B pixel, the vertical, horizontal, and diagonal line patterns are used as well as the G pixel, and these are correlated. You may comprise so that it may determine.
However, in the case of the R pixel and the B pixel, since the distance between the pixels to be added is longer than that of the G pixel, the risk of erroneous determination of the correlation determination is increased, the circuit scale is increased, and the human scale is increased. The addition pattern is determined by comprehensively judging that the sensitivity to the color change is lower than the luminance.

  Further, for the G pixel, not all of the above 12 addition patterns may be used, but only a part thereof may be used. For example, only the four patterns shown in FIGS. 16A to 16D may be used, or only the four patterns shown in FIGS. 17A to 17D may be used. Only the four patterns shown in (d) may be used.

  As described in the above example, pixel addition is performed using a highly correlated pixel that is closest to the pixel of interest, so that it is possible to improve sensitivity while minimizing degradation of image resolution.

  As described in the above example, by performing pixel addition immediately after being output from the image sensor (that is, before processing by the video signal processing circuit 7), pixel addition is not affected by video signal processing. High sensitivity signal can be generated. In the pixel addition performed in the subsequent stage of the video signal processing, since the color synchronization processing and the filter processing are performed using the pixels located in the vicinity, the horizontal resolution and the vertical resolution are reduced more than expected. Further, since the video signal processing is performed with the small amplitude signal, there is a possibility of gradation drop. By performing the pixel addition immediately after being output from the image sensor, the signal amplitude can be recovered by pixel addition before the image information is lost, so that there is an effect that the detailed image information can be visually recognized.

  In addition, since non-linear filter processing and gradation conversion processing are performed in the video signal processing, the signal amplitude may be lost when a low amplitude signal is input. For this reason, even if two pixels are added to the output signal of the video signal processing, there is a possibility that the image signal is not doubled. In the example described above, the pixels are added before the video signal processing. Therefore, when two pixels are added, there is an effect that the image signal is doubled.

  In the above example, the case where the sensitization magnification L up to 4 times is set has been described, but it may be set to 4 times or more. The setting of the sensitization magnification L of 4 times or more needs to be used with attention to gradation drop.

  As described in the above example, pixels with the same filter color are added, so that a highly sensitive color image can be obtained without color mixing.

  When a captured image at low illuminance is amplified by an analog amplifier, noise becomes larger than a signal. Further, gradation reduction occurs when amplified by a digital amplifier. As described in the above example, the present invention achieves higher sensitivity by pixel addition of peripheral pixels, so noise is smaller than that of signals. For example, when two pixels are added, the signal component is doubled, the noise component is square root doubled, and a relatively pure signal component is increased. In addition, since the pixels located in the vicinity have a high correlation as a property of the image, a highly effective sensitivity improvement is realized by adding a plurality of pixels closest to the target pixel.

In the above example, the configuration using the CCD image sensor 2 as shown in FIG. 1 as one configuration example of the solid-state image sensor has been described. However, the two-dimensional image sensor is not limited to the CCD image sensor, Any thing is good. Further, not limited to the interline transfer CCD, a frame transfer CCD or a frame interline transfer CCD may be used.
FIG. 22 shows a configuration using the CMOS image sensor 20. The CMOS image sensor may be a device with a single imaging function or a device with integrated peripheral functions.
FIG. 22 shows a case of a CMOS image sensor in which peripheral functions are integrated.
The functions of the CCD image pickup device 2, the correlated double sampling processing circuit 3, the programmable gain amplification circuit 4, the A / D conversion circuit 5, and the timing generation circuit 10 in FIG. 1 are included in the CMOS image pickup device 20.

Embodiment 2. FIG.
FIG. 23 shows an imaging apparatus according to Embodiment 2 of the present invention. In FIG. 23, the detection circuit 13 is added, and the control circuit 12b is provided instead of the control circuit 12 of FIG. Play.

  The detection circuit 13 detects the magnitude of the signal supplied from the pixel addition circuit 6, obtains a signal amplitude level, for example, an average level detection value ASA, and outputs it as illuminance information.

In the above detection, the detection circuit 13 obtains the calculated value ASA of the average level of the signal amplitude by dividing the sum of the pixel values of all effective pixels by the total number of effective pixels.
Such calculation of the average level is performed, for example, every vertical scanning cycle, and is executed by, for example, integration processing and division processing. The “calculated value” of the average level of the signal amplitude obtained as described above may be referred to as “detected value” of the average level of the signal amplitude.

  Note that the division by the total number of effective pixels in the calculation of the average level of the signal amplitude described above may be realized by a bit shift process of a digital value when the number of pixels is given by 2 to the nth power (n is an integer). . Further, since the total number of effective pixels is a constant in the same system, division of the total number of effective pixels may be omitted.

The control circuit 12b is similar to the control circuit 12 of the first embodiment, but has an additional function as follows. That is, the control circuit 12b controls the aperture of the lens 1 based on the detection value ASA of the average level of the signal amplitude supplied from the detection circuit 13, and from the photoelectric conversion element of the CCD image pickup device 2 generated by the timing generation circuit 10. The charge read timing and charge forced discharge timing control (accordingly, charge accumulation time, ie, exposure time control), amplification gain control of the programmable gain amplifier circuit 4, and pixel addition processing of the pixel adder circuit 6 are controlled.
Further, the video signal processing circuit 7 calculates the level of noise included in the output of the pixel addition circuit 6 every vertical scanning period, and supplies it to the control circuit 12b.

  In the above example, the calculation of the average level of the signal amplitude and the calculation of the noise level are performed every vertical scanning period. However, the signal processing time in the detection circuit 13 and the video signal processing circuit 7 and Considering the transmission time from the detection circuit 13 and the video signal processing circuit 7 to the control circuit 12b, it may be performed only once in several vertical scans.

  In the above example, the detection circuit 13 calculates the average level of the signal amplitude, but there may be a case where peak detection of the signal amplitude is performed. The output of the detection circuit 13 is generated so as to increase the visibility of the subject of interest. For example, when it is not desired to saturate a highlight portion to white, peak detection is performed. When a halftone is clearly visible even if the highlight portion is saturated to white, average detection is performed.

  As will be described in detail below, the sensitivity control by the pixel addition circuit 6 can also be controlled as part of the exposure control, so that there is an effect that an image can be obtained in which the subject can always be viewed under optimum conditions even if the illuminance environment changes. The pixel addition circuit 6 adjusts the signal amplitude by changing the addition coefficient.

  The control circuit 12b performs automatic exposure control so that the detection value ASA of the average level of the signal amplitude obtained from the detection circuit 13 is constant. When the signal amplitude is large in imaging in a bright environment, the control circuit 12b controls the aperture of the lens 1 to reduce the amount of light incident on the CCD image sensor 2, or the forced charge discharge timing by the timing generation circuit 10. In this adjustment, the exposure time is reduced by controlling to forcibly discharge the charge accumulated in the photoelectric conversion element of the CCD image pickup device 2.

  When the signal amplitude becomes smaller due to imaging in a dark environment, the control circuit 12b amplifies the imaging signal by controlling to increase the amplification gain of the programmable gain amplifier circuit 4. However, when the amplification gain is too large, noise becomes conspicuous and an image with poor visibility is obtained. As another method, the control circuit 12b can extend the exposure time by controlling the charge reading from the photoelectric conversion element of the CCD image pickup device 2 to be thinned out in units of vertical scanning periods. However, if the exposure time is too long, the moving subject becomes an afterimage, resulting in an image with poor visibility. Furthermore, a circuit for performing interpolation (frame interpolation) of missing images in units of vertical scanning cycles is required.

Similar to the first embodiment, the control circuit 12b of the present embodiment sets the sensitization magnification L to the pixel addition circuit 6 from 1 to 4. The sensitization magnification L is set (adjusted) based on the illuminance information from the detection circuit 13 and the exposure parameters. As described in the first embodiment, the addition coefficients KR, KG, and KB are adjusted by changing the sensitization magnification L. Therefore, the addition coefficient is adjusted based on the illuminance information.
Further, by forming a plurality of combinations consisting of a pixel of interest and a pixel located in the vicinity region of the pixel of interest and located at a specific position with respect to the pixel of interest, a change width is calculated for each of the formed combinations, Pixels belonging to the combination having the smallest change width are added.
By adding the four most highly correlated pixels located in the vicinity region of the pixel of interest, the sensitivity can be improved by a factor of 4, and the visibility can be greatly improved even in an extremely dark environment. Further, since the decrease in frame frequency can be prevented or suppressed, there is no deterioration in dynamic resolution, and deterioration in horizontal resolution and vertical resolution can be minimized.

Hereinafter, an example of a procedure for sensitivity adjustment when the subject illuminance changes will be described. First, description will be made assuming that the exposure time is maintained at a constant value (standard exposure time) Tr.
When the illuminance of the subject gradually decreases and the detection value ASA of the average level of the signal amplitude decreases, the aperture of the lens 1 is controlled in the open direction (range Sa in FIG. 24A), and the average level of the signal amplitude To maintain. Here, “maintaining the average level of the signal amplitude” means maintaining the average level of the signal amplitude of the output of the pixel addition circuit 6, and hence the average level of the signal amplitude expressed by the output of the detection circuit 13. means.

  After the aperture of the lens 1 is opened (fully opened), control is performed to increase the amplification gain of the programmable gain amplifier circuit 4 (range Sb in FIG. 24B), and the average level of the signal amplitude is maintained. After the amplification gain of the programmable gain amplifier circuit 4 increases and becomes larger than the predetermined upper limit value UGL of the amplification gain, the sensitization magnification L of the pixel addition circuit 6 is controlled to increase (FIG. 24 (c) ) Range Sc), maintaining the average level of the signal amplitude.

  The average level ASA can be maintained by controlling the sensitization magnification L until the sensitization magnification L reaches the maximum value (L = 4). When the subject illuminance further decreases, the average level ASA starts to decrease.

  At the standard exposure time Tr, the lens diaphragm is opened, the amplification gain is maximized, and the illuminance HL at which the output of the pixel addition circuit 6 becomes a predetermined level when the sensitization magnification L is 1 is set as the high illuminance side reference value. When the illuminance is 1/4 of the high illuminance reference value HL, that is, at the standard exposure time Tr, the output of the pixel addition circuit 6 is predetermined when the lens aperture is opened, the amplification gain is maximized, and the sensitization magnification L is 4. The illuminance LL at the level of is set as the low illuminance side reference value.

  When the subject illuminance gradually becomes brighter and exceeds the low illuminance side reference value LL and the detection value ASA of the average level of the signal amplitude is about to increase, the pixel addition circuit 6 is controlled to reduce the sensitization magnification L ( The range Sc) in FIG. 24 (c) maintains the average level of the signal amplitude. When the sensitization magnification L is reduced to 1, the control is performed so as to reduce the amplification gain of the programmable gain amplifier circuit 4, and the average level of the signal amplitude is maintained. After the amplification gain of the programmable gain amplifier circuit 4 decreases (range Sb in FIG. 24B) and becomes smaller than the predetermined lower limit value LGL, the diaphragm of the lens 1 is controlled in the light shielding direction (FIG. 24 ( The average level of the signal amplitude is maintained in the range Sa) of a) (FIG. 24 (e)). As the illuminance further increases, the average level ASA increases.

As a result of the above control, the average level ASA of the signal amplitude can be kept constant in the range from the lower limit LL to the upper limit UL as shown by the solid line in FIG.
Although the exposure time is constant, the exposure time may be controlled according to the illuminance of the subject. For example, the exposure time may be increased when the signal amplitude does not reach a sufficient value even when the illuminance decreases and the sensitization magnification L reaches the maximum value (range in FIG. 24D). Sd). Conversely, if the illuminance increases and the diaphragm is maximized (the F value is maximized), or if the signal amplitude is too large, the exposure time may be shortened (see FIG. 24D). Range Se).
When such exposure time control is applied, the average level ASA of the signal amplitude can be kept constant in the range from the lower limit LLe to the upper limit ULe as shown by the dotted line in FIG.

  The predetermined upper limit value UGL of the amplification gain is determined based on a noise level detection value ANL detected by the video signal processing circuit 7. (Considering that it is necessary to increase the amplification gain when the illuminance of the subject decreases and the S / N of the output of the image sensor 2 decreases accordingly) The detected value ANL of the noise level is the average level of the signal amplitude. The amplification gain of the programmable gain amplifier circuit 4 when the predetermined noise ratio (first predetermined noise ratio, that is, the allowable upper limit value) NPR1 with respect to the detection value ASA is set as the predetermined upper limit value UGL. The first predetermined noise ratio NPR1 is set to 1/50, for example.

  The calculated noise level ANL is obtained by extracting noise components by noise reduction processing and dividing the sum of absolute values of noise components in the entire effective pixel range by the total number of effective pixels. By the noise reduction process, a noise reduction signal NRS in which the noise of the input signal is reduced is obtained. A noise component can be extracted by subtracting the noise reduction signal NRS from the input signal (a signal before the noise reduction process is performed in the video signal processing circuit 7). The “calculated value” of the noise level obtained as described above may be referred to as “detected value”.

  Since the allowable noise level for visual recognition of the subject varies depending on the application, the first predetermined noise ratio NPR1 varies depending on the application of the imaging apparatus, such as whether S / N is important or image resolution is important. The control circuit 12b dynamically observes the gain set in the programmable gain amplification circuit 4 and the detection value ANL of the noise level supplied from the detection circuit 13 to the control circuit 12b, and dynamically increases the predetermined upper limit value UGL of the amplification gain. The programmable gain amplifying circuit 4 and the pixel adding circuit 6 may be controlled, and the noise level detection value ANL is compared with the detection value ASA of the average level of the signal amplitude before the imaging apparatus is shipped from the factory. A storage unit (volatile memory backed up by a non-volatile memory or battery) that measures the amplification gain reaching a predetermined noise ratio NPR1 of 1 and retains the stored contents even when the imaging device is turned off as the predetermined upper limit value UGL 16), and the control circuit 12b refers to the predetermined upper limit value UGL of the above-mentioned amplification gain. Amplifying circuit 4 and may control the pixel addition circuit 6.

  The predetermined lower limit value LGL of the amplification gain is determined based on a noise level detection value ANL supplied from the video signal processing circuit 7 to the control circuit 12b. The amplification gain of the programmable gain amplifying circuit 4 when the detection value ANL of the noise level falls below a predetermined noise ratio (second predetermined noise ratio) NPR2 with respect to the detection value ASA of the average level of the signal amplitude is the predetermined gain. The lower limit is LGL. The second predetermined noise ratio is determined based on the first predetermined noise ratio NPR1 and the sensitivity magnification (4 times) by the pixel addition circuit 6. For example, the second predetermined noise ratio NPR2 is defined as 1/200 (= {(1/50) × (1/4)}).

  Since the allowable noise level for visual recognition of the subject differs depending on the application, the second predetermined noise ratio NPR2 varies depending on the application of the imaging device, such as whether S / N is important or image resolution is important. The control circuit 12b dynamically observes the gain set in the programmable gain amplifier circuit 4 and the detection value ANL of the noise level supplied from the detection circuit 13 to the control circuit 12b, and dynamically determines the predetermined lower limit value LGL of the amplification gain. The programmable gain amplifying circuit 4 and the pixel adding circuit 6 may be controlled, and the noise level detection value ANL is compared with the detection value ASA of the average level of the signal amplitude before the imaging apparatus is shipped from the factory. 2, the amplification gain reaching a predetermined noise ratio NPR2 of 2 is measured, and the predetermined lower limit value LGL is written to the storage unit 16 that can retain the stored contents even when the power of the imaging apparatus is turned off. The programmable gain amplifier circuit 4 and the pixel addition circuit 6 may be controlled with reference to the lower limit value LGL.

  The control circuit 12b controls the aperture of the lens 1, the exposure time of the CCD image pickup device 2, the amplification gain of the programmable gain amplification circuit 4, and the signal amplitude adjustment function by pixel addition in the pixel addition circuit 6 to control the average level of the signal amplitude (pixel The average level of the signal amplitude of the output of the adder circuit 6 is maintained.

Since the control as described above is performed, the illuminance of the subject is determined based on the amplitude ASA of the signal obtained by the detection circuit 13 and the exposure control parameters (aperture, amplification gain, sensitization sensitivity, exposure time) at that time. Can be requested.
In other words, the converted value obtained by performing the reverse calculation based on the exposure control parameter with respect to the output of the detection circuit is a value corresponding to the illuminance of the subject. In other words, it is determined whether the illuminance obtained by the reverse calculation is higher than the high illuminance side reference value, lower luminance side luminance value or less, or in the range between these reference values, and pixel addition is performed according to the determination result (Sensitization sensitivity) can be controlled.

  Since it is configured as described above, it is possible to output an image with optimal brightness with good visibility by sequentially switching aperture control, amplification gain control, pixel addition control, and exposure time control during exposure control. There is an effect that can be done.

  In the pixel addition circuit, the sensitization magnification can be set not by the number of pixels to be added but by the addition coefficient, and the sensitization magnification L can be set by a value including not only an integer but also a decimal. Among them, the pixel addition control is also seamlessly switched using the addition coefficient with a value after the decimal point, and there is an effect that an easy-to-view image can be output without a sudden change in the brightness of the image in the process of illuminance change.

Embodiment 3 FIG.
FIG. 25 shows an imaging apparatus according to Embodiment 3 of the present invention. In FIG. 25, except for the point that the photometry unit 14 is added and the point that the control circuit 12c is provided instead of the control circuit 12, the same effect as the first embodiment is obtained.

  The photometry unit 14 measures the illuminance of the subject in the direction in which the light enters the lens 1. Mounting and positioning of the illuminance sensor (not shown) of the photometry unit 14 is determined based on the optical axis of the lens, and the illuminance of the subject imaged by the lens 1 is measured.

  The control circuit 12c is similar to the control circuit 12 of the first embodiment, but has an additional function as follows. That is, the control circuit 12 c controls the aperture of the lens 1 based on the illuminance value supplied from the photometry unit 14, the charge readout timing from the photoelectric conversion element of the CCD image sensor 2 generated by the timing generation circuit 10, and the charge forcing. Control of the discharge timing (accordingly, control of charge accumulation time, that is, exposure time), control of amplification gain of the programmable gain amplifier circuit 4, and control of pixel addition processing of the pixel adder circuit 6 are performed.

The control circuit 12c determines the aperture of the lens 1, the exposure time of the CCD image pickup device 2, the amplification gain of the programmable gain amplification circuit 4, and the sensitization magnification of the pixel addition circuit 6 according to the set value table held in the storage unit 16. Set up.
In the setting value table, the aperture of the lens 1, the exposure time of the CCD image pickup device 2, the amplification gain of the programmable gain amplification circuit 4, and the sensitization magnification of the pixel addition circuit 6 are registered for each illuminance value.

  When the illuminance is bright, the exposure time of the image sensor 2 is set to the standard exposure time Tr based on the frame rate, the amplification gain of the programmable gain amplification circuit 4 is set to 1 time, and the sensitization magnification L of the pixel addition circuit 6 is set to 1 time. Then, the aperture of the lens 1 is reduced (range Sa in FIG. 24A). When the lens 1 reaches the maximum aperture and the illuminance further increases, the exposure time of the image sensor 2 is controlled to be shorter than the standard exposure time Tr (range Se in FIG. 24D).

  When the illuminance decreases, the exposure time of the image sensor 2 is set to the standard exposure time Tr based on the frame rate, the amplification gain of the programmable gain amplification circuit 4 is set to 1 time, and the sensitization magnification of the pixel addition circuit 6 is set to 1 time. The aperture of the lens 1 is opened (range Sa in FIG. 24A). When the lens 1 is opened and the illuminance is further darkened, the amplification gain of the programmable gain amplifier circuit 4 is increased from 1 (range Sb in FIG. 24B). The amplification gain of the programmable gain amplifying circuit 4 is the above upper limit value (the value of the amplification gain when the noise level included in the output of the pixel adding circuit 6 reaches the first predetermined ratio, that is, the noise When the level becomes a maximum gain value that satisfies the constraint that the first predetermined ratio (the upper limit value of the allowable range) is not exceeded, and the illuminance further decreases, the sensitization magnification of the pixel addition circuit 6 is reduced from 1 ×. It is increased (range Sc in FIG. 24C). When it becomes darker, the exposure time is lengthened (range Sd in FIG. 24D).

  Since it is configured as described above, an aperture image, amplification gain control, pixel addition control, and exposure time control can be sequentially switched during exposure control to output an image with optimal brightness with good visibility. There is an effect that can be done.

  In the pixel addition circuit, the sensitization magnification can be set not by the number of pixels to be added but by the addition coefficient, and the sensitization magnification L can be set by a value including not only an integer but also a decimal. Among them, the pixel addition control is also seamlessly switched using the addition coefficient with a value after the decimal point, and there is an effect that an easy-to-view image can be output without a sudden change in the brightness of the image in the process of illuminance change.

  In the second and third embodiments, it has been described that the exposure time is lengthened when the signal amplitude is not sufficient even when the sensitization magnification is maximized. This is a result of giving priority to keeping the frame rate unchanged. It is. When the resolution is more important than the frame rate, the control to increase the exposure time is performed first, and the sensitization magnification is increased when the signal amplitude is not sufficient even if the exposure time is increased (for example, up to a predetermined value). Alternatively, the control for increasing the sensitization magnification and the control for extending the exposure time may be performed in parallel.

  DESCRIPTION OF SYMBOLS 1 Lens, 2 CCD image pick-up element, 3 Correlated double sampling processing circuit, 4 Programmable gain amplifier circuit, 5 A / D conversion circuit, 6 Pixel addition circuit, 7 Video signal processing circuit, 8 Video signal output terminal, 9 Drive circuit, DESCRIPTION OF SYMBOLS 10 Timing generation circuit, 11 Synchronization signal generation circuit, 12 Control circuit, 13 Detection circuit, 14 Photometry part, 15 Input terminal, 16 Output terminal, 17 Control terminal, 18 Synchronization signal input terminal, 19 Delay circuit, 20 CMOS image sensor, 30 selection addition circuit, 40 noise reduction circuit, 40A pixel extraction unit, 40B weighted addition unit, 40C coefficient setting circuit, 50 area selection circuit, 50A pixel extraction unit, 50B correlation determination unit, 301 input terminal, 302 output terminal, 311 312 2-line delay circuit, 317 system Terminal, 318 synchronization signal input terminal, 319 pixel position input terminal, 322 to 325 one line delay circuit, 330 to 331 four pixel delay circuit, 332 to 337 two pixel delay circuit, 342 to 365 one pixel delay circuit, 370 pixel selection circuit , 390 addition circuit, 401 input terminal, 402 output terminal, 417 control terminal, 418 synchronization signal input terminal, 422 to 425 1 line delay circuit, 432 to 455 1 pixel delay circuit, 461 to 485 multiplication circuit, 490 addition circuit, 501 Input terminal, 502 output terminal, 511-512 2-line delay circuit, 518 synchronization signal input terminal, 522-525 1-line delay circuit, 530-531 4-pixel delay circuit, 532-537 2-pixel delay circuit, 542-565 1 pixel Delay circuit, 5 0 pixel selection circuit, 571 to 582 change width calculation circuit, the minimum value calculation circuit 585, 590 pixel specifying circuit, 595 correlation area detection unit.

Claims (11)

An imaging unit having a plurality of types of pixels that detect light of a plurality of different color components and output corresponding imaging signals;
A noise reduction circuit for reducing noise of an imaging signal output from the imaging unit;
Based on the imaging signal in which noise has been reduced by the noise reduction circuit, for each target pixel, a region having a high correlation among the regions formed by pixels that detect light of the same color component around each target pixel. An area selection circuit to select;
The imaging signals of the target pixel outputted from the imaging unit, and a selection adding circuit for outputting the addition result as an addition pixel signal by adding the image pickup signal of a pixel included in the realm selected by the area selecting circuit Prepared,
The plurality of color components includes first, second and third color components;
The plurality of types of pixels are
A first type of pixel that detects light of the first color component and generates a corresponding imaging signal; a second type of pixel that detects light of the second color component and generates a corresponding imaging signal; A third type of pixel that detects light of the third color component and generates a corresponding imaging signal; and a fourth type of pixel that detects light of the second color component and generates a corresponding imaging signal; Including
The first to fourth types of pixels are regularly arranged in the screen with a combination of two horizontal pixels and two vertical pixels as a basic unit,
The second type and fourth type pixels are arranged side by side in the direction of one diagonal line in the basic unit, and the first type and third type pixels are arranged side by side in the direction of the other diagonal line. ,
The region selection circuit includes:
A pixel extraction unit that simultaneously extracts signals representing pixel values of a pixel of interest and its surrounding pixels by delaying an imaging signal output from the noise reduction circuit by a predetermined time;
Of the pixels extracted by the pixel extraction unit, forming a plurality of combinations of a target pixel and a plurality of pixels that detect light of the same color component as the target pixel, and among the plurality of combinations, imaging apparatus characterized by comprising a maximum and minimum value of the difference between the correlation determination unit determines that constitutes the high have area of said correlation combining the minimum pixel value of pixels constituting the combination . The correlation determination unit
Of the pixels extracted by the pixel extraction unit, a plurality of combinations of a pixel of interest and a plurality of pixels that detect light of the same color component as the pixel of interest are formed, and the plurality of combinations are output. A selection circuit;
For each of a plurality of combinations output from the selection circuit, a change width calculation circuit that calculates a difference between the maximum value and the minimum value as a change width;
Characterized in that it comprises a said minimum of the combination that caused the ones of the variation calculated by the variation calculation circuit, wherein the determining the correlation region detecting unit constitutes a high There area correlation The imaging apparatus according to claim 1. A combination formed by the correlation determination unit using a pixel that outputs an imaging signal of the second color component as a target pixel,
A combination of a pixel of interest, a pixel located two lines before the pixel of interest, a pixel located one pixel before the pixel of interest, and a pixel located one pixel before the line of interest;
A combination of a pixel of interest, a pixel located one pixel before the pixel of interest one pixel after, a pixel located two pixels after the pixel of interest, and a pixel located one pixel after the pixel of interest;
A combination of a pixel of interest, a pixel located one pixel before the pixel of interest, a pixel located two pixels before the pixel of interest, and a pixel located one pixel before the pixel of interest;
The pixel of interest, the pixel located one pixel before the pixel of interest one pixel before, the pixel located one pixel after the pixel of interest one pixel after, and the combination of the pixel located two lines after the pixel of interest The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. A combination formed by the correlation determination unit using a pixel that outputs an imaging signal of the second color component as a target pixel,
A combination of a target pixel, a pixel located two lines before the target pixel, a pixel located four lines before the target pixel, and a pixel located two lines after the target pixel;
A combination of a pixel of interest, a pixel located two lines before the pixel of interest, a pixel located two lines after the pixel of interest, and a pixel located four lines after the pixel of interest;
A combination of a pixel of interest, a pixel located 2 pixels before the pixel of interest, a pixel located 4 pixels before the pixel of interest, and a pixel located 2 pixels after the pixel of interest;
The pixel of interest, a pixel located two pixels before the pixel of interest, a pixel located two pixels after the pixel of interest, and a combination of pixels located four pixels after the pixel of interest are included. The imaging device according to any one of the above. A combination formed by the correlation determination unit using a pixel that outputs an imaging signal of the second color component as a target pixel,
A combination of a pixel of interest, a pixel located two pixels before the pixel of interest two pixels before, a pixel located one pixel before the pixel of interest one pixel before, and a pixel located one pixel after the pixel of interest one line after When,
A combination of a target pixel, a pixel positioned one pixel before the target pixel one pixel before, a pixel positioned one pixel after the target pixel one line after, and a pixel positioned two pixels after the target pixel two lines ,
A combination of a pixel of interest, a pixel located two pixels before the pixel of interest two pixels before, a pixel located one pixel before the pixel of interest one pixel before, and a pixel located one pixel before the pixel of interest one pixel before ,
A combination of a pixel of interest, a pixel located one pixel before the pixel of interest one pixel before, a pixel located one pixel after the pixel of interest one pixel before, and a pixel located two pixels before the pixel of interest two pixels before The imaging device according to any one of claims 1 to 4, wherein the imaging device includes: A combination formed by the correlation determination unit using a pixel that outputs an imaging signal of the first or third color component as a target pixel,
A combination of a pixel of interest, a pixel located two lines before the pixel of interest, a pixel located two pixels before the line of interest, and a pixel located two pixels before the pixel of interest;
A combination of a target pixel, a pixel located two lines before the target pixel, a pixel located two pixels before the second line of the target pixel, and a pixel located two pixels after the target pixel;
A combination of a target pixel, a pixel located two pixels before the target pixel, a pixel located two lines after the target pixel, and a pixel located two pixels before the second line of the target pixel;
And a combination of a pixel located two pixels after the target pixel, a pixel located two lines after the target pixel, and a pixel located two pixels after the second line of the target pixel. The imaging device according to any one of 1 to 5. The addition in the selective addition circuit is performed by weighted addition using an addition coefficient determined based on the sensitization magnification,
An illuminance information generator for generating illuminance information representing the illuminance of the subject;
The imaging apparatus according to claim 1, further comprising a control circuit that determines the sensitization magnification based on the illuminance information.   The imaging apparatus according to claim 7, wherein the sensitization magnification is set to a value larger than 1 when the illuminance indicated by the illuminance information becomes a predetermined value or less. A control circuit that controls the lens aperture and exposure time of the imaging unit; and a signal amplification unit that amplifies the imaging signal output from the imaging unit;
The predetermined illuminance is set such that the lens aperture is opened, the exposure time is the standard exposure time, the amplification gain of the signal amplifying unit, and the noise level included in the selective addition circuit does not exceed a predetermined value. 9. The imaging apparatus according to claim 8, wherein when the maximum gain value to be satisfied is set and the sensitization magnification of the selective adding circuit is 1, the output of the selective adding circuit is an illuminance at a predetermined level. . The imaging according to any one of claims 7 to 9, wherein the illuminance information generation unit includes a photometry unit that measures the illuminance based on light from a subject and generates the illuminance information. apparatus.   10. The illuminance information generation unit generates the illuminance information based on a level of the addition pixel signal generated by the selective addition circuit and an exposure control parameter. 10. The imaging device described.

Images