JP2007174032A - Imaging device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of reducing smears without generating vertical black streaks due to over-correction, even when luminance signal values are saturated from pixels in an effective pixel region. <P>SOLUTION: The imaging device has a solid-state imaging element and a signal processor provided with a black reference detector 14 for detecting the luminance signal value of a pixel as a black reference and that of a pixel in which smears occur; a luminance signal detector 15 for detecting the luminance signal value of each pixel in the effective pixel region; an orthogonal region average value detector 18 for detecting the luminance signal value of the average value of the prescribed number of pixels, after horizontally and continuously selecting the prescribed number of the pixels from the pixels whose luminance signal values have been detected by the luminance signal detection part; a correction value generator 16 for generating a correction value corresponding to the luminance signal value of the pixel, in which smears occur, from each detected luminance signal value detected with the black reference detector 14, the luminance signal detector 15, and the orthogonal region average value detector 18; and a corrector 17 for correcting the luminance signal value of the pixel, in which smears occur, from the correction value generated with the correction value generator 16. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image pickup apparatus using a solid-state image pickup device, and in particular, has a function of removing a noise component that is highly correlated in one vertical and horizontal direction in an image pickup device such as a vertical direction or a horizontal direction called smear. The present invention relates to an imaging apparatus.

  For example, a CCD (Charge Coupled Device) imaging device includes a photoelectric conversion unit in which a plurality of photoelectric conversion elements that output charges corresponding to the intensity of received light are arranged in a matrix of a plurality of rows and columns as pixels, A vertical transfer unit (hereinafter also referred to as a vertical transfer path or VCCD) that transfers charges output from the photoelectric conversion element in the vertical direction, and a horizontal transfer unit (hereinafter referred to as vertical transfer path or VCCD) that transfers charges output from the vertical transfer unit in the horizontal direction. , Horizontal transfer path, and HCCD). In general, at least one of the upper and lower outer peripheral edge regions of the effective pixel region used for imaging in the photoelectric conversion unit is external light input to several photoelectric conversion elements in several rows of the peripheral region. A light shielding region for outputting a luminance signal value serving as a black reference is formed.

  In a CCD imaging device, depending on imaging conditions, a noise component having a high correlation may be generated in the vertical (VCCD) direction or the horizontal (HCCD) direction. As an example of noise having a high correlation in one direction such as the VCCD direction or the HCCD direction, for example, a noise component generated in a vertical (vertical) direction called smear is known.

  Hereinafter, the principle of smear generation will be briefly described. When a CCD image pickup device picks up strong light (= very bright light) such as sunlight or strong illumination light, for example, a photoelectric conversion element (hereinafter referred to as a photodiode or PD) in the strong light region on the image pickup device. (Noted) generates not only the charges necessary for the imaging signal but also an extra large amount of charges. A lot of the generated extra charge becomes a high-level noise signal, and the photoelectric signal in the area (= dark area) receiving the weak light formed in the VCCD direction in the photoelectric conversion unit of the CCD image sensor. Leak into the conversion element. Then, in the photoelectric conversion element in the dark region, the leaked high level noise signal is superimposed on the original low level imaging signal. In this way, smear occurs in the photoelectric conversion element in the dark area in the VCCD direction.

  The smear generated in the CCD image pickup device has a problem that the visual quality of the picked-up image is remarkably lowered. In addition, for example, when smear occurs in an imaging device used for biometrics that recognizes body-specific data such as a biometric fingerprint pattern or vein pattern for personal authentication, the feature points required for recognition are high. There is a case where it is buried in the noise component of the level and the recognition rate of the feature point is lowered.

  Further, as noise having a high correlation in one direction other than smear, for example, noise caused by variations in HCCD sensitivity in an image sensor having a plurality of HCCDs is known. A CCD image sensor including a plurality of HCCDs is, for example, a high pixel and is used to realize frame transfer at high speed. However, when a plurality of HCCDs are provided, the amplification characteristics of the charge output unit vary, and thus a difference value (amount of change) occurs in the luminance signal value between the HCCDs, which may reduce the visual quality. In particular, there is a problem in that the visual quality is remarkably deteriorated particularly in the dark part of the imaging screen.

  In the conventional imaging apparatus, various methods have been proposed in order to remove noise having high correlation in one direction in the CCD imaging device as described above. For example, the difference between the luminance signal value of the black reference pixel output from the light shielding region and the luminance signal value of the pixel in which noise is generated is calculated, and the pixel in which noise is output from the effective pixel region and the pixel There is known a method of providing a signal processing unit that detects a luminance signal value and uniformly subtracts a difference value obtained from a light shielding region from a luminance signal value of a noise generation pixel in the effective pixel region.

  More specifically, a dummy area of a plurality of lines in which only noise components are generated is provided on the upper and lower peripheral sides of the black reference pixel area in the solid-state imaging device, and a signal from each pixel in the dummy area is supplied to the signal processing unit. A line memory for storing as a dummy signal; an adder for generating a dummy signal (= correction signal) for one line by averaging the dummy signals of a plurality of lines; and each line in the effective pixel area of the solid-state imaging device By providing a subtractor that subtracts the dummy signal (= correction signal) value of one line obtained by averaging from the pixel output signal value, the noise (smear) value is obtained from the output signal value of each pixel of one line in the effective pixel region. Is known (see, for example, Patent Document 1).

  Also known is a method of correcting the luminance signal value itself of a pixel serving as a black reference. Specifically, the horizontal black reference pixel region is provided on the upper and lower outer peripheral sides of the effective pixel region in the solid-state imaging device, and the vertical black reference pixel region is also provided on the left and right outer peripheral sides. An average value is calculated from the integrated output values of each pixel in the area, and when the average value exceeds a predetermined value at which it can be determined that noise (smear) has occurred, the luminance signal value of the black reference pixel is Switch from the output of the reference pixel area to the output of the vertical black reference pixel area. In this way, it is possible to avoid a situation in which the black reference luminance signal value abnormally increases when smear occurs (see, for example, Patent Document 2).

JP 2000-050165 A (pages 9-12, FIG. 1) JP 2000-138869 A (page 2-8, FIG. 2)

  However, in the case of Patent Document 1, there is an influence in which random noise components in the vertical direction are superimposed on the output of pixels in which smear does not occur in the dummy signal of one line obtained by averaging. Therefore, when the dummy signal value of one line obtained by averaging is subtracted from the output signal value of each pixel of one line in the effective pixel region in the solid-state imaging device, the influence of smear can be reduced with respect to the position of the pixel where smear occurs. However, there is a problem that noise in the vertical direction occurs due to the influence of random noise components in the dummy area at the position of the pixel where smear does not occur.

  In addition, the effective pixel area has hundreds or thousands of lines of pixels as compared with the above-described several black reference pixel areas (or dummy areas). The level of the luminance signal value varies greatly depending on the direction. For example, if the effective pixel area is above the sun or a high-illumination lighting device, and there is a dark-colored plant or the like below, the level of the luminance signal value near the upper sun is very high. The saturation point of the output of the photoelectric conversion element is reached or close to the saturation point, and the level of the luminance signal value in the vicinity of the lower plant becomes relatively low. On the other hand, one type of dummy signal (= correction signal) is generated from the black reference pixel region or the dummy region. Therefore, only when the level of the luminance signal value changes linearly in all the pixels in the vertical direction of the pixel in which the smear in the effective pixel area has occurred, the correction is performed by using one type of dummy signal from the above-described dummy area. Will be possible.

  However, in practice, for example, the output of the luminance signal value of the pixel that captures the sun or a high-illuminance lighting device is often saturated (saturated) before linearly increasing. From this, the increase amount of the luminance signal value of the pixel that images the illuminating device of the sun or high illuminance decreases with respect to the increase amount of the luminance signal value of the pixel that images the dark plant or the like. Therefore, when the luminance signal value of each pixel in the column where smear has occurred in the effective pixel region is corrected by the one kind of dummy signal described above, the saturated luminance signal value of the pixel such as the sun or a high-illuminance lighting device is excessive. As a result, there is a problem that the pixels that are arranged in the vertical direction, which is originally bright (bright), are darkened like black streaks.

  The present invention has been made to solve the above-described problems, and suppresses the occurrence of noise in the vertical direction due to the influence of random noise other than noise that is highly correlated in one direction. An object of the present invention is to provide an imaging device that reduces noise having high correlation in one direction without generating vertical black stripes due to overcorrection even when the luminance signal value is saturated.

In order to achieve the above object, the imaging apparatus of the present invention provides:
A photoelectric conversion unit in which a plurality of photoelectric conversion elements that output charges corresponding to the intensity of received light are arranged in a matrix of a plurality of rows and a plurality of columns as pixels,
A vertical transfer unit that transfers charges output from each photoelectric conversion element in the vertical direction;
And a horizontal transfer unit for transferring the charges output from the vertical transfer unit in the horizontal direction,
At least one of the upper, lower, left, and right outer peripheral edge regions of the effective pixel region used for imaging in the photoelectric conversion unit is configured to shield external light input to each of the photoelectric conversion elements in a plurality of rows of the peripheral region. A solid-state imaging device in which a light-shielding region for outputting a luminance signal value serving as a black reference is formed;
Noise that is highly correlated in one direction with the luminance signal value of the pixel serving as the black reference detected from the output of the light-shielding region, from the luminance signal value of the pixel in which highly correlated noise is generated in one direction in the effective pixel region An image pickup apparatus including a signal processing unit having at least a function of subtracting a difference from a luminance signal value of a pixel in which
The signal processing unit
A black reference detection unit that detects a luminance signal value of at least a pixel serving as a black reference and a luminance signal value of a pixel in which highly correlated noise is generated in one direction from the luminance signal of each pixel in the light shielding region;
A luminance signal detector that detects a luminance signal value of each pixel in the effective pixel region;
A predetermined number of pixels are continuously selected in a horizontal direction orthogonal to a vertical direction in which pixels having a highly correlated noise in one direction are arranged with respect to the pixels in which the luminance signal value is detected by the luminance signal detection unit. An orthogonal region average value detection unit for detecting a luminance signal value of an average value of each pixel;
The luminance signal value of the pixel in which the high-correlation noise is generated in one direction of the effective pixel region is input with the luminance signal value detected by the black reference detection unit, the luminance signal detection unit, and the orthogonal region average value detection unit. A correction value generation unit that generates a correction value corresponding to the value;
A correction unit that receives a correction value generated by the correction value generation unit and corrects a luminance signal value of a pixel in which highly correlated noise is generated in one direction of the effective pixel region.

  According to the present invention, the occurrence of noise in the vertical direction due to the influence of random noise other than highly correlated noise in one direction is suppressed, and even when the luminance signal value from the pixel in the effective pixel region is saturated, overcorrection is achieved. It is possible to reduce highly correlated noise in one direction without generating vertical black sun.

Embodiment 1 FIG.
In the first embodiment, a description will be given of a case of a smear in which noise highly correlated in one direction generated in an effective pixel region of a solid-state imaging device is generated in each pixel continuous in the vertical (vertical) direction.

FIG. 1 is a block diagram showing a basic configuration of the imaging apparatus according to Embodiment 1 of the present invention.
The lens 1 has a focus adjustment function that optically condenses (focuses) the light emitted from the subject so that the solid-state imaging device 2 can take an image regardless of the position of the subject. The lens 1 of the present embodiment will be described below as a configuration that automatically performs a focus adjustment function, for example, but a configuration that manually switches a plurality of fixed focal points according to the application of the imaging device, or a pan A configuration that does not require adjustment by focusing may be used. Furthermore, a zoom function for continuously changing the angle of view from the wide-angle side to the telephoto side by combining a plurality of lenses and adjusting the distance between the lenses may be provided.

  The solid-state imaging device 2 has a plurality of photoelectric conversion elements such as photodiodes arranged two-dimensionally on its surface as pixels, and receives light focused by the lens 1 for each pixel (incident light from a subject). Is done. The received light is photoelectrically converted into electric charges (hereinafter also referred to as imaging charges and electric signals) corresponding to the intensity of the light. The photoelectrically converted electrical signal output is read out by a predetermined driving method using a solid-state imaging device driving pulse PA input from the timing control unit 6 and output as an imaging signal output SA.

  The analog signal processing unit 3 performs correlated double sampling (CDS) on the input imaging signal output SA using the sample hold pulse PB from the timing control unit 6 and the analog signal processing control signal CC from the control unit 7. In addition to performing the process, an automatic signal amplification (AGC) process for automatically controlling the amplification gain is performed and output as an analog amplified signal output SB.

  The A / D conversion unit 4 converts the input analog amplified signal output SB into a digital signal using the A / D clock PC from the timing control unit 6 and outputs the digital signal as a digital signal output SC. As the resolution of the A / D converter 4, it is sufficient that the resolution is 8 bits for a general display device. However, for example, when there is a purpose to improve color resolution by signal processing in the subsequent stage, there is a purpose to improve gradation characteristics, there is a purpose to detect even a slight contrast, there is a purpose to expand the dynamic range, etc. The number of bits is increased according to the various purposes, and it is common, and the resolution may be 10 bits or 14 bits. In the present embodiment, the maximum value of the digital signal output SC is described as A / D conversion output max. The A / D conversion output max is 255 when the A / D conversion unit 4 has an 8-bit resolution, and 1023 when the A / D conversion unit 4 has a 10-bit resolution. In the present embodiment, a case where the A / D conversion unit 4 has an 8-bit resolution will be described.

  The digital signal processing unit 5 is a processing unit (described later with reference to FIG. 3) that corrects noise having a high correlation in one of the vertical and horizontal directions of the display screen, such as smear. Means, processing means for interpolating values between pixels, and filter processing means. Further, processing means for converting, for example, an RGB signal into a color difference value (change amount) signal (YCbCr) so that it can be displayed on a subsequent display device (display device) or easily processed by a display circuit, processing for correcting white balance A processing means for processing a video signal, such as a processing means for digital amplification, a processing means for γ (gamma) correction, a color interpolation processing means, and an edge enhancement correction unit. The input digital signal output SC is subjected to the above-described various processing including noise correction processing, and is output as a digital video signal output SE.

  The digital video signal output SE output from the digital signal processing unit 5 is input to a display device such as an LCD (not shown) to display the video. A statistical processing signal SF is output from the digital signal processing unit 5 to the control unit 7. The statistical processing signal SF is, for example, the maximum value of the digital signal output SC in one frame as a result of dividing one captured image into a plurality of blocks in the horizontal and vertical directions and integrating the digital signal output SC for each block. This is a signal resulting from performing statistical processing such as a histogram on the minimum value and the digital video signal output SE.

  The timing control unit 6 includes a solid-state image sensor driving pulse PA including a vertical transfer pulse, a horizontal transfer pulse, an electronic shutter pulse, a reset pulse, and the like for controlling the driving timing of the solid-state image sensor 2, and an analog signal processing unit. 3 generates a sample hold pulse PB for controlling the timing for sampling and holding the image signal output SA signal and the black reference signal, and an A / D clock PC for controlling the timing of the A / D converter 4. Supply to each processing means.

  The control unit 7 receives, for example, a statistical processing signal SF input from the digital signal processing unit 5 by serial communication using a general-purpose bus as a communication medium, and controls the timing control unit 6 according to the statistical processing signal SF. For generating the timing control signal CA, the analog signal processing control signal CC for controlling the analog signal processing unit 3, and the digital signal processing control signal CB for controlling the digital signal processing unit 5 and outputting them to each means To do.

  The timing control signal CA is a signal for controlling the phase of each of the pulses generated by the timing control unit 6 and the output timing. For example, in the case of the solid-state image sensor driving pulse PA, the all-pixel imaging mode or the monitor imaging is performed. It is possible to control each pulse when switching the driving mode of the solid-state imaging device 2 such as the mode, and the timing of the charge discharge pulse related to the shutter speed for controlling the exposure.

  The digital signal processing control signal CB is, for example, a process of correcting white balance performed by the digital signal processing unit 5 on the digital signal output SC output from the A / D conversion unit 4 or a process of digital amplification. A signal whose gain can be controlled. For example, the analog signal processing control signal CC is obtained by setting the amplification amount (amplified signal output SB) of the automatic signal amplification processing performed by the analog signal processing unit 3 to the imaging signal output SA output from the solid-state imaging device 2 as the shutter speed. Since it can be controlled in synchronization, it is a signal that can control exposure.

FIG. 2 is a diagram showing a schematic configuration of the solid-state imaging device 2 of FIG.
The photodiodes 8 (PD) are elements that are two-dimensionally arranged on the surface of the solid-state imaging device 2 and receive incident light from a subject focused by the lens 1. Further, it is photoelectrically converted into an electric signal (imaging charge) indicating a luminance signal value at a level corresponding to the intensity of the received light, and is output. The imaging charges converted by the photodiode 8 are read out from a transfer gate unit (not shown) by a transfer gate pulse and output to a vertical charge transfer unit 9 described later.

  The vertical charge transfer unit 9 converts the image pickup charge input from the photodiode 8 into the vertical (V) direction of the image pickup element (downward arrow direction shown in FIG. 2) according to the drive waveforms of the vertical charge transfer pulses φV1 to φV4. Forward to. The horizontal charge transfer unit 10 converts the image pickup charge input from the vertical charge transfer unit 9 into the horizontal (H) direction of the image pickup device (the left-pointing arrow shown in FIG. 2) according to the drive waveforms of the horizontal charge transfer pulses φH1 to φH2. Direction). The imaging signal output unit 11 outputs the imaging charge input from the horizontal charge transfer unit 10 as the imaging signal output SA. The effective pixel region 12 is a region in the solid-state imaging device 2 where pixels that receive incident light from a subject, perform photoelectric conversion, and output imaging charges are disposed.

  The black reference pixel region 13 (the lower end region 13a, the upper end region 13b, the left end region 13c, and the right end region 13d) is also described as a light shielding region, and is effective for obtaining a reference for an optical black reference signal in the solid-state imaging device 2. This is an area in which pixels in which the light from the lens 1 is shielded at the upper, lower, left and right outer peripheral ends of the pixel area 12 are arranged. The black reference pixel is generally referred to as optical black. As a means for shielding the black reference pixel, it is sufficient if the area of the black reference pixel can be shielded. For example, a mechanical shutter formed so as to obtain a light shielding effect with a material such as aluminum before the pixel of the solid-state imaging device 2 or the lens 1 or What is necessary is just to arrange | position a light-shielding object.

  The solid-state imaging device 2 has a horizontal transfer pulse (φH1, φH2) for driving a two-layer horizontal CCD from a timing control unit 6 and a vertical transfer pulse (φV1, φV2, φV3) for driving a four-layer vertical CCD. , ΦV4), a reset gate pulse (φRG) for resetting the pixel charge, and an electronic shutter control pulse (φSUB) for realizing an electronic shutter with a function of discharging the accumulated pixel charge from the substrate potential of the CCD. By driving according to the procedure, the charge accumulated in the accumulation means can be read out as the imaging signal output SA.

  As the solid-state image pickup device 2, any other transfer type described above may be used as long as it has a black reference pixel capable of detecting a black reference signal and transfers charges. For example, a frame transfer type, a frame An interline transfer type or interline transfer type image sensor may be used.

  Further, as the solid-state imaging device 2, for example, a monochrome solid-state imaging device in which no color filter is arranged, a color solid-state imaging device in which primary or complementary color filters are arranged, and a line-type solid-state imaging device may be used. Furthermore, the driving method of the solid-state imaging device 2 is not limited to the configuration using the horizontal transfer pulse for driving the above-mentioned two-layer horizontal CCD and the vertical transfer pulse for driving the four-layer vertical CCD. A solid-state imaging device 2 that uses a vertical transfer pulse for driving a three-layer vertical CCD may be used.

FIG. 3 is a block diagram showing a schematic configuration of a noise correction unit provided in the digital signal processing unit 5 of FIG.
The noise correction unit shown in FIG. 3 has the following configuration for correcting (removing) noise having a high correlation in one direction such as smear generated in each pixel in the effective pixel region 12 of the solid-state imaging device 2.

  The black reference detection unit 14 generates a black reference signal from the luminance signal output of the black reference pixel region 13 provided in the solid-state imaging device 2 in order to generate a black reference signal from the digital signal output SC output from the AD conversion unit 4. While detecting the reference signal level, the difference value (change amount) of the luminance signal value between adjacent pixels in the black reference pixel region 13 and the position information of the pixel are detected. In particular, the horizontal position of the pixel in which the change amount of the luminance signal value has changed beyond a predetermined value is detected. Here, the predetermined value has a variance σ due to, for example, the black reference pixel signal changing randomly due to thermal noise, and is set to a value exceeding this σ. By setting the predetermined value in this way, it is possible to detect a noise signal having a high correlation in one direction such as smear. With this function, it is possible to detect a difference value (amount of change) between a black reference signal level of a normal pixel in the black reference pixel region 13 and a signal level of a pixel adjacent to the pixel where smear has occurred.

  When the difference value (change amount) of the luminance signal value between adjacent pixels in the black reference pixel region 13 is equal to or less than a predetermined value, the black reference detection unit 14 averages the signals of the adjacent pixels so far. When the difference value (change amount) of the luminance signal value between adjacent pixels exceeds a predetermined value, the average value of the signal of each adjacent pixel after the change is the amount of change in the luminance signal value of the adjacent pixel. Is output as the detection result. For example, when the highly correlated noise in one direction is smear, the black reference signal level is detected from the outputs of the lower end region 13a and the upper end region 13b in the black reference pixel region 13, and is adjacent in the horizontal (H) direction. The amount of change in luminance signal value between the pixels and the position information of the pixels are detected.

  The difference value between the black reference signal level of the normal pixel and the signal level of the pixel in which smear has occurred is the luminance signal value of the pixel in which smear has occurred in the effective pixel region as the first correction value. It is used to correct what is a linear change region that is not saturated. Further, the black reference detection unit 14 may be configured by using, for example, a non-linear filter as processing means for detecting a difference value (change amount) of luminance signal values between adjacent pixels. When selecting the adjacent pixel, if the difference value from the luminance signal value of the previous adjacent pixel is less than a predetermined value at which it is determined that smear has occurred, the difference value is detected to be less than the predetermined value. If the difference value with the luminance signal value of the previous adjacent pixel exceeds a predetermined value at which it is determined that smear has occurred, it is detected that the difference value exceeds the predetermined value. Select from the adjacent pixels after the selected pixel. In this way, by selecting only the adjacent pixels having a difference value less than the predetermined value, a black reference pixel can be obtained when smear occurs in the vertical (vertical) direction of the effective region and extends to the black reference pixel region. The black clamp (black reference) signal SCclp and the smear amount value ΔSm1 (n) can be obtained from the average value of the luminance signal values of the respective pixels in the horizontal (lateral) direction of the region.

  For each pixel in the effective pixel area 12 of the solid-state imaging device 2, the luminance signal detection unit 15 is a digital value that indicates the level of the luminance signal before correction proportional to the imaging signal output SA output from each pixel as a digital value. The signal output SC is detected.

  The correction value generation unit 16 is correlated in one direction based on the detection result output of the luminance signal detection unit 15, the detection result output of the black reference detection unit 14, and the detection result output of the orthogonal region average value detection unit 18. Generate high noise correction values. At this time, the correction value generation unit 16 uses the first correction value used by the output of the black reference detection unit 14 and the luminance signal detection unit according to the output of the luminance signal detection unit 15 and the output of the black reference detection unit 14. The second correction value using a value obtained by subtracting the output of the orthogonal region average value detection unit 18 from the 15 saturated signal values is selectively switched. The second correction value will be described later in detail with reference to FIGS.

  When the correction value generation unit 16 switches between the first correction value and the second correction value, first, the output of the change amount of the luminance signal value between adjacent pixels of the black reference detection unit 14 indicates that smear has occurred. If the value exceeds the value and the output of the luminance signal detection unit 15 does not reach 255 in the case of 8-bit resolution, for example, it is not saturated, the first correction value is output. On the other hand, when the change output of the luminance signal value between adjacent pixels of the black reference detection unit 14 exceeds a predetermined value and the output of the luminance signal detection unit 15 is saturated, the second correction value is output.

  Further, the correction value generation unit 16 of the present embodiment has processing means for calculating a third correction value by performing a linear operation based on the first correction value and the second correction value, The first correction value, the second correction value, and the first correction value according to the luminance signal level before correction of the luminance signal detection unit 15 and the output of the change amount of the luminance signal value between adjacent pixels of the black reference detection unit 14. The three correction values are selectively output. The third correction value will be described later with reference to FIG.

  The correction unit 17 corrects the digital signal output SD corrected by subtracting the correction value generated by the correction value generation unit 16 from the digital signal output SC proportional to the imaging signal output SA output from the solid-state imaging device 2. Is output.

  The orthogonal area average value detection unit 18 has, in the effective pixel area of the solid-state imaging device 2, arranged in one line area in a direction orthogonal to each pixel arranged in one direction in which highly correlated noise occurs. Then, for example, several adjacent pixels such as 2 or 4 are selected, the luminance signals for the selected several pixels are averaged, and the result is detected as an average value and output to the correction value generation unit 16.

  Here, before describing the second correction value, first, in the case where smear occurs in the captured image of the solid-state imaging device 2, the luminance signal in the region of one line in the vertical and horizontal directions of the effective pixel region 12 is described. The change in output will be described.

  FIG. 4 shows an example of an imaging screen when smear does not occur for comparison with the case where smear occurs on the left side, and smear may occur in the imaging screen on the right side. It is a diagram showing the level of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line VLp in the vertical (vertical) direction at the horizontal (lateral) position Ph. FIG. 5 is a diagram showing the vertical pixel position on the horizontal axis by rotating 90 degrees counterclockwise to make it easy to see the diagram showing the level of the right digital signal output SC in FIG.

  In the imaging screen of FIG. 4, the sun that outputs strong light (= very bright light), a tree that outputs relatively dark reflected light, and a person that outputs relatively dark reflected light are imaged. In the case of FIG. 4, in the photoelectric conversion element in the vertical pixel area (A area) of strong light near the sun on the imaging element, a large amount of electric charge is output and the captured luminance signal value is also high, which is a saturated area. This shows a case where it does not leak to a photoelectric conversion element in a dark area that overflows and receives other weak light. In other words, the photoelectric conversion element in the vicinity of the sun in this case generates a large amount of charge necessary for the imaging signal, but its luminance signal value is near the saturation value but less than the saturation value.

  The photoelectric conversion element in the vertical pixel region (B region) near the tree generates charges necessary for the imaging signal from the relatively dark reflected light and has a low luminance signal value. Therefore, the digital signal output SC of each pixel along the two-dot chain line VLp in the vertical (longitudinal) direction of the horizontal position Ph indicates that the vertical pixel is in the A region as shown in the right side of FIG. 4 and the characteristic line Lv1 in FIG. Is not saturated, but has a relatively high value, and in the B region, the vertical direction pixel has a low value.

  FIG. 6 is a diagram illustrating an example of an imaging screen when smear 20 occurs due to strong sunlight in the image of FIG. FIG. 7 shows the characteristics of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph where the smear 20 is generated in the imaging screen of FIG. It is a figure which shows line Lv2 (solid line) by making a vertical direction pixel position into a horizontal axis. In FIG. 7, for comparison, the characteristic line of the digital signal output SC when the smear of FIG. 5 does not occur is indicated by a broken line Lv1.

  In FIG. 6, in each pixel of the solar portion (A region) aligned on the vertical (vertical) direction line VLp at the horizontal (horizontal) position Ph, more charges than the saturation level are generated by strong light. A large amount of the generated extra charge becomes a high-level noise signal and is formed in the vertical (vertical) direction line VLp direction (VCCD direction) in the photoelectric conversion unit of the CCD image sensor, for example, B The light leaks into the photoelectric conversion element in the region including the region that receives weak light (= dark region). In the photoelectric conversion element in the dark region, the leaked high level noise signal is superimposed on the original low level luminance signal. In this way, the smear 20 is generated in the photoelectric conversion element in the dark region in the vertical (vertical) line VLp direction.

  The smear 20 generated in the CCD image sensor significantly reduces the visual quality of the captured image as shown in FIG. Therefore, although it is a problem when used for an ordinary digital camera or the like, the smear 20 is particularly used in an imaging device used for biometrics for recognizing body-specific data such as a fingerprint pattern or a vein pattern of a living body for personal authentication. If it occurs, the feature points required for recognition will be buried in the high-level noise component, which will reduce the recognition rate of the feature points, which may be a serious obstacle to the recognition performance. There is.

  The signal level of the A / D converter 4 indicated by the characteristic line Lv2 (solid line) of the digital signal output SC in FIG. 7 is entirely affected by the smear 20 compared to the shape of the characteristic line Lv1 (dashed line) in FIG. Is rising. However, in the vicinity of the A region, clipping is performed with a saturation value (255 output in the 8-bit A / D conversion unit), and it cannot be increased and becomes flat. On the other hand, in the region B, the signal of the A / D conversion unit 4 is superimposed as a whole due to the influence of the smear 20 without being clipped, and its level is increased. Note that the signal level of the A / D converter 4 that is increased by the smear 20 is not shown in FIG. 6, but reaches the shielding area (for example, 13a and 13b in FIG. 2) where the black reference pixel is arranged. Reaching and increasing. This will be described later with reference to FIG.

FIG. 8 is a diagram illustrating an example of an imaging screen when the luminance signal value of the pixel in which the smear of FIG. 6 has occurred is simply subjected to batch subtraction according to the conventional method.
FIG. 9 collectively shows digital signal outputs SC from the AD conversion unit 4 for the pixels arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph where the smear 20 is generated in the imaging screen of FIG. It is a figure which shows the characteristic line Lv3 (solid line) correct | amended by the subtraction process by making a vertical direction pixel position into a horizontal axis. In FIG. 9, the characteristic line of the digital signal output SC when the smear 20 of FIG. 7 is generated is shown by a broken line Lv2 for comparison.

  8 and 9, the luminance signal value of the smear 20 detected in the shielding area where the black reference pixel is arranged is vertical at the horizontal (lateral) position Ph where the smear 20 is generated in the effective pixel area. Subtracted collectively from the luminance signal values of the pixels arranged on the line VLp in the (vertical) direction.

  When FIG. 5 is compared with FIG. 9, by simply subtracting the level of the luminance signal of the smear 20 obtained from the black reference pixel in the shielding area from the level of the luminance signal increased by the smear 20 of FIG. For the region, the luminance signal level increased by the smear 20 can be corrected to the original luminance signal level shown in FIG. However, in the area A including the sun, when the smear 20 is generated, after the value of the digital signal output SC does not increase due to clipping, the smear generation pixel detected by the black reference pixel in the shielding area Since the luminance signal value is subtracted, the luminance signal is smaller than the original luminance signal level shown in FIG. 5, and includes a darker sun portion than the surrounding sun portion as shown in the correction unit 21 of FIG. It becomes an image.

FIG. 10 is a diagram illustrating an example of an imaging screen in the case where the smear 22 is generated in the case where the solid-state imaging device 2 in FIG. 1 is a frame transfer type.
In FIG. 10, in each pixel of the solar portion (A region) aligned on the vertical (vertical) direction line VLp at the horizontal (horizontal) position Ph, more charges than the saturation level are generated by strong light. A large amount of the generated extra charge becomes a high-level noise signal and is formed on the lower side in the vertical (vertical) direction line VLp direction (VCCD direction) in the photoelectric conversion unit of the CCD image pickup device. Leaks into the photoelectric conversion element in the area (= dark area) receiving the weak light. Charges do not leak to the upper side of the A region. In the photoelectric conversion element in the dark region, the leaked high level noise signal is superimposed on the original low level luminance signal. In this way, the smear 22 is generated in the photoelectric conversion element in the lower dark region in the vertical (vertical) line VLp direction. No smear 22 occurs in the photoelectric conversion element on the upper side of the A region. That is, in the frame transfer type solid-state imaging device as shown in FIG. 10, the smear 22 is generated in the luminance signal value only in the pixel on one side (lower side) from the sun, but the smear 22 is not generated in the upper pixel.

FIG. 11 is a diagram illustrating an example of an imaging screen in a case where the simple subtraction process is performed on the luminance signal value of the smear occurrence pixel in the frame transfer type individual imaging element of FIG.
In FIG. 10, the smear 22 is generated only below the sun portion (A area) and is not generated further above the A area. Therefore, the smear detected by the black reference pixel is the same as in FIG. When the luminance signal value of the generated pixel is simply subtracted, the luminance signal level in the correction area 23 including the area A similar to FIG. 8 and up to the sky above the sun decreases. In this case, as shown in FIG. 11, the luminance signal of the correction area 23 including the area above the area A in addition to the area A of the pixels arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph. The value decreases, resulting in a dark image.

  As described with reference to FIGS. 4 to 11, the luminance signal value of the smear generation pixel detected by the black reference pixel in the shielding area is subtracted collectively from the luminance signal value in the smear generation area of the effective pixel area. The conventional smear correction method has a problem that the sun portion (A region) becomes dark due to overcorrection. Further, in the case of a frame transfer type individual imaging device, in addition to the sun portion (A region) There is a problem that the upper region also becomes dark.

  Next, it is provided in the digital signal processing unit 5 in order to realize control in which the characteristic sun portion (A region) does not become dark due to overcorrection by correcting noise highly correlated in one direction of the present embodiment. A more detailed operation content of the noise correction unit shown in FIG. 3 will be described with reference to FIGS. 12 to 25a.

First, the operation of the black reference detection unit 14 will be described with reference to FIGS.
12 illustrates the black reference detection unit 14 of FIG. 3 in the light-shielding regions arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph where the smear 20 of the imaging screen of FIG. 6 has occurred. It is a figure which cuts out a part of captured image of each pixel vicinity, and shows as an example.
FIG. 12 shows, for example, a light shielding region (E region) such as the black reference pixel region 13a shown in FIG. 2 and a part of the effective pixel region 12 for correcting noise due to smear. No smear 20 is generated in the horizontal (H) pixel positions 1 to 8 and 17 to 20, and a relatively dark image is picked up in the effective pixel region 12, and the black reference pixel region 13a is shielded from light and black. Indicates a detectable state. On the other hand, in the horizontal (H) pixel positions 8 to 15, the smear 20 is generated, so that the luminance signal value is increased, and the captured image is whitened.

FIG. 13 is a diagram illustrating an example of the digital signal output SC of the black reference detection unit 14 corresponding to each pixel illustrated in FIG.
The signal values SClv1 to SClv3 in FIG. 13 are affected by random noise. For this reason, the digital signal value SClv1 in the dark region where the light is shielded from the horizontal (H) pixel position H = 0 to 8 includes a noise component that changes in amplitude (range) from SC1 to SC2. Similarly, the digital signal value SClv2 in the region where the smear 20 at the horizontal (H) pixel position H = 9 to 15 is generated includes a noise component that changes in amplitude from SC5 to SC6, and the horizontal (H) pixel. The light-shielded dark region digital signal value SClv3 at positions H = 16 to 20 again includes a noise component that changes in amplitude from SC1 to SC2.

  FIG. 14 is a diagram illustrating an example of an average value obtained by performing an averaging process on the digital signal output SC of the black reference detection unit 14 illustrated in FIG. 13 using a non-linear filter. The black reference detection unit 14 includes a non-linear filter for detecting a difference value between luminance signal values of adjacent pixels. The nonlinear filter will be described later with reference to FIG.

  As described above, the black reference detection unit 14 also detects the difference value of the luminance signal value of each adjacent pixel in the light shielding region and outputs it together with the position information of each pixel in which each value is detected. The unit 14 may be configured to obtain the average of the pixel data for which the black reference detection is performed in the horizontal pixel direction and output the luminance signal value of the smear generation pixel.

  When the difference value of the luminance signal value of each adjacent pixel is less than or equal to the predetermined value, the average value of the luminance signal value of each adjacent pixel up to the pixels where the difference value is detected to be the predetermined value or less is used as the black reference detection result When the difference value of the luminance signal value of each adjacent pixel exceeds the predetermined value, the average value of the luminance signal value of each adjacent pixel after the detected pixel is noise when the difference value exceeds the predetermined value. Output as component value.

  For example, the noise component that changes with the amplitude (range) of SC1 to SC2 in the digital signal value SClv1 in the dark region where the light is shielded from the horizontal (H) pixel position H = 0 to 8 in FIG. 13 is smoothed and averaged AVSC1. It becomes. Similarly, the noise component that changes with the amplitude of SC1 to SC2 in the digital signal value SClv3 in the light-shielded dark region at the horizontal (H) pixel position H = 16 to 20 is smoothed and has the same level as the average value AVSC1. The average value is AVSC3. The average value AVSC1 and the average value AVSC3 are output from the black reference detection unit 14 as black reference values.

  On the other hand, the noise component changing with the amplitude of SC5 to SC6 in the digital signal value SClv2 in the region where the smear 20 at the horizontal (H) pixel position H = 9 to 15 in FIG. 13 is smoothed is averaged. The value AVSC2. The average value AVSC2 is output from the black reference detection unit 14 as a noise component value such as a smear amount value. As the first correction value of the present embodiment, an average value of the smear amount values is used.

FIG. 15 is a diagram illustrating an interval linear function of the ε filter provided in the black reference detection unit 14.
In the present embodiment, as the non-linear filter, the f (x) axis value of the x-axis value less than −ε and the value exceeding ε is 0, and the x-axis value is f (x) of the value −ε to ε. Although the ε filter whose axis value increases linearly is used, the nonlinear filter used in the black reference detection unit 14 of the present invention is not limited to the ε filter, and the luminance signal value of the smear generation pixel in the black reference pixel region Any filter may be used as long as it can detect the luminance signal value of the occurrence position and the smear generation pixel. For example, a configuration using a nonlinear filter such as a median filter or a stack filter or a configuration using a linear filter may be used.

ｘ（ｎ）： 画素位置ｎのＡＤ変換部４からのデジタル信号出力ＳＣ（ｎ）の値
ｙ（ｎ）： 画素位置ｎのフィルタ出力値
ａｋ： フィルタ係数
ｎ： 水平（Ｈ）画素位置（正の整数）
又、上記（１）式のｆ（ ）は、区間線形関数であり、以下の式（３）により定義される。

で与えられる。
尚、図１５の場合は、α＝０と定義した場合の区間線形関数を示している。 In general, a first-order ε filter is defined by the following equations (1) and (2).

x (n): value of the digital signal output SC (n) from the AD conversion unit 4 at the pixel position n y (n): filter output value at the pixel position n ak: filter coefficient n: horizontal (H) pixel position (positive Integer)
Further, f () in the above equation (1) is an interval linear function and is defined by the following equation (3).

Given in.
In the case of FIG. 15, an interval linear function when α = 0 is shown.

△Ｓｍ１（ｎ）： 画素位置ｎのスミア量の値
ＳＣｃｌｐ： 黒クランプ（黒基準）信号量の値 Further, the luminance signal value at the pixel at the horizontal (H) pixel position n where the smear of the black reference detection unit 14 is generated is defined by the following equation (4).

ΔSm1 (n): value of smear amount at pixel position n SCclp: value of black clamp (black reference) signal amount

  The average value AVSC1 of the digital signal values SCn at the positions from the horizontal (H) pixel positions 0 to 8 in FIG. 14 from the equation (4) is, for example, the horizontal pixel position n at which the digital signal value is considered to be relatively stable. The average value AVSC1 is ΔSm1 (3) + SCclp because the sum of the smear amount value and the black clamp signal amount value in this case may be obtained. However, at the horizontal (H) pixel positions 0 to 8, Since smear has not occurred, ΔSm1 (3) = 0, so AVSC1 = SCclp.

  Similarly, the sum of the smear amount value and the black clamp signal amount value when the horizontal pixel position n is 19 at the horizontal (H) pixel positions 16 to 20 may be obtained, and the average value AVSC3 ΔSm1 (19) + SCclp, but since smear does not occur even at the horizontal (H) pixel positions 16 to 20, since ΔSm1 (19) = 0, AVSC3 = SCclp.

  On the other hand, at the positions from the horizontal (H) pixel positions 9 to 15, the sum of the smear amount value and the black clamp signal amount value when the horizontal pixel position n is 10 can be obtained. AVSC2 is ΔSm1 (10) + SCclp, but since smear has occurred at the horizontal (H) pixel positions 9 to 15, ΔSm1 (10) does not become 0, and remains as it is, and AVSC2 = Δ Sm1 (10) + SCclp.

  The black reference detection unit 14 not only outputs a black clamp (black reference) signal SCclp but also outputs each horizontal (H) pixel position n and a smear amount value ΔSm1 (n) of the smear generation pixel. be able to. For example, as shown in the examples of FIGS. 12 to 14, the average values AVSC1 and AVSC3 of the output of each pixel whose smear amount value ΔSm1 (n) is 0 are output as the black clamp (black reference) signal SCclp. Then, the average value AVSC2 of the output of each pixel (9 <n <15) whose smear amount value ΔSm1 (n) is not 0 is output as the first correction value.

Here, instead of outputting ΔSm (0) to ΔSm (8) at horizontal (H) pixel positions 0 to 8 that do not exceed the predetermined value, 0 may be output. By outputting 0, the random pixel remaining in the average value is not affected by the effective pixel to be corrected.
That is, it is possible to correct only the area where smear occurs, and the digital signal output SC can be used as it is in the area where smear does not occur.

  Next, the luminance signal detection unit 15 is a processing unit that detects a signal value of the digital signal output SC before correction. For example, the luminance signal detection unit 15 detects the digital signal output SC2 at the horizontal (H) pixel position n = 7 in FIG. At position n = 12, the digital signal output SC5 is detected, and the level of the detected signal is output as the value of the luminance signal before correction.

  As described above, the orthogonal region average value detection unit 18 detects the horizontal (H) pixel position for each pixel arranged in one direction in which smear occurs in the effective pixel region, that is, for each pixel to be corrected, for example. Then, several pixels adjacent to the smear of the pixel in the orthogonal direction (horizontal direction) are selected, and the average value of the luminance signals for the several pixels is detected.

  Further, by using the orthogonal area average value detection unit 18, the smear occurrence area and its pixel position in the effective area are clarified, and the distribution of digital signal values in the smear occurrence area can be detected. Therefore, it is possible to detect the amount of effective images in the effective area. In addition, when correcting the digital signal value in the smear generation area, the smear generation area, its pixel position, and its digital signal value (average value) are clarified. Therefore, it is possible to generate a correction value for the optimum smear correction. In addition, by using the average value, it is possible to reduce the smear term that is calculated when the signal is detected, so that the signal detection accuracy can be improved.

  For example, if the luminance signal value near the light source such as the sun is saturated due to smear, but the luminance signal value of each pixel in the vertical direction is smeared but not saturated, The luminance signal value can be corrected using the first correction value described above, and the luminance signal value of the saturated pixel can be corrected using a second correction value described later. In this way, it is possible to generate an optimal smear correction value for the subject imaged at each pixel position.

In the present embodiment, the average value of the luminance signal values of adjacent pixels only in the horizontal (horizontal) direction of the black reference pixel region is used. However, the present invention is not limited thereto, and is adjacent to the pixel from which the signal is detected in the horizontal direction. It is also possible to use a pixel signal at a position adjacent in the vertical direction to each pixel at the pixel position.
However, when selecting pixels adjacent in the vertical direction, when obtaining a black reference signal value, it is necessary to select a pixel in which no smear has occurred due to the above-described difference value. In order to obtain (one correction value), it is necessary to select a pixel in which smear has occurred.

  The correction value generation unit 16 includes a smear generation pixel position detected by the black reference detection unit 14 and a luminance signal value of the smear generation pixel, a luminance signal output before correction detected by the luminance signal detection unit 15, and an orthogonal area average. Based on the horizontal black reference pixel or smear signal detection output detected by the value detection unit 18, the luminance signal value of the smear generation pixel is calculated, and whether or not the luminance signal value has reached a saturation value is determined. If the luminance signal value has not reached the saturation value, the first correction value is output, and if the luminance signal value has reached the saturation value, a second correction value to be described later is generated and output. To do.

  In the correction unit 17, the correction value (either the first correction value or the second correction value) determined by the correction value generation unit 16 with respect to the digital signal output SC output from the A / D conversion unit 4. The output of is subtracted and output. As a result, for the output from the pixel whose luminance signal value has reached the saturation value due to smearing, correction considering the saturation value is performed, and the output from the pixel whose luminance signal value has not reached the saturation value is performed. On the other hand, correction based on the smear level detected in the black reference pixel region is performed. That is, it is possible to generate an optimal smear correction value for the subject imaged at each pixel position.

  The correction unit 17 corrects the digital signal output SD corrected by subtracting the correction value generated by the correction value generation unit 16 from the digital signal output SC proportional to the imaging signal output SA output from the solid-state imaging device 2. Is output.

Next, a second correction value generation method and a correction method using the first and second correction values will be schematically described.
FIG. 16 is an enlarged view of a part of the pixel area of the solid-state imaging device 2 used in the present embodiment.
In FIG. 16, a Ca area is an area composed of a plurality of adjacent pixels which are adjacent in the horizontal direction from the C pixel where it is determined that smear has occurred and are similarly determined that smear has occurred. On the other hand, the Da region is a region that is adjacent in the horizontal direction from the D pixel that is determined not to have smear, and that is similarly composed of a plurality of adjacent pixels that are determined to have no smear.

  In this case, when selecting several adjacent pixels in each area of Ca and Da, the luminance signal value of the adjacent pixel is determined as in the above-described determination of the black reference pixel and the smeared pixel. Depending on whether the difference value is greater than or equal to the predetermined value, the process of selecting the previous pixel or the subsequent pixel is not performed, and the adjacent pixels are selected in order according to the processing order. To go.

FIG. 17 is a diagram illustrating photoelectric conversion characteristics of the digital signal output SC with respect to the incident light amounts of the Ca region and the Da region illustrated in FIG. 16 in order to explain the operation principle of the correction value generation unit 16 of the present embodiment. is there.
In FIG. 17, the vertical axis represents the digital signal output SC from the AD conversion unit 4, and the horizontal axis represents the amount of light incident on the photodiode constituting each pixel of the solid-state imaging device 2. The average value characteristic D of the Da region is the maximum value (= saturation value) of the A / D converter 4 from the black clamp (black reference) signal SCclp (incident light amount is 0) when the incident light amount changes from 0 to A4. The characteristic that the digital signal output changes linearly up to 255 (incident light amount: A4) is shown.

  On the other hand, the characteristic C of the average value in the Ca region is a characteristic of the digital signal output SC when smear occurs. When the incident light quantity is 0 to A2, the signal for each incident light quantity is compared with the characteristic D by smear. It shows the characteristic that the output is increased by the difference value of SCa-SCclp. In the characteristic C, the amount of incident light is A2, and the digital signal output SC of the A / D conversion unit 4 reaches the maximum value max = 255 of the A / D conversion output.

  In other words, FIG. 17 shows the characteristic C of the average value of the A / D conversion output in the Ca area where smear shown in FIG. 16 shows the A / D conversion output in the Da area where smear does not occur. Compared to the characteristic D of the average value, it is shown that the saturation is reached at max = 255 in the dynamic range of the A / D conversion unit 4 at (A2) with a dark incident light amount. Therefore, in the characteristic C, the change in the incident light amount between A2 and A4 is not reflected in the AD conversion output, and a constant value of max = 255 is output.

  In this characteristic C, even if the incident light quantity becomes A2 or more which is a constant value, the A / D conversion output of the A / D conversion unit 4 is saturated and does not increase at a maximum dynamic range of max = 255. If the incident light quantity becomes equal to or greater than A2, which is a constant value, the luminance signal value of the smear generation pixel occupying the digital signal output SC is decreased by a linear ratio similar to the increase rate in the characteristic C so far. It is considered equivalent to In the present embodiment, the difference value between the characteristic C and the characteristic D in which the amount of incident light is between A2 and A4 is set as the second correction value. That is, when the amount of incident light when smear occurs is between A2 and A4, the smear value ΔSm (A3) = 255−SCd.

  Therefore, in the characteristic C of FIG. 17 showing the A / D conversion output of the A / D conversion unit 4 when smear occurs, the luminance signal value Δ of the smear occurrence pixel which is the first correction value up to the incident light amount A2. Correction may be performed using Sm (0) = SCa−SCclp. However, when the incident light amount is larger than A2, for example, when the incident light amount is A3, the smear amount ΔSm (A3) = 255−SCd that is the second correction value. Is used to correct the digital signal output of the smear-generated pixel. Regarding the magnitude relationship between ΔSm (0) and ΔSm (A3), ΔSm (0)> ΔSm (A3), and ΔSm (A3) is a value obtained by subtracting the value of characteristic D from the maximum value 255. It becomes.

  If the incident light quantity in FIG. 17 is A2 to A4, when the first correction value is simply subtracted as the correction value from the luminance signal value of the pixel where the smear has occurred, FIG. 8 or FIG. Since the luminance signal value of the smear generation pixel is corrected excessively beyond the original signal value of the pixel as in the sun portion of the A region shown, there is a possibility that the pixel becomes dark. In other words, when the incident light quantity is from A2 to A4, if the first correction value is used as the correction value, overcorrection occurs, and for example, a sun sink occurs in the sun area such as area A in FIG. The above quality will be deteriorated. In order to avoid the occurrence of the black sun, as a correction value in the case where the smear occurs when the incident light amount is A2 to A4 in FIG. 17, the smear amount ΔSm (A3) = second correction value is used. 255-SCd may be used.

  In order to generate the second correction value, the orthogonal region average value detection unit 18 performs C in the horizontal direction (H direction) orthogonal to the direction with high noise correlation (V direction) as shown in FIG. An average value of a range (Ca region, Da region) of ± 2 pixels centering on the pixel and the D pixel is obtained, and a change amount of the luminance signal value captured as the region is detected. Since pixel data adjacent to each other centering on the C pixel and the D pixel (or adjacent pixel data) has a high correlation value of the luminance signal, the luminance signal value captured as an area for several pixels (Ca area, Da area) Changes can be used.

Here, the operation of the orthogonal region average value detection unit 18 will be further described.
FIG. 18 is an enlarged view of a part of the pixel area of the solid-state imaging device 2 used in the present embodiment, as in FIG. 16, but the horizontal (where smear is generated more than in FIG. 16). The width (number of pixels) in the (lateral) direction is reduced to one pixel.

  In FIG. 18, an Ea area indicates an area including several pixels adjacent to the E pixel in which smear has been generated in the horizontal direction. Since each pixel adjacent to the vertical direction including the E pixel is affected by smear, the value of the digital signal output SC is large. On the other hand, the Da region shows a region including D pixels not affected by smear and several adjacent pixels not affected by smear in the horizontal direction, as shown in FIG. .

FIG. 19 is a diagram showing the photoelectric conversion characteristics of the digital signal output SC with respect to the incident light amount in the region Ea of FIG.
The vertical axis, horizontal axis, characteristic C, and characteristic D in FIG. 19 are the same as those in FIG. 17, and an average characteristic E of the A / D conversion output in the Ea region is added thereto.

  The straight line of the characteristic E in FIG. 19 is affected by the smear generated in the E pixel of FIG. 18 as compared with the straight line of the characteristic D, but the average value of the A / D conversion output of the five pixels in the Ea region is Therefore, it is a straight line slightly parallel to the top. That is, the value of the characteristic E is only slightly larger than the characteristic D by the amount of 1 pixel smear divided by 5 pixels.

  By selecting a plurality of pixels in the direction orthogonal to the smear generation direction and detecting the average value of the A / D conversion output signal values of the selected pixels of the E pixel in which the smear has occurred in this way. Thus, an approximate value of the A / D conversion output signal value of the original E pixel before the occurrence of smear can be detected. In other words, since the A / D conversion output signal value of the original E pixel before the occurrence of smear is an average value of a plurality of pixels in a direction orthogonal to the smear generation direction, an approximate value of the original value can be obtained. Can be estimated from there.

  This is because the A / D conversion output signal values of a specific pixel and surrounding pixels surrounding the specific pixel in a normal imaging state have a high correlation, and the average value is detected. In this case, the value without smear is reduced from the average value (= approximate value). In order to approximate the A / D conversion output signal value of the characteristic E to the A / D conversion output signal value of the characteristic D, the number of output data from the pixels for obtaining the average value may be increased. From the above, the A / D conversion output signal value of the characteristic E approximates the A / D conversion output signal value of the characteristic D, and if an A / D conversion output signal value of the characteristic E having a sufficient number of data is obtained. It can be understood that a value equivalent to the A / D conversion output signal value of the characteristic D can be obtained from the approximate value.

  Further, for example, when the black reference detection unit 14 detects the position of the pixel where smear has occurred (the horizontal pixel position of the E pixel in FIG. 20) and the smear signal value at that pixel position, By subtracting the smear signal value detected by the black reference detection unit 14 from the A / D conversion output signal value and then detecting the average value with other pixels in the Ea region, the A / D of the characteristic D Further approximation to the converted output signal value is possible. As a result, the smear correction accuracy can be further improved.

  FIG. 18 shows the case where smear occurs only in pixels arranged in one column in the vertical direction. However, when smear occurs across pixels in a plurality of lines in the vertical direction, for example, the average value described above is used. It is possible to cope with each area for obtaining the value by increasing the number of horizontal pixels in the area.

  In the following, with reference to FIGS. 20 to 23, the correction value generation unit 16 generates a second correction value by using the output of the orthogonal region average value detection unit 18, thereby overcorrection by the first correction value. A description will be given of what can be prevented.

FIG. 20 illustrates an example of an imaging screen (where the smear of the effective pixel region is omitted) when the smear illustrated in FIG. 6 occurs in order to explain the operation of the correction value generation unit 16. It is the figure shown with the black reference | standard pixel area | region. A smear 24 occurs in the black reference pixel region of FIG.
4, 6, 8, 10, and 11, a digital signal output from the AD conversion unit 4 for each pixel arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph of the captured image. The level of the SC has been described. In FIG. 20, the level of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the lines Fh, Gh, and Hh indicating the horizontal (lateral) direction region of the captured image is illustrated. This will be described with reference to FIGS.

The digital signal output SC of the horizontal region F and region G in FIG. 20 is shown in FIGS.
FIG. 21 is a diagram illustrating the level of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line Fh indicating the horizontal (lateral) direction region of the captured image of FIG. The solid line Lh1 in the figure is a line indicating the level of the digital signal output SC for each pixel along the line Fh indicating the horizontal (lateral) direction region, and the broken line Lhav1 is the digital signal output SC for each pixel. Is a line indicating the average value output from the orthogonal region average value detection unit 18 corresponding to the.

  Since the dashed line Lhav1 is an average value, the rise and fall changes are less steep than the solid line Lh1, but the luminance signal from the sun or the like is higher than the number of pixels in the horizontal direction where smear has occurred. Since the number of horizontal pixels as the level is large, the level SCL1 of the broken line Lhav1 of the average value is also a high output level.

  On the other hand, FIG. 22 is a diagram showing the level of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line Gh indicating the horizontal (lateral) direction region of the captured image of FIG. The solid line Lh2 in the figure is a line indicating the level of the digital signal output SC for each pixel along the line Gh indicating the region in the horizontal (lateral) direction, and the broken line Lhav2 is the digital signal output SC for each pixel. Is a line indicating the average value output from the orthogonal region average value detection unit 18 corresponding to the.

  Also in this case, since the broken line Lhav2 is an average value, the rise and fall changes are less steep than the solid line Lh1, but the number of pixels in the horizontal direction where smear occurs is the luminance signal from the sun or the like in FIG. Therefore, the level SCL2 of the broken line Lhav2 of the average value reaches only a lower output level than the level SCL1 of FIG.

Based on FIGS. 21 and 22, the following determination can be made.
(Judgment one)
As shown in FIG. 21, the number of pixels arranged in the horizontal direction for imaging a bright subject such as the sun is larger than the number of pixels arranged in the horizontal direction for imaging a smeared portion. Therefore, the horizontal imaging region becomes wider and the number of detected luminance signals also increases. For this reason, the average value Lhav1 in the horizontal direction, which rises relatively gently as compared with the digital signal output SC, also shows a large value because a sufficient rise time can be obtained.

(Judgment 2)
As shown in FIG. 22, when smear is generated in a subject image under a relatively dark condition, the number of pixels arranged in the horizontal direction in order to capture the smeared portion is a bright subject such as the sun. The digital signal output SC of the smear portion is only at a high level, and the digital signal output SC of the other portion is at a low level. Therefore, the average value Lhav2 in the horizontal direction, which rises relatively gently compared to the digital signal output SC, shows a small value because a sufficient rise time cannot be obtained.

(Judgment 3)
At the pixel position in the horizontal (H) direction where the luminance signal value is large because the sun is imaged on the line Fh indicating the region shown in FIG. 20, the situation shown in the above judgment 1 as shown in FIG. However, in the case of the situation shown in the first judgment, the second correction value is used.
On the other hand, at the pixel position in the horizontal (H) direction that has a large luminance signal value due to the occurrence of smear in the line Gh indicating the region shown in FIG. 20, as shown in FIG. In most cases, the first correction value is used when the situation shown in the second determination is made.

FIG. 23 is a diagram illustrating the level of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line Hh indicating the horizontal (lateral) direction region of the black pixel detection region of the captured image of FIG. . A solid line Lh3 in the figure is a line indicating the level of the digital signal output SC for each pixel along the line Hh indicating the region in the horizontal (lateral) direction.
In this way, by using the above determination 3 when the correction value generation unit 16 generates the correction value, overcorrection when the correction unit 17 corrects the digital signal output SC can be prevented. Note that the first correction value used in the case of judgment 2 is the luminance signal value (the smear amount of the smear amount) at the pixel at the horizontal (H) pixel position n where the smear of the black reference detection unit 14 of Equation (4) occurs. Value) ΔSm1 is used.

ここで、ＡＤｍａｘは、Ａ／Ｄ変換部４から出力されるデジタル信号出力ＳＣの最大値ｍａｘであり、８ビットの場合には２５５となり、１０ビットの場合には１０２３となる。ａｍは係数である。

As a second correction value used in the case of determination 1, in order to prevent overcorrection, the output of the average value of the orthogonal region average value detection unit 18 is the maximum of the digital signal output SC output from the A / D conversion unit 4. A smear amount value ΔSm2 subtracted from the value max is defined as the following equation (5).

Here, ADmax is the maximum value max of the digital signal output SC output from the A / D converter 4, and is 255 for 8 bits and 1023 for 10 bits. am is a coefficient.

  In addition, when the solid-state imaging device 2 of FIG. 1 as shown in FIGS. 10 and 11 is a frame transfer type, a smear 22 is generated in the luminance signal value only in the pixel on one side (lower side) from the sun. When the smear 22 does not occur in the upper pixel, it can be corrected by using the signal of the luminance signal detection unit 15 for the pixel in which the upper smear 22 does not occur in addition to the above.

(Judgment 4)
When the output of the luminance signal detection unit 15 is equal to or lower than the signal of the black reference detection unit 14, the correction process is not performed.

Here, the smear amount value ΔSm2 will be further described with reference to FIG.
Assuming that the characteristic D of the thin solid line in FIG. 17 is an average value in the orthogonal direction, in the region where the incident light quantity is A2 or more, the influence of the luminance signal value of the smear occurrence pixel gradually decreases as the light quantity increases. Just as you did. Therefore, in a region where there are many pixels that output a high luminance signal, such as the sun, a value obtained by subtracting the thin line characteristic D from the maximum value max of the digital signal output SC is used.
The correction unit 17 subtracts the correction value (first correction value or second correction value) determined and generated by the correction value generation unit 16 from the digital signal output SC of the A / D conversion unit 4. Make corrections.

但し、入射光量≧Ａ２のとき、ａ＝１，ｂ＝０
入射光量＜Ａ２のとき、 ａ＝０，ｂ＝１ When the correction value determined and generated by the correction value generation unit 16 is represented by a general-purpose equation, the following equation (7) is obtained.

However, when the incident light quantity ≧ A2, a = 1, b = 0
When incident light quantity <A2, a = 0, b = 1

  Here, for example, when the amount of incident light in FIG. 17 is in the vicinity of A2, there is a case where an intermediate correction value between the first correction value and the second correction value is necessary because the correction value changes. In that case, on the basis of statistical processing and experimental data, the third correction value is defined by setting the values of a and b with respect to the incident light amount of the above equation (7), for example, a = b = 0.5, etc. Optimization can also be performed by using the third correction value.

  The outline of the processing flow described so far is shown in the flowcharts of FIGS. The contents shown in S103 (recording process) of FIG. 25 can be realized by the same process as that of the conventional means. 24 and 25 can be handled by software used in a CPU (μ processor), a PC (personal computer), or the like.

  FIG. 24 is a flowchart showing processing related to the present invention in the digital signal processing unit 5 of the present embodiment, and FIG. 25 shows the change amount (difference value) of the digital signal output SC of the adjacent pixels in step 1 of FIG. Is a flowchart showing in more detail the process of detecting.

  First, in the process of FIG. 24, the black reference detection unit 14 of the digital signal processing unit 5 detects the black reference from the black reference detection area and also detects the change amount (difference value) of the digital signal output SC of the adjacent pixels. The processing is started (S1), and the horizontal pixel position number j and the vertical pixel position number k in the effective pixel area are reset to 0 (S2).

  The luminance signal detection unit 15 detects a luminance value (digital signal output SC) before correction from the effective pixel region (S3), and the orthogonal region average value detection unit 18 generates a one-way smear in the effective pixel region. The horizontal (H) pixel position of each pixel lined up is detected, several pixels adjacent to the smear of the pixel in the orthogonal direction (horizontal direction) are selected, and the average value of the luminance signals for the several pixels is detected ( S4).

  In the correction value generation unit 16, the output of the black reference detection unit 14, the output of the luminance signal detection unit 15, and the output of the orthogonal area average value detection unit 18 are respectively input, and the first correction is made based on the input contents. Depending on the value, the second correction value, and the amount of incident light, one correction value is selected and generated from the third correction value (S5). The correction unit 17 performs correction by subtracting the correction value selected by the correction value generation unit 16 from the luminance value (digital signal output SC) before correction input from the AD trunk unit 4 (S6). .

When the correction of the luminance value of the pixel position number j in the horizontal direction in the effective pixel area is finished, has the pixel position number j reached the maximum value Hmax in the horizontal direction, that is, whether the pixel at the right end in the horizontal direction in the effective pixel area has reached? It is determined whether or not (S7).
When the pixel position number j is not Hmax (S7: No), 1 is added to the pixel position number j (j + 1), and the process returns to Step 3.
When the pixel position number j is Hmax (S7: Yes), it is determined whether or not the pixel position number k has reached the maximum value Vmax in the vertical direction, that is, the lowermost pixel in the vertical direction within the effective pixel region. (S8).

If the pixel position number k is not Vmax (S8: No), 1 is added to the pixel position number k (k + 1), and the process returns to step 3.
If the pixel position number k is Vmax (S8: Yes), the process ends.

  In the process of FIG. 25, when the change amount of the digital signal output (SC) is detected by the black reference detection unit 14, the horizontal pixel position number i in the black reference detection region is reset to 0 (S101).

  The black reference detection unit 14 performs a filtering process on the input digital signal output (SC) using a nonlinear filter such as an ε filter, and as a result, obtains a filter output F (i) (S102). The filter output F (i) is recorded as MEM (i) in the storage device inside the black reference detection unit 14 (S103).

Whether or not the pixel position number i has reached the horizontal maximum value Hmax, that is, the rightmost pixel in the horizontal direction in the effective pixel area when the filtering of the horizontal pixel position number i in the black reference detection area is finished. Is determined (S104).
When the pixel position number i is not Hmax (S104: No), 1 is added to the pixel position number i (i + 1), and the process returns to step 102.
When the pixel position number i is Hmax (S104: Yes), the process of the black reference detection unit 14 is terminated and the process proceeds to step S2 in FIG.

  As described above, in the present embodiment, by using the orthogonal area average value detection unit 18, the smear generation area, the pixel position, and the digital signal value (average value) are clarified, and the smear and the high in the effective area are increased. Since it is possible to discriminate the subject from luminance, it is possible to detect the distribution of digital signal values in the smear generation region. Therefore, when correcting the digital signal value in the smear generation region, it is possible to generate an optimal smear correction value for the subject imaged at each pixel position. As a result, the luminance value (digital signal output SC) of the effective image in the effective region can be detected without causing blackening or the like in the high luminance image portion. Further, by using the average value, it is possible to reduce the smear term that is calculated when detecting the signal, so that the signal detection accuracy can be improved.

  In addition, the noise correction unit for correcting smear provided in the digital signal processing unit 5 according to the present embodiment does not increase the processing load with only simple calculation. For example, a line memory, a comparator, a condition determination logic, etc. Since it can be configured with relatively small gate-scale hardware, and no additional line memory or special mechanism parts are required, the cost can be reduced (low cost), and the circuit scale can be reduced, thereby reducing the size.

  Further, in the above smear correction, the processing time after detecting the luminance signal value of the smear occurrence pixel only needs to be a delay of the gate signal. For example, the smear correction value is obtained and corrected within the horizontal blanking period. Thus, the present invention can be applied to a moving image in which the processing time for smear correction cannot be made long.

Embodiment 2. FIG.
In the first embodiment, the case where the noise having a high correlation in one direction generated in the effective pixel region of the solid-state imaging device is a smear is described. However, as described above, HCCD is used as the noise having a high correlation in one direction. There may be noise in which a difference (variation) occurs in luminance signal values between HCCDs due to variations in amplification characteristics (sensitivity) of charge output units in a plurality of imaging devices. In the second embodiment, a case where the luminance signal values between the HCCDs of the multiple HCCD image sensors vary will be described.

  Note that a CCD image pickup device including a plurality of HCCDs is used because, for example, an image pickup device having a large number of effective pixels needs to read out charges at a high speed and perform frame transfer. However, when a plurality of HCCDs are provided, the amplification characteristics of the charge output unit vary, and thus a difference value (offset) occurs in the luminance signal value between the HCCDs, which may reduce the visual quality. In particular, there is a problem that the visual quality is remarkably deteriorated particularly in the dark part of the imaging screen.

FIG. 26 is a diagram illustrating a configuration of an individual imaging device including a two-channel horizontal charge transfer path (HCCD) according to the present embodiment.
In FIG. 26, the imaging region is divided into two parts, a left imaging region 19 and a right imaging region 22, and each of the divided imaging regions includes a photodiode and a vertical charge transfer unit.

  A left horizontal charge transfer unit 20 is provided below the imaging region 19, and a left imaging charge output unit 21 is provided at the left end thereof. Similarly, a right horizontal charge transfer unit 23 is provided below the imaging region 22, and a right imaging charge output unit 24 is provided at the right end thereof.

  The basic functions of the photodiode, the vertical charge transfer unit, the horizontal charge transfer units 20 and 23, and the imaging charge output units 21 and 24 shown in FIG. 26 are the same as those of the photodiode 8 shown in FIG. Since the functions of the vertical charge transfer unit 9, the horizontal charge transfer unit 10, and the imaging signal output unit 11 are the same, detailed description thereof is omitted, but the imaging region is divided into left and right parts, and the horizontal charge transfer unit 20, 23 is different in that the transfer direction of charges in each of the two channels is two channels in opposite directions on the left and right.

FIG. 27 is a diagram showing the digital signal output SC of the solid-state imaging device provided with the two-channel horizontal charge transfer path of the present embodiment.
As shown in FIG. 26, the solid-state imaging device 2 is divided into left and right parts and an HCCD having two channels of outputs is provided. The left and right output characteristics have a difference value (variation) of ΔSCG as shown in FIG. In some cases, a difference value occurs between the left and right signals, so that the captured image is displayed with a step difference in luminance between the left and right.

  The configuration of the noise correction unit and the processing content shown in the first embodiment can also be applied to an individual imaging device provided with such a two-channel horizontal charge transfer path. An optimal correction value can be generated. As a result, the luminance value (digital signal output SC) of the effective image in the effective region can be output without generating a step in luminance between the left and right.

  The configuration of the noise correction unit and the processing content thereof described in the first embodiment can be applied to vertical noise caused by VCCD defects and thermal noise when long-time exposure is performed. Is possible.

1 is a block diagram showing a basic configuration of an imaging apparatus according to Embodiment 1 of the present invention. It is a figure which shows schematic structure of the solid-state image sensor 2 of FIG. It is a block diagram which shows schematic structure of the noise correction | amendment part provided in the digital signal processing part 5 of FIG. For comparison with the case where smear occurs on the left side, an example of an imaged screen when smear does not occur is shown, and on the right side there is a possibility that smear may occur in the imaged screen (no occurrence) It is a figure which shows the level of the digital signal output SC from the AD conversion part 4 about each pixel located in a line (VLp) of the vertical (vertical) direction in (horizontal) position Ph. FIG. 5 is a diagram illustrating the vertical pixel position on the horizontal axis by rotating 90 degrees counterclockwise to make the diagram showing the level of the digital signal output SC on the right side of FIG. 4 easier to see. It is a figure which shows an example of the imaging screen when the smear 20 generate | occur | produces with the strong light of the sun in the image of FIG. The characteristic line Lv2 (solid line) of the digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph where the smear 20 is generated in the imaging screen of FIG. ) With the vertical pixel position on the horizontal axis. It is the figure which showed an example of the imaging screen in the case of carrying out the simple subtraction process of the luminance signal value of the pixel which the smear generate | occur | produced of FIG. 6 of the conventional method. The digital signal output SC from the AD conversion unit 4 for each pixel arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph where the smear 20 is generated in the imaging screen of FIG. It is a figure which shows the characteristic line Lv3 (solid line) which made the vertical direction pixel position horizontal axis. It is the figure which showed an example of the imaging screen when the smear 22 generate | occur | produces when the solid-state image sensor 2 of FIG. 1 is a frame transfer type. It is the figure which showed an example of the imaging screen at the time of implementing a simple subtraction process with respect to the luminance signal value of a smear generation | occurrence | production pixel to the frame transfer type individual image sensor of FIG. In order to describe the black reference detection unit 14 in FIG. 3, in the vicinity of each pixel of the light shielding region arranged on the line VLp in the vertical (vertical) direction at the horizontal (horizontal) position Ph where the smear 20 of the imaging screen in FIG. 6 occurs. It is a figure which cuts out a part of captured image and shows as an example. It is a figure which shows an example of digital signal output SC of the black reference | standard detection part 14 corresponding to each pixel shown in FIG. It is a figure which shows an example of the average value which performed the averaging process using the non-linear filter for digital signal output SC of the black reference | standard detection part 14 shown in FIG. It is a figure which shows the interval linear function of the epsilon filter provided in the black reference | standard detection part. FIG. 3 is a diagram illustrating a part of a pixel area of a solid-state imaging device 2 used in Embodiment 1 that is cut out and enlarged. FIG. 17 is a diagram illustrating photoelectric conversion characteristics of the digital signal output SC with respect to incident light amounts of the C pixel and the D pixel illustrated in FIG. 16 in order to explain the operation principle of the correction value generation unit 16 according to the first embodiment. It is the figure which cut out and expanded and showed the one part pixel area of the solid-state image sensor 2 used by this Embodiment similarly to FIG. It is the figure which showed the photoelectric conversion characteristic of digital signal output SC with respect to the incident light quantity of Ea area | region of FIG. In order to explain the operation of the correction value generation unit 16, an example of the imaging screen (the description of the smear of the effective pixel region is omitted) when the smear shown in FIG. It is the figure shown with. It is a figure which shows the level of the digital signal output SC from the AD conversion part 4 about each pixel located in a line Fh which shows the area | region of the horizontal (horizontal) direction of the captured image of FIG. It is a figure which shows the level of the digital signal output SC from the AD conversion part 4 about each pixel located in a line Gh which shows the area | region of the horizontal (horizontal) direction of the captured image of FIG. It is a figure which shows the level of digital signal output SC from the AD conversion part 4 about each pixel located in a line Hh which shows the area | region of the horizontal (horizontal) direction of the black pixel detection area | region of the captured image of FIG. It is a flowchart which shows the process regarding this invention in the digital signal processing part 5 of this Embodiment, 25 is a flowchart showing in more detail the process of detecting the amount of change (difference value) in the digital signal output SC of adjacent pixels in step 1 of FIG. 6 is a diagram illustrating a configuration of a solid-state imaging device including a two-channel horizontal charge transfer path (HCCD) according to Embodiment 2. FIG. FIG. 6 is a diagram illustrating a digital signal output SC of an individual imaging device provided with a two-channel horizontal charge transfer path according to the second embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Lens, 2 Solid-state image sensor, 3 Analog signal processing part, 4 A / D conversion part, 5 Digital signal processing part, 6 Timing control part, 7 Control part, 8 Photodiode, 9 Vertical charge transfer part, 10 Horizontal charge transfer 11, charge output means, 12 effective pixel area, 13 black reference pixel area, 14 black reference detection section, 15 luminance signal detection section, 16 orthogonal area average value detection section, 17 correction section, and 18 correction value generation section.

Claims (7)

A photoelectric conversion unit in which a plurality of photoelectric conversion elements that output charges corresponding to the intensity of received light are arranged in a matrix of a plurality of rows and a plurality of columns as pixels,
A vertical transfer unit that transfers charges output from each photoelectric conversion element in the vertical direction;
And a horizontal transfer unit for transferring the charges output from the vertical transfer unit in the horizontal direction,
At least one of the upper, lower, left, and right outer peripheral edge regions of the effective pixel region used for imaging in the photoelectric conversion unit is configured to shield external light input to each of the photoelectric conversion elements in a plurality of rows of the peripheral region. A solid-state imaging device in which a light-shielding region for outputting a luminance signal value serving as a black reference is formed;
Noise that is highly correlated in one direction with the luminance signal value of the pixel serving as the black reference detected from the output of the light-shielding region, from the luminance signal value of the pixel in which highly correlated noise is generated in one direction in the effective pixel region An image pickup apparatus including a signal processing unit having at least a function of subtracting a difference from a luminance signal value of a pixel in which
The signal processing unit
A black reference detection unit that detects a luminance signal value of at least a pixel serving as a black reference and a luminance signal value of a pixel in which highly correlated noise is generated in one direction from the luminance signal of each pixel in the light shielding region;
A luminance signal detector that detects a luminance signal value of each pixel in the effective pixel region;
A predetermined number of pixels are continuously selected in a horizontal direction orthogonal to a vertical direction in which pixels having a highly correlated noise in one direction are arranged with respect to the pixels in which the luminance signal value is detected by the luminance signal detection unit. An orthogonal region average value detection unit for detecting a luminance signal value of an average value of each pixel;
The luminance signal value of the pixel in which the high-correlation noise is generated in one direction of the effective pixel region is input with the luminance signal value detected by the black reference detection unit, the luminance signal detection unit, and the orthogonal region average value detection unit. A correction value generation unit that generates a correction value corresponding to the value;
A correction unit that receives a correction value generated by the correction value generation unit and corrects a luminance signal value of a pixel in which highly correlated noise is generated in one direction of the effective pixel region. apparatus. The correction value generation unit
Based on the luminance signal value of the pixel in which highly correlated noise is detected in one direction detected by the black reference detection unit and the luminance signal value of the pixel detected by the luminance signal detection unit,
A first correction value that is a difference value between a luminance signal value of a pixel serving as a black reference of the black reference detection unit and a luminance signal value of a pixel in which highly correlated noise is generated in one direction;
The output is selected from the saturated luminance signal value and a second correction value that is a difference value between average values of the orthogonal region average value detection unit corresponding to the saturated pixel. Imaging device. The selection of the output of the correction value generator is as follows:
When the luminance signal value of the pixel in which the highly correlated noise is generated in one direction of the black reference detection unit exceeds a predetermined value and the luminance signal value of the luminance signal detection unit corresponding to the pixel is not saturated To select the first correction value,
When the luminance signal value of a pixel in which highly correlated noise occurs in one direction of the black reference detection unit exceeds a predetermined value, and the luminance signal value of the luminance signal detection unit corresponding to the pixel is saturated The imaging device according to claim 2, wherein the second correction value is selected. The correction value generation unit
A third correction value is calculated by combining the first correction value and the second correction value at a predetermined ratio that changes linearly,
In the vicinity of a change point where the luminance signal value of the luminance signal detection unit corresponding to the pixel is saturated when the luminance signal value of the pixel in which the highly correlated noise occurs in one direction of the black reference detection unit exceeds a predetermined value The imaging apparatus according to claim 3, wherein the third correction value is selected and output. In addition to the luminance signal value, the black reference detection unit further includes:
The difference value of the luminance signal value of each pixel which adjoins in the said light-shielding area | region is also detected, and each said value is output with the positional information on each pixel by which it detected. Imaging device. The black reference detection unit
When the difference value between the luminance signal values of the adjacent pixels is equal to or less than a predetermined value,
Output the average value of the luminance signal value of each adjacent pixel up to the pixels where the difference value is detected to be less than or equal to the predetermined value,
When the difference value of the luminance signal value of each adjacent pixel exceeds a predetermined value,
6. The imaging apparatus according to claim 5, further comprising a processing unit that outputs an average value of luminance signal values of adjacent pixels after the detected pixel when the difference value exceeds a predetermined value as a black reference detection result. . The black reference detection unit
The imaging apparatus according to claim 5, further comprising a non-linear filter for detecting a difference value between luminance signal values of the adjacent pixels.

Images