JP2007028026A - Imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging apparatus capable of eliminating random noise from correction data and correcting longitudinal stripes and shading in a column direction. <P>SOLUTION: The imaging apparatus is configured to include: an image sensor 2 wherein a plurality of pixels each converting an object light into an electric image signal are arranged two-dimensionally; a mechanical shutter 1 for shutting off the object light applied to the image sensor; a filter processing section 8 for applying prescribed filter processing to the image signal by each processing unit of rows or columns of the pixel arrangement of the image sensor; a dark state correction data acquisition section 5 for storing a processing result by the filter processing section applied to the image signal at light shut-off by the mechanical shutter as correction data in a dark state; and a subtractor 6 for subtracting the dark state correction data stored in the dark state correction data acquisition section from the image signal and providing an output of a subtraction result as a corrected image signal. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an imaging device that uses an imaging device and outputs a video signal.

  Solid-state imaging devices such as CCD image sensors and MOS image sensors are widely used in digital cameras and the like. The output signal of the CCD image sensor or the MOS type image sensor includes fixed pattern noise caused by dark current unevenness. In addition, since many MOS image sensors perform processing in parallel with a column, streaky fixed pattern noise is generated in the column direction. As a method for correcting fixed pattern noise, a method is known in which an output signal of a dark image sensor is stored in a memory as correction data and subtracted from a captured image signal by a subtractor.

  However, when subtraction is performed using correction data including random variation (random noise) for each pixel, there is a problem that random noise of the image data after subtraction becomes larger than the image before subtraction.

In order to solve this problem, as disclosed in Japanese Patent Application Laid-Open No. 11-333192, one having a processing unit that smoothes and removes random noise of correction data by moving average processing has been proposed. Further, as disclosed in Japanese Patent Laid-Open No. 2000-261730, a method is also proposed in which an average value of dark output is calculated for each column, stored in a line memory as correction data, and subtracted from image data. Has been.
JP-A-11-315992 JP 2000-261730 A

  However, those proposed in the above publications have the following problems. That is, in the method of removing random noise from the correction data disclosed in Japanese Patent Application Laid-Open No. 11-333192, an increase in random noise can be suppressed. However, the occurrence of stripes in the column direction (vertical stripes) cannot be suppressed, and no correction is mentioned. Further, in the method of storing the average value for each column of dark images disclosed in Japanese Patent Laid-Open No. 2000-261730 in the line memory as correction data, streaking in the column direction can be suppressed without increasing random noise. There is a problem that shading correction cannot be supported.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an imaging apparatus capable of correcting random stripes and shading in the column direction while removing random noise from correction data.

  In order to solve the above-described problem, the invention according to claim 1 is directed to an imaging unit in which a plurality of pixels that convert subject light into an electrical signal and convert it into an electrical image signal are two-dimensionally arranged, and the imaging unit A shutter that blocks the subject light, a filter processing unit that performs a predetermined filter process for each processing unit on the image signal, with a row or column of the pixel array of the imaging unit as one processing unit, The processing result by the filter processing unit for the image signal at the time of light shielding by the shutter is stored in the dark correction data acquisition unit that holds the dark correction data as dark correction data and the dark correction data acquisition unit from the image signal. The image pickup apparatus is configured to include a subtracter that subtracts the dark correction data and outputs the result as a corrected image signal.

  According to a second aspect of the present invention, in the imaging apparatus according to the first aspect, the filter processing unit performs a defect removal process on the defective pixel with respect to the image signal, and smoothes the image signal after the defect removal process. It is characterized in that the processing is performed.

  According to a third aspect of the present invention, in the imaging apparatus according to the second aspect, the filter processing unit relates to a plurality of pixels in the vicinity of the row or column to which the pixel to be processed belongs, for the pixel to be processed, as the smoothing process. The image signal is averaged, and the result is output as a smoothed image signal for the pixel to be processed.

  According to a fourth aspect of the present invention, in the imaging apparatus according to the second aspect, the filter processing unit includes a median filter process as the defect removal process.

  According to a fifth aspect of the present invention, in the imaging apparatus according to any one of the first to fourth aspects, the dark correction data acquisition unit includes a frame memory that holds a plurality of dark correction data with different light shielding conditions, and an image pickup And a controller that controls to output the dark correction data of the light shielding condition corresponding to the condition from the frame memory to the subtractor.

  According to a sixth aspect of the present invention, in the imaging apparatus according to any one of the first to fifth aspects, the filter processing unit sets processing conditions for the filter processing according to a noise generation factor included in the image signal. It is characterized by changing.

  According to a seventh aspect of the present invention, in the imaging apparatus according to the sixth aspect, the filter processing unit changes a processing condition of the filter processing according to a temperature of the imaging unit.

  According to an eighth aspect of the present invention, in the imaging apparatus according to the sixth aspect, the filter processing unit changes a processing condition of the filter processing in accordance with an accumulation time of the imaging unit.

  According to a ninth aspect of the present invention, in the imaging device according to the sixth aspect, the filter processing unit changes a processing condition of the filter processing according to a gain applied to the image signal output from the imaging unit. It is characterized by.

  According to a tenth aspect of the present invention, in the imaging apparatus according to any one of the sixth to ninth aspects, the filter processing unit is configured such that the processing condition of the filter processing to be changed is the number of pixels used for the filtering processing. It is characterized by being.

  According to the first aspect of the present invention, it is possible to realize an imaging apparatus capable of reducing shading-like fixed pattern noise and vertical streak-like fixed pattern noise from image data. According to the second aspect of the invention, an increase in random noise can be suppressed when dark correction data is subtracted from an image signal. In addition, since the defect removal process is performed before the smoothing process, correction data that is not affected by the defect can be created. According to the invention of claim 3, it is possible to remove random noise from the dark image by a simple method. Moreover, according to the invention which concerns on Claim 4, a defect can be removed with a simple method before smoothing processing. According to the fifth aspect of the present invention, it is not necessary to create correction data every time shooting is performed. According to the invention of claim 6, it is possible to perform a correction process in accordance with a captured image. Moreover, according to the invention which concerns on Claim 7, it can respond to the random noise and defect of the image at the time of darkness from which quantity changes according to temperature. According to the eighth aspect of the present invention, it is possible to deal with random noise and defects in dark images whose amount varies depending on the accumulation time. Moreover, according to the invention which concerns on Claim 9, it can respond to the random noise from which a magnitude | size changes according to a gain. According to the invention of claim 10, more appropriate defect removal processing or smoothing processing can be performed.

  Next, the best mode for carrying out the present invention will be described.

  First, Example 1 according to the present invention will be described. FIG. 1 is a block diagram showing Embodiment 1 of an image pickup apparatus according to the present invention. The image pickup apparatus according to this embodiment includes a mechanical shutter 1, an image sensor 2, a preprocessing unit 3, a shutter driver 4, and dark correction data. The acquisition unit 5, the subtraction unit 6, the signal processing unit 7, and the filter processing unit 8 are configured. The dark correction data acquisition unit 5 includes a switch 5a, a controller 5b, and a frame memory 5c. The preprocessing unit 3 converts an analog signal into a digital signal by an amplifier 3a that amplifies the output of the image sensor 2. The A / D converter 3b is used.

  The output of the image sensor 2 is output to the switch 5a of the dark correction data acquisition unit 5 via the amplifier 3a and the A / D converter 3b. The output of the switch 5a is output to the frame memory 5c or the subtraction unit 6. The output of the frame memory 5 c is connected to the filter processing unit 8 and the subtracter 6. The output of the filter processing unit 8 is connected to the frame memory 5c. The output of the subtracter is connected to the signal processing unit 7. The controller 5b of the dark correction data acquisition unit 5 is connected to the switch 5a, the shutter driver 4 and the frame memory 5c, and the shutter driver 4 is connected to the mechanical shutter 1.

  Next, the operation of the imaging apparatus according to the first embodiment configured as described above will be described. The correction processing of the imaging apparatus according to the present embodiment is roughly divided into correction data creation processing and correction processing. In the correction data creation process, correction data is created. In the correction process, an actual correction process is performed on the video signal from the image sensor based on the correction data created in the correction data creation process. FIG. 2 is a flowchart showing the correction data creation process, and FIG. 3 is a flowchart showing the operation of the correction process.

  Next, correction data creation processing will be described with reference to the flowchart of FIG. First, in step S11, the controller 5b closes the mechanical shutter 1 via the shutter driver 4 and takes a dark image. The output of the image sensor 2 is amplified by the amplifier 3a and then A / D converted by the A / D converter 3b to become dark image data, which is held in the frame memory 5c via the switch 5a.

  Next, in step S12, the filter processing unit 8 performs median filter processing for each column in order to remove singular data such as defects from the dark image data acquired in step S11 and held in the frame memory 5c. Done. In step S13, the filter processing unit 8 performs a moving average process for each column in order to remove random noise on the dark image data subjected to the median filter process for each column in step S12.

  Next, the moving average processing for each column when the image data is 10 × 10 pixels will be described with reference to FIG. The moving average of the pixels to be processed in the fifth column and the fifth row is given as an average value of, for example, five pixels in the column direction centering on the processing pixel in the fifth column and the fifth row.

  In step S14, the calculation result obtained by the processing up to step S13 is stored in the frame memory 5c as correction data. This completes the dark correction data creation process.

  Next, the correction process will be described with reference to the flowchart of FIG. First, in step S21, the controller 5b controls the opening and closing of the mechanical shutter 1 via the shutter driver 4, and an image is captured, and image data relating to the captured image is read from the image sensor 2. Then, after being amplified by the amplifier 3a, A / D conversion is performed by the A / D converter 3b.

  Next, in step S22, the image data read in step S21 is output to the subtraction unit 6 via the switch 5a, and output from the frame memory 5c by the controller 5b. In step S14 of the correction data generation process. The obtained correction data is subtracted and corrected. A captured image corrected by the above procedure is obtained.

  In the median filter processing in the correction data creation processing, it is preferable that the median filter range is increased when the number of defective pixels increases. Examples of cases where the number of defective pixels increases include when the image sensor is hot or when the accumulation time is long.

  In the moving average process, when the random noise is large, it is preferable to increase the number of moving average pixels (5 pixels in the example shown in FIG. 4). Examples of the case where the random noise becomes large include when the image sensor is at a high temperature or when the gain of the preprocessing unit is high. It is preferable that the moving average number of pixels can be changed under these conditions.

  According to the present embodiment, vertical dark lines caused by offset variation for each column can be obtained without increasing random noise by using the median filter and moving average processing for each column as dark correction data. As well as the shading in the column direction can be corrected. It is not always necessary to take a dark image every time it is taken, and it may be appropriately performed according to the stability of the system and the characteristics of the image sensor.

  In the imaging apparatus shown in FIG. 1, the image sensor 2 and the amplifier 3a and the A / D converter 3b constituting the preprocessing unit 3 are shown as individual blocks, but these may be integrated into one chip. Needless to say.

  Next, a second embodiment according to the present invention will be described. FIG. 5 is a block diagram illustrating a configuration of the imaging apparatus according to the second embodiment. Components corresponding to those of the imaging apparatus according to the first embodiment illustrated in FIG. 1 are denoted by the same reference numerals. Yes. The imaging apparatus according to this embodiment includes a mechanical shutter 1, an image sensor 2, a preprocessing unit 3, a shutter driver 4, a dark correction data acquisition unit 105, a subtraction unit 6, a signal processing unit 7, and a filter processing unit 8. Yes. The dark correction data acquisition unit 105 includes a switch 105a, a controller 105b, and a plurality of frame memories 105c. The preprocessing unit 3 amplifies the output of the image sensor, and an analog signal is converted into a digital signal. An A / D converter 3b for conversion is configured.

  In the present embodiment, a plurality of frame memories are used in the dark correction data acquisition unit of the first embodiment, which makes it possible to have a plurality of correction data with different imaging conditions, thereby improving the correction effect. be able to. The output of the image sensor 2 is connected to the switch 105a of the dark correction data acquisition unit 105 via the amplifier 3a and the A / D converter 3b. The output of the switch 105a is connected to a plurality of frame memories 105c or the subtracting unit 6. The outputs of the plurality of frame memories 105c are connected to the filter processing unit 8 and the subtracter 6. The output of the filter processing unit 8 is connected to a plurality of frame memories 105c. The output of the subtracter 6 is connected to the signal processing unit 7. The controller 105b of the dark correction data acquisition unit 105 is connected to the switch 105a, the shutter driver 4, and a plurality of frame memories 105c. The shutter driver 4 is connected to the mechanical shutter 1.

  Next, the operation of the imaging apparatus according to the second embodiment configured as described above will be described. The correction processing according to the present embodiment is roughly divided into correction data creation processing and correction processing. In the correction data creation process, correction data is created. In the correction process, an actual correction process is performed on the video signal from the image sensor based on the correction data created in the correction data creation process. FIG. 6 is a flowchart showing the correction data generation process, and FIG. 7 is a flowchart showing the operation of the correction process.

  Next, the correction data creation process will be described with reference to the flowchart of FIG. First, in step S31, the controller 105b closes the mechanical shutter 1 via the shutter driver 4, and causes a dark image to be taken under certain imaging conditions. The output of the image sensor 2 is amplified by the amplifier 3a and then A / D converted by the A / D converter 3b to become dark image data, which is held in the frame memory 105c via the switch 5a.

  Next, in step S32, the filter processing unit 8 performs median filter processing for each column in order to remove singular data such as defects from the dark image data acquired in step S31 and held in the frame memory 105c. Is done. In step S33, in order to remove random noise from the dark image data subjected to the median filter processing for each column in step S32 in the filter processing unit 8, the same moving average as that of the first embodiment for each column. Processing is performed. In step S34, the calculation result obtained by the processing up to step S33 is stored in the frame memory 105c as correction data. In step S35, it is determined whether or not correction data of a predetermined number of imaging conditions has been created. If not, the above-described steps S31 to S34 are repeated while changing the imaging conditions. To do.

  Next, correction processing will be described with reference to the flowchart of FIG. First, in step S41, the controller 105b controls the opening and closing of the mechanical shutter 1 via the shutter driver 4 to capture an image, and image data relating to the captured image is read from the image sensor 2. Then, after being amplified by the amplifier 3a, A / D conversion is performed by the A / D converter 3b.

  Next, in step S42, the image data read out in step S41 is output to the subtraction unit 6 via the switch 105a, and the correction data stored in the frame memory 105c in step S34 of the correction data generation process from the image data. Of the data, correction data that meets or close to the imaging conditions is selected by the controller 105b, and is subtracted by the subtractor 6 to perform correction processing. A captured image corrected by the above procedure is obtained. As a plurality of imaging conditions, for example, those with different accumulation times of dark images, or those with different gains of the preprocessing unit can be cited.

  According to the second embodiment, by preparing a plurality of dark correction data, it is possible to perform correction processing according to the imaging conditions, and it is not necessary to create correction data every time shooting is performed.

  In each of the above embodiments, median filter processing and moving average processing are performed for each column to obtain dark correction data. However, median filter processing and moving average processing are performed row by row. Similarly, dark correction data can be obtained.

1 is a block diagram illustrating a configuration of a first embodiment of an imaging apparatus according to the present invention. 6 is a flowchart for explaining a correction data creation processing operation in the first embodiment shown in FIG. 1. 3 is a flowchart for explaining a correction processing operation in the first embodiment shown in FIG. 1. It is explanatory drawing which shows the moving average process operation | movement for every row | line | column in the correction data creation process in Example 1 shown in FIG. It is a block diagram which shows the structure of the imaging device which concerns on Example 2 of this invention. It is a flowchart for demonstrating the correction data creation process operation in Example 2 shown in FIG. It is a flowchart for demonstrating the correction | amendment processing operation in Example 2 shown in FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Mechanical shutter 2 Image sensor 3 Pre-processing part 3a Amplifier 3b A / D converter 4 Shutter driver 5,105 Dark correction data acquisition part 5a, 105a Switch 5b, 105b Controller 5c Frame memory
105c Multiple frame memories 6 Subtraction unit 7 Signal processing unit 8 Filter processing unit

Claims (10)

  An imaging unit in which a plurality of pixels that convert subject light into an electrical signal and convert it into an electrical image signal are two-dimensionally arranged, a shutter that shields the subject light from the imaging unit, and the imaging unit A processing unit that performs a predetermined filtering process on the image signal for each processing unit, and a process performed by the filter processing unit on the image signal when light is blocked by the shutter The dark correction data acquisition unit that holds the result as dark correction data, and the dark correction data held in the dark correction data acquisition unit is subtracted from the image signal, and is output as a corrected image signal. An imaging device having a subtractor.   The imaging apparatus according to claim 1, wherein the filter processing unit performs a defect removal process on a defective pixel with respect to the image signal, and performs a smoothing process on the image signal after the defect removal process. .   As the smoothing process, the filter processing unit adds and averages image signals related to a plurality of pixels in the vicinity of the row or column to which the pixel to be processed belongs, and performs smoothing on the pixel to be processed. The imaging apparatus according to claim 2, further comprising a process of outputting as an image signal after conversion.   The imaging apparatus according to claim 2, wherein the filter processing unit includes a median filter process as the defect removal process.   The dark correction data acquisition unit outputs a frame memory holding a plurality of dark correction data having different light shielding conditions and dark correction data of the light shielding conditions corresponding to the imaging conditions from the frame memory to the subtractor. The imaging apparatus according to claim 1, further comprising a controller that controls the image pickup apparatus.   The imaging apparatus according to claim 1, wherein the filter processing unit changes a processing condition of the filter processing according to a noise generation factor included in the image signal.   The imaging apparatus according to claim 6, wherein the filter processing unit changes a processing condition of the filter processing according to a temperature of the imaging unit.   The imaging apparatus according to claim 6, wherein the filter processing unit changes a processing condition of the filter processing according to an accumulation time of the imaging unit.   The imaging apparatus according to claim 6, wherein the filter processing unit changes a processing condition of the filter processing in accordance with a gain applied to the image signal output from the imaging unit.   The image processing apparatus according to claim 6, wherein the filter processing unit is configured such that the processing condition of the filter processing to be changed is the number of pixels used for the filter processing.

Images