JP4498253B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that processes an output image signal of a solid-state imaging device.

  Light from the subject was converted into an electrical signal by a solid-state imaging device such as a CCD (Charge Coupled Device) or CMOS (Complementally MOS) sensor, and corrections such as white balance correction and clamp correction to make the black level a reference level were performed. Later, digital cameras configured to perform image compression and record on a recording medium have been put into practical use. In the solid-state imaging device used in this digital camera, each pixel is configured by a photodiode.

  In the photodiode, a current called a dark current always flows even in the absence of incident light. The magnitude of the dark current varies among individual photodiodes, and this causes fixed pattern noise in the image signal output from the solid-state imaging device.

  As a method for removing fixed pattern noise from an image signal captured by a solid-state imaging device, an image signal captured in a state where the shutter is closed in advance (that is, a state where incident light is blocked) is stored in a memory, and the shutter is opened. A method of subtracting the image signal stored in the memory from the image signal picked up in this way is used (for example, Patent Document 1). This method makes use of the fact that the image signal captured with the shutter closed is almost a fixed pattern noise component, and makes it possible to remove fixed pattern noise significantly. Because it occurs frequently, it consumes a lot of power.

  Therefore, when recording a still image, a conventional digital camera does not perform fixed pattern noise removal on a preview image captured to determine the range of a still image to be finally captured, Fixed pattern noise removal is performed only on an image captured as a still image to be recorded. This reduces the number of memory accesses by removing fixed pattern noise, and suppresses power consumption and power consumption.

By the way, in order to perform white balance correction on an image captured as a still image to be recorded on a recording medium, red (R), green (G), and blue (B) 3 included in the preview image captured before that. It is necessary to obtain an accumulated value of the magnitude of the image signal for each color and determine a gain to be multiplied by each of R, G, and B so that the accumulated values are equal. Also, in the case of performing clamp correction, the size of the black level output from the solid-state imaging device is specified using the preview image captured immediately before that.
JP 2005-130045 A

  However, the preview image includes a fixed pattern noise component, and the white balance correction gain is determined in a state including the fixed pattern noise component. When this gain is multiplied by a still image from which fixed pattern noise has been removed, the accumulated values of R, G, and B colors do not match by the amount of the fixed pattern noise component, and accurate white balance correction cannot be performed.

  Also in the clamp correction, the black level obtained from the preview image includes a fixed pattern noise component. Therefore, if this black level is subtracted from a still image from which fixed pattern noise has been removed, an extra amount of the fixed pattern noise component is added. It will be clamped to a dark image.

  In this way, when the still image correction parameters such as white balance correction and clamp correction are determined using the preview image including the fixed pattern noise component, the fixed pattern noise component is removed when the fixed pattern noise is removed from the still image. However, there is a problem that the image quality deteriorates due to the error.

  The present invention has been made in view of such circumstances, and an object thereof is to provide an image processing apparatus capable of suppressing image quality degradation when performing image correction on an image from which noise components have been removed. is there.

  One embodiment of the present invention is an image processing apparatus. The apparatus includes: an imaging device that captures an image; a noise removal unit that removes a noise component from an image signal of at least one of the images captured by the imaging device; and an image of the image captured by the imaging device An image correction unit that performs image correction on the signal, and the image correction unit performs image correction on the image signal of the first image from which noise components have been removed by the noise removal unit. The image correction parameter is determined based on the magnitude of the image signal of the second image taken before the first image is taken by the apparatus and the magnitude of the noise component removed by the noise removing unit. .

  According to this aspect, when image correction is performed on the image signal of the first image from which the noise component has been removed, the parameter used for this image correction is determined as compared with the first image as in the past. In addition, not only the image signal of the second image captured first but also the noise component to be removed by the noise removing unit in addition to the image signal of the second image is considered. Thereby, the image signal included in the first image from which noise has been removed can be predicted from the image signal of the second image and the noise component. Therefore, the influence of noise can be suppressed in determining the image correction parameter, compared with the conventional case where the image correction parameter is determined only from the image signal of the second image. When image correction is performed on the image, it is possible to suppress image quality deterioration.

  In this aspect, the noise removing unit may include a memory that stores a noise component to be removed, and the noise component is an image signal captured in a state where light is blocked in the imaging device. Also good.

  Since the image captured with the light blocked in the imaging device is dominated by noise due to dark current, the image signal of this image is stored as a noise component, and the image signal of the image captured separately by the imaging device Therefore, it is possible to remove noise with high accuracy by removing this noise component.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, and the like are also effective as an aspect of the present invention.

  According to the present invention, it is possible to obtain an image processing apparatus capable of suppressing image quality degradation when performing image correction on an image from which noise components have been removed.

  Hereinafter, as an embodiment of an image processing apparatus according to the present invention, a digital camera will be described as an example. FIG. 1 is a diagram showing a configuration of a digital camera 100 according to Embodiment 1 of the present invention. This configuration can be realized in terms of hardware by a CPU, memory, or other LSI of an arbitrary computer, and in terms of software, it is realized by a program having an encoding function loaded in the memory. The functional block realized by those cooperation is drawn. Accordingly, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The digital camera 100 includes an imaging device 10, an AFE (Analog Front End) 20, a noise removal unit 30, an image correction unit 40, a display device 50, an image compression unit 60, and a recording medium 70, and light from a subject is captured. It is incident on the device 10.

  The imaging device 10 includes a lens 11, a shutter 12, and an imaging unit 13, and light incident on the imaging device 10 enters the imaging unit 13 through the lens 11 and the shutter 12. In addition, the imaging apparatus 10 can block light incident on the imaging unit 13 by closing the shutter 12.

  The imaging unit 13 includes, for example, a CCD (Charge Coupled Devices), a CMOS (Complementary Metal-Oxide Semiconductor) sensor, and the like. The imaging unit 13 includes photodiodes arranged in a matrix, and each pixel includes a pixel. Composed. The imaging unit 13 converts the light incident on each pixel into an electrical signal corresponding to the intensity, and sequentially outputs this as an image signal pixel by pixel.

  The imaging unit 13 includes an effective pixel region where light is incident and an OPB (Optical Black) region which is a pixel region where light is not always incident. The OPB area is an area provided for defining the black level of the image signal output from the imaging unit 13.

  The AFE 20 amplifies the image signal output from the imaging device 10 and converts it into, for example, a 10-bit digital signal. The noise removing unit 30, the image correcting unit 40, and the image compressing unit 60 perform respective processes on the image signal converted into a digital signal by the AFE 20.

  When the still image is recorded on the recording medium 70, the noise removing unit 30 removes fixed pattern noise from the image signal of the still image captured for recording. For the preview image, the fixed pattern noise is not removed, and the image signal output from the AFE 20 is output as it is.

  The noise removing unit 30 includes a memory 31, a subtracting unit 32, and a selecting unit 33. The memory 31 is a memory for holding an image signal captured in advance with the shutter 12 closed in the imaging apparatus 10. The image signal held in the memory 31 can be regarded as a fixed pattern noise component in each pixel. This fixed pattern noise component is picked up immediately before a recording still image is picked up and stored in the memory 31.

  When the image signal of the recording still image is input from the AFE 20, the subtraction unit 32 reads out the fixed pattern noise component at the pixel position from the memory 31 in accordance with the pixel position of the input image signal. The fixed pattern noise is removed by subtracting the fixed pattern noise component read from the memory 31 from the image signal input from the AFE 20.

  The selection unit 33 selects the output of the noise removal unit 30. That is, when the noise removing unit 30 performs noise removal from the image signal of the recording still image, the selecting unit 33 selects to output the image signal after the noise removal. In addition, when the image signal of the preview image is input from the AFE 20 to the noise removing unit 30, the selection unit 33 selects to output the image signal input from the AFE 20. Further, even when a fixed pattern noise component imaged immediately before the recording still image is captured by the noise removing unit 30 is input from the AFE 20, it is selected to output the fixed pattern noise component as it is.

  The image correction unit 40 performs image correction such as white balance correction and clamp correction on the preview image and the still image recorded on the recording medium 70. The image correction unit 40 includes a parameter determination unit 41 and a correction unit 42.

  The parameter determination unit 41 determines a correction parameter to be used when performing image correction. The correction parameter is determined differently between the preview image and the still image recorded on the recording medium 70. First, since noise is not removed from the preview image, the correction parameter is determined using a method similar to the conventional method, that is, using the image signal of the preview image captured immediately before. For example, in the white balance correction, the cumulative values of the image signals of the three colors red (R), green (G), and blue (B) included in the preview image captured immediately before are obtained, and the respective cumulative values are obtained. The gains multiplied by R, G, and B are determined so that the values are equal. In the clamp correction, the black level is determined by obtaining the average value of the image signals output from the pixels included in the OPB area of the imaging device 10 among the image signals of the preview image captured immediately before.

  On the other hand, noise is removed from the still image to be recorded on the recording medium 70 by the noise removing unit 40. Therefore, the correction parameters are the preview image taken immediately before taking the still image, and the memory 31 of the noise removing unit 30. The correction parameter is determined using the fixed pattern noise component picked up for storage. For example, in the white balance correction, the magnitudes of the image signals of the R, G, and B3 colors included in the fixed pattern noise component are calculated from the cumulative values of the image signals of the R, G, and B3 colors included in the preview image. A value obtained by subtracting the accumulated value is calculated for each color, and gains to be multiplied by R, G, and B are determined so that the calculated values of the respective colors are equal. In addition, the clamp correction is performed by subtracting the fixed pattern noise component of the corresponding pixel from the image signal output from the pixel included in the OPB area of the imaging device 10 from the image signal of the preview image, and removing the fixed pattern noise. Thus, the black level is determined by obtaining the average value of the image signals after removal of the fixed pattern noise.

  The correction unit 42 performs image correction using the correction parameter determined by the parameter determination unit 41, and outputs an image signal subjected to image correction to the display device 50 and the image compression unit 60.

  The display device 50 displays the preview image corrected by the image correction unit 40 and the recorded still image. The image compression unit 60 performs image compression processing on the image signal corrected by the image correction unit 40 in accordance with an image compression standard such as JPEG (Joint Photographic Coding Experts Group). The recording medium 70 records and stores the image signal compressed by the image compression unit 60.

  Based on such a configuration, the operation of the digital camera 100 shown in FIG. 1 will be described below. When the operation mode of the digital camera 100 is set to the imaging mode, the digital camera 100 continues to capture the preview image with the imaging device 10 until a still image imaging command is input. The captured preview image is converted into a digital signal by the AFE 20 and then subjected to image correction such as white balance correction and clamp correction by the image correction unit 40, so that the display-corrected preview image is displayed on the display device 50. Is done. At this time, the parameter for image correction used in the image correction unit 40 is determined using the image signal of the preview image captured immediately before, as in the conventional case.

  Next, when a still image capturing command is input, the digital camera 100 captures an image by closing the shutter 12 of the image capturing apparatus 10, converts the image signal into a digital signal by the AFE 20, and stores this image signal as a fixed pattern noise component. The data is stored in the memory 31 of the removal unit 30. Further, the image correction unit 40 determines a correction parameter to be used for correcting the image signal of the still image using the fixed pattern noise component and the image signal of the preview image captured immediately before.

  Subsequently, the digital camera 100 opens the shutter 12 of the imaging device 10 again, and the imaging device 10 captures a still image for recording. The captured image signal of the still image is converted into a digital signal by the AFE 20, and then the fixed pattern noise is removed by subtracting the fixed pattern noise previously stored in the memory 31 by the noise removing unit 30. Then, the image signal of the still image from which the fixed pattern noise has been removed is subjected to image correction using the correction parameters previously determined by the image correction unit 40 and displayed on the display device 50. Further, the image corrected image is compressed by the image compression unit 60 and then recorded on the recording medium 70.

  As described above, according to the embodiment of the present invention, when performing image correction after removing noise components from a still image to be recorded on the recording medium 70, when determining parameters used for this image correction, As described above, not only the image signal of the preview image captured immediately before the still image is used, but also the size of the fixed pattern noise component removed by the noise removing unit 30 is considered in addition to the preview image. . As a result, the approximate size of the image signal included in the still image from which the fixed pattern noise has been removed can be predicted from the size of the image signal of the preview image captured immediately before and the size of the fixed pattern noise component. Therefore, it is possible to suppress the influence of fixed pattern noise in determining image correction parameters, and to remove noise components, compared to the conventional case of determining image correction parameters only from the image signal of the preview image. It is possible to suppress image quality degradation when capturing a still image.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  In the above embodiment, an example of a digital camera has been described. However, the present invention is not limited to this, and the present invention is not limited to this. Applicable. For example, the image processing apparatus may be a mobile terminal.

  In the above embodiment, the fixed pattern noise component is imaged and stored in the memory 31 when the still image recording command is input. However, when the digital camera 100 is in the imaging mode, the fixed pattern noise component is captured. A noise component may be imaged and stored in the memory 31. In addition, the noise removing unit 30 has a function for correcting fluctuations in the size of the fixed pattern noise component due to a temperature change of the imaging device 10 when the fixed pattern noise component is imaged and when the fixed pattern noise is removed. You may have.

  In the above embodiment, the fixed pattern noise is removed from the still image to be recorded on the recording medium 70, and the size of the image signal of the preview image captured immediately before and the size of the fixed pattern noise are used. Although an example of determining the image correction parameter for the still image has been shown, the present invention is not limited to this, and when the fixed pattern noise of a certain image is removed by the noise removing unit, the correction parameter for this image is set to the front of this image. An image processing apparatus that is determined from the size of the image signal of the captured image and the size of the fixed pattern noise component is also included in the scope of the present invention.

1 is a configuration diagram of a digital camera 100 according to an embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Lens 12 Shutter 13 Imaging part 30 Noise removal part 31 Memory 32 Subtraction part 40 Image correction part 41 Parameter determination part 42 Correction part 50 Display apparatus 60 Image compression part 70 Recording medium 100 Digital camera

Claims (1)

An imaging device for capturing an image;
A noise removing unit that removes a fixed pattern noise component from an image signal of at least one of images captured by the imaging device;
An image correction unit that performs image correction including white balance correction or clamp correction on an image signal of an image captured by the imaging device;
The fixed pattern noise component is stored in a memory unit included in the noise removing unit as an image signal captured in a state where light is blocked in the imaging device,
In the image correction unit, the fixed pattern noise component in the noise removing unit has been removed, when performing the image correction on the image signal of a still image to be recorded on the recording medium, imaging the still image by the imaging device Subtracting the magnitude of the fixed pattern noise component removed by the noise removal unit from the magnitude of the image signal of the preview image captured before the determination , determine the image correction parameters ,
When performing the image correction on the image signal of the preview image, the size of the image signal of the preview image including the fixed pattern noise component captured immediately before the preview image is An image processing apparatus for determining an image correction parameter .

Images