JP6551590B2 - Imaging apparatus and noise correction method - Google Patents

Description

  The present invention relates to an imaging device and a noise correction method for correcting noise included in an image.

  Various methods are known for removing noise from a captured image obtained by capturing. Patent Document 1 first divides an image signal of one frame obtained by capturing a black image into a plurality of blocks, and calculates an average for each block. Then, these are resized to the resolution of the captured image, and the resized signal is subtracted from the captured image to remove noise.

JP 2009-21815

  However, noise is generated due to various causes, and the form differs depending on the cause. Therefore, in order to remove noise from the captured image for each different form, it is assumed that a plurality of noise images are prepared according to the noise and the above-described technique is applied to a method of subtracting these from the captured image. At this time, if a plurality of noise images are resized and then subtracted from the captured image, an error occurs due to the resizing, and noise may not be accurately removed from the captured image. If noise can not be removed correctly, there is a possibility that the user who saw the photographed image may feel discomfort. In addition, if a plurality of noise images are resized, the number of processing times and the processing time increase by the number of times of resizing.

  The present invention has been made in view of these problems, and it is an object of the present invention to provide an imaging device and a noise correction method that appropriately correct noise contained in a captured image while suppressing increases in the number of times of processing and processing time.

  An imaging apparatus according to a first aspect of the present invention includes: a storage unit that stores a plurality of noise images created for each of a plurality of acquisition conditions; an imaging element that captures a subject image using imaging conditions; Adjust each noise image using the acquisition conditions and shooting conditions to create an adjusted noise image, add the adjusted noise images together to create a combined noise image, and convert the resolution of the combined noise image And a noise correction unit that generates a subtraction noise image and corrects a captured image using the subtraction noise image.

  The noise correction unit preferably corrects the captured image by subtracting the subtraction noise image from the captured image.

  The noise correction unit preferably creates a combined noise image by adding together a plurality of adjusted noise images that have been bit-expanded. Errors due to overflow can be minimized.

  The noise correction unit returns the summed noise image to the same number of bits as the plurality of adjusted noise images, converts the resolution of the summed noise image to create a subtraction noise image, and corrects the photographed image using the subtraction noise image. Is preferred.

  It is preferable that the noise correction unit sums a plurality of adjustment noise images to create a summed noise image.

  The noise correction unit preferably converts the resolution of the summed noise image so as to be the same as the resolution of the captured image to create a subtraction noise image.

  A noise correction method according to a second aspect of the present invention comprises the steps of: inputting a photographed image photographed under a predetermined photographing condition; reading a plurality of noise images generated for each of a plurality of acquisition conditions from a storage unit; A unit that adjusts each of the plurality of noise images using an acquisition condition and a shooting condition to create an adjustment noise image; and a noise correction unit adds the plurality of adjustment noise images to obtain a combined noise image. A step of creating, a step in which the noise correction unit converts the resolution of the combined noise image to create a subtraction noise image, and a step in which the noise correction unit corrects the captured image using the subtraction noise image. Features.

  According to the present invention, it is possible to obtain an imaging device and a noise correction method for appropriately correcting noise included in a captured image while suppressing an increase in the number of times of processing and processing time.

1 is a block diagram of an imaging apparatus according to a first embodiment of the present invention. It is the figure which showed the 1st noise image roughly. It is the figure which showed the 2nd noise image roughly. It is the figure which showed the 1st expansion noise image roughly. It is the figure which showed the 2nd expansion noise image roughly. It is the figure which showed the subtraction noise image roughly. It is the figure which showed the noise image obtained by extending | stretching the 1st noise image roughly. It is the figure which showed the noise image obtained by extending | stretching a 2nd noise image roughly. It is the figure which showed roughly the subtraction noise image by a well-known method. It is a figure showing an error between a subtraction noise image according to a known method and a subtraction noise image having a value close to a value to be actually corrected. It is the figure which showed roughly the noise image obtained by adding a 1st noise image and a 2nd noise image. It is the figure which showed roughly the subtraction noise image obtained by extending | stretching a noise image. It is the figure which showed the image by this embodiment roughly. It is the flowchart which showed the 1st correction process. It is the figure which showed the 1st expansion noise image roughly. It is the figure which showed the 2nd expansion noise image roughly. It is the figure which showed roughly the 1st adjustment noise image obtained by multiplying an adjustment coefficient, without carrying out bit expansion | extension. It is the figure which showed roughly the 2nd adjustment noise image obtained by multiplying an adjustment coefficient, without carrying out bit expansion | extension. It is the figure which showed roughly the 1st adjustment noise image obtained by multiplying an adjustment coefficient after bit expansion. It is the figure which showed roughly the 2nd adjustment noise image obtained by multiplying an adjustment coefficient after bit expansion. FIG. 19 is a view schematically showing an image obtained by subtracting the second adjustment noise image shown in FIG. 18 from the first adjustment noise image shown in FIG. 17. FIG. 21 is a view schematically showing an image obtained by subtracting the second adjustment noise image shown in FIG. 20 from the first adjustment noise image shown in FIG. 19. It is the flowchart which showed the 2nd correction process.

  An imaging apparatus and a noise correction method according to an embodiment of the present invention will be described below with reference to the drawings. FIG. 1 shows a digital still camera 10 which is a first embodiment of an imaging device. First, the configuration of the digital still camera 10 will be described with reference to FIG.

  The digital still camera 10 includes a photographing lens 11, an aperture mechanism 12, a mechanical shutter 13, a CMOS 14 as an imaging device, a preprocessing unit 15, a signal processing unit 16 as a noise correction unit, an interface 17, a display unit 18, and an external memory. 19 is mainly provided.

  The photographing lens 11 is composed of a plurality of optical lenses and forms an object image on the CMOS 14. A diaphragm mechanism 12, a mechanical shutter 13, and a CMOS 14 are provided on the optical axis of the photographing lens 11.

  The diaphragm mechanism 12 changes the size of the optical path from the photographing lens 11 in accordance with a signal from the control unit 161 described later.

  The mechanical shutter 13 opens and closes an optical path from the photographing lens 11 in accordance with a signal from a control unit 161 described later.

  The CMOS 14 receives a subject image from the shooting lens 11 and outputs a shot image. The temperature, exposure time, and gain of the CMOS 14 when a photographed image is created are referred to as photographing conditions. Gain is also called photographing sensitivity.

  The preprocessing unit 15 receives a captured image from the CMOS 14 and performs processing such as A / D conversion and gain control.

  The signal processing device 16 includes a control unit 161, a processing unit 162, and an internal memory 163 which is a storage unit.

  The control unit 161 is configured by, for example, an LSI or a microcomputer, and controls operations of the aperture mechanism 12, the mechanical shutter 13, the CMOS 14, the processing unit 162, and the internal memory 163.

  The processing unit 162 is configured by, for example, an LSI, a microcomputer, or the like, removes noise from the captured image received from the preprocessing unit 15, and outputs a corrected image. The process for removing noise is referred to as a first correction process. The first correction process will be described later.

  The internal memory 163 is a semiconductor memory, and temporarily stores captured images, and stores a program for operating the control unit 161 and the processing unit 162, and a noise image described later.

  The corrected image output by the processing unit 162 is transmitted to the display unit 18, the external memory 19, and the external device 20 via the interface 17.

  The display unit 18 is, for example, a liquid crystal display or an organic EL display provided on the back of the digital still camera 10, and displays a corrected image.

  The external memory 19 is a non-volatile memory such as a flash memory, for example, and stores a corrected image.

  The external device 20 is, for example, an external monitor or a computer, and displays or stores the correction image.

  Next, the first correction process will be described with reference to FIGS. The first correction process is a process of correcting noise included in a captured image using a plurality of noise images.

  The CMOS 14 has a plurality of photodiodes arranged in a matrix. Photodiodes arranged in the horizontal direction are called lines (rows). The photodiode converts incident light into electrical energy and accumulates it. By reading this line by line, a captured image is output.

  When an object image is captured using the CMOS 14, noise is generated in the captured image. The main reasons for the generation of noise are as follows. The first is due to dark current generated in each photodiode. The second is due to shading due to characteristic variation of each photodiode and performance error of each circuit provided around the photodiode. The third reason is that the period from the start of charge accumulation to the readout is different for each line. More specifically, since this period is different from line to line, the amount of dark current accumulated differs from line to line, causing a difference in characteristics in the vertical direction. Due to these causes, each CMOS 14 has different fixed pattern noises.

  The noise image is created in advance according to these causes and stored in the internal memory 163. The temperature, exposure time, and gain values of the CMOS 14 when the noise image is created are called acquisition conditions, and are stored in the internal memory 163 in association with the noise image.

  When imaging, the processing unit 162 compares the imaging condition with the acquisition condition to calculate an adjustment coefficient. More specifically, the temperature ratio that is the ratio of the temperature of the shooting condition to the temperature of the acquisition condition, the exposure time ratio that is the ratio of the exposure time of the shooting condition to the exposure time of the acquisition condition, and the gain of the shooting condition and the acquisition condition The gain ratio, which is the ratio to the gain, is determined, and the temperature ratio is multiplied by the exposure time ratio and the gain ratio. The value obtained in this way is used as the adjustment coefficient.

  FIG. 2 schematically shows a first noise image and FIG. 3 schematically shows a second noise image. Here, for simplicity of explanation, it is assumed that the first and second noise images have already been multiplied by the adjustment coefficient. Hereinafter, in FIGS. 2 to 13, small squares indicate pixels of the noise image, and numerals shown inside indicate noises of the pixels.

  FIG. 4 schematically shows a first magnified noise image obtained by expanding the first noise image to the same size as the photographed image using linear interpolation, and FIG. 5 similarly shows the second magnified noise image. 2 shows a second magnified noise image obtained by decompressing the noise image of FIG. In a known technique, noise included in a captured image is corrected by subtracting an image obtained by adding the first enlarged noise image and the second enlarged noise image from the captured image. FIG. 6 shows a subtraction noise image obtained by adding the first enlarged noise image and the second enlarged noise image. The subtracted noise image is an image composed of noise included in the captured image, and is used to correct noise included in the captured image.

  On the other hand, FIG. 7 schematically shows a noise image obtained by expanding the first noise image to the same size as the photographed image using bilinear interpolation, and FIG. 8 similarly shows the second noise. The noise image obtained by extending | stretching an image is shown. The noise images of FIGS. 7 and 8 contain a number after the decimal point, and the rounding error is smaller than the first and second magnified noise images. FIG. 9 shows a subtraction noise image obtained by adding the noise image of FIG. 7 and the noise image of FIG. Since the rounding error of the noise images of FIGS. 7 and 8 is smaller than that of the first and second enlarged noise images, the subtracted noise image of FIG. 9 has a value close to the value to be actually corrected.

  FIG. 10 is an image obtained by subtracting the subtraction noise image shown in FIG. 6 from the subtraction noise image shown in FIG. 9 and having a subtraction noise image by a known method and a value close to a value to be actually corrected. Error. That is, the subtraction noise image according to the known method has an error as shown in FIG.

  FIG. 11 shows a noise image obtained by adding the first noise image and the second noise image according to this embodiment, and FIG. 12 is the same as the photographed image using the linear interpolation method according to this embodiment. FIG. 12 schematically illustrates a subtracted noise image obtained by expanding the noise image of FIG. 11 to size.

  FIG. 13 is an image obtained by subtracting the subtraction noise image shown in FIG. 12 from the subtraction noise image shown in FIG. 9, and a subtraction noise image having a subtraction noise image according to the present embodiment and a value close to a value to be actually corrected. Error. When the image according to the present embodiment shown in FIG. 13 is compared with the image according to the known technique shown in FIG. 10, the image according to the present embodiment shown in FIG. 13 has a smaller error than the image according to the known technique shown in FIG. Have. That is, if the subtraction noise image shown in FIG. 12 is used, the noise contained in the photographed image can be corrected more appropriately than the known method.

  Next, the flow of the first correction process will be described with reference to FIG. The first correction process is a process executed by the signal processing device 16.

  In the first step S141, the signal processing device 16 causes the CMOS 14 to perform imaging.

  In the next step S <b> 142, the signal processing device 16 acquires a captured image from the CMOS 14 via the preprocessing unit 15 and temporarily stores it in the internal memory 163.

  In the next step S143, the processing unit 162 reads the first noise image from the internal memory 163. In the next step S144, the processing unit 162 multiplies the first noise image by the adjustment coefficient to create a first adjusted noise image. That is, in steps S143 and S144, the noise image as reduced is adjusted according to the imaging condition.

  In the next step S145, the processing unit 162 reads the second noise image from the internal memory 163.

  In the next step S146, the processing unit 162 multiplies the second noise image by the adjustment coefficient to create a second adjusted noise image. That is, in steps S145 and S146, the noise image as reduced is adjusted according to the imaging condition.

  In the next step S147, the processing unit 162 adds the first adjustment noise image and the second adjustment noise image to create a combined noise image.

  In the next step S148, the processing unit 162 expands the combined noise image to the same size as the captured image using a linear interpolation method, and creates a subtracted noise image.

  In the next step S149, the processing unit 162 subtracts the subtraction noise image from the captured image to create a corrected noise image.

  In the next step S150, the processing unit 162 performs predetermined image processing on the corrected noise image. Then, the process ends.

  According to the first correction process, even if a plurality of noise images are used, the number of times of resolution conversion and the process of subtracting the subtraction noise image from the photographed image can be one, and thereby the time required for the calculation Can be reduced.

  In addition, calculation error caused by rounding to an integer can be reduced by setting the number of resolution conversion processes to one.

  By correcting the noise contained in the captured image using a plurality of noise images, it is possible to properly correct noise that can not be corrected with one noise image. In other words, noise with different dependency conditions can be corrected appropriately.

  In addition, there is a possibility that the noise is corrected excessively in the correction using one noise image, but even if it is corrected excessively, the noise corrected excessively by using another noise image As a result, it is possible to appropriately correct the noise included in the captured image.

  By creating and storing a noise image in advance, it is possible to save time for creating a noise image and to shorten the shooting time, compared to a method for creating a noise image when shooting. Further, as compared with the configuration in which a plurality of noise images are enlarged, the number of times of enlargement can be reduced to one and the processing time for enlargement can be reduced.

  Next, the second correction process will be described with reference to FIGS. In the second correction process, the bit number of luminance of each pixel is increased and then the adjustment coefficient is multiplied to the noise image to create a summed noise image and then restore the original bit number to create a summed noise image, This is a process for correcting noise included in a captured image.

  If the luminance represented by 8 bits is multiplied by the adjustment coefficient, overflow (digit error) may occur, and the accuracy of the obtained value may be lowered. Therefore, overflow is prevented by temporarily converting the luminance represented by 8 bits into 16 bits and multiplying the value by the adjustment coefficient.

  The luminance of the first noise image shown in FIG. 15 is represented by 8 bits.

  FIG. 15 schematically shows a first noise image represented by 8 bits, and FIG. 16 schematically shows a second noise image represented by 8 bits.

  FIG. 17 shows each value of the first adjustment noise image obtained by multiplying the first noise image by the adjustment coefficient 10. Here, the maximum number of 8 bits is 255. Any value obtained by multiplying the first noise image by the adjustment factor 10 exceeds 255. Therefore, each value of the first adjustment noise image which is 8 bits overflows and takes a value of 255.

  FIG. 18 shows each value of the second adjustment noise image obtained by multiplying the second noise image by the adjustment coefficient 10. As in the first adjusted noise image, each value of the second adjusted noise image overflows and takes a value of 255.

  FIG. 19 shows a first adjusted noise image obtained by expanding the first noise image to 16 bits and then multiplying the adjustment coefficient 10. Each value of the first adjustment noise image takes the correct value without overflowing.

  FIG. 20 shows a second adjusted noise image obtained by expanding the second noise image to 16 bits and then multiplying the adjustment coefficient 10. Similar to the first adjusted noise image, each value of the second adjusted noise image takes the correct value without overflowing.

  FIG. 21 shows an image obtained by subtracting the second adjustment noise image shown in FIG. 18 from the first adjustment noise image shown in FIG. 17 and then returning it to 8 bits. Each value becomes 0 due to overflow.

  FIG. 22 shows an image obtained by subtracting the second adjustment noise image shown in FIG. 20 from the first adjustment noise image shown in FIG. 19 and then returning it to 8 bits. Since each value does not overflow, each value takes the correct value.

  Next, the flow of the second correction process will be described with reference to FIG. The second correction process is a process executed by the signal processing device 16.

  In the first step S231, the signal processing device 16 causes the CMOS 14 to perform imaging.

  In the next step S232, the signal processing device 16 acquires a captured image from the CMOS 14 via the preprocessing unit 15, and temporarily stores the captured image in the internal memory 163.

  In the next step S233, the processing unit 162 reads the first noise image from the internal memory 163. In the next step S234, the processing unit 162 decompresses each value of the first noise image to 16 bits, and then multiplies the first noise image by the adjustment coefficient to create a first adjusted noise image. That is, in steps S233 and S234, the noise image as reduced is adjusted according to the imaging condition.

  In the next step S235, the processing unit 162 reads the second noise image from the internal memory 163.

  In the next step S236, the processing unit 162 decompresses each value of the second noise image to 16 bits, and then multiplies the second noise image by the adjustment coefficient to create a second adjusted noise image. That is, in steps S235 and S236, the noise image as reduced is adjusted according to the imaging condition.

  In the next step S237, the processing unit 162 adds the first adjustment noise image and the second adjustment noise image, and then returns each value to 8 bits to create a combined noise image.

  In the next step S238, the processing unit 162 expands the combined noise image to the same size as the captured image using a linear interpolation method, and creates a subtracted noise image.

  In the next step S239, the processing unit 162 subtracts the subtracted noise image from the captured image to create a corrected noise image.

  In the next step S240, the processing unit 162 performs predetermined image processing on the corrected noise image. Then, the process ends.

  According to the second correction process, since an overflow that occurs by multiplying the adjustment coefficient can be prevented, a subtraction noise image can be created with high accuracy. Thereby, the noise contained in a picked-up image can be corrected appropriately.

  In addition, by performing bit expansion on the compressed first and second noise images, it is possible to perform adjustment coefficient multiplication with a smaller amount of data compared to the case where the compressed image is bit expanded.

  According to this embodiment, it is possible to appropriately correct the noise included in the photographed image.

  For the sake of simplicity of explanation, the entire pixel data has been described as an object, but an acquisition condition may be set for each line and stored in the storage unit to obtain an adjustment coefficient. In this case, the lines may be horizontal or vertical.

  The conditions included in the imaging conditions and the acquisition conditions are not limited to the temperature, exposure time, and gain of the CMOS 14. It is also possible to use other elements that cause noise.

  The photographing lens 11 may be provided so as to be removable.

  The imaging device is not limited to the CMOS 14 and may be a CCD.

10 digital still camera 11 shooting lens 12 aperture mechanism 13 mechanical shutter 14 CMOS
15 Preprocessing unit 16 Signal processing device 17 Interface 18 Display unit 19 External memory 20 External device 161 Control unit 162 Processing unit 163 Internal memory

Claims (6)

A storage unit that stores a plurality of noise images created for each of a plurality of acquisition conditions;
An imaging element that captures a subject image using shooting conditions and outputs the shot image;
Adjusting each of the plurality of noise images using the acquisition condition and the imaging condition to create an adjusted noise image;
Add the multiple adjustment noise images to create a combined noise image,
Converting the resolution of the summed noise image to create a subtracted noise image;
A noise correction unit that corrects the photographed image using the subtraction noise image.   The imaging device according to claim 1, wherein the noise correction unit corrects the photographed image by subtracting the subtraction noise image from the photographed image.   The imaging apparatus according to claim 1, wherein the noise correction unit adds the plurality of bit-decompressed adjustment noise images to create the summed noise image. The noise correction unit returns the summed noise image to the same number of bits as the plurality of adjusted noise images.
Converting the resolution of the summed noise image to create the subtraction noise image;
The imaging device according to claim 3, wherein the photographed image is corrected using the subtraction noise image.   5. The imaging device according to claim 1, wherein the noise correction unit converts the resolution of the combined noise image to be the same as the resolution of the captured image and creates a subtraction noise image. Inputting a photographed image photographed under a predetermined photographing condition;
Reading out a plurality of noise images created for each of a plurality of acquisition conditions from the storage unit;
A noise correction unit adjusts each of a plurality of noise images using the acquisition condition and the shooting condition to create an adjusted noise image;
The noise correction unit adds together a plurality of adjustment noise images to create a combined noise image;
A noise correction unit converting the resolution of the summed noise image to create a subtraction noise image;
And a noise correction unit correcting the photographed image using the subtraction noise image.

Images