JP2006311086A - Noise suppressor suppressing image noise by using black image, electronic camera, and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique which does not increase random noise for noise suppression using a black image. <P>SOLUTION: A noise suppressor according to the present invention includes an input section, a black image processing section, and a noise suppression section. The input section inputs image data obtained by imaging a field by an imaging section and a plurality of black image data obtained by imaging the field by the imaging section in a light shield state. The black image processing section generates black image data B by suppressing noncorrelative random noise between the plurality of dark image data. The noise suppression section suppresses noise of the image data based upon black image data B. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a noise suppression device that suppresses image noise using a dark image, an electronic camera, and an image processing program.

In general, when long-time shooting is performed with an electronic camera, fixed pattern noise appears in image data. As a device for removing this type of noise, a conventional device of Patent Document 1 is known.
In this conventional apparatus, first, normally captured image data and dark image data captured with the shutter closed are prepared. The conventional apparatus subtracts the dark image data from the image data in units of pixels, thereby removing the in-phase fixed pattern noise.
JP 2000-125204 A (Claims 1 and 2 etc.)

By the way, the dark image data taken for a long time includes random noise in addition to fixed pattern noise. In the subtraction processing of the conventional device described above, this random noise is reversed in phase and added to the image data, so that the random noise of the image data is increased in reverse.
Therefore, an object of the present invention is to provide a technique in which random noise does not increase in noise suppression using a dark image.

<< 1 >> The noise suppression device of the present invention includes an input unit, a dark image processing unit, and a noise suppression unit.
The input unit captures image data obtained by imaging a light image with the imaging unit and a plurality of dark image data obtained by imaging with the imaging unit in a light-shielded state.
The dark image processing unit generates dark image data B by suppressing non-correlated random noise between a plurality of dark image data.
The noise suppression unit suppresses noise in the image data based on the dark image data B.

<< 2 >> Preferably, the dark image processing unit detects a non-correlated part from level variations among a plurality of dark image data. The dark image processing unit obtains dark image data B by suppressing fluctuations of uncorrelated portions on a composite image of a plurality of dark image data.

<< 3 >> Preferably, the dark image processing unit detects a minute level portion where a plurality of dark image data are all equal to or less than a predetermined threshold. The dark image processing unit obtains dark image data B by suppressing the fluctuation of the minute level portion on the composite image of the plurality of dark image data.

<< 4 >> Preferably, the total charge accumulation time of the plurality of dark image data is set shorter than the charge accumulation time of the image data. In this case, the dark image processing unit applies to the dark image data B a compensation gain that compensates for a reduction in the gain of the fixed pattern noise that accompanies this time reduction.

<< 5 >> The electronic camera of the present invention includes a noise suppression device, an imaging unit, a light shielding mechanism, and a control unit.
The imaging unit performs photoelectric conversion.
The light shielding mechanism has a function of shielding the imaging unit.
The control unit drives the imaging unit to capture an image of the object scene and generate image data. In addition, the control unit drives the imaging unit that is blocked by the light blocking mechanism a plurality of times to generate a plurality of dark image data.
In particular, the noise suppression device is the device according to any one of the above << 1 >> to << 4 >>, and captures image data and a plurality of dark image data to suppress noise in the image data.

<< 6 >> The image processing program of the present invention is a program for causing a computer to function as the input unit, the dark image processing unit, and the noise suppression unit according to any one of the above << 1 >> to << 4 >>. .

  The present invention generates a plurality of dark image data. The tendency of occurrence of fixed pattern noise is similar between a plurality of dark image data, and shows high correlation. On the other hand, random noise is randomly generated between a plurality of dark image data, and thus shows non-correlation. Therefore, by comparing a plurality of dark image data, uncorrelated random noise and correlated fixed pattern noise can be accurately distinguished. The present invention suppresses fixed pattern noise in image data with reference to dark image data B in which random noise is suppressed by this correlation determination. As a result, it is possible to appropriately suppress the fixed pattern noise in the image data while suppressing the adverse effect of an increase in random noise.

<< Description of Configuration of this Embodiment >>
FIG. 1 is a block diagram showing the electronic camera 11.
In FIG. 1, a photographing lens 12 is attached to the electronic camera 11. The lens control unit 12a performs focus driving, aperture control, and the like of the photographing lens 12. In the image space of the photographic lens 12, the light receiving surface of the image sensor 13 is disposed via the shielding mechanism 12b. The shielding mechanism 12b may be used as a mechanical shutter or a diaphragm, or may be a dedicated shielding mechanism. The shielding mechanism 12 b has a function of shielding the light receiving surface of the imaging element 13 by blocking the incident optical path of the photographing lens 12 with a signal from the imaging control unit 14.

On the other hand, the imaging device 13 is driven by the imaging control unit 14. Image data output from the image sensor 13 is processed via the signal processing unit 15 and the A / D conversion unit 16 and then temporarily stored in the memory 17.
This memory 17 is connected to a bus 18. The bus 18 is also connected with a lens control unit 12a, an imaging control unit 14, a microprocessor 19, an image processing unit 20, a recording unit 22, and a monitor display unit 23.
The microprocessor 19 is connected to an operation unit 19a such as a release button. The recording unit 22 is loaded with a recording medium 22a.

<< Explanation of operation of this embodiment >>
FIG. 2 is a flowchart for explaining the operation of the present embodiment.
Hereinafter, the operation will be described along the step numbers shown in FIG.

Step S1: When the microprocessor 19 detects a release instruction from the user using the operation unit 19a or the like, the microprocessor 19 instructs the imaging control unit 14 to start an imaging operation. The imaging control unit 14 causes the imaging device 13 to start photoelectric conversion of the subject image with the shielding mechanism 12b opened, and reads image data from the imaging device 13 after a preset charge accumulation time has elapsed. The image data is digitized via the signal processing unit 15 and the A / D conversion unit 16 and then temporarily stored in the memory 17.

Step S2: The microprocessor 19 determines a generation schedule of a plurality of dark image data corresponding to the charge accumulation time of the image data, and instructs the imaging control unit 14 to generate dark image data.
For example, when generating N (N ≧ 2) dark image data, it is preferable to set 1 / N times the charge accumulation time of the image data as the charge accumulation time for one dark image data.
For example, when N pieces of dark image data are generated, the time reduction rate of the total charge accumulation time of the dark image data is R (0 <R <1). In this case, it is preferable to set R / N times the charge accumulation time of the image data as the charge accumulation time for one dark image data.
For example, when generating N pieces of dark image data, the time extension rate of the total charge accumulation time of the dark image data is set to E (E> 1). In this case, it is preferable to set E / N times the charge accumulation time of the image data as the charge accumulation time for one dark image data.
In addition, for example, it is also preferable that the charge accumulation time of all or part of N dark image data is intentionally shifted. In particular, in this case, the linearity or non-linear relationship between the two can be identified by performing mathematical approximation or regression analysis on the time axis relationship between the different charge accumulation times and the generated fixed pattern noise. . As a result, the fixed pattern noise during the charge accumulation time of the image data can be estimated with higher accuracy.

Step S3: The imaging control unit 14 shields the shielding mechanism 12b and keeps the light receiving surface of the image sensor 13 in the dark state in response to the generation instruction of the dark image data.

Step S4: The imaging control unit 14 drives the dark imaging element 13, and performs charge accumulation and image reading at least twice according to the generation schedule. The plurality of dark image data generated at this time is sequentially processed via the signal processing unit 15 and the A / D conversion unit 16 and then stored in the memory 17.

Step S5: The microprocessor 19 determines the set values L, J, and K by referring to the correspondence relationship stored in advance by the charge accumulation time of the dark image data. The set values L, J, and K are set in the image processing unit 20 as parameters.
This set value L is a threshold value for discriminating a minute level portion where no fixed pattern noise is clearly generated from the dark image data. Such a set value L varies depending on the noise characteristics of the image sensor 13, the charge accumulation time, or the imaging sensitivity. Therefore, it is preferable to obtain in advance experimentally in association with the charge accumulation time of dark image data or imaging sensitivity. As the set value L, it is preferable to adopt the lower value of “(a) Fixed pattern noise lower limit threshold”, “(b) Random noise upper limit threshold”, or (a) (b). In addition, when the charge accumulation times and imaging sensitivities of a plurality of dark image data are different, it is preferable to determine the set value L for each dark image data.
The set value J is a threshold value for discriminating whether or not the signal level is correlated between the dark image data. Such a setting value J varies depending on the noise characteristics of the image sensor 13, the charge accumulation time, or the imaging sensitivity. Therefore, it is preferable to obtain in advance experimentally in association with the charge accumulation time of dark image data or imaging sensitivity. Note that the set value J is a value obtained by statistically calculating the variation width of fixed pattern noise of individual dark image data (when the charge accumulation time is different, the average signal level is adjusted by applying a normalization coefficient) ( It is preferable to employ a value corresponding to the standard deviation σ.
Further, the set value K is a value indicating the virtual signal level when the dark image data B (described later) is not affected by random noise. This set value K is preferably obtained in advance experimentally in association with the total charge accumulation time of a plurality of dark image data or imaging sensitivity. The set value K may be determined by detecting a direct current level or the like from the dark image data obtained in step S4.

Step S6: The image processing unit 20 initializes the scanning position [i, j] of the dark image data to [0,0], and reads the value of the scanning position [i, j] from each of the plurality of dark image data.

Step S7: The image processing unit 20 determines whether or not the values of the scanning positions [i, j] of the plurality of dark image data are all equal to or less than the set value L.
If the scan position [i, j] is equal to or less than the corresponding set value L in all dark image data, the image processing unit 20 determines that the position is a minute level that does not include fixed pattern noise, and operates in step S9. To migrate.
In other cases, the image processing unit 20 shifts the operation to step S8.

Step S8: The image processing unit 20 calculates a level difference (absolute value) of the scanning position [i, j] between a plurality of dark image data. When there are three or more dark image data, the average of the difference between the maximum and minimum values of the scanning position [i, j], the standard deviation of the scanning position [i, j], and the level difference between individual images It is preferable to obtain the level difference and determine the level difference.
Next, the image processing unit 20 determines whether or not the obtained level difference is less than the set value J.
Here, when the level difference is less than the set value J, it can be determined that the scanning position [i, j] is fixed pattern noise because the correlation is high. In this case, the image processing unit 20 shifts the operation to step S10.
On the other hand, when the level difference is equal to or larger than the set value J, it is a non-correlated part, and therefore it can be determined that the scanning position [i, j] is random noise. In this case, the image processing unit 20 shifts the operation to step S9.

When sequentially processing a plurality of pieces of dark image data one by one, it is preferable to unconditionally move to step S10 because there is no comparison target for the first dark image data. In this case, the determination in step S8 is started from the second and subsequent dark image data.
For one or more dark image data, the operation may be shifted from step S7 to step S9 for the scanning position [i, j] that is equal to or less than the set value L without performing the determination in step S8. . Such an operation makes it possible to shorten the processing time.

Step S9: The image processing unit 20 replaces the scanning position [i, j] of the dark image data B (preferably one of the dark image data in the memory 17 in order to save memory capacity) with the set value K. . After this processing, the operation proceeds to step S11.

Step S10: The image processing unit 20 adds the values of the scanning positions [i, j] for a plurality of dark image data, obtains an added value, and stores the added values at the scanning positions [i, j] of the dark image data B.

Step S11: The image processing unit 20 determines whether or not scanning of dark image data is completed. If the scanning is not completed, the operation proceeds to step S12. On the other hand, when the scanning is completed, the operation proceeds to step S13.

Step S12: The image processing unit 20 advances the scanning position [i, j] next. The image processing unit 20 reads the values of the new scanning position [i, j] from the plurality of dark image data. After this processing, the image processing unit 20 returns the operation to step S7.

Step S13: The image processing unit 20 determines the compensation gain of the dark image data B based on the sum S of the charge accumulation time BT of the dark image data and the charge accumulation time PT of the image data. If the level of the fixed pattern noise changes linearly with respect to the charge accumulation time, “PT / S times” may be used as the compensation gain. If nonlinear, the compensation gain corresponding to S and PT (or BT and PT) may be obtained.
The image processing unit 20 applies the obtained compensation gain to the dark image data B, so that the level of the fixed pattern noise is substantially uniform between the dark image data B and the image data.

Step S14: The image processing unit 20 subtracts the dark image data B from the image data in the memory 17 in units of pixels. By this subtraction process, image data in which fixed pattern noise is suppressed is obtained.

<< Effects of this embodiment >>
As described above, in the present embodiment, a plurality of dark image data are generated.
Since the fixed pattern noise is regularly generated, high correlation is exhibited between a plurality of dark image data. On the other hand, since random noise occurs irregularly, it shows non-correlation between a plurality of dark image data. Therefore, as in this embodiment, dark image data with reduced random noise can be obtained by suppressing non-correlated points between a plurality of dark image data. By suppressing the image noise based on the dark image data, it is possible to suppress the fixed pattern noise while suppressing the influence of the random noise.

  Further, in the present embodiment, a minute level portion where a plurality of dark image data are all at a minute level is detected. This minute level portion is a portion that cannot be fixed pattern noise whose level is likely to rise. Such a minute level portion is a portion including only random noise, and can be determined as a point that is harmful and not advantageous in removing fixed pattern noise. Therefore, it is possible to further suppress the influence of random noise by suppressing the level of the minute level portion. As a result, it is possible to suppress fixed pattern noise while suppressing the influence of random noise.

  Since the influence of random noise can be suppressed in this way, it becomes possible to allow random noise in dark image data as compared with the conventional case. Accordingly, it is possible to realize settings such as shortening the total charge accumulation time of dark image data. In this case, the electronic camera 11 can be quickly released from the dark image data imaging sequence, and the electronic camera 11 excellent in usability can be realized.

<< Additional items of embodiment >>
In the above-described embodiment, fixed pattern noise is removed by subtracting dark image data from image data. However, the embodiment is not limited to this.

  For example, the image processing unit 20 evaluates the correlation of a plurality of dark image data, and detects the occurrence of fixed pattern noise having high correlation (or by removing the occurrence of random noise indicating non-correlation). , Detect where fixed pattern noise occurs). Next, the image processing unit 20 performs noise suppression processing with reference to peripheral pixels and the like on the pixel position in the image data corresponding to the occurrence location. In such processing, it becomes possible to accurately discriminate between fixed pattern noise and random noise in dark image data from the difference in correlation. As a result, it is possible to suppress the influence of random noise on the image data also in this process.

In the above-described embodiment, the non-correlated random noise is suppressed by replacing the non-correlated part in the dark image data with the set value K. However, the embodiment is not limited to this.
For example, when three or more dark image data are generated, non-correlated random noise can be suppressed by image composition that performs an intermediate value operation (median) on three or more dark image data.

  Further, for example, when three or more dark image data are generated, uncorrelated random noise can be suppressed by image synthesis such as averaging except pixel values having a large level difference.

  In the above-described embodiment, the image data is generated first, and the dark image data is generated later. However, the embodiment is not limited to this. Some or all of the plurality of dark image data may be generated prior to the image data. Alternatively, some or all of the plurality of dark image data may be generated at a timing independent of image data capture, such as when the power is turned on. In such an operation, it is possible to shorten or omit the dark image data shooting sequence after shooting the image data.

  In the above-described embodiment, the electronic camera 11 (including the noise suppression device) has been described. However, the embodiment is not limited to this. For example, a noise suppression device may be configured independently of the electronic camera. Further, for example, the above-described noise suppression processing (for example, shown in FIG. 2) may be softwareized to create an image processing program. By executing this image processing program on a computer, a noise suppression device can be realized on the computer.

  In the above-described embodiment, the case where the noise suppression process is performed on the data generated by the electronic camera has been described. However, the embodiment is not limited to this. For example, the above-described noise suppression processing can be performed using image data and dark image data created by an image generation device such as a scanner device.

  Further, such a server on the Internet (image album server or the like) may provide a noise suppression service using image data and dark image data transmitted from a user.

  The present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-mentioned embodiment is only a mere illustration in all points, and should not be interpreted limitedly. The scope of the present invention is indicated by the scope of claims, and is not restricted by the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

  As described above, the present invention is a technique that can be used for image noise suppression and the like.

1 is a block diagram showing an electronic camera 11. FIG. It is a flowchart explaining operation | movement of this embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Electronic camera, 12 ... Shooting lens, 12b ... Shielding mechanism, 13 ... Imaging device, 14 ... Imaging control part, 17 ... Memory, 19 ... Microprocessor, 20 ... Image processing part

Claims (6)

An input unit that captures image data obtained by imaging a light image with an imaging unit and a plurality of dark image data obtained by imaging with the imaging unit in a light-shielded state;
A dark image processing unit that generates dark image data B by suppressing non-correlated random noise between the plurality of dark image data;
A noise suppression device comprising: a noise suppression unit configured to suppress noise of the image data based on the dark image data B. The noise suppression device according to claim 1,
The dark image processing unit
A non-correlated part is detected from a level variation between the plurality of dark image data, and the dark image data B is obtained by suppressing fluctuation of the non-correlated part to a composite image of the plurality of dark image data. Noise suppression device. In the noise suppression device according to claim 1 or 2,
The dark image processing unit
The dark image data is detected by detecting a minute level portion in which the plurality of dark image data are all equal to or less than a predetermined threshold value, and suppressing the fluctuation of the minute level portion on a composite image of the plurality of dark image data. What is required is a noise suppression device. In the noise suppression device according to any one of claims 1 to 3,
When the total charge accumulation time of the plurality of dark image data is set to be shorter than the charge accumulation time of the image data,
The noise suppression device, wherein the dark image processing unit applies a compensation gain that compensates for a decrease in gain of fixed pattern noise accompanying the time reduction to the dark image data B. The noise suppression device according to any one of claims 1 to 4,
An imaging unit that performs photoelectric conversion;
A light shielding mechanism for shielding the imaging unit;
A controller that drives the image pickup unit to pick up an image of the object scene to generate image data, and drives the image pickup unit that is blocked by the light blocking mechanism a plurality of times to generate a plurality of dark image data; With
The electronic camera, wherein the noise suppression device captures the image data and the plurality of dark image data to suppress noise of the image data.   An image processing program for causing a computer to function as the input unit, the dark image processing unit, and the noise suppression unit according to any one of claims 1 to 4.

Images