JP4946417B2 - IMAGING DEVICE, IMAGE PROCESSING DEVICE, IMAGE PROCESSING METHOD, PROGRAM FOR IMAGE PROCESSING METHOD, AND RECORDING MEDIUM CONTAINING PROGRAM FOR IMAGE PROCESSING METHOD - Google Patents

Description

  The present invention relates to an imaging apparatus, an image processing apparatus, an image processing method, a program for the image processing method, and a recording medium on which the program for the image processing method is recorded, and can be applied to, for example, a digital still camera. According to the present invention, the tap coefficient of the conditional average filter is set to a value of 0 at a predetermined location, and high-frequency aliasing is suppressed by a low-pass filter, so that noise can be sufficiently suppressed by a simple process and configuration. To.

  2. Description of the Related Art Conventionally, in a digital still camera, RAW data sequentially obtained by a pixel array of an image sensor is demosaiced to generate a full color image, and then converted into image data of luminance signals and color difference signals and recorded on a recording medium. In such image processing of a digital still camera or the like, noise is suppressed using an ε filter, which is a conditional average filter, as disclosed in, for example, Japanese Patent Application Laid-Open No. 2004-172726.

  By the way, with a digital still camera, there are cases where shooting is performed at an extremely fast shutter speed, and in such shooting, the S / N ratio of the imaging result may be significantly degraded. In a digital still camera, it is desired to efficiently suppress noise even for such an imaging result having a significantly deteriorated S / N ratio. For this purpose, for example, when the noise is suppressed using an ε filter, the ε It is necessary to improve the performance of the filter.

Here, in the ε filter, if the number of taps is increased to increase the number of pixels used in the averaging process, the noise suppression performance can be improved. However, increasing the number of taps complicates the ε filter configuration. Further, when the ε filter is configured by arithmetic processing, the arithmetic processing amount is remarkably increased, and processing takes time.
JP 2004-172726 A

  The present invention has been made in consideration of the above points, and an imaging apparatus, an image processing apparatus, an image processing method, an image processing method program, and an image processing method capable of sufficiently suppressing noise by a simple process and configuration The present invention intends to propose a recording medium on which the program is recorded.

  In order to solve the above-described problem, the invention of claim 1 is applied to an imaging device, acquires an imaging result and outputs image data, and outputs output data while suppressing noise in the image data. A noise suppression unit, wherein the noise suppression unit band-limits the image data, and two-dimensionally outputs the output data by band-limiting the image data output from the low-pass filter. The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at a predetermined position in the horizontal direction and the vertical direction, and the tap coefficient is set to a value of 0. The aliasing of the area is suppressed by the low-pass filter.

  The invention of claim 11 is applied to an image processing apparatus that outputs output data by suppressing noise in the image data, and is output from the two-dimensional low-pass filter that limits the band of the image data and the low-pass filter. A two-dimensional conditional average filter that limits the band of image data to output the output data, and the tap coefficient of the two-dimensional conditional average filter has a value of 0 at predetermined locations in the horizontal and vertical directions. The high-frequency aliasing caused by setting the tap coefficient to 0 is suppressed by the low-pass filter.

  The invention of claim 12 is applied to an image processing method for outputting output data by suppressing noise in image data, and a first band limiting step for band-limiting the image data and outputting a low-frequency component. And a second band limiting step of band-limiting the low-frequency component and outputting the output data, and band limitation by the second band limiting step is performed by a two-dimensional conditional average filter The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at a predetermined position in the horizontal direction and the vertical direction, and high-frequency aliasing due to the tap coefficient being set to a value of 0, Suppression is performed in the first band limiting step.

  The invention of claim 13 is applied to a program of an image processing method for outputting output data by suppressing noise of image data, and a first band limitation for band-limiting the image data and outputting a low-frequency component. And a second band limiting step for band-limiting the low-frequency component and outputting the output data, and the band limitation by the second band limitation step is a two-dimensional conditional average. Band limiting by a filter, the tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at a predetermined location in the horizontal direction and the vertical direction, and the high frequency aliasing by setting the tap coefficient to a value of 0 Is suppressed in the first band limiting step.

  The invention of claim 14 is applied to a recording medium on which a program of an image processing method for outputting output data by suppressing noise of image data is recorded, and the program of the image processing method limits the band of the image data. A first band limiting step for outputting a low frequency component and a second band limiting step for limiting the band of the low frequency component and outputting the output data. The band limitation by this step is the band limitation by the two-dimensional conditional average filter, and the tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at a predetermined position in the horizontal direction and the vertical direction. High frequency aliasing due to the coefficient set to 0 is suppressed in the first band limiting step.

  According to the configuration of claim 1, claim 11, claim 13, claim 13, or claim 14, for taps whose tap coefficient is set to 0, calculation can be omitted. Can be simplified. Further, high-frequency aliasing that occurs when the tap coefficient is set to 0 is suppressed by a low-pass filter, so that aliasing is not sufficiently noticeable in practice, and noise can be sufficiently suppressed.

  According to the present invention, noise can be sufficiently suppressed by a simple process and configuration.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

(1) Configuration of Embodiment FIG. 2 is a block diagram showing a digital still camera of an embodiment of the present invention. In the digital still camera 1, the image sensor 3 includes a CCD (Charge Coupled Device) solid-state image sensor, a CMOS (Complementary Metal-Oxide Semiconductor) solid-state image sensor, and the like. The image pickup device 3 photoelectrically converts an optical image formed on the image pickup surface by a lens unit (not shown), and sequentially outputs red, blue, and green pixel values in an order corresponding to the Bayer array.

  The preprocessing unit 4 performs correlated double sampling processing, automatic gain adjustment processing, and analog-digital conversion processing on the output signal of the image sensor 3, and outputs RAW data D1.

  The optical correction unit 5 is constituted by, for example, a digital signal processor. The optical correction unit 5 performs defect correction processing, white balance adjustment processing, and noise suppression processing on the RAW data D1 output from the preprocessing unit 4, and outputs RAW data D2. In this noise suppression process, the optical correction unit 5 switches the characteristics of the noise suppression process step by step under the control of a controller (not shown) so that the amount of noise suppression increases as the shutter speed increases.

  The image processing unit 6 performs demosaic processing, resolution conversion processing, gamma correction processing, image quality correction processing, and the like on the RAW data D2, converts the data into luminance signal and color difference signal image data D3, and outputs the image data. The digital still camera 1 displays the image data D3 processed by the image processing unit 6 on a display device (not shown) and displays a monitor image of the imaging result.

  The encoder (ENC) 7 compresses and outputs the image data D3 output from the image processing unit 6 by a still image encoding method such as JPEG (Joint Photographic Coding Experts Group).

  The interface (IF) 8 records the output data of the encoder 7 or the RAW data D2 output from the optical correction unit 5 on the recording medium 9. The recording medium 9 is a memory card in this embodiment, and records various data output from the interface 8. The recording medium 9 is not limited to a memory card, and various recording media such as an optical disk and a magnetic disk can be widely applied.

  FIG. 1 is a block diagram showing a configuration relating to noise removal processing of the optical correction unit 5. The optical correction unit 5 inputs the RAW data D1 output from the preprocessing unit 4 via the line buffer 11.

  The peripheral pixel reference unit 12 is composed of a series circuit of a plurality of line buffers each capable of outputting the image data D1, and sequentially inputs and transfers the RAW data D1 output from the buffer 11, while transferring from the plurality of line buffers. By simultaneously outputting in parallel, the target pixel is sequentially set on the mosaic image constituted by the RAW data D1, and the image data of the target pixel is output together with the image data of the peripheral pixels.

  The high-pass filter 13 band-limits the RAW data output from the surrounding pixel reference unit 12 and selectively outputs a high-frequency component. Here, FIG. 3 shows the frequency characteristics of the high-pass filter 13.

  The high-frequency noise removing unit 14 is constituted by a two-dimensional median filter or the like, and suppresses noise from the output data of the high-pass filter 13 and outputs it.

  The low-pass filter 15 band-limits the RAW data output from the surrounding pixel reference unit 12 and selectively outputs a low-frequency component. Further, the tap coefficient is switched under the control of the tap coefficient switching unit 17 and the RAW data output from the surrounding pixel reference unit 12 is output without being processed. FIG. 4 shows the characteristics of the low-pass filter 15 when a low-frequency component is extracted from the RAW data.

  The low-frequency noise removing unit 16 is configured by, for example, a two-dimensional ε filter having a predetermined number of taps, and suppresses noise from the output data of the low-pass filter 15 and outputs the result.

  The tap coefficient switching unit 17 switches the tap coefficients of the low-pass filter 15 and the low-frequency noise removing unit 16 under the control of a controller (not shown), and switches the noise suppression characteristics of the low-frequency component.

  The image synthesizing unit 18 adds the output data of the high-frequency noise removing unit 14 and the low-frequency noise removing unit 16, and outputs RAW data D2 in which noise is suppressed.

  FIG. 5 is a flowchart showing a processing procedure of the optical correction unit 5 having the configuration shown in FIG. Here, width and height are the numbers of pixels in the horizontal and vertical directions of the RAW data D1 to be processed, and i and j are the vertical and horizontal positions of the pixel of interest, respectively, with height and width. It is a variable indicated by contrast.

  When this processing procedure is started, the optical correction unit 5 proceeds from step SP1 to step SP2, and initializes a variable i indicating the position in the vertical direction to a value of zero. In the subsequent step SP3, the variable j indicating the horizontal position is initialized to the value 0. Subsequently, in step SP4, the optical correction unit 5 reads out the RAW data D1 of the target pixel specified by the variables i and j together with the RAW data D1 of the peripheral pixels and inputs the read data to the high-pass filter 13. The high-pass filter 13 extracts the high-frequency component by band-limiting the RAW data D1, and the high-frequency noise removing unit 14 suppresses the noise of the high-frequency component in the subsequent step SP5.

  Subsequently, in step SP6, the optical correction unit 5 reads out the RAW data D1 of the target pixel specified by the variables i and j together with the RAW data D1 of the peripheral pixels and inputs the read data to the low-pass filter 15. Further, the low-pass filter 15 limits the band of the RAW data D1 to extract a low-frequency component, and in the subsequent step SP7, the low-frequency noise removing unit 16 suppresses the noise of the low-frequency component. Subsequently, in step SP8, the optical correcting unit 5 adds and outputs the output data of the high frequency noise removing unit 14 and the low frequency noise removing unit 16 in the image synthesizing unit 18, and in step SP9, the horizontal position is determined. The indicated variable j is incremented by a value 1, and in the subsequent step SP10, the variable j is compared with the number of pixels in the horizontal direction width to determine whether or not the processing for one line has not been completed.

  If a positive result is obtained here, the optical correction unit 5 returns to step SP4 and repeats the process for the subsequent pixel of interest in the horizontal direction. On the other hand, if a negative result is obtained in step SP10, the optical correction unit 5 proceeds to step SP11, increments the variable i indicating the position in the vertical direction by a value of 1, and then sets the variable i in the vertical direction in step SP12. It is determined whether or not the processing for one screen has not been completed yet.

  If an affirmative result is obtained here, the optical correction unit 5 returns to step SP3 and starts processing of the subsequent line. On the other hand, if a negative result is obtained in step SP12, the process proceeds to step SP13 to complete the noise suppression process.

  Here, FIG. 6 is a characteristic curve diagram for explaining the low-frequency noise removing unit 16. In FIG. 6, symbol L1 is a frequency characteristic of a one-dimensional low-pass filter, and symbol L2 is a frequency characteristic of a one-dimensional ε filter at a minute amplitude. Here, the low-pass filter is a 3-tap filter having tap coefficients of [1, 2, 1]. The ε filter is a 13-tap filter with all tap coefficients having a value of 1. In this ε filter, a NULL point is generated for each predetermined frequency as the frequency increases, and the attenuation rate gradually increases as the frequency increases. Symbol L1 + L2 is a frequency characteristic of a series circuit of the low-pass filter and the ε filter. In this case, the frequency characteristic of the ε filter is further suppressed on the high frequency side due to the frequency characteristic of the low-pass filter.

  On the other hand, symbol L3 in FIG. 7 is a frequency characteristic when the tap coefficients of the ε filter shown in FIG. 6 are alternately set to a value 1 and a value 0. Therefore, in the example of the code L3, the ε filter has tap coefficients [1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1]. In the case of this ε filter, as compared with the case of the ε filter shown in FIG. 6, the attenuation rate decreases on the high frequency side, the gain increases at the frequency Fs / 2, and high frequency aliasing occurs. However, when connected in series with a one-dimensional low-pass filter having a tap coefficient [1, 2, 1] indicated by the symbol L1, the high-frequency side frequency characteristic is suppressed by the frequency characteristic of the low-pass filter, as indicated by the symbol L1 + L3. As a result, the high-frequency aliasing becomes inconspicuous. Actually, when the gain on the high frequency side becomes 0.2 or less, the noise on the high frequency side becomes inconspicuous, and the noise can be sufficiently suppressed. Even if the aliasing cannot be completely prevented, the image quality degradation due to noise becomes relatively conspicuous in the image with degraded S / N ratio due to the extremely high shutter speed. Can be used.

  Here, in the ε filter, when the tap coefficients are alternately set to the value 1 and the value 0, the calculation amount is 7/13 (= 53 [%]) as compared with the case where all the tap coefficients are set to the value 1. The amount of calculation can be greatly reduced. If the ε filter having the configuration shown in FIG. 7 is applied two-dimensionally, the amount of calculation is (7/13) × (7/13) (= 29 [ %]).

  However, when the ε filter having the configuration shown in FIG. 7 is simply applied two-dimensionally, the pattern near the NULL point is affected, and the image quality of the stripe pattern is deteriorated. Further, the tap coefficient of a specific pixel is continuously set to a value 0 within a certain area, and this pixel information is not reflected in the noise removal processing within this certain area, which is not desirable. Therefore, as shown in FIG. 8, the characteristics of a two-dimensional ε filter having different characteristics in the vertical direction and the horizontal direction are considered. Here, the two-dimensional ε filter shown in FIG. 8 is set such that the arrangement of all tap coefficients of 0 is different in the vertical direction and the horizontal direction, and the vertical direction and the horizontal direction are set to different frequency characteristics. It is a filter. Specifically, in the ε filter, tap coefficients in the third, sixth, eighth, and eleventh columns from the left in the horizontal direction are all set to 0, and the third row from the top in the vertical direction. The tap coefficients of the 5th, 6th, 8th, 9th and 11th lines are all set to 0.

  Here, when the tap coefficients in the horizontal direction and the vertical direction shown in FIG. 8 are added in the horizontal direction and the vertical direction, respectively, [3, 2, 0, 2, 3, 0, 5, 0, 3, 2, 0, 2, 3] are obtained, and [3, 2, 0, 5, 0, 0, 5] as indicated by the symbol F2 by addition in the horizontal direction. , 0, 0, 5, 0, 2, 3] are obtained. In the following, the tap coefficients of the codes F1 and F2 are referred to as the total tap coefficients in the vertical direction and the horizontal direction, respectively. This ε filter can consider the frequency characteristics in the horizontal direction and the vertical direction based on the total tap coefficients in the horizontal direction and the vertical direction.

  Here, a symbol L4 in FIG. 9 is a frequency characteristic by the total tap coefficient in the vertical direction. In this case, although the aliasing becomes large near FS / 4, as shown by reference numeral L1 + L4, when connected in series with a one-dimensional low-pass filter having tap coefficients [1, 2, 1], the frequency characteristics of this low-pass filter As a result, the frequency characteristic on the high frequency side is suppressed, and the aliasing of the high frequency becomes inconspicuous. Also in the example of FIG. 9, the gain on the high frequency side is 0.2 or less.

  Moreover, the code | symbol L5 of FIG. 10 is a frequency characteristic by the total tap coefficient of a horizontal direction. In this case as well, although the aliasing increases near FS / 4, the frequency of this low-pass filter is obtained when it is connected in series with a one-dimensional low-pass filter with tap coefficients [1, 2 and 1] as indicated by reference numeral L1 + L5. The frequency characteristic on the high frequency side is suppressed by the characteristics, and the aliasing of the high frequency becomes inconspicuous. Also in the example of FIG. 9, the gain on the high frequency side is 0.2 or less.

  Here, in the ε filter having the characteristic shown in FIG. 8, the amount of calculation is 25/169 (= 15 [%]) as compared with the 13 pixel × 13 pixel ε filter in which all tap coefficients are set to a value of 1. Can be reduced.

  In practice, in the ε filter having the characteristics shown in FIG. 8, the noise suppression performance is determined at the lowest-order NULL point frequency, and the lower-frequency noise can be suppressed as the NULL point frequency is lower. At higher frequencies than the NULL point, the gain increases due to aliasing. However, if the increase in gain due to aliasing can be suppressed to 0.2 or less due to the overall characteristics of the low-pass filter, the gain can be sufficiently put into practical use.

  Therefore, the tap coefficient switching unit 17 divides the shutter speed into three stages, and sets the tap coefficient of the low-pass filter 15 so as to have a flat frequency characteristic in the first section with the slowest shutter speed. Further, the ε filter of the low-frequency noise removing unit 16 is set so that the tap coefficient is a value 1 only in a certain range centered on the pixel of interest and the remaining tap coefficients are a value 0. Therefore, in this case, the optical correction unit 5 simply suppresses the noise only with the ε filter, and synthesizes the high frequency component with the noise suppressed with the output data of the ε filter and outputs the RAW data D2.

  On the other hand, the tap coefficient switching unit 17 taps the low-pass filter 15 so that the tap coefficient of the pixel of interest has a value 2 and the other tap coefficients have a value 1 in other shutter speed categories. Set. Therefore, in this case, the low-pass filter 15 has the frequency characteristics shown in FIG. 4 and the frequency characteristics shown in FIG. 6, FIG. 7, FIG. 9, and FIG. Extract band components.

  Further, the tap coefficient switching unit 17 converts the tap coefficients of the arrangement shown in FIG. 8 and the arrangement shown in FIG. 8 into 90 degrees in the second section on the slow shutter speed side among the other shutter speed classifications. The tap coefficient of the ε filter is switched for each predetermined pixel and / or for each predetermined line with the tap coefficients of the rotated arrangement. That is, in this case, the optical correction unit 5 simplifies the process by setting the tap coefficient of the ε filter to a value of 0 at a predetermined location so as to limit the band of the low frequency component extracted by the low pass filter with the ε filter. Further, the high frequency aliasing caused by setting the tap coefficient to 0 is suppressed by the low pass filter, and the gain is set to 0.2 or less. Further, the tap coefficient of the ε filter is set to be different between the vertical direction and the horizontal direction so that the frequency of the NULL point varies depending on the frequency characteristic in the horizontal direction and the frequency characteristic in the vertical direction. The tap coefficient is switched between the horizontal direction and the vertical direction for each predetermined pixel and / or for each predetermined line so as to change.

  Therefore, in this case, for example, when the low-frequency component is band-limited for the red pixel R, the ε filter, as shown in FIG. 11, out of the 13 pixels × 13 pixels of the red pixel R that is continuous in the horizontal direction and the vertical direction. Only the RAW data input to the tap whose tap coefficient is set to the value 1 is arithmetically processed to suppress noise. Note that the example of FIG. 11 is a case where the band is limited by the characteristics shown in FIG. 8, and pixels that are not used in the arithmetic processing that are pixels corresponding to the tap coefficient set to the value 0 are indicated by hatching. A target pixel is indicated by a symbol Rc.

  Further, the tap coefficient switching unit 17 further increases the horizontal and vertical tap coefficients in the third section on the fast shutter speed side among the other shutter speed sections as compared with the case shown in FIG. And the tap coefficient of the ε filter is set as in the second section. Therefore, in this case, as in the case of the second section, the optical correction unit 5 sets the horizontal and vertical tap coefficients to a value of 0 at a predetermined location to simplify the process, and sets the tap coefficient to a value of 0. The high-frequency aliasing caused by this is suppressed by the low-pass filter, and the gain is set to 0.2 or less. Further, the tap coefficient of the ε filter is set to be different between the vertical direction and the horizontal direction so that the frequency of the NULL point varies depending on the frequency characteristic in the horizontal direction and the frequency characteristic in the vertical direction. The tap coefficient is switched between the horizontal direction and the vertical direction for each predetermined pixel and / or for each predetermined line so as to change.

  In these first to third divisions, the ε filter sets the threshold value for holding the edge information to a constant value without any change. You may make it change with.

(2) Operation of Embodiment In the above configuration, in the digital still camera 1 (FIG. 2), the imaging signal output from the imaging device 3 is subjected to analog-digital conversion processing by the preprocessing unit 4 to generate RAW data D1. The noise of the RAW data D1 is suppressed by the optical correction unit 5. In the subsequent image processing unit 6, the RAW data D <b> 2 is demosaiced to generate full-color image data. The full-color image data is converted into image data D <b> 3 using a luminance signal and a color difference signal, and then the encoder 7 performs data compression. And recorded on the recording medium 9. When the user instructs recording using RAW data, the RAW data D <b> 2 output from the optical correction unit 5 is recorded on the recording medium 9.

  In the noise removal processing in the optical correction unit 5 (FIGS. 1 and 5), the RAW data D1 is input to the high-pass filter 13 and the low-pass filter 15 via the line buffer 11 and the peripheral pixel reference unit 12. Here, they are separated into a high frequency component and a low frequency component, respectively (FIGS. 3 and 4). These high frequency components and low frequency components are respectively input to the high frequency noise removing unit 14 and the low frequency noise removing unit 16 to suppress noise, and then synthesized by the image synthesis unit 18 to generate RAW data D2. . Therefore, in this digital still camera 1, the image data based on the RAW data D1 is band-separated into a high-frequency component and a low-frequency component, and noise suppression is performed, compared to a case where noise suppression is performed without any band separation. Therefore, it is possible to efficiently suppress noise by reducing image quality degradation.

  Here, in the suppression of noise on the low frequency side in the optical correction unit 5, when the RAW data D 1 having a sufficient S / N ratio is obtained with a low shutter speed, the RAW data D 1 is generated by the low-pass filter 15 at any band. Without being limited, the low-frequency component is extracted by the conditional average filter provided in the subsequent low-frequency noise removing unit 16, and noise suppression processing is further performed. More specifically, in this conditional average filter, a tap coefficient in a certain range from the center is set to a value 1, and a tap coefficient outside this certain range is set to a value 0. Therefore, in this case, the low-frequency component noise is suppressed in the RAW data D1 only by a partial tap of the conditional average filter.

  However, when the shutter speed increases, the S / N ratio of the RAW data D1 deteriorates. Even if all the tap coefficients of the conditional average filter are set to a value 1, noise suppression is performed only by the conditional average filter. In this case, noise cannot be sufficiently suppressed. In this case, it may be possible to improve the noise suppression performance by increasing the number of taps of the conditional average filter and increasing the number of pixels used for the averaging process. The processing of the weighted average filter becomes complicated.

  Therefore, in this embodiment, when the shutter speed becomes faster than a certain value, the tap coefficient of the conditional average filter is set to 0 at a predetermined location (FIG. 8). Since the taps set to the value 0 do not need to be subjected to arithmetic processing, all tap coefficients other than the value 0 are set when the tap coefficients of the conditional average filter are set to the value 0 at a predetermined location. As compared with the case of setting to, the processing of the conditional average filter can be simplified. Further, for such a tap with a value of 0, the configuration of the conditional average filter can be partially omitted to simplify the configuration. Further, taps whose coefficients are set to 0 can be distributed to tap coefficients other than 0 and can be averaged over a wide range with a conditional average filter having a small number of taps. Therefore, the configuration and processing of the digital still camera 1 can be simplified.

  Further, the high-frequency aliasing that occurs when the tap coefficient is set to a value of 0 at a predetermined location is suppressed by the low-pass filter 15 in the previous stage, and the gain is set to be 0.2 or less. As a result, the aliasing noise is made inconspicuous enough for practical use (FIGS. 6, 7, 9, and 10).

  In practice, the case where it is necessary to increase the noise suppression performance is the case of processing RAW data with a fast signal speed and a small signal level. In this case, the RAW data is an image having a relatively deteriorated S / N ratio with a relatively deteriorated image quality as compared with the case where the shutter speed is slow, and has a characteristic that the aliasing is not noticeable. Accordingly, if the aliasing gain is set to 0.2 or less, noise can be sufficiently suppressed practically.

  In this embodiment, when the shutter speed is further increased and the S / N ratio of the RAW data is further deteriorated, similarly, the tap coefficient of the conditional average filter is set to 0 at a predetermined location, Is set so that the number of taps in the conditional average filter is increased, and noise in the RAW data is further suppressed.

  In this way, when the tap coefficient of the conditional average filter is set to a value of 0 at a predetermined location, the conditional average filter is configured so that the frequency at the NULL point is different between the horizontal frequency characteristic and the vertical frequency characteristic. , The tap coefficient is set. That is, since the gain is locally reduced in the vicinity of the frequency at the NULL point, the signal level in the output data is reduced as compared with other frequency components. Therefore, when a repeated pattern corresponding to the frequency of the NULL point is photographed, the repeated pattern is blurred and photographed.

  However, if the frequency of the NULL point is set to be different between the horizontal direction and the vertical direction, the pitch of the repetitive pattern to be photographed can be made different between the horizontal direction and the vertical direction, and this blur is perceived. It is difficult to improve the image quality.

  Such image quality deterioration becomes remarkable when the pattern corresponding to the frequency of the NULL point is a striped pattern extending long in the vertical and horizontal directions. Therefore, the optical correction unit 5 switches the tap coefficient of the conditional average filter for each predetermined pixel and / or for each predetermined line in the horizontal direction and the vertical direction, and for each predetermined pixel and every predetermined line for switching the tap coefficient. The frequency of the NULL point in the horizontal direction and the vertical direction is set to change. Therefore, even a striped pattern with a predetermined pitch can be prevented from being blurred, and image quality deterioration can be effectively avoided.

  In addition, the configuration of storing the tap coefficient is simplified by executing the change in the frequency of the NULL point in the horizontal direction and the vertical direction by switching the tap coefficient of the conditional average filter between the horizontal direction and the vertical direction. be able to.

(3) Effects of the embodiment According to the above configuration, the tap coefficient of the conditional average filter is set to a value of 0 at a predetermined location, and the high-frequency aliasing is suppressed by the low-pass filter, thereby simplifying processing and configuration. Noise can be sufficiently suppressed.

  Also, in this case, the image data to be processed is band-separated to suppress noise, reducing the image quality degradation and suppressing noise more efficiently than when noise is suppressed without any band separation. can do.

  In this case, the noise can be suppressed with a simple configuration by applying the ε filter to the conditional average filter to suppress the noise.

  Further, by making the tap coefficient of the two-dimensional conditional average filter different between the vertical direction and the horizontal direction so that the frequency of the NULL point is different between the frequency characteristic in the horizontal direction and the frequency characteristic in the vertical direction, Blurred display can be prevented.

  Further, by switching the tap coefficient of the conditional average filter for each predetermined pixel and / or for each predetermined line so that the frequency of the NULL point changes, it is possible to prevent a blurred display of a striped pattern with a specific pitch. .

  In the digital still camera of this embodiment, the tap coefficient is set to 0 for the defective pixel, except for the case where the target pixel is a defective pixel, so that the noise removal processing is not performed using the output value of the defective pixel. The tap coefficient in the low-frequency noise removal unit 16 is switched as appropriate. This embodiment has the same configuration as that of the first embodiment except for the configuration related to the tap coefficient switching.

  That is, in this embodiment, a defective pixel is detected in an inspection process at the time of factory shipment, and position information of the defective pixel is stored in a memory (not shown). The tap coefficient switching unit 17 switches tap coefficients in the low-pass filter 15 and the low-frequency noise removing unit 16 based on the position information. Here, the switching of the tap coefficients in the low-frequency noise removing unit 16 is the switching between the horizontal direction and the vertical direction described above with respect to the first embodiment. The tap coefficient switching in the low-pass filter 15 is executed by assigning a tap coefficient having a value of 0 to the output value of the defective pixel.

  As in this embodiment, except when the target pixel is a defective pixel, if the tap coefficient of the conditional average filter is switched so that the tap coefficient is set to a value of 0 for the defective pixel, The same effects as those of the first embodiment can be obtained by preventing the image quality deterioration due to the influence.

  In the digital still camera of this embodiment, the image data used for the noise removal process is biased to reduce the frequency of access to the memory. Note that this embodiment is configured in the same way as the first embodiment or the second embodiment, except that the image data is biased to reduce the frequency of access to the memory.

  That is, the digital still camera of this embodiment stores the low-frequency component image data output from the low-pass filter 15 in the buffer memory and temporarily holds it. Image data to which tap coefficients other than 0 are assigned is selectively loaded from this buffer memory for each pixel of interest and processed by the low-frequency noise removing unit 16.

  In addition, in the process of loading from the buffer memory, the image data that has already been loaded into the low-frequency noise removing unit 16 in the noise removal process at the immediately preceding target pixel is stopped from being loaded. Subsequently, the noise removal processing of the target pixel is executed using the image data that has been processed.

  Further, at least in the process of the target pixel, the tap coefficient of the conditional average filter is switched so that the tap coefficient assigned to the same pixel is other than 0, and the same image data is repeated in the process of the continuous target pixel. The image data used for noise removal processing is biased.

  According to this embodiment, at least in the processing of the target pixel, the image used for the noise removal process is switched by switching the tap coefficient of the conditional average filter so that the tap coefficient assigned to the same pixel is other than 0. The frequency of access to the memory can be reduced by giving bias to the data, and the same effect as in the first or second embodiment can be obtained by processing the image data at a higher speed.

  In the digital still camera of this embodiment, in the noise removal processing of the continuous pixel of interest, the image data of the same pixel is not repeatedly used for the noise removal processing, and the noise removal processing of the continuous pixel of interest is processed in parallel. Speed up. In this embodiment, the configuration is the same as that of the first or second embodiment except for the configuration relating to the setting of the image data used for the noise removal processing.

  That is, in the digital still camera of this embodiment, the low-frequency component image data output from the low-pass filter 15 is stored in the buffer memory and temporarily held. Further, image data to which tap coefficients other than 0 are assigned is selectively loaded from this buffer memory and processed by the low frequency noise removing unit 16.

  The low-frequency noise removing unit 16 includes at least two conditional average filters. The two conditional average filters simultaneously perform noise removal processing on consecutive pixels of interest through an output buffer. Output sequentially. Further, in the two systems of noise removal processing, image data to which tap coefficients other than 0 are assigned is loaded from the buffer memory to the conditional average filters of each system simultaneously and in parallel.

  Further, in the continuous pixel of interest that executes the processing in parallel simultaneously, the tap coefficient of each conditional average filter is switched so that the tap coefficient of the same pixel does not become a value other than 0 continuously. The image data loaded on one of the weighted average filters is not loaded on the other.

  According to this embodiment, in the processing of at least the continuous pixel of interest, the tap pixel of the conditional average filter is switched so that the tap coefficient of the same pixel does not continuously become a value other than 0. The same effect as that of the first embodiment or the second embodiment can be obtained by increasing the processing speed when the noise removal processing is performed in parallel.

  This embodiment is applied to a digital video camera to shoot a moving image and record it on a recording medium. This digital video camera removes noise from the image data at the time of shooting this moving image in the same manner as in the above embodiments. At this time, the tap coefficient of the conditional average filter is switched for each predetermined field or every predetermined frame so that the frequency of the NULL point changes, and the frequency of the NULL point is changed in the time axis direction, thereby blurring a specific stripe pattern. To prevent. The digital video camera of this embodiment has the same configuration as each of the above-described embodiments except for the configuration relating to the processing of the moving image.

  As in this embodiment, even if the tap coefficient of the conditional average filter is switched every predetermined field or every predetermined frame so that the frequency of the NULL point changes, the blur of the specific stripe pattern is prevented and The same effect as the embodiment can be obtained.

  In the above-described embodiment, the case where high-frequency aliasing is suppressed to a gain of 0.2 or less using a low-pass filter has been described. However, the present invention is not limited to this, and suppression of aliasing is obtained by suppressing noise. Therefore, when the noise is significant, suppression of about 0.25 gain is sufficient. Further, in the case of an image quality in which high-frequency aliasing is not conspicuous, suppression with a gain of about 0.3 is sufficient. However, according to various examination results, even when the partial band of the aliasing is gain 0.2 or more, the gain of this partial band is suppressed to 0.35 or less, and the other aliasing band is gain 0. If it is suppressed to .2 or less, it is possible to suppress noise sufficiently without making high-frequency aliasing inconspicuous. When further improvement in image quality is required, all bands resulting from aliasing are gained by 0.2 or less. Therefore, noise can be sufficiently suppressed without making high-frequency aliasing inconspicuous.

  Further, in the above-described embodiment, the case where the raw data is divided into the high-frequency component and the low-frequency component and processed is described. However, the present invention is not limited to this, and the case where the raw data is divided into three or more bands is processed. Can also be widely applied. In this case, in addition to the low-frequency processing, the present invention may be applied to the mid-frequency processing.

  In the above-described embodiments, the case where the tap coefficient of the conditional average filter is set to the value 0 or the value 1 and the desired noise suppression characteristic is ensured is described. However, the present invention is not limited to this, and the conditional average is used. The filter tap coefficient may be set to a value 0 or other than 0 to ensure a desired noise suppression characteristic.

  In the above-described embodiments, the case where the ε filter is applied to the conditional average filter has been described. However, the present invention is not limited to this, and various conditional average filters such as a bilateral filter can be widely applied. .

  In the first embodiment, etc., the case where the tap coefficient is switched between the vertical direction and the horizontal direction has been described. However, the present invention is not limited to this, and a plurality of types of tap coefficient settings with different NULL point frequencies are prepared. Alternatively, these may be switched and used.

  In the above-described embodiment, the case where the characteristic of the low-pass filter is fixed and the tap coefficient of the conditional average filter is switched variously has been described. However, the present invention is not limited to this, and the tap of the conditional average filter is described. The characteristics of the low-pass filter may be switched in conjunction with the coefficient.

  In the above-described embodiment, the case where noise is removed by the low-pass filter and the low-frequency noise removing unit is described only when the S / N ratio of the image data is deteriorated. However, the present invention is not limited to this. The present invention can also be applied when the S / N ratio of the image data is not so deteriorated.

  In the above-described embodiments, the case of removing noise of image data due to RAW data has been described. However, the present invention is not limited to this, and the image data of luminance signal and color difference signal is set as a processing target to remove noise. For example, the present invention can be widely applied to removing noise from image data in various formats.

  Further, in the above-described embodiment, the case where noise is removed at the stage before demosaic processing has been described. However, the present invention is not limited to this, and when noise removal is performed after demosaic processing, during the processing in the image processing unit, after processing The present invention can be widely applied to noise removal in various processes such as noise removal.

  In the above-described embodiments, the case where the present invention is applied to a digital still camera and a digital video camera has been described. However, the present invention is not limited to this, and various image processing apparatuses that process image data and various image processing programs. Can be widely applied. The image processing program may be provided by being installed in advance in a computer, an image processing apparatus, or the like, or alternatively, provided by being recorded on various recording media such as an optical disk, a magnetic disk, or a memory card. Alternatively, it may be provided by downloading via a network such as the Internet.

  The present invention can be applied to, for example, a digital still camera.

It is a block diagram which shows the structure of the optical correction part of the digital still camera of Example 1 of this invention. It is a block diagram which shows the digital still camera of Example 1 of this invention. It is a characteristic curve figure which shows the characteristic of the high-pass filter of the optical correction | amendment part of FIG. It is a characteristic curve figure which shows the characteristic of the low-pass filter of the optical correction | amendment part of FIG. It is a flowchart which shows the process sequence of the optical correction part of FIG. It is a characteristic curve figure with which it uses for description at the time of applying the epsilon filter whose tap coefficients are all the values 1 to the low frequency noise removal part of the optical correction part of FIG. It is a characteristic curve figure with which it uses for description at the time of applying the epsilon filter whose tap coefficient is alternately the value 1 and the value 0 to the low frequency noise removal part of the optical correction part of FIG. It is a top view which shows the tap coefficient of the low frequency noise removal part of the optical correction part of FIG. It is a characteristic curve figure which shows the frequency characteristic by the total tap coefficient of the perpendicular direction of FIG. It is a characteristic curve figure which shows the frequency characteristic by the total tap coefficient of the horizontal direction of FIG. It is a top view with which it uses for description of the process of the red pixel by the tap coefficient of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Digital still camera, 3 ... Image sensor, 5 ... Optical correction part, 6 ... Image processing part, 13 ... High-pass filter, 14 ... High-pass noise removal part, 15 ... Low-pass Filter, 16 ... Low-frequency noise removal unit, 18 ... Image composition unit

Claims (13)

An imaging unit that acquires imaging results and outputs image data;
A noise suppression unit that suppresses noise of the image data and outputs output data ;
Have
The noise suppressor is
A two-dimensional low-pass filter for band-limiting the image data and outputting a low-frequency component ;
A two-dimensional conditional average filter that band-limits the low-frequency component output from the low-pass filter and suppresses noise of the low-frequency component ;
A high-pass filter that limits the band of the image data and outputs a high-frequency component;
A high-frequency component noise suppression unit that suppresses the high-frequency component noise;
Combining a high-frequency component output from the high-frequency component noise suppression unit with a low-frequency component output from the two-dimensional conditional average filter, and outputting the output data;
Have
The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at predetermined locations in the horizontal and vertical directions;
High-frequency aliasing due to the tap coefficient being set to a value of 0 was suppressed by the low-pass filter.
An imaging apparatus characterized by that. The imaging apparatus according to claim 1, wherein the two-dimensional conditional average filter is an ε filter. The imaging apparatus according to claim 1, wherein the image data is RAW data. The noise suppressor is
The tap coefficient of the two-dimensional conditional average filter is set to be different between the horizontal direction and the vertical direction so that the frequency at the NULL point is different between the frequency characteristic in the horizontal direction and the frequency characteristic in the vertical direction. The imaging apparatus according to claim 1, wherein the imaging apparatus is characterized. The noise suppressor is
The imaging apparatus according to claim 1, wherein the tap coefficient of the two-dimensional conditional average filter is switched for each predetermined pixel and / or for each predetermined line so that the frequency of the NULL point changes. The noise suppressor is
2. The tap coefficient of the two-dimensional conditional average filter is switched so that the tap coefficient is a value of 0 at the defective pixel except when the pixel of interest is a defective pixel of the imaging unit. The imaging device described in 1. The noise suppressor is
The imaging apparatus according to claim 1, wherein the tap coefficient of the two-dimensional conditional average filter is switched so that the tap coefficient of the same pixel becomes a value other than 0 in the processing of at least consecutive pixels of interest. The noise suppressor is
Including at least two systems of the two-dimensional conditional average filters that execute noise suppression processing of successive pixels of interest in parallel;
2. The tap coefficient of the two-dimensional conditional average filter is switched so that the tap coefficient of the same pixel does not continuously become a value other than 0 in the noise suppression processing of at least continuous pixels of interest. The imaging device described in 1. The image data is video image data,
The noise suppressor is
The imaging apparatus according to claim 1, wherein the tap coefficient of the two-dimensional conditional average filter is switched every predetermined field or every predetermined frame so that the frequency of the NULL point changes. An image processing apparatus for outputting an output data by suppressing the noise of the images data,
A two-dimensional low-pass filter for band-limiting the image data and outputting a low-frequency component ;
A two-dimensional conditional average filter that band-limits the low-frequency component output from the low-pass filter and suppresses noise of the low-frequency component ;
A high-pass filter that limits the band of the image data and outputs a high-frequency component;
A high-frequency component noise suppression unit that suppresses the high-frequency component noise;
Combining a high-frequency component output from the high-frequency component noise suppression unit with a low-frequency component output from the two-dimensional conditional average filter, and outputting the output data;
Have
The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at predetermined locations in the horizontal and vertical directions;
An image processing apparatus, wherein high-frequency aliasing caused by setting the tap coefficient to 0 is suppressed by the low-pass filter. In an image processing method for suppressing output noise and outputting output data,
A first band limiting step of band-limiting the image data and outputting a low-frequency component;
A second band limiting step of band limiting the low frequency component to suppress noise of the low frequency component ;
A third band limiting step of band-limiting the image data and outputting a high frequency component;
A step of noise suppression of the high frequency component for suppressing the noise of the high frequency component;
Combining the high-frequency component with suppressed noise with the low-frequency component with suppressed noise, and outputting the output data; and
Have
The bandwidth limitation by the second bandwidth limitation step is bandwidth limitation by a two-dimensional conditional average filter,
The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at predetermined locations in the horizontal and vertical directions;
An image processing method, wherein high-frequency aliasing caused by setting the tap coefficient to 0 is suppressed in the first band limiting step. In a program of an image processing method for outputting output data by suppressing noise in image data ,
A first band limiting step of band-limiting the image data and outputting a low-frequency component;
A second band limiting step of band limiting the low frequency component to suppress noise of the low frequency component ;
A third band limiting step of band-limiting the image data and outputting a high frequency component;
A step of noise suppression of the high frequency component for suppressing the noise of the high frequency component;
Combining the high-frequency component with suppressed noise with the low-frequency component with suppressed noise, and outputting the output data; and
And execute
The bandwidth limitation by the second bandwidth limitation step is bandwidth limitation by a two-dimensional conditional average filter,
The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at predetermined locations in the horizontal and vertical directions;
A program for an image processing method, wherein high-frequency aliasing caused by setting the tap coefficient to 0 is suppressed in the first band limiting step. In a recording medium recording a program of an image processing method for suppressing output noise and outputting output data,
The image processing method program is stored in a computer.
A first band limiting step of band-limiting the image data and outputting a low-frequency component;
A second band limiting step of band limiting the low frequency component to suppress noise of the low frequency component ;
A third band limiting step of band-limiting the image data and outputting a high frequency component;
A step of noise suppression of the high frequency component for suppressing the noise of the high frequency component;
Combining the high-frequency component with suppressed noise with the low-frequency component with suppressed noise, and outputting the output data; and
And execute
The bandwidth limitation by the second bandwidth limitation step is bandwidth limitation by a two-dimensional conditional average filter,
The tap coefficient of the two-dimensional conditional average filter is set to a value of 0 at predetermined locations in the horizontal and vertical directions;
A recording medium recording a program of an image processing method, wherein high-frequency aliasing caused by setting the tap coefficient to 0 is suppressed in the first band limiting step.

Images