JP5092536B2 - Image processing apparatus and program thereof - Google Patents

Description

  The present invention relates to an image processing apparatus and a program thereof, and more particularly, to an image processing apparatus and a program for performing filter processing on image data.

  In recent years, in an image processing apparatus that removes image noise using a nonlinear filter such as an ε filter, a plurality of ε filters are connected in cascade, and a predetermined interval is set for the coefficient of each ε filter (weight is 0). A technique for reducing the amount of calculation of filter processing by arranging the coefficients according to a predetermined rule has appeared (Patent Document 1).

Japanese Patent Laid-Open No. 2004-172726

  However, color noise occurs over a wide range of captured images during high-sensitivity shooting, and a filter that covers a wide range is required to remove color noise during high-sensitivity shooting. The amount increases and the processing time becomes longer.

  Therefore, the present invention has been made in view of such conventional problems, and an image processing apparatus capable of greatly reducing the amount of calculation associated with filter processing for removing noise generated over a wide range of a captured image, and The purpose is to provide the program.

In order to achieve the above object, the image processing apparatus according to the present invention specifies an arbitrary pixel as a target pixel from among the two-dimensionally arrayed pixels, a pixel value of the target pixel, and a plurality of pixels around the target pixel. An image processing apparatus including a filter processing unit that performs a filter process by calculating a new pixel value of the target pixel using an image processing filter that weights the pixel values of the surrounding pixels with a predetermined weight, A division filter that divides the weighting coefficient of the image processing filter into a plurality of weights so that one pixel is weighted, and the filter processing means is adapted to change a position of the target pixel in a horizontal direction or a vertical direction. Accordingly, the division filter divided into the plurality is periodically selected.

An image processing apparatus according to another aspect identifies an arbitrary pixel as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and a plurality of peripheral pixels around the target pixel An image processing apparatus comprising a filter processing unit that performs a filter process by calculating a new pixel value of the target pixel using an image processing filter that weights the pixel value of the pixel with a predetermined weight. The filter has a division filter divided into a plurality of weighting coefficients for a part of the pixels, and the filter processing means is arranged at the position of the target pixel from among the plurality of division filters. Filter processing is performed using one division filter selected in accordance with the division filter, and the division filter divided into the plurality is divided into the image processing filter in which the center of gravity of the weighting coefficient is set at the center. Data, and a filter which is divided into a plurality, characterized in that it is divided such that the sum of all the weighting coefficients of the plurality of division filter becomes equal to the weighting coefficients of the image processing filter.

An image processing apparatus according to another aspect identifies an arbitrary pixel as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and a plurality of peripheral pixels around the target pixel An image processing apparatus comprising a filter processing unit that performs a filter process by calculating a new pixel value of the target pixel using an image processing filter that weights the pixel value of the pixel with a predetermined weight. The filter has a division filter divided into a plurality of weighting coefficients for a part of the pixels, and the filter processing means is arranged at the position of the target pixel from among the plurality of division filters. Filter processing is performed using one division filter selected according to the division filter, and the division filter divided into a plurality of division filters has a center of weight of the weighting coefficient in each division filter. Characterized in that it is set in a position displaced.

An image processing apparatus according to another aspect identifies an arbitrary pixel as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and a plurality of peripheral pixels around the target pixel An image processing apparatus comprising a filter processing unit that performs a filter process by calculating a new pixel value of the target pixel using an image processing filter that weights the pixel value of the pixel with a predetermined weight. The filter has a division filter divided into a plurality of weighting coefficients for a part of the pixels, and the filter processing means is arranged at the position of the target pixel from among the plurality of division filters. Filtering is performed using one division filter selected according to the filter, and the interval of the weighting coefficient in the image processing filter is the shooting sensitivity or the pressure of the image data. Characterized in that determined by the block size of the system.

Further, the program according to the present invention specifies an arbitrary pixel as a target pixel among the two-dimensionally arrayed pixels, a pixel value of the target pixel, and pixel values of a plurality of peripheral pixels around the target pixel. Is a program that causes a computer to execute filter processing by calculating a new pixel value of the target pixel using an image processing filter that weights the image processing filter with a predetermined weight, wherein the filter processing includes weighting of the image processing filter One image processing periodically selected according to a change in the horizontal or vertical position of the pixel of interest from among a plurality of division filters in which coefficients are weighted to a part of the pixels Filtering is performed using a filter.

Further, the program according to another aspect specifies an arbitrary pixel as a target pixel from among the two-dimensionally arranged pixels, a pixel value of the target pixel, and a plurality of peripheral pixels around the target pixel. A program that causes a computer to execute a filter process by calculating a new pixel value of the target pixel using an image processing filter that weights a value with a predetermined weight, wherein the filter process is divided into the plurality Filter processing is performed using one division filter selected according to the position of the pixel of interest from among the division filters, and the interval of the weighting coefficient in the image processing filter is the imaging sensitivity or the compression of image data It is determined by the block size of the system.

  According to the present invention, the processing burden associated with the filter processing can be greatly reduced, and the processing time can be shortened.

Hereinafter, the present embodiment will be described in detail with reference to the drawings as an example in which the image processing apparatus of the present invention is applied to a digital camera.
[Embodiment]
A. Configuration of Digital Camera FIG. 1 is a block diagram showing a schematic electrical configuration of a digital camera 1 that implements the image processing apparatus of the present invention.
The digital camera 1 includes a photographing lens 2, a lens driving block 3, an aperture 4, a CCD 5, a driver 6, a TG (timing generator) 7, a unit circuit 8, a memory 9, a CPU 10, a DRAM 11, an image display unit 12, a flash memory 13, and a key. An input unit 14 and a bus 15 are provided.

  The photographing lens 2 includes a focus lens, a zoom lens, and the like that are constituted by a plurality of lens groups (not shown). A lens driving block 3 is connected to the photographing lens 2. The lens driving block 3 includes a focus motor and a zoom motor that drive the focus lens and the zoom lens along the optical axis direction, respectively, and a focus motor driver that drives the focus motor and the zoom motor according to a control signal sent from the CPU 10. And a zoom motor driver (not shown).

The diaphragm 4 includes a drive circuit (not shown), and the drive circuit operates the diaphragm 4 in accordance with a control signal sent from the CPU 10.
The diaphragm 4 is a mechanism that controls the amount of light that enters from the photographing lens 2.

  The CCD 5 is driven by the driver 6 and photoelectrically converts the intensity of light of each color of the RGB value of the subject image for every fixed period and outputs it to the unit circuit 8 as an imaging signal. The operation timing of the driver 6 and the unit circuit 8 is controlled by the CPU 10 via the TG 7. The CCD 5 has a Bayer color filter (RGB color filter) and also functions as an electronic shutter. The shutter speed of the electronic shutter is controlled by the CPU 10 via the driver 6 and TG7.

  A TG 7 is connected to the unit circuit 8, a CDS (Correlated Double Sampling) circuit that holds the imaged signal output from the CCD 5 by correlated double sampling, and an AGC that performs automatic gain adjustment of the imaged signal after the sampling. An (Automatic Gain Control) circuit and an A / D converter that converts an analog image pickup signal after the automatic gain adjustment into a digital signal, the image pickup signal output from the CCD 5 passes through the unit circuit 8 as a digital signal. It is sent to CPU10.

  The CPU 10 stores the image data sent from the unit circuit 8 in the buffer memory (DRAM 11), and performs gamma correction, interpolation processing, white balance processing, luminance color difference signal (YUV data) on the stored image data. Image processing, image data compression / decompression (for example, JPEG compression / decompression) processing, image data recording processing to the flash memory 13, and control of each part of the digital camera 1 It is a one-chip microcomputer.

In particular, the CPU 10 has a function of performing a color noise reduction process for reducing color noise of image data. The color noise reduction process targets a color difference signal component of a captured image and has a coefficient that differs depending on the position of the target pixel. It includes divided ε filter processing for reducing noise by applying an ε (epsilon) filter, and LPF processing for reducing noise by LPF (low pass filter). In this LPF process, an LPF process is performed to smooth regular high-frequency noise generated in the entire image by applying an ε filter having a different coefficient depending on the position of the target pixel by the divided ε filter process.
Here, the LPF is a linear filter that mainly removes high-frequency noise, while the ε filter is a non-linear filter that removes noise generated in various forms as well as high-frequency noise. The ε filter is excellent in the performance of removing noise at the edge portion of the image.

The memory 9 stores a control program and necessary data necessary for the CPU 10 to control each unit, and the CPU 10 operates according to the program.
The DRAM 11 is used as a buffer memory for temporarily storing image data sent to the CPU 10 after being imaged by the CCD 5 and also as a working memory for the CPU 10.
The flash memory 13 is a recording medium that stores compressed image data.

The image display unit 12 includes a color LCD and its drive circuit, and displays the subject imaged by the CCD 5 as a through image when in a shooting standby state, and is read out from the flash memory 13 and decompressed when a recorded image is reproduced. The recorded image is displayed.
The key input unit 14 includes a plurality of operation keys such as a shutter button that can be fully pressed halfway, a mode switching key, a face detection mode on / off switching key, a strobe mode switching key, a cross key, and a SET key. An operation signal corresponding to the operation is output to the CPU 10.

B. About the divided ε filter processing The following divided ε filter processing, which is a feature of the present invention, will be described.
Here, the case where the two-dimensional image processing filter (nonlinear filter) used for the divided ε filter processing is (5 × 5) will be described. However, the (3 × 3) image processing filter and the (9 × 9) image are used. The present invention can also be applied to processing filters and the like. In this way, when the number of taps of the image processing filter used for the divided ε filter processing is changed, the number of taps of the LPF processing performed thereafter may be changed corresponding to this change.
Here, in order to reduce the noise of the pixel value of a certain pixel using the image processing filter, the pixel values of the surrounding pixels are also used. The pixel for which the color noise is to be reduced is referred to as a target pixel, and the pixel value of the target pixel is referred to as a target pixel value.

FIG. 2 is a functional block diagram of the divided ε filter processing.
The divided ε filter processing includes a target pixel specifying unit 21, a processing target pixel extracting unit 22, a non-linear operation unit 23, and a linear filter processing unit 24. A combination of the non-linear operation unit 23 and the linear filter processing unit 24 constitutes a non-linear filter. In the present embodiment, the non-linear operation unit 23 and the linear filter processing unit 24 are divided. However, depending on the type of the non-linear filter, the non-linear operation unit and the linear filter processing unit may not be clearly divided. Alternatively, the non-linear operation unit 23 may be deleted, and the linear filter processing unit 24 may be configured as a non-linear filter processing unit.
When the pixel-of-interest specifying unit 21 acquires the image data of the color difference component (the pixel value of the color difference signal of each pixel of the CCD 5) out of the luminance color difference signals captured by the CCD 5 and generated by the CPU 10, the target color specifying signal 21 The position of the pixel to be the target pixel is specified from the pixels. The target pixel specifying unit 21 outputs the acquired image data of the color difference component to the processing target pixel extracting unit 22, and uses the processing target pixel extracting unit 22 and the linear filter process for the position information (x, y) of the specified target pixel. To the unit 24.

The processing target pixel extracting unit 22 extracts a pixel value to be processed based on the position information (x, y) of the target pixel sent from the target pixel specifying unit 21 and the image data of the color difference component. The pixel value to be processed is a target pixel value and surrounding pixel values.
Here, since the (5 × 5) image processing filter is used, the range of the peripheral pixels (peripheral pixels) is a range of two pixels on the left and right and two pixels on the upper and lower sides from the target pixel. That is, the range of surrounding pixels changes according to the number of taps of the image processing filter. For example, when the (3 × 3) image processing filter is used, the range of the surrounding pixels is one pixel left and right from the target pixel. When the (9 × 9) image processing filter is used, the range of surrounding pixels is the range of 4 pixels left and right and 4 pixels above and below from the target pixel.

FIG. 3A shows the state of the extracted pixel value to be processed. Among the pixels extracted as the processing target, the pixel value at the center (pixel value C) is the target pixel value. Other pixels have peripheral pixel values (pixel value Pn; n = 1, 2,..., 24).
The extracted pixel value is output to the non-linear operation unit 23.

The non-linear operation unit 23 performs a process of performing non-linear operation on the pixel values (peripheral pixel values) of the peripheral pixels sent from the processing target pixel extraction unit 22 using the target pixel value and the threshold value ε The calculation is performed on the pixel value, and the calculation result is output to the linear filter processing unit 24. The value of this calculation result (Tn; n = 1, 2,..., 24) is the new value of the peripheral pixel.
This non-linear calculation processing determines whether or not the absolute value of the difference between the surrounding pixel value and the target pixel value is larger than the threshold value ε, and if it is smaller than the threshold value ε, outputs the surrounding pixel value as the calculation result as it is. If the absolute value is not smaller than the threshold ε, the pixel value of interest is output as the calculation result. Note that the target pixel value C is output to the linear filter processing unit 24 as it is without performing any operation.

FIG. 3B shows the state of the calculation result. Since the calculation is not performed on the target pixel, the value of the target pixel is C as it is.
Further, T1, T2,..., T24 indicate values of calculation results of pixel values of P1, P2,..., P24, respectively, and the absolute value of the difference from the target pixel value is greater than the threshold value ε. If it is smaller, Tn is the value of Pn, and if the absolute value of the difference from the target pixel value is not smaller than the threshold ε, Tn is the value of the target pixel value C.

The linear filter processing unit 24 uses the pixel value of the calculation result and the value of the target pixel value C (pixel values as shown in FIG. 3B) and the linear image processing filter sent from the nonlinear calculation unit 23. A target pixel value C1 in which the color noise of the target pixel value C is reduced is calculated.
Conventionally, here, for example, an image processing filter as shown in FIGS. 3C and 3D, that is, an image processing filter in which one or more coefficients are evenly overlooked in the entire range (the pixels to be weighted are in the entire range). The image processing filter that is evenly overlooked) is used in common for all the target pixels in the image data, and there is a restriction in setting an interval for the coefficient, so that the calculation amount of the target pixel value C1 is increased. It was. That is, since this image processing filter is common to all the target pixels, the center of gravity of the coefficient is the center (target pixel), and the weight of the center (target pixel) is the largest. A process for calculating the target pixel value C1 by weighted averaging the target pixel value C and the peripheral pixel value Tn using this image processing filter is performed for all target pixels. It was given in common.

  Here, the description will be made using the image processing filter shown in FIG. 3C, and the calculation of the target pixel value C1 using the image processing filter of FIG. = {(T9 + T13 + T20 + T24) +2 (T10 + T11 + T12 + T14 + T15 + T16 + T17 + T18 + T19 + T21 + T22 + T23) +4 (T1 + T2 + T3 + T4 + T5 + T6 + T7 + T8 + T9 + C)} / 64 Note that the image processing filter shown in FIG. 2D is an image processing filter when the interval of the image processing filter shown in FIG.

On the other hand, in the present invention, an image processing filter as shown in FIG. 3C with restrictions on coefficient setting is divided into a plurality of image processing filters as shown in FIG. The target pixel value C1 is calculated using any one of the image processing filters divided for the pixel.
(1), (2), (3), and (4) in FIG. 4A show an example of each image processing filter divided when the image processing filter in FIG. 3C is divided into four. It is. In this division, as long as the divided image processing filters are summed up to form the original image processing filter, it is relatively free to divide the coefficients with intervals. In FIG. 4A, (1) is referred to as a division filter 1, (2) as a division filter 2, (3) as a division filter 3, and (4) as a division filter 4.

Each division filter shown in FIG. 4A is an image processing filter in which the original coefficients are divided so that the weighted ranges are different from each other while weighting a range of a part of the peripheral pixels. The divided image processing filters when FIG. 3D is divided into four are the image processing filters of (1), (2), (3), and (4) in FIG. The image processing filter has an interval of 2.
As can be seen from FIG. 4A, each divided filter is an image processing filter of (5 × 5) in terms of form, but if the portion whose value is “0” is excluded (3 × 3) Therefore, the calculation amount is substantially the same as that of the (3 × 3) image processing filter, and the calculation amount of the target pixel value C1 can be reduced.

For example, when the division filter 1 is used, the calculation amount can be reduced by calculating the target pixel value C1 = {(T9 + T11 + T16 + C) +2 (T2 + T4 + T10 + T14) +4 (T1)} / 16. That is, the amount of calculation is reduced by reducing the pixel values used for calculating the weighted average.
Each of the division filters has the center of gravity of the coefficient and the pixel with the largest weighting and the pixel of interest shifted, but the image processing filter is divided so that the pixel with the largest weighting matches the pixel of interest. Alternatively, the image processing filter may be divided so that the center of gravity of the coefficient becomes the target pixel.

  The linear filter processing unit 24 selects a division filter based on the position information (x, y) of the target pixel output from the target pixel specifying unit 21. Which division filter is used to calculate the target pixel value C1 is determined by the position of the target pixel value C. As shown in FIG. 4B, x of the position (x, y) of the target pixel value C is determined. If it is even and y is even, the image processing filter of (1) in FIG. 4A is used. If x is odd and y is even, the image processing filter of (2) is used. If x is even and y is odd, (3) If x is an odd number and y is an odd number, the image processing filter of (4) is used. In the captured image data, the position of the upper left pixel is (x, y) = (0, 0).

By specifying all the pixel values picked up by the CCD 5 by the target pixel specifying unit 21, the divided ε filter processing is performed on all the pixel values. As a result, it is possible to calculate image data from which color noise has been reduced and removed, and to significantly reduce the processing time.
In this divided ε filter processing, regular high-frequency noise is generated in the entire image by applying ε filters having different coefficients depending on the position. For this reason, after the divided ε filter processing, the entire image data is further subjected to LPF processing in order to remove (smooth) this regular high-frequency noise.

  In this LPF process, by using an image processing filter for LPF, the target pixel value C2 is calculated by taking a weighted average of the target pixel value C to be subjected to the LPF process and the surrounding pixel values. This LPF process is substantially the same as the ε filter process, but the target pixel value C and the peripheral pixel value Pn are calculated without performing a non-linear operation on the peripheral pixel value Pn of the target pixel value to calculate the pixel value Tn. Is used to calculate a new target pixel value C2. The high-frequency noise to be removed (smoothed) differs depending on the division method, but is generated in a cycle corresponding to the cycle in which the division filter is applied alternately. Therefore, the number of taps of the LPF is divided. It is sufficient to use a filter having a size corresponding to the period in which the filters are alternately applied. As a result, the amount of calculation can be suppressed without wastefully performing the LPF processing. Here, it is assumed that a (3 × 3) two-dimensional image processing filter as shown in FIG.

For example, when the target pixel value C as shown in FIG. 3A is calculated, the calculation formula by the LPF process is the target pixel value C2 = {(P1 + P3 + P6 + P8) + 2 × (P2 + P4 + P5 + P7) + 4 × C} / 12. It will be.
Needless to say, the target pixel value C and the peripheral pixel value Pn here are the target pixel value C1 that has already been subjected to the ε filter processing.

C. Operation of Digital Camera 1 The operation of the digital camera 1 in the embodiment will be described with reference to the flowchart of FIG.

First, in step S <b> 1, the CPU 10 acquires RGB image data captured by the CCD 5.
Next, in step S <b> 2, the CPU 10 generates image data of luminance color difference signals from the acquired image data, and separates the generated image data into image data of luminance components and image data of color difference components. That is, each captured pixel value is separated into a luminance signal and a color difference signal.

Next, in step S3, the CPU 10 performs a divided ε filter process on the image data of the separated color difference components. That is, by performing the divided ε filter processing on the pixel values of all the color difference signals, the color noise of the pixel values of the color difference signals of each pixel is reduced. The operation of this divided ε filter process will be described later.
Next, in step S4, the CPU 10 performs LPF processing on the pixel values of all color difference signals that have been subjected to the divided ε filter processing. Here, it is assumed that low-pass filter processing is performed using a (3 × 3) image processing filter.

D. Operation of Split ε Filter Process Next, the operation of the split ε filter process will be described with reference to the flowchart of FIG.
When the divided ε filter processing is started, that is, when the process proceeds to step S3 in FIG. 5, the process proceeds to step S11 in FIG. 6, and the target pixel specifying unit 21 acquires and specifies the pixel value of each pixel of the color difference signal. The pixel position is set to (x, y) = (0, 0). Thereby, the pixel of (x, y) = (0, 0) is specified as the target pixel. The pixel value of each pixel of the acquired color difference signal is output to the processing target image extraction unit 22, and the position information (x, y) of the identified target pixel is transmitted to the processing target pixel extraction unit 22 and the linear filter processing unit 24. Output.

  Next, in step S12, the processing target pixel extraction unit 22 extracts the pixel value C at the specified position (target pixel) and the surrounding pixel value Pn as pixels to be processed. The extracted pixel to be processed is output to the nonlinear arithmetic unit 23. The range of the peripheral pixels to be extracted changes depending on the number of taps of the image processing filter. In the case of the (5 × 5) image processing filter, the pixel values in the range of the left and right 2 pixels of the target pixel and the upper and lower 2 pixels Will be extracted. FIG. 3A shows a state of pixels extracted as a processing target.

  Next, in step S13, the non-linear operation unit 23 performs a process of performing non-linear operation on the peripheral pixel value Pn sent from the processing target extraction unit 22 using the target pixel value C and the threshold ε. The calculation is performed on the pixel value, and the value Tn of the calculation result is output to the linear filter processing unit 24. Note that the target pixel value C is output to the linear filter processing unit 24 without performing any operation.

  This non-linear calculation processing 23 determines whether or not the absolute value of the difference between the peripheral pixel value Pn and the target pixel value C is larger than the threshold value ε. If the absolute value of the difference is larger than the threshold value ε, the pixel value of interest is output as the calculation result. FIG. 3B shows the state of the calculation result of FIG.

Next, in step S14, the linear filter processing unit 24 sets the division filters corresponding to y & 1 and x & 1 as image processing filters used for the filter processing.
This x & 1 calculates the value of the least significant bit when x is expressed in binary, and the position of x of the identified pixel of interest is an even number (0, 2, 3, 4,...). In this case, x & 1 = 0, and when the position of x is an odd number (1, 3, 5, 7,...), X & 1 = 1. Similarly, in the case of y & 1, when the position of y is an even number (0, 2, 3, 4,...), Y & 1 = 0 and the position of y is an odd number (1, 3, 5, 7,... In the case of ()), y & 1 = 1.

When x & 1 = 0 and y & 1 = 0, the filter is set to the division filter 1, and when x & 1 = 1, and when y & 1 = 0, the filter is set to the division filter 2, and x & 1 = 0. , Y & 1 = 1, the division filter 3 is set, x & 1 = 1, and y & 1 = 1, the division filter 4 is set.
Next, in step S15, the linear filter processing unit 24 reduces the color noise of the target pixel value C using the pixel value (including the target pixel value C) of the calculation result and the set division filter. Is calculated.

Next, in step S <b> 16, the target pixel specifying unit 21 determines whether or not the x position of the position (x, y) of the target pixel currently specified is the image width.
For example, if there are 0 to 399 pixels on the x axis of the CCD 5, it is determined whether or not x = 399.
If it is determined in step S16 that the x position of the target pixel currently specified is not the image width, the process proceeds to step S17, and the target pixel specifying unit 21 determines the position of the target pixel to be newly specified as the current x position. Is set to the incremented position (x, y) = (x + 1, y), and the process returns to step S12. As a result, the ε filter processing can be performed on all the pixels in one line.

  On the other hand, if it is determined in step S16 that the x position of the currently specified pixel of interest is the image width, the process proceeds to step S18, and the pixel-of-interest specifying unit 21 determines the position (x, It is determined whether the y position in y) is the image height. For example, when there are 0 to 299 pixels on the y-axis of the CCD 5, it is determined whether or not y = 299.

If it is determined in step S18 that the position of y of the pixel of interest currently specified is not the image height, the process proceeds to step S19, and the pixel-of-interest specifying unit 21 sets the position of the newly specified pixel of interest to x, The current position y is incremented to position (x, y) = (0, y + 1), and the process returns to step S12. Thereby, the divided ε filter processing can be performed on the pixels of all lines.
If it is determined in step S18 that the currently set y position of the target pixel is the image height, the process ends, and the process proceeds to step S4 in FIG.

  As described above, in the embodiment, the image processing filter in which the weighted pixels are evenly seen over the entire range is weighted with respect to a range of a part of the peripheral pixels, and the weighted ranges are different. Since the ε filter processing is performed using any one of the divided filters according to the position of the target pixel among the divided filters divided into four, the processing burden associated with the ε filter processing can be greatly reduced. Time can be shortened. Further, regular high-frequency noise generated in the entire image is removed (smoothed) by the LPF process by applying an ε filter having a different coefficient depending on the position. Therefore, when dividing the original image processing filter This eliminates the need to set the center of gravity of the coefficients in each divided filter at the center, thereby making it possible to further reduce the filter processing time by providing more coefficient intervals.

[Modification]
E. The above embodiment may be the following modes.
(01) In the above embodiment, the center of gravity of the coefficient in each divided filter is set at a position away from the center (the target pixel). However, the center of gravity of the coefficient in each divided filter is set at the center (the same position as the target pixel). May be. Further, in such a division, since the center of gravity of the coefficient is not changed, the subsequent LPF processing may be omitted. By omitting the LPF processing, the processing load can be reduced and the processing time can be shortened. it can.

FIG. 7A is a diagram illustrating an example of a state of each divided filter that is divided into four so that the center of gravity of the coefficient is in the center (the same position as the target pixel).
FIG. 7A shows an example in which an (5 × 5) image processing filter as shown in FIG. 3C is divided into four. In this case as well, the sum of the divided image processing filters is the original image processing filter. Here, it can be seen that the weighted pixels differ for each filter.

  Also at this time, as described in the above embodiment, the division filter to be used is changed according to the position of the target pixel. That is, when x & 1 = 0 and y & 1 = 0, the division filter shown in (1) of FIG. 7A is set, and when x & 1 = 1 and y & 1 = 0, FIG. When the divided filter shown in (2) of (a) is set to x & 1 = 0 and y & 1 = 1, the divided filter shown in (3) of FIG. 7A is set, and x & 1 = 1. If it is present and y & 1 = 1, the division filter shown in (4) of FIG.

For example, when the division filter shown in (1) of FIG. 7A is used, the pixel value of interest C1 = {(T9 + T13 + T20 + T24) +2 (T11 + T16 + T17 + T22) +4 (C)} / 16 can be calculated. When the division filter shown in (2) of FIG. 7A is used, the pixel value of interest can be calculated by the following formula: C1 = (T10 + T12 + T14 + T15 + T18 + T19 + T21 + T23) / 8.
When the division filter shown in (3) of FIG. 7A is used, it can be calculated by the calculation formula of the target pixel value C1 = (T2 + T4 + T5 + T7) / 4, and the division shown in (4) of FIG. When a filter is used, the pixel value of interest can be calculated by the following formula: C1 = (T1 + T3 + T6 + T8) / 4.

In this way, the pixel value C1 with reduced noise is calculated using any one of the divided filters divided into four so that the center of gravity of the coefficient is in the center (the same position as the target pixel). Accordingly, the calculation amount of the pixel value C1 can be significantly reduced compared to the conventional technique (from the technique of calculating using an undivided image processing filter).
Further, considering the calculation of the pixel value with reduced noise of each pixel value, the calculation amount can be further suppressed than the calculation amount by the division filter described in the first embodiment.

(02) In the above embodiment, the (5 × 5) image processing filter is divided into four division filters. For example, when the (3 × 3) image processing filter is divided into four, FIG. As shown in (b) and (c).
Here, the image processing filter used for the ε filter processing is shown in FIG. 4C, and FIGS. 7B and 7C show the (3 × 3) image processing filters shown in FIG. It is a figure which shows an example of the mode of each division | segmentation filter at the time of dividing | segmenting.
In this case as well, the total of the divided image processing filters becomes the original image processing filter, and the division filter to be used is changed according to the position of the target pixel.

(03) In the above embodiment, the image processing filter is divided into four, but it may be divided into two.
(1) and (2) in FIG. 8 (a) show the state of the division filter when the image processing filter as shown in FIG. 4 (a) is divided into two. Similarly, each divided filter is an image processing filter that weights a range of a part of the peripheral pixels and has different weighting ranges.
In the ε filter processing in the case of two divisions, the x position of the target pixel is an even number, the y position is an even number, the x position of the target pixel is an odd number, and the y position is an odd number. 7 (a) When the divided filter 1 is used and the x position of the target pixel is an odd number and the y position is an even number, the x position of the target pixel is an even number and the y position is an odd number. The division filter 2 of 7 (a) is used.

Moreover, it is not limited to two divisions, and may be 16 divisions.
FIG. 9 shows a state of the division filter when the image processing filter as shown in FIG. Similarly, each divided filter is an image processing filter that weights a range of a part of the peripheral pixels and has different weighting ranges.
Here, a number that is a multiple of 4 (including 0) is an even number 1 (0, 4, 8, 12,...), And a number obtained by adding 2 to a multiple of 4 (including 0) is an even number 2. (2, 6, 10, 14,...) 1 plus a multiple of 4 (including 0) is an odd number 1 (1, 5, 9, 13,...), 3 is 4 A number obtained by adding a multiple (including 0) is an odd number 2 (3, 7, 11, 15...).

In the ε filter processing in the case of two divisions, when the position of y of the target pixel is an even number 1, and the position of x is an even number 1, the division filter shown in (1) of FIG. When the position of x of the pixel is an odd number 1, the division filter shown in (2) of FIG. 8 is set. When the position of x is an even number 2, the division filter shown in (3) of FIG. When the position of x is an odd number 2, the division filter shown in (4) of FIG. 8 is set.
If the position of y of the target pixel is an odd number 1 and the position of x is an even number 1, the division filter shown in (5) of FIG. 8 is set, and the position of x of the target pixel is an odd number 1. 8 is set, the division filter shown in (6) of FIG. 8 is set. When the position of x is even 2, the division filter shown in (7) of FIG. 8 is set, and the position of x is odd 2. In this case, the division filter shown in (8) of FIG. 8 is set.

Further, when the position of y of the target pixel is an even number 2 and the position of x is an even number 1, the division filter shown in (9) of FIG. 8 is set, and the position of x of the target pixel is an odd number 1. 8 is set, the division filter shown in (10) of FIG. 8 is set. When the position of x is even 2, the division filter shown in (11) of FIG. 8 is set, and the position of x is odd 2. In this case, the division filter shown in (12) of FIG. 8 is set.
If the position of y of the pixel of interest is an odd number 2 and the position of x is an even number 1, the division filter shown in (13) of FIG. 8 is set, and the position of x of the pixel of interest is an odd number 1. 8 is set, the division filter shown in (14) of FIG. 8 is set. When the position of x is even 2, the division filter shown in (15) of FIG. 8 is set, and the position of x is odd 2. In this case, the division filter shown in (16) of FIG. 8 is set.

  Note that the division is not limited to two divisions or sixteen divisions, and may be nine divisions or the like. In the case of 2 divisions, the number of taps of the image processing filter used for the LPF processing is the same 3 × 3 tap as in the case of 3 divisions, but in the case of 9 divisions or 16 divisions, 5 × 5 taps are suitable. In addition, although the ε filter processing is performed using the two-dimensional image processing filter, the ε filter processing and the LPF processing may be performed using the one-dimensional image processing filter.

[04] In the above embodiments and modifications (1) to (3), the aspect of the division filter has been described. The image processing filter may be divided so as to be weighted. Further, the divided filter may be further divided.
Since the ε filter processing is performed using any one of the divided filters corresponding to the position of the target pixel, the processing burden associated with the ε filter processing can be greatly reduced, and the processing time can be shortened.

(05) In the above-described embodiment, the divided ε filter processing is performed on all imaged pixel values. However, the pixels on which the divided ε filter processing is performed may be thinned out.
FIG. 8B is a diagram illustrating a state of a pixel to be subjected to ε filter processing.
The shaded area in FIG. 8B represents thinned pixels that are not subjected to the divided ε filter processing. For the thinned pixels, an average value of the pixel values subjected to the adjacent divided ε filter processing is taken. Since only the divided ε filter processing is not performed on the pixels to be thinned out (the pixels in the hatched portion) (because they are not specified as the target pixel), the pixel values that are not thinned out are divided. When the ε filter process is performed, it is used as a peripheral pixel value.
As a result, the processing load can be greatly reduced and the processing time can be shortened as compared with the case where the ε filter processing is performed on all imaged pixel values.

(06) Also, the interval may be changed according to the compressed block size of the image data.
Hereinafter, the change of the interval according to the compression block size will be described in more detail using JPEG format compression as an example.
A JPEG format is generally used as a method for compressing image data picked up by the CCD 5.
The compression by this JPEG format is a process of displaying a block of n × m pixels as a frequency component by performing discrete cosine transformation, quantizing the frequency component, and encoding it with Huffman coding. Do this for each block.

At this time, there is often a certain tendency for each block of n × m pixels at the quantization stage, and when the JPEG image is enlarged, the boundary lines of these blocks can be visually recognized. Also, a block is often yellowish and the block next to it is often reddish.
Therefore, in the conventional method of performing weighted averaging of adjacent peripheral pixels, the pixel values are weighted and averaged while maintaining the color tendency for each block, and color noise cannot be removed cleanly.

Therefore, color noise can be efficiently removed by using pixels included in adjacent blocks instead of neighboring pixels adjacent to the target pixel for weighted averaging.
That is, if the block size is 8 × 8 pixels, the interval of the image processing filter is set to 8, and if the block size is 16 × 16 pixels, the interval of the image processing filter is set to 16. Noise can be removed.
In the above modification, the interval is changed according to the compressed block size. However, the higher the shooting sensitivity is, the larger the range in which color noise is generated. Therefore, the interval is changed according to the shooting sensitivity. May be. Thereby, color noise can be effectively removed.

(07) In the above embodiment, each pixel value imaged by the CCD 5 is subjected to the divided ε filter processing. However, the image size captured by the CCD 5 is reduced, and the reduced image is obtained. You may make it expand to the image data before reduction after performing a division | segmentation (epsilon) filter process with respect to data. As a result, it is possible to obtain the same result as when the ε filter process having a tap number several times that of the unreduced image data is simply performed.
Also, the higher the shooting sensitivity, the larger the range in which color noise occurs, so the reduction ratio may be increased. In this case, the enlargement rate changes according to the reduction rate. Thereby, color noise can be reduced effectively, the processing burden associated with the ε filter processing can be greatly reduced, and the processing time can be shortened.

  (08) In the above embodiment, the divided ε filter processing of the present invention is performed once, but the present invention is applied to the configuration of the prior art in which a plurality of ε filters are connected in cascade. In this case, the divided ε filter processing may be performed a plurality of times by applying the present invention to the ε filters connected in cascade. That is, the divided ε filter processing may be performed once or more on the image data on which the divided ε filter processing of the present invention has been performed. Thereby, color noise can be reduced effectively, the processing burden associated with the ε filter processing can be greatly reduced, and the processing time can be shortened.

Further, the interval of the image processing filter may be changed by a power of 2 for each number of ε filter processes. For example, in the case of the first ε filter processing, the interval of the image processing filter is 4 (2 ² ), and in the case of the second ε filter processing, the interval of the image processing filter is 2 (2 ¹ ). In the case of filter processing, the interval of the image processing filter may be set to 1 (2 ⁰ ). Thereby, a side lobe can be suppressed.

(09) In the above embodiment, the divided ε filter processing in which the ε filter is divided has been described. However, in other nonlinear filters such as a bilateral filter, the division processing as in the present invention is performed. Also good. Further, the division processing as in the present invention may be performed on a normal LPF instead of the nonlinear filter.
In the above-described embodiment, the ε filter is divided only for the color difference component, taking advantage of the characteristic that the color difference component is easier to remove the noise component without adversely affecting the signal component than the luminance component. Although the normal ε filter processing that does not divide the luminance component is applied, the divided ε filter processing may be applied to the luminance component.

  (10) Moreover, the aspect which combines arbitrarily the said modification (01)-(09) within the range which does not produce contradiction may be sufficient.

(11) The above embodiments of the present invention are merely examples as the best embodiments, and are described in order to better understand the principle and structure of the present invention. And is not intended to limit the scope of the appended claims.
Therefore, it should be understood that all the various variations and modifications that can be made to the above-described embodiments of the present invention are included in the scope of the present invention and protected by the appended claims.

  Finally, in each of the above-described embodiments, the case where the image processing apparatus of the present invention is applied to the digital camera 1 has been described. However, the present invention is not limited to the above-described embodiment. Any device can be applied.

1 is a block diagram of a digital camera according to an embodiment of the present invention. It is a functional block diagram of ε filter processing. It is a figure for demonstrating (epsilon) filter process. It is a figure for demonstrating (epsilon) filter process. It is a flowchart which shows operation | movement of the digital camera 1 in embodiment. It is a flowchart which shows the operation | movement of an epsilon filter process. It is a figure for demonstrating a modification. It is a figure for demonstrating a modification. It is a figure for demonstrating a modification.

Explanation of symbols

1 Digital Camera 2 Shooting Lens 3 Lens Drive Block 4 Aperture 5 CCD
6 Driver 7 TG
8 Unit circuit 9 Memory 10 CPU
11 DRAM
DESCRIPTION OF SYMBOLS 12 Image display part 13 Flash memory 14 Key input part 15 Bus 21 Target pixel specific part 22 Processing object pixel extraction part 23 Nonlinear calculation part 24 Linear filter processing part

Claims (14)

An image in which an arbitrary pixel is specified as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and pixel values of a plurality of peripheral pixels around the target pixel are weighted with a predetermined weight An image processing apparatus comprising filter processing means for performing filter processing by calculating a new pixel value of the target pixel using a processing filter,
A division filter that divides the weighting coefficient of the image processing filter into a plurality of weights so as to weight one part of pixels;
The filter processing means includes
An image processing apparatus, wherein the division filter divided into the plurality is periodically selected according to a change in a horizontal or vertical position of the target pixel. The divided filter divided into a plurality of
An image processing filter obtained by dividing the weighting coefficient of the image processing filter into four;
The filter processing means includes
The image processing according to claim 1, wherein four division filters having different weighting coefficients are periodically selected according to whether the horizontal position and the vertical position of the target pixel are odd or even. apparatus. The filter processing means includes
A pixel to be specified as a target pixel is specified by thinning out a checkered pattern from the two-dimensionally arranged pixel values, and a filter process is performed on the specified plurality of pixels. Then, the average value of the pixel values of the pixels subjected to the filtering process adjacent to the pixel is calculated, and the filtering process is performed on the thinned pixels by using the calculated average value as the pixel value of the pixel. The image processing apparatus according to claim 1, wherein the image processing apparatus performs the processing. The filter processing means includes
Performing filter processing using one divided filter periodically selected according to the position of the target pixel from among the plurality of divided filters, for all the two-dimensionally arranged pixels. The image processing apparatus according to claim 1, wherein: The filter processing means includes
The image processing apparatus according to claim 4, wherein the process of performing filter processing on all the pixel values arranged two-dimensionally is performed a plurality of times. The filter processing means includes
6. The image processing apparatus according to claim 5, wherein an interval of the division filter to be used is changed by a power of 2 for each number of times of filter processing.   The low-pass filter processing means for performing a low-pass filter process on the pixel values of all the two-dimensionally arranged pixels calculated by the filter processing means is provided. Image processing apparatus. The low-pass filter processing means includes
The image processing apparatus according to claim 7, wherein the number of taps corresponds to the number of divisions of the plurality of image processing filters. An image in which an arbitrary pixel is specified as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and pixel values of a plurality of peripheral pixels around the target pixel are weighted with a predetermined weight An image processing apparatus comprising filter processing means for performing filter processing by calculating a new pixel value of the target pixel using a processing filter,
A division filter that divides the weighting coefficient of the image processing filter into a plurality of weights so as to weight one part of pixels;
The filter processing means includes
Using one divided filter selected according to the position of the target pixel from among the plurality of divided filters, a filter process is performed,
The divided filter divided into a plurality of
The image processing filter in which the center of gravity of the weighting coefficient is set at the center is a filter obtained by dividing the image processing filter into a plurality of parts, and the weighting coefficients of the plurality of divided filters are all added to be equal to the weighting coefficient of the image processing filter. An image processing apparatus that is divided. An image in which an arbitrary pixel is specified as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and pixel values of a plurality of peripheral pixels around the target pixel are weighted with a predetermined weight An image processing apparatus comprising filter processing means for performing filter processing by calculating a new pixel value of the target pixel using a processing filter,
A division filter that divides the weighting coefficient of the image processing filter into a plurality of weights so as to weight one part of pixels;
The filter processing means includes
Using one divided filter selected according to the position of the target pixel from among the plurality of divided filters, a filter process is performed,
The divided filter divided into a plurality of
An image processing apparatus, wherein the center of gravity of a weighting coefficient in each division filter is set at a position shifted from the center. The divided filter divided into a plurality of
The image processing apparatus according to claim 10, wherein the weighting coefficient of the image processing filter is divided into a plurality of different weights. An image in which an arbitrary pixel is specified as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and pixel values of a plurality of peripheral pixels around the target pixel are weighted with a predetermined weight An image processing apparatus comprising filter processing means for performing filter processing by calculating a new pixel value of the target pixel using a processing filter,
A division filter that divides the weighting coefficient of the image processing filter into a plurality of weights so as to weight one part of pixels;
The filter processing means includes
Using one divided filter selected according to the position of the target pixel from among the plurality of divided filters, a filter process is performed,
The interval of weighting coefficients in the image processing filter is:
An image processing apparatus characterized by being determined by a photographing sensitivity or a block size of a compression method of image data. An image in which an arbitrary pixel is specified as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and pixel values of a plurality of peripheral pixels around the target pixel are weighted with a predetermined weight A program for causing a computer to execute a filter process by calculating a new pixel value of the target pixel using a processing filter,
The filtering process is
The weighting coefficient of the image processing filter is periodically selected from among the divided filters divided into a plurality of weights so that a part of the pixels is weighted according to a change in the horizontal or vertical position of the target pixel. A program characterized by performing filter processing using a single image processing filter. An image in which an arbitrary pixel is specified as a target pixel among the two-dimensionally arranged pixels, and a pixel value of the target pixel and pixel values of a plurality of peripheral pixels around the target pixel are weighted with a predetermined weight A program for causing a computer to execute a filter process by calculating a new pixel value of the target pixel using a processing filter,
The filtering process is
Using one divided filter selected according to the position of the target pixel from among the plurality of divided filters, a filter process is performed,
The interval of weighting coefficients in the image processing filter is:
A program characterized by being determined by the shooting sensitivity or the block size of the image data compression method.

Images