JP2015035775A - Image processing system and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing system, an image processing method, etc., that can reduce a noise amount more as an image size is smaller even when reduction processing on image data is performed at a reduction rate which is not an integral submultiple.SOLUTION: A reduction rate determination part 301 determines a reduction rate. A sampling phase determination part 302 determines a sampling phase after reduction processing according to the reduction rate determined by the reduction rate determination part 301. An LPF part 303 selects a pre-filter according to the reduction rate determined by the reduction rate determination part 301, and uses the pre-filter so as to perform pre-filter processing on image data output from an image conversion part 201. A thinning-out and interpolation processing part 304 reduces the image data output from the LPF part 303 through thinning-out or interpolation processing according to the sampling phase determined by the sampling phase determination part 302.

Description

  The present invention relates to an image processing apparatus and an image processing method that perform image data reduction processing and the like.

  Conventionally, when image data reduction processing is performed, the number of pixel sampling is reduced after applying a pre-filter for band limitation in order to prevent aliasing of the image signal. By applying a pre-filter according to the reduction ratio, it is possible to control the folding and resolution of the image after the reduction process. The reduction ratio indicates how much the size of the image data after the reduction processing is the ratio of the size of the image data before the reduction processing.

  For example, when an image is reduced by 1 / integer multiple such as 1/2 (50%), 1/3 (33%), and 1/4 (25%), a stronger band limiting filter is used as the reduction ratio is smaller. The pixels are thinned out from the sampling points of the image before the reduction process. The stronger the band limiting filter, the greater the weight of the neighboring pixel values in the weighted average. Therefore, an image with a reduced noise amount is obtained as the reduction rate is smaller. Therefore, when the image is reduced by 1 / integer multiple, the resolution and the amount of noise are reduced as the image size is reduced.

  However, when an image is reduced at a reduction ratio other than 1 / integer multiple, there are pixels where the pixel sampling position of the image after reduction processing matches or does not match the pixel sampling position of the image before reduction processing. For pixels whose pixel sampling position does not match the position before the reduction process, the pixel value is determined by the interpolation process. Therefore, the amount of noise varies depending on the pixel.

  Japanese Patent Application Laid-Open No. H10-228667 describes a method aimed at suppressing such noise amount variation.

JP 2001-43356 A JP 2009-177244 A

  However, in the conventional technique, when the image data is reduced at a reduction ratio other than 1 / integer multiple, the amount of noise may not be reduced even if the image size is reduced. The same applies to the method described in Patent Document 1.

  An object of the present invention is to provide an image processing apparatus, an image processing method, and the like that can reduce the amount of noise as the image size is reduced even when image data is reduced at a reduction ratio other than 1 / integer multiple. To do.

  An image processing apparatus according to the present invention includes a reduction unit that performs a reduction process of image data, and a sampling phase determination unit that determines a phase of pixel sampling in the reduction process. The pixel value of the image after the reduction process is determined by performing at least one of the pixel thinning process and the interpolation process of the image of the image, and the sampling phase determination means is an arbitrary one with different reduction ratios performed by the reduction means The phase of the pixel sampling is determined so that the noise amount after the reduction process with a small reduction ratio is equal to or less than the noise amount after the reduction process with a large reduction ratio between the two reduction processes.

  The image processing method according to the present invention includes a reduction step for performing reduction processing of image data, and a sampling phase determination step for determining a phase of pixel sampling in the reduction processing, wherein the reduction step is performed before the reduction processing. Determining a pixel value of the image after the reduction processing by performing at least one of pixel thinning processing and interpolation processing of the image, and the sampling phase determination step has a reduction rate performed in the reduction step. A step of determining the phase of the pixel sampling so that a noise amount after a reduction process with a small reduction ratio is equal to or less than a noise amount after a reduction process with a high reduction ratio between any two different reduction processes; It is characterized by.

  According to the present invention, the amount of noise can be reduced as the image size is reduced even when image data is reduced at a reduction ratio other than 1 / integer multiple.

1 is a block diagram illustrating an image processing apparatus according to an embodiment of the present invention. 3 is a block diagram illustrating a configuration of an image processing unit 104. FIG. It is a block diagram which shows the structure of the reduction process part 202 in 1st Embodiment. It is a flowchart which shows operation | movement of the reduction process part 202 in 1st Embodiment. It is a figure showing the phase of pixel sampling at the time of a reduction process. It is a figure which shows the change of the phase of pixel sampling in 1st Embodiment. It is a block diagram which shows the structure of the reduction process part 202 in 2nd Embodiment. It is a flowchart which shows operation | movement of the reduction process part 202 in 2nd Embodiment. 6 is a flowchart illustrating operations of a noise evaluation value calculation unit 705 and a sampling phase determination unit 702. It is a figure which shows the change of the phase of pixel sampling in 2nd Embodiment. It is a figure which shows the frequency characteristic and frequency weighting function of a filter. It is a block diagram which shows the structure of the reduction process part 202 in 4th Embodiment. It is a flowchart which shows operation | movement of the reduction process part 202 in 4th Embodiment. It is a flowchart which shows the other example of operation | movement of the reduction process part 202 in 4th Embodiment.

  Hereinafter, embodiments of the present invention will be specifically described with reference to the accompanying drawings.

(First embodiment)
First, the first embodiment will be described. The image processing apparatus according to the first embodiment is, for example, a digital camera that performs reduction processing of a captured image. FIG. 1 is a block diagram showing an image processing apparatus according to the first embodiment of the present invention. As shown in FIG. 1, the image processing apparatus according to the first embodiment includes an optical system 101, an image sensor 102, an A / D conversion unit 103, an image processing unit 104, a recording unit 105, a control unit 106, and an operation unit. 107 and a display unit 108 are included.

  The optical system 101 includes a focus lens, a diaphragm, and a shutter. At the time of shooting, the optical system 101 drives the focus lens to focus the subject, and controls the aperture and shutter to adjust the exposure amount. The imaging element 102 is a photoelectric conversion element such as a charge-coupled device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor. The image sensor 102 generates an analog electrical signal from the amount of light of the subject imaged in the optical system 101 by photoelectric conversion. The A / D (analog / digital) conversion unit 103 digitizes the input analog electric signal. The image processing unit 104 processes the signal digitized by the A / D conversion unit 103 to generate image data in a predetermined image format such as JPEG (joint photographic experts group). This processing includes, for example, reduction processing and noise reduction processing. The recording unit 105 records the image data processed by the image processing unit 104.

  The control unit 106 controls the overall operation of the image processing apparatus according to the present embodiment. The operation unit 107 is a part where the user gives an operation instruction to the image processing apparatus. The display unit 108 is, for example, a liquid crystal display installed on the back of the camera. The display unit 108 displays an image captured by the image sensor 102 and processed by the image processing unit 104, an image stored in the recording unit 105, and the like.

  Next, the image processing unit 104 will be described in more detail. FIG. 2 is a block diagram illustrating a configuration of the image processing unit 104. As shown in FIG. 2, the image processing unit 104 includes an image conversion unit 201, a reduction processing unit 202, a noise reduction unit 203, an edge enhancement unit 204, and an image conversion unit 205.

  The image conversion unit 201 performs a conversion process on the image data input from the A / D conversion unit 103. This conversion process includes, for example, a process in which each pixel has an RGB value by an interpolation process with respect to a Bayer signal having any RGB signal value in each pixel. The reduction processing unit 202 performs a reduction process on the image data output from the image conversion unit 201. The noise reduction unit 203 performs noise reduction processing on the image data output from the reduction processing unit 202. The edge enhancement unit 204 performs edge enhancement processing on the image data output from the noise reduction unit 203. The image conversion unit 205 converts the image data output from the edge enhancement unit 204 into a predetermined image format such as JPEG.

  Next, the reduction processing unit 202 will be described in more detail. FIG. 3 is a block diagram showing a configuration of the reduction processing unit 202 in the first embodiment. As illustrated in FIG. 3, the reduction processing unit 202 includes a reduction rate determination unit 301, a sampling phase determination unit 302, an LPF unit 303, and a thinning / interpolation processing unit 304.

  The reduction rate determination unit 301 determines the reduction rate. The reduction rate determination unit 301 may determine the reduction rate based on, for example, a user input, or may automatically determine without considering the user input by some method. The sampling phase determination unit 302 determines the sampling phase after the reduction process according to the reduction rate determined by the reduction rate determination unit 301. An LPF (low pass filter) unit 303 selects a pre-filter according to the reduction rate determined by the reduction rate determination unit 301 and uses this pre-filter to pre-filter the image data output from the image conversion unit 201. Process. This pre-filter process is, for example, a low-pass filter process, and is an example of a band limiting process before the image data reduction process. The thinning / interpolation processing unit 304 reduces the image data output from the LPF unit 303 by thinning processing or interpolation processing according to the sampling phase determined by the sampling phase determination unit 302.

  Next, an outline of the operation of the reduction processing unit 202 will be described. FIG. 4 is a flowchart showing the operation of the reduction processing unit 202 in the first embodiment.

  First, in step S401, the reduction rate determination unit 301 determines the reduction rate. As described above, the reduction rate determination unit 301 may determine the reduction rate based on the user's input, or may automatically determine without considering the user's input by some method. Next, in step S402, the LPF unit 303 selects a pre-filter according to the reduction ratio determined in step S401. In step S403, the sampling phase determination unit 302 determines a pixel sampling phase corresponding to the reduction ratio determined in step S401. Thereafter, in step S404, the LPF unit 303 performs prefiltering of the image data output from the image conversion unit 201, that is, input image data to the reduction processing unit 202, using the prefilter selected in step S402. Subsequently, in step S405, the thinning / interpolation processing unit 304 reduces the image data based on the reduction rate determined in step S401 and the pixel sampling phase determined in step S403. The process of step S403 may be performed at a timing other than between step S402 and step S404 as long as it is between step S401 and step S405.

  Here, the prefiltering process by the LPF unit 303 will be described. Here, an operation for reducing image data having RGB values for all pixels to 3/5 times and an operation for reducing image data having RGB values for all pixels to 1/2 times will be described as examples. Further, it is assumed that the pre-filter coefficient of the pre-filter most suitable for the reduction ratio of 3/5 times is [1 2 1] among the pre-filters included in the LPF unit 303 in consideration of aliasing and resolution. . Similarly, in consideration of aliasing and resolution, the prefilter coefficient of the prefilter that is most suitable for a reduction ratio of ½ of the prefilters that the LPF unit 303 has is [1 4 6 4 1]. Suppose there is. In the horizontal prefiltering process, if the prefilter coefficient is [1 2 1], each of the pixel value adjacent to the left of the pixel of interest, the pixel value of the pixel of interest, and the pixel value adjacent to the right of the pixel of interest Are averaged at a ratio of 1: 2: 1. Similarly, for the prefiltering process in the horizontal direction, if the prefilter coefficient is [1 4 6 4 1], the respective pixel values are set for five pixels arranged in the horizontal direction centered on the pixel of interest. Indicates that the averaging is performed at a ratio of 1: 4: 6: 4: 1.

When the pre-filter coefficient is a _i and the noise variance of the pixel to which the pre-filter coefficient is applied is σ _i and the noise variance of each pixel is independent, the noise after adding the pre-filter coefficient to each pixel is added. The variance is expressed as in equation (1) from the law of error propagation. It is assumed that the noise distribution follows a Gaussian distribution.

In addition, the noise standard deviation σ _p after the pre-filter processing is expressed by equation (2) when normalization of the pre-filter coefficient is taken into consideration.

Furthermore, when the standard deviation of the noise of each pixel is in the relation of the expression (3), the noise amount ratio α ₁ before and after the pre-filter process can be expressed as the expression (4). That means

This ratio α ₁ represents the noise reduction rate due to the prefiltering process. That is, the ratio α ₁ indicates how many times the noise has been reduced. When performing prefiltering with a reduction ratio of 3/5 and a prefilter coefficient of [1 2 1], the ratio α ₁ is about 0.612, and the reduction ratio is ½ and the prefilter coefficient is [1 4 When the pre-filter process of 6 4 1] is performed, the ratio α ₁ is about 0.523. That is, when the noise amount is evaluated by the expression (4), the noise is reduced when the prefiltering process with the prefilter coefficient [1 4 6 4 1] is performed. Equation (4) is a statistical representation of the noise reduction rate of the entire image, and when viewed locally, this noise reduction rate varies depending on the location of the image.

  As described above, the LPF unit 303 executes prefiltering using a prefilter most suitable for the reduction rate determined by the reduction rate determination unit 301 among the prefilters of the LPF unit 303 itself. The noise reduction rates in this prefiltering are arranged in the order of image size.

  Here, for reference, a simple decimation / interpolation process will be described. FIG. 5 is a diagram illustrating a sampling phase when an image is reduced to 3/5 times or 1/2 times.

  When the image is reduced by a factor of 2, all the pixel sampling positions of the image after the reduction process coincide with the pixel sampling positions of the image before the reduction process. For this reason, it is possible to reduce the image only by the thinning process.

On the other hand, when the image is reduced to 3/5 times, the pixel sampling position of the image after the reduction process is mixed with the pixel that matches the pixel sampling position of the image before the reduction process and the pixel that does not match. For this reason, an interpolation process is performed on the pixels whose pixel sampling position of the image after the reduction process does not coincide with the pixel sampling position of the image before the reduction process from the neighboring pixels of the image before the reduction process. Therefore, in the reduction of 3/5 times, pixels to be thinned out in a three-pixel cycle and pixels determined by the interpolation process are mixed. As a method of determining the pixel value so that the centers of gravity of the pixels of the image after the reduction processing are equally spaced, for example, a method of taking an average value by weighting neighboring pixel values by a distance ratio can be mentioned. In this method, in the case of 3/5 reduction, the sampling position of the pixel determined by the interpolation processing is a point obtained by internally dividing the sampling interval of the image before the reduction processing by 1: 2, as shown in FIG. . For this reason, for example, the pixel value is determined by performing bilinear interpolation with a weight of 1: 2. Such weighting is an example of an interpolation coefficient. By this interpolation processing, noise is reduced as in the case where the filter processing with the filter coefficient of [12] is performed. In the filtering process with a filter coefficient of [12], the ratio α ₁ (noise reduction rate) obtained by the equation (4) is about 0.745. Of the three pixels in one cycle, one pixel is thinned out, and two pixels are determined by interpolation processing. Therefore, when the noise reduction rate of the entire image is obtained as an average value, it is about 0.830. Note that when the reduction is performed only by the thinning process, the noise reduction rate is 1.

When the ratio α ₁ is used as the noise reduction rate k ₁ in the prefiltering process by the LPF unit 303 and the noise reduction rate in the subsequent thinning process is k ₂ , the noise reduction rate α by both these processes is expressed by the following equation (5). It is expressed as Therefore, the noise reduction rate α is related to a value in which the average noise amount per pixel of the image before the reduction process is used as a denominator and the average noise amount per pixel of the image after the reduction process is a numerator.

  Accordingly, when simple thinning / interpolation processing is performed, the noise reduction rate α at 3/5 reduction is about 0.508 (= 0.612 × 0.830), and at 1/2 reduction. The noise reduction rate α is about 0.523. As described above, when the noise reduction rates are arranged in the order of the image size at the time after the pre-filter processing, if the reduction is performed by simple thinning / interpolation processing thereafter, the noise reduction rate α is reversed. Sometimes.

  Therefore, in this embodiment, the thinning / interpolation processing unit 304 performs thinning / interpolation processing according to the sampling phase determined by the sampling phase determination unit 302 instead of the simple thinning / interpolation processing as described above.

For example, in the case of performing reduction of 1/2 times, the sampling phase determination unit 302 shifts the pixel sampling phase with respect to the thinning / interpolation processing unit 304 by 1/2 pixel with respect to the pixel sampling before the reduction processing. Instruct. FIG. 6 shows pixel sampling positions of the image after reduction processing in this case. Since all the pixels of the image after the reduction process exist in the middle of the pixel sampling position of the image before the reduction process, the thinning / interpolation processing unit 304 performs the interpolation process with a weight of 1: 1. At this time, the noise reduction rate k ₂ in the processing by the thinning / interpolation processing unit 304 is about 0.707, and the noise reduction rate α in the entire reduction processing unit 202 is about 0.370 (= 0.523 × 0.707). It becomes. Therefore, noise is reduced as compared with the case where the reduction is performed 3/5 times.

  As described above, the noise reduction rate α for each reduction rate when simple decimation / interpolation processing is performed is obtained from the reduction rate of the image processing apparatus and the prefilter coefficient of the LPF unit 303. Therefore, it is possible to determine in advance based on the noise reduction rate α how much the sampling phase shift in the thinning / interpolation processing unit 304 is to be arranged in order of image size. That is, what is the deviation of the sampling phase, and between any two reduction processes with different reduction ratios, the noise amount after the reduction process with a small reduction ratio is less than the noise amount after the reduction process with a large reduction ratio Can be determined in advance based on the noise reduction rate α. This noise reduction rate α is an example of a noise evaluation value. The sampling phase determination unit 302 determines a sampling phase in which such a deviation amount occurs. That is, the sampling phase determination unit 302 determines the sampling phase based on a predetermined relationship between the reduction rate and the sampling phase. The noise evaluation value is related to the interpolation coefficient and the reduction ratio in the thinning / interpolation processing unit 304. This noise evaluation value is further related to a change in the amount of noise due to the band limiting process in the LPF unit 303. In this embodiment, between any two reduction processes with different reduction ratios, the noise evaluation value in the reduction process with a small reduction ratio is equal to or less than the noise evaluation value in the reduction process with a large reduction ratio.

  According to the present embodiment, even when reduction processing is performed at a reduction ratio other than 1 / integer multiple, the amount of noise is arranged in order of image size by shifting the sampling phase at the time of reduction processing. be able to.

  In the present embodiment, the noise reduction rate by the prefiltering process is expressed as in equation (4), but the noise reduction rate may be expressed using another noise evaluation method. In this embodiment, the interpolation processing is bilinear interpolation, but other interpolation processing methods may be used.

  Further, in this embodiment, the pixel sampling phase is shifted so that the noise amount is in the order of the image size, but the pixel sampling phase is such that the noise amount is the same or the reversal of the relationship between the image size and the noise amount is not noticeable. May be shifted.

(Second Embodiment)
Next, a second embodiment will be described. In the second embodiment, the configuration of the reduction processing unit 202 is different from that of the first embodiment. Other configurations are the same as those of the first embodiment. FIG. 7 is a block diagram illustrating a configuration of the reduction processing unit 202 in the second embodiment. As shown in FIG. 7, the reduction processing unit 202 includes a reduction rate determination unit 701, a sampling phase determination unit 702, an LPF unit 703, a thinning / interpolation processing unit 704, and a noise evaluation value calculation unit 705.

  The reduction rate determination unit 701, the LPF unit 703, and the thinning / interpolation processing unit 704 perform the same operations as the reduction rate determination unit 301, the LPF unit 303, and the thinning / interpolation processing unit 304 of the first embodiment. The noise evaluation value calculation unit 705 calculates a predetermined noise evaluation value in the reduction process, for example, a noise reduction rate. The sampling phase determination unit 702 determines the sampling phase after the reduction process according to the reduction rate determined by the reduction rate determination unit 701 and the noise evaluation value calculated by the noise evaluation value calculation unit 705, for example, the noise reduction rate. .

  Next, an outline of the operation of the reduction processing unit 202 will be described. FIG. 8 is a flowchart showing the operation of the reduction processing unit 202 in the second embodiment.

  First, in step S801, the reduction rate determination unit 701 determines a reduction rate. Similar to the reduction rate determination unit 301, the reduction rate determination unit 701 may determine the reduction rate based on the user's input, or may automatically determine it without considering the user's input in some way. . Next, in step S802, the LPF unit 703 selects a prefilter corresponding to the reduction ratio determined in step S801. Thereafter, in step S803, the noise evaluation value calculation unit 705 calculates a noise evaluation value corresponding to the reduction ratio determined in step S801. Further, the sampling phase determination unit 702 determines a pixel sampling phase according to the noise evaluation value and the reduction ratio determined in step S801. Subsequently, in step S804, the LPF unit 703 performs prefiltering of the image data output from the image conversion unit 201, that is, input image data to the reduction processing unit 202, using the prefilter selected in step S802. . In step S805, the thinning / interpolation processing unit 704 reduces the image data based on the reduction rate determined in step S801 and the pixel sampling phase determined in step S803. The process of step S803 may be performed at a timing other than between step S802 and step S804 as long as it is between step S801 and step S805.

  Next, the operations of the noise evaluation value calculation unit 705 and the sampling phase determination unit 702 in step S803 will be described. FIG. 9 is a flowchart illustrating operations of the noise evaluation value calculation unit 705 and the sampling phase determination unit 702 in step S803.

  First, in step S901, the noise evaluation value calculation unit 705 determines the phase step value β. Next, in step S902, the noise evaluation value calculation unit 705 sets “0” as the initial value of the phase shift amount, and in step S903, the noise evaluation value calculation unit 705 sets the noise evaluation value, for example, in the first embodiment. The noise reduction rate α is calculated. Thereafter, in step S904, the noise evaluation value calculation unit 705 determines whether or not a predetermined condition that has a relationship between the image size and the noise evaluation value is satisfied. The certain predetermined condition is, for example, a condition that the noise reduction rate α after the reduction process is equalized or reduced as the reduction rate is smaller. If this condition is not satisfied, the noise evaluation value calculation unit 705 increases the phase shift amount by the phase step value β in step S905, returns to step S903, and calculates the noise evaluation value again. . On the other hand, if the above condition is satisfied, the sampling phase determination unit 702 determines the phase shift amount at that time as the sampling phase at the time of reduction in step S906. That is, the processing from step S903 to S905 is performed until the above condition is satisfied, and the phase shift amount when the condition is satisfied is determined as the sampling phase at the time of reduction.

  As described above, in this embodiment, the noise evaluation value calculation unit 705 shifts the phase during the reduction process for each step value, calculates the noise evaluation value for each phase shift amount, and satisfies a predetermined condition. Ask for things. As the noise evaluation value, for example, the noise reduction rate α in the first embodiment may be used, and S / N and the like listed as noise evaluation values in Patent Document 2 (Japanese Unexamined Patent Application Publication No. 2009-177244). Other variables may be used.

  Here, a specific example of the operations of the noise evaluation value calculation unit 705 and the sampling phase determination unit 702 will be described. Here again, an example of the operation when the image data is reduced to 3/5 times and the operation when the image data is reduced to 1/2 times will be described. Further, it is assumed that the phase step value β is 1/8 pixel, and the noise reduction rate α of the first embodiment is used as the noise evaluation value.

  First, in step S901, the phase step value β is determined to be 1/8. Next, in step S902, the phase shift amount is set to an initial value “0”, and in step S903, a noise evaluation value when the phase shift amount is 0 is calculated. As described in the first embodiment, the noise reduction rate α when the phase shift amount is 0 is about 0.508 when the reduction rate is 3/5, and is about 0 when the reduction rate is 1/2. .523. That is, the noise reduction rates α are not arranged in order of image size. Therefore, the determination in step S904 is “No”, and in step S905, the phase step value is shifted by 1/8 pixel as shown in FIG. 10 for the sampling phase when the reduction ratio is ½. . In step S903, the noise evaluation value is calculated again. The noise reduction rate at this time is about 0.462 from the equations (4) and (5). That is, the noise reduction rates α are arranged in order of image size. Therefore, the determination in step S904 is “Yes”, and in step S906, the sampling phase with the phase shift amount of 1/8 pixel is determined as the sampling phase during the reduction process.

  As described above, in the second embodiment, the sampling phase at the time of the reduction process can be automatically determined by the calculation performed by the noise evaluation value calculation unit 705.

  In the second embodiment, the sampling phase is determined when the noise evaluation values are arranged in the order of the image size. However, the present invention is not limited to this, and the noise evaluation value may be calculated until the phase shift amount becomes one pixel, and the phase shift amount when the noise evaluation value becomes the minimum value may be determined as the sampling phase during the reduction process. For example, when the reduction ratio is ½, as shown in FIG. 10, the noise reduction ratio is lowest when the sampling phase is shifted by ½ pixel.

(Third embodiment)
Next, a third embodiment will be described. In the third embodiment, a value different from the ratio α ₁ is used as the noise reduction rate k ₁ of the prefilter process. Other configurations are the same as those of the first embodiment. In the third embodiment, the noise reduction rate by the prefiltering process is expressed using the frequency characteristics of the filter. FIG. 11 is a diagram illustrating a frequency characteristic of a filter and a frequency weighting function. For example, the noise reduction rate k ₁ by the prefiltering process can be expressed as the following expression (6), _where g (x) is the frequency characteristic of the filter and fn is the Nyquist frequency of the image after the reduction process.

The noise reduction rate k ₁ obtained by the equation (6) corresponds to a value obtained by dividing the area of the region A in FIG. 11A by the sum of the area of the region A and the area of the region B. That is, the noise reduction rate k ₁ is an average value of gains of all frequencies up to the Nyquist frequency.

In addition, the noise reduction rate k ₁ can be expressed as in equation (7), taking into account only the gain of a specific frequency.

The noise reduction rate k ₁ obtained by the equation (7) corresponds to a value obtained by dividing the area of the region A ′ in FIG. 11B by the sum of the area of the region A ′ and the area of the region B ′. To do.

Alternatively, a weight may be determined for each frequency, and an average value of multiplication values with the frequency characteristics of the filter may be used as the noise reduction rate k ₁ . In this case, when the frequency weighting function is w (x), the noise reduction rate k ₁ can be expressed as shown in Equation (8).

As described above, it is possible to represent the noise reduction rate k ₁ from the characteristics of the prefilter, and this may be used for calculation of the noise reduction rate of the reduction processing unit 202. For the frequency weighting function w (x), for example, a large weight may be given to a frequency of noise that is easily noticeable by human eyes.

(Fourth embodiment)
Next, a fourth embodiment will be described. In the fourth embodiment, the configuration of the reduction processing unit 202 is different from that of the first embodiment. Other configurations are the same as those of the first embodiment. FIG. 12 is a block diagram illustrating a configuration of the reduction processing unit 202 in the fourth embodiment. As illustrated in FIG. 12, the reduction processing unit 202 includes a reduction rate determination unit 1201, a sampling phase determination unit 1202, an LPF unit 1203, a thinning / interpolation processing unit 1204, and a sensitivity information acquisition unit 1205.

  The reduction rate determination unit 1201, the LPF unit 1203, and the thinning / interpolation processing unit 1204 perform the same operations as the reduction rate determination unit 1201, the LPF unit 1203, and the thinning / interpolation processing unit 1204 of the first embodiment. The sensitivity information acquisition unit 1205 acquires sensitivity information at the time of shooting, that is, sensitivity information at the time of acquisition of image data, from the image sensor 102 through the control unit 106, and inputs the sensitivity information to the sampling phase determination unit 1202. The sampling phase determination unit 1202 determines the sampling phase after the reduction process according to the reduction rate determined by the reduction rate determination unit 701 and the sensitivity information input from the sensitivity information acquisition unit 1205.

  When shooting with low sensitivity, the amount of noise included in the image input to the reduction processing unit 202 is small, and the problem that the amount of noise is not arranged in order of image size is unlikely to appear in the image. On the other hand, as the shooting is performed with higher sensitivity, the amount of noise included in the image input to the reduction processing unit 202 increases, and the problem that the amounts of noise are not arranged in the order of image size becomes significant. For this reason, for example, when a phase shift process is performed on an image with a small amount of noise as in the first and second embodiments, it may appear that the resolution is only reduced. Therefore, it may be better not to perform the phase shifting process. Therefore, in the present embodiment, the operation of the sampling phase determination unit 1202 is changed according to the acquired sensitivity.

  Here, an outline of the operation of the reduction processing unit 202 will be described. FIG. 13 is a flowchart illustrating the operation of the reduction processing unit 202 according to the fourth embodiment.

  First, in step S1301, the reduction rate determination unit 1201 determines the reduction rate. Similar to the reduction rate determination unit 301, the reduction rate determination unit 1201 may determine the reduction rate based on the user's input, or may automatically determine it without considering the user's input by any method. . Next, in step S1302, the LPF unit 1203 selects a prefilter according to the reduction ratio determined in step S1301. Thereafter, the sensitivity information acquisition unit 1205 acquires sensitivity information in step S1303, and determines in step S1304 whether the sensitivity is equal to or higher than a threshold value. This threshold value may be determined by an input from the user operation unit 107, or may be automatically determined by the image processing apparatus according to the scene.

  If the sensitivity is equal to or greater than the threshold value, the process advances to step S1305, and the sampling phase determination unit 1202 determines the pixel sampling phase in the same manner as in the first embodiment. In step S1306, the LPF unit 1203 performs prefiltering of the image data output from the image conversion unit 201, that is, input image data to the reduction processing unit 202, using the prefilter selected in step S1302. In step S1307, the thinning / interpolation processing unit 1204 reduces the image data based on the reduction ratio determined in step S1301 and the pixel sampling phase determined in step S1305.

  On the other hand, if the sensitivity is less than the threshold value, the process proceeds to step S1308. In step S1308, the LPF unit 1203 uses the prefilter selected in step S1302 to output the image data output from the image conversion unit 201. Perform prefiltering. In step S1309, the thinning / interpolation processing unit 1204 reduces the image data based on the reduction ratio determined in step S1301. That is, the thinning / interpolation processing unit 1204 reduces the image data without changing the pixel sampling phase.

  According to the fourth embodiment, the operation of the sampling phase determination unit 1202 is changed according to the sensitivity information at the time of shooting, and whether or not the pixel sampling phase is changed can be appropriately switched.

  In the fourth embodiment, whether or not the sampling phase determination unit 1202 operates is switched according to the sensitivity threshold value, but the sampling phase shift amount may be changed according to the sensitivity. The operation of the reduction processing unit 202 in this case is shown in FIG.

  In this case, first, in step S1401, the reduction rate determination unit 1201 determines the reduction rate. Next, in step S1402, the LPF unit 1203 selects a pre-filter according to the reduction ratio determined in step S1301. Thereafter, in step S1403, the sensitivity information acquisition unit 1205 acquires sensitivity information and inputs the sensitivity information to the sampling phase determination unit 1202. Subsequently, in step S1404, the sampling phase determination unit 1202 determines the pixel sampling phase according to the sensitivity information input in step S1403 and the reduction ratio determined in step S1401. For example, when shooting is performed with high sensitivity, the sampling phase determination unit 1202 performs a 1/2 pixel phase shift in which the amount of noise is reduced most in 1/2 reduction, and as the shooting with low sensitivity is performed, The pixel sampling phase is determined so as to reduce the phase shift amount. The relationship between the reduction ratio, sensitivity, and shifted sampling phase may be determined in advance based on, for example, the noise reduction ratio calculated using Expressions (4) and (5). In step S1405, the LPF unit 1203 performs pre-filter processing on the input image data to the reduction processing unit 202. In step S1307, the thinning / interpolation processing unit 1204 reduces the image data.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  Each process of the present invention can also be realized by executing software (program) acquired via a network or various storage media by a processing device (CPU, processor) such as a personal computer.

  301, 701, 1201: Reduction rate determination unit 302, 702, 1202: Sampling phase determination unit 303, 703, 1203: LPF unit 304, 704, 1204: Decimation / interpolation processing unit 705: Noise evaluation value calculation unit 1205: Sensitivity information Acquisition department

Claims (12)

Reduction means for reducing the image data;
Sampling phase determining means for determining a phase of pixel sampling in the reduction processing;
Have
The reduction means determines a pixel value of the image after the reduction process by performing at least one of a pixel thinning process and an interpolation process of the image before the reduction process,
The sampling phase determining means is configured such that, between any two reduction processes with different reduction ratios performed by the reduction means, the noise amount after the reduction process with a small reduction ratio is equal to or less than the noise amount after the reduction process with a large reduction ratio. An image processing apparatus that determines a phase of the pixel sampling so that   The image processing apparatus according to claim 1, wherein the sampling phase determination unit determines a phase of the pixel sampling based on a noise evaluation value that represents a relationship between noise amounts before and after the reduction process.   The image processing apparatus according to claim 2, wherein the sampling phase determination unit determines the phase of the pixel sampling so that the noise evaluation values are arranged in order of reduction ratios.   The sampling phase determining means is configured to select the pixel so that a noise evaluation value in a reduction process with a small reduction ratio is equal to or less than a noise evaluation value in a reduction process with a high reduction ratio between any two reduction processes with different reduction ratios. The image processing apparatus according to claim 2, wherein a phase of sampling is determined.   The noise evaluation value is a value obtained by using an average noise amount per pixel of the image before the reduction process as a denominator and an average noise amount per pixel of the image after the reduction process as a numerator. The image processing apparatus according to claim 2.   The image processing apparatus according to claim 2, wherein the noise evaluation value is related to an interpolation coefficient or a reduction rate in the interpolation processing by the reduction unit. Low-pass filter processing means for performing band limiting processing of the image data before the reduction processing;
The image processing apparatus according to claim 2, wherein the noise evaluation value is related to a change in noise amount due to the band limitation process. Low-pass filter processing means for performing band limiting processing of the image data before the reduction processing;
The image processing apparatus according to claim 2, wherein the noise evaluation value is related to a coefficient or a frequency characteristic of a filter used for the band limitation process. Low-pass filter processing means for performing band limiting processing of the image data before the reduction processing;
9. The noise evaluation value according to claim 2, wherein the noise evaluation value is related to a frequency characteristic of a filter used for the band limiting process and a weight function that gives a different weight for each frequency of the filter. The image processing apparatus according to item. Sensitivity information acquisition means for acquiring sensitivity information at the time of acquisition of the image data,
The image processing apparatus according to claim 2, wherein the noise evaluation value is related to the sensitivity information. A reduction step for reducing the image data;
A sampling phase determining step for determining a phase of pixel sampling in the reduction processing;
Have
The reduction step includes a step of determining a pixel value of the image after the reduction process by performing at least one of a pixel thinning process and an interpolation process of the image before the reduction process,
In the sampling phase determination step, between any two reduction processes with different reduction ratios performed in the reduction step, a noise amount after the reduction process with a small reduction ratio is equal to or less than a noise amount after the reduction process with a large reduction ratio. An image processing method comprising a step of determining a phase of the pixel sampling so as to become. On the computer,
A reduction step for reducing the image data;
A sampling phase determining step for determining a phase of pixel sampling in the reduction processing;
And execute
The reduction step includes a step of determining a pixel value of the image after the reduction process by performing at least one of a pixel thinning process and an interpolation process of the image before the reduction process,
In the sampling phase determination step, between any two reduction processes with different reduction ratios performed in the reduction step, a noise amount after the reduction process with a small reduction ratio is equal to or less than a noise amount after the reduction process with a large reduction ratio. A program comprising the step of determining the phase of the pixel sampling so as to become.

Images