JP2011090445A - Image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To achieve an image processing method, along with an image processing apparatus and image processing program, capable of greatly reducing a calculation amount in a Bilateral filter where smoothing processing is performed to a flat part while maintaining an edge part of an image. <P>SOLUTION: The image processing method includes: a lateral direction averaging processing section that performs an averaging processing to only pixel on lines to each lateral line of an image; and a vertical direction averaging processing section that performs an averaging processing on only pixel on lines to each vertical line of the image. In the method, the lateral averaging processing to the input image is followed by the vertical averaging processing. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that perform smoothing on the entire image while retaining edge information in the image data.

  A photographic image can be roughly divided into two types of components: an “edge component” whose signal value changes in small increments such as the contour of an object and a “flat component” whose signal value changes gently such as the surface of an object. Noise is likely to be placed on the flat portion, and generally a smoothing process is performed to remove the noise. Further, for the backlight analysis, a process of taking out a flat component of an image and dividing it into a dark part and a bright part is performed. At this time, a smoothing process is also performed.

  A typical method for smoothing is a Gaussian filter. The Gaussian filter is a process in which each pixel of the image is weighted with a peripheral pixel and the distance is averaged.

  However, the Gaussian filter has a problem that the edge component is broken by smoothing. For example, when there is a pixel distribution in a one-dimensional space as shown in FIG. 7, if a Gaussian filter is applied to this, a pixel distribution with an edge as shown in FIG. 8 will be obtained. As a result, there is a problem that the image quality in the vicinity of the edge portion is deteriorated, and correct dark and bright portions cannot be acquired.

  In order to improve such problems, a Bilateral filter was devised. The Bilateral filter is an improvement of the Gaussian filter, and the “pixel value difference weighting” is further multiplied by the “weighting in the distance direction” in the Gaussian filter with respect to the peripheral pixels of the target pixel (Non-patent Document 1). As a result, an effect of applying a Gaussian filter with pixels having close pixel values is generated, and smoothing can be performed without breaking the edge component. For example, when there is a pixel distribution in a position dimensional space as shown in FIG. 7, if a Bilateral filter is applied to this, a pixel distribution as shown in FIG. 9 is obtained. As a result, a smoothing effect is obtained on the flat portion while maintaining the edge.

  Also, Patent Document 1 discloses a process that simulates a Bilateral filter. In this method, a pixel whose signal value difference from the target pixel is within a threshold range is labeled, and a Gaussian filter is applied only to the labeled pixel.

JP 2007-48104 A (first page, FIG. 1)

C. Tomasi and R. Manduchi. Bilateral Filtering for Gray and Color Images.Proceedings of the 1998 IEEE International Conference on Computer Vision, Bombay, India.

  However, since the Bilateral filter needs to calculate the weights of the surrounding pixels for all the pixels of the image, there is a problem that the calculation amount is very large and enormous processing time is required. In addition, the labeling method described in Patent Document 1 has a problem in that the image quality is greatly affected by the selection of the threshold value during labeling.

  The present invention has been made in view of the problems of the maintenance king, and can maintain an image processing effect of the Bilateral filter and can greatly reduce the amount of calculation, an image processing method, an image processing apparatus, and an image processing program It aims at realizing.

In an image processing apparatus that performs weighted averaging with peripheral pixels by weighting according to a distance and a pixel value difference with the peripheral pixels for each pixel of the image,
A horizontal direction averaging processing unit that performs the averaging process on each line in the horizontal direction of the image only with pixels on the line;
A vertical direction averaging processing unit that performs the averaging process only on pixels on the line for each vertical line on the image;
And an averaging process in the vertical direction after performing the averaging process in the horizontal direction on the input image.

  According to the present invention, a bilateral filter in a two-dimensional space can be approximated by a two-time process of a bilateral filter in a one-dimensional space, and the amount of calculation can be greatly reduced while maintaining the effect of the original bilateral filter. Is possible.

The figure which shows the structure of the process part in 1st Embodiment. The figure which shows the process sequence of the horizontal direction Bilateral filter in 1st Embodiment. The figure which shows the process sequence of the vertical direction Bilateral filter in 1st Embodiment. The figure which shows the process sequence of weighting calculation in 1st Embodiment. The figure which shows the structure of the process part in 2nd Embodiment. The figure which shows the process sequence of the Bilateral filter in 2nd Embodiment. The figure which shows an example of distribution of a pixel signal value. The figure which shows distribution of the signal value after applying a Gaussian filter with respect to a pixel signal. The figure which shows distribution of the signal value after applying a Bilateral filter with respect to a pixel signal. The figure which shows the state which decomposed | disassembled the "weight of distance" horizontally and vertically. The figure which shows the calculation amount at the time of calculating a filter with the formula of two-dimensional space. The figure which shows the calculation amount at the time of calculating by decomposing | disassembling a filter into the formula of one-dimensional space. The figure which shows the state which decomposed | disassembled the "weight of a pixel value difference" to horizontal and vertical direction.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  A first embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 shows a system configuration of an image processing apparatus according to the first embodiment. The image processing apparatus includes an image acquisition unit 101, a horizontal one-dimensional bilateral filter processing unit 102, a vertical one-dimensional bilateral filter processing unit 103, and an image output unit 104. First, when the image acquisition unit 101 first acquires image data to be subjected to image processing from a storage medium such as a memory or an HDD, the image data is transmitted to the horizontal one-dimensional Bilateral filter processing unit 102. The horizontal one-dimensional bilateral filter processing unit 102 performs the process shown in the flowchart of FIG. 2 to perform the horizontal one-dimensional bilateral filter processing on each horizontal line of the image data, and the vertical one-dimensional bilateral filter processing. The image data is transmitted to the unit 103. Next, the vertical one-dimensional Bilateral filter processing unit 103 performs the processing shown in the flowchart of FIG. 3 to perform the vertical one-dimensional Bilateral filter processing on each vertical line of the image data. Transmit image data. The image output unit 104 outputs the image data subjected to the image processing to a designated storage medium such as a memory or HDD.

  FIG. 2 is a flowchart showing the procedure of the horizontal one-dimensional Bilateral filter processing. First, in step S201, a variable i representing the horizontal line number of the image is prepared and initialized with line number 0. In step S202, it is checked whether the variable i is less than the image height. Every time Bilateral filter processing is performed on one line, i is incremented by 1. Therefore, if Bilateral filter processing is not completed for all lines, the process proceeds to step S203. The process ends. In process S203, a variable j for scanning each pixel on the line number i of the image is prepared and initialized with pixel number 0. In step S204, it is checked whether the variable j is less than the image width. Since j is incremented by 1 every time the pixel is subjected to Bilateral filter processing later, if Bilateral filter processing is not completed for all the pixels on line i, the process proceeds to step S205, but all are completed. In the case, the process proceeds to step S208. In the process S205, the “weighting” of each peripheral pixel (pixel on the line) with respect to the pixel j is calculated according to the distance between the target pixel j and each peripheral pixel and the pixel value difference. The processing procedure for performing this calculation is shown in the flowchart of FIG. A description of FIG. 4 will be given later. In step S206, each pixel on the line i is multiplied by the “weight”, and the target pixel j is updated with the value. In process S207, the value of j is incremented to move the target pixel to the next. The above is the description of the one-dimensional Bilateral filter processing in the horizontal direction.

  FIG. 3 is a flowchart showing the procedure of one-dimensional Bilateral filter processing in the vertical direction. First, in step S301, a variable i representing the vertical line number of the image is prepared and initialized with a line number 0. In step S302, it is checked whether the variable i is less than the image width. Each time Bilateral filter processing is performed on one line, i is incremented by 1. Therefore, if Bilateral filter processing is not completed for all lines, the process proceeds to step S303. The process ends. In step S303, a variable j for scanning each pixel on the line number i of the image is prepared and initialized with pixel number 0. In step S304, it is checked whether the variable j is less than the image height. Since j is incremented by 1 every time a pixel is subjected to Bilateral filter processing later, if Bilateral filter processing is not completed for all pixels on line i, the process proceeds to step S305, but all are completed. In the case, the process proceeds to step S308. In the process S305, the “weighting” of each peripheral pixel (pixel on the line) with respect to the pixel j is calculated according to the distance between the target pixel j and each peripheral pixel and the pixel value difference. The processing procedure for performing this calculation is shown in the flowchart of FIG. A description of FIG. 4 will be given later. Next, in process S306, the “weighting” is applied to each pixel on the line i, and the target pixel j is updated with the value. In process S307, the value of j is incremented to move the pixel of interest to the next. The above is the description of the one-dimensional Bilateral filter processing in the vertical direction.

  Next, a procedure for calculating the weights of the peripheral pixels for the pixel j on the line will be described with reference to FIG. First, in step S401, a weight array for storing a weighted calculation result is prepared, and in step S402, the weight array is initialized with a weight of zero. Thereafter, weights are sequentially calculated in the direction in which the pixel number decreases from the pixel j in steps S403 to S409, and then weights are sequentially calculated in the direction in which the pixel number increases from the pixel j in steps S410 to S416.

  In process S403, a variable k for advancing the pixel number in the direction of decreasing the pixel number is prepared and initialized with j. In process S404, a variable dist representing the distance weight and a variable diff representing the pixel value difference weight are prepared and initialized with 1.0. In step S405, it is checked whether the pixel number k is 0 or more and the product of the distance weight and the pixel value difference weight is larger than a predetermined threshold. For example, 0.01 may be considered as the predetermined threshold. By introducing this threshold value, a pixel higher than the pixel of interest j and a pixel ahead of a pixel having a large pixel value difference is excluded from the target of the filter calculation, thereby realizing high speed. Next, in S406, the distance weight of the pixel k is calculated and substituted into the dist variable. The distance weight is calculated using Equation 1.

  σ is a parameter representing the spread of Gaussian. For example, σ = 5 is used. xk is the coordinate of the pixel k, and xj is the coordinate of the pixel j. Here, k and j are represented as they are. Next, in step S407, the pixel value difference weight of the pixel k is calculated and substituted into the diff variable. The pixel value difference weight is calculated using Equation 2. σ is a parameter representing the spread of Gaussian. For example, σ = 50 is used. vk represents the pixel value of the pixel k, and vj represents the pixel value of the pixel j. In step S408, the product of dist and diff is substituted into the weight array weight. In step S409, the variable k is decremented by one and the process proceeds to a smaller pixel number. When the above processes of S403 to S409 are completed for the pixels having the number j or less, the process proceeds to process S410.

  In step S410, a variable k for advancing the pixel number in the direction in which the pixel number increases is prepared and initialized with j + 1. In process S411, a variable dist representing the distance weight and a variable diff representing the pixel value difference weight are prepared and initialized to 1.0. Next, in step S412, it is checked whether the pixel number k is 0 or more and the product of the distance weight and the pixel value difference weight is larger than a predetermined threshold value. For example, 0.01 may be considered as the predetermined threshold. By introducing this threshold value, a pixel higher than the pixel of interest j and a pixel ahead of a pixel having a large pixel value difference is excluded from the target of the filter calculation, thereby realizing high speed. Next, in S413, the distance weight of the pixel k is calculated and substituted into the dist variable. The distance weight is calculated using Equation 1. σ is a parameter representing the spread of Gaussian. For example, σ = 5 is used. xk is the coordinate of the pixel k, and xj is the coordinate of the pixel j. Here, k and j are represented as they are. Next, in step S414, the pixel value difference weight of the pixel k is calculated and substituted into the diff variable. The pixel value difference weight is calculated using Equation 2.

  σ is a parameter representing the spread of Gaussian. For example, σ = 50 is used. vk represents the pixel value of the pixel k, and vj represents the pixel value of the pixel j. In step S415, the product of dist and diff is substituted into the weight array weight. In step S416, the variable k is incremented by 1, and the process proceeds to a larger pixel number. When the processes of S410 to S416 are completed for the pixels with the number j + 1 or more, the process proceeds to process S417.

  In step S417, the weight array weight is output, and the weight calculation is terminated.

  According to the first embodiment described above, the Bilateral filter in the two-dimensional space can be approximated by the two-time processing of the Bilateral filter in the one-dimensional space, and the calculation amount can be reduced while maintaining the effect of the original Bilateral filter. A significant reduction can be realized. Further, when calculating the weights of the peripheral pixels, the sequential processing is completed when the weights are equal to or less than a certain threshold value, so that the calculation amount can be further reduced.

  A first embodiment of the present invention will be described with reference to the drawings.

  FIG. 5 shows a system configuration of the image processing apparatus according to the second embodiment. This image processing apparatus includes an image acquisition unit 501, a horizontal one-dimensional bilateral filter processing unit 502, an image rotation processing unit 503, and an image output unit 504. First, when the image acquisition unit 501 first acquires image data to be subjected to image processing from a storage medium such as a memory or an HDD, the image data is transmitted to the horizontal one-dimensional Bilateral filter processing unit 502. The horizontal one-dimensional Bilateral filter processing unit 502 performs image processing in cooperation with the image rotation processing unit 503 to obtain an image processing effect equivalent to that of the first embodiment. The processing procedure is shown in the flowchart of FIG. First, a horizontal one-dimensional bilateral filter process is performed in process S601. The content of this process is the same as the process flow of FIG. 2 in the first embodiment. Next, the image is transferred to the image rotation processing unit 503, and the image is rotated 90 degrees in step S602. Then, the image is transferred again to the horizontal one-dimensional Bilateral filter processing unit, and the horizontal one-dimensional Bilateral filter processing is performed in Step S603. Thereby, the same effect as the vertical Bilateral filter in the first embodiment can be realized. Thereafter, the image data is once again transferred to the image rotation processing unit 503, and the original image is restored after being rotated by 90 degrees in step S604. Finally, the image output unit 504 outputs the image data to a storage medium such as a memory or HDD.

  According to the second embodiment described above, the same effect as that of the first embodiment can be obtained with only one bilateral filter processing unit and one image rotation processing unit. Can be reduced in size.

<Description of algorithm>
The reason why the calculation amount can be greatly reduced by the first or second embodiment will be described while explaining the algorithm.

  The Gaussian filter that is the base of the Bilateral filter takes enormous processing time like the Bilateral filter, but the Gaussian filter can greatly reduce the processing time by devising the calculation method. The weight of the Gaussian filter is calculated by Equation 4 (the coefficient is 1 for simplification).

  Here, when the pixel xm is taken as a relay point between the pixel xk and the pixel xj as shown in FIG. 10, Equation 4 can be expressed as Equation 5.

  This indicates that Formula 4 which is a formula in the two-dimensional space can be decomposed into Formula 5 in the one-dimensional space. Therefore, the same effect as the Gaussian filter in the two-dimensional space can be realized by applying the Gaussian filter in the one-dimensional space on the horizontal line and further applying the Gaussian filter in the one-dimensional space on the vertical line. Moreover, the amount of calculation corresponds to a reduction from the fourth power to the third power in the O notation (the variable N is the number of pixels on one side of the image).

  The reason why the amount of calculation can be reduced is explained in an easy-to-understand manner. FIG. 11 and FIG. 12 show examples in which the Gaussian filter is calculated by adding the values of the peripheral pixels a to k to the target pixels u and v. At this time, FIG. 11 shows the calculation procedure in the case of using the equation in the two-dimensional space (Equation 4), and FIG. Yes. In FIG. 11, the values of the pixels a to k are added to the pixel u and the pixel v, respectively, and the calculation is performed 22 times in total. On the other hand, in FIG. 12, the values of the pixels a to k are first added to the pixel g, and then the values of the pixel g are added to the pixels u and v, and the calculation amount is 13 times in total. In this way, by reusing the calculation result of the horizontal one-dimensional spatial Gaussian filter when calculating the vertical one-dimensional spatial Gaussian filter, the computational complexity can be significantly reduced compared to the two-dimensional spatial Gaussian filter. It becomes possible.

  On the other hand, a Bilateral filter cannot be decomposed into a one-dimensional product like a Gaussian filter as it is. This is because the Bilateral filter uses a weight obtained by multiplying Equation 5 by the coefficient of Equation 6.

  Equation 6 depends on the “pixel ground difference” between the pixel of interest and the surrounding pixels, and the pixel value difference has no spatial relationship, so that it can be decomposed vertically and horizontally as in FIG. Usually not possible.

  However, most of the general edge components of photographic images are characterized by being in the form of closed lines (unbroken lines). If the edge component is a closed line, the difference value between two pixels can be approximately expressed by integration of gradient data of an arbitrary path. Therefore, by calculating the pixel value difference between two pixels xk and xj via xm as shown in FIG. 13, the calculation of the weight of the pixel value difference is expressed by Equation 7 which is approximately decomposed into a one-dimensional space equation. It becomes possible.

  By disassembling in one dimension, it is possible to obtain a similar effect to the bilateral filter in the two-dimensional space by sequentially applying the bilateral filter in the one-dimensional space in the vertical and horizontal directions. The amount of calculation corresponds to a reduction from the fourth power to the third power in O notation (the variable N is the number of pixels on one side of the image). The reason why the amount of calculation can be reduced is the same as that of the Gaussian filter.

Claims (12)

In an image processing apparatus that performs weighted averaging with peripheral pixels by weighting according to a distance and a pixel value difference with the peripheral pixels for each pixel of the image,
A horizontal averaging processing unit (102) that performs the averaging process on each line in the horizontal direction of the image only with pixels on the line;
A vertical averaging unit (103) that performs the averaging process on each line in the vertical direction of the image only with pixels on the line;
And an averaging process in the vertical direction after performing the averaging process in the horizontal direction on the input image. In an image processing apparatus that performs weighted averaging with peripheral pixels by weighting according to a distance and a pixel value difference with the peripheral pixels for each pixel of the image,
A horizontal averaging processing unit (102) that performs the averaging process on each line in the horizontal direction of the image only with pixels on the line;
A vertical averaging unit (103) that performs the averaging process on each line in the vertical direction of the image only with pixels on the line;
And an averaging process in the horizontal direction after the averaging process is performed on the input image. In an image processing apparatus that performs weighted averaging with peripheral pixels by weighting according to a distance and a pixel value difference with the peripheral pixels for each pixel of the image,
A horizontal direction averaging processing unit (502) that performs the averaging process on each line of the image only with pixels on the line;
A rotation processing unit (503) for rotating the image by 90 degrees;
An image processing apparatus characterized in that after performing an averaging process on an input image in the horizontal or vertical direction, the image is rotated 90 degrees, and further, the averaging process is performed in the horizontal or vertical direction. The image processing apparatus according to claim 1, wherein
Sequentially calculating weights of peripheral pixels from the pixel adjacent to the target pixel on the line toward the edge of the image when performing averaging processing on the target pixel;
An image processing apparatus. The image processing apparatus according to claim 1, wherein
When calculating the weights of neighboring pixels sequentially, the weight calculation ends when the weight falls below a predetermined threshold,
An image processing apparatus. In the image processing method for performing weighted averaging with the surrounding pixels by weighting according to the distance and the pixel value difference with the surrounding pixels for each pixel of the image,
Performing the averaging process on each line in the horizontal direction of the image only with pixels on the line (S201 to S208);
Performing the averaging process on each line in the vertical direction of the image only with pixels on the line (S301 to S308);
An image processing method comprising: The image processing method according to claim 6,
Performing the averaging process in the vertical direction after performing the averaging process in the horizontal direction,
An image processing method characterized by the above. The image processing method according to claim 6,
Performing the averaging process in the horizontal direction after performing the averaging process in the vertical direction,
An image processing method characterized by the above. In the image processing method for performing weighted averaging with the surrounding pixels by weighting according to the distance and the pixel value difference with the surrounding pixels for each pixel of the image,
Performing the averaging process on each line of the image with only pixels on the line (S601);
Rotating the averaged image by 90 degrees (S602);
Performing the averaging process on each line of the rotated image with only pixels on the line (S603);
Rotating the image after the averaging process by -90 degrees (S604);
An image processing method comprising: The image processing method according to claim 6, wherein:
Sequentially calculating weights of peripheral pixels from the pixel adjacent to the target pixel on the line toward the edge of the image when performing averaging processing on the target pixel;
An image processing method characterized by the above. The image processing method according to claim 6, wherein:
When calculating the weights of neighboring pixels sequentially, the weight calculation ends when the weight falls below a predetermined threshold,
An image processing method characterized by the above. In an image processing program that performs weighted averaging with surrounding pixels by weighting according to the distance and pixel value difference with the surrounding pixels for each pixel of the image,
A procedure for performing the averaging process on only the pixels on the horizontal direction of each line in the image;
A procedure for performing the averaging process on each line in the vertical direction of the image only with pixels on the line;
An image processing program comprising:

Images