JP5545128B2 - Jaggy mitigation processing apparatus and jaggy mitigation processing method - Google Patents

Description

  The present invention relates to a jaggy mitigation processing apparatus and a jaggy mitigation processing method for mitigating jaggies existing in an image, and more particularly to a technique for applying a level-set filter.

  Along with the recent high performance of computers, the application fields of computer image generation, composition, and processing have become ambiguous such as various designs, arts, entertainment, and security. In general, pictures created artificially using these computers (hereinafter referred to as “artificial images”) are different from those in which landscapes that actually exist in photographs (hereinafter referred to as “natural images”). Many of them have features that are not found in natural images, such as “the contours of objects are clearly drawn with lines”.

  When dealing with these images on a computer, aliasing noise is a problem. This is generally called jaggy and refers to a phenomenon in which the contour lines and patterns of objects existing on the image become jagged. Jaggy is an unavoidable problem in computer image processing that treats images as a set of discrete values, and is one of the major causes of image blurring, especially in artificial images with clear outlines.

  There is a method of applying a low-pass filter as a method for mitigating jaggies (Patent Document 1). However, when the low-pass filter is applied, there is a problem that a clear outline that is a feature of an artificial image is blurred.

  Therefore, a method (Level-set filter) for reducing jaggedness along the outline of the contour has been proposed (Non-Patent Document 1). In this method, the contour features are extracted from the local area in the pixel, and smoothing is performed so as not to impair the features. Can do. In particular, a great effect can be obtained for an artificial image having a clear outline.

Hereinafter, the level-set filter will be described in more detail with an example. This filter applies a differential filter to the local area first containing processed becomes a pixel _{I n} this, the horizontal _{(I x,} I _xx) · Vertical _{(I y,} I _yy) · diagonal _{(I xy)} Get the direction tilt. FIG. 7 shows differential filter coefficients for obtaining the respective slopes. And the feature-value based on a curvature is calculated with the calculated | required inclination (Formula 1). I _t in Formula 1 is the characteristic quantity to the local area.

The feature amount multiplied by the weight is added to the pixel to be processed (Equation 2), and further by repeating Equation 1 and Equation 2, the local region is smoothed into a more gentle state. The weight k often takes a value of about 0.1.

  This iterative process of feature extraction / smoothing is performed by estimating the features of the smooth contour shape that would have existed and reconstructing the contours that were jagged due to discretization. To get the same effect. Therefore, it is possible to remove the jagged edges while maintaining the outline of the outline.

JP-A-3-139774

Bryan S. Morse and Duane Schwartzwald, “Image magnification using level set reconstruction”, Proc. International Conf. Computer Vision, pp.333-341, 2001.

  Here, a processing apparatus (when temporarily applied to real-time image processing) that can apply the level-set filter at high speed is considered. The center of the filtering process is an iterative process of feature quantity extraction / smoothing, which corresponds to Expression 1 and Expression 2. If this is mounted as it is as an electronic circuit, the configuration shown in FIGS. FIG. 8 shows one example of a configuration when the level-set filter 100 is made into an electronic circuit. FIG. 9 shows a processing flow of the entire processing apparatus, and FIG. 10 shows a processing flow of the level-set filter 100. First, when an image is input, the processing apparatus sequentially stores the image in the line buffer 200 in units of pixels (FIG. 9, S1 → S2). Then, after 2 lines of pixels are accumulated, the pixels for 3 lines are input to the level-set filter 100 (S3 in FIG. 9). The level-set filter 100 executes a smoothing process on the center (= target point) of the 3 × 3 region in the local region buffer 110. The smoothing processing is as shown in Equations 1 and 2, and the characteristic amount is calculated by passing the slope obtained by applying the differential filter to the local region to the division block 120 (FIG. 10, S41 → S42 → S43). ). After that, the weight k is multiplied and the result is fed back to the pixel value of the point of interest (S44 → S45 in FIG. 10). When the degree of smoothing is not sufficient (FIG. 10, S46: NO), the processing from step S41 to S45 in FIG. 10 is repeated. Each time this process is repeated, the pixel value of the target point is updated, and as a result, the local region is smoothed (S47 in FIG. 10).

  As described above, the level-set filter can mitigate jaggedness existing in the image while maintaining the outline of the outline. However, the calculation cost is high because it includes repetitive calculation and division, and it is difficult to apply to real-time image processing at present.

  The present invention has been made to solve the above-mentioned problems, and an object thereof is to provide a jaggy mitigation processing apparatus and a jaggy mitigation processing method capable of mitigating jaggies while reducing the calculation cost.

  In order to solve the above problem, the jaggy mitigation processing device according to the embodiment of the present invention repeats the calculation of the feature amount for the local region and smoothes the local region, thereby mitigating the jaggy existing in the image. And an independent calculation unit that independently calculates a portion that does not require repetition of the feature amount calculation, and a portion that requires repetition of the feature amount calculation, and a calculation result thereof. And a feature amount calculation unit that calculates the feature amount using a calculation result of the independent calculation unit, and a plurality of the feature amount calculation units are connected in series, and the calculation result of the independent calculation unit is the plurality It is characterized by being input to each of the feature quantity calculation units.

  In addition, the jaggy mitigation processing apparatus according to the embodiment of the present invention is a jaggy mitigation processing apparatus that performs jaggies mitigation processing in an image by repeating calculation of a feature amount for a local area and smoothing the local area. A feature amount calculation unit that calculates the feature amount at least once the number of repetitions; and an approximation unit that obtains the subsequent feature amount by an approximate expression using a calculation result of the feature amount calculation unit. It is characterized by.

  In addition, the jaggy mitigation processing method according to the embodiment of the present invention is a jaggy mitigation processing method that performs jaggies mitigation processing in an image by repeating feature value calculation for a local region and smoothing the local region. An independent calculation step for independently calculating a portion that does not require repetition of the calculation of the feature amount, a portion that requires repetition of the calculation of the feature amount, and a calculation result and calculation of the independent calculation step. A feature amount calculation step for calculating the feature amount using a result, wherein the feature amount calculation step is repeated a plurality of times, and a calculation result of the independent calculation step is used in each of the plurality of feature amount calculation steps. It is characterized by being able to.

  In addition, the jaggy mitigation processing method according to the embodiment of the present invention is a jaggy mitigation processing method that performs jaggies mitigation processing in an image by repeating feature value calculation for a local region and smoothing the local region. A feature amount calculating step for calculating the feature amount at the first repetition number of times, and an approximation step for obtaining the subsequent feature amount by an approximate expression using a calculation result of the feature amount calculating step. It is characterized by.

  ADVANTAGE OF THE INVENTION According to this invention, it is possible to provide the jaggy mitigation processing apparatus and jaggy mitigation processing method which can relieve jaggies, reducing calculation cost.

It is a figure which shows the Level-set filter structure in Embodiment 1 to 3. It is a figure which shows the structure of the feature-value calculation block in Embodiment 1. FIG. It is a figure which shows the structure of the feature-value calculation block in Embodiment 2. FIG. It is a figure which shows the structure of the feature-value calculation block in Embodiment 3. It is a figure which shows the Level-set filter structure in Embodiment 4. Is a graph showing the correlation of the number of repetitions and the feature amount I _t. It is a figure which shows the filter coefficient which calculates the inclination of a local region. It is a figure which shows the conventional Level-set filter structure. It is a figure which shows the processing flow of the whole processing apparatus. It is a figure which shows the processing flow of a Level-set filter.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the configuration described in this embodiment is an example for easily explaining the features of the present invention, and does not limit the scope of the invention.

(Embodiment 1)
FIG. 1 is a diagram illustrating a filter configuration of the jaggy mitigation processing apparatus according to the first embodiment. This jaggy mitigation processing apparatus is an apparatus that performs jaggies mitigation processing in an image by repeating the calculation of feature values for a local area and smoothing the local area, as shown in FIG. A buffer 20 and a level-set filter 10 are provided. The level-set filter 10 includes a local area buffer 11, an independent calculation block 12, feature quantity calculation blocks 13, 14,. The local area buffer 11 is a 3 × 3 area centered on the point of interest. The independent calculation block 12 independently calculates a portion that does not require repetition of the feature amount calculation. The feature amount calculation blocks 13, 14,... Calculate a portion that requires repeated calculation of the feature amount, and calculate the feature amount using the calculation result and the calculation result of the independent calculation block 12. A plurality of feature amount calculation blocks 13, 14,... Are connected in series, and the calculation result of the independent calculation block 12 is input to each of the feature amount calculation blocks 13, 14,.

Analyzing Equation 1 of the level-set filter, the horizontal (I _x ), vertical (I _y ), and diagonal (I _xy ) elements depend only on the initial value of the local region and are not affected by the number of iterations. . Therefore, in this embodiment, the part implemented as a loop in the conventional filter configuration (FIG. 8) is divided into “parts that do not need to be repeated” and “parts that need to be repeated”. By making the “part that does not need to be repeated” independent of the feature amount calculation, the amount of calculation of the repetition calculation can be reduced.

FIG. 2 is a diagram illustrating a configuration of the feature amount calculation block 13 according to the first embodiment. Here, the feature quantity calculation block 13 will be described as an example, but the configuration of the other feature quantity calculation blocks 14,... Is the same. As shown in this figure, the feature quantity calculation block 13 calculates only I _xx and I _yy , and I _x , I _y , I _xy , I _x ^{^ 2} and I _y ^{^ 2} are calculated by the independent calculation block 12. . In this way, it is possible to minimize the calculation cost required for repeated calculations. The division block 13a in the feature quantity calculation block 13 calculates Equation 1, but this division block 13a may have a general configuration (see FIG. 8).

  On the other hand, an upper limit is set for the number of repetitions of feature quantity calculation. The method of setting the upper limit may be a method of fixing the feature quantity calculation blocks 13, 14,... To a predetermined number, or only a part of the predetermined number of feature quantity calculation blocks 13, 14,. The method may be used. It has been experimentally found that sufficient smoothing can be obtained if a level-set filter is repeatedly applied about 2 to 4 times when processing for removing jaggy existing in an image is realized. From this, as shown in FIG. 1, the Level-set filter 10 can be configured as a serial operation, and the existing high-speed design method such as high-speed / pipeline can be applied.

  As described above, according to the jaggy mitigation processing apparatus according to the first embodiment, the filter configuration in which the “part that does not need to be repeated” is made independent of the feature amount calculation, the jaggy is mitigated while reducing the calculation cost. can do. In addition, since an upper limit is set for the number of repetitions of the feature amount calculation, the time required for the smoothing process can be grasped. As a result, it is possible to realize a filter circuit that operates at a high speed with a small circuit amount. Such a configuration is particularly suitable when applied to real-time image processing.

(Embodiment 2)
Division having the highest calculation cost in the feature quantity calculation block 13 is division. If the denominator and numerator of the division are calculated based on Equation 1, both result in a long bit number. For example, when the input pixel value is an 8-bit gradation, the final denominator calculation result is 36 bits and the numerator calculation result is 25 bits. In this state, the bit length of the divider becomes long, which is not realistic from the viewpoint of manufacturing and mounting cost. Therefore, in the present embodiment, the following method is adopted in order to significantly reduce the bit length of the divider.

1 and 3 are diagrams illustrating a filter configuration of the jaggy mitigation processing apparatus according to the second embodiment. In the second embodiment, as shown in FIG. 3, barrel shifters are incorporated before and after the divider, and division is performed after limiting the effective bit length. Suppose that the denominator and numerator values in FIG. 2 are obtained as follows.

In order to perform this division, a divider of 1000000 (20bit) ÷ 50000 (16bit) is required. If this arithmetic unit is mounted in each division block, the circuit scale becomes very large. Therefore, the idea of floating-point arithmetic is applied to perform the operation after limiting the effective bit length in advance (reference: Behrooz Parhami, “Computer Arithmetic: Algorithms and Hardware Designs”, The Oxford Series in Electrical and Computer. Engineering, Sep 1999.). If the effective bit length of the denominator numerator is limited to 8 bits in advance in Equation 3, it can be modified as follows, and the divider can be 8 bits ÷ 8 bits.

  A calculation error occurs by reducing the effective bit length to 8 bits, but some errors can be allowed considering that the output result is an image. For example, the calculation result of Expression 4 is 20.0205, but the difference from Expression 3 using a multi-bit divider is about 0.02. In other words, there is an error of 0.02 on the pixel value, and this level of error cannot be recognized in a system where the observer is a human being.

  As described above, according to the jaggy mitigation processing apparatus in the second embodiment, the bit length of the divider can be significantly reduced as compared with the first embodiment. This makes it possible to further reduce the circuit and increase the speed.

(Embodiment 3)
In the second embodiment, the configuration in which the circuit amount can be reduced by reducing the effective bit length at the time of division is shown. In the third embodiment, the circuit in the division block is replaced with multiplication to further reduce the circuit and increase the speed.

1 and 4 are diagrams illustrating a filter configuration of the jaggy mitigation processing apparatus according to the third embodiment. In the third embodiment, as shown in FIG. 4, division is executed as multiplication by storing a reciprocal (1 / divisor) of a number that becomes a divisor in the division as a table and storing it in a memory. For example, when division is performed as multiplication in the situation of Equation 3, the following is obtained.

  Here, by calculating 1/195, which is the reciprocal number, by referring to the table, an operation equivalent to Equation 2 can be realized. Note that, as described in the above-mentioned reference, by increasing the accuracy of the reciprocal table, or by using interpolation or Newton's method together, sufficient accuracy can be maintained even when division by reciprocal is executed. In addition, by applying this concept and providing a two-dimensional table with the denominator and numerator as an index when the amount of memory is sufficient, it is also possible to obtain the feature value only by referring to the memory without performing an operation. .

  As described above, according to the jaggy mitigation processing apparatus in the third embodiment, division in a division block can be replaced with multiplication. Thereby, it is possible to further reduce the circuit and increase the speed as compared with the second embodiment.

(Embodiment 4)
FIG. 5 is a diagram illustrating a filter configuration of the jaggy mitigation processing apparatus according to the fourth embodiment. In the present embodiment, replacing the number of repetitions second and subsequent characteristic amount calculation block .SIGMA.I _t approximation block 15. This configuration is intended to eliminate “repetitive calculation”, which is a major factor that makes it difficult to implement a level-set filter.

In the normal Level-The set filter, as in Equation 2, Komu adding value obtained by multiplying the weight k to the feature amount I _t calculated in each iteration stage to its original value. If this is written down, it will become like Formula 6. Furthermore, when the number of repetitions is n, it will become like Formula 7 (f (I _n ) is equivalent to Formula 1).

  If the filter is mounted with this configuration, it is necessary to take a circuit configuration including a loop shown in FIG. Even if an upper limit is set for the number of repetitions of feature quantity calculation, it is necessary to adopt a configuration in which a plurality of feature quantity calculation blocks 13 shown in FIG. 1 are connected in series.

However, the feature amount I _t and repetition count to have a certain correlation is known experimentally. Using this property, by obtaining the approximate expression rather than repeated operation of any I _t, it is possible to realize the equation 2 equivalent operations and without the repetitive operation. In other words (equivalent is Σ later .SIGMA.I _t approximation block 15) to become as follows.

Figure 6 is a graph showing an example of the correlation of the repetition count and the feature amount I _t when applying the Level-The set filter to an image. The curve L1 will be described as an example. The pixel value at which the feature value I _t0 = 26 in the first iteration changes to I _t1 = 21, I _t2 = 17,... In the second and subsequent iterations. This tendency is the same regardless of the magnitude of _It0 . In such situations, it is possible to determine the amount of change I _t by example, the following approximate expression. Note that the numerical value 0.8 in Equation 9 may be a circuit mounted as a fixed value, or may be a circuit configuration given as a parameter from the outside.

When the example of FIG. 6 is applied to Equation 9, I _t1 is I _t0 × 0.8 ¹ or 20.8, and I _t2 is I _t0 × 0.8 ² or 16.64. In this way, comparing the case where the amount of change is obtained by an approximate expression and the case where the amount of change is obtained by repeated calculation, it can be seen that a large error that can be perceived by humans does not occur. Therefore, by mounting a circuit that approximates these sums, a level-set filter 10 without repetitive calculation can be configured as shown in FIG. The total sum when the approximate expression of the amount of change is Equation 9 and the number of repetitions is n is derived as follows using a formula.

On the side of the molecule there is the calculation of the power 0.8 ^n, but can be simplified by having the answers to each of L in the table. If If the L is not taken only up to 6, may be able to have the six patterns from 0.8 ¹ to 0.8 ⁶ in the memory, it affects the amount of hardware is small.

In the example of above, were subjected to approximation of feature quantity I _t at a power of operation, the approximate expression is not limited to this equation. That is, a linear expression, a quadratic expression, or their interpolation may be used. For example, it is conceivable to approximate the graph of FIG. 6 with a linear line. If case of approximating by a linear equation of the slope -0.2, I _tL can be expressed as Equation 11, the sum of these can be expressed as Equation 12. Note that the numerical value 0.2 in Equation 12 may be a circuit mounted as a fixed value, or may be a circuit configuration given as a parameter from the outside.

  As described above, according to the jaggy mitigation processing apparatus in the fourth embodiment, iterative calculation of feature values can be replaced with an approximate expression. As a result, the time required for the calculation is fixed, and the application to the real-time image processing becomes easier.

In addition, the mounting system of each block shown in Embodiment 1-4 is not ask | required. For example, it may be implemented as an electronic circuit or a software program for DSP (Digital Signal Processor) with a limited bit length. Further, the bit lengths and approximate expressions shown in the first to fourth embodiments are provided for the purpose of explanation, and are not limited to these. A configuration in which a part of Embodiments 1 to 4 is applied in combination is also effective. For example, a configuration in which only the division block of FIG. 8 is replaced with the multiplied block 13c shown in FIG. In addition, a configuration in which the configuration of FIG. 1 and the configuration of FIG. 5 are combined, the calculation up to the second iteration is performed based on Equation 1, and the calculation after the third is approximated is also conceivable. Thereby, it is possible to make the time required for calculation constant while improving the accuracy of output. Further, in FIG. 5 was the application of the .SIGMA.I _t approximation block 15 to the configuration of Embodiment 1 (FIG. 1), it is also possible to apply the .SIGMA.I _t approximation block 15 in the conventional configuration (FIG. 8).

10 ... Level-The set filter 12 ... independently calculation block 13, 14 ... feature amount calculation block 13a, 13b, 13c ... dividing block 15 ... .SIGMA.I _t approximation block

Claims (9)

A jaggy mitigation processing device that performs mitigation processing of jaggies existing in an image by repeating the calculation of a feature amount for a local region and smoothing the local region,
An independent calculation unit that independently calculates a part that does not require repeated calculation of the feature amount;
A feature amount calculation unit that calculates a portion that requires repetition of the calculation of the feature amount, and calculates the feature amount using the calculation result and the calculation result of the independent calculation unit;
The jaggy mitigation processing apparatus, wherein a plurality of the feature quantity calculation units are connected in series, and a calculation result of the independent calculation unit is input to each of the plurality of feature quantity calculation units. The independent calculation unit calculates independently the repeating is not required portion of the calculation of the feature quantity I _t of each element of the horizontal (I _x) · Vertical (I _y) · diagonal (I _xy) represented by Formula 13 The jaggy mitigation processing apparatus according to claim 1.

  The jaggy mitigation processing apparatus according to claim 1, wherein an upper limit is set for the number of repetitions of the calculation of the feature amount.   2. The jaggy mitigation processing apparatus according to claim 1, wherein the feature amount calculation unit incorporates a barrel shifter before and after the divider and performs division after limiting the effective bit length.   2. The jaggy mitigation processing apparatus according to claim 1, wherein the feature amount calculation unit executes division as multiplication by forming a table of reciprocals of numbers that are divisors in division in a divider and storing them in a memory. . A jaggy mitigation processing device that performs mitigation processing of jaggies existing in an image by repeating the calculation of a feature amount for a local region and smoothing the local region,
A feature amount calculation unit that calculates the feature amount at least the number of times of repetition;
An approximation unit that obtains the subsequent feature value by an approximate expression using the calculation result of the feature value calculation unit;
A jaggy mitigation processing device characterized by comprising: The jaggy mitigation processing apparatus according to claim 6, wherein the approximating unit obtains the total sum when the number of repetitions is n, using Formula 14 or Formula 15.

A jaggy mitigation processing method for mitigating jaggies present in an image by repeating the calculation of a feature amount for a local region and smoothing the local region,
An independent calculation step for independently calculating a portion that does not require repetition of calculation of the feature value;
A feature amount calculating step of calculating a portion that requires repetition of the calculation of the feature amount, and calculating the feature amount using the calculation result and the calculation result of the independent calculation step;
The feature amount calculation step is repeated a plurality of times, and the calculation result of the independent calculation step is used in each of the plurality of feature amount calculation steps. A jaggy mitigation processing method for mitigating jaggies present in an image by repeating the calculation of a feature amount for a local region and smoothing the local region,
A feature amount calculating step for calculating the feature amount at least the first iteration; and
An approximation step for obtaining the subsequent feature value by an approximate expression using the calculation result of the feature value calculation step;
A jaggy mitigation processing method characterized by comprising:

Images