JPWO2020031362A1 - Image processing device and image processing method - Google Patents

Abstract

A composite box filter process that generates a plurality of smoothed images (D1-1 to D1-4) corresponding to a plurality of box filters by performing smoothing with a plurality of box filters having different sizes or aspect ratios from each other. Correspondence between the composite box filter processing unit (102) performed on the input image (DIN), each of the plurality of pixels included in each of the plurality of smoothed images (D1-1 to D1-4), and the input image (DIN). A composition rule specifying unit (107) that specifies a composition rule for synthesizing a plurality of smoothed images (D1-1 to D1-4) according to a difference value from the pixel to be processed, and a plurality of smoothed images (D1 to D1-4) according to the composition rule. It is provided with a compositing processing unit (108) that generates a compositing filtered image (D3) by executing a compositing process for compositing D1-1 to D1-4).

Description

The present invention relates to an image processing apparatus and an image processing method.

There is a filtering technique that preserves and smoothes edges that reduce artifacts such as image noise or compression distortion while preserving the edges of the image. In this technique, smoothing is generally performed by focusing on a certain pixel and adding and averaging the pixels existing in the rectangular region around the pixel of interest. Image filtering generally increases the computational cost depending on the filter size.

Here, using an approach called the integral image method, the sum of pixel values or the sum of squares required for calculating the mean or standard deviation (variance) of rectangular regions, which is often performed in filtering, is independent of the kernel size. In addition, it is possible to calculate at high speed. If the smoothing filter processing is composed of a combination of processing to which the integral image method can be applied, a filter with low calculation cost can be designed.

Here, in the integral image method, the filter size is fixed for the entire image. If the filter size is set large, the edge area will be strongly smoothed. On the other hand, when the filter size is set small, the smoothing effect on the edge region and the flat region becomes low. Therefore, there is a demand for a method for enhancing the smoothing effect on the edge vicinity and the flat region.

For example, in Patent Document 1, an edge image is generated in advance for an input image, the attention pixel and the attention rectangular area of the input image are scanned in the filter processing, and when the attention rectangle area includes the edge pixel, "filter" An image processing device has been proposed in which smoothing is performed even in the vicinity of complicated edges by changing any of "size", "offset", and "overlap".

JP-A-2002-135592

However, in the conventional image processing apparatus, since the filter size is reduced in the vicinity of the edge, the smoothing effect in the vicinity of the edge is lowered. In addition, the conventional image processing device sequentially determines whether or not edge pixels are included in the rectangular region of interest, and changes the filter size based on the determination result. Therefore, an integral image method for processing the entire image can be applied. However, the calculation cost increases.

Therefore, one or more aspects of the present invention are aimed at making it possible to enhance the smoothing effect on the edge vicinity and the flat region while preserving the edge or the gradient by using a plurality of box filters. To do.

The image processing apparatus according to one aspect of the present invention is a composite that generates a plurality of smoothed images corresponding to the plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios. A compound box filter processing unit that performs box filter processing on an input image, the plurality of pixels included in each of the plurality of smoothed images, and the plurality of pixels according to the difference value between the corresponding pixels of the input image. A composition rule specifying unit that specifies a composition rule for synthesizing a smoothed image of the above, and a composition processing unit that generates a composition filter processed image by executing a composition process for synthesizing the plurality of smoothed images according to the composition rule. , It is characterized in that.

The image processing method according to one aspect of the present invention is a composite that generates a plurality of smoothed images corresponding to the plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios. Box filtering is performed on the input image, and the plurality of smoothed images are combined according to the difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixels of the input image. It is characterized in that a composite filtered image is generated by specifying a composite rule to be performed and executing a composite process of synthesizing the plurality of smoothed images according to the composite rule.

According to one or more aspects of the present invention, by using a plurality of box filters, it is possible to enhance the smoothing effect on the edge vicinity and the flat region while preserving the edge or the gradient.

It is a block diagram which shows schematic structure of the image processing apparatus which concerns on Embodiment 1. FIG. (A) to (E) are schematic diagrams for explaining an integral image technique. (A) to (D) are schematic views showing a box filter. It is the schematic for demonstrating the generation method of an integral image. It is a block diagram which shows typically an example of the intermediate image composition filter processing part. (A) and (B) are block diagrams showing a hardware configuration example. It is a flowchart which shows the image processing method. It is a block diagram which shows schematic structure of the image processing apparatus which concerns on a modification. It is a block diagram which shows schematic structure of the image processing apparatus which concerns on Embodiment 2. FIG.

Embodiment 1.
FIG. 1 is a block diagram schematically showing the configuration of the image processing apparatus 100 according to the first embodiment.
The image processing device 100 includes an input image composition filter processing unit 101 and an intermediate image composition filter processing unit 110.
The input image composition filter processing unit 101 generates a composition filter processing image D3 by filtering the input image DIN. Then, the input image composition filter processing unit 101 gives the composition filter processing image D3 and the composition weight maps D2-1 to D2-4 described later to the intermediate image composition filter processing unit 110.

The input image composition filter processing unit 101 includes a composite box filter processing unit 102, a composition rule specifying unit 107, and a composition processing unit 108.

The composite box filter processing unit 102 performs the composite box filter processing on the input image DIN. The composite box processing is a process of generating a plurality of smoothed images D1-1 to D1-4 corresponding to a plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios. is there. For example, the composite box filter processing unit 102 performs edge preservation smoothing on the input image DIN by using a plurality of box filters by an integral image method. Then, the composite box filter processing unit 102 gives the smoothed images D1-1 to D1-4, which are the results of the composite box filter processing, to the synthesis rule specifying unit 107 and the synthesis processing unit 108.

First, the integral image method will be described.
The integral image expresses the correspondence at a certain coordinate (x, y) when the image consisting of the pixel values obtained by summing the pixel values from the origin of the image to a certain pixel of interest is IMG and the integral image is INTEGRAL. Shown in (1).

By using the integral image, for example, the addition value V of the arbitrary rectangular region AR in the image shown in FIG. 2 (A) is shown in FIG. 2 (B) as shown in the equation (2). ) ~ It can be calculated by addition and subtraction of the integral images II1 to II4 shown in FIG. 2 (E).

Returning to FIG. 1, the composite box filter processing unit 102 includes a large box filter processing unit 103, a vertical box filter processing unit 104, a horizontal box filter processing unit 105, and a small box filter processing unit 106.
Each of the large box filter processing unit 103, the vertical box filter processing unit 104, the horizontal box filter processing unit 105, and the small box filter processing unit 106 uses a box filter having at least one of different sizes and different aspect ratios. Edge preservation smoothing filter processing is performed by the integral image method.

3 (A) to 3 (D) are schematic views showing a box filter used in the composite box filter processing unit 102.
The large box filter processing unit 103 uses, for example, a box filter BF1 having a width × height width of M × M pixels (M is an integer of 2 or more) as shown in FIG. 3 (A). Then, the input image DIN is subjected to edge preservation smoothing filter processing to generate a smoothed image D1-1.
The vertical box filter processing unit 104 uses, for example, a box filter BF2 having a size of N × M pixels (N is an integer of 1 or more satisfying N <M) as shown in FIG. 3 (B). Then, the input image DIN is subjected to edge preservation smoothing filter processing to generate a smoothed image D1-2.

The horizontal box filter processing unit 105 performs edge preservation smoothing filter processing on the input image DIN by using the box filter BF3 having the size of M × N pixels as shown in FIG. 3C, for example. , Generates a smoothed image D1-3.
The small box filter processing unit 106 performs edge preservation smoothing filter processing on the input image DIN by using the box filter BF4 having a size of N × N pixels as shown in FIG. 3 (D), for example. , A smoothed image D1-4 is generated.

Normally, an integral image is generated by performing a pixel accumulation operation twice in the horizontal direction and the vertical direction. Therefore, in order to generate four types of rectangular size integral images, it is usually necessary to perform cumulative operations of 2 × 4 = 8 times.

However, by using the box filters BF1 to BF4 as shown in FIG. 3, the cumulative calculation of M pixels in the horizontal direction and the accumulation of M pixels in the horizontal direction are performed as shown in FIG. Cumulative calculation of M pixels in the vertical direction using the calculation result, cumulative calculation of N pixels in the vertical direction using the cumulative calculation result of M pixels in the horizontal direction, cumulative calculation of N pixels in the horizontal direction, and horizontal Six cumulative operations, a cumulative operation of M pixels in the vertical direction using the cumulative operation of N pixels in the direction and a cumulative operation of N pixels in the vertical direction using the cumulative operation of N pixels in the horizontal direction. It is possible to generate four types of rectangular-sized integral images II1 to II4.

The configuration of the plurality of box filters used in the composite box filter processing unit 102 is not limited to the box filters BF1 to BF4 shown in FIG. The plurality of box filters used in the composite box filter processing unit 102 can be freely configured by box filters having different sizes and aspect ratios of at least one of them. The composite box filter processing unit 102 uses, for example, three types of box filters, the box filters BF1 to BF3 or the box filters BF2 to BF4, to reduce the calculation time as compared with the case of using the four types of box filters BF1 to BF4. Can be reduced.

When four types of box filters BF1 to BF4 are used, the box filter BF1 is the first box filter, the box filter BF2 is the second box filter, the box filter BF3 is the third box filter, and the box filter BF4. Is also referred to as a fourth box filter.
When three types of box filters BF1 to BF3 are used, the box filter BF1 is also referred to as a first box filter, the box filter BF2 is also referred to as a second box filter, and the box filter BF3 is also referred to as a third box filter.
When three types of box filters BF2 to BF4 are used, the box filter BF4 is also referred to as a first box filter, the box filter BF2 is also referred to as a second box filter, and the box filter BF3 is also referred to as a third box filter.

Returning to FIG. 1, the synthesis rule specifying unit 107 responds to the difference value between each of the plurality of pixels included in each of the plurality of smoothed images D1-1 to D1-4 and the corresponding pixels of the input image DIN. , Specify a composition rule for synthesizing a plurality of smoothed images D1-1 to D1-4.

For example, the composition rule specifying unit 107 calculates the weights of each of the plurality of pixels included in each of the plurality of smoothed images D1-1 to D1-4 so that the smaller the difference value, the larger the weight. A plurality of composite weight maps D2-1 to D2-4 showing the calculated weights for each pixel corresponding to the plurality of smoothed images D1-1 to D1-4 are generated as a composite rule. An example of the calculation method of the composite weight maps D2-1 to D2-4 is shown below.

The coordinate value of the pixel of interest in the smoothed image D1-i (i is any of 1 to 4) is represented by X, and the difference value between the pixel value in the coordinate value X and the pixel value in the coordinate value X of the input image DIN. Is represented as diff _i (X), and the composite weight of the pixel X of interest in the composite weight map D2-i corresponding to the smoothed image D1-i is represented as W (X, i). An example is shown in equations (3), (4) and (5).

As described above, the composition rule specifying unit 107 indicates the composition weight map D2-1 showing the composition weight of each pixel of the smoothed image D1-1 and the composition weight of each pixel of the smoothed image D1-2. The composite weight map D2-2, the composite weight map D2-3 showing the composite weight of each pixel of the smoothed image D1-3, and the composite weight map D2 showing the composite weight of each pixel of the smoothed image D1-4. Generate -4. Then, the composition rule specifying unit 107 gives the composition weight maps D2-1 to D2-4 to the composition processing unit 108 and the intermediate image composition filter processing unit 110.

The compositing processing unit 108 executes a compositing process for compositing a plurality of smoothed images D1-1 to D1-4 using the corresponding compositing weight maps D2-1 to D2-4, thereby performing a compositing filter processed image D3. To generate. For example, the synthesis processing unit 108 synthesizes a plurality of smoothed images D1-1 to D1-4 by performing weighted addition averaging according to a plurality of composite weight maps D2-1 to D2-4. Specifically, the pixel value of each pixel of the smoothed image D1-1 to D1-4 is multiplied by the composite weight of the corresponding pixel in the corresponding plurality of D2-1 to D2-4, and the result is multiplied. The composite filtered image D3 is generated by adding the multiplication value for each pixel. Then, the composition processing unit 108 gives the composition filter processed image D3 to the intermediate image composition filter processing unit 110.

As described above, the composition rule specifying unit 107 calculates the composition weight maps D2-1 to D2-4, and the composition processing unit 108 performs weighted addition averaging, whereby the pseudo contour in the switching area of the box filter, etc. It is possible to avoid the occurrence of the above-mentioned artifacts and improve the image quality.

The intermediate image composite filter processing unit 110 performs the composite box filter processing and the composite processing once or a plurality of times on the input image DIN or the composite filter processed image D3 by using the composite weight maps D2-1 to D2-4. As a result, the composite image DOUT is generated.

FIG. 5 is a block diagram schematically showing an example of the intermediate image composition filter processing unit 110.
The intermediate image compositing filter processing unit 110 includes a plurality of composite box filter processing units 111-1 to 111-k (k is an integer of 2 or more) and a plurality of compositing processing units 116-1 to 116-k.

The composite box filter processing units 111-1 to 111-k include large box filter processing units 112-1 to 112-k, vertical box filter processing units 113-1 to 113-k, and horizontal box filter processing units 114-1. It is provided with ~ 114-k and small box filter processing units 115-1 to 115-k.

Each of the plurality of composite box filter processing units 111-1 to 111-k performs the same processing as the composite box filter processing unit 102 shown in FIG. 1 on the input image.
Further, each of the synthesis processing units 116-1 to 116-k also performs the same processing as the synthesis processing unit 108 shown in FIG.

In the example shown in FIG. 5, the intermediate image compositing filter processing unit 110 includes a plurality of composite box filter processing units 111-1 to 111-k and a plurality of compositing processing units 116-1 to 116-k. However, it may be provided with one composite box filter processing unit and one synthesis processing unit.

In advanced filter processing such as edge preservation smoothing filter, box filtering is performed a plurality of times inside the processing. Here, the smoothing performance in the vicinity of the edge can be improved by replacing the normal box filtering process with the compound box filtering process.

A part or all of the input image composition filter processing unit 101 and the intermediate image composition filter processing unit 110 described above may be, for example, a single circuit, a composite circuit, or a program as shown in FIG. 6A. It can also be configured by a processing circuit 10 such as an ASIC processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuits) or an FPGA (Field Programmable Gate Array).

Further, a part or all of the input image composition filter processing unit 101 and the intermediate image composition filter processing unit 110 are stored in the memory 11 and the memory 11 as shown in FIG. 6B, for example. It can be configured by a processor 12 such as a CPU (Central Processing Unit) that executes a program. Such a program may be provided through a network, or may be recorded and provided on a recording medium. That is, such a program may be provided as, for example, a program product.

Next, the image processing method performed by the image processing apparatus 100 according to the first embodiment will be described.
FIG. 7 is a flowchart showing the image processing method according to the first embodiment.
First, the composite box filter processing unit 102 receives the input of the input image DIN (S10).

Next, the composite box filter processing unit 102 performs composite box filter processing on the input image DIN, and gives a plurality of smoothed images D1-1 to D1-4 to the synthesis rule specifying unit 107 and the synthesis processing unit 108 ( S11).

Next, the synthesis rule specifying unit 107 calculates the difference value between the pixel value of each pixel of the plurality of smoothed images D1-1 to D1-4 and the pixel value of the corresponding pixel of the input image DIN, and calculates the difference value thereof. A plurality of composite weight maps D2-1 to D2-4, in which the smaller the difference value, the larger the weight, are calculated corresponding to each of the plurality of smoothed images D1-1 to D1-4 (S12). ..

Next, the synthesis processing unit 108 performs a synthesis process of synthesizing a plurality of smoothed images D1-1 to D1-4 using a plurality of composite weight maps D2-1 to D2-4, thereby performing a composite filter process. Image D3 is generated (S13).

Then, the intermediate image composition filter processing unit 110 uses the plurality of composition weight maps D2-1 to D2-4 to perform the compound box filter processing and the composition processing once for the input image DIN or the composition filter processing image D3. Alternatively, it is performed a plurality of times to generate a composite image DOUT (S14).

As described above, according to the image processing apparatus 100 according to the first embodiment, image information such as edges is subjected to smoothing filter processing that reduces artifacts such as noise or compression distortion of the image while preserving the edges of the image. By adaptively changing the filter size according to the above, it is possible to enhance the smoothing effect on the vicinity of the edge and the flat region while preserving the edge or the gradient.

Further, the composite box filter processing unit 102 is composed of a large box filter processing unit 103, a vertical box filter processing unit 104, a horizontal box filter processing unit 105, and a small box filter processing unit 106, and thus has a general square shape. Compared with the case where the box filter processing unit 103 and the small box filter processing unit 106 are configured, the filter size in the edge vicinity region can be set larger, so that the smoothing effect in the edge vicinity region can be enhanced.

Further, by setting the filter sizes of the vertical box filter processing unit 104 and the horizontal box filter processing unit 105 to N × M pixels and M × N pixels, when an integral image is generated as compared with the case where an arbitrary size is set. The amount of calculation can be reduced.

Further, since the image processing apparatus 100 according to the first embodiment has a configuration including box filter processing and four arithmetic operations of pixels, an integral image method can be applied and high-speed processing operations can be performed.

The first embodiment is configured like the image processing device 100 shown in FIG. 1, but may be configured like the image processing device 100 # shown in FIG. 8, for example.
The image processing device 100 # shown in FIG. 8 is different from the image processing device 100 shown in FIG. 1 in the input image composition filter processing unit 101 #.
In the image processing apparatus 100 # shown in FIG. 8, the input image DIN is reduced by the reduction processing unit 120 before the input image DIN is input to the composite box filter processing unit 102, thereby reducing the overall calculation cost. I'm letting you. Other configurations are the same as those of the image processing apparatus 100 shown in FIG.

Embodiment 2.
The second embodiment shows an example in which the compound box filtering process is applied to a guided filter which is a kind of edge preservation smoothing filter.

FIG. 9 is a block diagram schematically showing the configuration of the image processing apparatus 200 according to the second embodiment.
The image processing device 200 includes an input image composition filter processing unit 101, an intermediate image composition filter processing unit 110, and a calculation unit 230.

The input image composition filter processing unit 101 and the intermediate image composition filter processing unit 110 in the image processing device 200 according to the second embodiment are the input image composition filter processing unit 101 and the intermediate image composition in the image processing device 100 according to the first embodiment. This is the same as the filter processing unit 110.
However, the composition rule specifying unit 107 also gives a plurality of composition weight maps D2-1 to D2-4 to the calculation unit 230, and the composition processing unit 108 also gives the composition filter processed image D3 to the calculation unit 230, and the intermediate image. The composite filter processing unit 110 gives the composite image DOUT to the calculation unit 230.

The calculation unit 230 performs a calculation in the filter processing using the guided filter.
First, the filter processing using the guided filter will be described.

ここで、入力画像Ｉのある画素ｘを中心とした局所領域をΩとした場合、式（７）が成立する。

ここで、ｙは、Ω（ｘ）を範囲とする局所領域の画素位置を示す。When the base component that is the output of the guided filter is q and the input image DIN is I, q and I are represented by a linear relationship as shown in equation (6).

Here, when the local region centered on the pixel x with the input image I is Ω, the equation (7) holds.

Here, y indicates a pixel position in a local region having a range of Ω (x).

Next, a method for deriving the linear regression coefficients a and b will be described.
The pixel value corresponding to the pixel position x of the variance varI of the input image I is expressed by the equation (8).

Next, in order to perform the smoothing process only in the region where the dispersion value is extremely small and to preserve the texture in the other regions, the dispersion varDIN is subjected to gradation conversion based on the equation (9) to obtain the coefficient a. eps is a constant parameter that determines the degree of edge preservation.

The coefficient b is derived based on the equation (10).

The calculation unit 230 derives the guided filter output value q based on the linear regression coefficients a and b obtained by the equations (9) and (10).

Here, the composite filter processed image D3 which is the processing result of the input image synthesis filter processing unit 101 for the input image I is MBF ₀ (I), and the composite image DOUT which is the processing result of the intermediate image synthesis filter processing unit 110 is MBF. _When expressing ₁ (I), a composite box filter was applied by replacing a part of the formula (8) with the following formula (11) and replacing the formula (10) with the following formula (12). A guided filter can be realized.

Specifically, the processing corresponding to the intermediate image composition filter processing unit 110 described in the first embodiment is for the square processing of the input image of the equation (8) and the linear approximation coefficients a and b of the equation (10). It is shown that the compound box filter processing or the synthesis processing is performed as the smoothing processing. In other words, the calculation unit 230 adds the composite image DOUT and the composite filter processed image D3 as the sum of the squares of the guided filter processing, and defines the linear relationship between the input image DIN and the output image DOUT # in the guided filter processing. The composite filtered image D3 is used to calculate the approximation coefficient.

According to the image processing apparatus 200 according to the second embodiment, by applying the composite box filter processing to a powerful and high-speed guided filter as an edge preservation smoothing filter, edge preservation is performed while ensuring high calculation speed. The effect of smoothing can be further enhanced.

As described above, smoothing filter processing that reduces artifacts such as image noise or compression distortion while preserving the edges of the image, adaptively changing the filter size according to the image information such as the edges. It is possible to enhance the smoothing effect on the vicinity of the edge and the flat region.

Further, although the embodiments 1 and 2 have been described above, the present invention is not limited to the embodiments 1 and 2.
For example, the synthesis rule specifying unit 107 generates the synthesis weight maps D2-1 to D2-4 using the above equations (3) to (5), but the first and second embodiments are as described above. It is not limited to the above examples.

When the composite weight maps D2-1 to D2-4 are generated using the above equations (3) to (5), usually, the smaller the size of the box filter, the smaller the difference value between the smoothed image and the input image. Become. Therefore, in most of the pixels of the composite weight maps D2-1 to D2-4, the composite weight of the pixels of the smoothed image by the small size box filter becomes heavy.
Here, by setting an offset that decreases as the area of the box filter increases or an offset that increases as the average value of the entire difference value of the smoothed image increases, and subtracting from the calculated difference value, the edge In areas other than the neighboring area, it is possible to adjust so that the composite weight of the pixels of the smoothed image by the box filter having the largest possible size becomes heavy.

As the offset that decreases as the area of the box filter increases, for example, an offset that increases as the reciprocal of the area of the box filter increases may be used.

Specifically, when the pixel value of the coordinate X of the i-th smoothed image is LPF _i (x) and the pixel value of the coordinate X of the input image DIN is IN (X), the coordinate X of the difference value diff _i The pixel value calculation method of is shown in the equation (13).

By subtracting the offset from the calculated difference value as described above, it is possible to control so that the influence of the box filter having the largest possible size is large in the area other than the edge vicinity area, and the smoothing effect on the flat area. Can be further enhanced.

Further, the composite weight w (X, i) may have a non-linear relationship by calculating by, for example, the following equation (14), regardless of the equation (5). Here, g is a constant parameter that adjusts the relationship between w1 and w.

Further, the composition rule specifying unit 107 sets the pixel value of each pixel of the smoothed images D1-1 to D1-4 and the pixel value of the corresponding pixel of the input image DIN in the composition weight map D2-1 to D2-4. The composite weight maps D2-1 to D2-4 may be generated using the equations (15) and (16) so that the pixel value having the smallest difference value from the above is selected. In such a case, w (X, i) is set to 0 or 1.

As described above, the composition rule specifying unit 107 is a plurality of differences calculated by one pixel of the input image DIN and a plurality of pixels corresponding to one pixel in the plurality of smoothed images D1-1 to D1-4. Among the values, the pixel corresponding to the smallest difference value is specified, and a plurality of composite weight maps D2-1 to 1 to indicate the specified pixel corresponding to the plurality of smoothed images D1-1 to D1-4. D2-4 can be generated as a synthesis rule.

In this case, the composition processing unit 108 selects a plurality of pixel values of the pixels specified from the plurality of smoothed images D1-1 to D1-4 according to the plurality of composition weight maps D2-1 to D2-4. The smoothed images D1-1 to D1-4 of the above are synthesized.
As a result, the amount of calculation can be reduced as compared with the case where a plurality of box filter processing results are switched and weighted addition averaging is performed.

Even when the combined weight maps D2-1 to D2-4 are generated using the equations (15) and (16), the offset is subtracted from the difference value as shown in the equation (13). May be good.

100, 200 image processing device, 101 input image composition filtering unit, 102 compound box filtering unit, 103 large box filtering unit, 104 vertical box filtering unit, 105 horizontal box filtering unit, 106 small box filtering unit, 107 Composite rule identification unit, 108 Composite processing unit, 110 Intermediate image composition filter processing unit, 111-1 to 111-k composite box filter processing unit, 116-1 to 116-k Multiple composition processing units, 120 reduction processing unit, 230 arithmetic unit.

The image processing apparatus according to the first aspect of the present invention generates a plurality of smoothed images corresponding to the plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios from each other. The composite box filter processing unit that performs the composite box filter processing on the input image, and the offset is subtracted from the difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixels of the input image. A plurality of composite weight maps are calculated for each of the plurality of pixels so that the smaller the value is, the larger the weight is, and the calculated weights are shown for each pixel corresponding to the plurality of smoothed images. a combining rule specifying unit which generates a combining rule, by performing the weighted averaging according to the plurality of combining weights maps, running synthesis process of synthesizing the plurality of smoothed images to generate a composite filtered image A compositing processing unit is provided, and the offset becomes smaller as the area of the box filter used in generating the smoothed image including the pixel value used in calculating the difference value becomes larger. It is characterized by.
Further, the image processing apparatus according to the second aspect of the present invention performs smoothing by a plurality of box filters having different sizes or aspect ratios, thereby performing a plurality of smoothed images corresponding to the plurality of box filters. The composite box filter processing unit that performs the composite box filter processing to generate the input image, and offset from the difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixels of the input image. Among the values obtained by subtracting, the pixel corresponding to the smallest value is specified, and a plurality of composite weight maps indicating the specified pixel for each pixel are generated as a composite rule corresponding to the plurality of smoothed images. The plurality of smoothed images are obtained by selecting the pixel values of the specified pixels from the plurality of smoothed images as pixel values of the composite filtered image according to the composite rule specifying unit to be performed and the plurality of composite weight maps. The smoothed image including the pixel value used in calculating the difference value is provided with a compositing processing unit that executes a compositing process for compositing and generates the compositing filtered image. It is characterized in that the larger the area of the box filter used at the time of generation, the smaller the size.

The image processing method according to the first aspect of the present invention generates a plurality of smoothed images corresponding to the plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios. The composite box filter processing is performed on the input image, and the smaller the value obtained by subtracting the offset from the difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixel of the input image, the larger the value. A weight to be a weight is calculated for each of the plurality of pixels, and a plurality of composite weight maps showing the calculated weight for each pixel corresponding to the plurality of smoothed images are generated as a composite rule. By performing weighted addition averaging according to the plurality of composite weight maps, a composite process for synthesizing the plurality of smoothed images is executed to generate a composite filtered image, and the offset calculates the difference value. An image processing method characterized in that the larger the area of the box filter used when generating the smoothed image including the pixel values used in the case, the smaller the area .
The image processing method according to the second aspect of the present invention generates a plurality of smoothed images corresponding to the plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios. The composite box filter processing is performed on the input image, and the most of the values obtained by subtracting the offset from the difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixels of the input image. A pixel corresponding to a small value is specified, and a plurality of composite weight maps indicating the specified pixel for each pixel are generated as a composite rule corresponding to the plurality of smoothed images, and the plurality of composite weight maps are generated. According to the above, by selecting the pixel value of the specified pixel from the plurality of smoothed images as the pixel value of the composite filter processed image, a composite process for synthesizing the plurality of smoothed images is executed, and the composite filter is executed. The processed image is generated, and the offset becomes smaller as the area of the box filter used in generating the smoothed image including the pixel value used in calculating the difference value becomes larger. And.

Claims (16)

A composite box filter process that performs a composite box filter process on an input image to generate a plurality of smoothed images corresponding to the plurality of box filters by executing smoothing by a plurality of box filters having different sizes or aspect ratios from each other. Department and
Specifying a composition rule that specifies a composition rule for synthesizing the plurality of smoothed images according to a difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixels of the input image. Department and
An image processing apparatus comprising: a compositing processing unit that generates a compositing filtered image by executing a compositing process for compositing the plurality of smoothed images according to the compositing rule. The composition rule specifying unit calculates the weight of each of the plurality of pixels so that the smaller the difference value is, the larger the weight is, and the calculated weight is calculated in correspondence with the plurality of smoothed images. A plurality of composite weight maps shown for each are generated as the composite rule, and
The image processing apparatus according to claim 1, wherein the composition processing unit synthesizes the plurality of smoothed images by performing weighted addition averaging according to the plurality of composite weight maps. The composition rule specifying unit is a plurality of the difference values calculated by one pixel included in the input image and a plurality of pixels corresponding to the one pixel included in the plurality of smoothed images. Among them, the pixel corresponding to the difference value having the smallest value is specified, and a plurality of composite weight maps showing the specified pixel for each pixel corresponding to the plurality of smoothed images are generated as the composite rule. ,
The compositing processing unit selects the pixel values of the specified pixels from the plurality of smoothed images as the pixel values of the compositing filtered image according to the plurality of composite weight maps, whereby the plurality of smoothed images. The image processing apparatus according to claim 1, further comprising synthesizing. The compositing rule specifying unit calculates the weight of each of the plurality of pixels so that the smaller the value obtained by subtracting the offset from the difference value, the larger the weight, and the above-mentioned is corresponding to the plurality of smoothed images. A plurality of composite weight maps showing the calculated weights for each pixel are generated as the composite rule.
The image processing apparatus according to claim 1, wherein the composition processing unit synthesizes the plurality of smoothed images by performing weighted addition averaging according to the plurality of composite weight maps. The composition rule specifying unit is based on a plurality of difference values calculated by one pixel included in the input image and a plurality of pixels corresponding to the one pixel included in the plurality of smoothed images. Among the plurality of values obtained by subtracting the offset, the pixel corresponding to the smallest value is specified, and a plurality of composite weight maps showing the specified pixel for each pixel corresponding to the plurality of smoothed images are obtained. Generated as a composite rule,
The compositing processing unit selects the pixel values of the specified pixels from the plurality of smoothed images as the pixel values of the compositing filtered image according to the plurality of composite weight maps, whereby the plurality of smoothed images. The image processing apparatus according to claim 1, further comprising synthesizing. 4. The offset is characterized in that it becomes smaller as the area of the box filter used in generating the smoothed image including the pixel value used in calculating the difference value becomes larger. The image processing apparatus according to 5. The image according to claim 4 or 5, wherein the offset becomes larger as the average value of the difference values calculated corresponding to the pixel values of the plurality of pixels included in the smoothed image becomes larger. Processing equipment. As the plurality of box filters, the composite box filter processing unit includes a square first box filter, a rectangular second box filter, and a rectangular third box filter having an aspect ratio different from that of the second box filter. The image processing apparatus according to any one of claims 1 to 7, wherein a box filter and a square fourth box filter having a size smaller than that of the first box filter are used. The vertical width of the second box filter is the same as the vertical width of the first box filter, and the horizontal width is the same as the horizontal width of the fourth box filter.
The third box filter according to claim 8, wherein the vertical width is the same as the vertical width of the fourth box filter, and the horizontal width is the same as the horizontal width of the first box filter. Image processing device. The composite box filter processing unit, as the plurality of box filters, includes a square first box filter, a rectangular second box filter, and a rectangular third box filter having an aspect ratio different from that of the second box filter. The image processing apparatus according to any one of claims 1 to 7, wherein a box filter is used. The second box filter has the same vertical width as the first box filter, and the horizontal width is smaller than the horizontal width of the first box filter.
The image according to claim 10, wherein the third box filter has the same vertical width as the horizontal width of the second box filter and the same horizontal width as the horizontal width of the first box filter. Processing equipment. The width of the second box filter is the same as that of the first box filter, and the vertical width is larger than the vertical width of the first box filter.
The third box filter according to claim 10, wherein the width of the third box filter is the same as the height of the second box filter, and the height of the third box filter is the same as the height of the first box filter. Image processing equipment. The image processing apparatus according to any one of claims 1 to 12, wherein the composite box filter processing unit performs the smoothing using an integral image. The input image or the composite filtered image is further provided with an intermediate image composite filter processing unit that generates a composite image by executing the composite box filter process and the composite process at least once. The image processing apparatus according to any one of claims 1 to 13. As the sum of squares of the guided filter processing, the composite image and the composite filter processed image are added, and the composite filter processed image is used to calculate an approximation coefficient that defines the linear relationship between the input image and the output image in the guided filter processing. The image processing apparatus according to claim 14, further comprising a calculation unit using the above. By performing smoothing with a plurality of box filters having different sizes or aspect ratios, a composite box filter process for generating a plurality of smoothed images corresponding to the plurality of box filters is performed on the input image.
A composition rule for synthesizing the plurality of smoothed images is specified according to the difference value between each of the plurality of pixels included in each of the plurality of smoothed images and the corresponding pixels of the input image.
An image processing method characterized in that a composite filtered image is generated by executing a composite process of synthesizing the plurality of smoothed images according to the composite rule.

Images