JP6883864B2 - Image processing equipment, image processing method, and image processing program - Google Patents

Description

The present invention relates to an image processing device for enlarging an input image with high quality, an image processing method, and a program thereof. In particular, a data-dependent triangle division method (Data) in which an input image is divided into regions by a plurality of triangles and pixels are interpolated. Dependent Triangulation (DDT method) is used to relate to an image processing apparatus capable of enlarging an input image with high quality, an image processing method, and a program thereof.

When enlarging, rotating, deforming, etc. a raster image such as a bitmap image, the angle of the edge existing in the image is calculated, and the pixels are interpolated using the vector data created based on the calculated edge angle. For example, the DDT method is known as a technique for interpolating (Non-Patent Document 1).
The DDT method divides the raster image into regions by a plurality of triangles based on the cost function and the angle of the edge. The triangular mesh data that divides the raster image into regions is vector data. In the DDT method, the pixel values of the pixels corresponding to the vertices of the triangle and the triangle mesh data which is the vector data are used to linearly interpolate the pixels in the region in the triangle mesh.
Patent Document 2 is mentioned as an image processing method in which the above DDT method is applied to image enlargement processing.

Japanese Unexamined Patent Publication No. 2018-32301

X.Yu, B.S. Morse, and T.W. Sederberg, “Image Reconstruction UsingData-Dependent Triangulation”, IEEE Computer Graphics and Applications, Vol.21, No. 3, pp. 62-68 May / Jun. (2001)

In Patent Document 1, when determining the shape of a triangular mesh, a process of confirming the consistency of each division pattern is executed after provisionally determining a division pattern that minimizes the cost function. If the connections between the divided patterns do not match, the process of modifying the patterns so that they match is executed, which may increase the processing load.
The present invention has been made in view of the above circumstances, and an object of the present invention is to suppress an increase in load in image processing.

In order to solve the above problems, the present invention is an image processing device that divides an original image composed of a plurality of original pixels into a mesh, and each pixel value in a first pixel region including the plurality of original pixels. An edge image generation means that calculates an edge angle based on the above and generates an intermediate image composed of intermediate pixels having pixel values corresponding to the calculated edge angles, and a plurality of the intermediate pixels. The intermediate pixel through which the diagonal line of the rectangular region corresponding to the selected detection pattern passes based on the detection pattern selected from the plurality of types of detection patterns of the diagonal line for detecting the diagonal line of the rectangular region to be formed. Is set as the determination target pixel , and when the edge angle indicated by all the determination target pixels is within a predetermined angle range with respect to the angle of the diagonal line corresponding to the selected detection pattern, the diagonal line is extracted. It is characterized by comprising an edge extraction means for performing.

According to one embodiment, it is possible to suppress an increase in load in image processing.

It is a block diagram which shows the functional structure of the image processing apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the hardware structure of the computer which concerns on one Embodiment of this invention. (A)-(c) is a figure explaining the outline of the edge calculation method by the image processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the intermediate image generated by the edge calculation part. It is a schematic diagram explaining an example of the diagonal line extraction method by an edge extraction part. (A) to (f) are diagrams showing an example of a diagonal detection pattern extracted as a dividing edge. (A) and (b) are schematic views explaining an example of the method of determining the angle range for determination. (A) to (d) are diagrams showing an example of the divided edges to be interpolated. It is a flowchart which showed the image processing procedure which concerns on one Embodiment of this invention.

Hereinafter, the present invention will be described in detail using the embodiments shown in the drawings. However, unless there is a specific description, the components, types, combinations, shapes, relative arrangements, etc. described in this embodiment are merely explanatory examples, not the purpose of limiting the scope of the present invention to that alone. ..

[Functional configuration]
FIG. 1 is a block diagram showing a functional configuration of an image processing apparatus according to an embodiment of the present invention.
The image processing device 100 is a device that enlarges, rotates, and transforms an input image such as a bitmap image (an example of a raster image) to generate a high-resolution or high-quality output image. The image processing device 100 divides an input image (original image) composed of a plurality of pixels into a mesh-like region by a plurality of triangles utilizing edges in the input image. The area division information for dividing the input image into areas by a triangular mesh is vector data. The image processing device 100 enlarges the input image and generates an output image based on the area division information.
The image processing device 100 includes an edge calculation unit (pixel value acquisition means, edge calculation means) 110, a mesh generation unit 120, an edge enhancement unit 130, and a rasterization unit 140.
The edge calculation unit 110 is a means for selecting a first pixel region including a plurality of pixels (original pixels) from an input image and calculating an edge angle based on the pixel value of each pixel in the first pixel region. The edge calculation unit 110 also functions as an edge image generation means for generating an intermediate image composed of intermediate pixels having pixel values corresponding to the calculated angles of each edge. The edge calculation unit 110 includes a contrast calculation unit (difference calculation means) 111, a gradient calculation unit 112, and an outliner calculation unit 113.

The mesh generation unit 120 is a means for dividing the input image into regions by a plurality of triangles based on the edge angle calculated by the edge calculation unit 110. The mesh generation unit 120 includes an edge extraction unit (edge extraction means) 121, a detection pattern storage unit 122, a mesh interpolation unit 123, and an interpolation pattern storage unit 124.
The edge enhancement unit 130 generates a sharpened image in which the edges of the input image are emphasized.
The rasterization unit 140 enlarges the sharpened image output from the edge enhancement unit 130 by using the region division information output from the mesh generation unit 120, and also uses a bilinear method, a bicubic method, or the like as necessary. Pixel interpolation is performed to generate a high-resolution output image.
Each part constituting the image processing apparatus 100 is realized by an electronic circuit (hardware), but may be realized by software executed by a general computer.

[Computer configuration]
An example of the hardware configuration of the computer when each part of the image processing apparatus shown in FIG. 1 is realized by software will be described.
FIG. 2 is a block diagram showing a hardware configuration of a computer according to an embodiment of the present invention.
The image processing device 100 includes a CPU (Central Processing Unit) 151, a ROM (Read Only Memory) 152, a RAM (Random Access Memory) 153, and a bus 154. Each component is connected by a bus 154. The image processing device 100 may include any or all of the storage device 155, the external I / F 156, and the reader / writer 157.

The CPU 151 controls the entire image processing apparatus 100 by using a program or the like stored in the ROM 152. The ROM 152 is a non-volatile storage means for storing a control program for causing the CPU 151 to execute a process of controlling the entire image processing device 100. The control program for executing the process of controlling the entire image processing device 100 may be stored in the storage device 155 included in the image processing device 100. Here, the storage device 155 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.
The control program for executing the process of controlling the entire image processing device 100 is stored in the recording medium 162, read by the reader / writer 157 included in the image processing device 100, and stored in the ROM 152 or the storage device 155 included in the image processing device 100. It may be stored in. The recording medium 162 includes, for example, an SD memory card (SD Memory Card), an FD (Floppy Disk), a CD (Compact Disc), a DVD (Digital Versatile Disk), a BD (Blu-ray (registered trademark) Disk), and a flash memory. It is a non-temporary recording medium such as.

The RAM 153 is a volatile storage means for temporarily storing programs, data, and the like read from the ROM 152. Various functions are realized by the CPU 151 reading the control program from the ROM 152 at startup, expanding it into the RAM 153, and executing the control program using the RAM 153 as a workspace.
The external I / F 156 connects the image processing device 100 and the external storage device 161. The external storage device 161 stores various data such as an input image and an output image. The external storage device 161 is, for example, an HDD, an SSD, or the like.
The edge calculation unit 110, the mesh generation unit 120, the edge enhancement unit 130, and the rasterization unit 140 shown in FIG. 1 are each realized by the CPU 151 reading the control program from the ROM 152 or the like, expanding it into the RAM 153, and executing the control program. .. Further, the detection pattern storage unit 122 and the interpolation pattern storage unit 124 are realized by the storage device 155.

[Edge calculation unit]
<Outline of processing>
3A to 3C are diagrams illustrating an outline of an edge calculation method by an image processing apparatus according to an embodiment of the present invention.
The contrast calculation unit 111 selects a first pixel region 12 having a predetermined shape formed by a plurality of pixels 11 from the input image 10 shown in FIG. 3A, and the pixel value of each pixel 11 in the first pixel region 12 (For example, brightness value) is acquired. The contrast calculation unit 111 calculates the difference between the pixel values of the pixels in the first pixel region 12, that is, the contrast C (f_cont), based on the pixel values of the pixels 11 constituting the first pixel region 12 (FIG. 3). (B)).

The gradient calculation unit 112 and the outliner calculation unit 113 calculate the angle of the edge 13 of the first pixel region 12 (FIG. 3 (c)). In this example, the gradient calculation unit 112 calculates the angle of the edge 13 in the range of 0 to 90 degrees, and the outliner calculation unit 113 calculates the direction of the edge 13 (lower right direction or lower left direction).
The gradient calculation unit 112 sets the reference area 15 (15a, 15b) including the first pixel area 12 according to the magnitude of the contrast C of the first pixel area 12 (FIG. 3 (c)). That is, the gradient calculation unit 112 sets the first pixel region 12 as the reference region 15a when the contrast C is less than the threshold value Cτ, and the second pixel wider than the first pixel region 12 when the contrast C is the threshold value Cτ or more. The area is set as the reference area 15b. The gradient calculation unit 112 acquires the pixel value of each pixel 11 of the set reference region 15, and calculates the angle of the edge 13 of the first pixel region 12 using the pixel value of each pixel in the reference region 15.
The outliner calculation unit 113 acquires the pixel value of each pixel 11 in the first pixel region 12, and calculates the orientation of the edge 13 of the first pixel region 12 using the pixel value of each pixel in the first pixel region 12.

・・・式（１）
のように算出される。なお、符号

は、小数点以下の数値を切り捨てる床関数である。
このように、コントラスト算出部１１１は、第一画素領域１２における最大輝度値と最小輝度値の差をコントラストＣとして算出することができる。 Hereinafter, the operation of each part will be described in detail with reference to specific examples.
<Contrast calculation unit>
<< In the case of 1-element image >>
As shown in FIG. 3A, the contrast calculation unit 111 sequentially sets the first pixel area 12 including a plurality of pixels 11 from the upper left of the input image 10, and each pixel included in each first pixel area 12. Get the brightness value of. The contrast calculation unit 111 calculates the contrast C of the first pixel region 12, in other words, the difference in the brightness values of the pixels in the first pixel region 12.
Here, when the first pixel area 12 is a 2 × 2 pixel area (2 × 2 matrix A) and the luminance value (component of the 2 × 2 matrix A) of each pixel is a, b, c, d, the contrast C (f_cont) is, for example,

... Equation (1)
It is calculated as. The code

Is a floor function that truncates numbers after the decimal point.
In this way, the contrast calculation unit 111 can calculate the difference between the maximum luminance value and the minimum luminance value in the first pixel region 12 as the contrast C.

・・・式（２）
のように算出される。
このように、コントラスト算出部１１１は、入力画像が、入力画像の各画素が夫々複数の要素を有した多要素画像である場合も１要素画像の場合と同様に、コントラストＣを算出することができる。 << For multi-factor images >>
Suppose that the input image is an RGB image (an example of a multi-element image, particularly a three-element image) in which each pixel has a brightness value for each element of R (Red), G (Green), and B (Blue). In the case of contrast C (f_cont), the first pixel area 12 is a 2 × 2 pixel area, and the brightness value (component of the matrix Ax) of each pixel related to the element X is ax, bx, cx, dx.

... Equation (2)
It is calculated as.
As described above, the contrast calculation unit 111 can calculate the contrast C even when the input image is a multi-element image in which each pixel of the input image has a plurality of elements, as in the case of the one-element image. it can.

<< Shape of first pixel area >>
In this example, the first pixel area is a square pixel area composed of 2 × 2 pixels, but the first pixel area may be another rectangular pixel area including a plurality of pixels. Further, the first pixel region may be a pixel region having an arbitrary shape including a plurality of pixels (for example, a circular pixel region). In this case, the contrast calculation unit 111 can set a pixel region having an arbitrary shape as the first pixel region by selecting a rectangular pixel region and masking unnecessary pixels.

<Gradient calculation unit>
As shown in FIG. 3C, the gradient calculation unit 112 sets the reference area 15 (15a, 15b) including the first pixel area 12 according to the magnitude of the contrast C. Specifically, the gradient calculation unit 112 sets the first pixel region 12 (reference region 15a) or a region wider than the first pixel region 12 (reference region 15b) as the reference region 15 according to the magnitude of the contrast C. Set. The gradient calculation unit 112 uses the brightness value of each pixel in the set reference region 15 to indicate the gradient G (f_grad, (gx)) in which the angle θ of the edge 13 of the first pixel region 12 is shown in the range of 0 degrees to 90 degrees. , Gy)) is calculated.

・・・式（３）
なお、記号「＊」は行列のたたみ込みを表し、各行列の夫々の成分の積を足し合わせることを意味する。参照領域１５ａの各画素の輝度値（行列Ａの成分）をａ，ｂ，ｃ，ｄとしてグラディエントＧ（f_grad）を具体的に示せば、

・・・式（４）
となる。式（４）に示すように、本例においてｇｘとｇｙは、共に正の数となる。 << 2 × 2 gradient function >>
The gradient calculation unit 112 sets the first pixel region 12 as the reference region 15a when the contrast C is less than the threshold value Cτ (FIG. 3C).
When the reference area 15a is a 2 × 2 pixel area and is expressed as a 2 × 2 matrix A, the gradient calculation unit 112 uses the 2 × 2 gradient function shown in the following equation (3) to move the reference area 15a in the horizontal direction. The amount of change (or rate of change) gx of the luminance value in the vertical direction and the amount of change (or rate of change) gy of the luminance value in the vertical direction are calculated.

... Equation (3)
The symbol "*" represents the convolution of the matrix, and means that the products of the components of each matrix are added together. If the gradient G (f_grad) is specifically shown with the luminance values (components of the matrix A) of each pixel in the reference region 15a as a, b, c, and d,

... Equation (4)
Will be. As shown in the formula (4), both gx and gy are positive numbers in this example.

・・・式（５） Suppose that the input image is an RGB image (an example of a multi-element image, particularly a three-element image) in which each pixel of the input image has a brightness value for each element of R (Red), G (Green), and B (Blue). If, the gradient G (f_grad) can be obtained as the sum of the gradient G related to each element X (X ∈ {R, G, B}).

... Equation (5)

<< 4 × 4 gradient function >>
The gradient calculation unit 112 sets a region (second pixel region) including the first pixel region 12 and larger than the first pixel region 12 as the reference region 15b when the contrast C is equal to or higher than the threshold value Cτ (FIG. 3 (FIG. 3). c)).
In this example, the gradient calculation unit 112 sets a 4 × 4 pixel area, which is one size larger than the first pixel area 12, as the reference area 15b.

・・・式（６） When the reference area 15b is a 4 × 4 pixel area (expressed as a 4 × 4 matrix B), the gradient calculation unit 112 is horizontal to the reference area 15b by the 4 × 4 gradient function shown in the following equation (6). The amount of change (or rate of change) gx of the luminance value in the direction and the amount of change (or rate of change) gy of the luminance value in the vertical direction are calculated.

... Equation (6)

The gradient calculation unit 112 calculates the edge angle (gradient) after multiplying the brightness value of each pixel in the reference region 15b by a weighting coefficient according to the positional relationship with the pixels included in the first pixel region 12. .. That is, the gradient calculation unit 112 multiplies the luminance value of the pixels constituting the first pixel region 12 by the weighting coefficient "+4" or "-4", and is adjacent to the first pixel region 12 in the horizontal or vertical direction. The brightness value of the pixel to be used is multiplied by the weighting coefficient "+2" or "-2", and the brightness value of the other pixels is multiplied by the weighting coefficient "+1" or "-1".
A power value of 2 is set for the weighting coefficient. By setting a power value of 2 to the weighting coefficient, a shift operation for calculating by shifting the bits can be used, and the processing speed can be improved.

・・・式（７）
即ち、グラディエント算出部１１２は、参照領域１５の水平方向における輝度値の変化率ｇｘと垂直方向における輝度値の変化率ｇｙとに基づいてエッジ１３の角度θを算出する。
なお、逆三角関数の計算は処理負荷が高いため、式（７）の代わりに、角度θの正接

・・・式（８）
を求めてもよい。
グラディエント算出部１１２は、コントラストＣが閾値Ｃτ以上の場合に、第一画素領域１２よりも広い参照領域１５ｂの各画素の輝度値に基づいてグラディエントＧを算出するため、より正確にエッジ１３の角度θを算出可能となる。また、参照領域１５ｂの各輝度値に対して、第一画素領域１２との位置関係に応じた重み付け係数を乗じることにより、より正確にエッジ１３の角度θを算出できる。 << Edge angle θ >>
The gradient calculation unit 112 uses an inverse trigonometric function to calculate the angle θ (0 ° ≦ θ ≦ 90 °) of the edge 13 of the first pixel region 12 from the gradient G (gx, gy) obtained by the formula (3) or the formula (6). ) Can be obtained.

... Equation (7)
That is, the gradient calculation unit 112 calculates the angle θ of the edge 13 based on the rate of change gx of the brightness value in the horizontal direction of the reference region 15 and the rate of change gy of the brightness value in the vertical direction.
Since the calculation of the inverse trigonometric function has a high processing load, the tangent of the angle θ is used instead of the equation (7).

... Equation (8)
May be sought.
When the contrast C is equal to or greater than the threshold value Cτ, the gradient calculation unit 112 calculates the gradient G based on the brightness value of each pixel in the reference region 15b wider than the first pixel region 12, so that the angle of the edge 13 is more accurately calculated. θ can be calculated. Further, the angle θ of the edge 13 can be calculated more accurately by multiplying each luminance value of the reference region 15b by a weighting coefficient according to the positional relationship with the first pixel region 12.

<Outlier calculation unit>
The outlier calculation unit 113 calculates an outlier (f_outlier) which is a value for estimating the strength and direction of the local edge 13 in the first pixel region 12.

・・・式（９）
のように算出される。なお、記号「＊」は行列のたたみ込みを表し、各行列の夫々の成分の積を足し合わせることを意味する。 For example, when the first pixel area 12 is a 2 × 2 pixel area (matrix A), the outlier (f_outlier) is

... Equation (9)
It is calculated as. The symbol "*" represents the convolution of the matrix, and means that the products of the components of each matrix are added together.

・・・式（１０）
即ち、アウトライヤ（f_outlier）の大きさは、斜め方向における輝度変化の大きさを示し、アウトライヤ（f_outlier）の符号は、エッジ１３の向きを表す。アウトライヤ（f_outlier）が正の数をとるときエッジ１３は右下方向を向き、負の数をとるときエッジ１３は左下方向に向いていることを示す。
グラディエントＧから求められるエッジの角度θが０〜９０度の範囲であっても、アウトライヤの値を求めることによって、エッジの角度θを０〜１８０度の範囲で特定することができる。 The outliers (f_outlier) are specifically shown with the luminance values (components of the matrix A) of each pixel in the first pixel region 12 as a, b, c, and d as follows.

... Equation (10)
That is, the magnitude of the outlier (f_outlier) indicates the magnitude of the change in brightness in the oblique direction, and the sign of the outlier (f_outlier) indicates the direction of the edge 13. When the outlier (f_outlier) takes a positive number, the edge 13 points in the lower right direction, and when the outlier (f_outlier) takes a negative number, the edge 13 points in the lower left direction.
Even if the edge angle θ obtained from the gradient G is in the range of 0 to 90 degrees, the edge angle θ can be specified in the range of 0 to 180 degrees by obtaining the outliner value.

<Intermediate image generated by the edge calculation unit>
FIG. 4 is a diagram showing an intermediate image generated by the edge calculation unit.
Finally, the edge calculation unit 110 determines the angle θ (0 to 90 degrees, or 0 to 180 degrees) of each edge 13 in each first pixel region 12, or a value indicating this (gradient (gx, gy), And an outliner) as pixel values, respectively, to generate an intermediate image 20 composed of a plurality of intermediate pixels 21. That is, the edge calculation unit 110 functions as an edge image generation means for generating an intermediate image 20 in which each pixel has edge angle information. In this example, the intermediate image 20 is composed of a plurality of images of a gradient image in which each intermediate pixel 21 has a gradient value and an outline image in which each intermediate pixel 21 has an outliner value.
The outliner and gradient values of each intermediate pixel 21 indicate the relationship between the four pixels 11 constituting the first pixel region 12 in the original image (input image 10). The intermediate image 20 is composed of a plurality of intermediate pixels 21 represented as a square region having the pixel center of each pixel 11 of the input image 10 as a vertex. Further, the intermediate image 20 has a positional relationship that is shifted by half a pixel in the lower right direction with respect to the input image 10.

<Modification example>
In the present embodiment, when setting the reference area 15 for calculating the gradient, the brightness contrast (difference in luminance value) of the first pixel region 12 is used, but the color contrast (color difference) and the like are used. The difference in pixel values other than the brightness value of the pixels may be used. Instead of the brightness contrast, a color contrast (difference in pixel values representing colors) may be used.
Unlike the above embodiment, the gradient calculation unit 112 sets a region wider than the first pixel region 12 as a reference region when the contrast C is less than the threshold value Cτ, and sets the first pixel when the contrast C is the threshold value Cτ or more. Area 12 may be set as a reference area.

[Mesh generator]
<Outline of processing>
The mesh generation unit 120 utilizes the edges in the input image 10 to divide the input image 10 into regions by a plurality of triangles. That is, the triangle that divides the input image 10 is selected so that the edge in the input image 10 becomes the edge of the triangle that divides the input image 10 into a region.
The edge extraction unit 121 is based on the edge angle θ (including a value indicating the edge angle) calculated by the edge calculation unit 110 or a value indicating this (for example, a gradient (gx, gy) and an outliner). The division edges (diagonal lines 23), which are the sides of the triangle that divides the input image 10 into regions, are sequentially extracted.
The mesh interpolation unit 123 interpolates the divided edges with respect to the incomplete mesh portion so that the entire input image 10 is divided into regions by triangles.

<Edge extraction part: Diagonal extraction method>
FIG. 5 is a schematic diagram illustrating an example of a diagonal line extraction method by the edge extraction unit. Hereinafter, an example will be described in which the rectangular region forming the diagonal line is the right 3 × 2 pixel region. The “right 3 × 2 pixel area” means a rectangular area in the drawing in which the diagonal line extends in the lower right direction and is composed of 3 × 2 pixels. When the diagonal line extends in the lower left direction in the figure, it is expressed as "left ... pixel area".
The edge extraction unit 121 selects a rectangular region 22 (here, a right 3 × 2 pixel region) formed by a plurality of intermediate pixels 21 from the intermediate image 20 shown in FIG. The edge extraction unit 121 sets an intermediate pixel through which the diagonal line 23 of the rectangular region 22 passes as a determination target pixel 24 (colored portion in the figure). When the angle θ of each edge 13 indicated by all the determination target pixels 24 in the rectangular region 22 is within a predetermined angle range with respect to the angle φ of the diagonal line 23, the edge extraction unit 121 inputs the diagonal line 23 to the input image. Is extracted as the dividing edge that constitutes the side of the triangle that divides the area.

<Detection pattern of split edge>
6 (a) to 6 (f) are diagrams showing an example of a diagonal detection pattern extracted as a dividing edge.
The detection pattern storage unit 122 (FIG. 1) stores a plurality of types of diagonal detection patterns. Specifically, the detection pattern storage unit 122 has a rectangular area 22, a right 3 × 2 pixel area, a left 3 × 2 pixel area (FIG. 6A), a right 2 × 3 pixel area, and a left 2 × 3 pixel. Region (FIG. 6 (b)), right 3 × 1 pixel region, left 3 × 1 pixel region (FIG. 6 (c)), right 1 × 3 pixel region, left 1 × 3 pixel region (FIG. 6 (d)) , Right 2 × 1 pixel area, left 2 × 1 pixel area (FIG. 6 (e)), right 1 × 2 pixel area, left 1 × 2 pixel area (FIG. 6 (f)). I remember. Since the diagonal lines of the right 1 × 1 pixel area and the left 1 × 1 pixel area are treated as the divided edges to be interpolated (see FIG. 7), they are not stored as detection patterns.
In this example, the detection pattern storage unit 122 stores the diagonal line of the rectangular area smaller than the 3 × 3 pixel area as the detection pattern, but the detection pattern storage unit 122 stores the diagonal line of the rectangular area larger than this as the detection pattern. Such a detection pattern may be stored.

The detection pattern storage unit 122 stores the detection patterns related to the diagonal lines having an angular relationship that does not overlap with each other. In particular, it is desirable that the detection pattern storage unit 122 stores a diagonal line other than the end point on the diagonal line that does not include the intersection (corner portion) of the intermediate pixels as the detection pattern. That is, the rectangular region forming the diagonal line stored as the detection pattern does not include a small rectangular region having a diagonal line at the same angle as the diagonal line of the rectangular region, and the lengths of the two sides orthogonal to each other are "elementary to each other". It is desirable that the rectangular area is related. When the lengths of the two sides are "disjoint", it means that the greatest common divisor of the lengths of the two orthogonal sides forming the rectangular region is 1.
For example, since the 6 × 4 pixel area includes four 3 × 2 pixel areas and includes the corners (intersection points) of the 3 × 2 pixel area on the diagonal line of the 6 × 4 pixel area, the detection pattern is stored. The unit 122 may store the diagonal line of the 3 × 2 pixel area as a detection pattern, but it is desirable that the diagonal line of the 6 × 4 pixel area is not stored as a detection pattern. This is because at the time of rasterization, the pixel value to be filled in the triangular mesh is determined by referring to the pixel value of the original pixel located at the apex of the triangular mesh. For example, when extracting the diagonal line of the 6 × 4 pixel area, the corner part of the 3 × 2 pixel area exists in the center of each side of the triangular mesh including the diagonal line, but the position is located in the center of each side. Since the pixel value of the original pixel is not referred to at the time of rasterization, the reproduction rate of the original image may decrease. By storing only the diagonal lines related to the rectangular regions that are "disjoint" in the detection pattern storage unit 122 as the detection pattern, the recall rate of the original image is improved at the time of rasterization, and an image closer to the original image can be obtained. It becomes possible to output.

In the detection pattern storage unit 122 (FIG. 1), the position of the determination target pixel 24 and the determination angle range (parameters a, b, outliner code) are associated with each other for the detection pattern of each diagonal line 23 for determination. It is stored as a table. Further, the detection pattern of each diagonal line 23 is associated with information relating to the position of the base point 23a of the diagonal line 23 to be detected and the position of the attention pixel 25 in which the determination is first performed among the determination target pixels 24. There is. The determination target pixel 24 having the base point 23a of the diagonal line 23 to be detected is the attention pixel 25. That is, it can be said that the detection pattern of each diagonal line 23 includes information indicating the positional relationship between the attention pixel 25 and the determination target pixel 24 other than the attention pixel 25.
The determination angle range stored in the detection pattern storage unit 122 is set so that the diagonal lines extracted by the edge extraction unit 121 do not intersect with each other.

・・・式（１１）
なお、θは各中間画素２１が示すエッジの角度である。 7 (a) and 7 (b) are schematic views illustrating an example of a method for determining an angle range for determination. Hereinafter, an example of determining an angle range for determination for detecting the diagonal line 23 related to the rectangular region 22 composed of the right 3 × 2 pixels will be described.
First, the diagonal line d1 of the rectangular area closest to the counterclockwise direction (right 2 × 1 pixel area) and the rectangular area closest to the clockwise direction (right 1 × 1 pixel area) with respect to the diagonal line 23 to be extracted. Find the diagonal line d2 of. Here, the angle of the diagonal line 23 is φ, the angle of the diagonal line d1 is α, and the angle of the diagonal line d2 is β.
Between the angle φ of the diagonal line 23 and the angle α of the diagonal line d1 divided equally (φ + α) / 2 and the angle φ of the diagonal line 23 and the angle β of the diagonal line d2 divided equally (φ + β) / 2. , The angle range for judgment.

... Equation (11)
Note that θ is the angle of the edge indicated by each intermediate pixel 21.

・・・式（１２）
しかし、逆三角関数の計算は複雑であり、これを中間画素ごとに計算する場合には、回路の複雑化と処理負荷の増大を招くという問題がある。そこで、式（１２）を以下の式（１３）ように変形して、グラディエント比（ｇｙ／ｇｘ）の下限値をパラメータａ、上限値をパラメータｂとして判定テーブル（図６）に記憶させることで、逆三角関数の計算を省略することが可能となる。

・・・式（１３） By the way, the angle θ of the edge of the intermediate pixel 21 is obtained by the following equation (12) when the gradient of the intermediate pixel 21 is (gx, gy).

... Equation (12)
However, the calculation of the inverse trigonometric function is complicated, and when this is calculated for each intermediate pixel, there is a problem that the circuit becomes complicated and the processing load increases. Therefore, the equation (12) is transformed into the following equation (13) and stored in the determination table (FIG. 6) with the lower limit value of the gradient ratio (gy / gx) as the parameter a and the upper limit value as the parameter b. , It is possible to omit the calculation of the inverse trigonometric function.

... Equation (13)

In the detection pattern shown in FIG. 6, the parameters a and b for determining each diagonal line (angle φ) can be set to arbitrary values satisfying a <tanφ and tanφ <b. Further, the angle range for determining each diagonal line may have a portion overlapping between the parameters for determining different diagonal lines. For example, in the detection pattern shown in FIG. 6, the parameter a in the left and right 3 × 1 pixel area shown in FIG. 6 (c) and the parameter b in the left and right 1 × 3 pixel area shown in FIG. 6 (d) are not determined. The edge judgment range is expanded so that the edge range does not occur. Further, since the diagonal lines of the left and right 2 × 1 pixel areas shown in FIG. 6 (e) and the diagonal lines of the left and right 1 × 2 pixel areas shown in FIG. 6 (f) are less likely to be erroneously detected, the angle range for determination is expanded. doing.
Further, in the left and right 3 × 1 pixel area and the left and right 1 × 3 pixel area, in order to prevent erroneous detection of the horizontal edge and the vertical edge, when the y component gy of the gradient is zero, it is a diagonal line. Is set not to be extracted.

<Diagonal detection procedure>
The edge extraction unit 121 executes the division edge extraction process so as to preferentially extract the diagonal line related to the rectangular area having a large area over the diagonal line related to the rectangular area having a small area. That is, the edge extraction unit 121 reads the detection pattern from the detection pattern storage unit 122, and executes the division edge extraction process so as to extract the diagonal lines in the order shown in FIGS. 6A to 6F. The edge extraction unit 121 can extract the diagonal lines so that the divided edges do not intersect by extracting the diagonal lines related to the rectangular area having a large area in preference to the diagonal lines related to the rectangular area having a small area.

Specifically, as shown in FIG. 5, the edge extraction unit 121 selects one intermediate pixel 21 in order from, for example, the upper left of the intermediate image 20 (FIG. 4), and sets it as the attention pixel 25. The edge extraction unit 121 compares the angle θ of the edge 13 of the pixel of interest 25 and the angle range for determining the diagonal line 23 in order from the detection pattern of the diagonal line 23 related to the rectangular region 22 having a large area. When the angle θ of the edge of the pixel of interest 25 is included in the angle range for determination of a certain diagonal line 23, the edge extraction unit 121 sets the determination target pixel 24 other than the pixel of interest 25 based on the detection pattern of the diagonal line 23. To do. The edge extraction unit 121 compares the angle θ of the edge 13 of the newly set determination target pixel 24 with the angle range for determination. When the angle θ of the edge 13 indicated by all the determination target pixels 24 is within the angle range for determination, the edge extraction unit 121 divides the diagonal line 23 into a triangle that divides the input image 10 (FIG. 3A) into a region. It is extracted as a dividing edge that constitutes the side of.
If the angle θ of the edge 13 indicated by any of the determination target pixels 24 deviates from the angle range for determination with respect to the angle φ of the diagonal line 23, the edge extraction unit 121 determines the angle θ of the edge 13 of the pixel of interest 25. Is compared with the angle range for determining the diagonal line relating to the rectangular region having the next largest area. The edge extraction unit 121 newly sets the determination target pixel 24 other than the attention pixel 25 based on the comparison result.
The edge extraction unit 121 extracts the diagonal line 23 as the divided edge from the entire region of the intermediate image 20 by sequentially executing the above processes.

<Mesh interpolation section>
Since a complete mesh cannot be formed only by the diagonal line 23 extracted by the edge extraction unit 121, the mesh interpolation unit 123 interpolates the divided edges for the portion where the mesh is incomplete, and divides the entire input image 10 into regions by triangles.
8 (a) to 8 (d) are diagrams showing an example of the divided edges to be interpolated. The divided edges shown in this figure are stored in the interpolation pattern storage unit 124 (FIG. 1).
The edge extraction unit 121 can divide the entire input image 10 into regions by triangles by inserting the divided edges in the order of FIGS. 8A to 8D for the portion where the mesh is incomplete. However, the edge extraction unit 121 inserts the illustrated divided edge only when it does not intersect the already extracted divided edge.
The edge extraction unit 121 of the mesh generation unit 120 may extract the divided edges by using the edge angle (or gradient value) of the original pixel calculated by a method other than the above.

〔flowchart〕
FIG. 9 is a flowchart showing an image processing procedure according to an embodiment of the present invention.
In step S1, the image processing device 100 acquires the input image 10 (FIG. 3A). The input image 10 is input to the edge calculation unit 110 and the edge enhancement unit 130.
In step S3, the contrast calculation unit 111 sequentially sets the first pixel region 12 including the plurality of pixels 11, and acquires the brightness value of each pixel 11 included in each first pixel region 12. The contrast calculation unit 111 calculates the contrast C of the first pixel region 12 based on the brightness value of each pixel 11 included in the first pixel region 12 (FIG. 3B).
In step S5, the gradient calculation unit 112 sets the reference area 15 based on the magnitude of the contrast C of the first pixel area 12. That is, the gradient calculation unit 112 sets the first pixel region 12 as the reference region 15a when the contrast C is less than the threshold value Cτ, and includes the first pixel region 12 when the contrast C is the threshold value Cτ or more, and is the first. A region larger than the pixel region 12 (second pixel region) is set as the reference region 15b (FIG. 3 (c)).

In step S7, the gradient calculation unit 112 acquires the brightness value of each pixel 11 included in the set reference area 15. The gradient calculation unit 112 calculates the gradient G (gx, gy) of the first pixel region 12 based on the brightness value of each pixel 11 included in the reference region 15. The gradient G is a value indicating the angle θ of the edge 13 existing in the first pixel region 12 in the range of 0 to 90 degrees. By this processing, the gradient calculation unit 112 generates a gradient image (intermediate image) composed of a plurality of pixels having the gradient G of each first pixel region 12 as a pixel value.
In step S9, the outliner calculation unit 113 calculates an outliner which is a value for estimating the strength and direction of the local edge 13 in the first pixel region 12. The outliner calculation unit 113 calculates the outliner based on the brightness value of each pixel 11 included in the first pixel region 12. By this processing, the outliner calculation unit 113 generates an outliner image (intermediate image) composed of a plurality of pixels having an outliner of each first pixel region 12 as a pixel value.
The edge calculation unit 110 outputs an intermediate image 20 (FIG. 4) in which each pixel has gradient G and outliner information to the mesh generation unit 120.

In step S11, the edge extraction unit 121 of the mesh generation unit 120 receives the input image 10 based on the angle θ of the edge 13 indicated by each intermediate pixel 21 of the intermediate image 20 generated by the gradient calculation unit 112 and the outliner calculation unit 113. Extract the dividing edges that make up the sides of the triangle that divides the area. That is, as shown in FIG. 5, the edge extraction unit 121 sets a rectangular region 22 formed by a plurality of intermediate pixels 21, and all of the determination target pixels 24 through which the diagonal line 23 of the rectangular region 22 passes pass through the diagonal line 23. When it is within a predetermined angle range including, the diagonal line 23 is extracted as a dividing edge constituting the side of the triangle that divides the input image 10 into a region. In this process, the edge extraction unit 121 extracts the diagonal line 23 related to the rectangular area 22 having a large area with priority over the diagonal line 23 related to the rectangular area 22 having a small area (a) to 6 (f). Diagonal lines are extracted in the order of the detection patterns shown in.

In step S13, the mesh interpolation unit 123 of the mesh generation unit 120 reads out the divided edge to be interpolated from the interpolation pattern storage unit 124, and sets the divided edge for the portion where the mesh is incomplete (a) to 8 (d). Interpolate in the order of, and the entire input image 10 is divided into regions by triangles. The area division information for dividing the input image 10 into areas is vector data, and is information that does not lose its sharpness even when enlarged or reduced.
In step S15, the edge enhancement unit 130 generates a sharpened image in which the edge of the input image 10 is emphasized and sharpened.
In step S17, the rasterization unit 140 enlarges the sharpened image output from the edge enhancement unit 130 by using the region division information output from the mesh generation unit 120. Further, the rasterization unit 140 determines the pixel value for filling the inside of the triangle mesh by using the pixel value of the pixel 11 corresponding to the apex of the triangle mesh, and fills the inside of the triangle mesh using the determined pixel value. Further, the rasterization unit 140 performs pixel interpolation using a bilinear method, a bicubic method, or the like as necessary to generate a high-resolution output image.

[Summary of Examples of Embodiments of the Present Invention, Actions, and Effects]
<First embodiment>
This aspect is an image processing device 100 that divides an original image (input image 10) composed of a plurality of original pixels (pixels 11) into a mesh, and each pixel in the first pixel region 12 including the plurality of original pixels. An edge image generation means (edge calculation unit 110) that calculates an edge angle θ based on the values and generates an intermediate image 20 composed of intermediate pixels 21 having pixel values corresponding to the calculated edge angles. ) And the intermediate pixel through which the diagonal line 23 of the rectangular region 22 formed by the plurality of intermediate pixels passes is defined as the determination target pixel 24, and the edge angles indicated by all the determination target pixels are in a predetermined angle range with respect to the diagonal angle. It is characterized by comprising an edge extraction means (edge extraction unit 121) for extracting the diagonal line when it is inside.
According to this aspect, since it is not necessary to achieve consistency of the division pattern by the mesh, it is possible to suppress an increase in the load in the image processing.

<Second embodiment>
In the image processing apparatus 100 according to this aspect, the edge extraction means (edge extraction unit 121) is characterized in that the diagonal line 23 related to the rectangular region 22 having a large area is preferentially extracted over the diagonal line related to the rectangular region having a small area. To do.
According to this aspect, the diagonal lines can be extracted so that the edges do not intersect.

<Third embodiment>
The image processing apparatus 100 according to this embodiment stores the detection pattern of each diagonal line 23 in association with the position of the determination target pixel 24 and the determination angle range (parameters a and b, outliner code). A means (detection pattern storage unit 122) is provided, and the storage means stores detection patterns related to diagonal lines having an angular relationship that does not overlap with each other.
According to this aspect, the diagonal lines can be extracted so that the edges do not intersect. That is, since it is not necessary to ensure the consistency of the division pattern by the mesh, it is possible to suppress an increase in the load in the image processing.

<Fourth Embodiment>
In the image processing apparatus 100 according to this aspect, the storage means (detection pattern storage unit 122) is characterized in that a detection pattern that does not include the intersection of the intermediate pixels 21 is stored on the diagonal line 23 other than the end point.
At the time of rasterization, the inside of the triangular mesh is filled with reference to the pixel value of the original pixel located at the apex of the triangular mesh. When the rectangular area forming the diagonal line to be extracted contains a plurality of small rectangular areas similar to this, the pixel of the original pixel corresponding to the corner part of each small rectangular area located in the middle of each side of the triangular mesh. There is a problem that the value is not reflected and the reproducibility of the original image is lowered.
According to this aspect, it is possible to generate an image having high reproducibility with respect to the original image.

<Fifth embodiment>
In the image processing apparatus 100 according to this aspect, the angle range (parameters a, b, outliner code) for determination is set so that the diagonal lines 23 extracted by the edge extraction means (edge extraction unit 121) do not intersect with each other. It is characterized by being.
According to this aspect, since it is not necessary to achieve consistency of the division pattern by the mesh, it is possible to suppress an increase in the load in the image processing.

10 ... Input image, 11 ... Pixel, 12 ... First pixel area, 13 ... Edge, 15 ... Reference area, 15a ... Reference area, 15b ... Reference area (second pixel area), 20 ... Intermediate image, 21 ... Intermediate pixel , 22 ... Rectangular area, 23 ... Diagonal line, 23a ... Base point, 24 ... Pixel to be determined, 25 ... Pixel of interest, 100 ... Image processing device, 110 ... Edge calculation unit (pixel value acquisition means, edge calculation means), 111 ... Contrast Calculation unit (difference calculation means), 112 ... gradient calculation unit, 113 ... outliner calculation unit, 120 ... mesh generation unit, 121 ... edge extraction unit, 122 ... detection pattern storage unit, 123 ... mesh interpolation unit, 124 ... interpolation pattern storage Unit, 130 ... Edge enhancement part, 140 ... Rasterize part, 151 ... CPU, 152 ... ROM, 153 ... RAM, 154 ... Bus, 155 ... Storage device, 156 ... External I / F, 157 ... Reader / writer, 161 ... External storage Device, 162 ... Recording medium, d1, d2 ... Diagonal

Claims (7)

An image processing device that divides an original image composed of a plurality of original pixels into meshes.
An intermediate image composed of intermediate pixels having pixel values corresponding to the calculated edge angles calculated by calculating the edge angle based on each pixel value in the first pixel region including the plurality of original pixels. Edge image generation means to generate
A diagonal line of a rectangular area corresponding to the selected detection pattern based on the detection pattern selected from a plurality of types of detection patterns of the diagonal line for detecting the diagonal line of the rectangular area formed by the plurality of intermediate pixels. The intermediate pixel through which is passed is set as the determination target pixel, and the angle of the edge indicated by all the determination target pixels is within a predetermined angle range with respect to the angle of the diagonal line corresponding to the selected detection pattern. When the edge extraction means for extracting the diagonal line and
An image processing device comprising. The image processing apparatus according to claim 1, wherein the edge extraction means extracts the diagonal line of the rectangular region having a large area in preference to the diagonal line of the rectangular region having a small area. Each of the diagonal detection patterns is provided with a storage means that stores the determination target pixel and the predetermined angle range used by the edge extraction means as a determination angle range in association with each other.
The image processing apparatus according to claim 1 or 2, wherein the storage means stores the detection patterns related to the diagonal lines that do not overlap each other. The image processing apparatus according to claim 3, wherein the storage means stores the detection pattern that does not include the intersection of the intermediate pixels on the diagonal line other than the end point. The image processing apparatus according to claim 3 or 4, wherein the angle range for determination is set so that the diagonal lines extracted by the edge extraction means do not intersect with each other. An image processing method that divides an original image composed of a plurality of original pixels into meshes.
An intermediate image composed of intermediate pixels having pixel values corresponding to the calculated edge angles calculated by calculating the edge angle based on each pixel value in the first pixel region including the plurality of original pixels. And the edge image generation step to generate
A diagonal line of a rectangular area corresponding to the selected detection pattern based on the detection pattern selected from a plurality of types of detection patterns of the diagonal line for detecting the diagonal line of the rectangular area formed by the plurality of intermediate pixels. The intermediate pixel through which is passed is set as the determination target pixel, and the angle of the edge indicated by all the determination target pixels is within a predetermined angle range with respect to the angle of the diagonal line corresponding to the selected detection pattern. When the edge extraction step to extract the diagonal line and
An image processing method characterized by executing. A computer that divides an original image composed of multiple original pixels into meshes,
An intermediate image composed of intermediate pixels having pixel values corresponding to the calculated edge angles calculated by calculating the edge angle based on each luminance value in the first pixel region including the plurality of original pixels. Edge image generation means to generate
A diagonal line of a rectangular area corresponding to the selected detection pattern based on the detection pattern selected from a plurality of types of detection patterns of the diagonal line for detecting the diagonal line of the rectangular area formed by the plurality of intermediate pixels. The intermediate pixel through which is passed is set as the determination target pixel, and the angle of the edge indicated by all the determination target pixels is within a predetermined angle range with respect to the angle of the diagonal line corresponding to the selected detection pattern. When the edge extraction means for extracting the diagonal line and
An image processing program characterized by functioning as.

Images