JP2021108222A - Image processing device, image processing method, and image processing program - Google Patents

Abstract

To improve calculation accuracy of an edge angle when enlarging an input image with high quality using a data-dependent triangulation method.SOLUTION: A contrast C of a first pixel region 12 is calculated from a pixel value of a pixel 11 included in the first pixel region 12. The first pixel region 12 is set as a reference region 15a when the contrast C is less than a threshold Cτ; a region including the first pixel region 12 and being larger than the first pixel region 12 is set as a reference region 15b when the contrast C is equal to or greater than a threshold Cτ. An angle θ of an edge 13 of the first pixel region is calculated using a pixel value of a pixel included in the reference region.SELECTED DRAWING: Figure 3

Description

The present invention relates to an image processing device for enlarging an input image with high quality, an image processing method, and a program thereof. In particular, a data-dependent triangle division method (Data) in which an input image is divided into regions by a plurality of triangles and pixels are interpolated. Dependent Triangulation (DDT method) is used to relate to an image processing apparatus capable of enlarging an input image with high quality, an image processing method, and a program thereof.

When enlarging, rotating, deforming, etc. a raster image such as a bitmap image, the angle of the edge existing in the image is calculated, and the pixels are interpolated using the vector data created based on the calculated edge angle. For example, the DDT method is known as a technique for interpolating (Non-Patent Document 1).
The DDT method divides the raster image into regions by a plurality of triangles based on the cost function and the angle of the edge. The triangular mesh data that divides the raster image into regions is vector data. In the DDT method, the pixel values of the pixels corresponding to the vertices of the triangle and the triangle mesh data which is the vector data are used to linearly interpolate the pixels in the region in the triangle mesh.
Patent Document 2 is mentioned as an image processing method in which the above DDT method is applied to image enlargement processing.

Japanese Unexamined Patent Publication No. 2018-32301

X.Yu, B.S. Morse, and T.W. Sederberg, “Image Reconstruction UsingData-Dependent Triangulation”, IEEE Computer Graphics and Applications, Vol.21, No. 3, pp. 62-68 May / Jun. (2001)

In Patent Document 1, since the edge angle of the pixel region consisting of 2 × 2 pixels in the raster image is calculated based only on the information of the 2 × 2 pixel region, the calculation accuracy of the edge angle may decrease. be.
The present invention has been made in view of the above circumstances, and an object of the present invention is to improve the accuracy of calculating the edge angle.

In order to solve the above problems, the present invention includes a first pixel region including a plurality of pixels, and acquires a pixel value of each pixel in a second pixel region wider than the first pixel region. Means, an edge calculating means for calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region, and the maximum value of the pixel value of the pixel in the first pixel region. The edge calculation means includes a difference calculation means for calculating the difference from the minimum value, and the edge calculation means sets a reference area including the first pixel area according to the difference between the maximum value and the minimum value of the pixel value. It is characterized in that the angle of the edge of the first pixel region is calculated by using the pixel value of each pixel in the reference region.

According to one embodiment, it is possible to improve the accuracy of calculating the edge angle.

It is a block diagram which shows the functional structure of the image processing apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the hardware structure of the computer which concerns on one Embodiment of this invention. (A)-(c) is a figure explaining the outline of the edge calculation method by the image processing apparatus which concerns on one Embodiment of this invention. It is a figure which shows the intermediate image generated by the edge calculation part. It is a schematic diagram explaining an example of the diagonal line extraction method by an edge extraction part. (A) to (f) are diagrams showing an example of a diagonal detection pattern extracted as a dividing edge. (A) and (b) are schematic views explaining an example of the method of determining the angle range for determination. (A) to (d) are diagrams showing an example of the divided edges to be interpolated. It is a flowchart which showed the image processing procedure which concerns on one Embodiment of this invention.

Hereinafter, the present invention will be described in detail using the embodiments shown in the drawings. However, unless there is a specific description, the components, types, combinations, shapes, relative arrangements, etc. described in this embodiment are merely explanatory examples, not the purpose of limiting the scope of the present invention to that alone. ..

[Functional configuration]
FIG. 1 is a block diagram showing a functional configuration of an image processing apparatus according to an embodiment of the present invention.
The image processing device 100 is a device that enlarges, rotates, and transforms an input image such as a bitmap image (an example of a raster image) to generate a high-resolution or high-quality output image. The image processing device 100 divides an input image (original image) composed of a plurality of pixels into a mesh-like region by a plurality of triangles utilizing edges in the input image. The area division information for dividing the input image into areas by a triangular mesh is vector data. The image processing device 100 enlarges the input image and generates an output image based on the area division information.
The image processing device 100 includes an edge calculation unit (pixel value acquisition means, edge calculation means) 110, a mesh generation unit 120, an edge enhancement unit 130, and a rasterization unit 140.
The edge calculation unit 110 is a means for selecting a first pixel region including a plurality of pixels (original pixels) from an input image and calculating an edge angle based on the pixel value of each pixel in the first pixel region. The edge calculation unit 110 also functions as an edge image generation means for generating an intermediate image composed of intermediate pixels having pixel values corresponding to the calculated angles of each edge. The edge calculation unit 110 includes a contrast calculation unit (difference calculation means) 111, a gradient calculation unit 112, and an outliner calculation unit 113.

The mesh generation unit 120 is a means for dividing the input image into regions by a plurality of triangles based on the edge angle calculated by the edge calculation unit 110. The mesh generation unit 120 includes an edge extraction unit (edge extraction means) 121, a detection pattern storage unit 122, a mesh interpolation unit 123, and an interpolation pattern storage unit 124.
The edge enhancement unit 130 generates a sharpened image in which the edges of the input image are emphasized.
The rasterization unit 140 enlarges the sharpened image output from the edge enhancement unit 130 by using the region division information output from the mesh generation unit 120, and also uses a bilinear method, a bicubic method, or the like as necessary. Pixel interpolation is performed to generate a high-resolution output image.
Each part constituting the image processing apparatus 100 is realized by an electronic circuit (hardware), but may be realized by software executed by a general computer.

[Computer configuration]
An example of the hardware configuration of the computer when each part of the image processing apparatus shown in FIG. 1 is realized by software will be described.
FIG. 2 is a block diagram showing a hardware configuration of a computer according to an embodiment of the present invention.
The image processing device 100 includes a CPU (Central Processing Unit) 151, a ROM (Read Only Memory) 152, a RAM (Random Access Memory) 153, and a bus 154. Each component is connected by a bus 154. The image processing device 100 may include any or all of the storage device 155, the external I / F 156, and the reader / writer 157.

The CPU 151 controls the entire image processing apparatus 100 by using a program or the like stored in the ROM 152. The ROM 152 is a non-volatile storage means for storing a control program for causing the CPU 151 to execute a process of controlling the entire image processing device 100. The control program for executing the process of controlling the entire image processing device 100 may be stored in the storage device 155 included in the image processing device 100. Here, the storage device 155 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like.
The control program for executing the process of controlling the entire image processing device 100 is stored in the recording medium 162, read by the reader / writer 157 included in the image processing device 100, and stored in the ROM 152 or the storage device 155 included in the image processing device 100. It may be stored in. The recording medium 162 includes, for example, an SD memory card (SD Memory Card), an FD (Floppy Disk), a CD (Compact Disc), a DVD (Digital Versatile Disk), a BD (Blu-ray (registered trademark) Disk), and a flash memory. It is a non-temporary recording medium such as.

The RAM 153 is a volatile storage means for temporarily storing programs, data, and the like read from the ROM 152. Various functions are realized by the CPU 151 reading the control program from the ROM 152 at startup, expanding it into the RAM 153, and executing the control program using the RAM 153 as a workspace.
The external I / F 156 connects the image processing device 100 and the external storage device 161. The external storage device 161 stores various data such as an input image and an output image. The external storage device 161 is, for example, an HDD, an SSD, or the like.
The edge calculation unit 110, the mesh generation unit 120, the edge enhancement unit 130, and the rasterization unit 140 shown in FIG. 1 are each realized by the CPU 151 reading the control program from the ROM 152 or the like, expanding it into the RAM 153, and executing the control program. .. Further, the detection pattern storage unit 122 and the interpolation pattern storage unit 124 are realized by the storage device 155.

[Edge calculation unit]
<Outline of processing>
3A to 3C are diagrams illustrating an outline of an edge calculation method by an image processing apparatus according to an embodiment of the present invention.
The contrast calculation unit 111 selects a first pixel region 12 having a predetermined shape formed by a plurality of pixels 11 from the input image 10 shown in FIG. 3A, and the pixel value of each pixel 11 in the first pixel region 12 (For example, brightness value) is acquired. The contrast calculation unit 111 calculates the difference between the pixel values of the pixels in the first pixel region 12, that is, the contrast C (f_cont), based on the pixel values of the pixels 11 constituting the first pixel region 12 (FIG. 3). (B)).
The gradient calculation unit 112 and the outliner calculation unit 113 calculate the angle of the edge 13 of the first pixel region 12 (FIG. 3 (c)). In this example, the gradient calculation unit 112 calculates the angle of the edge 13 in the range of 0 to 90 degrees, and the outliner calculation unit 113 calculates the direction of the edge 13 (lower right direction or lower left direction).

The gradient calculation unit 112 sets the reference area 15 (15a, 15b) including the first pixel area 12 according to the magnitude of the contrast C of the first pixel area 12 (FIG. 3 (c)). That is, the gradient calculation unit 112 sets the first pixel region 12 as the reference region 15a when the contrast C is less than the threshold value Cτ, and the second pixel wider than the first pixel region 12 when the contrast C is the threshold value Cτ or more. The area is set as the reference area 15b. The gradient calculation unit 112 acquires the pixel value of each pixel 11 of the set reference region 15, and calculates the angle of the edge 13 of the first pixel region 12 using the pixel value of each pixel in the reference region 15.
The outliner calculation unit 113 acquires the pixel value of each pixel 11 in the first pixel region 12, and calculates the direction of the edge 13 of the first pixel region 12 using the pixel value of each pixel in the first pixel region 12.
Hereinafter, the operation of each part will be described in detail with reference to specific examples.

・・・式（１）
のように算出される。なお、符号

は、小数点以下の数値を切り捨てる床関数である。
このように、コントラスト算出部１１１は、第一画素領域１２における最大輝度値と最小輝度値の差をコントラストＣとして算出することができる。 <Contrast calculation unit>
<< In the case of 1-element image >>
As shown in FIG. 3A, the contrast calculation unit 111 sequentially sets the first pixel area 12 including a plurality of pixels 11 from the upper left of the input image 10, and each pixel included in each first pixel area 12. Get the brightness value of. The contrast calculation unit 111 calculates the contrast C of the first pixel region 12, in other words, the difference in the brightness values of the pixels in the first pixel region 12.
Here, when the first pixel area 12 is a 2 × 2 pixel area (2 × 2 matrix A) and the luminance value (component of the 2 × 2 matrix A) of each pixel is a, b, c, d, the contrast C (f_cont) is, for example,

... Equation (1)
It is calculated as. The code

Is a floor function that truncates numbers after the decimal point.
In this way, the contrast calculation unit 111 can calculate the difference between the maximum luminance value and the minimum luminance value in the first pixel region 12 as the contrast C.

・・・式（２）
のように算出される。
このように、コントラスト算出部１１１は、入力画像が、入力画像の各画素が夫々複数の要素を有した多要素画像である場合も１要素画像の場合と同様に、コントラストＣを算出することができる。 << For multi-factor images >>
Suppose that the input image is an RGB image (an example of a multi-element image, particularly a three-element image) in which each pixel has a brightness value for each element of R (Red), G (Green), and B (Blue). In the case of contrast C (f_cont), the first pixel area 12 is a 2 × 2 pixel area, and the brightness value (component of the matrix Ax) of each pixel related to the element X is ax, bx, cx, dx.

... Equation (2)
It is calculated as.
As described above, the contrast calculation unit 111 can calculate the contrast C even when the input image is a multi-element image in which each pixel of the input image has a plurality of elements, as in the case of the one-element image. can.

<< Shape of first pixel area >>
In this example, the first pixel area is a square pixel area composed of 2 × 2 pixels, but the first pixel area may be another rectangular pixel area including a plurality of pixels. Further, the first pixel region may be a pixel region having an arbitrary shape including a plurality of pixels (for example, a circular pixel region). In this case, the contrast calculation unit 111 can set a pixel region having an arbitrary shape as the first pixel region by selecting a rectangular pixel region and masking unnecessary pixels.

<Gradient calculation unit>
As shown in FIG. 3C, the gradient calculation unit 112 sets the reference area 15 (15a, 15b) including the first pixel area 12 according to the magnitude of the contrast C. Specifically, the gradient calculation unit 112 sets the first pixel region 12 (reference region 15a) or a region wider than the first pixel region 12 (reference region 15b) as the reference region 15 according to the magnitude of the contrast C. Set. The gradient calculation unit 112 uses the brightness value of each pixel in the set reference region 15 to indicate the gradient G (f_grad, (gx)) in which the angle θ of the edge 13 of the first pixel region 12 is shown in the range of 0 degrees to 90 degrees. , Gy)) is calculated.

・・・式（３）
なお、記号「＊」は行列のたたみ込みを表し、各行列の夫々の成分の積を足し合わせることを意味する。参照領域１５ａの各画素の輝度値（行列Ａの成分）をａ，ｂ，ｃ，ｄとしてグラディエントＧ（f_grad）を具体的に示せば、

・・・式（４）
となる。式（４）に示すように、本例においてｇｘとｇｙは、共に正の数となる。 << 2 × 2 gradient function >>
The gradient calculation unit 112 sets the first pixel region 12 as the reference region 15a when the contrast C is less than the threshold value Cτ (FIG. 3C).
When the reference area 15a is a 2 × 2 pixel area and is expressed as a 2 × 2 matrix A, the gradient calculation unit 112 uses the 2 × 2 gradient function shown in the following equation (3) to move the reference area 15a in the horizontal direction. The amount of change (or rate of change) gx of the brightness value in the vertical direction and the amount of change (or rate of change) gy of the brightness value in the vertical direction are calculated.

... Equation (3)
The symbol "*" represents the convolution of the matrix, and means that the products of the components of each matrix are added together. If the gradient G (f_grad) is specifically shown with the luminance values (components of the matrix A) of each pixel in the reference region 15a as a, b, c, and d,

... Equation (4)
Will be. As shown in the formula (4), both gx and gy are positive numbers in this example.

・・・式（５） Suppose that the input image is an RGB image in which each pixel of the input image has a brightness value for each element of R (Red), G (Green), and B (Blue) (multi-element image, particularly an example of a three-element image). If, the gradient G (f_grad) can be obtained as the sum of the gradient G related to each element X (X ∈ {R, G, B}).

... Equation (5)

<< 4 × 4 gradient function >>
The gradient calculation unit 112 sets a region (second pixel region) including the first pixel region 12 and larger than the first pixel region 12 as the reference region 15b when the contrast C is equal to or higher than the threshold value Cτ (FIG. 3 (FIG. 3). c)).
In this example, the gradient calculation unit 112 sets a 4 × 4 pixel area, which is one size larger than the first pixel area 12, as the reference area 15b.

・・・式（６） When the reference area 15b is a 4 × 4 pixel area (expressed as a 4 × 4 matrix B), the gradient calculation unit 112 is horizontal to the reference area 15b by the 4 × 4 gradient function shown in the following equation (6). The amount of change (or rate of change) gx of the luminance value in the direction and the amount of change (or rate of change) gy of the luminance value in the vertical direction are calculated.

... Equation (6)

The gradient calculation unit 112 calculates the edge angle (gradient) after multiplying the brightness value of each pixel in the reference region 15b by a weighting coefficient according to the positional relationship with the pixels included in the first pixel region 12. .. That is, the gradient calculation unit 112 multiplies the luminance value of the pixels constituting the first pixel region 12 by the weighting coefficient "+4" or "-4", and is adjacent to the first pixel region 12 in the horizontal or vertical direction. The brightness value of the pixel to be used is multiplied by the weighting coefficient "+2" or "-2", and the brightness value of the other pixels is multiplied by the weighting coefficient "+1" or "-1".
A power value of 2 is set for the weighting coefficient. By setting a power value of 2 to the weighting coefficient, a shift operation for calculating by shifting the bits can be used, and the processing speed can be improved.

・・・式（７）
即ち、グラディエント算出部１１２は、参照領域１５の水平方向における輝度値の変化率ｇｘと垂直方向における輝度値の変化率ｇｙとに基づいてエッジ１３の角度θを算出する。
なお、逆三角関数の計算は処理負荷が高いため、式（７）の代わりに、角度θの正接

・・・式（８）
を求めてもよい。 << Edge angle θ >>
The gradient calculation unit 112 uses an inverse trigonometric function to calculate the angle θ (0 ° ≦ θ ≦ 90 °) of the edge 13 of the first pixel region 12 from the gradient G (gx, gy) obtained by the formula (3) or the formula (6). ) Can be obtained.

... Equation (7)
That is, the gradient calculation unit 112 calculates the angle θ of the edge 13 based on the rate of change gx of the brightness value in the horizontal direction of the reference region 15 and the rate of change gy of the brightness value in the vertical direction.
Since the calculation of the inverse trigonometric function has a high processing load, the tangent of the angle θ is used instead of the equation (7).

... Equation (8)
May be sought.

When the contrast C is equal to or greater than the threshold value Cτ, the gradient calculation unit 112 calculates the gradient G based on the brightness value of each pixel in the reference region 15b wider than the first pixel region 12, so that the angle of the edge 13 is more accurately calculated. θ can be calculated. Further, the angle θ of the edge 13 can be calculated more accurately by multiplying each luminance value of the reference region 15b by a weighting coefficient according to the positional relationship with the first pixel region 12.

・・・式（９）
のように算出される。なお、記号「＊」は行列のたたみ込みを表し、各行列の夫々の成分の積を足し合わせることを意味する。 <Outlier calculation unit>
The outlier calculation unit 113 calculates an outlier (f_outlier) which is a value for estimating the strength and direction of the local edge 13 in the first pixel region 12.
For example, when the first pixel area 12 is a 2 × 2 pixel area (matrix A), the outlier (f_outlier) is

... Equation (9)
It is calculated as. The symbol "*" represents the convolution of the matrix, and means that the products of the components of each matrix are added together.

・・・式（１０）
即ち、アウトライヤ（f_outlier）の大きさは、斜め方向における輝度変化の大きさを示し、アウトライヤ（f_outlier）の符号は、エッジ１３の向きを表す。アウトライヤ（f_outlier）が正の数をとるときエッジ１３は右下方向を向き、負の数をとるときエッジ１３は左下方向に向いていることを示す。
グラディエントＧから求められるエッジの角度θが０〜９０度の範囲であっても、アウトライヤの値を求めることによって、エッジの角度θを０〜１８０度の範囲で特定することができる。 The outliers (f_outlier) are specifically shown with the luminance values (components of the matrix A) of each pixel in the first pixel region 12 as a, b, c, and d as follows.

... Equation (10)
That is, the magnitude of the outlier (f_outlier) indicates the magnitude of the change in brightness in the oblique direction, and the sign of the outlier (f_outlier) indicates the direction of the edge 13. When the outlier (f_outlier) takes a positive number, the edge 13 points in the lower right direction, and when the outlier (f_outlier) takes a negative number, the edge 13 points in the lower left direction.
Even if the edge angle θ obtained from the gradient G is in the range of 0 to 90 degrees, the edge angle θ can be specified in the range of 0 to 180 degrees by obtaining the outliner value.

<Intermediate image generated by the edge calculation unit>
FIG. 4 is a diagram showing an intermediate image generated by the edge calculation unit.
Finally, the edge calculation unit 110 determines the angle θ (0 to 90 degrees, or 0 to 180 degrees) of each edge 13 in each first pixel region 12, or a value indicating this (gradient (gx, gy), And an outliner) as pixel values, respectively, to generate an intermediate image 20 composed of a plurality of intermediate pixels 21. That is, the edge calculation unit 110 functions as an edge image generation means for generating an intermediate image 20 in which each pixel has edge angle information. In this example, the intermediate image 20 is composed of a plurality of images of a gradient image in which each intermediate pixel 21 has a gradient value and an outline image in which each intermediate pixel 21 has an outliner value.
The outliner and gradient values of each intermediate pixel 21 indicate the relationship between the four pixels 11 constituting the first pixel region 12 in the original image (input image 10). The intermediate image 20 is composed of a plurality of intermediate pixels 21 represented as a square region having the pixel center of each pixel 11 of the input image 10 as a vertex. Further, the intermediate image 20 has a positional relationship that is shifted by half a pixel in the lower right direction with respect to the input image 10.

<Modification example>
In the present embodiment, when setting the reference area 15 for calculating the gradient, the brightness contrast (difference in luminance value) of the first pixel region 12 is used, but the color contrast (color difference) and the like are used. The difference in pixel values other than the brightness value of the pixels may be used. Instead of the brightness contrast, a color contrast (difference in pixel values representing colors) may be used.
Unlike the above embodiment, the gradient calculation unit 112 sets a region wider than the first pixel region 12 as a reference region when the contrast C is less than the threshold value Cτ, and sets the first pixel when the contrast C is the threshold value Cτ or more. Area 12 may be set as a reference area.

[Mesh generator]
<Outline of processing>
The mesh generation unit 120 utilizes the edges in the input image 10 to divide the input image 10 into regions by a plurality of triangles. That is, the triangle that divides the input image 10 is selected so that the edge in the input image 10 becomes the edge of the triangle that divides the input image 10 into a region.
The edge extraction unit 121 is based on the edge angle θ (including a value indicating the edge angle) calculated by the edge calculation unit 110 or a value indicating this (for example, a gradient (gx, gy) and an outliner). The division edges (diagonal lines 23), which are the sides of the triangle that divides the input image 10 into regions, are sequentially extracted.
The mesh interpolation unit 123 interpolates the divided edges with respect to the incomplete mesh portion so that the entire input image 10 is divided into regions by triangles.

<Edge extraction part: Diagonal extraction method>
FIG. 5 is a schematic diagram illustrating an example of a diagonal line extraction method by the edge extraction unit. Hereinafter, an example will be described in which the rectangular region forming the diagonal line is the right 3 × 2 pixel region. The “right 3 × 2 pixel area” means a rectangular area in the drawing in which the diagonal line extends in the lower right direction and is composed of 3 × 2 pixels. When the diagonal line extends in the lower left direction in the figure, it is expressed as "left ... pixel area".
The edge extraction unit 121 selects a rectangular region 22 (here, a right 3 × 2 pixel region) formed by a plurality of intermediate pixels 21 from the intermediate image 20 shown in FIG. The edge extraction unit 121 sets an intermediate pixel through which the diagonal line 23 of the rectangular region 22 passes as a determination target pixel 24 (colored portion in the figure). When the angle θ of each edge 13 indicated by all the determination target pixels 24 in the rectangular region 22 is within a predetermined angle range with respect to the angle φ of the diagonal line 23, the edge extraction unit 121 inputs the diagonal line 23 to the input image. Is extracted as the dividing edge that constitutes the side of the triangle that divides the area.

<Detection pattern of split edge>
6 (a) to 6 (f) are diagrams showing an example of a diagonal detection pattern extracted as a dividing edge.
The detection pattern storage unit 122 (FIG. 1) stores a plurality of types of diagonal detection patterns. Specifically, the detection pattern storage unit 122 has a rectangular area 22, a right 3 × 2 pixel area, a left 3 × 2 pixel area (FIG. 6A), a right 2 × 3 pixel area, and a left 2 × 3 pixel. Region (FIG. 6 (b)), right 3 × 1 pixel region, left 3 × 1 pixel region (FIG. 6 (c)), right 1 × 3 pixel region, left 1 × 3 pixel region (FIG. 6 (d)) , Right 2 × 1 pixel area, left 2 × 1 pixel area (FIG. 6 (e)), right 1 × 2 pixel area, left 1 × 2 pixel area (FIG. 6 (f)). I remember. Since the diagonal lines of the right 1 × 1 pixel area and the left 1 × 1 pixel area are treated as the divided edges to be interpolated (see FIG. 7), they are not stored as the detection pattern.
In this example, the detection pattern storage unit 122 stores the diagonal line of the rectangular area smaller than the 3 × 3 pixel area as the detection pattern, but the detection pattern storage unit 122 stores the diagonal line of the rectangular area larger than this as the detection pattern. Such a detection pattern may be stored.

The detection pattern storage unit 122 stores the detection patterns related to the diagonal lines having an angular relationship that does not overlap with each other. In particular, it is desirable that the detection pattern storage unit 122 stores a diagonal line other than the end point on the diagonal line that does not include the intersection (corner portion) of the intermediate pixels as the detection pattern. That is, the rectangular region forming the diagonal line stored as the detection pattern does not include a small rectangular region having a diagonal line at the same angle as the diagonal line of the rectangular region, and the lengths of the two sides orthogonal to each other are "elementary to each other". It is desirable that the rectangular area is related. When the lengths of the two sides are "disjoint", it means that the greatest common divisor of the lengths of the two orthogonal sides forming the rectangular region is 1.
For example, since the 6 × 4 pixel area includes four 3 × 2 pixel areas and includes the corners (intersection points) of the 3 × 2 pixel area on the diagonal line of the 6 × 4 pixel area, the detection pattern is stored. The unit 122 may store the diagonal line of the 3 × 2 pixel area as a detection pattern, but it is desirable that the diagonal line of the 6 × 4 pixel area is not stored as a detection pattern. This is because at the time of rasterization, the pixel value to be filled in the triangular mesh is determined by referring to the pixel value of the original pixel located at the apex of the triangular mesh. For example, when extracting the diagonal line of the 6 × 4 pixel area, the corner part of the 3 × 2 pixel area exists in the center of each side of the triangular mesh including the diagonal line, but the position is located in the center of each side. Since the pixel value of the original pixel is not referred to at the time of rasterization, the reproduction rate of the original image may decrease. By storing only the diagonal lines related to the rectangular regions that are "disjoint" in the detection pattern storage unit 122 as the detection pattern, the recall rate of the original image is improved at the time of rasterization, and an image closer to the original image can be obtained. It becomes possible to output.

In the detection pattern storage unit 122 (FIG. 1), the position of the determination target pixel 24 and the determination angle range (parameters a, b, outliner code) are associated with each other for the detection pattern of each diagonal line 23 for determination. It is stored as a table. Further, the detection pattern of each diagonal line 23 is associated with information relating to the position of the base point 23a of the diagonal line 23 to be detected and the position of the attention pixel 25 in which the determination is first performed among the determination target pixels 24. There is. The determination target pixel 24 having the base point 23a of the diagonal line 23 to be detected is the attention pixel 25. That is, it can be said that the detection pattern of each diagonal line 23 includes information indicating the positional relationship between the attention pixel 25 and the determination target pixel 24 other than the attention pixel 25.
The determination angle range stored in the detection pattern storage unit 122 is set so that the diagonal lines extracted by the edge extraction unit 121 do not intersect with each other.

・・・式（１１）
なお、θは各中間画素２１が示すエッジの角度である。 7 (a) and 7 (b) are schematic views illustrating an example of a method for determining an angle range for determination. Hereinafter, an example of determining an angle range for determination for detecting the diagonal line 23 related to the rectangular region 22 composed of the right 3 × 2 pixels will be described.
First, the diagonal line d1 of the rectangular area closest to the counterclockwise direction (right 2 × 1 pixel area) and the rectangular area closest to the clockwise direction (right 1 × 1 pixel area) with respect to the diagonal line 23 to be extracted. Find the diagonal line d2 of. Here, the angle of the diagonal line 23 is φ, the angle of the diagonal line d1 is α, and the angle of the diagonal line d2 is β.
Between the angle φ of the diagonal line 23 and the angle α of the diagonal line d1 divided equally (φ + α) / 2 and the angle φ of the diagonal line 23 and the angle β of the diagonal line d2 divided equally (φ + β) / 2. , The angle range for judgment.

... Equation (11)
Note that θ is the angle of the edge indicated by each intermediate pixel 21.

・・・式（１２）
しかし、逆三角関数の計算は複雑であり、これを中間画素ごとに計算する場合には、回路の複雑化と処理負荷の増大を招くという問題がある。そこで、式（１２）を以下の式（１３）ように変形して、グラディエント比（ｇｙ／ｇｘ）の下限値をパラメータａ、上限値をパラメータｂとして判定テーブル（図６）に記憶させることで、逆三角関数の計算を省略することが可能となる。

・・・式（１３） By the way, the angle θ of the edge of the intermediate pixel 21 is obtained by the following equation (12) when the gradient of the intermediate pixel 21 is (gx, gy).

... Equation (12)
However, the calculation of the inverse trigonometric function is complicated, and when this is calculated for each intermediate pixel, there is a problem that the circuit becomes complicated and the processing load increases. Therefore, the equation (12) is transformed into the following equation (13) and stored in the determination table (FIG. 6) with the lower limit value of the gradient ratio (gy / gx) as the parameter a and the upper limit value as the parameter b. , It is possible to omit the calculation of the inverse trigonometric function.

... Equation (13)

In the detection pattern shown in FIG. 6, the parameters a and b for determining each diagonal line (angle φ) can be set to arbitrary values satisfying a <tanφ and tanφ <b. Further, the angle range for determining each diagonal line may have a portion overlapping between the parameters for determining different diagonal lines. For example, in the detection pattern shown in FIG. 6, the parameter a in the left and right 3 × 1 pixel area shown in FIG. 6 (c) and the parameter b in the left and right 1 × 3 pixel area shown in FIG. 6 (d) are not determined. The edge judgment range is expanded so that the edge range does not occur. Further, since the diagonal lines of the left and right 2 × 1 pixel areas shown in FIG. 6 (e) and the diagonal lines of the left and right 1 × 2 pixel areas shown in FIG. 6 (f) are less likely to be erroneously detected, the angle range for determination is expanded. is doing.
Further, in the left and right 3 × 1 pixel area and the left and right 1 × 3 pixel area, in order to prevent erroneous detection of the horizontal edge and the vertical edge, when the y component gy of the gradient is zero, it is a diagonal line. Is set not to be extracted.

<Diagonal detection procedure>
The edge extraction unit 121 executes the division edge extraction process so as to preferentially extract the diagonal line related to the rectangular area having a large area over the diagonal line related to the rectangular area having a small area. That is, the edge extraction unit 121 reads the detection pattern from the detection pattern storage unit 122, and executes the division edge extraction process so as to extract the diagonal lines in the order shown in FIGS. 6A to 6F. The edge extraction unit 121 can extract the diagonal lines so that the divided edges do not intersect by extracting the diagonal lines related to the rectangular area having a large area in preference to the diagonal lines related to the rectangular area having a small area.

Specifically, as shown in FIG. 5, the edge extraction unit 121 selects one intermediate pixel 21 in order from, for example, the upper left of the intermediate image 20 (FIG. 4), and sets it as the attention pixel 25. The edge extraction unit 121 compares the angle θ of the edge 13 of the pixel of interest 25 and the angle range for determining the diagonal line 23 in order from the detection pattern of the diagonal line 23 related to the rectangular region 22 having a large area. When the angle θ of the edge of the pixel of interest 25 is included in the angle range for determination of a certain diagonal line 23, the edge extraction unit 121 sets the determination target pixels 24 other than the pixel of interest 25 based on the detection pattern of the diagonal line 23. do. The edge extraction unit 121 compares the angle θ of the edge 13 of the newly set determination target pixel 24 with the angle range for determination. When the angle θ of the edge 13 indicated by all the determination target pixels 24 is within the angle range for determination, the edge extraction unit 121 divides the diagonal line 23 into a triangle that divides the input image 10 (FIG. 3A) into a region. It is extracted as a dividing edge that constitutes the side of.
If the angle θ of the edge 13 indicated by any of the determination target pixels 24 deviates from the angle range for determination with respect to the angle φ of the diagonal line 23, the edge extraction unit 121 determines the angle θ of the edge 13 of the pixel of interest 25. Is compared with the angle range for determining the diagonal line relating to the rectangular region having the next largest area. The edge extraction unit 121 newly sets the determination target pixel 24 other than the attention pixel 25 based on the comparison result.
The edge extraction unit 121 extracts the diagonal line 23 as the divided edge from the entire region of the intermediate image 20 by sequentially executing the above processes.

<Mesh interpolation section>
Since a complete mesh cannot be formed only by the diagonal line 23 extracted by the edge extraction unit 121, the mesh interpolation unit 123 interpolates the divided edges for the portion where the mesh is incomplete, and divides the entire input image 10 into regions by triangles.
8 (a) to 8 (d) are diagrams showing an example of the divided edges to be interpolated. The divided edges shown in this figure are stored in the interpolation pattern storage unit 124 (FIG. 1).
The edge extraction unit 121 can divide the entire input image 10 into regions by triangles by inserting the divided edges in the order of FIGS. 8A to 8D for the portion where the mesh is incomplete. However, the edge extraction unit 121 inserts the illustrated divided edge only when it does not intersect the already extracted divided edge.
The edge extraction unit 121 of the mesh generation unit 120 may extract the divided edge by using the edge angle (or gradient value) of the original pixel calculated by a method other than the above.

〔flowchart〕
FIG. 9 is a flowchart showing an image processing procedure according to an embodiment of the present invention.
In step S1, the image processing device 100 acquires the input image 10 (FIG. 3A). The input image 10 is input to the edge calculation unit 110 and the edge enhancement unit 130.
In step S3, the contrast calculation unit 111 sequentially sets the first pixel region 12 including the plurality of pixels 11, and acquires the brightness value of each pixel 11 included in each first pixel region 12. The contrast calculation unit 111 calculates the contrast C of the first pixel region 12 based on the brightness value of each pixel 11 included in the first pixel region 12 (FIG. 3B).
In step S5, the gradient calculation unit 112 sets the reference area 15 based on the magnitude of the contrast C of the first pixel area 12. That is, the gradient calculation unit 112 sets the first pixel region 12 as the reference region 15a when the contrast C is less than the threshold value Cτ, and includes the first pixel region 12 when the contrast C is the threshold value Cτ or more, and is the first. A region larger than the pixel region 12 (second pixel region) is set as the reference region 15b (FIG. 3 (c)).

In step S7, the gradient calculation unit 112 acquires the brightness value of each pixel 11 included in the set reference area 15. The gradient calculation unit 112 calculates the gradient G (gx, gy) of the first pixel region 12 based on the brightness value of each pixel 11 included in the reference region 15. The gradient G is a value indicating the angle θ of the edge 13 existing in the first pixel region 12 in the range of 0 to 90 degrees. By this processing, the gradient calculation unit 112 generates a gradient image (intermediate image) composed of a plurality of pixels having the gradient G of each first pixel region 12 as a pixel value.
In step S9, the outliner calculation unit 113 calculates an outliner which is a value for estimating the strength and direction of the local edge 13 in the first pixel region 12. The outliner calculation unit 113 calculates the outliner based on the brightness value of each pixel 11 included in the first pixel region 12. By this processing, the outliner calculation unit 113 generates an outliner image (intermediate image) composed of a plurality of pixels having an outliner of each first pixel region 12 as a pixel value.
The edge calculation unit 110 outputs an intermediate image 20 (FIG. 4) in which each pixel has gradient G and outliner information to the mesh generation unit 120.

In step S11, the edge extraction unit 121 of the mesh generation unit 120 receives the input image 10 based on the angle θ of the edge 13 indicated by each intermediate pixel 21 of the intermediate image 20 generated by the gradient calculation unit 112 and the outliner calculation unit 113. Extract the dividing edges that make up the sides of the triangle that divides the area. That is, as shown in FIG. 5, the edge extraction unit 121 sets a rectangular region 22 formed by a plurality of intermediate pixels 21, and all of the determination target pixels 24 through which the diagonal line 23 of the rectangular region 22 passes pass through the diagonal line 23. When it is within a predetermined angle range including, the diagonal line 23 is extracted as a dividing edge constituting the side of the triangle that divides the input image 10 into a region. In this process, the edge extraction unit 121 extracts the diagonal line 23 related to the rectangular area 22 having a large area with priority over the diagonal line 23 related to the rectangular area 22 having a small area (a) to 6 (f). Diagonal lines are extracted in the order of the detection patterns shown in.

In step S13, the mesh interpolation unit 123 of the mesh generation unit 120 reads out the divided edge to be interpolated from the interpolation pattern storage unit 124, and sets the divided edge for the portion where the mesh is incomplete (a) to 8 (d). Interpolate in the order of, and the entire input image 10 is divided into regions by triangles. The area division information for dividing the input image 10 into areas is vector data, and is information that does not lose its sharpness even when enlarged or reduced.
In step S15, the edge enhancement unit 130 generates a sharpened image in which the edge of the input image 10 is emphasized and sharpened.
In step S17, the rasterization unit 140 enlarges the sharpened image output from the edge enhancement unit 130 by using the region division information output from the mesh generation unit 120. Further, the rasterization unit 140 determines the pixel value for filling the inside of the triangle mesh by using the pixel value of the pixel 11 corresponding to the apex of the triangle mesh, and fills the inside of the triangle mesh using the determined pixel value. Further, the rasterization unit 140 performs pixel interpolation using a bilinear method, a bicubic method, or the like as necessary to generate a high-resolution output image.

[Summary of Examples of Embodiments of the Present Invention, Actions, and Effects]
<First embodiment>
The image processing apparatus 100 according to this aspect includes a first pixel region 12 including a plurality of pixels 11, and a pixel that acquires a pixel value of each pixel in a second pixel region (reference region 15b) wider than the first pixel region. A value acquisition means (edge calculation unit 110) and an edge calculation means (edge calculation unit 110) for calculating the edge angle θ of the first pixel area using the pixel values of each pixel in the second pixel area are provided. It is characterized by.
According to this aspect, since the edge angle of the first pixel area is calculated using the pixel value of each pixel of the second pixel area wider than the first pixel area, it is possible to improve the calculation accuracy of the edge angle. It will be possible.

<Second embodiment>
The image processing device 100 according to this aspect includes a difference calculation means (contrast calculation unit 111) for calculating the difference (contrast C) of the pixel values of the pixels 11 in the first pixel region 12, and an edge calculation means (edge calculation unit 110). ) Sets the reference area 15 (15a, 15b) including the first pixel area according to the difference in the pixel values, and uses the pixel value of each pixel in the reference area to set the edge angle θ of the first pixel area. It is characterized by calculating.
According to this aspect, the reference area is set according to the difference in the pixel values, and the edge angle of the first pixel area is calculated using the pixel value of each pixel of the set reference area. Therefore, the edge angle is calculated. It is possible to improve the accuracy.

<Third embodiment>
In the image processing apparatus 100 according to this aspect, the edge calculation means (edge calculation unit 110) sets the first pixel area 12 or the second pixel area (reference area 15b) in the reference area 15 according to the difference in pixel values. It is characterized by doing.
In this embodiment, the edge calculation means sets the first pixel area or the second pixel area as a reference area according to the difference in pixel values, and uses the pixel values of each pixel of the set reference area to obtain the first pixel area. Calculate the edge angle. That is, the edge calculation means calculates the edge angle of the first pixel region by using the pixel value of each pixel constituting the first pixel region (reference region 15a) according to the difference in the pixel values. The edge angle of the first pixel region is calculated using the pixel values of each pixel constituting the second pixel region (reference region 15b) wider than the pixel region.
According to this aspect, the first pixel area or the second pixel area is set as the reference area according to the difference in the pixel values, and the edge angle of the first pixel area is calculated. Therefore, the calculation accuracy of the edge angle is calculated. Can be improved.

<Fourth Embodiment>
In the image processing apparatus 100 according to this aspect, the edge calculation means (edge calculation unit 110) determines the edge based on the rate of change gx of the pixel value in the horizontal direction and the rate of change gy of the pixel value in the vertical direction of the reference region 15. It is characterized in that the angle θ is calculated.
The angle θ may be calculated directly from the rate of change of the pixel values gx and gy by an inverse trigonometric function, or may be calculated indirectly by obtaining the tangent (gy / gx) of the angle θ. Since the calculation of the inverse trigonometric function has a high processing load, the processing load related to the calculation can be reduced by adopting the latter method.

<Fifth embodiment>
In the image processing apparatus 100 according to this aspect, the edge calculation means (edge calculation unit 110) corresponds to the positional relationship between the pixel values of each pixel 11 in the reference region 15 and the pixels included in the first pixel region 12. It is characterized in that the edge angle θ is calculated after multiplying by a weighting coefficient.
According to this aspect, the edge angle can be calculated more accurately while referring to the pixel values of the pixels located around the first pixel region as needed.

<Sixth Embodiment>
The image processing apparatus 100 according to this aspect is characterized in that a power value of 2 is set for the weighting coefficient.
By setting a power value of 2 to the weighting coefficient, a shift operation for calculating by shifting the bits can be used, and the processing speed can be improved.

10 ... Input image, 11 ... Pixel, 12 ... First pixel area, 13 ... Edge, 15 ... Reference area, 15a ... Reference area, 15b ... Reference area (second pixel area), 20 ... Intermediate image, 21 ... Intermediate pixel , 22 ... Rectangular area, 23 ... Diagonal line, 23a ... Base point, 24 ... Pixel to be determined, 25 ... Pixel of interest, 100 ... Image processing device, 110 ... Edge calculation unit (pixel value acquisition means, edge calculation means), 111 ... Contrast Calculation unit (difference calculation means), 112 ... gradient calculation unit, 113 ... outliner calculation unit, 120 ... mesh generation unit, 121 ... edge extraction unit, 122 ... detection pattern storage unit, 123 ... mesh interpolation unit, 124 ... interpolation pattern storage Unit, 130 ... Edge enhancement part, 140 ... Rasterize part, 151 ... CPU, 152 ... ROM, 153 ... RAM, 154 ... Bus, 155 ... Storage device, 156 ... External I / F, 157 ... Reader / writer, 161 ... External storage Device, 162 ... Recording medium, d1, d2 ... Diagonal

Claims (9)

A pixel value acquisition means that includes a first pixel region including a plurality of pixels and acquires a pixel value of each pixel in a second pixel region that is wider than the first pixel region.
An edge calculation means for calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region.
A difference calculation means for calculating the difference between the maximum value and the minimum value of the pixel value of the pixel in the first pixel region is provided.
The edge calculation means sets a reference area including the first pixel area according to the difference between the maximum value and the minimum value of the pixel value, and uses the pixel value of each pixel in the reference area to obtain the first pixel value. An image processing device for calculating the angle of the edge of a pixel region. The first aspect of claim 1, wherein the edge calculation means sets the first pixel area or the second pixel area in the reference area according to the difference between the maximum value and the minimum value of the pixel value. Image processing equipment. The edge calculation means according to claim 1 or 2, wherein the edge calculation means calculates the angle of the edge based on the rate of change of the pixel value in the horizontal direction and the rate of change of the pixel value in the vertical direction of the reference region. Image processing device. A pixel value acquisition means that includes a first pixel region including a plurality of pixels and acquires a pixel value of each pixel in a second pixel region that is wider than the first pixel region.
An edge calculation means for calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region.
A difference calculation means for calculating the difference between the pixel values of the pixels in the first pixel region is provided.
The edge calculation means sets a reference area including the first pixel area according to the difference in the pixel values, and the pixel value of each pixel in the reference area is set with the pixel included in the first pixel area. An image processing apparatus characterized in that the angle of the edge of the first pixel region is calculated by multiplying the weighting coefficient according to the positional relationship by using the pixel value of each pixel in the reference region. The image processing apparatus according to claim 4, wherein a power value of 2 is set for the weighting coefficient. A pixel value acquisition step of acquiring a pixel value of each pixel in a second pixel area that includes a first pixel area including a plurality of pixels and is wider than the first pixel area.
An edge calculation step of calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region, and
Including a difference calculation step of calculating the difference between the maximum value and the minimum value of the pixel value of the pixel in the first pixel region.
In the edge calculation step, a reference area including the first pixel area is set according to the difference between the maximum value and the minimum value of the pixel value, and the pixel value of each pixel in the reference area is used to set the first pixel value. An image processing method characterized by calculating the angle of the edge of a pixel region. Computer,
A pixel value acquisition means that includes a first pixel region including a plurality of pixels and acquires a pixel value of each pixel in a second pixel region that is wider than the first pixel region.
An edge calculation means for calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region.
It functions as a difference calculation means for calculating the difference between the maximum value and the minimum value of the pixel value of the pixel in the first pixel region, and also functions as a difference calculation means.
The edge calculation means sets a reference area including the first pixel area according to the difference between the maximum value and the minimum value of the pixel value, and the first pixel value of each pixel in the reference area is used. An image processing program characterized in that it functions to calculate the angle of the edge of a pixel region. A pixel value acquisition step of acquiring a pixel value of each pixel in a second pixel area that includes a first pixel area including a plurality of pixels and is wider than the first pixel area.
An edge calculation step of calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region, and
Includes a difference calculation step of calculating the difference in pixel values of the pixels in the first pixel region.
In the edge calculation step, a reference area including the first pixel area is set according to the difference in the pixel values, and the pixel value of each pixel in the reference area is set with the pixel included in the first pixel area. An image processing method characterized in that the angle of the edge of the first pixel region is calculated using the pixel value of each pixel in the reference region after multiplying by a weighting coefficient according to the positional relationship. Computer,
A pixel value acquisition means that includes a first pixel region including a plurality of pixels and acquires a pixel value of each pixel in a second pixel region that is wider than the first pixel region.
An edge calculation means for calculating the edge angle of the first pixel region using the pixel value of each pixel in the second pixel region.
It functions as a difference calculation means for calculating the difference between the pixel values of the pixels in the first pixel region, and also functions as a difference calculation means.
The edge calculation means sets a reference area including the first pixel area according to the difference in the pixel values, and the pixel value of each pixel in the reference area is set with the pixel included in the first pixel area. Image processing characterized in that it functions to calculate the angle of the edge of the first pixel region by using the pixel value of each pixel in the reference region after multiplying by a weighting coefficient according to the positional relationship. program.

Images