JP2594212B2 - Adaptive and hierarchical grid representation generator for images - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an adaptive and hierarchical grid representation generator for images, and more particularly to a computer vision or computer graphics field, which uses image processing techniques to obtain distance, shading, color images, and the like. An adaptive and hierarchical grid of images used in technical fields such as triangular patch representation of the general image, image compression and decompression for image communication, CAD modeling of arbitrarily shaped objects for graphics display, etc. The present invention relates to an expression generation device.

[0002]

2. Description of the Related Art Conventionally, image compression and decompression are generally performed based on frequency analysis of an image. However, since it is difficult to perform frequency analysis on the entire image due to computational restrictions, processing is performed by dividing the input image into a plurality of rectangular regions (blocks). However, there arises a problem that continuity of the analysis result between blocks is lost.

On the other hand, in the field of computer vision and image processing in recent years, techniques for compressing (sampling) and restoring an image using a grid representation have been proposed. Examples of a method of generating an adaptive grid representation include a method of moving each grid point position of the regular grid and a method of correcting the number of grid points and the connection relation. In the method of moving the grid points, if the shape change of the target surface is not uniform, the grid points are moved over a wide range, and a large amount of calculation time is required to obtain a grid representation with uniform accuracy over the entire surface. In addition, the method of increasing or decreasing the number of grid points has a problem that the connection relationship between the existing grid points is often lost, and cracks (gaps) are generated in the approximate polygon obtained from the grid representation.

Therefore, a main object of the present invention is to adaptively compress and decompress an image using means for generating an adaptive grid representation based on the geometric characteristics of the entire image. It is an object of the present invention to provide an adaptive and hierarchical lattice representation generation device for images.

[0005]

SUMMARY OF THE INVENTION The present invention is directed to an initial regular lattice plane which is set at equal intervals and a sparse lattice width in the horizontal and vertical directions and includes a line segment for defining a triangular patch expression.
A grid initialization unit for setting a distance image obtained by measuring the three-dimensional shape as an input image; a value of a pixel value at each set grid point position; a slope measured by the pixel value;
A hierarchical lattice expression generating means for generating a hierarchical lattice expression for sequentially dividing a lattice surface into similar triangular patches based on the curvature which is a change in inclination to form an approximate polygon, and a generated lattice surface Means for determining the division of the approximate polygon based on the curvature and the hierarchy of the triangular patch around the lattice plane in order to avoid the gap of the approximate polygon generated in the process of dividing .

[0006]

According to the present invention, there is provided an apparatus for generating an adaptive and hierarchical lattice representation of an image, comprising an initial regular lattice plane which is set at equal intervals and a sparse lattice width in the horizontal and vertical directions and includes a line segment for expressing a triangular patch. A distance image obtained by measuring a three-dimensional shape is set as an input image, and a grid surface is determined based on a pixel value at each grid point position, a slope measured at the pixel value, and a curvature that is a change in the slope. Are sequentially divided into similar triangular patches to generate a hierarchical grid representation that forms an approximate polygon. At this time, in order to avoid a gap between the approximate polygons generated in the process of dividing the lattice plane, the division of the approximate polygon is determined based on the curvature and the hierarchy of the triangular patches around the lattice plane.

[0007]

FIG. 2 is a diagram for explaining 3D curvature. First, with reference to FIG.
The D curvature will be described. The normal at point P on an arbitrary curved surface n, the tangent and t _i, consider a plane P _i including n and t _i. If the maximum curvature κ1 (the curvature of the intersection curve S1) and the minimum curvature κ2 (the curvature of the intersection curve S2) among the curvatures (called the normal curvature) of the intersection curve of the curved surface and the plane P _i are obtained, the curvature of the surface at the point P is obtained. Local shapes can be accurately described based on differential geometry.

FIG. 3 is a diagram showing a local shape of eight classification points and distribution positions in a curvature space. Each point on the curved surface can be classified into the following eight types according to the local shape determined by the combination of the signs of the maximum curvature κ1 and the minimum curvature κ2. Convex ellipse point (κ1 <0, κ2 <0) Convex paraboloid (κ1 = 0, κ2 <0) Convex hyperbolic point (κ1 <0 <κ1, | κ1 | <| κ2 |) Minimum point (κ1 = − κ2> 0) concave hyperbolic point (κ2 <0 <κ1, | κ2 | <| κ1 |) concave parabolic point (κ1> 0, κ2 = 0) concave elliptical point (κ1> 0, κ2> 0) plane point (Κ1 = 0, κ2 = 0) FIG. 4 is a diagram showing a two-dimensional array of square lattice elements having an initial lattice width for explaining the principle of the present invention. Next, the principle of the present invention will be described with reference to FIG. FIG.
As shown in (a), the initial adaptive grid (plane) is a regular grid, and is a square grid element (hereinafter referred to as a grid element) obtained by dividing the X and Y directions of the reconstructed plane (playback display screen) at equal intervals. Is given as an array of Then, on each grid point P ₀ ,
Height Z, normal n, principal curvature (κ1, κ2), principal direction (e
1, e2) is calculated, to restore the shape of the lattice point P _0.

In the following description, the lattice point P ₀ = (x ₀ , y
₀ ) is a surface point P ₀ that projects P ₀ on a curved surface: Z ₀ = f
(X _0, y ₀₎ represents the generation of lattice points P _i means to perform height of the surface point P _i, the normal, the shape recovery of the principal curvatures and the main direction.

The initial grid element shown in FIG. 4A is composed of four grid points (P) adjacent in the x and y directions shown in FIG.
₀ , P ₁ , P ₂ , P ₃ ). Restored curvature values at each grid point of the grid elements (κ0, κ1, κ2, κ3 ) represents the magnitude and direction of the uneven change of the curved region near to the boundary of the Quad _0. Assuming that the maximum and minimum curvature values at all initial lattice points are κ _max and κ _min , respectively, κ _range = κ _max −
κ _min represents the curvature distribution range of the entire curvature. From the distribution state of all the curvature values in the κ _range , the curvature rank n ₀ in the lattice element Quad ₀ is represented by the following equation.

[0011] n ₀ = RanΚ (Κ ₀₎ new grid points along the boundary line of the grating elements Quad ₀ according to the rank n ₀ of curvature at the grating elements Quad ₀ is generated recursively divided into two grid elements n ₀ Repeated times. Here, recursive means finding a similar shape by itself, and recursive bisection means dividing a lattice element into two similar lattice elements.

Each component is composed of the initial four components shown in FIG.
Lattice points _{(P 0, P 1, P} 2, P 3) and the newly generated grid points P ₄ with four routes triangles at the center point of the grating element shown in FIG. 4 (c) (patches) Tr _i [I], i = 0,
It is divided into 1, 2, and 3. This is called initialization. Each root triangle and Motohen four sides of the component, respectively, the left end point of Motohen, right endpoint is initialized with the center point P ₄ counterclockwise. Each triangle has a curvature rank (n _i , n
_{If any of i + 1} , n ₄ ) exceeds the current recursion depth split threshold, the current triangle is split into two right and left child triangles to form a binary tree.

FIG. 5 is a diagram showing a process of forming two adjacent lattice elements into a three-level binary tree structure. As shown in FIG. 5, two adjacent components Qd (i ₀ , j ₀ ) and Qd (i
₀ , j _{0 + 1} ) has a high curvature along the boundary, the Tr [2] and Qd (i ₀ , j) of the component Qd (i ₀ , j ₀ )
Tr [0] of ( _{0 + 1} ) is recursively divided three times to form a three-level binary tree structure. In FIG. 5, each downward arrow indicates a layer.

The current recursion depth of the parent triangle is n and the base length is l _n (= InitQuadSize · (1 / √
2) ⁿ ), l _{n + 1} (= InitQuadSize · (1
The triangle of / √2) ^{n + 1} ) is divided into two sub-triangles each having the left side and the right side of the parent triangle as bases when any of the following conditions are satisfied: The curvature rank κ = f (n _i , n _{i + 1} ) along the base is greater than the current subdivision depth n (the hierarchy of the binary split tree). The curvature rank κ = f (n _{i + 1} , n ₄ ) along the left side is larger than the current subdivision depth n + 1. The curvature rank κ = f (n ₄ , n _i ) along the right side is greater than the current subdivision depth n + 1.

The condition of the above three conditions is immediately determined. However, in the conditions (1) and (2), the magnitude of the change in the unevenness (curvature rank) of the curved surface area bounded by two child triangles is sequentially and recursively determined in the peripheral direction as the reference result.

FIG. 6 is a diagram for explaining a method of dividing a parent triangle into two. As described with reference to FIG. 5, when the division of the parent triangle is determined, as shown in FIG.
A new grid point M is generated at the midpoint of the base, and is divided into two child triangles having the left and right sides of the parent triangle as bases. The recursion depth of the child triangle is n + 1, and the base length is l _{n + 1} =
InitQuadSize · (1 / √2) ^{n + 1} ), the left side length (= right side length) is l _{n + 2} (= InitQuadSize
・ (1 / √2) ^{n + 2} ). Further, the left and right child triangles are set as parent triangles, and the grid is divided recursively into two.

FIG. 6A shows the inheritance rule of the three lattice points (P ₀ , P ₁ , P ₂ ) of the parent triangle and the midpoint M of the base side when the child triangle is generated. Defined according to the grid point inheritance rules,
FIG. 6 shows the transition process of the first point (the left end point of the base) of the left child triangle.
(B). Each time the grid division proceeds once, the left end point of the base side of the left child triangle moves to the position of the distance d (= base length of the parent triangle × √2) in the + 45 ° direction, and the grid division becomes It can be seen that it approaches the center.

The grid division depth (recursive depth) is determined by the 3D curvature rank K of the grid element and the maximum grid width InitQuadSiz.
e and the minimum grid width MinQuadSize. The upper limit n _max is given by the following equation.

N _max ≦ log√2 (InitQuadS
(size) -log√2 (MinQuadSize) The maximum and minimum grid widths correspond to the size (base length) of the maximum and minimum triangular patches. Therefore, the grid division depth is determined by the required accuracy under visibility (triangular patch expression accuracy) together with the curved surface shape.

FIG. 7 is a diagram showing a state in which cracks have occurred in the approximate polygon obtained from the lattice representation. When processing to increase or decrease the number of grid points according to the shape, change, or continuity of each grid element is performed, different grid divisions are often performed between adjacent grid elements. For this reason, the connection relationship between the existing grid points is impaired, and the approximate polygon obtained from the grid representation has a problem that a crack (gap) as shown in FIG. 7 occurs. The occurrence of such cracks
If any of the above conditions (1) to (5) are satisfied, it can be avoided by dividing into left and right child triangles each having the left side and the right side as bases.

FIG. 8 is a diagram for explaining a neighborhood search range necessary to divide each grid element. The division determination of the parent triangle based on the above conditions and is performed as follows.

[0022] (1) As shown in FIG. 8 (b), sets the distant grid points of d = (Motohencho * 1 / √2) opposite the Motohen E _i of the parent triangle T, group An opposite parent triangle T 'sharing an edge is defined, and T and T' are divided into two areas of influence RI (E
_i ). T in affected regions RI (E _i), by reflecting any one of division determination T 'on the other, can be prevented cracking along Motohen E _i.

(2) For each parent triangle in each of the forward and reverse directions, and for each of the left and right child triangles based on the left side and right side of each parent triangle, if (1) is established, the parent Require the need for triangulation.

(3) If (1) is not established, the current child triangle is defined as a parent triangle, and the left and right child triangles are defined recursively with the left and right sides as bases, and the condition of (1) is determined. Perform

It should be noted, the division of the parent triangle is required under both normal and reverse directions of the change in shape of the divided region of influence of RI Motohen E _i (E _i). For this reason, even when the discontinuity of the 3D curvature occurs near either the forward side or the backward side of the base, the base is required to be divided, and both the forward and reverse triangles are symmetrically formed.
Divided. The larger the discontinuity of the 3D curvature value in RI (E _i ), the denser the grid points are generated near E _i , and the more grid points are generated on E _i at the same time. The discontinuous area is covered with a large number of acute triangles connecting these lattice points, thereby avoiding cracks.

The distance d from the base of the root triangle to the tip of the search area near the search (recursion) depth n is given by:

[0027]

(Equation 1)

Where k = [n / 2], InitQua
dSize is the initial grid width. When the 3D curvature is high near the base of the parent triangle, the search depth is shallow and the search distance is short. Conversely, when the 3D curvature is low in the vicinity of the base side, the search depth becomes deeper because the shape is searched farther away in order to confirm the necessity of grid point generation. However, FIG.
As shown in (3), since the search distance is given by an asymptotic series in (3), the upper limit of the search distance does not exceed the initialization grid width.

Therefore, as shown in FIG. 4C, the neighborhood search range required to divide each grid element is:
A total of 7 (=) surrounded by (1) each lattice element, (2) four lattice elements adjacent to four sides of each lattice element, and (3) four oblique half lattice elements contacting at each of the four end points. 1 + 4 + 1/2 ×
4) * InitQuadSize ² is limited to the size of the region. Therefore, the generation of a new grid point in each grid element does not affect the vicinity search range, and grid points can be generated by generating grid points in parallel between the grids.

The upper limit of the search depth (recursion depth) is given below by the maximum (initial) grid width and the minimum grid width.

N _max ≦ log√2 (InitQuadS
size) −log√2 (MinQuadSize) where MinQuadSize is the specified minimum grid width.

That is, the division of each lattice element is performed by 7 * Ini
It is completely determined by the shape of the neighboring region that is limited to the size of tQuadSize ^2. Also, from the above equation, it is possible to generate an adaptive grid by parallel calculation of how many times each grid element is divided.

FIG. 1 is a block diagram showing an embodiment for realizing the principle described with reference to FIGS. In this embodiment, the target image is a distance image obtained by a laser range finder or the like, and the 3D curvature described with reference to FIG. 2 calculated using the first and second derivatives of the pixel values is used as its geometric feature. Can be

In FIG. 1, the regular lattice generating apparatus 1 defines a square regular lattice (plane) in which the number of lattices in the X direction = X ₀ and the prime number of the elements in the Y direction = Y _{0 in} the input image as shown in FIG. The lattice initialization device 2 initializes the defined lattice (plane) and the data structure of each lattice element as shown in FIG. The neighborhood shape search device 3 recursively searches for the shape change of the initialized neighborhood neighborhood of each grid element in the “outside (asymptotic to the area boundary line) direction” as shown in FIG. Grid dividing device 4
Divides the grid elements recursively in the “inward direction (asymptotic to the center of the grid element)” from the boundary of each grid element based on the search result, as shown in FIG. The lattice expression generation device 5 generates a lattice expression (triangle patch expression) of the input image from each recursively divided lattice element.

An experimental result of generating an adaptive grid generation algorithm according to the present invention in C language will be described with reference to FIGS. In this experiment, a distance image of a real male face was used.

FIG. 9A shows an original distance image, and FIG. 9B shows the original distance image of FIG. 9A expressed by a convex elliptic point, a convex parabolic point, a convex hyperbolic point, or the like described with reference to FIG. (C) shows a portion having large unevenness by a line. In particular, FIG.
From (c), it can be seen that important facial features such as the nose, chin, and forehead are distributed in a region having a large 3D curvature value.

FIGS. 10A and 10B are diagrams showing the results of shape restoration using the initial grid. In particular, FIG. 10A shows a face expressed by a grid, and FIG. 10B shows a face image approximated by a polygon. (C) shows the result of shading the approximate polygon using the normal restored at each grid point.

FIG. 11 shows a case where the recursion depth is set to 1 and FIG.
FIG. 12 shows a case where the recursion depth is set to 5 when the recursion depth is set to 3.
2 shows the curvature reconstruction when the face image is selected, and it can be seen that the feature of the face image is expressed more finely as the recursion depth increases.

In this experiment, the number of grid points at each recursion depth 1, depth 2 and depth 3 (finally used points / searched points) were 1083/2386 and 1670/47, respectively.
51 points and 2483/9224 points. The calculation time is
As shown in Table 1, the recursive depth 1, depth 2, and depth 3 were 8 seconds, 13 seconds, and 28 seconds, respectively.

[0040]

[Table 1]

Table 2 shows the experimental results when no regular lattice was used.

[0042]

[Table 2]

As is apparent from a comparison between Table 1 and Table 2, in Table 1, the lattice points 2483 and the number of triangular patches 4749 are shown.
On the other hand, in Table 2, the grid points are 5146 and the number of triangular patches is 9618, and it can be confirmed that the use of the regular grid can reduce the number of grid points and triangular patches and achieve a polygon approximate expression with the same accuracy. .

In the above-described embodiment, the case where the distance image is used has been described. However, the present invention is not limited to this, and can be applied to adaptive compression and restoration of general images such as grayscale images and color images. .

[0045]

As described above, according to the present invention, a hierarchical triangular patch representation of a three-dimensional free-form surface is generated from a distance image using a lattice surface adapted to the uneven shape of curvature and its complexity. A complicated curved surface shape can be effectively and faithfully visualized.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of one embodiment of the present invention.

FIG. 2 is a diagram for describing 3D curvature.

FIG. 3 is a diagram showing local shapes of eight classification points and distribution positions in a curvature space.

FIG. 4 is a diagram showing a two-dimensional array of square lattice elements having an initial lattice width.

FIG. 5 is a diagram showing a process of forming two adjacent lattice elements into a three-stage binary tree structure.

FIG. 6 is a diagram illustrating inheritance rules of three lattice points of a parent triangle and a midpoint of a base side when a child triangle is generated.

FIG. 7 is a diagram showing a state in which a cladding has occurred in an approximate polygon obtained from a lattice representation.

FIG. 8 is a diagram for explaining a neighborhood search range required to divide a grid element.

FIG. 9 is a diagram showing a state in which the shape of a face image is described.

FIG. 10 is a diagram showing a result of shape restoration using an initial grid.

FIG. 11 is a diagram showing a shape restoration result using a two-division adaptive grid.

FIG. 12 is a diagram illustrating a shape restoration result using a 4-division adaptive grid.

FIG. 13 is a diagram showing a result of shape restoration using an 8-division adaptive grid.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Regular lattice generator 2 Grid initialization device 3 Neighborhood shape search device 4 Grid division device 5 Grid expression generation device

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Nobuyoshi Terashima Kyoto Prefecture

Claims (1)

(57) [Claims] 1. A distance image obtained by measuring a three-dimensional shape on an initial regular lattice plane which is set at equal intervals and a sparse lattice width in the horizontal and vertical directions and includes a line segment for defining a triangular patch expression, as an input image Grid initializing means for setting, based on the value of the pixel value at each grid point position set by the grid initializing means, the slope measured at the pixel value, and the curvature which is a change in the slope, A hierarchical lattice expression generating means for generating a hierarchical lattice expression for forming an approximate polygon by sequentially dividing the lattice plane into triangular patches of similar shape, and dividing the lattice plane generated by the hierarchical lattice expression generating means Means for determining a division of the approximate polygon based on the curvature and the hierarchy of the triangular patch around the lattice plane in order to avoid a gap between the approximate polygons generated in the process. , An adaptive and hierarchical grid representation generator for images.