KR100397085B1 - Method for the graph representation of 3-dimensional object image for image retrival - Google Patents

Abstract

The present invention relates to a graph representation method of a three-dimensional image for retrieval of a three-dimensional object image. According to the method of the present invention, a volumetric step of expressing a triangular mesh representing a three-dimensional object image in volume (voxel), and skeletonizing the three-dimensional object image of the volumetric triangular mesh into a skeleton having a hierarchical structure for each layer And generating a node of the 3D image graph by expanding the 3D object image skeletonized for each layer using the layer value.

The three-dimensional image represented by the method of the present invention not only loses geometric information, but also has an excellent compression effect and can be usefully used for three-dimensional object recognition or efficient management of a large amount of databases due to its intuition-like shape. have.

Description

Graph generation method for retrieving 3D object images {METHOD FOR THE GRAPH REPRESENTATION OF 3-DIMENSIONAL OBJECT IMAGE FOR IMAGE RETRIVAL}

The present invention relates to a feature extraction method for recognizing a 3D object image, and more particularly, to a graph generating method for retrieving a 3D object image.

Recently, with the development of measurement technology of 3D distance data such as laser scanner and spatial coding technique, accurate 3D distance information of the object surface can be obtained. The acquired distance image is very useful for modeling a 3D object, and is particularly applied to fields such as computer aided design, reverse engineering, and computer graphics.

As the three-dimensional data obtained from the distance image increases exponentially, construction of three-dimensional images and model databases has been attempted in many ways. However, in general, a 3D image obtained by a laser scanner is difficult to store or transmit as it is enlarged due to huge coordinate information. Therefore, a suitable descriptor is needed for efficient management of the database. In addition, the shape descriptor used in the existing two-dimensional image retrieval technique brings a decrease in efficiency in the management of the three-dimensional image database. This is because the geometric information is not lost in the 3D image unlike the 2D image. Therefore, a new shape descriptor for 3D image database management is needed.

As a three-dimensional shape descriptor of the prior art, EGI (Extended Gaussian Image), 3D FD (3D Fourier Descriptor), MAT (Medial Axis Tramsform), and the like have been proposed. In the case where a 3D object is approximated as a polygon, EGI is a technique for describing an object by expressing a normal vector of each polygon on a sphere surface based on an angle. This EGI technique is also used for watermarking of 3D images. However, since the EGI technique only considers the local features of the object, it is difficult to describe the overall shape.

The 3D FD method is a method that a 3D image is expressed as a frequency component and used as a descriptor like various frequency conversions. This method is suitable for abstracting three-dimensional images by including most of the information in the low frequency region, but has a problem that is sensitive to the error of some part of the object and the movement of the center point.

Since MAT has been proposed by Blum, research has been conducted on various aspects of two-dimensional images. Although the study of skeletonization in 3D images is not progressing in earnest, methods using mathematical morphology or computational geometry are commonly used. MAT of 3D objects has the advantage of being similar to human intuition, but it is difficult to describe the shape of 3D image because it is composed of line segments and faces. Such existing three-dimensional shape descriptors can express the shape of an image, but there is no research to apply them so that they can be used for actual recognition or retrieval. In addition, it is difficult to apply an appropriate algorithm to a 3D image composed of a polygonal mesh due to the use of a real coordinate system.

There is a growing interest in content-based retrieval and browsing for more efficient indexing of rapidly growing three-dimensional image databases. In October 1996, the Moving Picture Expert Group (MPEG) began discussing MPEG-7 in the subtitle "MultimediaContent Description Interface." MPEG-7 is currently underway with the aim of finalizing international standards in November 2001 and aims to effectively provide multimedia information retrieval. However, there is no discussion about the descriptor of a full-scale three-dimensional image. In the case of three-dimensional images, unlike two-dimensional images, it is not affected by the direction of the occlusion or the viewpoint of the object. Therefore, it is appropriate to define a shape descriptor different from the two-dimensional image, and various methods have been proposed. . However, these descriptors do not provide proper indexing because of definitions that do not match human intuition. Therefore, it is necessary to propose a graph similar to human intuition as a new shape descriptor for three-dimensional images.

Therefore, an object of the present invention is to solve the above-described problem, and an object thereof is to provide a graph generation method for indexing and retrieving a 3D image.

Another method of the present invention is to provide a feature extraction method for use in indexing and retrieval of three-dimensional images.

According to the present invention for achieving the above object, the volumetric digestion step of representing a triangular mesh representing a three-dimensional object image in volume (voxel); Skeletonizing the 3D object image of the volumetric triangular mesh into a skeleton having a hierarchical structure for each layer; And generating a node of the 3D image graph by expanding the 3D object image skeletonized for each layer using the layer value.

1 is a block diagram suitable for implementing a method for generating a graph for searching a 3D image according to the present invention;

2 is a view illustrating a volume digestion process of a 3D image according to the present invention;

3 is a view illustrating a region filling process of a 3D image according to the present invention;

FIG. 4 is a diagram illustrating a 3D image according to the results of FIG. 2 and FIG. 3;

5 illustrates a skeletal hierarchical structure in accordance with the present invention;

6 is a view illustrating a node generation process according to the present invention;

7 is an exemplary diagram illustrating a result of generating nodes and edges for volumetric telephone image;

8, 9 and 10 illustrate the experimental results of the present invention, respectively.

<Description of the symbols for the main parts of the drawings>

100: volume digestion block 200: skeletal block

300: node generation block 400: node merge block

500: 3D image database

Hereinafter, with reference to the accompanying drawings will be described in detail the operation of the preferred embodiment according to the present invention.

Referring to FIG. 1, there is shown a block diagram suitable for implementing a graph generation method for retrieving a 3D image according to the present invention. The system of the present invention includes a volume digest block 100, a skeleton block 200, a node generation block 300, and a node merge block 400.

The volume digest block 100 corresponds to a pixel corresponding to a pixel of a two-dimensional image of an image of a three-dimensional object represented by a triangle mesh as illustrated in FIG. 8 (a), 9 (a), or 10 (a). As a means of expressing in the form of (voxel), this volume digestion process is largely composed of two steps.

The first is the process of expressing the 3D object image represented by the triangular mesh in volume, and the second is the process of filling the interior of the 3D object image by applying the mathematical morphological area filling algorithm based on the result of the first step. .

Computer graphics aim to convert a scene in three-dimensional space into a two-dimensional image. In computer graphics, raster graphics are used to display images. This method is a technique for representing scenes in three-dimensional space as pixels, and jagged patterns generated when converting straight lines or polygons to pixels. pattern) and rendering speed decreases. Digital radar analyzer (DDA) algorithm is mainly used to express straight lines in pixels in raster graphics. The DDA algorithm is an algorithm for determining a corresponding pixel when representing a straight line in raster graphics.

In the volume digestion block 100 of the present invention, the DDA algorithm is applied to the triangular mesh volumetric digestion of a three-dimensional object.

For example, FIG. 2 illustrates one triangle in a triangle mesh image as illustrated in FIG. 8 (a), FIG. 9 (a), or FIG. 10 (a). Suppose the vertices of the reference triangle are M and the other two vertices are L and R. At this time, the line segments connecting L and R are divided by equal intervals, and each interval is set as an increment Δ. Move P from L to R according to the set increment (Δ). At this time, when the volume digestion of the line connecting M and P is performed, the volume digestion for the triangle LMR is performed.

In more detail, if the volumetric digestion is performed by applying the DDA algorithm on the line connecting M and L, the volume of the line ML is obtained, and P is moved from L to R by an increment (Δ) to make the DDA algorithm. Is applied, the volume of the line segment MP is obtained. By repeating this method up to the line segment MR, the volume digestion of one triangular mesh of the three-dimensional object image is completed. When the above-described series of processes are repeatedly applied to all triangles of the triangle mesh 3D image, volumetric digestion of the 3D image is completed. That is, volume digestion may be referred to as a process of expressing the boundary of an object of a 3D image as a volume. The result of performing volume digestion is illustrated in FIG. 4A.

The second step is to express the inside of the boundary as a volume element for the 3D image on which volume digestion is performed.

Typically, when volumetric digestion is performed on the boundary of a three-dimensional object image represented by a triangular mesh, the interior of the object is also filled with a volume to apply a graph generation algorithm. It has the disadvantage of knowing information about.

The area filling method of the present invention adds two layers of volume in each axial direction to a volume of a portion other than the three-dimensional object image boundary made based on the bounding box of the object image. Then, the newly added volume place is not included in the 3D object by the definition of the bounding box, and can be defined as the outside of the object. Using the volume defined in this way, it is possible to fill the outer region of the 3D object image by applying the region morphology algorithm of mathematical morphology, and obtain the image with the inside of the image filled from the inverse of the image with the outer region filled. .

In more detail, the volume digesting block 100 of the present invention first sets the bounding box 130 surrounding the three-dimensional object 150, as illustrated in FIG. 3A.

Then, as illustrated in FIG. 3B, the outer region 140 is filled with a range of not including the three-dimensional object 150 outside the bounding box 130. The expression filling out here means padding with a specific volume.

The next process is again specified until the volume point representing the boundary of the three-dimensional object 150 is found inward from the bounding box 130 inside the volume area padded to a specific value, as illustrated in FIG. 3C. The inner area 140 of the bounding box 130 is filled by padding the volume of the value. As a result, the padding areas 140 and 160 completely surrounding the outside of the 3D object 150 and the inside area of the 3D object 150 are distinguished.

Subsequently, the area of the volume place padded with a specific value, that is, the areas outside the boundary 140 and 160 of the 3D object 150 and the inside of the boundary of the 3D object 150 are reversed. Then, the inside of the boundary of the three-dimensional object 150 is padded with a volume of a specific value, thereby filling the inside of the boundary of the three-dimensional object 150. The result is illustrated in FIG. 4B, and FIG. 4C is a diagram representing the boundary and the outside together.

The skeletalization block 200 repeatedly performs a process of expressing a skeleton of a hierarchical structure by using Equation 1 below on a three-dimensional object image in which volume digestion and region filling are completed.

here, Is a skeleton, Is a three-dimensional image expressed in volume, Is a hierarchy and B is a structural element for computation of mathematical morphology. , Are mathematical morphological calculations, respectively, which mean contraction and thermal operations.

That is, the skeletalization performed by the skeletal block 200 means a mathematical morphology that can only represent the skeleton of the important part of the object as a result of removing the important part from the original object image. For example, FIGS. 5A to 5E illustrate skeletal processes of three-dimensional images in which volume digestion is performed at a resolution of 38 * 15 * 34 for a telephone composed of 672 triangle meshes. Here, FIG. 5F refers to an image in which the skeletons of the hierarchical structures of FIGS. 5A to 5E are combined.

The node generation block 300 expresses a skeleton of each layer generated by the skeletal block 200 as nodes and generates nodes and edges from the 3D object image by generating edges from the connection information between the nodes. To perform.

According to the present invention, the node generation block 300 generates a node by expanding the skeleton of each layer by a layer value from an upper layer to a lower layer.

For example, FIG. 6A is a diagram illustrating a pixel block on a two-dimensional plane of a 12 * 7 object image that is volumetricized by a volumetric block before performing skeletalization. FIG. 6B illustrates a skeletal result of a 12 * 7 image having three hierarchies as a two-dimensional plane, and the skeleton 310 shows an image having a layer k of 2 and a skeleton 320. Denotes an image having a layer k of 1 and a skeleton 330 having five pixels.

In FIG. 6B, since the topmost skeleton 310 has k = 2, two expansion processes are performed, and the result is shown in FIG. 6C.

After the expansion of the uppermost skeleton 310 is performed, the expansion of the uppermost skeleton 310 is performed by the layer value k, that is, one expansion. In this case, expansion is not performed on the skeleton included in the expansion result of the previous layer, that is, 320 '. The result is shown in FIG. 6D.

Meanwhile, FIG. 6E illustrates the result of the node being created by zero expansion for the skeleton 330 having the layer k = 0. The expansion result obtained for each layer is generated as a union that is merged with each other as illustrated in FIG. 6F. When the expansion results overlap, the result of the upper layer is followed.

In order to actually become a node of the graph, it is necessary to label the nodes by extracting the connection components for the union as shown in FIG. 6F, which is implemented by using the connection component extraction algorithm of mathematical morphology. The connection component extraction algorithm is an algorithm that determines how many parts the entire image is composed by extracting a part having the same value among the pixels.

The edges are configured by determining whether the generated nodes are connected or not. In the node generation scheme of the present invention, the node generation results do not coincide with the existing image. This is because the skeleton contained in the node of the higher level is not used to generate the node. That is, although the existing image can be restored from the result of skeletalization, the node generation result does not coincide with the existing image because part of the skeleton is lost in the node generation process. Therefore, in order to determine the connectivity between nodes, edges are extracted while comparing a separate image reconstructed from a skeleton with a node generation result.

FIG. 7 is the result of 53 nodes and edges generated for the phone image shown in FIG. 6F volumetricized at a resolution of 38 * 15 * 34. FIG. 7A shows the different colors of nodes according to hierarchy in the volumetric telephone image. Figure 7b is a representation of the sphere by labeling each volume by node and by setting the radius of the skeleton to a radius around the average value of the volume corresponding to each node.

When the graph of the 3D object image is generated by the node generation method according to the present invention, as shown in the telephone image of FIG. Therefore, the post-processing process is required to express the appropriate graph through the merging process between nodes. In the present invention, it is determined whether to merge the nodes by setting the size condition and the distance condition. The merging process of nodes is a process of generating an appropriate graph by repeatedly applying a size condition and then applying a distance condition. In principle, when two nodes are merged, a node having a lower hierarchy is merged into a node having a higher hierarchy.

A node is a set of volume elements generated from a skeleton and has a hierarchy of skeletons. The higher the hierarchy of nodes and the larger the number of volume components constituting the node, the more important the image. Therefore, by setting an appropriate threshold and merging nodes having a smaller number of volumes than the threshold, the graph can be represented by an appropriate number of nodes. However, in the case of limb animals whose legs are smaller than those of the torso, a node having a low hierarchy and a small number of volumetric elements is generated in the leg portion, and the nodes may be merged with other nodes to generate a graph of missing features.

Accordingly, the node merging block 400 of the present invention sets a threshold for the nodes of each layer and determines whether to merge the nodes according to the layer. However, when the threshold is set for each layer and the merge is performed, the number of nodes decreases toward the higher layer, resulting in some of the nodes of the upper layer being merged with other nodes. Accordingly, a threshold is set for the total number of volume locations of the nodes, and a size condition for determining whether to merge nodes is set for a layer having a threshold smaller than this threshold as shown in the following equation.

In Equation 2, Means the hierarchy of the skeleton, silver The number of nodes included in the hierarchy.

Nodes are sorted in ascending order of number of volume, Is In the hierarchy The number of volumes in the first node. Is The threshold obtained from the histogram of the number of volumes of nodes included in the hierarchy, Is the threshold obtained from the histogram of the number of volumes of all nodes. The threshold setting method uses a technique that maximizes the degree of separation when a given set is divided into two parts.

By considering whether nodes are merged in each layer in the size condition, there is a possibility that a meaningless part of the nodes in the lower layer is actually included in the graph. There is also the possibility that the graph will contain parts that need to be merged into one of the nodes of the same layer.

Therefore, according to the present invention, in order to merge these nodes, a distance condition for comparing the distance between the centers of each node with the sum of the hierarchical values of each node is set as in Equation 3 below.

In Equation 3, Wow Means the hierarchy of the skeleton, Is the center of the node, Means Euclidean distance. Since the volume is a cuboid, it is a constant to consider the length of the diagonal.

The telephone image, which is a three-dimensional object generated by merging by the node merging block 400, is illustrated in, for example, FIG. 8C, 9C or 10C, and these images are stored in a three-dimensional image database, and later three-dimensional. It is used as an index graph for content-based retrieval and browsing of images.

Experimental Example

In order to evaluate the performance of the graph representation method of the proposed 3D image, simulations were performed on three 3D images. Three-dimensional images were represented by triangular mesh, and volume digestion was implemented by dividing the shortest side by 15 equal to the bounding box of the image. The final graph representation stores the information of nodes and edges, and nodes are radiused spheres and edges are represented by line segments. In the present invention, the experiment was performed under the hardware of Intel Pentium III 600MHz, 192MB memory and Windows NT Server 4.0 operating system.

8, 9, and 10 are experimental results for the telephone, toy, and cow. Tables 1 and 2 describe the results of the volume digestion, skeletalization, and graph representation techniques, and the execution time of each algorithm. In Table 2, the number of volumes for image representation refers to the number of volumes required to represent an image when volume digestion is performed for a given 3D image, and the number of volumes for graph representation represents the image of nodes and edges. When expressed as a graph, it means the number of volumes required to construct a node.

object Number of points Number of meshes Volumetric resolution Layer of skeleton Number of graph nodes cellphone 338 672 38 * 15 * 34 4 3 Eldest son 1944 3808 29 * 23 * 15 3 8 small 2904 5804 46 * 28 * 15 5 6

object Volume fire extinguishing time (sec) Skeletal execution time (sec) Graph creation execution time (sec) Volume count for image representation Volume count for graph representation cellphone 2.093 1.953 9.784 3664 2999 Eldest son 1.202 0.791 4.236 1945 1858 small 2.454 2.543 13.410 5975 5129

For the three 3D images of FIGS. 8, 9 and 10, it can be seen that the number of volume elements for image representation and the number of volume elements for graph representation do not coincide. It means not to perform a restore.

8 (a), 9 (a) and 10 (a) show the experimental images in a wireframe, and FIGS. 8 (b), 9 (b) and 10 (b) show volumetric digestion. The resulting image is represented by an appropriate node, and Figs. 8 (c), 9 (c) and 10 (c) show the results of the graphs of Figs. 8 (a), 9 (a) and 10 (a). to be. The sphere radii of FIGS. 8 (c), 9 (c) and 10 (c) are determined according to the hierarchy of nodes.

Looking at the results of the graph representation, it can be seen that the phone image is represented by a handset, a handset, a handle, and a toy or a cow is represented by a head, a body, and four legs. In particular, it can be seen that in the case of toy dogs, the tail and ears are represented by nodes to describe the image in more detail.

As described above, the graph of the three-dimensional image stored in the database using the graph representation method of the present invention not only loses geometric information by using skeletalization, but also has an excellent compression effect and is similar in shape to human intuition. It can be useful for 3D object recognition and efficient management of large amounts of databases. In addition, in the case of an object having a similar structure, discriminative indexing and recognition can be performed on a simple scale based on the number of volume places stored in the node and the distance information between the nodes stored in the edge. In addition, by using the mathematical morphology to approach the three-dimensional image from the perspective of the volume place, it has the advantage that it is more resistant to noise than the method using the computational geometry.

Claims (9)

In the graph representation method for searching a three-dimensional object image, A volume digestion step of expressing a triangular mesh representing a three-dimensional object image in volume; Skeletonizing the 3D object image of the volumetric triangular mesh into a skeleton having a hierarchical structure for each layer; And generating a node of a 3D image graph by expanding the 3D object image skeletonized for each layer using the layer value. The method of claim 1, wherein the volume digesting step: Volume-diminishing the three-dimensional object image to establish a boundary of the three-dimensional object image; And filling the region of the boundary of the 3D object image. The method of claim 2, wherein the boundary setting of the 3D object image comprises: Dividing a line segment connecting two vertices of each triangle of the triangle mesh at equal intervals and setting each interval as an increment; And performing volumetric digestion of all the triangles of the triangular mesh by performing volumetric digestion of another line segment generated while moving by the set increment along the line segment. Graph representation method for 3D object image retrieval. The method of claim 2, wherein the filling of the region is: Setting a bounding box surrounding a boundary of the 3D object image; Padding a range of outer regions that do not include the object outside the bounding box with a volume of a specific value; Padding with the volume of the particular value from within the padded volume region to a volume representing the boundary of the object; And inverting the inside of the object boundary by padding the inside of the object boundary with the inside of the object boundary by inverting the volume of the specific value inversely to the inside of the object boundary. Graph representation method. The method of claim 1, wherein the node creation step is: Performing expansion by the hierarchical value of the corresponding skeleton sequentially from the highest skeleton to the lowest skeleton; Combining the expanded skeleton for each layer into a skeleton image of the hierarchical structure; And extracting the connection components of the merged hierarchical skeletons as nodes of a 3D image graph and generating edges of the 3D image graph according to the information connecting the nodes. Graph representation method for image search. 6. The method of claim 5, wherein the hierarchical skeletalization excludes the expansion of the skeleton of the lower layer included in the expansion result of the previous upper layer. The method of claim 1 wherein the method is: And a node merging step of merging nodes having a lower hierarchy into nodes having a higher hierarchy with respect to the nodes of the 3D image graph. 8. The method of claim 7, wherein the node merging step merges nodes by setting size and distance conditions; The size condition satisfies the following equation, In the above equation, Is a hierarchy of skeletons, silver The number of nodes in the hierarchy, Is Nodes are sorted in ascending order of number of volume, In the hierarchy Number of volumes in the first node, Is The threshold obtained from the histogram of the number of volume places of the nodes included in the hierarchy, Is a threshold obtained from the histogram of the number of volumes of all nodes; The distance condition satisfies the following equation, In the above equation, Wow Is a hierarchy of skeletons, Is the center of the node, Is Euclidean street, The graphing method for the three-dimensional object image search, characterized in that the constant. The method of claim 1 wherein the method is: And storing the node and edge information of the three-dimensional image graph in a database as search and browsing information representing the three-dimensional object image.