KR102534282B1 - Method and apparatus for reducing the capacity of mesh data - Google Patents

Abstract

A method for reducing the capacity of mesh data is disclosed. A method for reducing the size of mesh data according to the present invention includes obtaining mesh data representing a 3D structure of space and including a plurality of vertices, comparing the mesh data with a 2D image of the space, and obtaining from the 2D image Extracting vertices belonging to the same plane of the same object from the mesh data using visual characteristics, and removing some vertices from the vertices within the same plane.

Description

Mesh data capacity reduction method and apparatus {METHOD AND APPARATUS FOR REDUCING THE CAPACITY OF MESH DATA}

The present invention relates to a mesh data capacity reduction method and apparatus capable of reducing the capacity of mesh data while minimizing shape distortion.

Recently, a technique for representing a three-dimensional structure of space using data such as cloud points is developing. In addition, mesh data including polygons, edges, and vertices is widely used to represent the 3D structure of space.

Since mesh data has a very large capacity, there is a problem in that its utilization is greatly restricted in storage, distribution, work using mesh data, and the like. In addition, existing technologies that reduce the capacity of image data by removing some dots, such as resolution conversion, cause problems such as resolution degradation and distortion of image data.

The present invention is to solve the above problems, and an object of the present invention is to provide a method and apparatus for reducing the size of mesh data, which can reduce the size of mesh data while minimizing shape distortion.

A method for reducing the capacity of mesh data according to an embodiment of the present invention includes obtaining mesh data representing a 3D structure of a space and including a plurality of vertices, comparing the mesh data with a 2D image of the space, and Extracting vertices belonging to the same plane of the same object from the mesh data by using visual characteristics obtained from, and removing some vertices from the vertices within the same plane.

Meanwhile, in the step of extracting vertices belonging to the same surface of the same object from the mesh data, when one surface of the same object includes a protrusion or depression, selecting a surface excluding the protrusion or depression as the same surface. can do.

Meanwhile, the removing of some vertices from vertices within the same plane may include maintaining vertices belonging to a boundary of the same plane and removing some of the vertices from vertices inside the boundary.

In this case, the removing of the partial vertices from the vertices inside the boundary may include removing the partial vertices so that at least one vertex among the vertices inside the boundary remains.

Meanwhile, the removing of some vertices from vertices within the same plane includes determining a degree to which vertices are removed from the same plane using visual characteristics obtained from the 2D image, and The visual characteristic to be used may include the type of the same object and a part to which the same surface belongs in the same object.

Meanwhile, the removing some of the vertices from the vertices within the same plane may include determining a degree to which vertices are to be removed from the same plane based on a degree of curvature of the same plane.

According to the present invention, there is an advantage in that a large amount of vertices, edges, and polygons in mesh data can be removed, thereby significantly reducing the capacity of mesh data. For example, a point cloud obtained from a lidar device may include millions of points, and when mesh data is constructed with these points, the capacity becomes very large. However, in the present invention, since the number of points is reduced to one-tenth/one-twentieth level, there is an advantage in that the capacity of mesh data can be significantly reduced and the utilization of mesh data can be increased.

According to the present invention, there is an advantage in that distortion of a shape can be minimized by using visual characteristics obtained from a 2D image while significantly reducing the size of mesh data.

1 is a block diagram for explaining an apparatus for reducing capacity of mesh data according to the present invention.
2 is a diagram for explaining a method for reducing the capacity of mesh data according to the present invention.
3 is a diagram illustrating mesh data and 2D images according to the present invention.
4 is a diagram for explaining a method of extracting vertices belonging to the same surface of the same object according to the present invention.
5 is a diagram for explaining a method of removing some vertices from vertices belonging to the same plane of the same object according to the present invention.
6 is a diagram for explaining a method of correcting mesh data using a 2D image of a space according to the present invention.

Hereinafter, the embodiments disclosed in this specification will be described in detail with reference to the accompanying drawings, but the same or similar elements are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. The suffixes "module" and "unit" for components used in the following description are given or used together in consideration of ease of writing the specification, and do not have meanings or roles that are distinct from each other by themselves. In addition, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiment disclosed in this specification, the detailed description thereof will be omitted. In addition, the accompanying drawings are only for easy understanding of the embodiments disclosed in this specification, the technical idea disclosed in this specification is not limited by the accompanying drawings, and all changes included in the spirit and technical scope of the present invention , it should be understood to include equivalents or substitutes.

Terms including ordinal numbers, such as first and second, may be used to describe various components, but the components are not limited by the terms. These terms are only used for the purpose of distinguishing one component from another.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

In implementing the present invention, components may be subdivided for convenience of description, but these components may be implemented in one device or module, or one component may be divided into multiple devices or modules may be implemented in

1 is a block diagram for explaining an apparatus for reducing capacity of mesh data according to the present invention.

The device 100 for reducing the size of mesh data according to the present invention (hereinafter, referred to as “device 100”) includes a communication unit 110, a control unit 120, a memory 130, an input unit 140, and an output unit 150. ) may be included.

All of the components shown in FIG. 1 are not essential components, and the color conversion device 800 may include fewer or more components.

The communication unit 110 may provide an interface for connecting the device 100 to a wired/wireless network including an internet network. Also, the device 100 may communicate with an external device through a network connected to the communication unit 110 or another network linked to the connected network.

In addition, the communication unit 110 may further include an input interface and receive data from the outside.

The controller 120 may control overall operations of the device 100 .

In addition, the memory 130 may store programs for processing and controlling each signal in the control unit 120, data for providing a GUI, signal-processed images, audio or data signals, and other data. Also, the memory 130 may store application programs or other data for operation of the device 100 . Also, the memory 130 may store at least one of mesh data and 2D data.

The input unit 140 may receive user input from the outside. To this end, the input unit 140 may include an interface such as a keyboard, a touch pad, or an input terminal.

The output unit 150 may output data processed in the device 100 . For example, the output unit 150 includes a display unit and may display mesh data, 2D images, and a synthesized image of mesh data and 2D images.

2 is a diagram for explaining a method for reducing the capacity of mesh data according to the present invention.

A method for reducing the size of mesh data according to the present invention includes obtaining mesh data representing a 3D structure of space and including a plurality of vertices (S210), comparing the mesh data with a 2D image of the space, and obtaining the mesh data It may include extracting vertices belonging to the same plane from (S220), and removing some vertices from the vertices within the same plane (S230).

First, S210 will be described with reference to FIG. 3 .

3 is a diagram illustrating mesh data and 2D images according to the present invention.

The mesh data may include at least one of a polygon 313 , an edge 312 , and a vertex 311 to represent a spatial structure. In detail, various objects may exist in the space, and the surfaces of these objects may be expressed as a set of polygons 313 . In addition, the polygon 313 is distinguished from other polygons by a plurality of edges 312, and the edge 312 may be created by connecting a vertex 311 and other vertices.

Mesh data may also represent the 3D structure of space. Specifically, the mesh data may include information capable of expressing the 3D structure of space.

For example, the mesh data may include 3D coordinate information of each vertex 311 or other information capable of tracking the 3D coordinates of each vertex 311 . However, it is not limited thereto, and various pieces of information capable of indicating a 3D location in space may be matched to at least one of polygons, edges, and vertices. Also, since polygons, edges, and vertices are formed on the surface of an object in space, mesh data may represent a 3D structure of space.

Meanwhile, the controller 120 may obtain mesh data including a plurality of vertices (S210).

As an example of the mesh data, the mesh data may be generated using a device for measuring a 3D structure of space, such as a LiDAR device. In this case, a point cloud is generated by a LiDAR device, and some or all of the points included in the point cloud may constitute vertices of mesh data.

However, it is not limited thereto, and mesh data may be generated using various data representing a 3D structure of space.

The control unit 120 may directly generate mesh data or receive mesh data through the communication unit 110 .

Meanwhile, as shown in FIG. 3, mesh data includes a large number of polygons, edges, and vertices. Therefore, since mesh data has a very large capacity, there is a problem that its utilization is greatly restricted in storage, distribution, work using mesh data, and the like. As an example, it is assumed that multiple workers are connected to the cloud and collaborate on the metaverse space. However, since mesh data has a very large capacity, it is very difficult to be compatible between operators. Therefore, in the present invention, the problem is solved by removing some of the vertices in the mesh data.

In addition, existing technologies that reduce the capacity of image data by removing some dots, such as resolution conversion, cause problems such as resolution degradation and distortion of image data. However, the present invention proposes a method for reducing the capacity without affecting the quality of data.

In the present invention, vertices are removed after comparing mesh data of a space with a 2D image of the space. Therefore, mesh data representing the 3D structure of space and 2D images of the same space are required.

As shown in FIG. 3 , the controller 120 may synthesize mesh data representing a 3D structure of a space and a 2D image of the space, and remove vertices using the synthesized image. However, it is not limited to this, and the 2D coordinates of objects in the 2D image and the 2D coordinates of the vertices in the mesh data (ie, the 2D coordinates converted from the 3D coordinates of the vertices to suit the composition of the 2D image) are obtained without synthesizing the two images. Vertices can also be removed by comparing them.

4 is a diagram for explaining a method of extracting vertices belonging to the same surface of the same object according to the present invention.

4 shows mesh data including a plurality of vertices and a 2D image.

The controller 120 may compare the mesh data representing the 3D structure of the space with the 2D image of the space, and extract vertices belonging to the same plane of the same object from the mesh data (S220).

First, the controller 120 may select the same object using visual characteristics obtained from a 2D image of a space. Here, the visual characteristic may include an object extracted from a 2D image of space.

Specifically, in order to preserve the 3D structure of objects in space, it is necessary to distinguish objects from each other. For example, if vertices are removed by treating the first object and the second object as the same object, distortion occurs in the 3D structure.

In addition, since it is difficult to classify objects only with mesh data, the controller 120 may classify objects in space using a 2D image and select one object among them as the same object. For example, the controller 120 may select a building among various objects in a space as the same object. For another example, the controller 120 may select a road as the same object among various objects in the space.

In addition, the controller 120 can distinguish two objects even when another object exists on top of one object in the 2D image. For example, when a window 420 exists on the building of FIG. 4 , the controller 120 may select only the building excluding the window 420 as the same object. In this case, the controller 120 may select the window 420 as a separate object different from the building and separately perform a process of removing some vertices.

Meanwhile, the controller 120 may select the same surface of the same object by using the visual characteristics obtained from the 2D image of the space. Here, the visual characteristics may include a surface of an object extracted from a 2D image of a space.

Specifically, in order to preserve the 3D structure of an object, it is necessary to preserve each facet of the object. Also, since it is difficult to classify faces only with mesh data, the controller 120 may classify a plurality of faces of an object using a 2D image and select one face. For example, the controller 120 may distinguish a plurality of surfaces of an object using boundaries (corners, etc.) of the surfaces in the 2D image and select one object. 4 shows a state in which the side surface 410 of the building is selected.

On the other hand, in selecting the same surface, different colors existing on one surface are not considered. For example, it is assumed that the upper surface of the road consists of a gray area and a yellow area (for example, a center line). However, mesh data is data for representing the shape of an object, and does not represent the color of the object. Accordingly, the controller 120 may select one surface including a plurality of colors as the same surface.

On the other hand, it is assumed that there are protrusions or depressions on one surface. Protrusions or depressions affect the shape of an object, and structural characteristics of the protrusions or depressions are not preserved when the protrusions or depressions are treated as one plane with the surroundings.

Accordingly, when one surface of the same object includes a protrusion or depression, the controller 120 may select a surface excluding the protrusion or depression as the same surface. For example, when a speed bump is installed on the upper surface of the road, the controller 120 may select the rest of the upper surface of the road except for the speed bump as the same surface. In this case, the control unit 120 may select the speed bump as a separate object different from the road, and separately perform processes for selecting the same surface and removing some vertices.

Even in this case, the controller 120 may select the same surface using visual characteristics acquired from the 2D image. Here, the visual characteristic may include the type of object.

Specifically, the controller 120 may select a protrusion or depression using the type of object in the 2D image. For example, a cab on the roof of a taxi or a speed bump on a road has a protruding structure. Therefore, the controller 120 can select the speed bump in the 2D image as the protrusion and separate the selected protrusion from the surrounding surface.

Also, the visual characteristics acquired from the 2D image may include information for selecting a protrusion or depression. For example, the controller 120 may select a protrusion or depression using shape information (curvature, shadow, etc.) extractable from a 2D image.

Additionally, the controller 120 may select the same object and the same face by using both the visual characteristics obtained from the 2D image and the structural characteristics obtained from the mesh data.

For example, the control unit 120 may classify objects and set boundaries between objects using visual characteristics acquired from 2D images and structural characteristics (bent structures, curved structures, etc.) obtained from mesh data. . In addition, the controller 120 accurately classifies the faces of the object or selects protrusions/depressions using visual characteristics obtained from 2D images and structural characteristics obtained from mesh data (a structure bent at a boundary, a curved structure, etc.) can do.

Meanwhile, when the same face of the same object is selected, the controller 120 may extract vertices belonging to the same face of the same object from mesh data. For example, when a side 410 of a building exists as shown in FIG. 4 , the controller 120 may extract vertices belonging to the corresponding side from mesh data.

5 is a diagram for explaining a method of removing some vertices from vertices belonging to the same plane of the same object according to the present invention.

5A is a diagram showing the same face of the same object before vertex removal.

The controller 120 may remove some vertices from vertices within the same plane (S230). In this case, the controller 120 may randomly remove some vertices from vertices within the same plane or may remove some vertices based on various criteria.

Meanwhile, vertices belonging to the same plane are divided into vertices belonging to the boundary of the same plane (eg, 511) and vertices inside the boundary (eg, 512 and 513).

Here, the boundary of the same plane distinguishes the same plane from another plane, and when a vertex (eg, 511) belonging to the boundary of the same plane is mistakenly removed, a loss is applied to the object shape itself. Accordingly, the controller 120 may maintain vertices (eg, 511) belonging to the boundary of the same plane and remove some vertices from the vertices inside the boundary.

5B is a diagram showing the same face of the same object after vertex removal. Referring to FIG. 5B , it can be seen that only vertices inside the boundary are removed while vertices belonging to the boundary of the same plane are not removed.

Meanwhile, all vertices belonging to the boundary may be maintained, but the present invention is not limited thereto, and the controller 120 may remove some vertices even from the vertices belonging to the boundary. However, even in this case, the degree of removing vertices belonging to the boundary may be smaller than the degree of removing vertices inside the boundary. For example, the controller 120 may remove 30% of the vertices belonging to the boundary and 90% of the vertices inside the boundary.

Meanwhile, the control unit 120 may remove some vertices so that at least one vertex among the vertices inside the boundary remains. Specifically, when all vertices inside the boundary are removed, edges and polygons are not formed on the corresponding face, and therefore, information about the shape of the face does not remain in the mesh data. That is, when querying mesh data, it may be recognized as having no corresponding face (that is, having a hole in it). Accordingly, the control unit 120 may leave at least one vertex inside the boundary and remove the remaining vertices. In FIG. 5B , it is illustrated that four vertices including the first vertex 512 and the second vertex 513 remain.

The control unit 120 may remove some vertices from vertices within the same plane while minimizing loss applied to the shape of the object.

Specifically, the shape of the surface may vary. Specifically, while some surfaces are perfectly flat, other surfaces may be somewhat curved. For example, while the windows of a car are flat, the bonnet of a car has a curved shape.

If it is assumed that the face is perfectly flat, even if there is only one vertex inside the boundary, the shape of the face can be perfectly expressed. However, if the surface has a curved shape and vertices are excessively removed, the shape cannot be perfectly expressed.

Accordingly, as an example, the controller 120 may determine the degree to which vertices are removed from the same surface based on the degree of curvature of the same surface. Specifically, when the first surface is flat and the second surface is curved, the degree of removing vertices from the first surface may be greater than the degree of removing vertices from the second surface. For example, the controller 120 may remove 99% of the vertices belonging to the first surface and 60% of the vertices belonging to the second surface. In this case, the controller 120 may calculate the degree of curvature of the same surface using at least one of visual characteristics obtained from the 2D image of the space and structural characteristics obtained from the mesh data.

As another embodiment, the controller 120 may determine the degree to which vertices are removed from the same plane by using visual characteristics obtained from a 2D image of the space. Specifically, the control unit 120 determines the degree to which vertices are removed from the same plane by using at least one of the type of the same object, the part to which the same plane belongs in the same object, and the shape information (eg, curvature, etc.) expressed from the 2D image. can decide

For example, when the same object is a road, the upper surface of the road has a slightly curved structure for drainage. In this case, the controller 120 may determine the degree to which vertices are removed from the upper surface of the road based on the type of object (road).

For another example, when the same object is a building, the side of the building has a flat structure. In this case, the controller 120 may determine the degree to which vertices are removed from the side of the building based on the type of object (building) and the part to which the same side belongs (side of the building).

Meanwhile, the controller 120 may also perform the processing of S220 to S230 on the other side of the same object and other objects. Also, when the processing is completed, the controller 120 may store new mesh data from which some of the vertices are removed in the memory 130 .

In addition, the controller 120 may store the new mesh data separately from the 2D image and utilize it separately from the 2D image. For example, in a situation where only a 3D structure of space is required, the controller 120 may transmit only new mesh data to the outside or process a job using the new mesh data. However, it is not limited thereto, and new mesh data may be used in combination with a 2D image or a texture extracted from a 2D image.

According to the present invention, there is an advantage in that a large amount of vertices, edges, and polygons in mesh data can be removed, thereby significantly reducing the capacity of mesh data. For example, a point cloud obtained from a lidar device may include millions of points, and when mesh data is constructed with these points, the capacity becomes very large. However, in the present invention, since the number of points is reduced to one-tenth/one-twentieth level, there is an advantage in that the capacity of mesh data can be significantly reduced and the utilization of mesh data can be increased.

In addition, according to the present invention, there is an advantage in that distortion of a shape can be minimized by using visual characteristics acquired from a 2D image while significantly reducing the size of mesh data.

6 is a diagram for explaining a method of correcting mesh data using a 2D image of a space according to the present invention.

A portion marked in pink in FIG. 6A is a portion for which points are not obtained due to diffuse reflection by water or the like. Therefore, the mesh data is in a state in which structural information (vertex, mesh, polygon) for the corresponding area is not included.

In this case, the controller 120 can restore structural information of the corresponding region using the visual characteristics of the 2D image. For example, the control unit 120 determines the structure of the corresponding region by using the type of object disposed in the corresponding region, the part to which the corresponding face belongs in the object, and shape information (eg, border, curvature, etc.) expressed from the 2D image. information can be restored. In this case, mesh data may be restored as shown in FIG. 6B. Also, the controller 120 may reconstruct a 2D image by synthesizing an image texture with the reconstructed mesh data.

The above-described present invention can be implemented as computer readable code on a medium on which a program is recorded. The computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drive (HDD), Solid State Disk (SSD), Silicon Disk Drive (SDD), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. there is Also, the computer may include a processor of a server. Accordingly, the above detailed description should not be construed as limiting in all respects and should be considered as illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

110: communication unit 120: control unit
130: memory 140: input unit
150: output unit

Claims (6)

obtaining mesh data representing a 3D structure of space and including a plurality of vertices;
extracting visual characteristics including an object in the space and a surface of the object from a 2D image of the space, and extracting vertices belonging to the same surface of the same object in the space from the mesh data using the visual characteristic; and
Removing some of the vertices within the boundary of the same plane among the vertices belonging to the same plane;
A method for reducing the size of mesh data. According to claim 1,
The step of extracting vertices belonging to the same plane of the same object in the space from the mesh data,
When one surface of the same object includes a protrusion or depression, selecting a surface excluding the protrusion or depression as the same surface;
A method for reducing the size of mesh data. According to claim 1,
The step of removing some vertices,
Maintaining vertices belonging to the boundary of the same plane and removing some of the vertices from vertices inside the boundary;
A method for reducing the size of mesh data. According to claim 3,
The step of removing some vertices,
removing some of the vertices so that at least one of the vertices inside the boundary remains;
A method for reducing the size of mesh data. According to claim 1,
The step of removing some vertices,
Determining the degree to which vertices are removed from the same plane using visual characteristics obtained from the 2D image;
Visual characteristics obtained from the 2D image,
Including the type of the same object and a part to which the same surface belongs in the same object
A method for reducing the size of mesh data. According to claim 1,
The step of removing some vertices,
Based on the degree of curvature of the same surface, determining the degree to which vertices are removed from the same surface; comprising
A method for reducing the size of mesh data.

Images