KR20240020800A - Mobile digital twin implement method and apparatus using space modeling - Google Patents

Abstract

A method and device for implementing a mobile digital twin through spatial modeling are disclosed. A mobile digital twin implementation device through spatial modeling according to an embodiment of the present invention is a mobile device that implements a digital twin of real space, and captures real space and objects placed therein with a camera module to provide color images and depth. A mobile scanning unit that generates images; a modeling unit that processes the color image and the depth image to generate a point cloud and generates a three-dimensional model from the point cloud; a mesh analysis unit that analyzes the mesh data of the three-dimensional model to classify planes belonging to the same object and classifies and assigns space and object properties; a space object separation unit that divides the planes into a space model plane and an object model plane based on properties given to the planes, and separates the space model and the object model from each other; and a storage unit that stores the space model and the object model in memory or a database.

Description

Mobile digital twin implementation method and apparatus using space modeling}

The present invention relates to modeling technology, and more specifically, to a method and device for implementing a mobile digital twin through spatial modeling.

3D scanning refers to a technology that uses an image sensor to create a 3D model representing the 3D shape of an object. There are two types of 3D scanning methods: a manual method in which a person scans various parts of an object while holding a scanner and adjusting its position, and an automatic method in which various parts of an object are scanned while the position of the scanner or object is automatically adjusted by a robot.

A 3D model can be created using scan data obtained through such 3D scanning. Previously, there was object-level modeling, but modeling through mobile scanning of an entire space such as an office or home had its limitations.

Korean Patent Publication No. 10-2020-0045321 (published May 4, 2020) - Spatial scanning device and its information processing method

The present invention is a method of implementing a mobile digital twin through spatial modeling that can implement a digital twin on a mobile through spatial scanning and modeling using a mobile terminal and provide users with a 3D digital recreation of real space and objects. and to provide devices.

The present invention creates a three-dimensional model of a space captured using a camera installed in a general mobile terminal owned by the user, and separates objects placed in the space to create a model of pure space through spatial modeling. It is intended to provide a digital twin implementation method and device.

Other objects of the present invention will become clearer through the preferred embodiments described below.

According to one aspect of the present invention, a mobile device that implements a digital twin of a real space, comprising: a mobile scanning unit that captures real space and objects placed therein with a camera module to generate a color image and a depth image; a modeling unit that processes the color image and the depth image to generate a point cloud and generates a three-dimensional model from the point cloud; a mesh analysis unit that analyzes the mesh data of the three-dimensional model to classify planes belonging to the same object and classifies and assigns space and object properties; a space object separation unit that divides the planes into a space model plane and an object model plane based on properties given to the planes, and separates the space model and the object model from each other; and a storage unit that stores the space model and the object model in a memory or database. A mobile scan digital twin implementation device is provided.

The space model may be a result of modeling the space itself, excluding the object from the real space where the object is placed.

The mobile scanning unit displays the shooting screen in a preview format through the display unit during the mobile scanning process, and covers areas where points belonging to the point cloud have not been created or have not been created completely with points of a specific color, so that the preview screen The point of the specific color can be induced to disappear.

The modeling unit may perform a data preparation step of generating triangular mesh data using points of the point cloud.

The mesh analysis unit includes a data separation step of grouping triangles of the mesh data into mesh groups, which are a set of triangles constituting the same plane, where triangles of the mesh data are connected to each other and the angle between the corresponding normal and the reference normal is small; And a group classification step may be performed to classify the mesh groups into space groups and object groups depending on whether the spatial conditions are met.

The spatial object separation unit may perform spatial merge and object merge on the divided groups, and may ultimately perform a spatial merge step and an object merge step to separate the separated groups into the spatial model and the object model.

It may further include an editor that deletes the object model from the 3D model according to a user command, and displays only the spatial model or displays another object model by placing it at a random position in the spatial model.

Meanwhile, according to another aspect of the present invention, there is a computer program stored in a computer-readable medium for performing a mobile scan digital implementation method, wherein the computer program causes the computer to perform the following steps, wherein the steps include modeling Generating a point cloud by processing the color image and depth image generated by the mobile scanning unit in the unit; A data preparation step of generating triangular mesh data using points of the point cloud in the modeling unit; A data separation step of analyzing the mesh data in a mesh analysis unit and grouping it into mesh groups, which are sets of triangles constituting the same plane; A group classification step in which the mesh analysis unit classifies the mesh groups into space groups and object groups according to whether spatial conditions are met; And a computer program stored in a computer-readable medium including a spatial merge step and an object merge step of performing spatial merge and object merge on the group classified in the spatial object separation unit and finally separating the group into the spatial model and the object model. This is provided.

In the data preparation step, the modeling unit may index co-located vertices and adjacent triangles and manage them in advance as a first dictionary and a second dictionary.

In the data separation step, by using the first dictionary and the second dictionary to find the same location vertex and adjacent triangles, the triangles of the mesh data are connected to each other, and triangles with a small angle between the corresponding normal and the reference normal are created. It can be grouped into mesh groups, which are sets of triangles constituting the same plane.

In the group classification step, checking whether the number of triangles included in each mesh group exceeds a predetermined number; determining an object group if the number is less than a predetermined number; If the number is more than a predetermined number and satisfies the floor conditions, wall conditions, and ceiling conditions of the space consisting of the floor, wall, and ceiling, the properties are set to floor, wall, and ceiling respectively and determined as a space group, and the floor condition, the wall condition, and If all of the ceiling conditions are not met, determining the object group may be included.

In the spatial merge step, it is determined whether the spatial merge condition is satisfied for each object group, and if the spatial merge condition is satisfied, the point of the object group can be added to the spatial group, and the object group can be deleted.

In the object merging step, object recombination is performed through expansion of the object plane, but expansion into the spatial plane is blocked, thereby creating a new object group created as the object model.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to an embodiment of the present invention, there is an effect of implementing a digital twin on a mobile device through spatial scanning and modeling using a mobile terminal and providing a three-dimensional digital recreation of real spaces and objects to users.

In addition, it has the effect of creating a three-dimensional model of the space captured using the camera installed in the user's general mobile terminal and creating a model of pure space by separating objects placed in the space.

1 is a configuration diagram of a mobile digital twin implementation device through spatial modeling according to an embodiment of the present invention;
2 is a flowchart of the mobile scanning process performed in the mobile scanning unit;
Figure 3 is a flowchart of a method for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention;
Figure 4 is a diagram showing the detailed process of the data preparation stage;
Figure 5 is a diagram showing the detailed process of the data separation step;
Figure 6 is a diagram showing the detailed process of the group classification step;
Figure 7 is a diagram showing the detailed process of the spatial merge step;
Figure 8 is a diagram showing the detailed process of the object merge step;
Figure 9 is an example of a preview screen during mobile scanning and a modeling screen after mobile scanning is completed;
Figure 10 is an example diagram showing the separation of a space model and an object model.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to designate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the components of the embodiments described with reference to each drawing are not limited to the corresponding embodiments, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention, and may also be included in separate embodiments. Even if the description is omitted, it is natural that a plurality of embodiments may be re-implemented as a single integrated embodiment.

In addition, when describing with reference to the accompanying drawings, identical or related reference numbers will be assigned to identical or related elements regardless of the drawing symbols, and overlapping descriptions thereof will be omitted. In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Figure 1 is a configuration diagram of an apparatus for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention.

A method and device for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention implements a digital twin by scanning and modeling real spaces and objects using a mobile terminal, recreating them in three-dimensional digital form, and providing them to the user. It is characterized by A three-dimensional model is created by scanning a real space such as an office or home using the camera of a mobile device that the user generally carries for phone calls, internet, etc., and objects placed in the space are separated from the space. You can create a purely spatial model.

Referring to FIG. 1, the mobile digital twin implementation device 100 according to this embodiment may be implemented as a mobile terminal such as a smartphone or tablet PC. However, the mobile digital twin implementation device 100 is not limited to this. The mobile digital twin implementation device 100 may be implemented as a separate device for performing the mobile digital twin implementation method according to an embodiment of the present invention.

The mobile digital twin implementation device 100 may include a mobile scanning unit 110, a modeling unit 120, a mesh analysis unit 130, a spatial object separation unit 140, and a storage unit 150.

The mobile scan unit 110 takes images of the real space to be 3D modeled. The real space subject to filming may be an indoor space such as an office or home.

The mobile scan unit 110 is a camera module provided in a mobile terminal and may include a video camera for color image capture and a depth sensor such as LiDAR for depth image capture.

The image captured by the mobile scanning unit 110 may be displayed on the screen in a preview format through a display unit provided in the mobile terminal. Through the preview, users can check the current scan area and check the progress of the scan.

The modeling unit 120 processes data from the image captured by the mobile scanning unit 110 to create a point cloud, and creates a plurality of meshes from the point cloud to create a three-dimensional model of the space where objects are placed. Create.

The mesh analysis unit 130 analyzes the mesh data of the three-dimensional model generated by the modeling unit 120, classifies planes belonging to the same object, and classifies and assigns space and object properties.

The spatial object separator 140 may distinguish and separate a plane of a spatial model for a space and a plane of an object model for an object placed in the space based on the properties given to the planes.

The storage unit 150 can store the final modeled spatial model and object model in the terminal's memory, or transmit them to an external server connected to a network and store them in a separate database. The saved space model and object model can be loaded into the viewer/editor 160 or shared with other users according to user input.

In this embodiment, the space model refers to the modeling result of the space itself excluding the object from the real space where the object is placed, and the object model refers to the modeling result of the object itself.

The viewer 160a displays the space model and/or object model stored in the storage unit 150 on the screen of the display unit. Functions such as rotation, enlargement, reduction, and size display allow you to check the modeled model through mobile scanning from various viewpoints.

The editor 160b may perform an editing function on the model displayed on the screen in response to user input. Editing functions may include object deletion, box clipping, plane cropping, object placement, etc.

By using the editor 160b to place a desired object model within the space model, the user can virtually place the object and check whether it fits the space without purchasing and placing the object in person.

The data processing process performed in the above-described modeling unit 120, mesh analysis unit 130, and spatial object separation unit 140 includes a data preparation step, a data separation step, a group classification step, a spatial merge step, and an object merge step. can be included and performed sequentially.

Hereinafter, the mobile digital twin implementation method performed in the mobile digital twin implementation device 100 through spatial modeling described above will be described in detail with reference to the related drawings.

FIG. 2 is a flowchart of the mobile scanning process performed in the mobile scanning unit, FIG. 3 is a flowchart of a method of implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention, and FIG. 4 shows a detailed process of the data preparation step. Figure 5 is a diagram showing the detailed process of the data separation step, Figure 6 is a diagram showing the detailed process of the group classification step, Figure 7 is a diagram showing the detailed process of the spatial merge step, and Figure 8 is an object This diagram shows the detailed process of the merge stage.

First, referring to FIG. 2, a mobile scan process performed by the mobile scan unit 110 is shown.

The mobile scan unit 110 may perform a mobile scan process using a camera module provided in the mobile terminal (step S200).

The camera module may include a video camera such as a CCD camera or CMOS camera for capturing a color image (RGB Image), and a depth sensor for capturing a depth image. Accordingly, a color image can be acquired using a video camera (step S205) and a depth image can be acquired using a depth sensor (step S210).

Alternatively, the camera module may be an RGB-D camera that can acquire both color images and depth images.

In addition, the camera module is equipped with an augmented reality kit (AR kit) for implementing augmented reality, so that information on the location of the camera currently being filmed can be obtained (step S215). Additionally, lens characteristic information regarding the lens installed in the camera can be additionally obtained.

Using the depth image captured by the depth sensor and the camera position information acquired by the augmented reality kit, the mobile scanning unit 110 generates a point cloud, which is a set of points about the space currently being photographed, and as needed. can update it (step S220).

The generated and updated point cloud may be projected into the camera coordinate system (step S225). The camera coordinate system is created based on camera position information acquired by the augmented reality kit, and thus the coordinates of each point of the relative point cloud can be obtained.

For the points of the point cloud projected into the camera coordinate system, the color of each point can be designated as a color related to the corresponding coordinate of the color image area and output (step S230).

The screen captured by the camera module may be displayed in a preview format through the display unit and provided to the user. On the preview screen, parts of the point cloud that have not been created or whose creation has not been completed are displayed in a pre-specified manner, giving users who are filming real space the information that needs to be collected to create/update the point cloud for 3D modeling. You can check the points intuitively.

Here, the predefined method displays the point parts for which the point cloud has been created and updated as a normally captured screen, and the part for which the point cloud has not yet been created is in a specific color (a rare color in normal shooting situations, for example, orange). It may be covered with points (see (a) of FIG. 9). By applying this display method to encourage user behavior so that points of a specific color disappear as much as possible from the preview screen, enough points for 3D modeling can be collected to complete the point cloud.

When the scan is completed by the mobile scanning unit 110 and a point cloud is generated, it can be considered that basic data for 3D modeling has been prepared. Afterwards, the mobile digital twin implementation method can be executed as shown in FIG. 3. Depending on the situation, the mobile scanning process may be included and executed together with the mobile digital twin implementation method.

Referring to Figure 3, the mobile digital twin implementation method includes a data preparation step (S300), a data separation step (S305), a group classification step (S310), a spatial merge (small object merge) step (S315), and an object merge (object recombination) step. ) may include step (S320).

Referring to FIG. 4, in the data preparation step (S330), the modeling unit 120 generates triangle-type mesh data using points of the point cloud generated by the mobile scanning unit 110.

Mesh data may be a data set consisting of sub-data of vertices, triangles, and normals. A vertex is the vertex of a triangle, and the normal is a unit vector perpendicular to the plane of the triangle.

Here, in a situation where there is multiple mesh data, some vertices may exist in the same location but belong to different meshes and are logically separated. In this case, the vertex ID may be different, and during data processing such as subsequent merging or model deletion by the editor, a problem may occur in which one vertex is deleted but the other vertex is not deleted even though it is in the same location. .

Therefore, in the data preparation stage, even if the vertex IDs are different, the process of identifying and indexing vertices at the same position and managing the index in a separate first dictionary (same position vertex index dictionary) can be preceded. Through this, it is possible to immediately identify the vertex at the same location through the first dictionary without having to calculate it separately each time, thereby reducing the amount of calculation, allowing modeling, viewing, and editor tasks to be performed smoothly even on mobile devices.

Additionally, in the process of analyzing triangle data, information about triangles with adjacent vertices that are not at the same location but are located within tolerance is collected. In the plane expansion process described later, the planes are connected through the triangle expansion method. Information about triangles whose vertices are adjacent to one triangle is indexed, and the index is stored in a separate second dictionary (Vertice adjacent triangle index dictionary). It can be managed with Afterwards, adjacent triangles can be immediately identified through the second dictionary without having to be calculated separately each time, thereby reducing the amount of calculation, allowing modeling, viewing, and editor tasks to be performed smoothly even on mobile devices.

When data preparation is completed, in the data separation step (S305), the mesh analysis unit 130 finds a set of triangles that are connected to each other and have a small angle between the corresponding normal and the reference normal, and treats them as triangles constituting the same plane.

Referring to FIG. 5, it is checked whether the vertices have been reviewed (S305-1). If it is a vertex that has not been reviewed, add it to the checklist (S305-2). Check whether the checklist is empty (S305-3), and if not, extract the vertices of the checklist (S305-4). For the corresponding vertex, vertices at the same location are found through the first dictionary prepared in the data preparation step (S300) (S305-5), and triangles (adjacent triangles) having adjacent vertices associated with the corresponding vertex are found through the second dictionary. (S305-6). By using a dictionary, you can quickly find co-located vertices and adjacent triangles and reduce the amount of calculation for search.

Additionally, the benchmark normal of the vertex (or the mesh to which the vertex belongs) is found (S305-7). The reference normal may be the normal vector of a triangle belonging to the mesh to which the corresponding vertex belongs.

Compare the normal line of the adjacent triangle with the reference normal line to determine whether the angle between them is within a preset angle (S305-8). Angle comparison may be made depending on whether the dot product of the two normals exceeds a predetermined reference value (eg, 0.75).

If it is within a preset angle (i.e., if the inner product of two normals exceeds a predetermined standard value), the triangle to which the current vertex belongs and the adjacent triangle are considered to have normals in the same direction, and can be determined to be triangles belonging to the same plane. (S305-9).

For triangles belonging to the same plane, vertices belonging to the triangle are extracted (S305-10), added to the checklist again, and the above-described process is repeated.

Through the above-described process, it is possible to find a set of triangles that are connected to each other and face the same direction, that is, forming the same plane. By growing triangles in this way, the triangles corresponding to each plane that makes up space or objects can be separated and grouped separately.

Data grouped into coplanar triangles goes through a group classification step (S310).

Referring to FIG. 6, the mesh analysis unit 130 performs analysis on data (mesh group) grouped in the same plane. The purpose of this analysis is to classify into space groups and object groups depending on whether the spatial conditions are met.

First, check whether the number of triangles grouped and included in one plane exceeds a predetermined number (for example, 10) (S310-1).

If the number is less than a predetermined number, it is considered not to be a plane related to a space with a large plane and is determined to belong to an object group (S310-9).

If the number exceeds a predetermined number, it is determined whether each condition of a space having a large plane is met. The plane that makes up space includes the floor, walls, and ceiling.

First, determine whether the floor condition is met (S310-2), and if the floor condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to floor (S310-3). And it is determined to belong to the space group (S310-8).

Next, determine whether the wall condition is met (S310-4), and if the wall condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to wall (S310-5). And it is determined to belong to the space group (S310-8).

Next, determine whether the ceiling condition is met (S310-6), and if the ceiling condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to ceiling (S310-7). And it is determined to belong to the space group (S310-8).

The order of judgment regarding floor conditions, wall conditions, and ceiling conditions among space conditions is only an example, and the order may be changed.

If the floor condition, wall condition, and ceiling condition are all not met, the plane in question does not constitute a space, but can be viewed as a plane that constitutes one of the objects placed in the space. In this case, the points and mesh groups belonging to the plane is determined to belong to the object group (S310-9).

In this embodiment, the floor conditions are (1) whether the normal direction of the plane is upward, (2) whether the area is a certain size or more, and (3) whether it is located in a low position.

The wall conditions are (1) whether the normal direction of the plane is horizontal, (2) whether the area is over a certain size, and (3) whether the height is over a certain height (for example, 2 m).

Ceiling conditions are (1) whether the plane's normal direction is downward, and (2) whether it is located at a high location.

When the analysis by the mesh analysis unit 130 is completed and divided into a space group and an object group, the space object separation unit 140 performs a merge on the divided groups and finally completely separates the space model and the object model. .

Referring to FIG. 7, in the spatial merge step (S315), the spatial object separation unit 140 performs a check operation on all object groups targeting spatial groups (S315-1). Space groups can be broadly divided into floors, walls, and ceilings, and walls can be reclassified into multiple walls according to the normal direction.

During the check process, each object group (S315-2) checks whether it satisfies the spatial merge conditions (S315-3), and if it satisfies the spatial merge conditions, the points belonging to the object group are added to the spatial group, and the corresponding object Delete the group (S315-4).

The space merge conditions are (1) whether it is included in the space group currently being checked, whether the normal direction is parallel, and whether the number of points is less than a certain number (for example, 100).

Through the spatial merge step (S315), small object meshes within the spatial mesh are merged, making it possible to set the mesh data constituting the space as a spatial group among the remaining data that was not classified during the group classification process.

Mesh data set as a space group can be assigned floor, wall, and ceiling properties according to the above-mentioned classification conditions, and can be integrated as a space model excluding objects placed inside the mobile scanned space.

Next, the object merge step (S320) is performed through object recombination.

Referring to FIG. 8, in the object merging step (S320), the spatial object separation unit 140 performs a check operation on vertices belonging to an object group that does not belong to a spatial group (S320-1).

Check whether the vertex has been checked (S320-2), and if it is an unchecked vertex, add it to the checklist (S320-3). Check the checklist (S320-4), and if the checklist is empty, create a new object group, and if it is not empty, extract the vertices in the checklist (S320-5).

Check whether the corresponding vertex is an object vertex (S320-6). Objects placed within a space may be in contact with one or more planes that make up the space. In this case, during the connection process, the object plane may extend to the spatial plane through the point where it meets the spatial plane, so S320-6 is performed to block this.

In the case of an object vertex, vertices at the same location are found using the first dictionary (S320-7). Then, adjacent triangles are also found using the second dictionary (S320-8). And connected vertices can be obtained using the searched vertices and triangles (S320-9). Obtained vertices can be added to the check risk to expand connectivity.

Since the object is a three-dimensional object with planes in front, back, left and right, and up and down, the planes in each direction can be connected to each other by connected vertices, creating one object group (S320-10). For a newly created object group, the minimum bounding box containing it can be calculated (S320-11) and function as one object model.

Figure 9 is an example diagram of a preview screen during mobile scanning and a modeling screen after mobile scanning is completed.

Referring to (a) of FIG. 9, as described above, while creating a point cloud during the scanning process by the mobile scanning unit 110, the parts that require additional point collection are displayed in a designated color (orange in the drawing) so that the user can By filming the relevant part for a longer period of time, it is possible to obtain sufficient points for 3D modeling.

Referring to (b) of FIG. 9, when scanning of the space and the objects placed therein is completed through mobile scanning of an arbitrary space, a viewer or editor screen is shown where the corresponding model can be confirmed.

In the modeling process, not only color images but also depth images are used, so the length value for each modeled model can be obtained. The width, height, and height of the bounding box set for each model are displayed on the screen, informing the user of the actual size of the space where the current mobile scan was performed.

Figure 10 is an example diagram showing the separation of a space model and an object model.

Referring to FIG. 10, a case in which a chair is placed in a mobile scanned real space is illustrated. A digital twin is implemented by the mobile scan digital twin implementation device 100 according to this embodiment, so that the spatial model of the real space and the object model of the objects placed therein can be separated and managed as in reality.

Therefore, as shown in (a) of FIG. 10, the space and the chair, which cannot be separated in the two-dimensional image, can be separated from each other in the three-dimensional modeling state through digital twin implementation. In other words, only the chair model can be selected separately from the space.

When the selected chair model is deleted, the chair model disappears as shown in (b) of FIG. 10. However, in this case, there is no captured image of the floor surface where the chair was, so it may be an empty surface area. In response to this, through predetermined data processing, it is supplemented to display a texture similar to the surrounding floor as shown in (c) of Figure 10, so that the empty surface of the space model due to deletion of the object model is displayed without awkwardness from the user's perspective. It may be possible.

The reality is that during the 3D modeling process for real space, it is not easy to perform mobile scanning with all objects placed in the space removed. In this situation, according to this embodiment, it is possible to create and utilize a model for the space itself, excluding objects, by 3D modeling space and objects together and then separating the space model and object model.

The method of implementing a mobile scan digital twin through spatial modeling described above can also be implemented in the form of a recording medium containing instructions executable by a computer, such as an application or program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.

The method of implementing a mobile scan digital twin through spatial modeling described above can be executed by an application that is installed by default on the terminal (this may include programs included in the platform or operating system, etc. that are installed by default in the terminal), and the user can use the application. It may also be executed by an application (i.e. program) installed directly on the master terminal through an application providing server such as a store server, application, or web server related to the service. In this sense, the method of implementing a mobile scan digital twin through spatial modeling described above is implemented as an application (i.e., program) installed by default on the terminal or directly installed by the user and can be recorded on a computer-readable recording medium such as a terminal. You can.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art can vary the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be modified and changed.

100: Mobile scanning digital twin implementation device
110: mobile scanning unit 120: modeling unit
130: mesh analysis unit 140: spatial object separation unit
150: storage unit 160: viewer/editor

Claims (13)

As a mobile device that implements a digital twin of real space,
A mobile scanning unit that captures real space and objects placed within it with a camera module to generate color images and depth images;
a modeling unit that processes the color image and the depth image to generate a point cloud and generates a three-dimensional model from the point cloud;
a mesh analysis unit that analyzes the mesh data of the three-dimensional model to classify planes belonging to the same object and classifies and assigns space and object properties;
a space object separation unit that divides the planes into a space model plane and an object model plane based on properties given to the planes, and separates the space model and the object model from each other; and
A mobile scan digital twin implementation device including a storage unit that stores the space model and the object model in memory or a database.
According to paragraph 1,
The space model is a mobile scan digital twin implementation device, characterized in that the space model is a result of modeling the space itself by excluding the object from the real space where the object is placed.
According to paragraph 1,
The mobile scanning unit displays the shooting screen in a preview format through the display unit during the mobile scanning process,
A mobile scan digital twin implementation device, characterized in that areas where points belonging to the point cloud have not been created or have not been completely created are covered with points of a specific color, thereby causing the points of the specific color to disappear from the preview screen.
According to paragraph 1,
The modeling unit is a mobile scan digital twin implementation device, wherein the modeling unit performs a data preparation step of generating triangular mesh data using points of the point cloud.
According to clause 4,
The mesh analysis unit includes a data separation step of grouping triangles of the mesh data into mesh groups, which are a set of triangles constituting the same plane, where triangles of the mesh data are connected to each other and the angle between the corresponding normal and the reference normal is small; and
A mobile scan digital twin implementation device characterized in that a group classification step is performed to classify the mesh groups into space groups and object groups depending on whether spatial conditions are met.
According to clause 5,
The spatial object separation unit performs a spatial merge and an object merge on the divided groups, and finally performs a spatial merge step and an object merge step to separate the separated groups into the space model and the object model. Implementing a mobile scan digital twin. Device.
According to paragraph 1,
A mobile scan digital twin implementation device further comprising an editor that deletes the object model from the 3D model according to a user command, and displays only the spatial model or displays another object model by placing it at a random position in the spatial model.
A computer program stored on a computer-readable medium for performing a mobile scan digital implementation method, the computer program causing a computer to perform the following steps, the steps comprising:
Generating a point cloud by processing the color image and depth image generated by the mobile scanning unit in the modeling unit;
A data preparation step of generating triangular mesh data using points of the point cloud in the modeling unit;
A data separation step of analyzing the mesh data in a mesh analysis unit and grouping it into mesh groups, which are sets of triangles constituting the same plane;
A group classification step in which the mesh analysis unit classifies the mesh groups into space groups and object groups according to whether spatial conditions are met; and
A computer program stored in a computer-readable medium including a space merge step and an object merge step of performing spatial merge and object merge on the group classified in the spatial object separation unit and finally separating the group into the spatial model and the object model.
According to clause 8,
In the data preparation step, the modeling unit indexes co-located vertices and adjacent triangles and manages them in advance as a first dictionary and a second dictionary.
According to clause 9,
In the data separation step, by using the first dictionary and the second dictionary to find the same location vertex and adjacent triangles, the triangles of the mesh data are connected to each other, and triangles with a small angle between the corresponding normal and the reference normal are created. A computer program stored on a computer-readable medium characterized by grouping into mesh groups, which are sets of triangles constituting the same plane.
According to clause 10,
In the group classification step,
checking whether the number of triangles included in each mesh group exceeds a predetermined number;
determining an object group if the number is less than a predetermined number;
If the number is more than a predetermined number and satisfies the floor conditions, wall conditions, and ceiling conditions of the space consisting of the floor, wall, and ceiling, the properties are set to floor, wall, and ceiling respectively and determined as a space group, and the floor condition, the wall condition, and A computer program stored in a computer-readable medium, comprising determining the object group when all of the ceiling conditions are not met.
According to clause 11,
In the spatial merge step, it is determined whether the spatial merge condition is satisfied for each object group, and if the spatial merge condition is satisfied, the point of the object group is added to the spatial group, and the object group is deleted. A computer program stored on a computer-readable medium.
According to clause 12,
A computer program stored in a computer-readable medium, characterized in that, in the object merging step, object recombination is performed through expansion of the object plane, but blocking expansion into the spatial plane to generate a new object group created as the object model.