KR20240021542A - Mobile digital twin implement apparatus using space modeling and virtual plane generating method - Google Patents

Abstract

A mobile digital twin implementation device and virtual plane creation method through spatial modeling are disclosed. A computer program stored in a computer-readable medium to perform a virtual plane creation method when implementing a mobile scan digital twin according to an embodiment of the present invention, the computer program causes the computer to perform the following steps, The steps include generating a 3D model by mobile scanning the real space and objects placed therein, and separating the space model and the object model through space merging and object merging; selecting the object model to be deleted in response to a user command; Deleting the selected object model; and generating a virtual plane to fill a deleted area occurring in a plane of the space model that is in contact with the object model.

Description

{Mobile digital twin implement apparatus using space modeling and virtual plane generating method}

The present invention relates to modeling technology, and more specifically, to a mobile digital twin implementation device and virtual plane generation method through spatial modeling.

3D scanning refers to a technology that uses an image sensor to create a 3D model representing the 3D shape of an object. There are two types of 3D scanning methods: a manual method in which a person scans various parts of an object while holding a scanner and adjusting its position, and an automatic method in which various parts of an object are scanned while the position of the scanner or object is automatically adjusted by a robot.

A 3D model can be created using scan data obtained through such 3D scanning. Previously, there was object-level modeling, but modeling through mobile scanning of an entire space such as an office or home had its limitations.

Korean Patent Publication No. 10-2020-0045321 (published May 4, 2020) - Spatial scanning device and its information processing method

The present invention is a mobile digital twin implementation device through spatial modeling that implements a digital twin on mobile through spatial scanning and modeling using a mobile terminal and provides users with a 3D digital recreation of real space and objects. and to provide a method for creating a virtual plane.

The present invention creates a three-dimensional model of a space captured using a camera installed in a general mobile terminal owned by a user, separates objects placed in the space, and then places virtual walls and/or floors in the area where the objects were deleted. The purpose is to provide a mobile digital twin implementation device and a virtual plane creation method through spatial modeling that can create a model of pure space by creating a ceiling.

Other objects of the present invention will become clearer through the preferred embodiments described below.

According to one aspect of the present invention, there is a computer program stored in a computer-readable medium for performing a virtual plane creation method when implementing a mobile scan digital twin, wherein the computer program causes the computer to perform the following steps, The steps include generating a 3D model by mobile scanning the real space and objects placed therein, and separating the space model and the object model through space merging and object merging; selecting the object model to be deleted in response to a user command; Deleting the selected object model; A computer program stored in a computer-readable medium is provided, including generating a virtual plane to fill a deleted area occurring in a plane of the spatial model that is in contact with the object model.

The virtual plane creation step includes finding a spatial mesh adjacent to the deletion area and dividing it into a wall and a floor; acquiring a projection point by projecting a peripheral point of the deleted area of the wall onto the floor; creating a first closed line using peripheral points of the deleted area of the wall and the projection point, and creating a first closed plane whose interior is filled with a plane mesh; creating a second closed line using peripheral points of the deletion area of the floor and the projection point, and creating a second closed plane filling the inside of the projection point with a planar mesh; and generating the virtual plane by combining the first closed plane and the second closed plane.

In the step of generating the first closed plane, an XY plane mesh may be created when the normal is in the Z-axis direction, and a YZ plane mesh may be created when the normal is in the X-axis direction.

When the object model has only one surface in contact with the space model, the virtual plane can be created by creating a closed line using peripheral points of the deletion area and filling the inside of the closed line with a plane mesh.

When the object model has two surfaces in contact with the space model, the deletion area is divided into a first deletion area on the wall and a second deletion area on the floor, and the peripheral points of the first deletion area are projected onto the floor. A first closed plane is created by filling the inside of the first closed line created using one projection point with a plane mesh, and the surrounding points of the second deletion area and the inside of the second closed line created using the projection point are created. A second closed plane can be created by filling it with a planar mesh, and the virtual plane can be created by combining the first closed plane and the second closed plane.

When the object model has three surfaces in contact with the space model, the deletion area is divided into a 1-1 deletion area and a 1-2 deletion area of the adjacent wall portion and a second deletion area of the floor portion, A 1-1 projection point and a 1-2 projection point that project peripheral points of the 1-1 deletion area and the 1-2 deletion area onto the floor, and the 1-1 deletion area and the 1-2 deletion area. 2 A first closed plane is created by filling the interior of the 1-1 closed line and the 1-2 closed line created using a second projection point by projecting a peripheral point of any one of the deleted areas to another neighboring wall with a plane mesh. And a 1-2 closed plane is created, and the interior of the second closed line generated using the peripheral points of the second deletion area, the 1-1 projection point, and the 1-2 projection point is filled with a planar mesh. A second closed plane is created, and the virtual plane can be created by combining the 1-1 closed plane, the 1-2 closed plane, and the second closed plane.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to an embodiment of the present invention, there is an effect of implementing a digital twin on a mobile device through spatial scanning and modeling using a mobile terminal and providing a three-dimensional digital recreation of real spaces and objects to users.

In addition, a three-dimensional model of the space captured using the camera installed in the user's general mobile terminal is created, objects placed in the space are separated, and virtual walls and/or floors/ceilings are created in the areas where the objects were deleted. There is also the effect of creating a model of pure space by creating .

1 is a configuration diagram of a mobile digital twin implementation device through spatial modeling according to an embodiment of the present invention;
2 is a flowchart of the mobile scanning process performed in the mobile scanning unit;
Figure 3 is a flowchart of a method for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention;
Figure 4 is a diagram showing the detailed process of the data preparation stage;
Figure 5 is a diagram showing the detailed process of the data separation step;
Figure 6 is a diagram showing the detailed process of the group classification step;
Figure 7 is a diagram showing the detailed process of the spatial merge step;
Figure 8 is a diagram showing the detailed process of the object merge step;
Figure 9 is an example of a preview screen during mobile scanning and a modeling screen after mobile scanning is completed;
Figure 10 is an example diagram showing the separation of a space model and an object model;
11 is a flowchart of a virtual plane generation method according to an embodiment of the present invention;
12 shows the selection process of an object model to be deleted,
13 shows the deletion process for a mesh group,
14 shows the deletion process for a group of objects;
15 shows the creation process of a virtual plane for the deletion area,
16 shows the virtual plane creation process at the intersection;
Figures 17 and 18 are examples of deletion of an object model in contact with one side of space and creation of a virtual plane;
Figures 19 and 20 are examples of deletion of an object model in contact with two sides of space and creation of a virtual plane;
Figures 21 and 22 are examples of deleting an object model in contact with three sides of space and creating a virtual plane.
.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the components of the embodiments described with reference to each drawing are not limited to the corresponding embodiments, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention, and may also be included in separate embodiments. Even if the description is omitted, it is natural that a plurality of embodiments may be re-implemented as a single integrated embodiment.

In addition, when describing with reference to the accompanying drawings, identical or related reference numbers will be assigned to identical or related elements regardless of the drawing symbols, and overlapping descriptions thereof will be omitted. In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Figure 1 is a configuration diagram of an apparatus for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention.

A method and device for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention implements a digital twin by scanning and modeling real spaces and objects using a mobile terminal, recreating them in three-dimensional digital form, and providing them to the user. It is characterized by A three-dimensional model is created by scanning a real space such as an office or home using the camera of a mobile device that the user generally carries for phone calls, internet, etc., and objects placed in the space are separated from the space. You can create a purely spatial model. When objects placed in a space are separated and deleted, virtual walls and floors/ceilings are created for some areas of the space that were in contact with the objects, creating a space model similar to when a space is mobile scanned without objects. enables you to obtain.

Referring to FIG. 1, the mobile digital twin implementation device 100 according to this embodiment may be implemented as a mobile terminal such as a smartphone or tablet PC. However, the mobile digital twin implementation device 100 is not limited to this. The mobile digital twin implementation device 100 may be implemented as a separate device for performing the mobile digital twin implementation method according to an embodiment of the present invention.

The mobile digital twin implementation device 100 may include a mobile scanning unit 110, a modeling unit 120, a mesh analysis unit 130, a spatial object separation unit 140, and a storage unit 150.

The mobile scan unit 110 takes images of the real space to be 3D modeled. The real space subject to filming may be an indoor space such as an office or home.

The mobile scan unit 110 is a camera module provided in a mobile terminal and may include a video camera for color image capture and a depth sensor such as LiDAR for depth image capture.

The image captured by the mobile scanning unit 110 may be displayed on the screen in a preview format through a display unit provided in the mobile terminal. Through the preview, users can check the current scan area and check the progress of the scan.

The modeling unit 120 processes data from the image captured by the mobile scanning unit 110 to create a point cloud, and creates a plurality of meshes from the point cloud to create a three-dimensional model of the space where objects are placed. Create.

The mesh analysis unit 130 analyzes the mesh data of the three-dimensional model generated by the modeling unit 120, classifies planes belonging to the same object, and separately assigns space and object properties.

The spatial object separator 140 may distinguish and separate a plane of a spatial model for a space and a plane of an object model for an object placed in the space based on the properties given to the planes.

The storage unit 150 can store the final modeled spatial model and object model in the terminal's memory, or transmit them to an external server connected to a network and store them in a separate database. The saved space model and object model can be loaded into the viewer/editor 160 or shared with other users according to user input.

In this embodiment, the space model refers to the modeling result of the space itself excluding the object from the real space where the object is placed, and the object model refers to the modeling result of the object itself.

The viewer 160a displays the space model and/or object model stored in the storage unit 150 on the screen of the display unit. Functions such as rotation, enlargement, reduction, and size display allow you to check the modeled model through mobile scanning from various viewpoints.

The editor 160b may perform an editing function on the model displayed on the screen in response to user input. Editing functions may include object deletion, box clipping, plane cropping, object placement, etc.

The user can use the editor 160bb to select and delete a specific object model from among the 3D models currently displayed through the display unit. In this case, when a specific object model is deleted, there are no points belonging to the previously created point cloud in some areas of the space plane (wall, floor, ceiling, etc.) in contact with the object model, so a mesh for plane construction is required. cannot be created. Therefore, in this embodiment, a filled-in three-dimensional modeling can be made even for areas where no points exist among the planes of the spatial model by creating a virtual plane in the corresponding area through predetermined data processing.

Additionally, by using the editor 160b to place a desired object model within the space model, the user can virtually place the object and check whether it fits the space without purchasing and placing the object in person.

The data processing process performed in the above-described modeling unit 120, mesh analysis unit 130, and spatial object separation unit 140 includes a data preparation step, a data separation step, a group classification step, a spatial merge step, and an object merge step. can be included and performed sequentially.

Hereinafter, the mobile digital twin implementation method performed in the mobile digital twin implementation device 100 through spatial modeling described above will be described in detail with reference to the related drawings.

FIG. 2 is a flowchart of the mobile scanning process performed in the mobile scanning unit, FIG. 3 is a flowchart of a method of implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention, and FIG. 4 shows a detailed process of the data preparation step. Figure 5 is a diagram showing the detailed process of the data separation step, Figure 6 is a diagram showing the detailed process of the group classification step, Figure 7 is a diagram showing the detailed process of the spatial merge step, and Figure 8 is an object This diagram shows the detailed process of the merge stage.

First, referring to FIG. 2, a mobile scan process performed by the mobile scan unit 110 is shown.

The mobile scan unit 110 may perform a mobile scan process using a camera module provided in the mobile terminal (step S200).

The camera module may include a video camera such as a CCD camera or CMOS camera for capturing a color image (RGB Image), and a depth sensor for capturing a depth image. Accordingly, a color image can be acquired using a video camera (step S205) and a depth image can be acquired using a depth sensor (step S210).

Alternatively, the camera module may be an RGB-D camera that can acquire both color images and depth images.

In addition, the camera module is equipped with an augmented reality kit (AR kit) for implementing augmented reality, so that information on the location of the camera currently being filmed can be obtained (step S215). Additionally, lens characteristic information regarding the lens installed in the camera can be additionally obtained.

Using the depth image captured by the depth sensor and the camera position information acquired by the augmented reality kit, the mobile scanning unit 110 generates a point cloud, which is a set of points about the space currently being photographed, and as needed. can update it (step S220).

The generated and updated point cloud may be projected into the camera coordinate system (step S225). The camera coordinate system is created based on camera position information acquired by the augmented reality kit, and thus the coordinates of each point of the relative point cloud can be obtained.

For the points of the point cloud projected into the camera coordinate system, the color of each point can be designated as a color related to the corresponding coordinate of the color image area and output (step S230).

The screen captured by the camera module may be displayed in a preview format through the display unit and provided to the user. On the preview screen, parts of the point cloud that have not been created or whose creation has not been completed are displayed in a pre-specified manner, giving users who are filming real space the information that needs to be collected to create/update the point cloud for 3D modeling. You can check the points intuitively.

Here, the predefined method displays the point parts for which the point cloud has been created and updated as a normally captured screen, and the part for which the point cloud has not yet been created is in a specific color (a rare color in normal shooting situations, for example, orange). It may be covered with points (see (a) of FIG. 9). By applying this display method to encourage user behavior so that points of a specific color disappear as much as possible from the preview screen, enough points for 3D modeling can be collected to complete the point cloud.

When the scan is completed by the mobile scanning unit 110 and a point cloud is generated, it can be considered that basic data for 3D modeling has been prepared. Afterwards, the mobile digital twin implementation method can be executed as shown in FIG. 3. Depending on the situation, the mobile scanning process may be included and executed together with the mobile digital twin implementation method.

Referring to Figure 3, the mobile digital twin implementation method includes a data preparation step (S300), a data separation step (S305), a group classification step (S310), a spatial merge (small object merge) step (S315), and an object merge (object recombination) step. ) may include step (S320).

Referring to FIG. 4, in the data preparation step (S330), the modeling unit 120 generates triangle-type mesh data using points of the point cloud generated by the mobile scanning unit 110.

Mesh data may be a data set consisting of sub-data of vertices, triangles, and normals. A vertex is the vertex of a triangle, and the normal is a unit vector perpendicular to the plane of the triangle.

Here, in a situation where there is multiple mesh data, some vertices exist at the same location, but there may be cases where they belong to different meshes and are logically separated. In this case, the vertex ID may be different, and during data processing such as subsequent merging or model deletion by the editor, a problem may occur in which one vertex is deleted but the other vertex is not deleted even though it is in the same location. .

Therefore, in the data preparation stage, even if the vertex IDs are different, the process of identifying and indexing vertices at the same position and managing the index in a separate first dictionary (same position vertex index dictionary) can be preceded. Through this, it is possible to immediately identify the vertex at the same location through the first dictionary without having to calculate it separately each time, thereby reducing the amount of calculation, allowing modeling, viewing, and editor tasks to be performed smoothly even on mobile devices.

Additionally, in the process of analyzing triangle data, information about triangles with adjacent vertices that are not at the same location but are located within tolerance is collected. In the plane expansion process described later, the planes are connected through the triangle expansion method. Information about triangles whose vertices are adjacent to one triangle is indexed, and the index is stored in a separate second dictionary (Vertice adjacent triangle index dictionary). It can be managed with Afterwards, adjacent triangles can be immediately identified through the second dictionary without having to be calculated separately each time, thereby reducing the amount of calculation, allowing modeling, viewing, and editor tasks to be performed smoothly even on mobile devices.

When data preparation is completed, in the data separation step (S305), the mesh analysis unit 130 finds a set of triangles that are connected to each other and have a small angle between the corresponding normal and the reference normal, and treats them as triangles constituting the same plane.

Referring to FIG. 5, it is checked whether the vertices have been reviewed (S305-1). If it is a vertex that has not been reviewed, add it to the checklist (S305-2). Check whether the checklist is empty (S305-3), and if not, extract the vertices of the checklist (S305-4). For the corresponding vertex, vertices at the same location are found through the first dictionary prepared in the data preparation step (S300) (S305-5), and triangles (adjacent triangles) having adjacent vertices associated with the corresponding vertex are found through the second dictionary. (S305-6). By using a dictionary, you can quickly find co-located vertices and adjacent triangles and reduce the amount of calculation for search.

Additionally, the benchmark normal of the vertex (or the mesh to which the vertex belongs) is found (S305-7). The reference normal may be the normal vector of a triangle belonging to the mesh to which the corresponding vertex belongs.

Compare the normal line of the adjacent triangle with the reference normal line to determine whether the angle between them is within a preset angle (S305-8). Angle comparison may be made depending on whether the dot product of the two normals exceeds a predetermined reference value (eg, 0.75).

If it is within a preset angle (i.e., if the inner product of two normals exceeds a predetermined standard value), the triangle to which the current vertex belongs and the adjacent triangle are considered to have normals in the same direction, and can be determined to be triangles belonging to the same plane. (S305-9).

For triangles belonging to the same plane, vertices belonging to the triangle are extracted (S305-10), added to the checklist again, and the above-described process is repeated.

Through the above-described process, it is possible to find a set of triangles that are connected to each other and face the same direction, that is, forming the same plane. By growing triangles in this way, the triangles corresponding to each plane that makes up space or objects can be separated and grouped separately.

Data grouped into coplanar triangles goes through a group classification step (S310).

Referring to FIG. 6, the mesh analysis unit 130 performs analysis on data (mesh group) grouped in the same plane. The purpose of this analysis is to classify into space groups and object groups depending on whether the spatial conditions are met.

First, check whether the number of triangles grouped and included in one plane exceeds a predetermined number (for example, 10) (S310-1).

If the number is less than a predetermined number, it is considered not to be a plane related to a space with a large plane and is determined to belong to an object group (S310-9).

If the number exceeds a predetermined number, it is determined whether each condition of a space having a large plane is met. The plane that makes up space includes the floor, walls, and ceiling.

First, determine whether the floor condition is met (S310-2), and if the floor condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to floor (S310-3). And it is determined to belong to the space group (S310-8).

Next, determine whether the wall condition is met (S310-4), and if the wall condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to wall (S310-5). And it is determined to belong to the space group (S310-8).

Next, determine whether the ceiling condition is met (S310-6), and if the ceiling condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to ceiling (S310-7). And it is determined to belong to the space group (S310-8).

The order of judgment regarding floor conditions, wall conditions, and ceiling conditions among space conditions is only an example, and the order may be changed.

If the floor condition, wall condition, and ceiling condition are all not met, the plane in question does not constitute a space, but can be viewed as a plane that constitutes one of the objects placed in the space. In this case, the points and mesh groups belonging to the plane is determined to belong to the object group (S310-9).

In this embodiment, the floor conditions are (1) whether the normal direction of the plane is upward, (2) whether the area is a certain size or more, and (3) whether it is located in a low position.

The wall conditions are (1) whether the normal direction of the plane is horizontal, (2) whether the area is over a certain size, and (3) whether the height is over a certain height (for example, 2 m).

Ceiling conditions are (1) whether the plane's normal direction is downward, and (2) whether it is located at a high location.

When the analysis by the mesh analysis unit 130 is completed and divided into a space group and an object group, the space object separation unit 140 performs a merge on the divided groups and finally completely separates the space model and the object model. .

Referring to FIG. 7, in the spatial merge step (S315), the spatial object separation unit 140 performs a check operation on all object groups targeting spatial groups (S315-1). Space groups can be broadly divided into floors, walls, and ceilings, and walls can be reclassified into multiple walls according to the normal direction.

During the check process, each object group (S315-2) checks whether it satisfies the spatial merge conditions (S315-3), and if it satisfies the spatial merge conditions, the points belonging to the object group are added to the spatial group, and the corresponding object Delete the group (S315-4).

The space merge conditions are (1) whether it is included in the space group currently being checked, whether the normal direction is parallel, and whether the number of points is less than a certain number (for example, 100).

Through the spatial merge step (S315), small object meshes within the spatial mesh are merged, making it possible to set the mesh data constituting the space as a spatial group among the remaining data that was not classified during the group classification process.

Mesh data set as a space group can be assigned floor, wall, and ceiling properties according to the above-mentioned classification conditions, and can be integrated as a space model excluding objects placed inside the mobile scanned space.

Next, the object merge step (S320) is performed through object recombination.

Referring to FIG. 8, in the object merging step (S320), the spatial object separation unit 140 performs a check operation on vertices belonging to an object group that does not belong to a spatial group (S320-1).

Check whether the vertex has been checked (S320-2), and if it is an unchecked vertex, add it to the checklist (S320-3). Check the checklist (S320-4), and if the checklist is empty, create a new object group, and if it is not empty, extract the vertices in the checklist (S320-5).

Check whether the corresponding vertex is an object vertex (S320-6). Objects placed within a space may be in contact with one or more planes that make up the space. In this case, during the connection process, the object plane may extend to the spatial plane through the point where it meets the spatial plane, so S320-6 is performed to block this.

In the case of an object vertex, vertices at the same location are found using the first dictionary (S320-7). Then, adjacent triangles are also found using the second dictionary (S320-8). And connected vertices can be obtained using the searched vertices and triangles (S320-9). Obtained vertices can be added to the check risk to expand connectivity.

Since the object is a three-dimensional object with planes in front, back, left and right, and up and down, the planes in each direction can be connected to each other by connected vertices, creating one object group (S320-10). For a newly created object group, the minimum bounding box containing it can be calculated (S320-11) and function as one object model.

Figure 9 is an example diagram of a preview screen during mobile scanning and a modeling screen after mobile scanning is completed.

Referring to (a) of FIG. 9, as described above, while creating a point cloud during the scanning process by the mobile scanning unit 110, the parts that require additional point collection are displayed in a designated color (orange in the drawing) so that the user can By filming the relevant part for a longer period of time, it is possible to obtain sufficient points for 3D modeling.

Referring to (b) of FIG. 9, when scanning of the space and the objects placed therein is completed through mobile scanning of an arbitrary space, a viewer or editor screen is shown where the corresponding model can be confirmed.

In the modeling process, not only color images but also depth images are used, so the length value for each modeled model can be obtained. The width, height, and height of the bounding box set for each model are displayed on the screen, informing the user of the actual size of the space where the current mobile scan was performed.

Figure 10 is an example diagram showing the separation of a space model and an object model.

Referring to FIG. 10, a case in which a chair is placed in a mobile scanned real space is illustrated. A digital twin is implemented by the mobile scan digital twin implementation device 100 according to this embodiment, so that the spatial model of the real space and the object model of the objects placed therein can be separated and managed as in reality.

Therefore, as shown in (a) of FIG. 10, the space and the chair, which cannot be separated in the two-dimensional image, can be separated from each other in the three-dimensional modeling state through digital twin implementation. In other words, only the chair model can be selected separately from the space.

When the selected chair model is deleted, the chair model disappears as shown in (b) of FIG. 10. However, in this case, there is no captured image of the floor surface where the chair was, so it may be an empty surface area. In response to this, through predetermined data processing, it is supplemented to display a texture similar to the surrounding floor as shown in (c) of Figure 10, so that the empty surface of the space model due to deletion of the object model is displayed without awkwardness from the user's perspective. It may be possible.

Hereinafter, a method of processing the area of the spatial model remaining when the object model is deleted, as shown in FIG. 10, will be described with reference to the related drawings.

Figure 11 is a flowchart of a virtual plane creation method according to an embodiment of the present invention, Figure 12 is a selection process of an object model to be deleted, Figure 13 is a deletion process for a mesh group, and Figure 14 is an object group 15 is a virtual plane creation process for the deletion area, FIG. 16 is a virtual plane creation process at an intersection, and FIGS. 17 and 18 are a deletion and virtual representation of an object model in contact with one side of the space. Figures 19 and 20 are examples of the creation of a plane, and Figures 19 and 20 are examples of the deletion of an object model in contact with two sides of space and the creation of a virtual plane, and Figures 21 and 22 are examples of the deletion of an object model in contact with three sides of a space. and an example of creating a virtual plane.

Referring to FIG. 11, in the method of creating a virtual plane according to this embodiment, when an object model is selected and deleted according to a user command in a three-dimensional model including a space model and an object model, after deletion of the object model, the object model and the space It is characterized in that the plane of the spatial model has continuity by creating a virtual plane mesh for the area where the models were in contact (deleted area). This virtual plane creation method can be performed in the editor 160b.

First, the editor 160b selects an object to be deleted, that is, an object model, in response to a user command input through a user input unit (not shown) (step S400). It is assumed that the user input unit is a touch screen, but various types of input devices such as a keyboard, keypad, and mouse may be applied.

Referring to FIG. 12, it is determined whether there is an object group including clicked points (S402) selected by a user command among the object group (S410) (S403). In other words, it checks whether there is a selected point within the object group bounds.

If there is an object group including the selected point within the bound, the object group with the smallest bound is found (S404). This is because the 3D model is displayed on a 2D screen through the display unit, so although they are different object groups, they can be located at the same point from the user's perspective.

Therefore, in this embodiment, when two or more object groups are selected, the object group with the smaller bound is considered to be selected. This is according to one embodiment, and the closest object group based on the user's viewpoint may be considered to be selected.

The bounds of the object group that have the smallest bound or are closest can be displayed as selected bounds (S405).

Referring to FIG. 13, with respect to the bound selected for deletion, it is checked whether there is a mesh group adjacent to the selected bound among the random mesh groups (S406a) (S407a). If there is an adjacent mesh group, the mesh group can also be added to the deletion list (S408a). Here, the mesh group is island data that was determined not to belong to the walls, floor, and ceiling of the space model in the space merge step (S315).

Referring to FIG. 14, with respect to the bound selected for deletion, it is checked whether there is an object group adjacent to the selected bound among any object groups (S406b) (S407b). If there is an adjacent object group, the corresponding object group can also be added to the deletion list (S408b). Here, the object group is island data that was determined not to belong to the plane of the object model in the object merge step (S320).

Through the above-described process, it is possible to select an object to be deleted corresponding to a user command, that is, a mesh group/object group adjacent to the object model.

Referring again to FIG. 11, next, deletion processing is performed on the selected object (step S410). The deletion process separates the points belonging to the selected bounds and removes them from the point cloud that constitutes the 3D model.

Through this deletion process, points related to the object selected from the 3D model may be deleted.

In this case, the object has been deleted, but the object (especially the object model) may have a portion that touches one or more of the planes (wall, floor, ceiling) of the spatial model. This part cannot be captured in the mobile scanning process described above and therefore cannot be created as a point cloud. Therefore, during deletion processing, a deletion area is created on some planes of the spatial model.

Accordingly, the editor 160b creates a virtual plane to fill the deleted area (step S420), thereby preventing the three-dimensional modeling from becoming incomplete due to the corresponding area of the spatial model displayed after the object is deleted being empty. .

Referring to Figure 15, the virtual plane creation process for the deletion area is shown in detail.

If a deletion area occurs on some plane of the spatial model (S421), a spatial mesh adjacent to the area can be found (S422). Adjacent space meshes can be divided into walls (S423) and floors/ceilings (S424).

In the case of the deleted area of the wall, the projection point can be obtained by projecting the first surrounding point onto the floor/ceiling (S425). Using the first peripheral points and projection points of the deleted area of the wall, it is possible to create a first closed plane in the wall portion (S426).

For floors/ceilings, additional points are available (S427). Additional points may be projection points. Or it could be one of the vertices needed to create a planar mesh.

For the deleted area of the floor/ceiling, a second closed plane can be created in the floor/ceiling area using the second peripheral points and additional points (S428).

By combining the first closed plane of the wall portion and the second closed plane of the floor/ceiling portion, a virtual plane can be created to fill the empty portion of the deleted area that was in contact with the deleted object model (S429).

The detailed process of creating the first closed plane and/or the second closed plane is shown in Figure 16. Here, the spatial model is modeling data for a three-dimensional space consisting of

The projection point existing in the deletion area is located at the corner where the wall and the floor/ceiling meet, and can be divided into intersection groups (S430).

Check whether the intersection group is located on the wall (S431), and if it is located on the wall, check whether the normal line is in the Z-axis direction (S432).

If the normal is in the Z-axis direction, the wall can be seen as forming an XY plane. By deleting the vertex (projection point) and creating an (S433).

If the normal is not in the Z-axis direction, it can be viewed as being in the X-axis direction, and in this case, the wall in question can be viewed as forming the YZ plane. Therefore, a virtual plane can be created to fill the empty area by deleting the vertex and creating a YZ plane mesh within the deleted area (S434).

If the intersection group is not a wall part, check whether it is a floor part (S435). If it is located on the floor, you can create a virtual plane to fill the empty area by deleting the vertices and creating a plane mesh corresponding to the floor plane, that is, an XY plane mesh (S436).

If the intersection group is not a wall or floor, check whether it is a ceiling (S437). If it is located in the ceiling area, you can create a virtual plane to fill the empty area by deleting the vertices and creating a plane mesh corresponding to the ceiling plane, that is, an XY plane mesh (S438).

Referring to Figures 17 and 18, a case is shown where the object model to be deleted touches one surface (floor surface) of the space model.

When the object model 550 is selected and deleted, an empty deletion area 560 is created on the floor plane of the space model (see (b) of FIG. 17).

In this case, the object model 550 has only one plane in contact with the space model 500, and the plane to which the deletion area belongs is the same as the plane in contact with it.

Accordingly, a closed line 565 corresponding to the deletion area 560 is created using the surrounding points 561 of the deletion area 560 (see (a) of FIG. 18). Additionally, a virtual plane 570 is created by creating a plane mesh inside the closed line 565 (see (b) of FIG. 18), so that there is no empty area in the space model 500.

Referring to FIGS. 19 and 20 , a case is shown where an object model to be deleted touches two surfaces of a space model (the floor surface and the wall surface in the drawings).

When the object model 550 is selected and deleted, an empty first deletion area 560a and a second deletion area 560b are created on the wall and floor surfaces of the space model (see (b) of FIG. 19).

The projection point 563a can be obtained by projecting the peripheral point 561a of the first deletion area 560a onto the floor (see (a) of FIG. 20). The projection point 563a can be viewed as an intersection located at the corner where the wall and the floor meet. Using the projection point 563a, the first closed line 565a for the first deletion area 560a, and the first closed line 565a are drawn. 2 A second closing line 565b may be created for the deletion area 560b.

The first closed plane 570a can be created by creating a planar mesh within the first closed line 565a. In this case, since the first closed line 565a is located in the wall portion and the normal vector is in the X-axis direction, a YZ plane mesh can be created according to S434 of FIG. 16.

Additionally, the second closed plane 570b can be created by creating a planar mesh within the second closed line 565b. In this case, the second closed line 570b is located at the bottom, so an XZ plane mesh can be created according to S436 of FIG. 16.

And by combining the first closed plane 570a and the second closed plane 570b to create a virtual plane 570 (see (d) of FIG. 20), it is possible to ensure that there are no empty areas in the space model 500. there is.

Referring to Figures 21 and 22, a case where an object model to be deleted is in contact with three surfaces of a space model (the floor surface and two walls in the drawings) is shown.

When the object model 550 is selected and deleted, the empty 1st-1st deletion area 560a-1, 1-2nd deletion area 560a-2, and second deletion area on the wall and floor of the space model ( 560b) is created (see (b) of FIG. 20).

The peripheral point 561a-1 of the 1-1 deletion area 560a-1 and the peripheral point 561a-2 of the 1-2 deletion area 560a-2 are projected onto the floor to create the 1-1 projection point. (563a-1) and the 1-2 projection point (563a-2) can be obtained (see (a) of FIG. 22). The 1-1 projection point 563a-1 and the 1-2 projection point 563a-2 can be viewed as intersection points located at the corners where the wall and the floor meet.

Additionally, if it is in contact with two walls, it is necessary to estimate the edge between the walls. Accordingly, by projecting the peripheral point 561c from the deletion area belonging to one of the two walls (e.g., the 1-2 deletion area 560a-2) to the other one (the 1-1 deletion area 560a-1), The second projection point 563c can be obtained.

1-1 deletion area 560a-1 using the 1-1 projection point 563a-1, 1-2 projection point 563a-2, and 2nd projection point 563c. 1 closure line 565a-1, 1-2 closure line 565a-2 for the 1-2 deletion area 560a-2, and second closure line 565b for the 2nd deletion area 560b. can be created.

The 1-1 closed plane 570a-1 can be created by creating a plane mesh within the 1-1 closed line 565a-1. In this case, since the 1-1 closed line 565a-1 is located in the wall portion and the normal vector is in the Y-axis direction, a YZ plane mesh can be created according to S434 of FIG. 16.

The 1-2 closed plane 570a-2 can be created by creating a plane mesh within the 1-2 closed line 565a-2. In this case, since the 1-2 closed line 565a-2 is located in the wall portion and the normal vector is in the Z-axis direction, an XY plane mesh can be created according to S433 of FIG. 16.

Additionally, the second closed plane 570b can be created by creating a planar mesh within the second closed line 565b. In this case, the second closed line 565b is located at the bottom, so an XZ plane mesh can be created according to S436 of FIG. 16.

And by creating a virtual plane 570 by combining the 1-1st closed plane 570a-1, the 1-2nd closed plane 570a-2, and the second closed plane 570b ((d) of FIG. 22) Reference), it is possible to ensure that there are no empty areas in the space model 500.

The reality is that during the 3D modeling process for real space, it is not easy to perform mobile scanning with all objects placed in the space removed. In this situation, according to this embodiment, it is possible to create and utilize a model for the space itself, excluding objects, by 3D modeling space and objects together and then separating the space model and object model.

In the process of excluding objects, an empty area is created on the plane of the spatial model with no mesh or points. By creating a virtual plane by creating a mesh in the empty area through estimation of some areas of the spatial model that are not visible because the object model is obscured, it is possible to prevent the spatial model from becoming an incomplete three-dimensional model.

The above-described mobile scan digital twin implementation method and virtual plane creation method through spatial modeling can also be implemented in the form of a recording medium containing instructions executable by a computer, such as an application or program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.

The method of implementing a mobile scan digital twin through spatial modeling and the method of creating a virtual plane described above can be executed by an application installed by default on the terminal (this may include programs included in the platform or operating system, etc., installed by default on the terminal). It may also be executed by an application (i.e., program) installed by the user directly on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the method of implementing a mobile scan digital twin through spatial modeling and the method of creating a virtual plane are implemented as an application (i.e. program) installed by default on the terminal or directly installed by the user and can be read by a computer such as a terminal. It can be recorded on a recording medium.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art can vary the present invention without departing from the spirit and scope of the present invention as set forth in the claims below. You will understand that it can be modified and changed.

100: Mobile scanning digital twin implementation device
110: mobile scanning unit 120: modeling unit
130: mesh analysis unit 140: spatial object separation unit
150: storage unit 160: viewer/editor
170: Styler
500: Space model 550: Object model
560, 560a, 560b, 560a-1, 560a-2: Deleted area
561, 561a, 561a-1, 561a-2, 561c: Nearby points
563a, 563a-1, 563a-2, 563c: Projection points
570: virtual plane

Claims (6)

A computer program stored in a computer-readable medium for performing a virtual plane creation method when implementing a mobile scan digital twin, wherein the computer program causes a computer to perform the following steps, the steps comprising:
Creating a 3D model by mobile scanning real space and objects placed therein, and separating the space model and object model through space merging and object merging;
selecting the object model to be deleted in response to a user command;
Deleting the selected object model; and
A computer program stored in a computer-readable medium, comprising the step of generating a virtual plane to fill a deleted area occurring in a plane of the spatial model that contacts the object model.
According to paragraph 1,
The virtual plane creation step is,
Finding a spatial mesh adjacent to the deletion area and dividing it into a wall and a floor;
acquiring a projection point by projecting a peripheral point of the deleted area of the wall onto the floor;
creating a first closed line using peripheral points of the deleted area of the wall and the projection point, and creating a first closed plane whose interior is filled with a plane mesh;
creating a second closed line using peripheral points of the deletion area of the floor and the projection point, and creating a second closed plane filling the inside of the projection point with a planar mesh; and
Combining the first closed plane and the second closed plane to generate the virtual plane.
According to paragraph 2,
The step of generating the first closed plane includes generating an XY plane mesh when the normal is in the Z-axis direction, and generating a YZ plane mesh when the normal is in the X-axis direction. A stored computer program.
According to paragraph 1,
When the object model has one surface in contact with the space model, a closed line is created using peripheral points of the deletion area, and the virtual plane is created by filling the inside of the closed line with a plane mesh. A computer program stored on a computer-readable medium.
According to paragraph 1,
If the object model has two surfaces in contact with the space model,
The deletion area is divided into a first deletion area of the wall portion and a second deletion area of the floor portion,
A first closed plane is created by filling the interior of the first closed line created using projection points obtained by projecting peripheral points of the first deletion area onto the floor with a plane mesh,
A second closed plane is created by filling the interior of the second closed line created using the projection point and the peripheral points of the second deletion area with a planar mesh,
A computer program stored in a computer-readable medium, wherein the virtual plane is generated by combining the first closed plane and the second closed plane.
According to paragraph 1,
If the object model has three surfaces in contact with the space model,
The deletion area is divided into a 1-1 deletion area and a 1-2 deletion area of the adjacent wall portion and a second deletion area of the floor portion,
A 1-1 projection point and a 1-2 projection point that project peripheral points of the 1-1 deletion area and the 1-2 deletion area onto the floor, and the 1-1 deletion area and the 1-2 deletion area. 2 A first closed plane is created by filling the interior of the 1-1 closed line and the 1-2 closed line created using a second projection point by projecting a peripheral point of any one of the deletion areas to another neighboring wall with a plane mesh. and a first-second closed plane is created,
A second closed plane is created by filling the interior of a second closed line created using the peripheral points of the second deletion area, the 1-1 projection point, and the 1-2 projection point with a planar mesh,
A computer program stored in a computer-readable medium, wherein the virtual plane is generated by combining the 1-1 closed plane, the 1-2 closed plane, and the second closed plane.