KR20240022097A - Mobile digital twin implement apparatus using space modeling and modeling object editing method - Google Patents

Abstract

A mobile digital twin implementation device and modeling object editing method through spatial modeling are disclosed. A computer program stored in a computer-readable medium to perform a modeling object editing method when implementing a mobile scan digital twin according to an embodiment of the present invention, the computer program causes the computer to perform the following steps, The steps include loading a modeling object on the screen in an editor; Receiving an edit command for the modeling object; And a step of deleting some points and mesh of the modeling object in response to the editing command, wherein the modeling object may be a three-dimensional model implemented as a digital twin by mobile scanning a real space where real objects are placed. there is.

Description

{Mobile digital twin implement apparatus using space modeling and modeling object editing method}

The present invention relates to modeling technology, and more specifically, to a mobile digital twin implementation device and modeling object editing method through spatial modeling.

3D scanning refers to a technology that uses an image sensor to create a 3D model representing the 3D shape of an object. There are two types of 3D scanning methods: a manual method in which a person scans various parts of an object while holding a scanner and adjusting its position, and an automatic method in which various parts of an object are scanned while the position of the scanner or object is automatically adjusted by a robot.

A 3D model can be created using scan data obtained through such 3D scanning. Previously, there was object-level modeling, but modeling through mobile scanning of an entire space such as an office or home had its limitations.

Korean Patent Publication No. 10-2020-0045321 (published May 4, 2020) - Spatial scanning device and its information processing method

The present invention is a mobile digital twin implementation device through spatial modeling that implements a digital twin on mobile through spatial scanning and modeling using a mobile terminal and provides users with a 3D digital recreation of real space and objects. and to provide a modeling object editing method.

The present invention creates a 3D model of a space captured using a camera installed in a general mobile terminal owned by the user, and edits the generated 3D model, such as clipping and cropping, with easy manipulation to produce the desired portion. The purpose is to provide a mobile digital twin implementation device and modeling object editing method through spatial modeling that allows for more smooth verification of.

Other objects of the present invention will become clearer through the preferred embodiments described below.

According to one aspect of the present invention, there is a computer program stored in a computer-readable medium for performing a modeling object editing method when implementing a mobile scan digital twin, wherein the computer program causes the computer to perform the following steps, The steps include loading a modeling object on the screen in an editor; Receiving an edit command for the modeling object; And a step of deleting some points and mesh of the modeling object in response to the editing command, wherein the modeling object is a three-dimensional model implemented as a digital twin by mobile scanning a real space where real objects are placed. A computer program stored on a computer-readable medium having the characteristics is provided.

The editing command is a plane crop command, and the deletion processing step includes: receiving a start point and an end point on the screen according to user selection when the plane crop function is activated by the edit command; Automatically generating a cutting surface using the starting point and the ending point; displaying a straight line where the cut surface meets the screen on the screen; receiving a selection input for a portion of the screen divided by the straight line; And it may include the step of deleting a portion of the modeling object divided by the cutting surface corresponding to the selection.

The step of automatically generating the cutting surface includes finding one plane including all three points of the user's viewpoint corresponding to the starting point, the ending point, and the camera position for the modeling object displayed on the screen, and defining the corresponding plane. It can be set as a cutting surface.

The editing command is a box clipping command, and the deletion processing step includes: displaying a clipping box corresponding to the modeling object on the screen when the box clipping function is activated by the editing command; adjusting one or more of the shape and rotation of the clipping box; It may include the step of using the planes constituting the clipping box as cutting surfaces and deleting an outer portion of the modeling object divided by the planes.

In the adjustment step, when one of the planes of the clipping box is selected, the selected plane is moved perpendicular to the plane to extend or shorten the clipping box in the moving direction.

Other aspects, features and advantages in addition to those described above will become apparent from the following drawings, claims and detailed description of the invention.

According to an embodiment of the present invention, there is an effect of implementing a digital twin on a mobile device through spatial scanning and modeling using a mobile terminal and providing a three-dimensional digital recreation of real spaces and objects to users.

In addition, a 3D model of the space captured using the camera installed in the user's general mobile terminal is created, and editing such as clipping and cropping is performed on the created 3D model with easy manipulation to fit the desired area. It also has the effect of allowing you to check information more smoothly.

1 is a configuration diagram of a mobile digital twin implementation device through spatial modeling according to an embodiment of the present invention;
2 is a flowchart of the mobile scanning process performed in the mobile scanning unit;
Figure 3 is a flowchart of a method for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention;
Figure 4 is a diagram showing the detailed process of the data preparation stage;
Figure 5 is a diagram showing the detailed process of the data separation step;
Figure 6 is a diagram showing the detailed process of the group classification step;
Figure 7 is a diagram showing the detailed process of the spatial merge step;
Figure 8 is a diagram showing the detailed process of the object merge step;
Figure 9 is an example of a preview screen during mobile scanning and a modeling screen after mobile scanning is completed;
Figure 10 is an example diagram showing the separation of a space model and an object model;
Figure 11 is a diagram showing the box clipping function among the editing functions performed in the editor of the mobile scan digital twin implementation device according to an embodiment of the present invention;
Figure 12 is a flowchart of the plane cropping process among the editing functions performed in the editor of the mobile scan digital twin implementation device according to an embodiment of the present invention;
Figure 13 is a diagram showing a method of setting a cutting surface for plane cropping;
Figure 14 is an example of a plane cropping process.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all changes, equivalents, and substitutes included in the spirit and technical scope of the present invention.

When a component is said to be "connected" or "connected" to another component, it is understood that it may be directly connected to or connected to the other component, but that other components may exist in between. It should be. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Terms such as first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another.

The terms used herein are only used to describe specific embodiments and are not intended to limit the invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as “comprise” or “have” are intended to indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to indicate the presence of one or more other features. It should be understood that this does not exclude in advance the possibility of the existence or addition of elements, numbers, steps, operations, components, parts, or combinations thereof.

In addition, the components of the embodiments described with reference to each drawing are not limited to the corresponding embodiments, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention, and may also be included in separate embodiments. Even if the description is omitted, it is natural that a plurality of embodiments may be re-implemented as a single integrated embodiment.

In addition, when describing with reference to the accompanying drawings, identical or related reference numbers will be assigned to identical or related elements regardless of the drawing symbols, and overlapping descriptions thereof will be omitted. In describing the present invention, if it is determined that a detailed description of related known technologies may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

Figure 1 is a configuration diagram of an apparatus for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention.

A method and device for implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention implements a digital twin by scanning and modeling real spaces and objects using a mobile terminal, recreating them in three-dimensional digital form, and providing them to the user. It is characterized by A three-dimensional model is created by scanning a real space such as an office or home using the camera of a mobile device that the user generally carries for phone calls, internet, etc., and objects placed in the space are separated from the space. You can create a purely spatial model.

Referring to FIG. 1, the mobile digital twin implementation device 100 according to this embodiment may be implemented as a mobile terminal such as a smartphone or tablet PC. However, the mobile digital twin implementation device 100 is not limited to this. The mobile digital twin implementation device 100 may be implemented as a separate device for performing the mobile digital twin implementation method according to an embodiment of the present invention.

The mobile digital twin implementation device 100 may include a mobile scanning unit 110, a modeling unit 120, a mesh analysis unit 130, a spatial object separation unit 140, and a storage unit 150.

The mobile scan unit 110 takes images of the real space to be 3D modeled. The real space subject to filming may be an indoor space such as an office or home.

The mobile scan unit 110 is a camera module provided in a mobile terminal and may include a video camera for color image capture and a depth sensor such as LiDAR for depth image capture.

The image captured by the mobile scanning unit 110 may be displayed on the screen in a preview format through a display unit provided in the mobile terminal. Through the preview, users can check the current scan area and check the progress of the scan.

The modeling unit 120 processes data from the image captured by the mobile scanning unit 110 to create a point cloud, and creates a plurality of meshes from the point cloud to create a three-dimensional model of the space where objects are placed. Create.

The mesh analysis unit 130 analyzes the mesh data of the three-dimensional model generated by the modeling unit 120, classifies planes belonging to the same object, and separately assigns space and object properties.

The spatial object separator 140 may distinguish and separate a plane of a spatial model for a space and a plane of an object model for an object placed in the space based on the properties given to the planes.

The storage unit 150 can store the final modeled spatial model and object model in the terminal's memory, or transmit them to an external server connected to a network and store them in a separate database. The saved space model and object model can be loaded into the viewer/editor 160 or shared with other users according to user input.

In this embodiment, the space model refers to the modeling result of the space itself excluding the object from the real space where the object is placed, and the object model refers to the modeling result of the object itself.

The viewer 160a displays the space model and/or object model stored in the storage unit 150 on the screen of the display unit. Functions such as rotation, enlargement, reduction, and size display allow you to check the modeled model through mobile scanning from various viewpoints.

The editor 160b may perform an editing function on the model displayed on the screen in response to user input. Editing functions may include object deletion, box clipping, plane crop, object placement, etc.

By using the editor 160b to place a desired object model within the space model, the user can virtually place the object and check whether it fits the space without purchasing and placing the object in person.

The data processing process performed in the above-described modeling unit 120, mesh analysis unit 130, and spatial object separation unit 140 includes a data preparation step, a data separation step, a group classification step, a spatial merge step, and an object merge step. can be included and performed sequentially.

Hereinafter, the mobile digital twin implementation method performed in the mobile digital twin implementation device 100 through spatial modeling described above will be described in detail with reference to the related drawings.

FIG. 2 is a flowchart of the mobile scanning process performed in the mobile scanning unit, FIG. 3 is a flowchart of a method of implementing a mobile digital twin through spatial modeling according to an embodiment of the present invention, and FIG. 4 shows a detailed process of the data preparation step. Figure 5 is a diagram showing the detailed process of the data separation step, Figure 6 is a diagram showing the detailed process of the group classification step, Figure 7 is a diagram showing the detailed process of the spatial merge step, and Figure 8 is an object This diagram shows the detailed process of the merge stage.

First, referring to FIG. 2, a mobile scan process performed by the mobile scan unit 110 is shown.

The mobile scan unit 110 may perform a mobile scan process using a camera module provided in the mobile terminal (step S200).

The camera module may include a video camera such as a CCD camera or CMOS camera for capturing a color image (RGB Image), and a depth sensor for capturing a depth image. Accordingly, a color image can be acquired using a video camera (step S205) and a depth image can be acquired using a depth sensor (step S210).

Alternatively, the camera module may be an RGB-D camera that can acquire both color images and depth images.

In addition, the camera module is equipped with an augmented reality kit (AR kit) for implementing augmented reality, so that information on the location of the camera currently being filmed can be obtained (step S215). Additionally, lens characteristic information regarding the lens installed in the camera can be additionally obtained.

Using the depth image captured by the depth sensor and the camera position information acquired by the augmented reality kit, the mobile scanning unit 110 generates a point cloud, which is a set of points about the space currently being photographed, and as needed. can update it (step S220).

The generated and updated point cloud may be projected into the camera coordinate system (step S225). The camera coordinate system is created based on camera position information acquired by the augmented reality kit, and thus the coordinates of each point of the relative point cloud can be obtained.

For the points of the point cloud projected into the camera coordinate system, the color of each point can be designated as a color related to the corresponding coordinate of the color image area and output (step S230).

The screen captured by the camera module may be displayed in a preview format through the display unit and provided to the user. On the preview screen, parts of the point cloud that have not been created or whose creation has not been completed are displayed in a pre-specified manner, giving users who are filming real space the information that needs to be collected to create/update the point cloud for 3D modeling. You can check the points intuitively.

Here, the predefined method displays the point parts for which the point cloud has been created and updated as a normally captured screen, and the part for which the point cloud has not yet been created is in a specific color (a rare color in normal shooting situations, for example, orange). It may be covered with points (see (a) of FIG. 9). By applying this display method to encourage user behavior so that points of a specific color disappear as much as possible from the preview screen, enough points for 3D modeling can be collected to complete the point cloud.

When the scan is completed by the mobile scanning unit 110 and a point cloud is generated, it can be considered that basic data for 3D modeling has been prepared. Afterwards, the mobile digital twin implementation method can be executed as shown in FIG. 3. Depending on the situation, the mobile scanning process may be included and executed together with the mobile digital twin implementation method.

Referring to Figure 3, the mobile digital twin implementation method includes a data preparation step (S300), a data separation step (S305), a group classification step (S310), a spatial merge (small object merge) step (S315), and an object merge (object recombination) step. ) may include step (S320).

Referring to FIG. 4, in the data preparation step (S330), the modeling unit 120 generates triangle-type mesh data using points of the point cloud generated by the mobile scanning unit 110.

Mesh data may be a data set consisting of sub-data of vertices, triangles, and normals. A vertex is the vertex of a triangle, and the normal is a unit vector perpendicular to the plane of the triangle.

Here, in a situation where there is multiple mesh data, some vertices may exist in the same location but belong to different meshes and are logically separated. In this case, the vertex ID may be different, and during data processing such as subsequent merging or model deletion by the editor, a problem may occur in which one vertex is deleted but the other vertex is not deleted even though it is in the same location. .

Therefore, in the data preparation stage, even if the vertex IDs are different, the process of identifying and indexing vertices at the same position and managing the index in a separate first dictionary (same position vertex index dictionary) can be preceded. Through this, it is possible to immediately identify the vertex at the same location through the first dictionary without having to calculate it separately each time, thereby reducing the amount of calculation, allowing modeling, viewing, and editor tasks to be performed smoothly even on mobile devices.

Additionally, in the process of analyzing triangle data, information about triangles with adjacent vertices that are not at the same location but are located within tolerance is collected. In the plane expansion process described later, the planes are connected through the triangle expansion method. Information about triangles whose vertices are adjacent to one triangle is indexed, and the index is stored in a separate second dictionary (Vertice adjacent triangle index dictionary). It can be managed with Afterwards, adjacent triangles can be immediately identified through the second dictionary without having to be calculated separately each time, thereby reducing the amount of calculation, allowing modeling, viewing, and editor tasks to be performed smoothly even on mobile devices.

When data preparation is completed, in the data separation step (S305), the mesh analysis unit 130 finds a set of triangles that are connected to each other and have a small angle between the corresponding normal and the reference normal, and treats them as triangles constituting the same plane.

Referring to FIG. 5, it is checked whether the vertices have been reviewed (S305-1). If it is a vertex that has not been reviewed, add it to the checklist (S305-2). Check whether the checklist is empty (S305-3), and if not, extract the vertices of the checklist (S305-4). For the corresponding vertex, vertices at the same location are found through the first dictionary prepared in the data preparation step (S300) (S305-5), and triangles (adjacent triangles) having adjacent vertices associated with the corresponding vertex are found through the second dictionary. (S305-6). By using a dictionary, you can quickly find co-located vertices and adjacent triangles and reduce the amount of calculation for search.

Additionally, the benchmark normal of the vertex (or the mesh to which the vertex belongs) is found (S305-7). The reference normal may be the normal vector of a triangle belonging to the mesh to which the corresponding vertex belongs.

Compare the normal line of the adjacent triangle with the reference normal line to determine whether the angle between them is within a preset angle (S305-8). Angle comparison may be made depending on whether the dot product of the two normals exceeds a predetermined reference value (eg, 0.75).

If it is within a preset angle (i.e., if the inner product of two normals exceeds a predetermined standard value), the triangle to which the current vertex belongs and the adjacent triangle are considered to have normals in the same direction, and can be determined to be triangles belonging to the same plane. (S305-9).

For triangles belonging to the same plane, vertices belonging to the triangle are extracted (S305-10), added to the checklist again, and the above-described process is repeated.

Through the above-described process, it is possible to find a set of triangles that are connected to each other and face the same direction, that is, forming the same plane. By growing triangles in this way, the triangles corresponding to each plane that makes up space or objects can be separated and grouped separately.

Data grouped into coplanar triangles goes through a group classification step (S310).

Referring to FIG. 6, the mesh analysis unit 130 performs analysis on data (mesh group) grouped in the same plane. The purpose of this analysis is to classify into space groups and object groups depending on whether the spatial conditions are met.

First, check whether the number of triangles grouped and included in one plane exceeds a predetermined number (for example, 10) (S310-1).

If the number is less than a predetermined number, it is considered not to be a plane related to a space with a large plane and is determined to belong to an object group (S310-9).

If the number exceeds a predetermined number, it is determined whether each condition of a space having a large plane is met. The plane that makes up space includes the floor, walls, and ceiling.

First, determine whether the floor condition is met (S310-2), and if the floor condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to floor (S310-3). And it is determined to belong to the space group (S310-8).

Next, determine whether the wall condition is met (S310-4), and if the wall condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to wall (S310-5). And it is determined to belong to the space group (S310-8).

Next, determine whether the ceiling condition is met (S310-6), and if the ceiling condition is met, the properties of the point and mesh group belonging to the corresponding plane are set to ceiling (S310-7). And it is determined to belong to the space group (S310-8).

The order of judgment regarding floor conditions, wall conditions, and ceiling conditions among space conditions is only an example, and the order may be changed.

If the floor condition, wall condition, and ceiling condition are all not met, the plane in question does not constitute a space, but can be viewed as a plane that constitutes one of the objects placed in the space. In this case, the points and mesh groups belonging to the plane is determined to belong to the object group (S310-9).

In this embodiment, the floor conditions are (1) whether the normal direction of the plane is upward, (2) whether the area is a certain size or more, and (3) whether it is located in a low position.

The wall conditions are (1) whether the normal direction of the plane is horizontal, (2) whether the area is over a certain size, and (3) whether the height is over a certain height (for example, 2 m).

Ceiling conditions are (1) whether the plane's normal direction is downward, and (2) whether it is located at a high location.

When the analysis by the mesh analysis unit 130 is completed and divided into a space group and an object group, the space object separation unit 140 performs a merge on the divided groups and finally completely separates the space model and the object model. .

Referring to FIG. 7, in the spatial merge step (S315), the spatial object separation unit 140 performs a check operation on all object groups targeting spatial groups (S315-1). Space groups can be broadly divided into floors, walls, and ceilings, and walls can be reclassified into multiple walls according to the normal direction.

During the check process, each object group (S315-2) checks whether it satisfies the spatial merge conditions (S315-3), and if it satisfies the spatial merge conditions, the points belonging to the object group are added to the spatial group, and the corresponding object Delete the group (S315-4).

The space merge conditions are (1) whether it is included in the space group currently being checked, whether the normal direction is parallel, and whether the number of points is less than a certain number (for example, 100).

Through the spatial merge step (S315), small object meshes within the spatial mesh are merged, making it possible to set the mesh data constituting the space as a spatial group among the remaining data that was not classified during the group classification process.

Mesh data set as a space group can be assigned floor, wall, and ceiling properties according to the above-mentioned classification conditions, and can be integrated as a space model excluding objects placed inside the mobile scanned space.

Next, the object merge step (S320) is performed through object recombination.

Referring to FIG. 8, in the object merging step (S320), the spatial object separation unit 140 performs a check operation on vertices belonging to an object group that does not belong to a spatial group (S320-1).

Check whether the vertex has been checked (S320-2), and if it is an unchecked vertex, add it to the checklist (S320-3). Check the checklist (S230-4), and if the checklist is empty, create a new object group, and if it is not empty, extract the vertices in the checklist (S320-5).

Check whether the corresponding vertex is an object vertex (S320-6). Objects placed within a space may be in contact with one or more planes that make up the space. In this case, during the connection process, the object plane may extend to the spatial plane through the point where it meets the spatial plane, so S320-6 is performed to block this.

In the case of an object vertex, vertices at the same location are found using the first dictionary (S320-7). Then, adjacent triangles are also found using the second dictionary (S320-8). And connected vertices can be obtained using the searched vertices and triangles (S320-9). Obtained vertices can be added to the check risk to expand connectivity.

Since the object is a three-dimensional object with planes in front, back, left and right, and up and down, the planes in each direction can be connected to each other by connected vertices, creating one object group (S320-10). For a newly created object group, the minimum bounding box containing it can be calculated (S320-11) and function as one object model.

Figure 9 is an example diagram of a preview screen during mobile scanning and a modeling screen after mobile scanning is completed.

Referring to (a) of FIG. 9, as described above, while creating a point cloud during the scanning process by the mobile scanning unit 110, the parts that require additional point collection are displayed in a designated color (orange in the drawing) so that the user can By filming the relevant part for a longer period of time, it is possible to obtain sufficient points for 3D modeling.

Referring to (b) of FIG. 9, when scanning of the space and the objects placed therein is completed through mobile scanning of an arbitrary space, a viewer or editor screen is shown where the corresponding model can be confirmed.

In the modeling process, not only color images but also depth images are used, so the length value for each modeled model can be obtained. The width, height, and height of the bounding box set for each model are displayed on the screen, informing the user of the actual size of the space where the current mobile scan was performed.

Figure 10 is an example diagram showing the separation of a space model and an object model.

Referring to FIG. 10, a case in which a chair is placed in a mobile scanned real space is illustrated. A digital twin is implemented by the mobile scan digital twin implementation device 100 according to this embodiment, so that the spatial model of the real space and the object model of the objects placed therein can be separated and managed as in reality.

Therefore, as shown in (a) of FIG. 10, the space and the chair, which cannot be separated in the two-dimensional image, can be separated from each other in the three-dimensional modeling state through digital twin implementation. In other words, only the chair model can be selected separately from the space.

When the selected chair model is deleted, the chair model disappears as shown in (b) of FIG. 10. However, in this case, there is no captured image of the floor surface where the chair was, so it may be an empty surface area. In response to this, through predetermined data processing, it is supplemented to display a texture similar to the surrounding floor as shown in (c) of Figure 10, so that the empty surface of the space model due to deletion of the object model is displayed without awkwardness from the user's perspective. It may be possible.

Figure 11 is a diagram showing the box clipping function among the editing functions performed in the editor of the mobile scan digital twin implementation device according to an embodiment of the present invention.

The above-described mobile scan digital twin implementation device 100 can generate a three-dimensional model (modeling object) of real space and objects placed therein through a mobile scan process.

The modeling object 10 created in this way is shown in FIG. 11.

The editor 160b can perform box clipping among the editing functions according to user commands. Box clipping is an operation in which a box-shaped tool is displayed on the screen and only the part that exists inside the box remains by deleting points and meshes outside the box for the modeling object. You can check the cross section of a modeling object with a simple operation.

Referring to (a) of FIG. 11, a basic clipping box 20 is shown. The clipping box 20 may be set as default to have a size corresponding to the modeling object 10 selected by the user. For example, the clipping box 20 may be created identical to the bounding box of the selected modeling object 10.

In this case, the bounding box is a box that completely accommodates the modeling object 10 therein, and the modeling object 10a can be maintained as is even by the clipping box 20.

The shape of the clipping box 20 can be adjusted or rotated according to user input.

Referring to Figures 11 (b) and (c), a case where the shape of the clipping box 20 is adjusted is shown.

When the user additionally selects a plane while the clipping box 20 is selected, the corresponding plane can move in the vertical direction, allowing the cross-section of the selected modeling object 10 to be displayed smoothly.

When one of the planes of the clipping box 20 is selected, the clipping box 20 can be stretched or shortened in the direction of movement by moving the selected plane perpendicular to the plane.

In Figure 11(b), a case 20a is shown where the upper surface of the clipping box is selected and moved downward. In this case, the upper part of the modeling object 10b is deleted, and only the lower part with a cross section at the top remains.

In Figure 11 (c), a case (20b) is shown where the front of the clipping box is selected and moved to the rear. In this case, the front part of the modeling object 10c is deleted and it becomes an object with a vertical cross-section.

When a part of the modeling object is clipped by box clipping, the planes of the clipping boxes 20, 20a, and 20b become cutting surfaces, and the points and mesh of the modeling object 10 existing outside the clipping boxes 20, 20a, and 20b are deleted.

Figure 12 is a flowchart of the plane crop process among the editing functions performed in the editor of the mobile scan digital twin implementation device according to an embodiment of the present invention, Figure 13 is a diagram showing a method of setting a cutting surface for plane crop, and Figure 14 is This is an example of the plane cropping process.

In the editor 160b, a 3D model (modeling object) generated by a mobile scan process can be displayed on the screen through the display unit.

When the plane crop function is activated by a user command, check whether there is a click by the user on the screen where the modeling object is displayed (S500). Clicking is an act of specifying a point and refers to user selection actions such as touching a touch screen or pushing/clicking a button using a keyboard/mouse.

When the click is recognized, it is checked whether it is the first point (S505). If it is the first point, the point is added as the starting point (S) (S510).

When the next click is recognized, the second point is added as the end point (E) (S515).

When two points are added by user selection, the editor 160b automatically creates a cutting surface for plane cropping (S520).

Referring to FIG. 13, the concept of modeling objects displayed through the editor screen is shown. The modeling object is placed in a three-dimensional space, but from the user's viewpoint (camera viewpoint, P1), it is projected onto the display screen, which is a two-dimensional plane.

In this case, when the starting point (S) and the ending point (E) are selected by the user, the user's viewpoint (P1) is automatically set as the third point, and a plane made of three points is calculated. There is only one plane created with three points, and this plane can be set as the cutting surface for plane cropping.

By automatically setting the user's viewpoint (P1) as a point for creating a cutting surface, the user can be saved from the trouble of entering three points to create a cutting surface. Additionally, by cutting along the user's viewpoint within the screen the user is currently viewing, more intuitive planar cropping can be possible.

When the cutting surface is created, a line (CL) is created between two points (starting point and ending point) on the display screen (S520). This line (CL) is a part of the straight line where the cutting surface meets the display screen, and the deleted portion and remaining portion are determined based on the line.

A selection is received from the user regarding one of the two parts 51 and 52 of the screen 50 divided by the cutting surface (S530).

When a user selection is made through a click, deletion processing is performed on the selected part (S535). In other words, points and meshes in a selected part of the modeling object can be deleted.

Referring to FIG. 14, a starting point (S) and an ending point (E) are designated for the modeling object 10, and a line (CL) where a cutting surface generated using the user's viewpoint (P1) meets the display screen is displayed. When the lower part (52) of the two parts (51, 52) of the screen separated by the line (CL) is selected, the corresponding part is deleted from the modeling object (10) and the cropped remaining object (10d) is displayed. You can. When rotating the remaining object 10d, a cross section may be displayed by plane cropping.

Previously, in S535, it was explained that the selected part is deleted, but depending on the embodiment, the selected part may remain and the unselected part may be deleted.

The reality is that during the 3D modeling process for real space, it is not easy to perform mobile scanning with all objects placed in the space removed. In this situation, according to this embodiment, it is possible to create and utilize a model for the space itself, excluding objects, by 3D modeling space and objects together and then separating the space model and object model.

The above-described mobile scan digital twin implementation method and modeling object editing method through spatial modeling can also be implemented in the form of a recording medium containing instructions executable by a computer, such as an application or program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data.

The method of implementing a mobile scan digital twin through spatial modeling and the method of editing modeling objects described above can be executed by an application installed by default on the terminal (this may include programs included in the platform or operating system, etc., installed by default on the terminal). It may also be executed by an application (i.e., program) installed by the user directly on the master terminal through an application providing server such as an application store server, an application, or a web server related to the service. In this sense, the mobile scan digital twin implementation method and modeling object editing method through spatial modeling described above are implemented as an application (i.e. program) installed by default on the terminal or directly installed by the user and can be read by a computer such as a terminal. It can be recorded on a recording medium.

Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand the present invention in various ways without departing from the spirit and scope of the invention as set forth in the claims below. You will understand that it can be modified and changed.

100: Mobile scanning digital twin implementation device
110: mobile scanning unit 120: modeling unit
130: mesh analysis unit 140: spatial object separation unit
150: storage unit 160: viewer/editor
10: Modeling object 20, 20a, 20b: Clipping box
50: Display screen

Claims (5)

A computer program stored in a computer-readable medium for performing a modeling object editing method when implementing a mobile scan digital twin, wherein the computer program causes the computer to perform the following steps, the steps comprising:
Loading modeling objects on the screen in the editor;
Receiving an edit command for the modeling object; and
Including the step of deleting some points and mesh of the modeling object in response to the editing command,
A computer program stored in a computer-readable medium, characterized in that the modeling object is a three-dimensional model implemented as a digital twin by mobile scanning a real space where real objects are placed.
According to paragraph 1,
The editing command is a plane crop command,
The deletion processing step is,
When the plane crop function is activated by the editing command, receiving a starting point and an ending point on the screen according to the user's selection;
Automatically generating a cutting surface using the starting point and the ending point;
displaying a straight line where the cut surface meets the screen on the screen;
receiving a selection input for a portion of the screen divided by the straight line; and
A computer program stored in a computer-readable medium, comprising the step of deleting a portion of the modeling object distinguished by the cutting surface in response to the selection.
According to paragraph 2,
The step of automatically generating the cutting surface includes finding one plane including all three points of the user's viewpoint corresponding to the starting point, the ending point, and the camera position for the modeling object displayed on the screen, and defining the corresponding plane. A computer program stored on a computer-readable medium characterized by setting a cutting plane.
According to paragraph 1,
The editing command is a box clipping command,
The deletion processing step is,
When the box clipping function is activated by the editing command, displaying a clipping box corresponding to the modeling object on the screen;
adjusting one or more of the shape and rotation of the clipping box;
A computer program stored in a computer-readable medium, comprising the step of using the planes constituting the clipping box as cutting surfaces and deleting an outer portion of the modeling object divided by the planes.
According to clause 4,
The adjusting step includes, when one of the planes of the clipping box is selected, moving the selected plane perpendicular to the plane to lengthen or shorten the clipping box in the direction of movement. A computer program stored in .