KR102626755B1 - A Framework for Automatic Generation of Augmented reality Objects - Google Patents

Abstract

The present invention relates to a framework for automatic creation of augmented reality objects. A server equipped to store facility-related data and communicate with AR devices, and generates AR assets, which are facility-related augmented reality 3D objects that can be confirmed in an AR environment. (server); A web that is equipped to link with a server through communication and performs a management function of facility-related IFC (Industrial Foundation Classes) files uploaded by users connected through the Internet environment; An AR device that is equipped to communicate with a server and outputs and displays AR assets, which are augmented reality 3D objects created related to facilities that can be viewed in an AR environment, wherein the server is built based on Amazon Web Services (AWS), Node.js-based backend for server operation, AWS's RDS (Relational Database Service) as a database, AWS's S3 (Scalable Storage Service) for data storage, and real-time two-way communication with AR devices. It includes an RMB (Redis Message Broker) unit and a socket unit, and the IFC file uploaded by the user connected to the web is automatically stored in the S3 unit, and the user can access it as needed through the RDS unit and the S3 unit. Accordingly, the file is configured to be checked later, and when a user connected to the web requests an AR device view, the web converts the IFC file corresponding to the requested view into 3D File Format and stores it in S3. , The server is configured to receive the corresponding request signal from the socket unit and transmit (publish) the stored file name to the RMB unit.
According to the present invention, by incorporating cutting-edge digital technologies such as AR (augmented reality) in the maintenance and management of construction-based facilities, it is possible to pursue technological advancement and implement a more efficient maintenance system than the existing one, and in particular, augmented reality that can be used in AR applications. By reducing and automating the conversion process of complex 3D objects resulting from the creation of assets, which are real objects, the final goal, augmented reality assets, can be easily and automatically created from the basic information of the facility.

Description

A Framework for Automatic Generation of Augmented Reality Objects}

The present invention relates to a framework for the automatic creation of augmented reality objects that focuses on the processing of data files related to construction infrastructure, including bridges and highways, and the combination of augmented reality (AR) technology, and provides augmented reality that can be used in applications. A framework for the automatic creation of augmented reality objects that allows for easy and automatic creation of the final goal, augmented reality assets, from the basic information of the facility by reducing and automating the conversion process of complex 3D objects according to asset creation. will be.

In general, information about construction-based facilities is changing from existing 2D CAD to 3D CAD, and recently to BIM (Building Information Modeling) environment.

The BIM represents a standard process for digitizing and visualizing information about facilities, includes various information including design and configuration, and creates 3D models using software such as AutoCAD and ArchiCAD. These BIMs are expressed as IFC (Industrial Foundation Classes) files developed by BuildingSmart.

Here, the IFC is an open data standard used for sharing and exchanging building information, and can be defined as standardized digital data that supports interoperability between engineering, architecture, and construction fields.

In addition, if BIM is a framework for digital collaboration, IFC can be said to be a language that enables such collaboration. In addition, BIM technology provides an environment that can effectively link and manage and process various 3D information generated at each stage of a construction project. It is also applied to maintenance systems for various facilities.

Meanwhile, safety diagnosis and maintenance and reinforcement inspections are being conducted periodically on facilities such as bridges and highways in Korea.

However, because the current inspection method is based on managers' visual or document-based inspection methods, problems with reduced reliability, systematic management, and efficiency are emerging. Accordingly, in order to effectively maintain these facilities, information on the facilities is expressed as BIM-based 3D objects, and AR (augmented reality)/VR (virtual reality)/MR() researched in the field of HCI (human-computer interaction) Smart management techniques that manage with AR glasses and mobile devices are being attempted by applying the latest technologies such as mixed reality.

In order to develop an advanced maintenance system for facilities that apply the latest technologies such as AR (Augmented Reality)/VR (Virtual Reality)/MR (Mixed Reality), BIM data, which is basic information about the facility, is used as the final augmented reality object. Conversion work is required. in other words, The asset creation process, which is an augmented reality object, first creates an IFC file, which is basic data about the facility, using BIM software, and converts the generated BIM file into a 3D file format usable in the AR system. And the converted 3D file is finally converted into an AR asset that can be used in AR applications using various AR development tools.

However, because the AR asset (augmented reality object) creation process is complex, it is difficult to easily develop augmented reality systems, and various problems are appearing as follows.

First, many steps and conversion operations are required in the creation process of augmented reality objects. In particular, it takes a lot of time for beginners because each step requires the application of various software necessary for extracting and converting information about the object.

Second, there are many difficulties in additional work to process or optimize information for the created object to suit the purpose.

Third, since various hardware (H/W) such as specific AR glasses or mobile devices are used depending on the target area, it is necessary to create appropriate 3D objects, but there are difficulties associated with this.

Therefore, there is an urgent need for alternatives to solve the above-mentioned problems caused by applying cutting-edge digital technologies such as AR (augmented reality) in the maintenance and management of facilities, and measures for more efficient management are required.

Meanwhile, when looking at the prior art literature in relation to the solution of the prior art, Domestic Publication Patent No. 10-2013-0101202 discloses “Augmented reality-based safety diagnosis information visualization method and system for bridge structures” and was registered in Korea. Patent No. 10-2521565 discloses “A device and method for providing and playing augmented reality services using 3D graph neural network detection,” and Domestic Publication Patent No. 10-2013-0063876 discloses “Using cloud computing.” “Augmented reality system and generation method” is disclosed.

However, these prior technologies can be said to be simply reference documents and have technical differences in terms of technical configuration and solution method.

Republic of Korea Patent Publication No. 10-2013-0101202 Republic of Korea Patent Publication No. 10-2521565 Republic of Korea Patent Publication No. 10-2013-0063876

The present invention was developed in consideration of and to improve the above-mentioned conventional problems, and enables the implementation of a more efficient maintenance management system by incorporating cutting-edge digital technologies such as AR (augmented reality) in the maintenance and management of construction-based facilities. In particular, AR By reducing and automating the conversion process of complex 3D objects according to the creation of assets, which are augmented reality objects that can be used in applications, the final goal, augmented reality assets, can be easily and automatically created from the basic information of the facility. Automatic creation of augmented reality objects. The purpose is to provide a framework for

The present invention makes it easier to process IFC files containing BIM data, which is building and construction information modeling, but automates many steps for processing existing IFC files to shorten complex processing processes and conveniently create augmented reality assets. The purpose is to provide a framework for the automatic creation of augmented reality objects that ensures file compatibility in various environments by providing conversion functions to various formats for IFC files of target facilities.

The present invention allows to reduce the size and processing capacity of objects when creating 3D objects by optimizing file conversion, and skips intermediate processing through communication between the AR device and the server that constitutes the system and converts the IFC file from the AR device. The purpose is to provide a framework for the automatic creation of augmented reality objects that can be viewed immediately and thus increase convenience.

The framework for automatic creation of augmented reality objects according to the present invention to achieve the above purpose is equipped to store facility-related data and communicate with AR devices, and is a facility-related augmented reality 3D object that can be confirmed in an AR environment. A server that creates AR assets; A web that is equipped to link with a server through communication and performs a management function of facility-related IFC (Industrial Foundation Classes) files uploaded by users connected through the Internet environment; An AR device that is equipped to communicate with a server and outputs and displays AR assets, which are augmented reality 3D objects created related to facilities that can be viewed in an AR environment, wherein the server is built based on Amazon Web Services (AWS), Node.js-based backend for server operation, AWS's RDS (Relational Database Service) as a database, AWS's S3 (Scalable Storage Service) for data storage, and real-time two-way communication with AR devices. It includes an RMB (Redis Message Broker) unit and a socket unit, and the IFC file uploaded by the user connected to the web is automatically stored in the S3 unit, and the user can access it as needed through the RDS unit and the S3 unit. Accordingly, the file is configured to be checked later, and when a user connected to the web requests an AR device view, the web converts the IFC file corresponding to the requested view into 3D File Format and stores it in S3. , The server is configured to receive the corresponding request signal from the socket unit and transmit (publish) the stored file name to the RMB unit.

According to one type, the AR device is a HoloLens and is equipped with a Redis handler module for interaction with the server and Redis communication, and when the server transmits signal data to the AR device according to the user's request, The AR device receives the corresponding signal data requested by the user from the server, retrieves the corresponding 3D object from the S3 unit, outputs and displays it in an augmented reality (AR) environment, and provides specific signal data from the server according to the user's request. When transmitted (published) through Redis communication, the AR device can receive signal data transmitted from the server, search the corresponding 3D object in S3, load the searched 3D object to the AR device, and display it in the AR environment. .

According to one type, the web is built with a React-based frontend to operate a website capable of uploading files, and the IFC Viewer unit, IFC, allows users to upload IFC files and easily check the uploaded IFC files. It includes an IFC processing unit for optimal file processing and a 3D file format conversion unit for converting IFC data into a 3D file format. On the web, the IFC file uploaded by the user is converted into a 3D format file and uploaded to the S3 section. Then, it is configured to send specific signal data to the AR device, but when sending specific signal data to the AR device, socket communication is configured between the web and the server, and when signal data is sent from the web through socket communication, the server It can be configured to receive data from and finally transmit it to the AR device through Redis communication.

According to one type, the IFCLoader module and UploadS3 module are used on the web when opening an IFC file uploaded by a user. After loading the IFC file through the IFCLoader module, the IFC file is uploaded to the S3 section through the UploadS3 module. IFC function unit that functions as a callback for the InitializeViewer module and SetViewer module to render the IFC Viewer unit; The GetS3List module and Modal module are used to manage uploaded IFC files, and the Recent Files function section functions to check and manage the list of IFC files in the Modal module for IFC files uploaded through the UploadS3 module in the IFC function section. ; The ResetView module is used to delete the IFC file rendered in the IFC Viewer part, and when Reset is executed, the Reset function part functions to return to the initial Viewer state through the InitializeViewer module and SetViewer module; The RedisPublish module and SendSignal module are used to send signals to the AR device, and when the function is executed, the View In HoloLens Function unit functions to perform socket communication with the AR device through the ReceiveMessage module and SendMessage module; The Export module, Parse module, and CreateObjectUrl module are used to convert files with the extensions of GLTF, OBJ, PLY, and STL, and a Convert Function unit that functions to convert files in the formats with the above four extensions based on IFC files; In order to render IFC files on the web, the InitializeViewer module and SetViewer module are used, and in addition to being used as a callback of the Reset function unit, an IFC view function unit can set the state of the IFC viewer unit; The ReceiveMessage module and the SendMessage module are used for real-time communication with the RMD unit, and the communication function unit functions to interact with the IFC file in the AR device in real time together with the view in Hololens function unit.

According to one type, the web has a framework menu for a user interface, and includes a File menu, an Edit menu, and an Optimize menu displayed on the web screen so that the user can select, and the File menu allows the user to use IFC Open IFC, which allows you to upload files and check them in the IFC Viewer, Recent Files, where you can check the history of IFC files uploaded by users, check the upload date, or use the function to save and delete files, and IFC Viewer. It has four submenus: Reset, which allows you to return to the initial state, and View In Hololens, which allows you to view the IFC file currently being viewed in the IFC Viewer section on the AR device and transmits a signal to the AR device when clicked; The Edit menu includes Convert as GLTF, which allows you to convert and save IFC files uploaded through Open IFC into GLTF format files, Convert as OBJ, and Open IFC, which allows you to convert and save IFC files uploaded through Open IFC into OBJ format files. It has four submenus: Convert as PLY, which allows you to convert and save IFC files uploaded through Open IFC into PLY format files, and Convert as STL, which allows you to convert and save IFC files uploaded through IFC into STL format files; The Optimize menu may have a submenu of Optimize IFC that not only optimizes IFC files uploaded through Open IFC, but also reduces file size and allows them to be stored locally.

According to one type, the IFC processing unit processes an algorithm to improve efficiency in the data transmission, storage, and processing process by creating an IFC optimization file with a reduced capacity compared to the original IFC file, and processes the original IFC file through the algorithm. The operation of storing the file-side 3D model in Web Assembly memory, copying it from this Web Assembly memory, and returning a new buffer is performed. (1) The process of calculating the size of the 3D model, (2) The process of calculating the size of the 3D model. Process of creating a new array with a size equal to the header bytes added, (3) Process of extracting 3D model data, (4) Process of copying the extracted 3D model data to Web Assembly memory, (5) Process of extracting the extracted 3D model data It may be a sequential processing structure that includes (6) the process of creating a new buffer stored in the web assembly memory, and (6) the process of returning the created new buffer.

According to one type, the Local ifc file is loaded and displayed in the IFC Viewer through the submenu of Open IFC, and the loaded ifc file is automatically uploaded to AWS S3, so that the Local ifc file is displayed in the Recent Files submenu. Even without this, ifc files can be loaded from the server, and through the View In HoloLens submenu, the ifc file currently displayed in the IFC Viewer section is loaded and signaled on the AR device to check assets in the AR environment in real time. It can be configured to do so.

According to the present invention, by incorporating cutting-edge digital technologies such as AR (augmented reality) in the maintenance and management of construction-based facilities, it is possible to pursue technological advancement and implement a more efficient maintenance system than the existing one, and in particular, augmented reality that can be used in AR applications. By reducing and automating the conversion process of complex 3D objects resulting from the creation of assets, which are real objects, it can provide the useful effect of easily and automatically generating the final goal, augmented reality assets, from the basic information of the facility.

According to the present invention, it is possible to more easily process IFC files containing BIM data, which is building and construction information modeling, and automate many steps for processing existing IFC files to shorten the complex processing process and conveniently create augmented reality assets. It can be created, the entire file of the target facility can be processed, and by providing conversion functions for IFC files into various formats, it can provide a useful effect of ensuring file compatibility in various environments.

According to the present invention, the size and processing capacity of the object can be reduced when creating a 3D object through file conversion optimization, and intermediate processing can be omitted through communication between the AR device and the server that constitutes the system, and the IFC file can be converted to the AR device. It can provide useful effects that increase convenience, such as allowing you to view it immediately.

Figure 1 is a conceptual diagram showing a framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 2 is a block diagram showing a framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 3 is an example diagram showing the File menu on a web screen in the framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 4 is an example diagram showing the Edit menu on a web screen in the framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 5 is an example diagram showing the Optimize menu on a web screen in a facility maintenance system with a framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 6 is an example diagram showing IFC data loaded on a web screen through the IFC Viewer unit in the framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 7 shows the ifc file currently being viewed on the web screen on the AR device through the View In Hololens submenu function in the framework for automatic creation of augmented reality objects according to an embodiment of the present invention as an asset in the AR environment in real time. This is an example showing that the status can be checked.
Figure 8 is an example diagram showing the status of file conversion and export to another 3D object in the framework for automatic creation of augmented reality objects according to an embodiment of the present invention.
Figure 9 is an example diagram showing the output of the results of a 3D object to which the IFC optimization algorithm is applied in the framework for automatic creation of augmented reality objects according to an embodiment of the present invention.

Preferred embodiments of the present invention will be described with reference to the drawings as follows. Through this detailed description and drawings, the purpose and configuration of the present invention and its characteristics will be better understood.

1 to 9 are diagrams showing a framework for automatically creating augmented reality objects according to an embodiment of the present invention.

As shown in Figures 1 and 2, the framework for automatic creation of augmented reality objects according to an embodiment of the present invention consists of a server 100, a web 200, and an AR device 300.

The server 100 is equipped to store facility-related data and communicate with the AR device 300, and is configured to generate AR assets, which are facility-related augmented reality 3D objects that can be confirmed in an AR (augmented reality) environment. am.

The server 100 is built as a server based on Amazon Web Services (AWS) to enable sophisticated applications with improved flexibility, scalability, and stability.

The server 100 includes a Node.js-based backend unit 110 for operating the server, an AWS Relational Database Service (RDS) unit 120 for use as a database, and a Relational Database Service (RDS) unit 120 for use as a database. It includes an AWS S3 (Scalable Storage Service) unit 130, a RMB (Redis Message Broker) unit 140 and a socket unit 150 that enable real-time two-way communication with the AR device 300.

The IFC file uploaded by the user connected to the web 200 is automatically stored in the S3 unit 130.

The RDS unit 120 and S3 unit 130 allow the user to check the corresponding file later as needed.

When a user connected to the web 200 requests a view from the AR device 300, the web 200 converts the IFC file corresponding to the requested view into 3D File Format and stores it in the S3 unit 130. , the server 100 is configured to receive the corresponding request signal from the socket unit 150 and transmit (publish) the stored file name to the RMB unit 140.

The web 200 is equipped to link with the server 100 through communication and is configured to perform management functions of facility-related IFC (Industrial Foundation Classes) files uploaded by users connected through the Internet environment. .

The web 200 is built with a React-based frontend to operate a website capable of uploading files.

The web 200 includes an IFC Viewer unit 201 for allowing the user to upload an IFC file through an interface and display the uploaded IFC file for easy confirmation, and an IFC processing unit for optimally processing the IFC file uploaded by the user. It includes (202) and a 3D file format conversion unit 203 for converting the IFC data contained in the IFC file into a 3D file format.

The web 200 is configured to convert the IFC file uploaded by the user into a 3D Format File, upload it to the S3 unit 130 on the server 100, and then send specific signal data to the AR device 300. When sending specific signal data to the device 300, socket communication is performed between the web 200 and the server 100.

In addition, when the web 200 sends signal data to the server 100 through socket communication, the server 100 receives it and finally transmits it to the AR device 300 through Redis communication on the RMD unit 140. do.

The web 200 has a software configuration for performing each function, and includes a number of core functions and modules used for these functions.

In detail, the web 200 includes an IFC function unit 200a, a Recent Files function unit 200b, a Reset function unit 200c, a View In HoloLens Function unit 200d, a Convert Function unit 200e, and an IFC view function. Includes a part (200f) and a communication function part (200g).

The IFC function part uses the IFCLoader module and UploadS3 module when opening an IFC file uploaded by the user. After loading the IFC file through the IFCLoader module, the IFC file is uploaded to the S3 section through the UploadS3 module and then rendered to the IFC Viewer section. To do this, the InitializeViewer module and SetViewer module function as callbacks.

The Recent Files function part uses the GetS3List module and the Modal module to manage the IFC files uploaded by the user, and the IFC file uploaded through the UploadS3 module in the IFC function part is checked and manages the list of IFC files in the Modal module. It functions to do so.

The Reset function part uses the ResetView module to delete the IFC file rendered in the IFC Viewer unit, and when Reset is executed, it returns to the initial Viewer state through the InitializeViewer module and SetViewer module.

The View In HoloLens Function part uses the RedisPublish module and SendSignal module to send signals to the AR device, and when the function is executed, it functions to proceed with socket communication with the AR device through the ReceiveMessage module and SendMessage module.

The Convert Function unit uses the Export module, Parse module, and CreateObjectUrl module to convert files with the extensions of GLTF, OBJ, PLY, and STL, and functions to convert files in the formats with the above four extensions based on IFC files. .

The IFC view function unit uses the InitializeViewer module and SetViewer module to render IFC files on the web, and is used as a callback for the Reset function unit and functions to set the state of the IFC viewer unit.

The Communication function unit uses the ReceiveMessage module and SendMessage module for real-time communication with the RMD unit on the server side, and together with the view In Hololens function unit, it functions to interact with IFC files on the AR device in real time.

In addition, the web 200 has a framework menu for a user interface, and includes a File menu, an Edit menu, and an Optimize menu that are displayed on the web screen and available for selection by the user.

As shown in Figure 3, the File menu has four submenus: Open IFC, Recent Files, Reset, and View In Hololens.

The Open IFC submenu allows the user to upload an IFC file and check it in the IFC Viewer section, as shown in FIG. 6.

The Recent Files submenu allows users to view the history of IFC files uploaded by the user, check the upload date, or use the function to save and delete files.

The Reset submenu allows the IFC Viewer unit to return to its initial state.

The View In Hololens submenu allows the IFC file currently being viewed in the IFC Viewer section to be viewed directly on the AR device and, when clicked, transmits a signal to the AR device.

The File menu, which has the above-described four submenus, matches the IFC Viewer through activation of the IFC Viewer unit, and loads and displays local ifc files into the IFC Viewer unit through the Open IFC submenu, and displays the ifc files loaded in this way. By automatically uploading files to AWS S3, you can load ifc files from the server even if there are no local ifc files in the Recent Files submenu, and ifc files currently displayed in the IFC Viewer section on the AR device through the View In HoloLens submenu. By loading the file and giving a signal, you can check the asset in the AR environment in real time, as shown in Figure 7.

As shown in Figure 4, the Edit menu has four submenus: Convert as GLTF, Convert as OBJ, Convert as PLY, and Convert as STL.

The Convert as GLTF submenu allows you to convert and save an IFC file uploaded through Open IFC into a GLTF format file.

The Convert as OBJ submenu allows you to convert and save an IFC file uploaded through Open IFC into an OBJ format file.

The Convert as PLY submenu allows you to convert and save IFC files uploaded through Open IFC into PLY format files.

The Convert as STL submenu allows you to convert an IFC file uploaded through Open IFC into an STL format file and save it.

The Edit menu, which has the four submenus described above, matches file format conversion through activation of the 3D file format conversion unit, and is configured to support conversion to various file formats in consideration of the expandability of ifc files.

The present invention is configured to support conversion and export of 3D objects in GLTF, OBJ, PLY, and STL formats in order to increase scalability of IFC files and usability in various other systems. You can choose to convert to an object.

Figure 8 shows an example of supporting conversion and export of 3D objects for IFC files.

At this time, the processing sequence for converting to a desired object through the four submenus of the Edit menu is as follows.

(1) Upload the IFC file using the Framework menu for user interface.

(2) Initialize the IFC Viewer part.

(3) Load the uploaded IFC file and create an IFC Model.

(4) Parse the IFC Model with the selected extension.

(5) Convert the parsed object to a JSON string.

(6) Enter the JSON string and convert it into large binary data.

(7) When the user downloads, the conversion file is delivered.

As shown in FIG. 5, the Optimize menu has an Optimize IFC submenu.

The Optimize IFC submenu not only optimizes the IFC file uploaded through Open IFC, but also reduces the file size so that it can be stored locally.

The Optimize menu with the above-described Optimize IFC submenu matches the optimization of the uploaded IFC file through activation of the IFC processing unit, and transmits and stores data by creating an IFC optimization file with a reduced capacity compared to the original IFC file. , algorithm processing is performed to improve the efficiency of the processing process.

In addition, optimization of the uploaded IFC file involves storing the 3D model of the original IFC file in Web Assembly memory through an algorithm, copying it from this Web Assembly memory, and returning a new buffer. The processing order is as follows. .

(1) Calculate the size of the 3D model of the uploaded IFC file.

(2) Create a new array with a size equal to the size of the 3D model plus the required header bytes.

(3) Extract 3D model data.

(4) Copy the extracted 3D model data to Web Assembly memory.

(5) Create a new buffer that stores the extracted 3D model data in Web Assembly memory.

(6) Returns the new buffer created.

In other words, files created through an algorithm with this series of processing processes have a reduced capacity compared to the original IFC file, which solves the problem of time delays or space problems during data transmission, storage, and processing. Efficiency can be increased by doing so.

The optimization data of the IFC file is stored in the server 100.

Figure 9 shows the output state of a 3D object to which the IFC optimization algorithm is applied when the Optimize IFC submenu is selected.

The AR device 300 is equipped to communicate with the server 100 and is configured to output and display AR assets, which are augmented reality 3D objects created related to facilities that can be confirmed in an augmented reality (AR) environment.

The AR device 300 uses HoloLens.

The AR device 300 is equipped with a Redis handler module to interact with the server 100 and perform Redis communication.

In the AR device 300, when the server transmits signal data to the AR device according to the user's request, it receives the corresponding signal data requested by the user from the server 100 and creates a 3D object corresponding to the corresponding signal data in the S3 unit 130. It is taken from , printed and displayed in an augmented reality (AR) environment.

In addition, when the server 100 transmits (publishes) specific signal data through Redis communication according to the user's request, the AR device 300 receives the signal data transmitted from the server 100 and creates the corresponding 3D object. The S3 unit 130 retrieves the searched 3D object and loads it into the AR device 300 to display it in an augmented reality (AR) environment.

At this time, the AR device 300 is equipped with the Unity platform, software that enables 3D, virtual reality (VR), and augmented reality (AR) displays and a three-dimensional, immersive interactive experience.

The AR device 300 is equipped to immediately receive edit signal data when there is some modification to the augmented reality object information of the facility in the server 100 linked to the web 200, and receives edit signal data based on the edit signal data. It can be configured to immediately load and display edited and stored change data from the RDS unit 120, which is the database of the server 100.

Meanwhile, in the following, experiments were conducted on the conversion of 3D objects to test the optimization of IFC files. Tests were conducted with a total of four ifc files, and the experimental results are shown in Table 1 below.

Table 1 compares the capacities of the original ifc, optimized ifc, converted gltf, converted obj, converted ply, and converted stl files, and analyzes the capacity reduction by extension.

Table 1 above shows the results of applying the optimization method to various file formats, and it was observed that the capacity was reduced.

In detail, compared to the original ifc file, optimized IFC files have an average size reduction of 6%, files converted to GLTF have an average size reduction of 73%, files converted to OBJ have an average size reduction of 60%, and files converted to PLY. showed an average capacity reduction rate of 66%, and files converted to STL showed an average capacity reduction rate of 46%.

According to these file tests and their results, the optimization method showed a capacity reduction effect ranging from a minimum of 6% to a maximum of 73% based on the average for various file formats, and optimization of the ifc file in the system of the present invention The conversion function shows that it is possible to convert files into various formats while reducing file size.

According to these file tests and their results, it can be said that the present invention can efficiently use data storage space and have file compatibility in various environments.

Accordingly, through the present invention, IFC files containing building and construction information modeling (BIM) data can be processed more easily and can provide various advantages as follows.

First, by automating many steps to process IFC files, the complex processing process can be shortened, allowing you to conveniently create augmented reality assets.

Second, it is possible to process partial IFC files rather than the entire IFC file of the target facility.

Third, IFC files can be converted into various formats, providing file compatibility in various environments.

Fourth, capacity can be reduced when creating augmented reality 3D objects through file optimization and conversion.

Fifth, through communication between the server and the AR device, intermediate processing can be omitted, and convenience can be increased by allowing IFC files to be viewed immediately on the AR device (HoloLens).

The above description is an illustrative description of the present invention, and the embodiments published in the specification are not intended to limit the technical idea of the present invention, but are for illustrative purposes, so those skilled in the art Various modifications, transformations, or substitution of steps will be possible without departing from the technical idea of . Therefore, the scope of protection of the present invention should be interpreted based on the matters stated in the claims, and technical matters within the equivalent scope thereof should also be interpreted as being included in the scope of rights of the present invention.

100: server
110: Node.js-based backend
120: RDS (Relational Database Service) Department
130: S3 (Scalable Storage Service) part
140: RMB (Redis Message Broker) part
150: Socket part
200: web
201: IFC Viewer section
202: IFC processing unit
203: 3D file format conversion unit
300: AR device

Claims (7)

A server that is equipped to store facility-related data and communicate with AR devices, and generates AR assets, which are facility-related augmented reality 3D objects that can be viewed in an AR environment;
A web that is equipped to link with a server through communication and performs a management function of facility-related IFC (Industrial Foundation Classes) files uploaded by users connected through the Internet environment;
An AR device that is equipped to communicate with a server and outputs and displays AR assets, which are augmented reality 3D objects created related to facilities that can be viewed in an AR environment; Includes,
The server is built based on Amazon Web Services (AWS), with a Node.js-based backend for running the server, AWS's RDS (Relational Database Service) as a database, and AWS's S3 (S3) for data storage. It includes a Scalable Storage Service (RMB) unit, a RMB (Redis Message Broker) unit that enables real-time two-way communication with AR devices, and a socket unit,
The IFC file uploaded by the user connected to the web is automatically saved and stored in the S3 section, and the user can check the file later as needed through the RDS section and the S3 section,
When a user connected to the web requests an AR device view, the web converts the IFC file corresponding to the requested view into 3D File Format and stores it in the S3 section, and the server receives the corresponding request signal from the socket section. A framework for automatic creation of augmented reality objects, characterized in that the saved file name is transmitted (published) to the RMB department. According to clause 1,
The AR device is HoloLens,
A Redis handler module is installed to interact with the server and perform Redis communication,
When the server transmits signal data to the AR device according to the user's request, the AR device receives the corresponding signal data requested by the user from the server, retrieves the corresponding 3D object from the S3 unit, and enters the augmented reality (AR) environment. Print and display,
When the server transmits (publishes) specific signal data through Redis communication according to the user's request, the AR device receives the signal data transmitted from the server, searches the S3 for the corresponding 3D object, and sends the searched 3D object to the AR device. A framework for the automatic creation of augmented reality objects, characterized in that they are loaded and configured to be displayed in an AR environment. According to clause 1,
The web is built with a React-based frontend to operate a website capable of uploading files, with an IFC Viewer unit that allows users to upload IFC files and check the uploaded IFC files, and an IFC to process IFC files. It includes a processing unit and a 3D file format conversion unit to convert IFC data into a 3D file format.
On the web, the IFC file uploaded by the user is converted to a 3D Format File, uploaded to the S3 section, and then configured to send specific signal data to the AR device. However, when sending specific signal data to the AR device, a socket (socket) is established between the web and the server. Socket) A framework for automatic creation of augmented reality objects, configured to communicate, and configured to receive signal data from the server when sending signal data through socket communication on the web and finally transmit it to the AR device through Redis communication. According to clause 1,
The web is,
When opening an IFC file uploaded by a user, the IFCLoader module and UploadS3 module are used. After loading the IFC file through the IFCLoader module, the IFC file is uploaded to S3 through the UploadS3 module, and then the InitializeViewer module is used to render the IFC Viewer. IFC function part that functions to enable the SetViewer module to operate as a callback;
The GetS3List module and Modal module are used to manage uploaded IFC files, and the Recent Files function section functions to check and manage the list of IFC files in the Modal module for IFC files uploaded through the UploadS3 module in the IFC function section. ;
The ResetView module is used to delete the IFC file rendered in the IFC Viewer part, and when Reset is executed, the Reset function part functions to return to the initial Viewer state through the InitializeViewer module and SetViewer module;
The RedisPublish module and SendSignal module are used to send signals to the AR device, and when the function is executed, the View In HoloLens Function unit functions to perform socket communication with the AR device through the ReceiveMessage module and SendMessage module;
The Export module, Parse module, and CreateObjectUrl module are used to convert files with the extensions of GLTF, OBJ, PLY, and STL, and a Convert Function unit that functions to convert files in the formats with the above four extensions based on IFC files;
In order to render IFC files on the web, the InitializeViewer module and SetViewer module are used, and in addition to being used as a callback of the Reset function unit, an IFC view function unit can set the state of the IFC viewer unit;
A ReceiveMessage module and a SendMessage module are used for real-time communication with the RMD unit, and a Communication function unit that functions to interact with an IFC file in an AR device in real time together with the view In Hololens function unit; A framework for automatic creation of augmented reality objects, comprising: According to clause 1,
The web has a framework menu for a user interface, and includes a File menu, an Edit menu, and an Optimize menu displayed on the web screen so that the user can select,
The above File menu is Open IFC, which allows users to upload IFC files and check them in the IFC Viewer section. Users can check the history of IFC files uploaded by users, check the upload date, or use functions to save and delete files. Recent Files, Reset, which returns the IFC Viewer to its initial state, and View In Hololens, which allows you to view the IFC file currently being viewed in the IFC Viewer directly on the AR device and sends a signal to the AR device when clicked. Has submenus;
The Edit menu includes Convert as GLTF, which allows you to convert and save IFC files uploaded through Open IFC into GLTF format files, Convert as OBJ, and Open IFC, which allows you to convert and save IFC files uploaded through Open IFC into OBJ format files. It has four submenus: Convert as PLY, which allows you to convert and save IFC files uploaded through Open IFC into PLY format files, and Convert as STL, which allows you to convert and save IFC files uploaded through IFC into STL format files;
The Optimize menu is a framework for the automatic creation of augmented reality objects, characterized in that it has a submenu of Optimize IFC that reduces the size of the IFC file uploaded through Open IFC and allows it to be stored locally. According to clause 3,
The IFC processing unit,
By creating an IFC file with a reduced capacity compared to the original IFC file, an algorithm is processed to improve the efficiency of the data transmission, storage, and processing process.
Through an algorithm, the 3D model on the original IFC file is stored in Web Assembly memory, copied from this Web Assembly memory, and a new buffer is returned.
(1) The process of calculating the size of the 3D model,
(2) The process of creating a new array with a size equal to the size of the 3D model plus the required header bytes,
(3) Process of extracting 3D model data,
(4) The process of copying the extracted 3D model data to web assembly memory,
(5) Process of creating a new buffer storing the extracted 3D model data in web assembly memory,
(6) A framework for automatic creation of augmented reality objects, characterized by sequential processing in the process of returning the new buffer created. According to clause 5,
Through the Open IFC submenu, local ifc files are loaded and displayed in the IFC Viewer, and the loaded ifc files are automatically uploaded to AWS S3, so even if there is no local ifc file in the Recent Files submenu, ifc files are stored on the server. Configure to load files,
Automatic augmented reality objects, characterized in that the AR device loads the ifc file currently displayed in the IFC Viewer section through the View In HoloLens submenu and sends a signal so that assets in the AR environment can be checked in real time. A framework for creation.