KR102642392B1 - System for maintenance of facilities using augmented reality technology - Google Patents

Abstract

The present invention relates to a facility maintenance system using augmented reality technology. It is a perspective glasses-type display device that supports an extended reality (XR) experience that combines reality and AR content and implements augmented reality (AR) content on a transparent lens. It is a device that inspectors wear on their faces and uses AR glasses to support visual inspection of facilities; It includes a mobile device owned by a facility inspector, which exchanges data by performing two-way data communication through wireless communication with the AR glasses, and the AR glasses are used for visual inspection of the facility through perspective and acquisition of real-time images. An AR interface module, which is a component that connects reality and AR content without a sense of heterogeneity, enables confirmation of damage information caused to facilities subject to inspection in the real world from acquired images; A drawing data output module that displays facility drawing data (BIM) by retrieving past damage information of the facility subject to inspection stored in AR glass or by retrieving past damage information of the facility subject to inspection stored from a mobile device; An augmented reality expansion module that expands by processing visualization data and enhancing the sense of reality regarding damage information caused to facilities confirmed through the AR interface module; Includes an overlay mapping processing module that automatically inputs the confirmed damage information of the inspection target facility through visualization data processing and enhanced realism through an augmented reality expansion module by overlay mapping it on the currently displayed facility drawing data (BIM); , The mobile device is characterized as being one type selected from a smartphone, a tablet PC, and a personal digital assistant (PDA).
According to the present invention, by incorporating augmented reality technology when visually inspecting facilities including bridges, the visualization and sense of reality of the facility are increased, thereby increasing the effectiveness of facility inspection, diagnosis, and maintenance compared to existing visual inspection methods. The efficiency of visual inspection can be improved, errors due to visual inspection can be minimized compared to before, and digitization and advanced processing can be performed for damage information and condition evaluation of facilities.

Description

Facility maintenance system using augmented reality technology {SYSTEM FOR MAINTENANCE OF FACILITIES USING AUGMENTED REALITY TECHNOLOGY}

The present invention relates to a facility maintenance system using augmented reality technology. More specifically, the present invention relates to a facility maintenance system using augmented reality technology, and more specifically, by incorporating augmented reality technology during visual inspection of facilities, including bridges, etc., the visualization and sense of reality of the facility are increased, making the facility safer than existing methods. Augmented reality technology that improves efficiency while increasing the effectiveness of inspection, diagnosis, and maintenance management, minimizes errors, and performs digitalization and advanced processing for damage information and condition evaluation of facilities. It is about a facility maintenance system using .

In general, facilities are largely classified into bridges, tunnels, ports, dams, buildings, rivers, water and sewage systems, retaining walls and cut slopes, and utility tunnels.

Inspection and maintenance of these facilities are very important for safe use. Conventionally, facility maintenance still adheres to the manpower-based method. Facilities that are difficult for manpower to access, such as steep slopes (slopes), elevated bridges, and dam systems. Direct inspection is carried out using a rappel or aerial vehicle.

However, the conventional human-based method of inspection not only costs a lot of money and time, but also has problems such as low efficiency, high risk of safety accidents such as falls and collisions due to riding equipment, and the reliability of measurement data is not high. .

In addition, in the conventional facility inspection and diagnosis, tablets with drawing data stored in the field are mainly used as a method of checking facility drawings, but there is difficulty in confirming damage to the facility.

In addition, in the past, it was difficult to eliminate errors that may occur in the process of simply visually inspecting and confirming damage to facilities, and there was a problem that the accuracy and efficiency of facility inspection, diagnosis, and maintenance management were low.

Meanwhile, the revised Special Act on Safety and Maintenance of Facilities (Facility Safety Act) changed facility maintenance from a post-maintenance system to a proactive maintenance system through the introduction of performance evaluation, and the “Enforcement Decree” of the same Act made it possible to utilize new technologies. By doing so, it is recommended to lay the foundation for a preemptive maintenance system using cutting-edge technology.

Accordingly, facility maintenance technology that applies robots and artificial intelligence (AI) is being developed based on industry-academic cooperation, but the scope of application of robots in the field is extremely limited, and artificial intelligence (AI) is low in reliability, so it is based on actual visual inspection. There are limits to the application of facility maintenance and management, and improvements in the field are continuously required.

In relation to this, looking at prior art literature, Domestic Patent Publication No. 10-2430041 states that “obtaining a basic image including a facility using an image sensor, and obtaining a three-dimensional image of the internal structure of the facility through a wired or wireless communication network. And a user terminal that acquires a three-dimensional image of the repair history of the facility and outputs an augmented reality image generated based on the three-dimensional image of the internal structure of the facility and the three-dimensional image of the repair history of the facility; and receiving the location and direction of the user terminal from the user terminal, recognizing the facility based on the location and direction, and providing a three-dimensional image of the internal structure of the recognized facility and a three-dimensional maintenance history. and a server that provides images to the user terminal through the wired and wireless communication network, wherein the location of the user terminal is determined through at least one of an acceleration sensor and a gyro sensor of the user terminal, and the server provides the basic image, the The augmented reality image is created through a process of combining a 3D image of the internal structure of the facility and a 3D image of the repair history of the facility, and the facility is created from the basic image based on the location and direction of the user terminal. Recognizing, if the facility includes a plurality of objects, the server determines that if the real-time data transmission amount of the server is less than or equal to a predetermined standard throughput and the real-time data throughput of the server is less than or equal to the predetermined standard throughput, According to the results of comparing the real-time movement speed of the user terminal and each of a plurality of reference movement speeds, 3D images of the plurality of objects are selectively delivered to the user terminal, and the real-time data transmission amount of the server is equal to the predetermined reference transmission amount. or, if the real-time data throughput of the server exceeds a predetermined standard throughput, augmented reality technology is used to deliver only 3D images of objects located in a specific direction from the location of the user terminal to the user terminal. We are starting to construct a “facility inspection system that inspects facilities.”

In addition, in Korean Patent Publication No. 10-2022-0114831, “In the method of providing maintenance service for an AR-based object, (a) the management server measures the temperature of the object through a measurement sensor adjacent to the object. Obtaining sensing information including in real time, performing an operation in a prediction module using this as an input value, and calculating the state of the object through this; (b) the management server, information on each part of the object and the Implementing the object implemented based on the state of the object in augmented reality and displaying it on the user's terminal; and (c) the management server, based on the input received from the user's terminal, in augmented reality A method for providing maintenance services for AR-based objects including the step of controlling the object and a server using the same are disclosed.

However, although the above-mentioned prior art documents use augmented reality technology, they can be said to have technical differences from the technical configuration of the "facility maintenance system using augmented reality technology" of the present invention proposed below.

Republic of Korea Patent Publication No. 10-2430041 Republic of Korea Patent Publication No. 10-2022-0114831

The present invention was developed to improve and take into account the above-mentioned conventional problems. By incorporating augmented reality technology during visual inspection of facilities, including bridges, etc., it increases the visualization and sense of reality of the facility, making it easier to inspect facilities compared to existing methods. Augmented reality technology that enhances the effectiveness of diagnosis and maintenance, improves the efficiency of visual inspection, minimizes errors, and performs digitalization and advanced processing for damage information and condition evaluation of facilities. The purpose is to provide a facility maintenance system using .

The present invention enables users to check damage information on facilities subject to inspection through an augmented reality (AR) interface, and automates the status assessment of damage information on these facilities based on a progressive web app (PWA) to perform facility maintenance tasks. The purpose is to provide a facility maintenance system using augmented reality technology that can reduce time and cost due to automatic condition assessment and implement a new digital-based facility maintenance system.

The facility maintenance system using augmented reality technology to achieve the above purpose is a perspective glasses-type display device that supports an extended reality (XR) experience that combines reality and AR content and implements augmented reality (AR) content on a transparent lens. , a device worn on the face by facility inspectors, and AR glasses to support visual inspection of facilities; It includes a mobile device owned by a facility inspector, which exchanges data by performing two-way data communication through wireless communication with the AR glasses, and the AR glasses are used for visual inspection of the facility through perspective and acquisition of real-time images. An AR interface module, which is a component that connects reality and AR content without a sense of heterogeneity, enables confirmation of damage information caused to facilities subject to inspection in the real world from acquired images; A drawing data output module that displays facility drawing data (BIM) by retrieving past damage information of the facility subject to inspection stored in AR glass or by retrieving past damage information of the facility subject to inspection stored from a mobile device; An augmented reality expansion module that expands by processing visualization data and enhancing the sense of reality regarding damage information caused to facilities confirmed through the AR interface module; Includes an overlay mapping processing module that automatically inputs the confirmed damage information of the inspection target facility through visualization data processing and enhanced realism through an augmented reality expansion module by overlay mapping it on the currently displayed facility drawing data (BIM); , The mobile device is characterized as being one type selected from a smartphone, a tablet PC, and a personal digital assistant (PDA).

Here, at least one or more of the mobile device or the AR glasses may be configured to include a condition evaluation engine module that automatically performs a condition evaluation on damage information occurring in the facility confirmed through the AR interface module. .

Here, the condition evaluation engine module reduces the time and cost required for automatic condition evaluation according to damage information of the facility and supports use in a web browser even without an Internet connection for improved facility maintenance and efficiency. Based on the application Progressive Web App (PWA), the algorithm can be processed and designed to automatically perform status evaluation on the damage information of the facility.

Here, the state evaluation engine module includes an interlocking design unit that interlocks with the AR interface module based on a progressive web app (PWA); A facility condition evaluation logic design department that classifies facility members based on a progressive web app (PWA), processes them into a table, and has an algorithm to evaluate the condition of facility damage information; A system unit for facility inspection that has a back-end server for facility inspection and a database (DB) for data storage based on a progressive web app (PWA); Facility electronic network diagram engine unit equipped with a CAD engine and DWG drawing import module for creating electronic network diagrams for facilities based on a progressive web app (PWA); It may be configured to include a Progressive Web App (PWA) design unit equipped with a Progressive Web App (PWA)-based UX/UI interface and a damage detection module to detect the degree of damage from damage information on the facility.

Here, the AR glasses are designed to improve the clarity of image quality by minimizing information loss in image data by removing impulse noise that impairs image quality from images of facilities in the real world acquired in real time through perspective. It further includes a filter module for removing image noise, wherein the filter module for removing image noise includes: a first filter module that removes impulse noise that causes sudden changes in image data of an image obtained from a facility that is subject to inspection; A second filter module for removing noise by smoothing the image data primarily filtered through the first filter module according to the angular direction, wherein the first filter module and the first filter module for image data acquired from the facility to be inspected. It may be configured to continuously process and control switching so that the second filter modules are alternately and repeatedly filtered.

Here, the AR glass is an optical-based see-through display device that includes an optical path guide-type optical system that supports images according to the viewing angle of the facility inspector, or a video-based see-through display that is equipped with a camera and supports images by shooting. It can be configured as a perspective glasses-type display device equipped with any one of the device or a fusion-based see-through display device that implements the optical-based see-through display and the video-based see-through display as an interaction unit.

When the AR glasses are equipped with a fusion-based transparent display device, the projection point for the actual point of the facility being inspected is calculated based on the viewing angle of the facility inspector wearing the AR glasses, and at the same time, based on the viewing angle of the facility inspector. By shooting and observing the actual point of the recognized facility with a camera, the error between the actual point of the facility recognized by the viewing angle and the projection point obtained by calculation is analyzed, and the actual point of the facility recognized by the viewing angle and the projection point obtained by calculation are analyzed. By minimizing the error between the displays, damage information on the display and the facility can be determined more accurately for the facility being inspected.

According to the present invention, by incorporating augmented reality technology when visually inspecting facilities including bridges, the visualization and sense of reality of the facility are increased, thereby increasing the effectiveness of facility inspection, diagnosis, and maintenance compared to existing visual inspection methods. The efficiency of visual inspection can be improved, errors due to visual inspection can be minimized compared to the existing method, and the usefulness of performing digitalization and advanced processing for damage information and condition evaluation of facilities can be achieved.

According to the present invention, damage information on facilities subject to inspection can be confirmed more precisely using an augmented reality (AR) interface, and the status evaluation of damage information on such facilities is automated through a progressive web app (PWA)-based design. By supporting facility maintenance work, time and cost can be reduced through automatic status assessment of facility damage information, accuracy and reliability can be improved, and the usefulness of implementing a new digital-based facility maintenance system can be achieved. .

According to the present invention, from a technical point of view, it is possible to secure the development technology of a new system using augmented reality (AR) technology and create technology-centered high added value, and to implement a new system by confirming damage to facilities subject to inspection, such as bridges, using the AR interface method. It can lead technology, promote long-term cost reduction due to maintenance of facilities from an economic and industrial perspective, and allow for digitalization and advancement, so the management entity can inspect and diagnose (performance evaluation) - maintenance reinforcement (maintenance) for each facility. Management/Performance Improvement) - Management History Record - Feedback, etc. can establish a cyclical structure in the safety industry, and from a social perspective, an environment in which the performance rating for each facility is maintained above "average" based on management status rather than elapsed years. In addition to creating a safety system, even small-scale facilities that were relatively weak in management can always secure safety performance, and synergy effects such as replacing aging workers can be achieved through the spread of advanced digitalized maintenance technology. You can.

Figure 1 is a schematic block diagram shown to explain a facility maintenance system using augmented reality technology according to an embodiment of the present invention.
Figure 2 is a conceptual diagram showing a facility maintenance system using augmented reality technology according to an embodiment of the present invention.
Figure 3 is a conceptual diagram showing a comparison between a facility maintenance system using augmented reality technology according to an embodiment of the present invention and a conventional facility maintenance system.
Figure 4 is an exemplary diagram showing an example design of a condition evaluation engine module in a facility maintenance system using augmented reality technology according to an embodiment of the present invention.
Figure 5 is an exemplary diagram showing an example design of a CAD engine in a facility maintenance system using augmented reality technology according to an embodiment of the present invention.

Preferred embodiments of the present invention will be described with reference to the drawings as follows. Through this detailed description and drawings, the purpose and configuration of the present invention and its characteristics will be better understood.

Hereinafter, a facility maintenance system using augmented reality technology according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 5.

1 to 5 are diagrams showing a facility maintenance system using augmented reality technology according to an embodiment of the present invention.

As shown in Figures 1 to 5, the facility maintenance system using augmented reality technology according to an embodiment of the present invention performs two-way data communication between AR glasses 100, which are perspective glasses-type display devices, and wireless communication between them. It consists of a mobile device 200 that includes:

The AR glasses 100 are provided as a see-through glasses-type display device that supports an extended reality (XR) experience that combines reality and AR content and implements augmented reality (AR) content on transparent lenses.

The AR glass 100 is a wearable device used by facility inspectors by wearing it on a part of the body, especially the face, and is a display device with augmented reality (AR) implementation technology to support visual inspection of facilities.

The AR glass 100 may be provided as an optical-based transparent display device including an optical path guide-type optical system that supports images according to the viewing angle of the facility inspector, or a video display device equipped with a camera to support images by shooting. It can be provided as a transparent display device.

Sometimes, the AR glass 100 may be equipped with a fusion-based see-through display device that implements the optical-based see-through display and the video-based see-through display as an interaction unit.

That is, the AR glass 100 may be a see-through glasses-type display device equipped with any one of an optical-based see-through display device, a video-based see-through display device, and a fusion-based see-through display device.

In detail, the AR glass 100 includes an AR interface module 110, a drawing data output module 120, and an augmented reality expansion module ( 130), and may include an overlay mapping processing module 140.

The AR interface module 110 is a component that seamlessly connects reality and AR content for visual inspection of facilities and acquisition of real-time images through visual inspection of facility inspectors using AR glasses, and provides information about the real world from the acquired images. It is a component that enables confirmation of damage information occurring in the facility subject to inspection.

The drawing data output module 120 retrieves past damage information of the facility to be inspected stored in the AR glass or retrieves past damage information of the facility to be inspected stored from the mobile device 200 owned by the facility inspector connected through the mutual interface. It is a component that displays facility drawing data (BIM).

The augmented reality expansion module 130 is a component that performs the role of processing visual data and enhancing the sense of reality for damage information occurring in the facility subject to inspection, which is confirmed through the AR interface module 110.

The overlay mapping processing module 140 processes visualization data through the augmented reality expansion module 130 and overlays and maps damage information confirmed from the inspection target facility with an enhanced sense of reality onto the currently displayed facility drawing data (BIM). It is a component that is automatically entered.

Through overlay mapping by the overlay mapping processing module 140, previous damage information and current damage information for the facility to be inspected can be more clearly distinguished and inspected.

At this time, through the overlay mapping processing module 140, overlay mapping is processed using differences in color or line thickness so that the current newly confirmed facility damage information, which is automatically input through overlay mapping, can be easily distinguished from previously displayed damage information. It is desirable to configure it to do so.

As such, in the present invention, augmented reality technology is applied to the visual inspection of facilities, including bridges, through the AR glass 100, so that visualization and realism of the facility can be increased, making it possible to inspect and improve the facility compared to existing visual inspection methods. The effectiveness of diagnosis and maintenance can be improved, and compared to visual inspection methods that rely only on the existing inspector's field of view, efficiency can be improved and frequent errors can be minimized.

The mobile device 200 is equipped to exchange data with the AR glass 100, which is a transparent display device, by performing two-way data communication through wireless communication, and is an electronic device owned by the facility inspector.

The mobile device 200 may be any type selected from a smartphone, tablet PC, or personal digital assistant (PDA), and may be equipped with an application to enable interconnection with the AR glass 100 through an interface.

The mobile device 200 includes a condition evaluation engine module 210 that automatically performs a condition evaluation on damage information occurring in the facility to be inspected, which is confirmed through the AR interface module 110 of the AR glass 100. .

The condition evaluation engine module for performing this function may be mounted on the AR glasses 100, and sometimes may be mounted on both the AR glasses 100 and the mobile device 200.

In the present invention, a system that utilizes the condition evaluation engine module by mounting it on the mobile device 200 would be a more preferable embodiment.

The condition evaluation engine module 210 reduces the time and cost required for automatic condition evaluation according to damage information of the facility to be inspected and supports use in a web browser even without an Internet connection for improved facility maintenance and efficiency. It is desirable to process and design an algorithm based on Progressive Web App (PWA), an application that automatically performs status evaluation on the damage information of the facility.

For this purpose, the condition evaluation engine module 210 is based on a progressive web app (PWA) and has an interlocking design unit 211 that interlocks with the AR interface module 110 and a facility based on a progressive web app (PWA). A facility condition evaluation logic design unit 212 with an algorithm that classifies members and processes them into a table and performs a condition evaluation on facility damage information, and a bag for facility inspection based on a progressive web app (PWA). -A facility inspection system unit 213 with a back-end server and a database for data storage, and a CAD for creating an electronic network diagram for the facility based on a progressive web app (PWA) The facility electronic network equipped with the engine 214a and the DWG drawing import module 214b is also damaged from the engine unit 214, the progressive web app (PWA)-based UX/UI interface 215a, and facility damage information. It can be configured to include a progressive web app (PWA) design unit 215 equipped with a damage detection module 215b to detect the degree.

In this way, by designing and utilizing the condition evaluation engine module 210 based on a progressive web app (PWA), it is possible to increase damage detection efficiency according to damage information of the facility to be inspected, while reducing the time and cost required for automatic condition evaluation. Compared to existing methods, it is possible to reduce the accuracy of damage detection and condition evaluation, and it provides the advantage of facilitating advanced processing such as making it easier to store data through digitization and using the obtained data as big data. can do.

That is, the condition evaluation engine module 210 detects the degree of damage based on the Progressive Web App (PWA), an application, for damage information on the facility to be inspected, which is implemented by expanding the AR glass 100 into augmented reality, and determines the current facility In addition to calculating a comprehensive condition score, the condition evaluation grade is automatically calculated and displayed, and a condition evaluation report according to the condition evaluation of the facility subject to inspection can be created through algorithmic processing based on a progressive web app (PWA). there is.

In addition, in the present invention, in the AR glass 100, information loss of image data is minimized by removing impulse noise that impairs image quality from images of facilities in the real world acquired in real time through visual inspection by a facility inspector. A filter module 150 for removing image noise may be further included to increase the clarity of image quality.

At this time, the filter module 150 for removing image noise includes a first filter module 151 that removes impulse noise that causes sudden changes from the image data of the image acquired from the facility being inspected, and the first filter module ( It may be configured to include a second filter module 152 that removes noise by smoothing the image data first filtered through 151) according to the angular direction.

In addition, it is desirable to have a circuit configuration that continuously switches and controls the first filter module 151 and the second filter module 152 to repeatedly filter image data acquired from the facility being inspected. .

Here, the first filter module may be a non-linear filter, and the second filter module may be a linear filter.

Furthermore, in the present invention, the AR glass 100 can have the following configuration depending on the implementation type of the transparent display device provided.

When the AR glasses 100 are equipped with a fusion-based transparent display device, the projection point for the actual point of the facility to be inspected is calculated based on the viewing angle of the facility inspector wearing the AR glasses, and at the same time, the facility inspector's The actual point of the facility recognized based on the viewing angle is photographed and observed with a camera, and the error between the actual point of the facility recognized by the viewing angle and the projection point obtained by calculation is analyzed, and the actual point of the facility recognized by the viewing angle is calculated. By minimizing the error between the projection points obtained by , it is possible to have processing and configuration to correct the display and damage information on the facility side for the facility subject to inspection more accurately.

That is, the calculation module 161 calculates the projection point for the actual point of the facility to be inspected based on the viewing angle of the facility inspector, and photographs and observes the actual point of the facility recognized based on the viewing angle of the facility inspector with a camera to determine the viewing angle. An error analysis module 162 that analyzes the error between the actual point of the facility recognized by the system and the projection point obtained by calculation, and minimizes the error between the actual point of the facility recognized by the viewing angle and the projection point obtained by calculation to determine the target of inspection. It may include an error correction module 163 that corrects the display and damage information on the facility to more accurately identify the facility.

At this time, the error correction module 163 minimizes the error between the actual point of the facility recognized by the viewing angle and the projection point obtained by calculation, and corrects the error range so that it is close to '0' or the facility recognized by the viewing angle It can be configured to perform correction to reduce the error between the actual point and the projected point obtained by calculation to '1/2'.

Accordingly, the facility maintenance system using augmented reality technology according to the present invention consisting of the above-described configuration can provide useful effects in technical, economic, industrial, and social aspects as follows.

First, in terms of technology, it is possible to secure new system development technology using augmented reality (AR) technology and create technology-centered high added value, and lead new implementation technology by confirming damage to facilities subject to inspection, such as bridges, using the AR interface method. It can provide useful effects.

Second, from an economic and industrial perspective, it is possible to reduce long-term costs due to maintenance of facilities, and digitalization and advancement are possible, so the management entity inspects and diagnoses each facility (performance evaluation) and reinforces maintenance (maintenance/performance improvement). - It can provide a useful effect in establishing a circular structure in the safety industry that leads to management history records and feedback.

Third, from a social perspective, it is possible to create an environment in which the performance rating of each facility is maintained above “average” based on management status rather than the number of years elapsed, as well as ensuring safety performance at all times even for small-scale facilities with relatively weak management. The spread of advanced digitalized maintenance technology can provide useful effects that can lead to synergy effects, such as replacing aging workers.

The above description is an illustrative description of the present invention, and the embodiments published in the specification are not intended to limit the technical idea of the present invention, but are for illustrative purposes, so those skilled in the art Various modifications, transformations, or substitution of steps will be possible without departing from the technical idea of . Therefore, the scope of protection of the present invention should be interpreted based on the matters stated in the claims, and technical matters within the equivalent scope thereof should also be interpreted as being included in the scope of rights of the present invention.

100: AR glass 110: AR interface module
120: Drawing data output module 130: Augmented reality expansion module
140: Overlay mapping processing module 150: Filter module for image noise removal
200: Mobile device 210: Condition evaluation engine module
211: Interconnection design department 212: Facility condition evaluation logic design department
213: Facility inspection system unit 214: Facility electronic network engine unit
215: Progressive Web App Design Department

Claims (7)

It is a perspective glasses-type display device that supports an extended reality (XR) experience that combines reality and AR content and implements augmented reality (AR) content on transparent lenses. It is a device worn by facility inspectors on the face and allows visual inspection of facilities. AR glasses to support; In the facility maintenance system using augmented reality technology, which includes a mobile device owned by a facility inspector, which exchanges data by performing two-way data communication through wireless communication with the AR glasses,
The AR glass is a component that seamlessly connects reality and AR content for visual inspection of facilities and acquisition of real-time images through perspective. It enables confirmation of damage information on facilities subject to inspection in the real world from the acquired images. AR interface module; A drawing data output module that displays facility drawing data (BIM) by retrieving past damage information of the facility subject to inspection stored in AR glass or by retrieving past damage information of the facility subject to inspection stored from a mobile device; An augmented reality expansion module that expands by processing visualization data and enhancing the sense of reality regarding damage information caused to facilities confirmed through the AR interface module; An overlay mapping processing module that automatically inputs the confirmed damage information of the inspection target facility through visualization data processing and enhanced realism through an augmented reality expansion module by overlay mapping it on the currently displayed facility drawing data (BIM); Including,
The mobile device is any type selected from a smartphone, tablet PC, or personal digital assistant (PDA),
The AR glass consists of an optical-based see-through display device that includes an optical path guide-type optical system that supports images according to the viewing angle of the facility inspector, and a video-based see-through display device that is equipped with a camera and supports images by shooting as an interactive unit. A fusion-based transparent display device that implements,
An operation module that calculates the projection point for the actual point of the facility to be inspected based on the viewing angle of the facility inspector, and a camera to photograph and observe the actual point of the facility recognized based on the viewing angle of the facility inspector to determine the projection point of the facility recognized by the viewing angle. An error analysis module that analyzes the error between the actual point and the projection point obtained by calculation, and minimizes the error between the actual point of the facility recognized by the viewing angle and the projection point obtained by calculation, causing damage to the display and facility side of the facility being inspected. Includes an error correction module that corrects information to understand it more accurately,
In the error correction module, the error between the actual point of the facility recognized by the viewing angle and the projected point obtained by calculation is minimized, but the error range is corrected to be close to '0' or the actual point of the facility recognized by the viewing angle is calculated. It is configured to perform correction to reduce the error between projection points obtained by to '1/2',
At least one or more of the mobile device or AR glasses includes a condition evaluation engine module that automatically performs a condition evaluation on damage information occurring in the facility confirmed through the AR interface module; Including,
The condition evaluation engine module reduces the time and cost required for automatic condition evaluation based on damage information of facilities and provides Progressive Web, an application that supports use in a web browser even without an Internet connection for improved facility maintenance and efficiency. The algorithm is processed and designed based on an app (PWA) to automatically perform status assessment on the corresponding damage information of the facility.
An interoperability design department that interoperates with the AR interface module based on a progressive web app (PWA); A facility condition evaluation logic design department that classifies facility members based on a progressive web app (PWA), processes them into a table, and has an algorithm to evaluate the condition of facility damage information; A system unit for facility inspection that has a back-end server for facility inspection and a database (DB) for data storage based on a progressive web app (PWA); Facility electronic network diagram engine unit equipped with a CAD engine and DWG drawing import module for creating electronic network diagrams for facilities based on a progressive web app (PWA); A Progressive Web App (PWA) design department equipped with a Progressive Web App (PWA)-based UX/UI interface and a damage detection module to detect the degree of damage from damage information on facilities; A facility maintenance system using augmented reality technology comprising: delete delete delete According to clause 1,
The AR glasses are,
A filter module for image noise removal to minimize information loss in image data and increase the clarity of image quality by removing impulse noise that impairs image quality from images of facilities in the real world acquired in real time through perspective; It further includes,
The filter module for removing image noise is,
A first filter module that removes impulse noise that causes sudden changes in image data of images obtained from facilities subject to inspection;
a second filter module that removes noise by smoothing the image data primarily filtered through the first filter module according to the angular direction; Including,
A facility maintenance system using augmented reality technology, characterized in that it continuously switches and controls image data acquired from a facility subject to inspection to be alternately and repeatedly filtered by the first filter module and the second filter module. delete delete