KR102669058B1 - SAFETY INSPECTION SYSTEM FOR STRUCTURE USING SMART PORTABLE DEVICE, AND METHOD FOR THe SAME - Google Patents

Abstract

By utilizing the LiDAR sensor and depth camera built into smart mobile devices, the status and location of safety inspection facilities can be monitored through augmented reality/virtual reality visualization through real-time linkage between smart mobile devices and safety inspection servers without physical assistance devices. Accordingly, even beginners or non-experts can easily access it, and by managing the time history of the facility based on quantitative evaluation data on the safety inspection facility, a systematic and reasonable maintenance strategy can be established. In addition, the type of deterioration/damage of the facility can be identified through image classification and analysis based on the input data, and the quantitative degree and amount of deterioration/damage of the facility can be identified by using the classified and analyzed data. A facility safety inspection system and method using smart mobile devices is provided, which can easily calculate the grade of inspection facilities and establish a reasonable maintenance strategy and present it to the management entity.

Description

Facility safety inspection system and method using smart portable devices {SAFETY INSPECTION SYSTEM FOR STRUCTURE USING SMART PORTABLE DEVICE, AND METHOD FOR THE SAME}

The present invention relates to a facility safety inspection system. More specifically, the present invention relates to an augmented reality (AR)/virtual safety inspection facility using various functions and sensors built into smart mobile devices (or portable smart devices) such as smartphones. This relates to a facility safety inspection system and method using smart mobile devices that visualizes reality (VR) and evaluates the status of safety inspection facilities.

Currently, the safety inspection market for facilities or structures is mostly conducted only through visual inspection by personnel, making quantitative evaluation difficult as the experience and capabilities of the inspector affect the rating of the facility.

For example, some of the existing visual inspection methods are carried out in a qualitative evaluation method rather than a quantitative evaluation method, such as calculating the deterioration and damage grade based on the inspector's judgment. Therefore, the experience and capabilities of the inspector determine the facility grade. It is currently affecting.

Specifically, in the case of Korea, numerous infrastructure structures are being built along with economic development, but the systemization and advancement of information management of infrastructure structures such as roads, bridges, tunnels, dams, and ports in public use are insufficient. , the field information input method of the infrastructure network was not implemented in a manager- or user-friendly environment using the latest information technology, and as a result, there was a problem that it was difficult to secure the reliability of the collected data.

The existing facility management system does not efficiently link and circulate information at the management site and the information in the management system, and because it is focused only on information input, its role in providing useful information is insufficient. In addition, because the information input method uses text or a two-dimensional plane, the ease of input is very low.

In addition, in order to access the facility management system, which stores, accumulates, utilizes and analyzes information on facilities such as bridges and provides useful information to managers, and retrieves information, and inputs information through field surveys into the facility management system, portable devices have recently been used. Attempts are being made to use terminals (for example, smartphones).

Meanwhile, in the current construction field, the quality of construction work is being improved by applying the BIM (Building Information Modeling) method to comprehensively manage information generated during the three-dimensional design of public facilities and their entire life cycle. However, most information management focuses on solving problems during planning, design and construction. In other words, the users and utilizers of BIM in the planning, design, and construction stages are limited to engineers with specialized skills, and the use of it is limited to those in charge of management entities in charge of maintenance of facilities that are finally completed and in use. There is.

Meanwhile, in the case of field investigations stipulated in the current detailed guidelines for safety and maintenance of facilities, cracks, whiteness, peeling, delamination, exposure of rebar, material separation, water leakage, etc. of concrete materials are evaluated, and in the case of steel materials, , paint deterioration, corrosion, cracking, rusting, etc. are evaluated through visual inspection.

When evaluating such deterioration and damage by area ratio, the defect and damage area is divided by the survey unit area and evaluated as a percentage multiplied by 100. Additionally, when evaluating structural cracks, the evaluation criteria are different depending on the direction and width of the crack. , when evaluating non-structural cracks, there is a qualitative grading system such as local cracks, general cracks, etc.

In addition, when calculating the overall grade of the safety inspection facility, the weight of each deterioration and damage or the level of serious defects can be adjusted at the discretion of the responsible engineer, and methods for the grade and repair and reinforcement of the safety inspection facility are presented. Should be.

Specifically, when evaluating deterioration and damage by area ratio for safety inspection of facilities, the deterioration and damage area must be calculated. The method of calculating this is a rough method using a tape measure at the site to quantitatively calculate the area and damage. It is difficult to evaluate, and there are limitations in calculating strategic repair and reinforcement quantities based on the concept of two-dimensional area rather than the concept of volume that considers depth.

In addition, when evaluating structural cracks, crack widths are measured and recorded separately by bringing a crack measuring device, but in places where access is difficult, such as the lower part of a bridge or the upper part of a pier, it may be difficult to actually measure cracks directly. In addition, when evaluating non-structural cracks, there is no quantitative calculation standard, so they are evaluated locally. Accordingly, there is a qualitative evaluation system such as local cracks, overall cracks, etc., and the experience and capabilities of the technician affect the rating of the facility.

In addition, when calculating the overall grade of a safety inspection facility, many matters are decided at the discretion of the responsible engineer, and at this time, maintenance strategies such as repair and reinforcement methods are suggested based on experience, which may result in unnecessary investment. In addition, since the assessed deterioration and damage and repair and reinforcement methods are written and recorded in a report, a large amount of documentation is required, and information transfer is weak, making it difficult to intuitively understand the facility, making it realistically difficult for beginners or non-experts to access the facility. There is a problem.

Meanwhile, as prior art, Republic of Korea Patent Publication No. 2013-101202 discloses an invention titled “Augmented reality-based safety diagnosis information visualization method and system for bridge structures,” which is explained with reference to FIG. 1.

Figure 1 is a configuration diagram showing an augmented reality-based safety diagnosis information visualization system for bridge structures according to conventional technology.

Referring to FIG. 1, the augmented reality-based safety diagnosis information visualization system for bridge structures according to the prior art displays information collected from measurement data and GPS location information related to the bridge structure in a mobile terminal 10 in three-dimensional augmented reality. It includes an augmented reality visualization device 20 that visualizes and displays.

The augmented reality visualization device 20 can monitor bridge structures in real time anytime, anywhere. Here, augmented reality (AR) refers to a mixed reality technology that combines the real world with a virtual world with additional information in real time and displays it as a single image. To implement such augmented reality (AR), an AR application 23 is installed in the mobile terminal 10.

This AR application 23 is operated based on the Android OS 21, but can also be operated on other operating systems such as iOS or web browsers. Accordingly, augmented reality using the AR application 23 can provide real-time safety diagnosis of parts that are not easy to access, such as bridge structures, and the bridge can be repaired and reinforced at an appropriate time through such real-time safety diagnosis.

Specifically, the Android OS 21-based augmented reality visualization device 20 is divided into an engine 22 and an application 23. Here, the engine 22 includes a GPS receiving module 22a that receives location information from a plurality of GPS satellites 32, a communication module 22c that receives measurement data from the server 40, and structural information and location of the bridge structure. It includes an AR module 22b that collects information and measurement data and visualizes it in three-dimensional augmented reality, and the server 40 also includes a DB 41 and a distribution server 42.

This AR module 22b includes a camera and a gyro sensor. And the application 23 includes a UI (user interface) module 23a that provides a user-oriented interface on the screen of the mobile terminal 10, and a measurement data reception module 23b that receives the location and measurement data of the measurement device 31 of the bridge structure. ) and a user location module 23c that provides location information of the user holding the mobile terminal 10.

In the case of the augmented reality-based safety diagnosis information visualization system for bridge structures according to conventional technology, multiple sensors are installed on the bridge structure, and the internal and external factors of the bridge structure are detected and measured and stored in the server DB. The user selects the AR application installed on the mobile terminal to set the basic data of the bridge, but uses the camera and GPS module of the mobile terminal to collect structural information, location information, and measurement data of the structure to create 3D augmented reality. By visualizing it, it is possible to provide safety diagnosis evaluation information about the detailed condition of the structure.

In the case of the augmented reality-based safety diagnosis information visualization system for bridge structures according to conventional technology, in addition to GPS information for efficient maintenance of bridge structures, additional information is provided in real time in the real world by linking with GPS information on smartphones or tablet PCs. Bridge structures can be diagnosed by accessing real-time sensor data and visualizing historical information through a location recognition algorithm that uses augmented reality to show the real environment in three dimensions by mixing the virtual world with a single image.

According to the augmented reality-based safety diagnosis information visualization system for bridge structures according to conventional technology, safety diagnosis of bridges is performed at an appropriate time through real-time monitoring using augmented reality-based sensor recognition and diagnostic information visualization for safety diagnosis of bridge structures. This can be done, and by using this safety diagnosis information visualization system, sensor information transmitted in real time to the control center 33 through wired or wireless communication can be monitored regardless of time and place.

In addition, sensors for safety diagnosis are installed at each major part or point of the bridge structure, enabling diagnosis for each part or point as well as the entire bridge. In conjunction with the GPS module, the location of the sensor is accurately identified to increase data accuracy. , Measurement information can be easily obtained anytime and anywhere through a wireless communication network, and the maintenance convenience of bridge structures can be improved by using 3D augmented reality technology to easily diagnose difficult-to-access parts of the bridge. .

However, in the case of the augmented reality-based safety diagnosis information visualization system for bridge structures according to the conventional technology, which is intended to visualize the safety diagnosis information of bridge structures based on augmented reality, the configuration and operation of the augmented reality visualization device 20 is complicated. However, there are limitations in performing specific safety diagnosis.

Meanwhile, as another prior art, Republic of Korea Patent No. 10-2195890 discloses an invention titled “Smart Glass Safety Diagnosis System and Safety Diagnosis Method,” which will be described with reference to FIGS. 2A and 2B.

Figure 2a is a simplified configuration diagram of a smart glass safety diagnosis system according to the prior art, and Figure 2b is a diagram illustrating the operation of the smart glass safety diagnosis system.

2A and 2B, the smart glass safety diagnosis system according to the prior art includes a smart glass 50, a portable terminal 60, a server 70, and a database 80. do.

The smart glasses 50 are equipped with a camera 51, an interface device capable of inputting commands (for example, a microphone), a wireless communication device, and an Augmented Reality (AR) function, and can perform see-through functions and computer functions. It is a device in the form of glasses that can be worn by the user.

The portable terminal 60 is a terminal that is portable and has communication and computer functions. A smartphone, a tablet PC, etc. can be used, and a screen input device that allows content to be written by touch on the liquid crystal screen of the portable terminal 60 This may be a portable terminal capable of this.

In addition, a digital drawing 91 of the structure being diagnosed may be constantly output on the portable terminal 60, and if there is damage or deterioration in the structure that has already been discovered and managed, a marker 92 has already been created and displayed. It may be a state.

The marker 92 may be a simple mark such as a cross indicating the point where damage or deterioration is located on the drawing 91, but it is preferable that the marker 92 is a QR code, and the marker 92 is used as a QR code. On the drawing 91, all information regarding damage and deterioration, such as the location of damage and deterioration within the structure, the shape and length of damage and deterioration, the time of discovery, and the estimated time of occurrence, can be included and include smart glass ( When the recognition unit 52 of 50) recognizes the QR code, information related to damage and deterioration can be displayed on the smart glass 50 in augmented reality. At this time, the smart glass 50 and the portable terminal 60 can be used in the field as devices that the inspector wears, carries, and brings with him when inspecting a structure.

On the other hand, the server 70 and the database 80 may be located in a remote central control center or management office.

The server 70 includes a communication unit 71, a processor 72, and a storage unit 73, and is preferably a server equipped with artificial intelligence functions, and includes smart glasses 50, a portable terminal 60, and a wireless network. Communication can be connected through, and the degree of growth is read for damage and deterioration transmitted in video format by the smart glass 50 and the portable terminal 60 at the structure site, and for newly discovered and detected signs of suspected damage or deterioration, It can be performed including the function of reading whether there is damage or deterioration, determining the risk of damage or deterioration, and deciding whether to notify the relevant agency.

The database 80 stores the drawing 91 of the structure and can also store a large amount of data on general damage that may occur in the structure, images of deterioration, detailed characteristics, etc., so that the server 70 can store damage, damage, and damage transmitted from the structure site. It can provide data used when looking at images of deterioration or damage or suspected signs of deterioration, find related drawings (91), or interpret damage or deterioration for newly detected damage or suspected signs of deterioration, and provide safety diagnosis of structures. Drawings of structures that are updated can be saved in version order.

The server 70 may be locally connected to the database 80 by wire or wirelessly, and may be connected to the smart glass 50 and the portable terminal 60 at the structure site through a wireless network.

According to the smart glass safety diagnosis system according to the conventional technology, when the safety diagnosis of a structure is performed in the field, when the smart glass recognizes the QR code of damage and deterioration, the information on the damage and deterioration can be output to the smart glass in augmented reality. By doing so, safety diagnosis of structures can be performed quickly, accurately, safely, and conveniently. However, in the case of the smart glass safety diagnosis system according to the prior art, there is a problem in that it is difficult to apply easily because smart glasses 50 in the form of separate glasses must be provided.

Republic of Korea Patent No. 10-1984778 (registration date: May 27, 2019), title of invention: “Multiple collaboration method for diagnosing the exterior wall of a facility and device for performing the same.” Republic of Korea Patent No. 10-2195890 (registration date: December 21, 2020), title of invention: “Smart glass safety diagnosis system and safety diagnosis method” Republic of Korea Patent No. 10-1772916 (registration date: August 24, 2017), title of invention: "Method and system for detecting cracks on tunnel lining surface using automated crack detection program based on captured images and artificial intelligence algorithm." Republic of Korea Patent No. 10-2192341 (registration date: December 11, 2020), title of invention: “Smartphone-based mobile filming type facility inspection device” Republic of Korea Patent No. 10-1486443 (registration date: January 20, 2015), title of invention: “Mobile field survey-based multidimensional object-oriented bridge information processing system and method” Republic of Korea Patent No. 10-2189111 (registration date: December 3, 2020), title of invention: “Underground facility detection system using machine learning, augmented reality and virtual reality” Republic of Korea Patent Publication No. 2013-101202 (publication date: September 13, 2013), title of invention: “Augmented reality-based safety diagnosis information visualization method and system for bridge structures”

The technical problem that the present invention aims to achieve in order to solve the above-mentioned problems is to utilize the lidar sensor and depth camera built into the smart mobile device, thereby enabling real-time linkage between the smart mobile device and the safety inspection server without a physical auxiliary device. The purpose is to provide a facility safety inspection system and method using smart mobile devices that can identify the status and location of safety inspection facilities through augmented reality/virtual reality visualization.

Another technical task to be achieved by the present invention is facility safety inspection using smart portable devices, which can establish a systematic and reasonable maintenance strategy by managing the facility time history based on quantitative evaluation data on safety inspection facilities. It is intended to provide a system and method.

Another technical task to be achieved by the present invention is to identify the type of deterioration/damage of facilities through image classification and analysis based on input data, and to quantify the deterioration/damage of facilities by utilizing the classified and analyzed data. To provide a facility safety inspection system and method using smart mobile devices that can determine the extent and quantity, calculate the grade of safety inspection facilities, and establish a reasonable maintenance strategy and present it to the management entity. It is for.

As a means of achieving the above-mentioned technical problem, the facility safety inspection system using a smart portable device according to the present invention is a facility safety inspection system using a smart portable device with a built-in lidar sensor, depth camera, and GPS module. It is equipped with a LiDAR sensor, a depth camera, and a GPS module, and uses the LiDAR sensor and the depth camera to scan the exterior of the safety inspection facility, register the location of the safety inspection facility based on GPS, and A smart mobile device that transmits exterior scan data and depth images of inspection facilities together with the GPS location data; Visualize the safety inspection facility in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility received from the smart mobile device, calculate the facility grade by member/material of the safety inspection facility, and record the facility time history. Safety inspection server that analyzes and manages and establishes facility maintenance strategies; A safety inspection DB configured to store data received and processed by the safety inspection server and to store and manage the obtained data by augmenting/virtualizing it; And an interlocking system that allows the safety inspection server to analyze and manage the facility time history, and includes an interlocking system of the safety inspection facility through augmented reality/virtual reality visualization according to real-time interconnection of the smart mobile device and the safety inspection server. It is characterized by identifying status and location.

Here, after inputting the specifications of the safety inspection facility, the smart mobile device scans the exterior of the safety inspection facility using the built-in LiDAR sensor and depth camera while maintaining a certain distance from the safety inspection facility and GPS -Based on this, the location of the safety inspection facility can be registered, and exterior scan data and depth images of the safety inspection facility can be transmitted along with the GPS location data.

Here, the linked system may be a facility integrated information management system or a building information management (BIM) system.

Here, the smart portable device includes a lidar sensor that irradiates the safety inspection facility and generates exterior scan data of the safety inspection facility; A depth camera that captures the safety inspection facility and generates a 3D depth image; A GPS module that generates GPS location data according to the location of the smart portable device and confirms and registers the location of the safety inspection facility corresponding to the location of the smart portable device; A data collection unit that collects exterior scan data of the safety inspection facility generated from the LiDAR sensor, 3D depth image captured from the depth camera, and GPS location data; a facility specifications input unit that inputs the specifications of the safety inspection facility provided from the safety inspection server; And it may include a communication module that collects the exterior scan data, 3D depth image, and GPS location data of the safety inspection facility collected from the data collection unit and transmits it to the safety inspection server.

Here, the safety inspection server includes a data receiving unit that receives lidar sensor scan data, depth camera shooting data, and GPS location data transmitted from the smart mobile device; An augmented reality (AR)/virtual reality (VR) visualization module that visualizes the safety inspection facility in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility received by the data receiving unit; A facility grade calculation module that calculates a facility grade by member/material of the safety inspection facility; a facility time history management unit that analyzes and manages facility time history in conjunction with the linkage system; And the safety inspection server may include a facility maintenance strategy establishment unit in which the facility maintenance strategy is established according to an image analysis algorithm and notified to the management entity terminal.

Here, the facility grade calculation module includes an image classification and analysis unit that classifies and analyzes the type of deterioration/damage of the safety inspection facility using an image classification algorithm; A deterioration/damage area rate calculation unit that calculates the deterioration/damage area rate according to LiDAR scan data; A crack width calculation unit that calculates the crack width through image analysis; A deterioration/damage location display unit that displays the deterioration/damage location according to GPS location data; And it may include a facility grade calculation unit that calculates the grade for each member/material of the safety inspection facility.

Meanwhile, as another means to achieve the above-mentioned technical problem, the facility safety inspection method using a smart portable device according to the present invention is a facility safety inspection method using a smart portable device equipped with a lidar sensor, a depth camera, and a GPS module. In the inspection method, a) inputting specifications for the safety inspection facility at the starting point of the safety inspection facility using a smart mobile device; b) collecting exterior scan data and GPS location data of the safety inspection facility while maintaining a certain distance from the starting point of the safety inspection facility; c) transmitting data collected from the smart mobile device to a safety inspection server; d) the safety inspection server visualizing the safety inspection facility in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility; e) the safety inspection server calculating a facility grade for each member/material of the safety inspection facility; f) the safety inspection server analyzing and managing the facility time history by linking with the interconnection system; and g) a step where the safety inspection server establishes a maintenance strategy for the facility according to an image analysis algorithm and notifies it to the management terminal, but augmented reality/virtual reality according to real-time linkage between the smart mobile device and the safety inspection server. It is characterized by identifying the status and location of the safety inspection facility through visualization.

Here, step e) includes: e-1) classifying and analyzing the type of deterioration/damage of the safety inspection facility using an image classification algorithm; e-2) calculating the deterioration/damage area rate according to LiDAR scan data; e-3) calculating the crack width through image analysis; e-4) displaying the location of deterioration/damage according to GPS location data; and e-5) may include calculating the grade for each member/material of the safety inspection facility.

According to the present invention, by using the lidar sensor and depth camera built into the smart mobile device, safety inspection facilities can be inspected through augmented reality/virtual reality visualization through real-time linkage between the smart mobile device and the safety inspection server without physical auxiliary devices. The status and location of the device can be identified, and thus, even beginners or non-experts can easily access it.

According to the present invention, a systematic and reasonable maintenance strategy can be established by managing the time history of the facility based on quantitative evaluation data for the safety inspection facility.

According to the present invention, the type of deterioration/damage of the facility can be identified through image classification and analysis based on the input data, and the quantitative degree and amount of deterioration/damage of the facility can be determined by using the classified and analyzed data. The grade of safety inspection facilities can be easily calculated, and a reasonable maintenance strategy can be established and presented to the management entity.

Figure 1 is a configuration diagram showing an augmented reality-based safety diagnosis information visualization system for bridge structures according to conventional technology.
Figure 2a is a simplified configuration diagram of a smart glass safety diagnosis system according to the prior art, and Figure 2b is a diagram illustrating the operation of the smart glass safety diagnosis system.
Figure 3 is a schematic configuration diagram of a facility safety inspection system using a smart portable device according to an embodiment of the present invention.
Figure 4 is a diagram showing the acquisition of scan data of the exterior of a safety inspection facility using a smart mobile device in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.
Figure 5 is a diagram for explaining the configuration and operating principle of a LiDAR sensor mounted on a smart mobile device in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.
Figure 6 is a diagram illustrating the schematic operation of a smart mobile device and a safety inspection server in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.
Figure 7 is a diagram for explaining the specific operation of the safety inspection server in the facility safety inspection system using smart mobile devices according to an embodiment of the present invention.
Figure 8 is a detailed configuration diagram of a smart mobile device in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.
Figure 9 is a detailed configuration diagram of a safety inspection server in a facility safety inspection system using smart mobile devices according to an embodiment of the present invention.
Figure 10 is an operation flowchart of a facility safety inspection method using a smart portable device according to an embodiment of the present invention.
FIG. 11 is an operation flowchart specifically illustrating the facility grade calculation process for each member/material shown in FIG. 10.

Below, with reference to the attached drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in many different forms and is not limited to the embodiments described herein. In order to clearly explain the present invention in the drawings, parts unrelated to the description are omitted, and similar parts are given similar reference numerals throughout the specification.

Throughout the specification, when a part is said to “include” a certain element, this means that it may further include other elements rather than excluding other elements, unless specifically stated to the contrary. Additionally, terms such as “… unit” used in the specification refer to a unit that processes at least one function or operation, and may be implemented as hardware, software, or a combination of hardware and software.

[Facility safety inspection system using smart mobile devices]

Figure 3 is a schematic configuration diagram of a facility safety inspection system using a smart portable device according to an embodiment of the present invention.

Referring to Figure 3, the facility safety inspection system utilizing a smart portable device according to an embodiment of the present invention is a facility safety inspection system utilizing a smart portable device 200 with a built-in lidar sensor, depth camera, and GPS module. It is comprised of a smart mobile device 200, a safety inspection server 300, a safety inspection DB 400, an interlocking system 500, a management entity terminal 600, and other equipment 700.

The smart mobile device 200 is equipped with a lidar sensor 210, a depth camera 220, and a GPS module 240, and generates exterior scan data for the safety inspection facility 100 and a depth image of the depth camera, Transmitted along with GPS location data.

Specifically, the smart portable device 200 includes a lidar sensor 210, a depth camera 220, and a GPS module 240, and the specifications of the safety inspection facility 100 are set to the smart portable device 200. After entering, the exterior is scanned using the on-board LIDAR sensor 210 and depth camera 220 while maintaining a certain distance from the safety inspection facility 100 to detect the safety inspection facility (100) based on GPS. 100) to register the location. Here, the lidar sensor 210 and the depth camera 220 are built into a smart portable device 200 according to recent technological developments, and a facility safety inspection system using a smart portable device according to an embodiment of the present invention. Scan data on the appearance of the safety inspection facility 100 is generated using the lidar sensor 210 and the depth camera 220 built into the smart portable device 200.

In other words, the facility safety inspection system using a smart portable device according to an embodiment of the present invention is equipped with a LiDAR sensor, a depth camera, a gyroscope sensor, etc. in accordance with recent technological developments, making it a smart device with various functions. The state of the safety inspection facility 100 can be quantitatively evaluated using the portable device 200. Here, the safety inspection facility 100 may be an infrastructure structure such as a public road, bridge, tunnel, dam, or port, but is not limited thereto.

The safety inspection server 300 visualizes the safety inspection facility 100 in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility 100 received from the smart mobile device 200, and displays the safety inspection facility 100 in augmented reality/virtual reality. (100) Calculate the facility grade by member/material, analyze and manage the facility time history, and establish a maintenance strategy for the facility.

Specifically, the safety inspection server 300 uses exterior scan data of the safety inspection facility 100 to visualize the safety inspection facility 100 in augmented reality or virtual reality, and uses an image classification algorithm to perform the safety inspection. After classifying the type of deterioration/damage of the facility 100, the damaged area rate is calculated using the classified image analysis data and LIDAR detection data, the crack width is calculated through image analysis, and the GPS-based damage location is displayed. After calculating the damage grade for each member/material of the safety inspection facility 100 using an evaluation algorithm using the analyzed data, the time history of the facility is recorded by linking with an interconnection system 500 such as an FMS or BIM system. , and establishes and presents a facility maintenance strategy using image analysis and machine learning.

The safety inspection DB 400 stores the data received and processed by the safety inspection server 300, and is a database implemented to store and manage the obtained data by augmenting/virtual reality, and also includes an image classification algorithm, An algorithm that can calculate grades by analyzing stored data, a machine learning algorithm, and an algorithm that can propose an appropriate maintenance strategy by comparing and analyzing the calculated grades over time are stored.

The interlocking system 500 is connected and linked wired or wirelessly with the safety inspection server 300 so that the safety inspection server 300 can analyze and manage the facility time history, for example, an integrated facility information management system. It may be, but is not limited to, a Facility Management System (FMS) or Building Information Management (BIM) system.

The management entity terminal 600 is provided with the facility maintenance strategy established and presented by the safety inspection server 300.

Additionally, other equipment 700 may be field test equipment that can be linked with the smart portable device 200. Here, the interconnectable field test equipment may be non-destructive testing equipment, etc., but is not limited thereto, and may vary depending on the type of the safety inspection facility 100, and the other equipment 700 may be a drone or unmanned equipment. Alternatively, the smart portable device 200 may be mounted on a drone or unmanned equipment so that the smart portable device 200 can be deployed in an inaccessible or dangerous place.

Meanwhile, Figure 4 is a diagram showing the acquisition of scan data of the exterior of a safety inspection facility using a smart mobile device in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.

In the case of a facility safety inspection system using a smart mobile device according to an embodiment of the present invention, as shown in FIG. 4, an inspector approaches a safety inspection facility, for example, a bridge, and performs the safety inspection on specifications, location, etc. Check basic data about the facility (100). At this time, if necessary, data and information related to the evaluation of the safety inspection facility 100 may be input and transmitted through additional field experiments using experimental equipment that can be linked with the smart portable device 200.

In addition, the exterior of the safety inspection facility 100 is scanned using the LIDAR sensor 210, depth camera 220, and gyroscope sensor 230 mounted on the smart mobile device 200 to obtain scan data. , In addition, the location of deterioration or damage of the safety inspection facility 100 is registered or displayed using the GPS module 240, etc.

Meanwhile, Figure 5 is a diagram for explaining the configuration and operating principle of a LiDAR sensor mounted on a smart mobile device in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.

Referring to FIG. 5, in the facility safety inspection system using a smart portable device according to an embodiment of the present invention, the LiDAR sensor 210 mounted on the smart portable device shines a laser on the safety inspection facility 100 as a target. As a result, the distance, direction, speed, temperature, material distribution and concentration characteristics, etc. to the safety inspection facility 100 can be detected.

This LiDAR sensor 210 is generally used for more precise observation of atmospheric physical properties and distance measurement by utilizing the advantages of a laser that can generate pulse signals with high energy density and short period. Specifically, these LiDAR sensors 210 include simple LiDAR sensors for measuring long-distance distances on the ground and cracking down on vehicle speed violations, and recently, laser scanners for 3D image restoration, and 3D sensors for future unmanned cars. As it is used as a core technology for image sensors, its utility and importance are gradually increasing.

The configuration of the LiDAR sensor is sometimes very complex depending on the application field, but the basic configuration is simply divided into a laser transmitter, a laser detector, a part for signal collection and processing, and data transmission, as shown in Figure 5. It can be. In addition, LiDAR sensors can be divided into time-of-flight (TOF) method and phase-shift method depending on the modulation method of the laser signal. The TOF method is capable of measuring distance by measuring the time when a laser emits a pulse signal and reflected pulse signals from objects within the measurement range arrive at the receiver. Additionally, the phase-shift method calculates time and distance by emitting a laser beam that is continuously modulated with a specific frequency and measuring the amount of phase change in the signal reflected back from an object within the measurement range.

Meanwhile, Figure 6 is a diagram illustrating the schematic operation of a smart mobile device and a safety inspection server in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.

In the case of a facility safety inspection system using a smart mobile device according to an embodiment of the present invention, as shown in FIG. 6, the safety inspection server 300 uses the grade for each member to inspect the entire safety inspection facility 100. The rating results can be extracted, and the entire facility rating for each member can be expressed in a specified format on the augmented reality/virtual reality visualization screen.

In addition, the safety inspection server 300 registers all data in conjunction with the FMS or BIM system and the Garten linkage system 500, and can determine whether progressive/non-progressive repair and reinforcement are performed through time history management and image analysis. .

In addition, the safety inspection server 300 uses the grade information, deterioration/damage information, image analysis information, and time history information of the safety inspection facility 100 to establish a systematic and reasonable facility maintenance strategy and propose it to the management entity. I do it.

Meanwhile, Figure 7 is a diagram for explaining the specific operation of the safety inspection server in the facility safety inspection system using smart mobile devices according to an embodiment of the present invention.

In the facility safety inspection system using a smart mobile device according to an embodiment of the present invention, as shown in FIG. 7, the safety inspection server 300 uses the exterior scan data of the safety inspection facility 100 to check the safety. The inspection facility 100 is made into augmented/virtual reality, and the location of deterioration/damage of the safety inspection facility 100 is expressed according to the designated symbol and format.

In addition, the safety inspection server 300 analyzes scan data obtained through an exterior scan of the safety inspection facility 100 to classify and analyze the type of deterioration/damage, and through this analysis, grades the facility. You can extract the information you need. Thereafter, the safety inspection facility 100 can be graded by member or material through the information extracted for grading the facility and auxiliary information obtained through additional field experiments.

Meanwhile, Figure 8 is a detailed configuration diagram of a smart mobile device in a facility safety inspection system using a smart mobile device according to an embodiment of the present invention.

Referring to FIG. 8, in the facility safety inspection system using smart mobile devices according to an embodiment of the present invention, the smart mobile device 200 includes a LiDAR sensor 210 and a depth camera 220. , a gyroscope sensor 230, a GPS module 240, a data collection unit 250, a control unit 260, a facility data input unit 270, a display 280, and a communication module 290.

The LiDAR (Light Detection And Ranging: LiDAR) sensor 210 irradiates the safety inspection facility 100 and generates exterior scan data of the safety inspection facility 100.

The depth camera (220) captures the safety inspection facility 100 and generates a 3D depth image. Specifically, this depth camera 220 is a ToF (Time Of Flight Camera) type camera, and can generate a 3D depth image by photographing the safety inspection facility 100, which is an object, according to a predetermined exposure time (Integration Time). You can.

The gyroscope sensor 230 generates acceleration data in response to the movement of the smart mobile device 200.

The GPS module 240 generates GPS location data according to the location of the smart portable device 200, and confirms and registers the location of the safety inspection facility 100 corresponding to the location of the smart portable device 200. You can.

The data collection unit 250 collects exterior scan data of the safety inspection facility 100 generated from the LiDAR sensor 210, a 3D depth image captured from the depth camera 220, and GPS location data.

The control unit 260 controls the operation of the LiDAR sensor 210 that scans the exterior of the safety inspection facility 100, the shooting operation of the depth camera 220, and the operation of the GPS module 240.

The facility specifications input unit 270 inputs the specifications of the safety inspection facility 100 provided from the safety inspection server 300 through the display 280 screen.

The display 280 displays exterior scan data of the safety inspection facility 100 generated from the LiDAR sensor 210 and a 3D depth image captured from the depth camera 220.

The communication module 290 collects the exterior scan data, 3D depth image, and GPS location data of the safety inspection facility 100 collected from the data collection unit 250 and transmits it to the safety inspection server 300.

Accordingly, the smart mobile device 200 can generate exterior scan data for the safety inspection facility 100 and a depth image from a depth camera, and transmit them to the safety inspection server 300 along with GPS location data.

Meanwhile, Figure 9 is a detailed configuration diagram of a safety inspection server in a facility safety inspection system using smart mobile devices according to an embodiment of the present invention.

Referring to FIG. 9, in the facility safety inspection system using smart mobile devices according to an embodiment of the present invention, the safety inspection server 300 includes a data reception unit 310 and an augmented reality (AR)/virtual reality (VR) visualization module. (320), a facility grade calculation module (330), a facility time history management unit (340), and a facility maintenance strategy establishment unit (350), and the facility grade calculation module (330) includes an image classification and analysis unit (331). , it may include a deterioration/damage area rate calculation unit 332, a crack width calculation unit 333, a deterioration/damage location display unit 334, and a facility grade calculation unit 335.

The data receiver 310 receives lidar sensor scan data, depth camera shooting data, and GPS location data transmitted from the smart mobile device 200.

The augmented reality (AR)/virtual reality (VR) visualization module 320 displays the safety inspection facility 100 in augmented reality/virtual reality (VR) according to the exterior scan data of the safety inspection facility 100 received by the data receiver 310. Visualize virtual reality.

The facility grade calculation module 330 calculates the facility grade for each member/material of the safety inspection facility 100. Specifically, the image classification and analysis unit 331 of the facility grade calculation module 330 uses an image classification algorithm to classify and analyze the type of deterioration/damage of the safety inspection facility 100. In addition, the deterioration/damage area rate calculation unit 332 of the facility grade calculation module 330 calculates the deterioration/damage area rate according to the LIDAR scan data. In addition, the crack width calculation unit 333 of the facility grade calculation module 330 calculates the crack width through image analysis. Additionally, the deterioration/damage location display unit 334 of the facility grade calculation module 330 displays the deterioration/damage location according to GPS location data. Accordingly, the facility grade calculation unit 335 of the facility grade calculation module 330 calculates the grade for each member/material of the safety inspection facility 100.

The facility time history management unit 340 analyzes and manages the facility time history in conjunction with the interconnection system 500.

The facility maintenance strategy establishment unit 350 notifies the safety inspection server 300 of establishing a facility maintenance strategy according to an image analysis machine learning algorithm to the management entity terminal 600. At this time, it is obvious to those skilled in the art that the image analysis machine learning can be used to establish a maintenance strategy for the facility, so detailed description will be omitted.

In the end, the facility safety inspection system using smart portable devices according to an embodiment of the present invention provides augmented reality/virtual reality visualization through real-time linkage of the safety inspection server 300 and smart portable devices 200 without physical auxiliary devices. Because the status and location of the safety inspection facility 100 can be determined through this, even beginners or non-experts can easily access it, and by managing the facility time history based on quantitative evaluation data on the safety inspection facility, it is a systematic and A reasonable maintenance strategy can be established.

In addition, the facility safety inspection system using a smart mobile device according to an embodiment of the present invention identifies the type of deterioration/damage of the facility through image classification and analysis based on the input data, and also identifies the type of deterioration/damage of the facility and the classified and analyzed data. By using , the quantitative degree and quantity of deterioration/damage of facilities can be identified, the grade of safety inspection facilities (100) can be calculated, and a reasonable maintenance strategy can be established and presented to the management entity.

[Facility safety inspection method using smart mobile devices]

FIG. 10 is an operation flowchart of a facility safety inspection method using a smart portable device according to an embodiment of the present invention, and FIG. 11 is an operation flowchart specifically showing the facility grade calculation process for each member/material shown in FIG. 10.

10 and 11, the facility safety inspection method using a smart portable device according to an embodiment of the present invention first arrives at the starting point of the safety inspection facility 100 and uses a smart portable device of the facility safety inspection system ( 200) is driven, and the specifications of the safety inspection facility 100 are input into the driven smart portable device 200 (S110).

Next, after registering the starting point of the safety inspection facility 100, the lidar sensor 210, depth camera 220, etc. of the smart portable device 200 while maintaining a certain distance from the safety inspection facility 100. The exterior of the safety inspection facility 100 is scanned, the location is registered through a GPS-based triangulation technique through the GPS module 240 of the smart mobile device 200, and data is collected (S120) ).

Next, the data collected from the smart mobile device 200 is transmitted to the safety inspection server 300 (S130). Specifically, after inputting the specifications of the safety inspection facility 100, the smart portable device 200 detects the built-in LiDAR sensor 210 and depth while maintaining a certain distance from the safety inspection facility 100. The exterior of the safety inspection facility 100 is scanned using the camera 220 to register the location of the safety inspection facility 100 based on GPS, and the exterior scan data and depth image for the safety inspection facility 100 are recorded. is transmitted together with the GPS location data.

Next, the augmented reality/virtual reality visualization module 320 of the safety inspection server 300 uses the exterior scan data of the safety inspection facility 100 to display the safety inspection facility 100 in augmented reality (AR)/ Visualize virtual reality (VR) (S140). Here, the safety inspection server 300 includes a data reception unit 310, an augmented reality/virtual reality visualization module 320, a facility grade calculation module 330, a facility time history management unit 340, and a facility maintenance strategy establishment unit ( 350).

Next, the facility grade calculation module 330 of the safety inspection server 300 calculates the facility grade by member/material of the safety inspection facility 100 using the collected data and evaluation algorithm (S150) . Specifically, the facility grade calculation module 330 includes an image classification and analysis unit 331, a deterioration/damage area rate calculation unit 332, a crack width calculation unit 333, a deterioration/damage location display unit 334, and a facility grade. Includes calculation unit 335.

Specifically, the image classification and analysis unit 331 of the facility grade calculation module 330 classifies and analyzes the type of deterioration/damage using an image classification algorithm (S151). Thereafter, the deterioration/damage area rate calculation unit 332 of the facility grade calculation module 330 calculates the deterioration/damage area rate using LIDAR scan data (S152). Thereafter, the crack width calculation unit 333 of the facility grade calculation module 330 calculates the crack width through image analysis (S153). Thereafter, the deterioration/damage location display unit 334 of the facility grade calculation module 330 displays the deterioration/damage location using GPS-based triangulation (S154). Thereafter, the facility grade calculation unit 335 of the facility grade calculation module 330 uses the analyzed data and evaluation algorithm to calculate the evaluation grade for each member/material of the safety inspection facility 100 (S155). .

Next, the facility time history management unit 340 of the safety inspection server 300 responds to changes over time in the safety inspection facility 100 by linking with the interconnection system, for example, FMS or BIM system. Analyze and manage history (S160).

Next, the facility maintenance strategy establishment unit 350 of the safety inspection server 300 establishes a facility maintenance strategy using an image analysis machine learning algorithm and presents it to the management terminal 600 (S170).

Ultimately, according to the embodiment of the present invention, the visual inspection of facilities conducted under the “Special Act on Safety and Maintenance of Facilities” can be conducted more systematically and quantitatively. In addition, by utilizing various functions and sensors added to smart mobile devices, we promote more systematic maintenance of facilities through quantitative and clear safety inspections, and provide reasonable maintenance strategies to management entities to lower and utilize the life cycle costs of facilities. Performance can be maximized. In addition, it enables quantitative and systematic evaluation in the evaluation of facilities, and allows smart portable devices to be mounted on drones or unmanned equipment and then deployed in difficult-to-access or dangerous places, making it easier for management entities to conduct safety inspections and access them. You can. In addition, quantitative evaluation of safety inspection facilities becomes possible and the cost for safety inspection can be reduced. Using this, various types of businesses are possible, such as technical training for safety inspection-related companies and development of customized facilities systems through cooperation with management entities such as the Korea Land and Safety Management Agency and the Korea Expressway Corporation.

The description of the present invention described above is for illustrative purposes, and those skilled in the art will understand that the present invention can be easily modified into other specific forms without changing the technical idea or essential features of the present invention. will be. Therefore, the embodiments described above should be understood in all respects as illustrative and not restrictive. For example, each component described as single may be implemented in a distributed manner, and similarly, components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims described below rather than the detailed description above, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

100: Safety inspection facility 200: Smart mobile device
300: Safety inspection server 400: Safety inspection DB
500: Linkage system 600: Management entity terminal
700: Other equipment
210: LiDAR sensor 220: Depth camera
230: Gyroscope sensor 240: GPS module
250: data collection unit 260: control unit
270: Facility specifications input unit 280: Display
290: Communication module 310: Data receiving unit
320: Augmented reality (AR)/virtual reality (VR) visualization module
330: Facility grade calculation module 340: Facility time history management department
350: Facility maintenance strategy establishment department 331: Image classification and analysis department
332: Deterioration/damage area rate calculation unit 333: Crack width calculation unit
334: Deterioration/damage location display unit 335: Facility grade calculation unit

Claims (12)

In the facility safety inspection system using a smart mobile device 200 with a built-in lidar sensor, depth camera, and GPS module,
A LIDAR sensor 210 that is irradiated to the safety inspection facility 100 to generate exterior scan data of the safety inspection facility 100, and a LIDAR sensor 210 that captures the safety inspection facility 100 to generate a 3D depth image. Generates GPS location data according to the location of the depth camera 220 and the smart mobile device 200, and confirms the location of the safety inspection facility 100 corresponding to the location of the smart mobile device 200. It is equipped with a GPS module 240 that registers, scans the exterior of the safety inspection facility 100 using the lidar sensor 210 and the depth camera 220, and uses the GPS-based safety inspection facility 100 A smart mobile device (200) that registers the location of and transmits exterior scan data and depth images for the safety inspection facility (100) together with the GPS location data;
The safety inspection facility 100 is visualized in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility 100 received from the smart mobile device 200, and each member/member of the safety inspection facility 100 is visualized in augmented reality/virtual reality. A safety inspection server 300 that calculates facility grades by material, analyzes and manages facility time history, and establishes facility maintenance strategies;
A safety inspection DB (400) configured to store data received and processed by the safety inspection server (300) and to store and manage the obtained data by augmenting/virtualizing it; and
The safety inspection server 300 includes an interlocking system 500 that is linked to analyze and manage facility time history,
The lidar sensor 210,
A laser transmitter that emits a laser toward the safety inspection facility (100);
A laser detection unit that detects reflected pulse signals in which the laser emitted from the laser transmitter is reflected and returned to the safety inspection facility 100; and
It includes a signal collection and processing unit that stores the data detected from the laser detection unit,
The smart mobile device 200,
A gyroscope sensor 230 that generates acceleration data in response to the movement of the smart mobile device 200;
Collecting exterior scan data of the safety inspection facility 100 generated from the LiDAR sensor 210, a 3D depth image captured from the depth camera 220, and GPS location data of the GPS module 240. Data collection unit 250;
A control unit 260 that controls the lidar sensor 210, depth camera 220, and GPS module 240;
A display 280 that displays exterior scan data of the safety inspection facility 100 generated from the LiDAR sensor 210 and a 3D depth image captured by the depth camera 220;
a facility specifications input unit 270 that allows input of the specifications of the safety inspection facility 100 provided from the safety inspection server 300 through the display 280 screen; and
A communication module 290 that collects the exterior scan data, 3D depth image, and GPS location data of the safety inspection facility 100 collected from the data collection unit 250 and transmits them to the safety inspection server 300. Contains more,
The safety inspection server 300,
A data receiver 310 that receives lidar sensor scan data, depth camera shooting data, and GPS location data transmitted from the smart portable device 200;
An augmented reality (AR)/virtual reality (VR) visualization module that visualizes the safety inspection facility 100 in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility 100 received by the data receiver 310. (320);
A facility grade calculation module 330 that calculates a facility grade by member/material of the safety inspection facility 100;
a facility time history management unit 340 that analyzes and manages facility time history in conjunction with the interconnection system 500; and
The safety inspection server 300 includes a facility maintenance strategy establishment unit 350 that establishes a facility maintenance strategy according to an image analysis machine learning algorithm and notifies it to the management entity terminal 600,
The facility grade calculation module 330,
An image classification and analysis unit 331 that classifies and analyzes the type of deterioration/damage of the safety inspection facility 100 using an image classification algorithm;
A deterioration/damage area rate calculation unit 332 that calculates a deterioration/damage area rate according to LiDAR scan data;
A crack width calculation unit 333 that calculates the crack width through image analysis;
A deterioration/damage location display unit 334 that displays the deterioration/damage location according to GPS location data; and
It includes a facility grade calculation unit 335 that calculates the grade by member/material of the safety inspection facility,
It further includes other equipment (700) consisting of a non-destructive inspection equipment that can be linked with the smart portable device (200) or a drone on which the smart portable device (200) is mounted,
A smart portable device characterized in that the status and location of the safety inspection facility 100 can be determined through augmented reality/virtual reality visualization through real-time linkage between the smart portable device 200 and the safety inspection server 300. Utilized facility safety inspection system. According to paragraph 1,
The interlocking system 500 is a facility safety inspection system using a smart mobile device, characterized in that it is a facility management system (FMS) or a building information management (BIM) system. Using the facility safety inspection system using smart mobile devices in Paragraph 1,
a) inputting specifications for the safety inspection facility (100) at the starting point of the safety inspection facility (100) using a smart mobile device (200);
b) collecting exterior scan data and GPS location data of the safety inspection facility 100 while maintaining a certain distance from the starting point of the safety inspection facility 100;
c) transmitting data collected from the smart mobile device 200 to the safety inspection server 300;
d) the safety inspection server 300 visualizing the safety inspection facility 100 in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility 100;
e) the safety inspection server 300 calculating a facility grade for each member/material of the safety inspection facility 100;
f) the safety inspection server 300 analyzes and manages the facility time history in conjunction with the interlocking system 500; and
g) The safety inspection server 300 establishes a facility maintenance strategy according to an image analysis algorithm and notifies the management terminal 600,
Utilizing a smart mobile device characterized in that the status and location of the safety inspection facility 100 are identified through augmented reality/virtual reality visualization through real-time linkage between the smart mobile device 200 and the safety inspection server 300. How to inspect the safety of a facility. The method of claim 3, wherein step e) is:
e-1) Classifying and analyzing the type of deterioration/damage of the safety inspection facility 100 using an image classification algorithm;
e-2) calculating the deterioration/damage area rate according to LiDAR scan data;
e-3) calculating the crack width through image analysis;
e-4) displaying the location of deterioration/damage according to GPS location data; and
e-5) A facility safety inspection method using a smart mobile device including the step of calculating the grade for each member/material of the safety inspection facility 100. According to paragraph 3,
In step c), the smart portable device 200 inputs the specifications of the safety inspection facility 100 and then maintains a certain distance from the safety inspection facility 100 by using the on-board LiDAR sensor 210 and The exterior of the safety inspection facility 100 is scanned using the depth camera 220 to register the location of the safety inspection facility 100 based on GPS, and the exterior scan data and depth for the safety inspection facility 100 are recorded. A facility safety inspection method using a smart mobile device, characterized by transmitting video together with the GPS location data. The method of claim 3, wherein the smart portable device 200 is:
A lidar sensor 210 that is irradiated to the safety inspection facility 100 and generates exterior scan data of the safety inspection facility 100;
A depth camera 220 that captures the safety inspection facility 100 and generates a 3D depth image;
A GPS module 240 that generates GPS location data according to the location of the smart portable device 200 and confirms and registers the location of the safety inspection facility 100 corresponding to the location of the smart portable device 200;
A data collection unit 250 that collects the exterior scan data of the safety inspection facility 100 generated from the LiDAR sensor 210, the 3D depth image and GPS location data captured by the depth camera 220;
a facility specifications input unit 270 that inputs the specifications of the safety inspection facility 100 provided from the safety inspection server 300; and
Includes a communication module 290 that collects the exterior scan data, 3D depth image, and GPS location data of the safety inspection facility 100 collected from the data collection unit 250 and transmits them to the safety inspection server 300. Facility safety inspection method using smart mobile devices. The method of claim 3, wherein the safety inspection server 300,
A data receiver 310 that receives lidar sensor scan data, depth camera shooting data, and GPS location data transmitted from the smart portable device 200;
An augmented reality (AR)/virtual reality (VR) visualization module that visualizes the safety inspection facility 100 in augmented reality/virtual reality according to the exterior scan data of the safety inspection facility 100 received by the data receiver 310. (320);
A facility grade calculation module 330 that calculates a facility grade for each member/material of the safety inspection facility 100;
a facility time history management unit 340 that analyzes and manages facility time history in conjunction with the interconnection system 500; and
Facility safety using smart mobile devices, including a facility maintenance strategy establishment unit 350 in which the safety inspection server 300 establishes a facility maintenance strategy according to an image analysis algorithm and notifies it to the management terminal 600. How to check. The method of claim 7, wherein the facility grade calculation module 330,
An image classification and analysis unit 331 that classifies and analyzes the type of deterioration/damage of the safety inspection facility 100 using an image classification algorithm;
A deterioration/damage area rate calculation unit 332 that calculates a deterioration/damage area rate according to LiDAR scan data;
A crack width calculation unit 333 that calculates the crack width through image analysis;
A deterioration/damage location display unit 334 that displays the deterioration/damage location according to GPS location data; and
A facility safety inspection method using a smart mobile device that includes a facility grade calculation unit 335 that calculates the grade by member/material of the safety inspection facility. delete delete delete delete