KR102603964B1 - artificial intelligence drone an unmanned station system for automatic check of dam watershed and water resource facility - Google Patents

Abstract

The present invention relates to an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities. An unmanned station (100) built around a dam, takes off and lands at the upper landing site of the unmanned station, and flies around the dam and water resources facilities. A drone (200) for constant monitoring, an unmanned boat (300) that detaches from the lower detachment site of the unmanned station and constantly monitors the water area around dams and water resources facilities, and a control server that controls the flight of the drone and navigation of the unmanned boat ( 400), it constantly monitors the aerial area and water area around dams and water resources facilities, demonstrates virtual 3D digital twin virtual environment dam modeling, and detects accidents that may occur in real dams and water resources facilities. We aim to provide an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities that can be managed in a way that prevents situational problems in advance.

Description

Artificial intelligence drone an unmanned station system for automatic check of dam watershed and water resource facility}

The present invention relates to an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities. Specifically, the unmanned station system required for the use of drones and unmanned boats capable of unmanned autonomous driving and the docking of these drones and unmanned boats. In addition, by securing a configuration that includes a control server reflecting a 3D digital twin, management is possible for constant monitoring of dams and water resources facilities, as well as problems such as cracks, defects, cracks, and other problems that may occur in dams and water resources facilities. This is an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities that can predict, prevent, and manage water damage in advance.

In general, the management of existing dam basins and inspection of the condition of facilities are conducted through direct investigation by employees working in dam facility organizations, so detailed and accurate inspections are required for areas of dam basins or facilities that are difficult to access by manpower. It is not being done.

Of course, recently, due to the activation of the use of drones, it is possible to monitor the area around the dam basin using drones, and to inspect structures around the dam by acquiring images collected from drones. However, a professional pilot who specializes in operating drones is also possible. In the absence or absence of a drone, it is difficult to operate a drone, and especially in unexpected weather conditions such as rain or strong winds, the drone flight itself is impossible. Not only is it not easy to manage the dam basin or check the condition of the facility, but it also requires an emergency. And detailed and accurate investigation work is inevitably delayed.

In addition, the manual control of a drone has a large variation depending on each individual's control ability, and the inconvenience of having to move to the shooting and recording point has been pointed out. In particular, when the drone is in automatic flight mode, only vertical shooting of the terrain based on a 2D image map is possible. In addition, automatic flight or close-up oblique photography of drones, which are required for precise condition inspection of dam slopes or the elevations and sides of facility structures, is even more impossible.

Patent Document 001: Patent Application No. 10-2016-0158980

The present invention to solve the above-mentioned problems requires urgent, detailed and accurate inspection and investigation of dam basin areas and dam facility areas that are difficult to access even by manpower, or management of the dam basin and inspection of the condition of the dam facility structure. Using drones and unmanned boats, pre-produced 3D design twin dam modeling data and images collected through autonomous unmanned flight and navigation of drones and unmanned boats are used to efficiently manage and crack dams and water resources facilities. The purpose is to provide an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities that can be managed in a way to prevent unexpected problems such as defects, cracks, and others.

The present invention for achieving the above-described objectives includes an unmanned station (100) built around a dam, a drone (200) that takes off and lands at the upper takeoff and landing site of the unmanned station and constantly monitors the area around the dam and water resources facilities, and the unmanned station's An unmanned boat 300 that detaches from the lower detachment site and constantly monitors the water area around the dam and water resources facilities, and a control server 400 that controls the flight of the drone and the navigation of the unmanned boat, and a constant automatic operation of the dam basin and water resources facilities. One example of this feature is the artificial intelligence drone unmanned station system for inspection.

The monitoring unit 230 installed on the drone 200 is configured in the form of a multi-camera 231, and the multi-camera 231 includes a thermal imaging sensor 232, a multi-spectral sensor 233, and an air quality measurement sensor ( 234), and a LiDAR sensor 235, an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities is an example of this feature.

The monitoring unit 330 installed on the unmanned boat 300 is configured in the form of a multi-camera 331, and the multi-camera 331 includes a thermal imaging sensor 332, a multi-spectral sensor 333, and a water quality measurement sensor ( 334), an example of this is the artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities, which is composed of a lidar sensor (335).

A 3D digital image production tool 410 reflecting the digital twin is installed in the control server 400, and the 3D digital image production tool 410 is a 3D digital image production tool for producing 3D digital twin virtual environment dam modeling. An example of the characteristics of an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities, which includes a program 411 and a simulation program 412 capable of demonstrating dam modeling in a dimensional digital twin virtual environment. There is.

The control server 400 has an example feature of an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities, which has been strengthened through 'two-way real-time question-leading-response' of the artificial intelligence approval department.

As discussed above, according to the present invention, in the management of areas around dams and water resources facilities, drones and unmanned boats equipped with artificial intelligence are used for constant monitoring regardless of day or night, and drones and unmanned boats capable of autonomous driving are controlled. By demonstrating dam modeling in a virtual 3D digital twin virtual environment through a 3D digital image production tool that reflects the digital twin technology installed on the control server, the surrounding cracks, defects, and defects that may occur in actual dams and water resources facilities in the real world are demonstrated. It has the effect of predicting and preventing cracks and other problems in advance.

In addition, according to the present invention, through the use of drones and unmanned boats capable of autonomous driving and the use of a 3D digital image production tool reflecting digital twin technology installed in the control server, malfunction or replacement timing of building parts comprised in dams and water resources facilities are monitored, etc. By predicting, it has the effect of increasing the production efficiency of management of dams and water resources facilities and dramatically reducing both cost and time.

Figure 1 is a diagram illustrating in simple blocks the configuration of an artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating the configuration of the drone shown in FIG. 1 in simple blocks.
FIG. 3 is a diagram illustrating the configuration of the unmanned boat shown in FIG. 1 in simple blocks.
FIG. 4 is a diagram illustrating the configuration of the control server shown in FIG. 1 in simple blocks.

The present invention should be interpreted in a way that includes the scope of rights reaching the technical idea through various modified embodiments, and the present embodiments are merely intended to ensure that the disclosure of the present invention is complete and to provide common knowledge in the technical field to which the present invention pertains. It is provided to fully inform those who have the scope of the invention, and the present invention is only defined by the scope of the claims.

Moreover, since the drawings attached to the present invention are attached as a way to help understand the description of the present invention, the technical idea of the present invention should not be interpreted as limited by the attached drawings.

As shown in FIG. 1, the artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities according to an embodiment of the present invention includes an unmanned station 100 and a drone 200 exploited on top of the unmanned station. , an unmanned boat 300 mounted on the lower part of the unmanned station, and a control server 400 that controls autonomous unmanned flight of the drone and autonomous unmanned navigation of the unmanned boat.

At the top of the unmanned station 100, there is a landing pad for takeoff and landing of the drone 200, and a drone detachment device (not shown) that uncouples or attaches the drone 200 when the drone 200 takes off and lands. It can be installed at the bottom of the unmanned station 100, and there is a detachment site for detaching and detaching the unmanned boat 300, as well as a boat detachment for decoupling or attaching the unmanned boat 300 when detaching or detaching the unmanned boat 300. Additional devices (not shown) may be installed.

A charging unit 110 is further installed at the take-off and landing site constructed on top of the unmanned station 100 to charge and provide power to the rechargeable battery mounted on the drone 200 when the drone 200 lands, An IOT sensor unit 120 for wireless communication between the drone 200 and the unmanned boat 300 is further installed.

Meanwhile, the drone 200 is equipped with a first artificial intelligence unit 210 and a flight control unit 220, and is capable of wireless communication with the IOT sensor unit 120 installed in the unmanned station 100, making it autonomous. Along with the flight, you can judge whether you can fly and decide whether to take off or not.

The first artificial intelligence unit 210 can determine whether the drone 200 can fly by analyzing weather and environmental information collected and transmitted from the IOT sensor unit 120 using an artificial intelligence method, and if so, it can determine whether the drone 200 can fly. If it is determined that during the flight of the drone 200, a recorded video of a part of the dam basin or part of the dam facility that requires management of the dam basin and inspection of the condition of the dam facility structure or is difficult to access even by manpower, It is possible to autonomously perform the collected video information and determine the processing task of transmitting the video in real time to the control server 500, which will be described later.

Of course, autonomous shooting of images through the decision of the first artificial intelligence unit 210 during the flight of the drone 200 can be implemented through the monitoring unit 230 installed on the drone 200, such as The monitoring unit 230 may be composed of a multi-camera 231, which includes a thermal imaging sensor 232, a multi-spectral sensor 233, an air quality measurement sensor 234, and a lidar sensor. It may be configured in a manner including (235).

The flight control unit 220 mounted on the drone 200 may be mounted in the structure of a flight control board that enables autonomous flight control of the drone 200 and setting of the flight path of the drone 200.

Meanwhile, the unmanned boat 300 is equipped with a second artificial intelligence unit 310 and a navigation control unit 320, and wireless communication with the IOT sensor unit 120 installed in the unmanned station 100 is possible, With autonomous navigation, you can decide for yourself whether to sail or not.

The second artificial intelligence unit 310 can determine whether the unmanned boat 300 can sail by analyzing the weather and environmental information collected and transmitted from the IOT sensor unit 120 using artificial intelligence. If it is determined that during the navigation of the unmanned boat 300, the collected image information, such as a recording of the captured image of the dam basin area or dam facility area below the water surface or the water surface that can be captured by the drone 200, may be collected. It is possible to determine the processing task of transmitting the video in real time to the control server 500, which will be described later.

Of course, autonomous shooting of images through the decision of the second artificial intelligence unit 310 during the flight of the unmanned boat 300 can be implemented through the monitoring unit 320 installed on the unmanned boat 300. , This monitoring unit 330 may be composed of a multi-camera 331, which includes a thermal imaging sensor 332, a multi-spectral sensor 333, an air quality measurement sensor 334, and a lidar. It may be configured in a way that includes a sensor 335.

The navigation control unit 320 mounted on the unmanned boat 300 can be mounted in the structure of a navigation control board that enables autonomous navigation control of the unmanned boat 300 as well as setting the navigation path of the unmanned boat 300. there is.

The thermal imaging sensors 232 and 332 above have the advantage of being able to fly a drone (100) and sail an unmanned boat (300) to monitor the status and monitoring of the dam basin survey site or dam facility survey site at night. In addition, it will be possible to identify drowning phenomena that may occur in the dam basin or dam facility inspection areas at night, as well as the activity status of other animals.

In particular, the above multi-spectral sensors 233 and 333 are capable of Ethernet connection and have the advantage of being able to be connected to a gas battery with an extended voltage range, and can be used in the irradiated area of the dam basin in the visible and invisible light bands. Multispectral image data for the irradiated area of the dam facility can be collected, and through this, RGB color composite images can also be generated. The reflectance of the irradiated area of the dam basin or dam facility can be optimally detected to determine the dam basin. It can provide the information necessary to evaluate the condition of the survey site or the survey site of the dam facility.

In addition, these multi-spectral sensors 233 and 333 have the advantage of accurately and repeatedly correcting multi-spectral image data acquired through a single flight of the drone 100 or navigation of the unmanned boat 300. there is.

In addition, the lidar sensors 325 and 335 above can detect the distance to an object and various physical properties by illuminating the investigation target in the dam basin or dam facility area, and the topographical features in the dam basin or dam facility structure. It can detect the back and model it as a 3D image, and in particular, it can function as an eye for the flight of the drone (100) and the navigation of the unmanned boat (300).

Moreover, the first and second artificial intelligence units ()() mounted on the drone 100 and the unmanned boat 300 can be linked to the ECU that controls flight and navigation, and the electronic signals of this ECU are transmitted to the control server. Can be communicated with (400).

In particular, the ECU analyzes the driving process of the drone 100 and the unmanned boat 300 according to GPS reception and the detection of changes in the movement of the object through the operation of the speed detection sensor of the moving object linked to the LiDAR sensor detection operation. It is possible to avoid collisions between drones and unmanned boats with obstacles (moving or fixed obstacles) through the collision prevention program built into the ECU.

In other words, by matching the stable distance value according to the virtual change value of the speed of the drone and unmanned boat with an arbitrary collision avoidance stable distance value, the drone and unmanned boat can be guided to go beyond the detection range of obstacles (moving or fixed obstacles). There is.

Therefore, the above-mentioned monitoring units 230 and 330 installed on the drone 100 and the unmanned boat 300, respectively, perform remote monitoring, reconnaissance, and condition inspection tasks in areas with restricted access to dam basins, dam facility structures, or waterfront facilities. As it can be used quickly and on a regular basis, it can ensure safety and accuracy compared to human investigation, dramatically save time and effort, and also reduce the risk of collision.

Meanwhile, the control server 400 has a 3D virtual platform on which a 3D digital image production tool 410 is installed. This 3D digital image production tool 410 includes a 3D digital image production program 411 and a simulation program ( 412).

The 3D digital image production tool 410 uses the 3D digital twin dam model as a base map and uses GNSS (Global Navigation Satellite System) surveying of the dam basin, drone photogrammetry, unmanned boat photogrammetry, and 3D scanning technology. By acquiring and processing high-resolution images and point cloud data, 3D twin virtual environment dam modeling of the dam basin can be built in advance.

This 3D digital twin virtual environment dam modeling can ultimately be processed and produced through a 3D digital image production program (411), and a 3D twin virtual environment dam modeling simulation can be produced in virtual reality through the simulation program (412). It is possible to demonstrate and interpret the management status of the dam basin area or dam facility structure area.

In particular, the 3D twin virtual environment dam modeling simulation reflects digital twin technology, so cracks around the dam that can occur through simulation are eliminated when inspecting and managing the condition of the dam basin, dam facility structure, or waterfront facility. , it can be used to identify and solve problems such as defects, cracks, and other problems.

In other words, you can create a twin of a dam basin, dam facility structure, or waterfront facility in the computer's virtual reality, and predict in advance the results of simulations of problems that may occur in reality with dedicated simulation monitors connected to the control server 400. , By predicting unexpected accidents in advance through these simulations and applying them to the real world, problems with unexpected situations that may occur around dams can be prevented in advance.

In this way, the 3D twin virtual environment dam modeling, which is expressed in the same way as the physical objects around the dam and on the computer, can be built/updated through the 3D digital image production tool 410, and can also be linked to data in related fields around the dam. /Can also be fused.

Meanwhile, since the digital twin is reflected in the 3D digital image production tool 410 of the control server 400, complementarity must be strengthened according to the risk of exposure of technology to the architectural structure of the dam and water resources facilities. Strengthening complementarity is achieved through a 'two-way real-time question-leading-response' method in which the artificial intelligence approval unit (not shown) installed in the control server 400 is used to respond to someone's computer during the approval process for accessing the control server 400. can be transmitted, and can only be approved if someone (e.g. a hacker) responds to this 'two-way real-time question-leading-response' by responding, which can be implemented through a complementary processing method.

In the 'two-way real-time question guidance response', the artificial intelligence approval department (not shown) can request someone's (e.g. hacker) computer to send a question several times to induce a response for approval by the control server 400. , in this way, someone (e.g. a hacker)'s computer must respond to the requested questions several times.

Of course, the data content responded to the artificial intelligence approval unit (not shown) from someone's (e.g. hacker) computer is analyzed by the artificial intelligence approval unit (not shown) through the already learned big data DB (hacking data) and re-processed. Through learning, it is possible to determine whether the data returned is genuine or not as accessed from the hacker's computer.

In other words, the artificial intelligence approval unit (not shown) strengthens real-time complementarity by exchanging two-way communication with the computer of someone (e.g., a hacker) accessing the control server 400, even if the hacker's Even if there is an act of stealing authentication information by remotely accessing the authentication terminal, hacking methods, etc. can be neutralized due to the limitation of not being able to respond to the personal information stolen in the response through the 'two-party real-time question-leading-response' method.

In other words, the web communication format called 'two-way real-time question-leading-response' is a variety of personal information that can prove the identity of someone (e.g., a hacker) from the Artificial Intelligence Approval Department (not shown) to a computer or terminal of someone (e.g., a hacker). Since the content of response data that proves the identity of someone (e.g. a hacker) is requested in a way that leads to an informed question, the personal information of the hacker stolen with malicious code is used to prove the identity required in 'two-way real-time question-leading-response'. Due to the inability to respond to various questions about personal information, the stolen hacking has no choice but to be neutralized.

Moreover, the contents of the question guidance window and the question response window required in 'two-way real-time question guidance and response' can be made several times on the computer of the artificial intelligence approval department (not shown) and someone (e.g. a hacker), and the artificial intelligence approval The content of the questions asked in the question guidance window through the communication window of the Department (not shown) is answered by the Department of Artificial Intelligence Approval (not shown), which learns the security channel created based on 'virtualization-based network separation technology'. Through the application of artificial intelligence, it can be processed with various question pattern methods necessary for someone (for example, a hacker) to recognize the person's identity, and personal information stolen from these various question pattern methods cannot be used alone.

The combination of 'virtualization-based network separation technology' and artificial intelligence technology means isolating and controlling the network of authentication terminals to completely block and eliminate the act of hackers remotely accessing authentication terminals and stealing authentication information. Even if this is achieved, it will not be possible to cope with various question pattern methods through intelligent learning of artificial intelligence. Authentication information can be exchanged by creating a separate dedicated network (VPN) to transmit only authentication information, and the input method is also automated. Hacking methods can be neutralized.

In this way, the complementarity resulting from the reflection of the digital twin in the 3D digital image production tool 410 of the control server 400 is strengthened through the 'two-way real-time question-leading-response' processing method of the artificial intelligence approval department (not shown) described above. It can be.

This digital twin can be reflected in the management server 400 in a way that utilizes components such as the Internet of Things, Industry 4.0, finite element method, cloud, and artificial intelligence, allowing modeling simulation in virtual reality for dam management. Problems of unexpected situations that may occur in dams can be prevented in advance, and complementarity of the management server 400 can also be strengthened.

As described above, the present invention can build a 3D digital twin virtual environment model of the dam basin in advance through digital twin dam basin management using artificial intelligence drones, and provides remote monitoring and reconnaissance for restricted access areas around the dam. It can speed up the status inspection of work, dam structures, waterfront facilities, etc., ensure safety and accuracy as it can be used at all times during the day and night, and dramatically save time and labor.

In addition, the information transmitted in real time from the drone 100 and the unmanned boat 300 to the control server 400 is compared and analyzed with the base digital twin data to determine dam facility maintenance, measurement and repair history visualization, and facility physical properties. It can be used for visualization work linked to information, hydrology, and measurement information, and can also promote intelligence in inspection work through AI analysis of concrete crack information, water leakage, etc.

Moreover, after the drone 100 completes its mission by autonomous flight for a certain period of time, it can safely land at the unmanned station 200 installed around the dam to charge the battery, and can be used in each function of the drone 100. A report that checks for internal abnormalities can also be sent. Of course, battery charging and report transmission for the unmanned boat (300) are also possible.

In particular, during the monitoring work using the drone (100) in the dam basin, if bad weather such as strong winds or rain continues, the unmanned boat (300) mounted on the lower part of the unmanned station (200) determines whether to set sail through environmental information analysis. Artificial intelligence determines that it can autonomously perform the task of collecting and transmitting information about the dam basin and then return.

The unmanned boat 300 is equipped with a product capable of optical zoom and rotation based on a system that can avoid water plants and floating objects, and is equipped with a variety of sensors such as thermal imaging, multi-spectral, water quality measurement sensor, and lidar for environmental monitoring. Information can be collected by mounting mission equipment and transmitted in real time using communication equipment.

Unmanned station (100) charging unit (110)
IOT sensor unit (120)
Drone (200) 1st Artificial Intelligence Department (210)
Flight control unit (220, control board) Monitoring unit (230)
Multi-camera (231) Thermal image sensor (232)
Multi-spectral sensor (233) Air quality measurement sensor (234)
LiDAR sensor (235)
Unmanned Boat (300) 2nd Artificial Intelligence Department (310)
Navigation control unit (320) Monitoring unit (330)
Multi-camera (331) Thermal image sensor (332)
Multi-spectral sensor (333) Water quality measurement sensor (334)
Lidar sensor (335)
Control server (400) 3D digital image production tool (410)
3D digital image production program (411) Simulation program (412)

Claims (5)

delete delete delete delete Unmanned station (100) built around the dam; A drone 200 that takes off and lands at the upper takeoff and landing site of the unmanned station and constantly monitors the area around dams and water resources facilities; An unmanned boat (300) that detaches from the lower detachment site of the unmanned station and constantly monitors the water area around dams and water resources facilities; A control server 400 that controls the flight of the drone and navigation of the unmanned boat; Including,
The monitoring unit 230 installed in the drone 200 is configured in the form of a multi-camera 231; The multi-camera 231 is configured to include a thermal imaging sensor 232, a multi-spectral sensor 233, an air quality measurement sensor 234, and a lidar sensor 235,
The monitoring unit 330 installed on the unmanned boat 300 is configured in the form of a multi-camera 331; The multi-camera 331 is composed of a thermal imaging sensor 332, a multi-spectral sensor 333, a water quality measurement sensor 334, and a lidar sensor 335,
The first and second artificial intelligence units (210) and (310) mounted on the drone (200) and the unmanned boat (300) are linked to the ECU that controls flight and navigation, and the electronic signals of the ECU are sent to the control server (400). ) is communicated with,
The ECU analyzes the detection of movement changes in objects through the operation of a speed detection sensor of a moving object linked to the LiDAR sensor detection operation during the driving process of the drone 200 and the unmanned boat 300 according to GPS reception, Avoids collision with obstacles between drones and unmanned boats through the collision prevention program built into the ECU;
A 3D digital image production tool 410 reflecting the digital twin is installed in the control server 400; The 3D digital image production tool 410 includes a 3D digital image production program 411 for producing 3D digital twin virtual environment dam modeling and a simulation program 412 for demonstrating 3D digital twin virtual environment dam modeling. It consists of a composition comprising;
In the control server 400, supplementation is strengthened through 'two-way real-time question guidance and response' of the artificial intelligence approval department,
The contents of the question guidance window and the question response window required in the above 'two-way real-time question guidance and response' are processed several times on the artificial intelligence approval department and the hacker's computer, and the question guidance window is provided through the communication window of the artificial intelligence approval department. The content of the questions asked is processed in the form of a pattern of questions necessary to recognize the identity of the hacker through the application of artificial intelligence to the artificial intelligence approval department, which learns the security channel created based on 'virtualization-based network separation technology'. An artificial intelligence drone unmanned station system for regular automatic inspection of dam basins and water resources facilities.