JP2020201850A - Control method of flying body - Google Patents

Abstract

To provide a more effective inspection method for bridges.SOLUTION: An embodiment of the present invention is a control method of a flying body, and further includes the steps of: controlling the flying body so as to move to a structure; referring to information related to the structure to set a flight route; taking an image of part of the structure; and transmitting the captured image.SELECTED DRAWING: Figure 1

Description

The present invention relates to a method for controlling an air vehicle.

In the past, maintenance and inspection of railway bridges was carried out by assembling scaffolding on the bridges, walking by workers, and visually observing them, which took time and had an impact on the running of the railway. ..

In particular, visual inspection of the underside of a bridge is difficult due to the location of roads and rivers. As a system to support such inspection, a rail is installed at the bottom of the bridge and a robot camera is placed on the rail. A system that captures the underside of a bridge by traveling (Patent Document 1), and an inspection vehicle with multiple cameras installed on the roof is driven on the road under the bridge to photograph the underside of the bridge and map it to a drawing. (Patent Document 2).

Japanese Unexamined Patent Publication No. 2016-079615 JP-A-2017-106170

However, although the system disclosed in Patent Document can realize the support for bridge inspection, the system disclosed in Patent Document 1 requires further installation of structures such as rails, which requires time and time. It costs money. Further, the system disclosed in Patent Document 2 is difficult to inspect when there is no road on the lower surface of the bridge, for example, when the terrain is a river or a canyon.

Therefore, an object of the present invention is to solve such a problem and to provide a more efficient bridge inspection method.

One embodiment of the present invention is a method for controlling an air vehicle, which includes a step of controlling the air vehicle to move to a structure, a step of referring to information related to the structure, and a step of setting a flight route. Includes a step of imaging a portion of the structure.

According to the present invention, it is possible to provide a more efficient bridge inspection method.

FIG. 1 is an overview view of an inspection method by the flying object 10 according to the first embodiment. FIG. 2 is a functional block diagram of the flying object 10 according to the first embodiment. FIG. 3 is a functional block diagram showing controls related to inspection of the flying object according to the first embodiment. FIG. 4 is a flowchart showing the control of the flying object system according to the first embodiment. FIG. 5 shows an example in which mapping is performed on the drawing data of the bridge in the first embodiment. FIG. 6 is a flowchart showing the control related to the inspection of the flying object according to the second embodiment.

The contents of the embodiments of the present invention will be described in a list. The flying object according to the embodiment of the present invention has the following configuration.
[Item 1]
It ’s a control method for the aircraft.
Steps to control the movement of the flying object to the structure,
Steps to set a flight route by referring to the information related to the structure,
The step of imaging a part of the structure and
How to control the flying object, including.
[Item 2]
The method for controlling an air vehicle according to claim 1.
A method of controlling an air vehicle, further comprising a step of transmitting the captured image.
[Item 3]
The method for controlling an air vehicle according to claim 1.
A method for controlling an air vehicle, further comprising a step of confirming the presence or absence of damage based on the captured image.
[Item 3]
The method for controlling an air vehicle according to claim 1.
A method for controlling an air vehicle, which comprises referring to the drawing information of the structure and mapping the captured image to the drawing information.
[Item 4]
The method for controlling an air vehicle according to claim 3.
A method for controlling an air vehicle, which comprises referring to drawing information of the structure and mapping the position of the damage to the drawing information when damage is confirmed in the confirmation step.

(First Embodiment)
Hereinafter, the flying object according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an overview view of a bridge inspection method by the flying object 10 according to the first embodiment.

As shown in FIG. 1, the flying object 10 can fly around a structure (for example, a bridge 1) and image each part of the bridge 1 with a camera. Since the airframe 10 is not restricted in movement, it can fly at any place including the upper surface and the lower surface of the bridge, and can fly as far as possible inside the bridge in light of the size of the airframe. Further, although not shown, the captured image can be transmitted to an external device such as a management terminal via a network.

FIG. 2 is a functional block diagram of the flying object 10 according to the first embodiment. First, the flight controller 21 can have one or more processors such as a programmable processor (eg, a central processing unit (CPU)). The camera 5 is mounted on the airframe via a gimbal, and the gimbal can, for example, rotate the camera 5 in the vertical direction with respect to the airframe. It is preferable that the machine can rotate in three axial directions (pitch angle, roll angle, yaw angle). In addition, when it is difficult to take a picture in a certain direction when viewed from the flying object 10, such as above or below the flying object 10 due to the relationship between the camera and the aircraft, for example, a plurality of cameras are placed on the upper surface and the lower surface of the flying object 10. You can also have. Further, by using a stereo camera as a camera, it is possible to measure the depth, distance, etc. of the image captured in the subsequent image processing.

Further, the flight controller 21 has a memory 22 and can access the memory. Memory 22 stores logic, code, and / or program instructions that the flight controller can execute to perform one or more steps.

The memory 22 may include, for example, a separable medium such as an SD card or random access memory (RAM) or an external storage device. The data acquired from the cameras and sensors may be directly transmitted and stored in the memory. For example, still image / moving image data taken by a camera or the like is recorded in an internal memory or an external memory. The camera is installed on the aircraft via a gimbal.

The flight controller 21 includes a control module configured to control the state of the flying object. For example, the control module adjusts the spatial placement, velocity, and / or acceleration of an air vehicle with six degrees of freedom (translational motion x, y and z, and rotational motion θ _x , θ _y and θ _z ). , Controls the propulsion mechanism (motor, etc.) of the flying object via ESC. The motor rotates the propeller to generate lift of the flying object. The control module can control one or more of the states of the mounting unit and the sensors.

The flight controller 21 is configured to transmit and / or receive data from one or more external devices (eg, transmitter / receiver (propo) 23, terminal, display device, or other remote controller). It is possible to communicate with the unit 24. The transmitter / receiver 23 can use any suitable communication means such as wired communication or wireless communication.

For example, the transmission / reception unit 24 uses one or more of a local area network (LAN), a wide area network (WAN), infrared rays, wireless, WiFi, a point-to-point (P2P) network, a telecommunications network, and cloud communication. can do.

The transmitter / receiver 24 may transmit and / or receive one or more of data acquired by sensors, processing results generated by a flight controller, predetermined control data, user commands from a terminal or a remote controller, and the like. it can.

Sensors 25 according to this embodiment may include inertial sensors (accelerometers, gyro sensors), GPS sensors, proximity sensors (eg, riders), or vision / image sensors (eg, cameras).

Further, the inspection unit 31 stores the information necessary for inspection such as image data captured by the camera 5 in the memory 32, and executes the bridge inspection. As another example, it is also possible to provide all or a part of the functions of the inspection unit 31 in a management terminal or the like outside the flying object (not shown).

Next, a method of controlling the flying object in the present embodiment will be described with reference to FIGS. 3 and 4. FIG. 3 is a functional block diagram relating to the vehicle body control related to the inspection of the bridge of the vehicle body according to the present embodiment. Further, FIG. 4 is a flowchart of the control of the flight body system including the flight body and the management terminal according to the present embodiment.

The control method for the flying object to inspect the bridge in the present embodiment can be realized by executing the control program in the CPU in the flight controller 21 and / or the CPU in the inspection unit 31. Further, as another example, all or a part of the functions of the flight controller 21 and / or the inspection unit 31 can be provided in the external device 28 (for example, a server or a management terminal), and the necessary processing can be performed on the flying object. 10 and the external device 28 can be executed in cooperation with each other. In this embodiment, the flying object 10 and an external device 28 such as a management terminal cooperate with each other to inspect the bridge.

First, in FIG. 3, the bridge information acquisition unit 51 refers to, for example, the position information of the bridge stored in the bridge information storage unit 61 of the memory 22 and confirms the position to be investigated. With respect to the position information confirmed by referring to the bridge information, the current position information acquisition unit 52 acquires the current position information from the GPS sensor 54, the acceleration sensor 55, etc., and the flight control unit 53 controls the motor 26 for the purpose. The flying object 10 can be controlled to move to the inspection position which is the ground (S401). The acquired current position information is stored in the current position information storage unit 62 in the memory 22. In addition, this step can be replaced by manually operating the flying object, and when GPS does not operate, the flying object is recognized by recognizing the bridge by image recognition technology without depending on the position information. It is also possible to replace it by controlling it to fly around the bridge. Further, in this step, the flight route is set for each inspection item by referring to the bridge information or the information about the bridge structure and each part to be inspected included in the drawing information 83 and the like stored in the memory 32 of the inspection unit 31. be able to. For example, image recognition technology can be used to recognize, for example, the underside of a bridge and control it to fly a predetermined route.

Next, when the flying object 10 is positioned at the part of the bridge to be investigated, the bridge inspection unit 71 of the inspection unit 31 refers to, for example, drawing information 83, and whether the target track has already been investigated. You can check if it is not. In this example, it is assumed that the surveyed position is mapped in the drawing, but any other format can be used as long as it can be understood as position information. Further, the flying object 10 can also make an inquiry to an external device 28 such as a management terminal via the transmission / reception unit 24 and the network for this information. Further, in another example, the flying object 10 can fly by understanding the investigated / uninvestigated positions in advance and setting the routing of the uninvestigated positions in advance. In yet another example, the flying object 10 can keep the camera turned off at the investigated position. As a result, battery resources can be saved.

Next, the image pickup control unit 74 of the inspection unit 31 controls the camera 5 to take an image of a target bridge portion (S402). At the time of imaging, an area camera such as a so-called digital camera can also be used in the sense of acquiring an image that changes visually. Further, when shooting at night, it is possible to use an infrared camera or shoot with LED lighting. Further, as described above, by using a stereo camera, the depth and distance of images captured from a plurality of parallax images can be recognized in a later image processing step. The captured image can be stored in the memory 32.

Subsequently, the notification generation unit 72 of the inspection unit 31 transmits a notification to an external device 28 such as a management terminal via the transmission / reception unit 24 (S403). The content of the notification can include position information, bridge part information, images, and the like. The notification process can be performed periodically by accumulating images, but it is preferably executed at the same time as the imaging.

The external device 28 such as the management terminal receives the image transmitted from the flying object 10 (S404). Subsequently, the external device 28 such as the management terminal inspects the bridge based on the received image (S405). In the inspection, for example, an image processed by an image processing unit (not shown) of a management terminal using a known method such as binarization of an image and detection of a feature amount of an edge or a line segment is used. You can also do it. Alternatively, the presence or absence of damage can be confirmed by comparing the image-processed inspection image of the bridge portion with the image-processed normal image captured at normal times. In addition to the presence or absence of damage, it is also possible to measure the magnitude of damage using a known method such as triangulation based on a parallax image obtained by a stereo camera. In addition, normal images of each part of the bridge to be inspected are machine-learned, damage discrimination rules are extracted as learning data from the features of edges and corners, and the features of the image captured by the unmanned aircraft 10 are used. It is also possible to recognize the damage by comparing with the training data. Further, when it is determined to be damaged, a score can be calculated based on the above-mentioned damage magnitude and feature amount, and it can be used later for prioritizing for determining the presence or absence of repair.

Subsequently, the external device 28 such as the management terminal maps the image, the damaged part, and the like on the drawing data of each part of the bridge based on the image received in S404 and the inspection result obtained in S405. The image is corrected to match the format of the specified drawing data, and the damaged part is given an icon indicating the damage or a special color scheme so that the inspector can easily recognize it later. be able to. In addition, depending on the degree of damage (score obtained from the size and feature amount), the size and coloring of a marker or the like indicating the damaged part can be changed. FIG. 5 shows an example of mapping on the drawing data. A plurality of captured images are arranged, and each image is indicated by an icon indicating a damaged part. The size of the icon differs depending on the size of the damage, and when one icon is selected, the details of the damage and the enlarged image are displayed. In addition, the portion of the flying object that cannot be imaged because it cannot fly can be displayed so that it can be identified on the drawing data. As a result, the person in charge of inspection can visually inspect the site.

In this way, by inspecting a bridge using an air vehicle, not only can the conventional visual work be automated, but also an inspection including a portion that is difficult with a known inspection device can be easily performed. This makes it possible to provide an efficient bridge inspection method.

(Second Embodiment)
FIG. 5 shows a flowchart relating to the vehicle body control related to the inspection of the bridge of the vehicle body according to the second embodiment. The feature of the flying object in the present embodiment is that the inspection carried out by the management terminal in the first embodiment is carried out by the flying object itself.

The structure and functional blocks of the flying object 20 of the present embodiment can be equivalent to the flying object 10 disclosed in the first embodiment shown in FIG. 2, except for the parts particularly mentioned.

The step (S601) of flying the structure (for example, a bridge) of FIG. 6 and the step (S602) of imaging a predetermined part of the bridge to be inspected are the methods according to the first embodiment described in FIG. Since it is equivalent to, the description thereof will be omitted. Further, the control block diagram of the flying object corresponding to the control method of the present embodiment can be the same as the diagram shown in FIG. 3, except for the matters particularly mentioned, and thus the description thereof will be omitted.

In FIG. 6, the bridge inspection unit 71 of the vehicle inspection unit 31 inspects the bridge based on the captured image (S603). In the inspection, for example, an image processed by the image processing unit 73 using a known method such as binarization processing of an image and detection of feature amounts of edges and corners can be used. Alternatively, the presence or absence of an abnormality can be confirmed by comparing the inspection image 81 of a predetermined portion of the image-processed bridge with the normal image 82 image-processed and image-processed at normal times. Alternatively, the presence or absence of damage can be confirmed by comparing the image-processed inspection image of the bridge portion with the image-processed normal image captured at normal times. In addition to the presence or absence of damage, it is also possible to measure the magnitude of damage using a known method such as triangulation based on a parallax image obtained by a stereo camera. In addition, normal images of each part of the bridge to be inspected are machine-learned, damage discrimination rules are extracted as learning data from the features of edges and corners, and the features of the image captured by the unmanned aircraft 10 are used. It is also possible to recognize the damage by comparing with the training data. Further, when it is determined to be damaged, a score can be calculated based on the above-mentioned damage magnitude and feature amount, and it can be used later for prioritizing for determining the presence or absence of repair.

Subsequently, the bridge inspection unit 71 confirms whether or not there is an abnormality in the track to be investigated as a result of the inspection (S604). Based on the image captured in S602 and the inspection result obtained in S603, the image, the damaged part, and the like are mapped on the drawing data of each part of the bridge. The image is corrected to match the format of the specified drawing data, and the damaged part is given an icon indicating the damage or a special color scheme so that the inspector can easily recognize it later. be able to. In addition, depending on the degree of damage (score obtained from the size and feature amount), the size and coloring of a marker or the like indicating the damaged part can be changed. In addition, the portion of the flying object that cannot be imaged because it cannot fly can be displayed so that it can be identified on the drawing data. As a result, the person in charge of inspection can visually inspect the site.

In the present embodiment, the notification generation unit 72 can transmit the drawing data generated in S604 to an external device 28 such as a management terminal via the network via the transmission / reception unit 24 (S605). In addition, the presence or absence of damage is determined together with the drawing data or separately from the drawing data, and if damage is found, a notification indicating that fact, the position information of the bridge where the damage was found, and the position information of the damage on the bridge , The degree of damage, and information such as images and drawing data can be transmitted to the external device 28.

As described above, in the present embodiment, since the flying object can take charge of all the inspections, it is possible to further improve the work efficiency.

The air vehicle of the present invention can be expected to be used as an industrial air vehicle mainly in the railway business, such as surveying, surveying, and observing structures such as bridges. In addition, the air vehicle of the present invention can be used in airplane-related industries such as multicopter drones, and further, through the present invention, it contributes to the improvement of safety related to the flight of these air vehicles and air vehicles. be able to.

The above-described embodiments are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes an equivalent thereof.

1 Bridge 5 Camera 10 Aircraft 21 Flight controller
22 Memory 23 Sensors 24 Transmitter / receiver 25 Transmitter / receiver 26 Motor 28 External device 31 Inspection unit 32 Memory 51 Bridge information acquisition unit 52 Current position information acquisition unit 53 Flight control unit 54 GPS sensor 55 Accelerometer 61 Bridge information storage unit 62 Current position Information storage unit 71 Bridge inspection unit 72 Notification generation unit 73 Image processing unit 74 Imaging control unit 81 Inspection image 82 Normal image 83 Drawing information

Claims (5)

It ’s a control method for the aircraft.
Steps to control the movement of the flying object to the structure,
Steps to set a flight route by referring to the information related to the structure,
The step of imaging a part of the structure and
How to control the flying object, including. The method for controlling an air vehicle according to claim 1.
A method of controlling an air vehicle, further comprising a step of transmitting the captured image. The method for controlling an air vehicle according to claim 1.
A method for controlling an air vehicle, further comprising a step of confirming the presence or absence of damage based on the captured image. The method for controlling an air vehicle according to claim 1.
A method for controlling an air vehicle, which comprises referring to the drawing information of the structure and mapping the captured image to the drawing information. The method for controlling an air vehicle according to claim 3.
A method for controlling an air vehicle, which comprises referring to drawing information of the structure and mapping the position of the damage to the drawing information when damage is confirmed in the confirmation step.