JP6833296B2 - Unmanned aerial vehicle, flight path creation method for unmanned aerial vehicle and plant control system - Google Patents

Description

Embodiments of the present invention relate to unmanned aerial vehicles, flight path creation methods for unmanned aerial vehicles, and plant control systems.

Humans are often inaccessible to the operating lines of a plant. For example, during the rolling process of a steel plant, the area around the rolling mill is dangerous and human access is severely restricted. Therefore, the necessary information is taken in by the automation controller (PLC) from the sensors that are deployed and wired in advance, and the PLC that stores the program in advance monitors and automatically controls the line based on the sensor information. , Issuing necessary alarms, or stopping the operation of the line.

In order to collect information that cannot be obtained only from sensor information, the rolling line of a steel plant is regularly set with line downtime to ensure safety and then collect information necessary for maintenance by humans.

In addition, even while the equipment or the like is in operation, a human may patrol a designated patrol route where safety is ensured and perform a visual inspection or the like on a regular basis.

In the prior art, most of the sensor inputs are digital I / O such as limit switches and relay contacts, or analog values such as flow rate and temperature. In addition, a surveillance image by an industrial surveillance camera (ITV) or a situation before and after the operation is stopped may be recorded in a moving image. However, the acquired camera image is not linked to the data acquired by the PLC and is often used independently.

Unmanned autonomous driving is advancing with technological development. It is said that there are still technical issues in the automatic driving of automobiles running on public roads until practical use, but the automatic driving of heavy machinery that works within a limited range is generally used (for example, patent documents). 1st grade).

GPS is used for automatic operation of heavy machinery, and automatic operation is possible by creating a route from GPS information. However, if there is a discrepancy between GPS and the actual terrain, automatic driving will be delayed.

In order to solve such a problem, the technique disclosed in Patent Document 1 feeds back actual terrain data and compares the data with GPS data to reduce differences due to actual terrain. It is something to do.

Not only heavy machinery but also unmanned aerial vehicles (drones) can generally create a route from GPS and automatically drive according to GPS if there are no obstacles.

Industrial plants such as oil refineries, chemical plants and power generation plants are automatically operated by distributed control systems (DCS). The DCS is equipped with a human-machine interface (HMI), and the HMI can notify a human being of the driving situation.

By the way, since human beings can make a comprehensive judgment including information from HMI, they can play a role of identifying physical risks such as equipment and devices existing in a plant and reducing them. By performing maintenance routines such as human visual inspections and regular routines, the risks that can occur in the plant can be reduced.

Patent Document 2 utilizes a drone to make the drone take on the role of a human being, thereby saving labor and improving productivity. Route generators called waypoints will be deployed in various places to generate routes for drones.

International release 2017/115879 Japanese Unexamined Patent Publication No. 2016-197404

The use of drones as a new technology is expected in various fields, and it has been put to practical use in some industries. Although there are various problems regarding the self-sustaining operation of the drone, the following two points are particularly focused on in the present invention.

(1) In order to establish an independent driving route, three-dimensional coordinate data such as GPS or a structure within a moving range is usually required. However, the accuracy of GPS is not sufficient in a narrow range, and it is difficult to use GPS itself in an indoor plant. In addition, it takes a lot of time and man-hours to construct a database having accurate 3D coordinate data in advance. Therefore, with the existing technology as described above, it is difficult to establish an independent operation route of the drone for monitoring in the plant line.

(2) Even if it is possible to operate independently while floating in the air, the charging time is long for the short operating time. For example, even if the drone is charged for several hours, the operable time is often limited to about 20 to 30 minutes.

The embodiment of the present invention has been made to solve the above-mentioned problems, does not require GPS or accurate three-dimensional coordinate data, can operate independently, and can move in the plant line in a short time. It is an object of the present invention to provide an unmanned air vehicle capable of patrol flight, a method for creating a flight path for the unmanned air vehicle, and a plant control system.

The unmanned aerial vehicle according to the embodiment patrols the plant and collects line inspection data. The unmanned vehicle is provided at different installation angles, and a plurality of first imaging units that acquire first captured images that are captured images around the flight path in which they fly, the first captured image, and the above-mentioned installation. A control unit including an image processing unit that processes a second image that is set at an angle and is an image of the periphery of the patrol path that has been acquired in advance by the same number of second imaging units as the plurality of first imaging units. , Equipped with. The control unit controls the flight attitude so that the first captured image matches the second captured image, and creates the flight path.

In the present embodiment, since the flight attitude is controlled so that the image captured by the self captures the image acquired in advance and the flight path is created, GPS and accurate three-dimensional coordinate data are not required. It can operate independently and can fly around the plant line in a short time.

FIG. 1A is a schematic diagram illustrating a patrol route of a plant. FIG. 1B is a schematic front view illustrating the drone according to the first embodiment. It is a schematic diagram which shows the image of the camera acquired by a human being, and the image of the camera which the drone has taken. It is a schematic diagram which illustrates the plant control system which concerns on 2nd Embodiment.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
The drawings are schematic or conceptual, and the relationship between the thickness and width of each part, the ratio of the sizes between the parts, and the like are not necessarily the same as the actual ones. Further, even when the same parts are represented, the dimensions and ratios may be different from each other depending on the drawings.
In addition, in the present specification and each figure, the same elements as those described above with respect to the above-mentioned figures are designated by the same reference numerals, and detailed description thereof will be omitted as appropriate.

(First Embodiment)
FIG. 1A is a schematic diagram illustrating a patrol route of a plant.
As shown in FIG. 1 (a), a plurality of devices and devices are installed in the plant 100. Plant 100 is, in this example, a hot rolling plant for steel. In the plant 100 of this example, the heating furnace 110, the rough rolling mill 112, the edge heater / bar heater 118, the finishing rolling mill 120, the water cooling device 126, and the winder 128 are linearly arranged. The rough rolling mill 112 is driven by the electric motor 114, and the electric motor 114 is controlled by the drive device 116. The finish rolling mill 120 is driven by an electric motor 122 and controlled by a drive device 124. In addition, transport equipment such as table rollers (not shown) is also driven by an electric motor and controlled by a drive device. In addition, the equipment and devices of the plant 100, including other devices such as the heating furnace 110, operate interlockingly or independently under set conditions under the control of the process computer and the process control controller.

These devices and devices are inspected on a daily basis. The inspection is carried out along a fixed patrol route 50. In the present embodiment, first, as the patrol route 50, the worker 2 wears a helmet 4 provided with a plurality of cameras 6a to 6d, patrols the preset patrol route 50, and determines the inspection points. Check the inspection items mainly visually. The patrol route is not limited to one route, and may be multiple routes depending on the frequency and contents of inspections.

In the following, the case of a hot rolling plant will be described as an example, but the drone of the embodiment is not limited to the hot rolling plant of steel, and any other plant as long as it is a plant that performs daily patrol inspection. However, it can be applied.

The helmet 4 worn by the worker 2 is provided with a plurality of cameras 6a to 6d (second imaging unit, second imaging device) at different angles. In this example, four cameras are provided, but it is preferable to provide at least three cameras in order to identify the position of the worker 2.

The cameras 6a to 6d patrol the worker 2 and acquire an image of the periphery of the patrol path 50 according to the direction in which the worker 2 faces. The image data acquired by the cameras 6a to 6d is associated with the acquisition time of the image data and is stored in the data server 8 as, for example, moving image data. The image data may be transferred and stored in the data server 8 in real time, or the image data may be stored in the cameras 6a to 6d, and the amount of image data becomes constant after the patrol is completed or during the patrol. The image data may be transferred to the data server 8 each time.

Worker 2 fills in the visually checked items on the process inspection sheet, etc., or inputs them using a tablet terminal, etc., but depending on the characteristics of the plant line, items other than visual inspection items May be checked. Inspection items other than visual inspection include, for example, abnormal noise and odor. When the worker 2 feels that an abnormal noise or odor is generated during the patrol, the worker 2 fills in the characteristics and the like on an inspection sheet and the like. Check data such as a process check sheet is stored in the data server 8 in association with the entry time.

FIG. 1B is a schematic front view illustrating the drone of the present embodiment.
As shown in FIG. 1B, the drone 10 includes a main body 12 and a sensor module 14. The main body 12 includes a propeller for the drone 10 to fly, a motor for driving the propeller, a controller (control unit, control device) for controlling the flight attitude, the traveling direction, and the like of the drone 10.

The sensor module 14 includes a plurality of cameras 16a to 16d (first imaging unit, first imaging device). The plurality of cameras 16a to 16d are attached to the sensor module 14 at the same positions and angles as the cameras 6a to 6d provided on the helmet 4 worn by the worker 2. The cameras 16a to 16d of the drone 10 acquire images around the flight path. As the flight path of the drone 10, a route is selected so that the image acquired by the human patrolling camera and the image captured while the drone 10 is flying match.

The drone 10 acquires image data acquired by the worker from the data server 8 and compares it with the image data captured by its own cameras 16a to 16d. The drone 10 sets the flight direction so that the image captured by the drone 10 matches the image around the patrol path 50 acquired by the worker 2 patrol.

The operation of the drone 10 of the present embodiment will be described.
FIG. 2 is a schematic diagram showing a camera image acquired by a human and a camera image captured by a drone.
FIG. 2 shows an example of images acquired by the four cameras 6a, 6b, 16a, 16b. The images 20 and 21 are images of the periphery of the patrol route 50 acquired by the worker 2 patrol, and are, for example, peripheral images at the point A in FIG. The image 20 is an image captured by the camera 6a of the worker 2 in this example. The image 21 is an image captured by the camera 6b of the worker 2 in this example.

Images 22 and 23 are images taken by the drone 10 while flying. Image 22 is an image captured by the camera 16a of the drone 10 in this example. Image 23 is an image captured by the camera 16b of the drone 10 in this example.

The cameras 16a and 16b of the drone 10 are set at positions and angles corresponding to the cameras 6a and 6b of the worker 2, respectively. The drone 10 searches for its own position and sets the flight path 52 so that the image 22 of the camera 16a matches the image 20 and the image 23 of the camera 16b matches the image 21.

When the worker 2 patrols, the patrol route 50 is patrolled, and the meters such as the control panel and the confirmation points of the devices and devices are visually confirmed. The drone 10 flies along the flight path 52 along the patrol route 50, and also acquires image data of a portion visually confirmed by the worker 2. The points of visual inspection are, for example, turning on / off / blinking of meters and indicator lights on the control panel.

The image data acquired by the drone 10 is stored in the data server 8. The drone 10 may store the image data and transfer the data to the data server 8 after the flight is completed, transfer the image data in real time, or transfer the data when the data capacity of the image data reaches a certain amount. It may be transferred to the server 8.

In this way, the drone 10 can set the flight position so that the image captured by the drone 10 matches the image acquired by the worker 2, and can also acquire the image of the inspection item acquired by the drone 10. .. The acquired image data of the inspection item is associated with the acquisition time and is managed together with the I / O data and the like collected by the process controller and the like on the server, for example.

Of the image data acquired by the worker 2 and the drone 10, the image portion of the inspection item such as the meter of the control panel may be different from each other. That is, in the automatic creation of the flight path 52 of the drone 10, the flight path 52 is set by matching the image data, and the image data including the image portion that is not directly related to the setting of the flight path 52 is acquired. May be executed at the same time and processing may be required. In such a case, for example, the following processing may be additionally executed.

First, the drone 10 acquires the image data, and the data server 8 or the like temporarily stores the image data as it is with the acquired contents.

After that, the data of the images 20 and 21 acquired by the worker 2 and the data of the images 22 and 23 acquired by the worker 2 are image-processed to extract the respective contour portions, and the patrol route 50 is created by matching or disagreeing the contours. Identify and determine flight path 52.

Since the arrangement of equipment and devices installed in the plant is rarely moved in a short period of time, it is possible to identify the situation around the patrol route 50 by extracting the outline of the captured image. it can. In general, since the meter or the like of the control panel is arranged inside the contour of the image, it is possible to reduce the influence of the fluctuation of the image portion other than the contour on the identification of the flight path. In this way, the drone 10 can identify its own flight path and acquire data on inspection items at the same time.

In addition, using a well-known image processing technique, it is possible to identify the flight path by image comparison and acquire inspection item data at the same time. For example, for each of the image required to identify the flight path and the image related to the inspection item, the feature amount after image processing is set in advance, and the difference from the set feature amount is used as the threshold value for each data. You may try to identify.

The flight path of the drone 10 may be appropriately modified by interposing an operator who wirelessly controls the drone 10 during the flight of the drone 10. When the drone 10 takes time to find the flight path 52, the flight time can be shortened by intervening the operator.

Further, by providing the sensor module 14 with an inertial sensor such as an acceleration sensor, the drone 10 can recognize how much it has moved in which direction. With the position where the drone 10 starts flying as the initial position, the drone 10 can store the data of its own movement position. The drone 10 stores the flight history if the operator intervenes in the correction of the flight path to find the flight path 52 that matches the patrol path 50 earlier. The flight path can be optimized by machine learning when the flight time is shortened by using the intervention of an operator. By optimizing the flight path, the flight time of the drone 10 operated by the power of the rechargeable battery can be shortened.

In this way, the drone 10 of the present embodiment can identify its own flight path 52 along the patrol route 50 of the worker 2. The flight path 52 is set so that the image captured by the drone 10 matches the image around the patrol path 50 acquired by the worker 2 in advance. Therefore, the flight path of the drone 10 can be automatically created without guiding by GPS or constructing a database having accurate three-dimensional coordinate data.

The drone 10 can acquire the same data as the visual data of the inspection items acquired by the worker 2 as image data. The acquired image data is associated with the acquisition time and can be used together with other I / O data and the like. Therefore, it is possible to realize automation and labor saving of patrol inspection.

The flight path 52 of the drone 10 can be created in a shorter time and the flight time can be shortened by appropriately patrolling the patrol route 50 by the worker 2. In addition, the flight path can be created in a shorter time by interposing an operator. Further, by providing the drone 10 with a position determination mechanism such as an inertial sensor and machine learning the intervention of the operator, an optimum flight path can be created and the flight time can be shortened.

(Second Embodiment)
In the above-described embodiment, the drone can autonomously establish a flight path along the patrol path without using the arrangement data of a device or the like based on GPS or accurate three-dimensional coordinates. In the control system, various sensors are provided to detect and grasp the status and conditions of each process, but by mounting various sensors on the drone and acquiring various data, the control system can be installed. It will be possible to grasp the more detailed situation of the target plant.

FIG. 3 is a schematic diagram illustrating the plant control system according to the present embodiment.
As shown in FIG. 3, the control system 200 includes a drone data collecting device 210 and a drone synchronization controller 212. The control system 200 further includes a comprehensive data collection device 214 and a comprehensive data viewing device 216. In addition, the control system 200 includes computers and the like connected to each other via a LAN. That is, the control system 200 includes a process controller 218, a process computer 220, an HMI terminal 222, and the like. In the control system 200, the drone data collecting device 210, the drone synchronization controller 212, the comprehensive data collecting device 214, and the comprehensive data viewing device 216 cooperate with the process controller 218 and the like to comprehensively collect control data and the like of the line to be controlled. It is used for analysis of line conditions, defect analysis, etc. by collecting, classifying, and extracting desired data.

When a plurality of process controllers 218 are used, they are connected to each other via a control LAN 224. The process controller 218 collects data of each process via an input / output module or the like according to set conditions, and supplies control data to each device or apparatus. The process controller 218 is, for example, an automation controller (PLC).

The process computer 220 is connected to the PLC 218 via the control LAN 224. The process computer 220 sets the condition group and the like specified by the higher-level computer system for the PLC 218. In addition, the process computer 220 collects I / O data such as sensors and regular work signals via the PLC 218. These I / O data are associated with the acquired time and are managed by the process computer 220.

The HMI terminal 222 is connected to the PLC 218 via the information system LAN 226. The HMI terminal 222 provides a human-machine interface for devices, devices, and the like connected to the PLC 218 and below. The operator can monitor the control system 200 and the like via the HMI terminal 222.

The drone data collecting device 210 collects various data collected by the drone 10 and stores the data in association with the time of collection. The data associated with the time includes image data acquired by the cameras 16a to 16d, respectively.

The drone synchronization controller 212 acquires various data from the drone data collection device 210 and uploads them to the control LAN 224. The uploaded data is stored in the integrated data collection device 214 together with the transmitted I / O data and the like under the control of PLC218.

The comprehensive data collection device 214 is connected to the control LAN 224. The comprehensive data collection device 214 collectively manages the data collected by the drone 10 and the control data collected via the PLC 218. Each piece of data is associated with the time when the data was acquired.

The comprehensive data browsing device 216 is connected to the control LAN 224. The comprehensive data viewing device 216 extracts each data managed by the comprehensive data collecting device 214 by designating the conditions, and outputs the extracted data. Since each data managed by the integrated data collection device 214 is associated with the time when the data was collected, when extracting the data, by specifying the time, the data associated with that time can be obtained. Can be output.

In the present embodiment, the drone 10 collects data of parameters such as sound data and odor data, which are not set in a normal control system, in addition to the image data in the case of the first embodiment described above. It is possible to identify the time when an abnormal noise or odor is generated due to an abnormality in a device or device, and to confirm whether or not there is an abnormality in the image data of the visual inspection location.

As described in the case of the first embodiment, the worker 2 may inspect not only the visual inspection but also the abnormal noise and odor in the line. In the present embodiment, the drone 10 detects and collects sound data by a microphone and the presence or absence of an offensive odor by an odor sensor in each process.

The drone 10 can enter an area where the worker 2 cannot enter even while the equipment or the like is in operation. It is also possible to fly the drone 10 to an area where humans cannot enter and acquire inspection data by intervening the operation of the operator. Therefore, by collecting data such as abnormal noise during operation of the device or the like, information that could not be acquired in the past can be collected in real time and used for determining the status of the line.

According to the embodiment described above, a drone or a drone that can operate independently without requiring GPS or accurate 3D coordinate data and can collect data in a plant line in a short time. It is possible to realize a patrol route creation method and a plant control system.

Although some embodiments of the present invention have been described above, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are also included in the scope of the invention and its equivalents described in the claims. In addition, the above-described embodiments can be implemented in combination with each other.

2 workers, 4 helmets, 6a-6d, 16a-16d cameras, 8 data servers, 10 drones, 12 main bodies, 14 sensor modules, 20-23 images, 100 plants, 110 heating furnaces, 112 rough rolling machines, 114, 122 Electric motor, 116, 124 control panel, 118 edge heater / bar heater, 120 finish rolling mill, 126 water cooling device, 128 winder, 200 control system, 210 drone data collector, 212 drone synchronization controller, 214 comprehensive data collector, 216 Comprehensive data browsing device, 218 process controller, 220 process computer, 222 HMI terminal, 224 control LAN, 226 information system LAN

Claims (4)

An unmanned aerial vehicle that patrols the plant and collects line inspection data.
A plurality of first imaging units that are provided at different installation angles and acquire first captured images that are captured images around the flight path in which they fly.
Image processing of the first captured image and the second captured image which is an image around the patrol path set in the installation angle and acquired in advance by the same number of second imaging units as the plurality of first imaging units. Control unit including image processing unit
With
The control unit is an unmanned aerial vehicle that creates the flight path by controlling the flight attitude so that the first captured image matches the second captured image. It is a method of creating a flight path for an unmanned aerial vehicle that patrols the plant and collects line inspection data.
A plurality of first image pickup devices provided at different installation angles acquire the first captured image which is an image taken around the flight path in which the aircraft flies.
An image of the second captured image, which is an image of the periphery of the patrol path acquired in advance by the same number of second imaging devices as the plurality of first imaging devices set at the installation angle, and the first captured image. Process and
A method for creating a flight path of an unmanned vehicle that creates the flight path by controlling the flight attitude so that the first captured image matches the second captured image by a control device. An unmanned aerial vehicle that patrols the plant and collects line inspection data.
A plurality of first imaging units that are provided at different installation angles and acquire first captured images that are captured images around the flight path in which they fly.
Image processing of the first captured image and the second captured image which is an image around the patrol path set in the installation angle and acquired in advance by the same number of second imaging units as the plurality of first imaging units. Control unit including image processing unit
Sensors capable of detecting data related to plant equipment and devices installed around the flight path, and
With
The control unit controls the flight attitude so that the first captured image matches the second captured image, creates the flight path, and stores the data detected by the sensor together with the time of detection. Unmanned air vehicle to do. The unmanned aerial vehicle according to claim 3 and
A drone data collection device that collects and stores the time and the data associated with the time.
A comprehensive data collection device that collects input / output data and control data acquired by the process controller of the plant, and
A drone synchronization controller that associates the data, the input / output data, and the control data with the time.
Plant control system with.

Images