CN116466691A - Support device and system including support device - Google Patents

Abstract

The present invention provides a support device that supports construction machinery based on image information provided by a camera mounted on an unmanned aerial vehicle. The support device includes: a state information acquisition unit that acquires state information indicating a state of the construction machine; a determination unit that determines a target position of the UAV and a movement method of the UAV to the target position; and a control unit that controls movement of the UAV to the target position based on the movement method determined by the determination unit. The determination unit determines the movement method based on the state information, so as to avoid interference between the UAV and the construction machine. Accordingly, the interference between the unmanned aerial vehicle and the construction machinery can be avoided, and the construction machinery can be supported based on the image information provided by the imaging device mounted on the unmanned aerial vehicle.

Description

Support device and system including support device

technical field

The present invention relates to a support device and a system including the support device.

Background technique

In the past, there is a known technique of using an imaging device installed on an autonomous aircraft to capture a space that cannot be photographed by an imaging device installed on the upper revolving body of a construction machine (for example, International Publication No. WO2017/131194).

However, in the above technologies, the aircraft may interfere with the construction machinery.

Contents of the invention

The purpose of the present invention is to avoid the interference between the unmanned aerial vehicle and the construction machinery, and to support the construction machinery based on the image information provided by the camera mounted on the unmanned aerial vehicle.

The present invention provides a support device that supports construction machinery based on image information provided by a camera mounted on an unmanned aerial vehicle. The support device includes: a state information acquisition unit that acquires state information representing the state of the construction machine; a decision unit that determines a target position of the UAV and a movement method of the UAV to the target position; and a control unit that controls the movement of the UAV to the target position based on the movement method determined by the decision unit, wherein the decision unit determines the movement method based on the state information to avoid interference between the UAV and the construction machine.

In addition, the invention also provides a system. The system includes at least one of the construction machine and the unmanned aerial vehicle, and the above-mentioned supporting device.

According to the present invention, interference between the unmanned aerial vehicle and construction machinery can be avoided, and the construction machinery can be supported based on the image information provided by the camera mounted on the unmanned aerial vehicle.

Description of drawings

FIG. 1 is a diagram illustrating an exemplary configuration of a system including a support device according to an embodiment of the present invention.

FIG. 2 is a functional diagram showing the configuration of the support device according to the embodiment of the present invention.

FIG. 3 is a diagram showing an example of a hardware configuration of a control system for a construction machine according to an embodiment of the present invention.

FIG. 4 is a flowchart showing the operation of the flight control device according to the embodiment of the present invention.

FIG. 5 is a plan view of the left and right positions of the unmanned aerial vehicle according to an embodiment of the present invention.

FIG. 6A is a diagram showing a flight path of an UAV passing through two waypoints.

FIG. 6B is a diagram showing the flight path of an UAV passing a waypoint.

FIG. 6C is an example diagram of the flight path of the unmanned aerial vehicle set when starting the operation.

FIG. 7A is an illustration of an example flight path of an UAV with cables attached.

FIG. 7B is an example diagram of the flight path of the UAV with the cables connected.

Detailed ways

Hereinafter, various embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is an exemplary diagram showing the configuration of a system including a support device according to an embodiment of the present invention, and FIG. 2 is a functional diagram showing the configuration of the support device according to this embodiment.

As shown in FIG. 1 , the system including the support device of this embodiment includes, for example, a flight control device 1 , an unmanned aerial vehicle 2 , and a construction machine 3 . In FIG. 1 , the flight control device 1 and the unmanned aerial vehicle 2 , and the flight control device 1 and the construction machine 3 are connected via a network. In this case, the flight control device 1 may be constituted by a server (server computer, management device), for example. In this case, the structure of the network is arbitrary, and the network may include a wireless communication network, the Internet, a VPN (Virtual Private Network), a WAN (Wide Area Network), a wired network, or any combination thereof.

The support device according to the present embodiment supports the operation of the construction machine 3 based on the image information transmitted from the imaging device 22 mounted on the unmanned aerial vehicle 2 . In this embodiment, the function of the support device is provided on the flight control device 1 . In addition, all the functions of the support device may be provided in either one of the unmanned aerial vehicle 2 and the construction machine 3 . In addition, the functions of the support device can also be separately provided in any two or three of the flight control device 1 , the unmanned aerial vehicle 2 and the construction machine 3 .

In addition, a device having a part or all of the functions of the support device may be mounted on the construction machine 3 .

As shown in FIG. 2 , the flight control device 1 constituting the support device according to the present embodiment includes a state information acquisition unit 11 , a determination unit 12 , a control unit 13 , a calculation unit 14 , a surrounding environment information acquisition unit 15 , and an information output unit 16 . The status information acquisition unit 11 acquires status information indicating the status of the construction machine 3 . The determination unit 12 determines the target position of the UAV 2 and the movement method (for example, path) of the UAV 2 to the target position. The control unit 13 controls the movement of the UAV 2 to the target position based on the movement method determined by the determination unit 12 . The calculation unit 14 calculates a parameter value indicating the possibility of interference between the UAV 2 and the construction machine 3 when the UAV 2 moves linearly from the current position to the target position based on the state information acquired by the state information acquisition unit 11 . The surrounding environment information acquisition unit 15 acquires surrounding environment information indicating the surrounding environment of the construction machine 3 . The information output unit 16 outputs predetermined information when the determination unit 12 cannot determine the above-mentioned movement method.

The unmanned aerial vehicle 2 is, for example, a rotorcraft, and in this case, includes a plurality of rotatable blades, an electric motor (actuator) for rotating the plurality of blades, a battery for supplying power to the electric motor, and the like. In addition, it is also possible to connect a power supply line to the UAV 2 from the ground to replace the battery or use it together.

The UAV 2 also includes a control device 21 and a camera device 22 .

The control device 21 controls the flight state of the UAV 2 (forward state, retreat state, rising state, descending state, hovering, etc.) according to the control information from the flight control device 1, and guides the UAV 2 to the target position. The target location is expressed in terms of, for example, latitude, precision, and altitude. In addition, the control device 21 also controls the posture of the UAV 2 at the target position.

The control device 21 acquires airframe information representing various states related to the airframe of the UAV 2 and controls the flight state of the UAV 2 based on the airframe information. Airframe information includes position information of the UAV 2, attitude information of the UAV 2, and the like. The positional information of the UAV 2 is expressed in, for example, latitude, longitude, and altitude. The location information of such UAV 2 can be acquired by GPS sensors. The attitude information of the UAV 2 includes, for example, relevant information about the rotation of the yaw axis, the roll axis, and the pitch axis of the UAV 2 around each axis. Such attitude information of the UAV 2 can be acquired by a sensor such as an inertial measurement unit (IMU: Inertial Measurement Unit) mounted on the UAV 2 .

In addition, the control device 21 also has a sending function for sending the image acquired by the camera 22 to the construction machine 3 or the flight control device 1 via the sending unit, and a receiving function for receiving the control information sent by the flight control device 1 through the receiving unit (not shown).

The photographing device 22 includes a camera mounted on the UAV 2 . The type and the like of the camera are arbitrary, and may be, for example, a wide-angle camera. The imaging device 22 utilizes imaging elements such as CCD (charge-Coupled device) and CMOS (complementary metal oxide semiconductor) to acquire an image of the front environment in front of the fuselage of the UAV 2 . The photographing device 22 may, for example, acquire the front environment image in real time, and provide the image to the control device 21 in a streaming form with a prescribed frame period.

The imaging device 22 preferably includes a gimbal (not shown). The gimbal serves to keep the optical axis of the imaging device 22 in a fixed direction (eg, a predetermined direction in the horizontal plane) when the posture of the UAV 2 changes.

The construction machine 3 performs prescribed work while cooperating with the unmanned aerial vehicle 2 . The construction machine 3 is, for example, a construction machine equipped with a crusher suitable for dismantling work, etc., and includes a crawler-type undercarriage 31 and a slewing upper body 32 mounted on the undercarriage 31 . A cab 34 is provided on the front left side of the upper slewing body 32 . A working mechanism 35 is provided at the front central portion of the upper slewing body 32 . In addition, the construction machine 3 may be in the form of a construction machine having a crane function, or may be in another form such as a crawler crane.

The working mechanism 35 includes a boom 35a mounted on the upper slewing body 32 and capable of moving up and down, a rotatable arm 35b connected to the distal end of the boom 35a, and a breaker 35c mounted at the distal end of the arm 35b. As is well known, the boom 35a, the arm 35b, and the crusher 35c are driven by a hydraulic cylinder or the like provided as a part of the working mechanism 35 .

In addition, the construction machine 3 may also have an apron for the unmanned aerial vehicle 2 to take off and land.

FIG. 3 is a diagram showing an example of the hardware configuration of the construction machine control system according to the present embodiment.

As shown in FIG. 3 , the construction machine 3 includes a control device 40 and peripheral equipment 50 .

The control device 40 includes a CPU (Central Processing Unit) 41 connected via a bus B, a RAM (Random Access Memory) 42, a ROM (Read Only Memory) 43, an auxiliary storage device 44, a drive device (reader) 45, a communication interface 47, and a wired transceiver 48 and a wireless transceiver 49 connected to the communication interface 47.

The supporting storage device 44 is, for example, HDD (Hard Disk Drive), SSI (Solid State Drive) or the like, and is a storage device for storing data related to application software and the like.

The wired transmission and reception unit 48 includes a transmission and reception unit capable of communicating through a wired network. The wired transceiver unit 48 is connected to a peripheral device 50 . Among them, a part or all of the peripheral devices 50 may be connected to the bus B, or may be connected to the wireless transceiver unit 49 .

The wireless transmission and reception unit 49 is a transmission and reception unit capable of communicating using a wireless network. The wireless network includes a wireless communication network of a mobile phone, the Internet, a VPN, a WAN, and the like. The wireless transceiver unit 49 may include a Near Field Communication (NFC: Near Field Communication) unit, a Bluetooth (registered trademark) communication unit, a Wi-Fi (Wireless Fidelity, registered trademark) transceiver unit, an infrared ray transceiver unit, and the like. The wireless transceiver unit 49 can communicate with the flight control device 1 in the form of a server. In addition, the wireless transceiver unit 49 also receives images acquired by the imaging device 22 and transmitted by the control device 21 .

The peripheral devices 50 are electronically controllable devices mounted on the construction machine 3 , various sensors, operation units, and the like. The peripheral equipment 50 may include, for example, an image output device 51, a buzzer, a voice output device (not shown), a hydraulic pressure generating device (not shown) for operating the working mechanism 35, various sensors 52 for detecting the operating states of various operating members, etc., and an operating unit 53 for further taking over the operation. Additionally, the hydraulic pressure generating means may be a hydraulic pump driven by an engine and/or an electric motor. When using a hydraulic pump driven by an electric motor, the hydraulic pressure generating device may include an inverter for driving the electric motor.

Various types of sensors 52 include gyro sensors, GPS (Global Positioning System) sensors, various angle sensors, acceleration sensors (tilt sensors). The GPS sensor acquires position information of the construction machine 3 . The position information of the construction machine 3 is expressed in latitude, longitude and altitude. In addition, the GPS sensor includes a GPS receiver, which calculates latitude, longitude, and altitude based on radio waves from satellites, by interfering lateral positions, and the like.

In addition, various sensors 52 include various sensors that acquire parameters related to the posture of the construction machine 3 , and various sensors acquire these parameters as posture information of the construction machine 3 . In this case, the various sensors that acquire the parameters related to the posture are, for example, a boom angle sensor, an arm angle sensor, a bucket angle sensor, a body tilt sensor, and the like. The boom angle sensor is a sensor that acquires the boom angle, and includes, for example, a rotation angle sensor that detects the rotation angle of the boom pin, a stroke sensor that detects the stroke amount of the boom cylinder, an inclination (acceleration) sensor that detects the inclination angle of the boom 35a, and the like. Also, the same applies to the arm angle sensor and bucket angle sensor. The fuselage inclination sensor is a sensor for acquiring the inclination angle of the fuselage, for example, detecting the inclination angle of the upper revolving body 32 relative to the horizontal plane.

In addition, the various sensors 52 include various sensors that acquire parameters related to the rotation angle of the upper revolving body 32 relative to the lower traveling body 31 as posture information of the construction machine 3, and these sensors acquire rotation angle information indicating the rotation angle of the upper revolving body 32. In this case, the various sensors for acquiring parameters related to the rotation angle of the upper revolving body 32 may be, for example, a geomagnetic sensor, a rotation angle sensor (such as a resolver, etc.), a gyro sensor, etc. that detects the rotation angle of the revolving mechanism that makes the upper revolving body 32 rotate relative to the lower running body 31 around the revolving axis.

The attitude information includes fixed (known) parameters such as the boom length, boom mount position, boom length, and arm length of the construction machine 3 other than the parameters acquired by various sensors 52, and parameters that may change during operation such as the boom angle, arm angle, and arm angle acquired by various sensors 52.

These position information of the construction machine 3 and posture information of the construction machine 3 correspond to state information of the construction machine 3 .

The image output device 51 is installed in the cab 34 so that the operator of the construction machine 3 can see it. The image captured by the camera 22 and received by the wireless transceiver 49 is displayed on the image output device 51 . As a result, the operator of the construction machine 3 can grasp, for example, the status of the work site, which cannot be seen directly, from the front environment image on the image output device 51 .

The structure of the image output device 51 is arbitrary, and it may be, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. In addition, as another embodiment, the image output device 51 may also be a portable device (such as a tablet terminal) brought into the cab 34 by the operator of the construction machine 3 .

In addition, it is also possible to display the image output from the image output device 51 on another display device instead of displaying it on the display device provided in the image output device 51 . In this case, the output target of the image output device 51 is arbitrary, and examples thereof include a display device such as a dashboard installed in the cab of a construction machine, a terminal device such as a tablet held by an operator, and a management screen managed by an on-site supervisor.

The operation unit 53 is provided in the cab 34 and receives instructions and operations from the operator of the construction machine 3 . The instruction operation may include an instruction operation for the flight control device 1 , and the instruction operation for the flight control device 1 is sent to the flight control device 1 via the wireless transceiver 49 .

In addition, the control device 40 may also be connected to a storage medium 46 . The storage medium 46 stores predetermined programs. The program stored in the storage medium 46 is installed to the auxiliary storage device 44 and the like of the control device 40 via the drive device 45 . The installed predetermined program can be executed by the CPU 41 of the control device 40 . For example, the storage medium 46 may be a CD (Compact Disc)-ROM, a floppy disk, a magneto-optical disk, etc., which store information optically, electrically, or magnetically, or a semiconductor memory such as ROM, flash memory, etc., which store information electrically. In addition, storage medium 46 does not contain carrier waves.

FIG. 4 is a flowchart showing the operation of the flight control device 1 according to the present embodiment.

After the flight control of the UAV 2 starts, in step S102 of FIG. 4 , the flight control device 1 judges whether the new target position of the UAV 2 has been set, and when the judgment is affirmative, the process proceeds to step S104. Here, the determination in step S102 is affirmative when an instruction operation by an operator or the like on the operation unit 53 , that is, an operation instructing to change the target position, or the like is received.

The setting of the above-mentioned new target position includes the setting of the new target position when the UAV 2 takes off. In addition, the setting of the new target position also includes the setting of the target position after change when the target position is changed during the shooting by the unmanned aerial vehicle 2 . It also includes the setting of taking the landing site of the unmanned aerial vehicle 2 as the target position when it lands.

In step S104 , the flight control device 1 functions as the status information acquiring unit 11 to acquire status information indicating the status of the construction machine 3 . As described above, the status information of the construction machine 3 corresponds to the position information of the construction machine 3 and the posture information of the construction machine 3 .

In step S106 , the flight control device 1 functions as the calculation unit 14 to acquire the current position of the UAV 2 . The current position corresponds to, for example, the initial position of the UAV 2 before the UAV 2 starts photographing, or the target position before change when the target position during the photographing by the UAV 2 is changed. Alternatively, the current position corresponds to the last photographing position (target position) when the UAV 2 finishes photographing. The flight control device 1 can also grasp the current position based on the airframe information sent from the control device 21 .

In step S108, the flight control device 1 plays the function of the calculation unit 14, based on the state information of the construction machine 3 acquired in step S104, calculates the value of the parameter (interference parameter) indicating the possibility of interference between the UAV 2 and the construction machine 3 when the UAV 2 moves linearly from the current position to the new target position. Here, for example, the flight control device 1 can calculate the shortest distance between the path and the construction machine 35 when the UAV 2 moves straight from the current position to the new target position (the distance between the above-mentioned path and the operation mechanism 35 when the UAV 2 is closest to the operation mechanism 35) as the above-mentioned parameter.

Here, the position (posture) of the working mechanism 35 is calculated based on the state information of the construction machine 3 , that is, the position information of the construction machine 3 and the posture information of the construction machine 3 . Therefore, for example, the position (orientation) of the working mechanism 35 can be grasped corresponding to the latitude, longitude, and altitude similar to the flight path, for example, without performing image processing or the like on the image captured by the imaging device 22 . Therefore, the load of processing for calculating the above-mentioned parameters can be reduced.

In step S110, the flight control device 1 functions as the calculation unit 14 to determine whether the parameter value (shortest distance) calculated in step S108 is above a predetermined threshold. If the determination is affirmative, the process proceeds to step S112. If the determination is negative, the process proceeds to step S114.

In step S112, the flight control device 1 plays the function of the determination unit 12, and selects the path (the path of the first movement mode (first candidate mode)) of the unmanned aerial vehicle 2 from the current position (also referred to as the original target position) to the new target position in a straight line as a temporary flight path, and the process proceeds to step S116.

In step S114, the flight control device 1 plays the function of the decision unit 12. Based on the state information acquired in step S104, it searches for a path for the UAV 2 to move from the current position to a new target position, that is, a path that avoids interference between the UAV 2 and the operating mechanism 35, and extracts the path. In addition, the flight control device 1 selects the route judged to be the best route from the extracted flight routes as the temporary flight route (the route of the second movement mode (second candidate mode)), and the process proceeds to step S116.

In step S116 , the flight control device 1 functions as the surrounding environment information acquiring unit 15 and acquires surrounding environment information indicating the surrounding environment of the construction machine 3 . Here, for example, the flight control device 1 may send the image captured by the camera 22 sent by the control device 21 to the image processing device 10 ( FIG. 1 ), and obtain surrounding environment information based on the image processing result of the image processing device 10 . The surrounding environment information includes the positions (coordinates) of obstacles that exist around the construction machine 3 and hinder the flight of the UAV 2 . In addition, the installation location of the image processing device 10 is arbitrary, for example, it can be installed on the flight control device 1 , the unmanned aerial vehicle 2 or the construction machine 3 .

In order to acquire surrounding environment information, radar or sonar sensors, etc. may also be installed on the unmanned aerial vehicle 2, instead of the above image processing device 10 or used together with the above image processing device 10 . In this case, the position (coordinates) of the obstacle can be detected by analyzing the radio wave or sound wave reflected by the obstacle in the processing device installed on the unmanned aerial vehicle 2, the flight control device 1, or the construction machine 3, for example.

In step S118, the flight control device 1 plays the function of the determining unit 12. Based on the surrounding environment information acquired in step S116, it is judged whether there is an obstacle on the temporary flight path selected in step S112 or step S114 that will hinder the flight of the UAV 2. If the judgment is affirmative, the processing proceeds to step S120. If the judgment is negative, the processing proceeds to step S124.

In step S120, the flight control device 1 plays the function of the decision unit 12, based on the state information obtained in step S104 and the surrounding environment information obtained in step S116, searches for a flight path that avoids the interference of the UAV 2 and the construction machinery 3 and avoids the interference of the UAV 2 and obstacles, and extracts the flight path.

In step S122, the flight control device 1 plays the function of the determining unit 12 to determine whether the flight path search in step S120 has extracted a flight path that avoids the interference of the UAV 2 and the engineering machinery 3 and avoids the interference of the UAV 2 and obstacles. If the judgment is affirmative, the process proceeds to step S124, and if the judgment is negative, the process proceeds to step S126.

In step S124, the flight control device 1 functions as the determining unit 12 to set the temporary flight path selected in step S112 or step S114, or the flight path selected from the flight paths extracted in step S120 as the final flight path, and the process proceeds to step S130.

On the other hand, in step S126, the flight control device 1 functions as the information output unit 16 to output information indicating that there is no flight path to be set, and the process proceeds to step S102. The information output from the information output unit 16 may include the reason why the flight path cannot be set, for example, information that a flight path within a predetermined distance to the construction machine 3 cannot be selected due to an obstacle. The information output from the information output unit 16 may be displayed on, for example, the image output device 51 or the like, and may include voice output information.

In step S130, the flight control device 1 functions as the control unit 13, and transmits control information for specifying the final flight path set in step S124 to the unmanned aerial vehicle 2, and the process proceeds to step S102. The transmitted information is received by the control device 21 . Under the control of the control device 21, the unmanned aerial vehicle 2 flies along the designated flight path to a new target position.

Therefore, in this embodiment, before the target position is updated (before the determination of step S102 is affirmative), the UAV 2 stays at the target position before the update, and continues to take pictures with the imaging device 22 at the target position. At the target position, the unmanned aerial vehicle 2 maintains a hovering state, and uses the photographing device 22 to photograph the operation (such as dismantling operation) status from a suitable photographing position, and the operator of the construction machine 3 can see the photographed image. Therefore, high-definition captured images can be presented to the operator without lowering the picture quality due to the shaking of the unmanned aerial vehicle 2 , thereby effectively supporting the operation of the construction machine 3 .

In addition, if the UAV 2 can move linearly to the new target position, the flight path for the UAV 2 to move linearly is selected, so that the flight path and the flight time can be minimized. At this time, since the unnecessary movement of the unmanned aerial vehicle 2 is small, the image captured by the photographing device 22 is not easy to shake, and the same image clarity can be maintained.

Also, in the event that a straight line movement to a new target position cannot be selected due to interference from the working mechanism 35, a flight path that bypasses the working mechanism 35 will be automatically selected. Therefore, the unmanned aerial vehicle 2 can be moved to a new target position without imposing a burden on the operator.

In addition, in this embodiment, a flight path avoiding obstacles is automatically selected. Therefore, the unmanned aerial vehicle 2 can be moved to a new target position without imposing a burden on the operator.

Next, the aforementioned current position (step S106) and target position (step S108) will be described.

First, before the unmanned aerial vehicle 2 takes off, the current position is the initial position for setting the unmanned aerial vehicle 2, and the new target position is the initial shooting position.

After the photographing device 22 starts photographing, if the photographing position is changed, the current position is the photographing position before the change, and the new target position is the photographing position after the change.

Also, for example, at the end of the work, the unmanned aerial vehicle 2 flies from the last shooting position to a predetermined position such as the initial position and lands. In this case, the current position is the last shooting position, and the new target position is the position where the UAV 2 lands, such as the initial position.

A position at which the state of the working mechanism 35 can be photographed, or a position at which at least a part of the work object, for example, at least a part of a building to be dismantled can be photographed, is specified as the photographing position. A position where both the state of the working mechanism 35 and the work object can be photographed may be specified as the photographing position.

The flight control device 1 can set the imaging device 22 from the left side of the operating mechanism 35 to the left side of the operating mechanism 35, and the imaging device 22 from the right side of the operating mechanism 35 to the right side of the operating mechanism 35 as the shooting positions. One of the left position and the right position is selected as the target position (shooting position).

The combination of the left position and the right position can be based on the state (position, direction) of the working mechanism 35 when the left position and the right position are set. In this case, the flight control device 1 may set the left position and the right position to be bilaterally symmetrical with respect to the working mechanism 35 .

In addition, the combination of the left side position and the right side position may be based on the state (position, direction) of the undercarriage 31 or the upper slewing body 32 . In this case, the flight control device 1 may set the left and right positions to be bilaterally symmetrical with respect to the lower traveling body 31 or the upper revolving body 32 .

In addition, the combination of the left position and the right position can also be set according to an operator's instruction. Also in this case, the flight control device 1 receives an instruction from the operator, and acts as a determination unit to determine the shooting position. The same goes for other location decisions.

FIG. 5 is a top view of the left and right positions of the UAV 2 . In FIG. 5 , the left position 2L and the right position 2R are set in front of the rotation axis of the boom 35a, that is, on the distal side in the direction in which the working mechanism 35 extends from the base end of the boom 35a, and are symmetrical positions with respect to the working mechanism 35 at the start of work.

A plurality of position combinations can also be prepared for the left position and the right position. In this case, one group may be selected from a plurality of position combinations according to an instruction of the operator or according to the state of the construction machine 3 (position and direction of each part of the construction machine 3 ).

In addition, the combination of the left position and the right position can also be changed in due course according to the state (position, direction) of the working mechanism 35 or according to the state (position, direction) of the lower traveling body 31 or the upper slewing body 32 . In this case, it is possible to secure an appropriate imaging position corresponding to the state of the construction machine 3 .

Select any of the left position and the right position as the shooting position. For example, the operator can choose any position by directing himself. In this case, the operator of the construction machine 3 can select an effective image by grasping, for example, the situation of the work site which cannot be seen directly. For example, in order to obtain a more effective image, the operator can switch the shooting position from the left position to the right position. In addition, the imaging position may be specified as the left position or the right position by the initial setting, and the imaging position specified by the initial setting may be switched according to an instruction of the operator.

In addition, the operator can also select any position among the left and right positions in good time according to the state (position, direction) of the working mechanism 35 or according to the state (position, direction) of the lower traveling body 31 or the upper slewing body 32.

The shooting position is not limited to the left position and the right position, and may be set to an arbitrary position instead of the left position and the right position, or in addition to the left position and the right position. In addition, the number of shooting positions is also arbitrary, and three or more shooting positions can be set, and the shooting positions can be selected in good time according to the instructions of the operator or according to the state of the construction machine 3 (the position and direction of each part of the construction machine 3 ).

Next, parameter values indicating the possibility of interference between the UAV 2 and the construction machine 3 will be described (step S108).

When calculating the above parameter values, the flight control device 1 obtains the shortest distance (minimum distance) between the path and the working mechanism 35 when the UAV 2 moves linearly from the current position to the new target position. The flight control device 1 calculates the state of the working mechanism 35 based on the above-mentioned state information of the construction machine 3 , that is, position information and posture information of the construction machine 3 . As described above, the posture information of the construction machine 3 includes fixed (known) parameters such as the boom length, boom mount position, boom length, and arm length of the construction machine 3 , and parameters that change during the operation, such as the boom angle, boom angle, and arm angle.

Assuming that the shortest distance is less than the predetermined threshold, the unmanned aerial vehicle 2 may touch the working mechanism 35 in the middle of moving straight from the current position to the new target position. Therefore, in this case (when the determination in step S108 is negative), the flight control device 1 does not select a path of linear movement, but searches for a path that avoids detours of the working mechanism 35 (step S114 ). On the other hand, in the case where the above-mentioned shortest distance is above the above-mentioned predetermined threshold (in the case of affirmative judgment in step S108), since the unmanned aerial vehicle 2 cannot contact the operating mechanism 35 in the middle of the straight-line movement, the flight control device 1 selects the path that moves straight from the current position to the new target position as the temporary flight path (step S112).

In the present embodiment, the flight control device 1 calculates the above-mentioned shortest distance as the value of the above-mentioned parameter. However, the above-mentioned parameter may be a binary parameter indicating whether linear movement is possible. For example, instead of calculating the shortest distance, the flight control device 1 can calculate the value of the above parameter (one of the two values) by using an algorithm that determines whether the UAV 2 flying on the branch path will interfere with the engineering machinery 3 . In this case, the time required to select the flight path can be shortened, especially when straight-line movement is possible.

Next, when a straight-line movement path cannot be selected (in the case of negative determination in step S110), a flight path (temporary flight path) that avoids interference between the UAV 2 and the construction machine 3 is selected (step S114).

The flight path that avoids interference between the unmanned aerial vehicle 2 and the construction machine 3 is extracted and selected as a path that can ensure a certain distance between the flight path and the operating mechanism 35 . In this case, the shortest distance between the flight path and the working mechanism 35 may be set to, for example, the same as or a distance greater than the above-mentioned predetermined threshold. In addition, the shortest distance between the unmanned aerial vehicle 2 and the operating mechanism 35 can be set with different lower limits according to different parts of the boom 35a or stick 35b, etc., and the shortest distance between each part and the unmanned aerial vehicle 2 is selected. The flight path is greater than the respective lower limit. For example, since the arm 35b (crusher 35c) generally moves faster than the boom 35a, the lower limit of the shortest distance of the arm 35b (crusher 35c) may be set to a value larger than the lower limit of the shortest distance of the boom 35a.

The trajectory of the flight path can be set as, for example, a flight path including passing points. In this case, the flight control device 1 can set a path sequentially connected by a straight line between the current position, the passing point, and the target position. One or more waypoints can be set.

FIG. 6A is a diagram showing a flight path passing through two via points. FIG. 6A shows the flight path of the unmanned aerial vehicle 2 moving from the left position 2L to the right position 2R through the passing point 201 and the passing point 202 in sequence. In this case, after the UAV 2 rises straight from the left position 2L to the passing point 201, it flies straight and horizontally to the passing point 202, and then descends straight to the right position 2R, thereby avoiding interference with the stick 53b.

FIG. 6B is a diagram showing a flight path passing through one via point. FIG. 6B shows the flight path of the UAV 2 moving from the left position 2L to the right position 2R via the waypoint 203 . In this case, the UAV 2 flies straight and horizontally from the left position 2L to the waypoint 203, and then flies straight and horizontally to the right position 2R, thereby avoiding interference with the boom 53a.

In addition, the flight path may be set as a curved path, such as an arc-shaped path, which can ensure a certain distance or more from the working mechanism 35 throughout the flight path. For example, the flight control device 1 can set an arc-shaped path avoiding the working mechanism 35 , such as the path 101 shown in FIG. 5 . The path 101 shown in FIG. 5 represents the path of the UAV 2 flying horizontally from the left position 2L to the right position 2R.

FIG. 6C is a diagram showing an example of a flight path set at the start of work. In FIG. 6C , it shows the flight path 102 of the UAV 2 at the initial position ₃₀₁ moving to the left position 2L as the target position. In this example, the UAV 2 ascends from the initial position 301 to the waypoint 204 , and then flies horizontally to the waypoint 205 . Then, the UAV 2 ascends from the passing point 205 to the left position 2L. In this case, if the unmanned aerial vehicle 2 moves linearly from the initial position 301 to the left position 2L, it may interfere with the boom 53a. However, the flight control device 1 can prevent the UAV 2 from touching the boom 53 a by selecting the flight path 102 passing through the waypoint 204 and the waypoint 205 . In this case, a distance Δ equal to or greater than a threshold (predetermined threshold) having the shortest distance set for avoiding contact is ensured between the selected flight path 102 and the boom 53a.

The flight control device 1 can select multiple flight paths as flight paths for avoiding interference between the UAV 2 and the engineering machinery 3 , but it is preferable to select a path with a relatively short flight distance or a relatively short moving time.

As one of the selection methods, the flight control device 1 may consider selecting the shortest flight path among flight paths that avoid interference between the UAV 2 and the engineering machinery 3 . For example, in the case of setting a route that sequentially connects the current position, the passing points, and the target position with straight lines, the flying distance can be shortened by increasing the number of passing points. On the other hand, if the number of waypoints is increased, the number of times of changing direction or stopping at the waypoints will increase, and the travel time will become longer. Therefore, the flight control device 1 preferably selects a flight path that meets the required conditions. For example, in the case of emphasizing the movement time, the flight control device 1 can select the flight path with the shortest movement time among the flight paths that avoid the interference between the UAV 2 and the engineering machinery 3 .

Next, a method of setting a flight path when there is an obstacle that hinders the flight of the unmanned aerial vehicle 2 on the temporary flight path (when the determination in step S118 is affirmative) will be described.

In this case, the flight control device 1 sets a flight path in which the shortest distance between the flight path and the obstacle is greater than or equal to a predetermined threshold among the flight paths capable of avoiding interference between the UAV 2 and the construction machine 3 . Thereby, the possibility that the UAV 2 touches an obstacle can be excluded. In addition, the flight path may be set to shorten the flight distance, especially to minimize the flight distance, or may be set to shorten the flight time, especially to minimize the flight time. The flight path may be a combination of straight-line paths connecting passing points, or may be composed of curves such as arc-shaped curves.

Next, the flight path when the cable is connected to the unmanned aerial vehicle 2 will be described.

FIG. 7A and FIG. 7B are diagrams showing examples of the flight path in the case where the UAV 2 is connected with a cable. The cable 400 is, for example, a cable for supplying power to the UAV 2 and a cable for communication between the UAV 2 and the construction machinery 3 . In addition, the case where the cable 400 is used as a rope for tethering the UAV 2 to limit its flight range is also included.

The example in FIG. 7A shows the flight path of the UAV 2 moving from the initial position 301 to the initial shooting position, that is, the right position 2R. In this case, after the UAV 2 rises straight from the initial position 301 to the passing point 207, it flies straight and horizontally toward the right position 2R and reaches the right position 2R. In this case, a passing point 207 is provided on the flight path, so that not only the UAV 2 will not come into contact with the boom 53a or the stick 53b, but also the cable 400 extending from the UAV 2 will not come into contact with the boom 53a or the stick 53b and become entangled.

The example of FIG. 7B shows the flight path of the UAV 2 moving from the left position 2L to the right position 2R. In this case, the unmanned aerial vehicle 2 flies straight and horizontally from the left position 2L to the passing point 208 , then flies straight and horizontally to the right position 2R and reaches the right position 2R. In this case, the passing point 208 is set on the flight path so that not only the UAV 2 will not come into contact with the boom 53a or the stick 53b, but also the cable 400 extending from the UAV 2 will not come into contact with the boom 53a or the stick 53b and become entangled.

Thus, when the UAV 2 is connected to the cable 400, if the UAV 2 moves straight from the current position to the target position, even if the UAV 2 itself cannot interfere with the operating mechanism 35, the cable 400 may still contact the operating mechanism 35. At this time, choose a flight path that detours from the current position to the target position to avoid contact.

In addition, in order to prevent the cable 400 from being loosened excessively, for example, a mechanism such as a winding machine for winding the cable 400 with a constant tension may be provided. In this case, the cable 400 has a shape close to a straight line, and therefore, the risk that the cable 400 blown by the wind will come into contact with the working mechanism 35 inadvertently can be eliminated.

As mentioned above, in this embodiment, before the target position is updated, the UAV 2 stays at the target position before the update, and continues to take pictures with the imaging device 22 at the target position. Therefore, high-definition captured images can be presented to the operator without lowering the picture quality due to the shaking of the unmanned aerial vehicle 2 , thereby effectively supporting the operation of the construction machine 3 .

In addition, when the UAV 2 can move straight to the new target position, the flight path such as the UAV 2 moving straight is selected, so the shortest flight path and the shortest flight time can be realized. Moreover, when the UAV 2 cannot choose to move straight to the new target position due to interference between the UAV 2 and the operating mechanism 35 , the UAV 2 automatically selects a flight path that bypasses the operating mechanism 35 or the like. Therefore, the unmanned aerial vehicle 2 can be moved to a new target position without imposing a burden on the operator.

As mentioned above, although each embodiment of this invention was described in detail, this invention is not limited to this specific embodiment, Various deformation|transformation and changes are possible within the scope described in a claim. In addition, all or a plurality of components of each of the above-described embodiments may be combined.

For example, in the above-mentioned embodiment, the embodiment in which the support device of the present invention is applied to support the actual operation of the construction machine 1 has been described, but the support device of the present invention can also be applied to support the inspection of the construction machine 1 itself, the running of the construction machine 1, or other actions. For example, the assisting device of the present invention can be applied to assisting inspection work of tall objects such as attachments when the machine is stopped, or monitoring the surroundings of a traveler who moves the construction machine 1 . As mentioned above, the content of the above-mentioned embodiment can also be applied to various support of the construction machine 1.

The present invention provides a support device that supports construction machinery based on image information provided by a camera mounted on an unmanned aerial vehicle. The assisting device includes: a state information acquisition unit that acquires state information indicating the state of the construction machine; a determination unit that determines a target position of the UAV and a movement method of the UAV to the target position; and a control unit that controls the movement of the UAV to the target position based on the movement method determined by the determination unit, wherein the determination unit determines the movement method based on the state information to avoid interference between the UAV and the construction machine.

In the above configuration, the construction machine may include: a lower running body; an upper revolving body mounted on the lower running body; and a working mechanism provided on the upper revolving body, and the state information includes posture information indicating the posture of at least one of the construction machine and the working mechanism.

In the above structure, a calculation unit may further be included, which calculates a value of a parameter based on the state information, and the parameter indicates the possibility of interference between the UAV and the construction machine when the UAV moves linearly from the current position to the target position, wherein the determination unit determines the movement mode based on the value of the parameter calculated by the calculation unit.

In the above configuration, the determination unit may determine the target position as a position where the imaging device can capture the working mechanism.

In the above configuration, the determination unit may determine the target position as a position at which the imaging device can capture a part of the working mechanism or the work object.

In the above configuration, the target position may include a left position and a right position. The left position is a position where the imaging device can capture the working mechanism from the left side of the working mechanism (more specifically, the left side in the left-right direction of the upper revolving body), and the right position is a position where the imaging device can capture the working mechanism from the right side of the working mechanism (more specifically, the right side in the left-right direction of the upper revolving body). way. For example, when the UAV is located on the right side of the working mechanism and moves to the target position set on the left side of the working mechanism, or, when the UAV is located on the left side of the working mechanism and moves to the target position set on the right side of the working mechanism, the determination unit determines the movement method based on the state information, so as to avoid interference between the UAV and the construction machinery.

In the above structure, when the determination unit determines the target position at a specified time, the determined target position is used as the new target position, and the target position before the determined target position is used as the original target position, then the determination unit determines one of the candidate modes selected from the first candidate mode and the second candidate mode as the movement mode. How to move via the point to the new target location.

In the above structure, the second candidate method may also include linear movement of the UAV from the original target position to the waypoint, and linear movement of the UAV from the waypoint to the new target position.

In the above configuration, the determination unit may determine the passing point based on the status information when the second candidate mode is selected.

In the above configuration, the determination unit may determine the passing point such that the shortest distance between the UAV and the construction machine when moving based on the second candidate method is greater than the shortest distance between the UAV and the construction machine when moving based on the first candidate method.

In the above configuration, a surrounding environment information acquiring unit may be included to acquire surrounding environment information indicating a surrounding environment of the construction machine, wherein the determining unit determines the movement method based on the state information and the surrounding environment information.

The above configuration may further include an information output unit configured to output predetermined information when the determination unit cannot determine the movement method.

In addition, the invention also provides a system. The system includes at least one of the construction machine and the unmanned aerial vehicle, and the above-mentioned supporting device.

Claims (13)

1. A supporting device, characterized in that, based on the image information provided by the camera mounted on the unmanned aerial vehicle, the supporting construction machine is supported, and the supporting device includes: a status information acquiring unit that acquires status information indicating the status of the construction machine; a determination unit that determines a target position of the UAV and a movement method of the UAV to the target position; and, a control unit that controls movement of the UAV to the target position based on the movement manner determined by the determination unit, wherein, The determination unit determines the movement method based on the state information, so as to avoid interference between the UAV and the construction machine. 2. The support device according to claim 1, wherein: The construction machinery includes: lower walking body; an upper revolving body mounted on the lower running body; and, The working mechanism is arranged on the upper slewing body, wherein, The status information includes posture information indicating a posture of at least one of the construction machine and the work mechanism. 3. The supporting device according to claim 2, further comprising: A calculation unit that calculates a value of a parameter based on the state information, the parameter representing the possibility of interference between the UAV and the construction machine when the UAV moves linearly from the current position to the target position, wherein, The determination unit determines the movement method based on the value of the parameter calculated by the calculation unit. 4. The support device according to claim 2 or 3, wherein: The determination unit determines the target position as a position where the imaging device can capture the working mechanism. 5. The support device according to claim 2 or 3, wherein: The determination unit determines the target position as a position at which the imaging device can capture a part of the working mechanism or the work object. 6. The support device according to claim 4, wherein: The target position includes a left position and a right position, the left position is a position where the photographing device can photograph the operating mechanism from the left side of the operating mechanism, and the right position is a position where the photographing device can photograph the operating mechanism from the right side of the operating mechanism, The determining unit determines the movement method when the target position changes between the left position and the right position. 7. The supporting device according to any one of claims 1 to 3, characterized in that, When the determination unit determines the target position at a predetermined time, the determined target position is used as the new target position, and the target position before the determined target position is used as the original target position, then the determination unit determines one of the candidate modes selected from the first candidate mode and the second candidate mode as the movement mode, The first candidate method is a method in which the UAV moves from the original target position to the new target position in a straight line, and the second candidate method is a method in which the UAV moves from the original target position to the new target position via a specified waypoint. 8. The support device according to claim 7, wherein: The second candidate way includes linear movement of the UAV from the original target position to the waypoint, and linear movement of the UAV from the waypoint to the new target position. 9. The support device according to claim 7, wherein: The determining unit determines the passing point based on the status information when the second candidate mode is selected. 10. The support device according to claim 9, wherein: The determination unit determines the passing point such that the shortest distance between the UAV and the construction machine when moving based on the second candidate method is greater than the shortest distance between the UAV and the construction machine when moving based on the first candidate method. 11. The support device according to any one of claims 1 to 3, further comprising: The surrounding environment information acquisition unit acquires surrounding environment information indicating the surrounding environment of the construction machine, wherein, The determination unit determines the movement method based on the state information and the surrounding environment information. 12. The support device according to any one of claims 1 to 3, further comprising: The information output unit outputs predetermined information when the determination unit cannot determine the movement method. 13. A system characterized by comprising: A support device as claimed in any one of claims 1 to 12; and, At least one of the construction machine and the unmanned aerial vehicle.