CN116466691A

CN116466691A - Support device and system including support device

Info

Publication number: CN116466691A
Application number: CN202310038081.3A
Authority: CN
Inventors: 坂本丈尚; 福田智宙
Original assignee: Kobelco Construction Machinery Co Ltd
Current assignee: Kobelco Construction Machinery Co Ltd
Priority date: 2022-01-20
Filing date: 2023-01-10
Publication date: 2023-07-21
Also published as: JP2023105836A

Abstract

The invention provides a supporting device for supporting a construction machine based on image information provided by an imaging device mounted on an unmanned aerial vehicle. The support device includes: a state information acquisition unit that acquires state information indicating a state of the construction machine; a determining unit that determines a target position of the unmanned aerial vehicle and a movement pattern of the unmanned aerial vehicle to the target position; and a control unit that controls movement of the unmanned aerial vehicle to the target position based on the movement pattern determined by the determination unit. The determination unit determines the movement pattern based on the state information so as to avoid interference between the unmanned aerial vehicle and the construction machine. Accordingly, the unmanned aerial vehicle and the construction machine can be prevented from interfering with each other, and the construction machine can be supported based on the image information provided by the imaging device mounted on the unmanned aerial vehicle.

Description

Support device and system including support device

Technical Field

The present invention relates to a support apparatus and a system including the support apparatus.

Background

Conventionally, a technique for capturing an image of a space that cannot be captured by an imaging device mounted on an upper revolving structure of a construction machine by using an imaging device mounted on an autonomous aircraft has been known (for example, international publication No. WO 2017/131194).

However, in the above-described technique, the aircraft may interfere with the construction machine.

Disclosure of Invention

The purpose of the present invention is to support a construction machine based on image information provided by an imaging device mounted on an unmanned aerial vehicle while avoiding interference between the unmanned aerial vehicle and the construction machine.

The invention provides a supporting device for supporting a construction machine based on image information provided by an imaging device mounted on an unmanned aerial vehicle. The support device includes: a state information acquisition unit that acquires state information indicating a state of the construction machine; a determining unit configured to determine a target position of the unmanned aerial vehicle and a movement pattern of the unmanned aerial vehicle to the target position; and a control unit configured to control movement of the unmanned aerial vehicle to the target position based on the movement pattern determined by the determination unit, wherein the determination unit determines the movement pattern based on the state information so as to avoid interference between the unmanned aerial vehicle and the construction machine.

In addition, the invention also provides a system. The system includes at least one of the construction machine and the unmanned aerial vehicle, and the support device.

According to the present invention, the unmanned aerial vehicle and the construction machine can be prevented from interfering with each other, and the construction machine can be supported based on the image information provided by the imaging device mounted on the unmanned aerial vehicle.

Drawings

Fig. 1 is an exemplary diagram of a system including a support device according to an embodiment of the present invention.

Fig. 2 is a functional diagram of a configuration of a support device according to an embodiment of the present invention.

Fig. 3 is a diagram showing an example of a hardware configuration of a control system of a construction machine according to an embodiment of the present invention.

Fig. 4 is a flowchart showing the operation of the flight control device according to the embodiment of the present invention.

Fig. 5 is a top view of a left side position and a right side position of an unmanned aerial vehicle according to an embodiment of the present invention.

Fig. 6A is a diagram showing a flight path of an unmanned aerial vehicle through two passing points.

Fig. 6B is a diagram showing a flight path of an unmanned aerial vehicle through a pass-by point.

Fig. 6C is an exemplary diagram of a flight path of the unmanned aerial vehicle set at the start of a mission.

Fig. 7A is an example diagram of a flight path of an unmanned aerial vehicle with a cable connected.

Fig. 7B is an example diagram of a flight path of an unmanned aerial vehicle with a cable connected.

Detailed Description

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

Fig. 1 is an exemplary diagram of a configuration of a system including a support device according to an embodiment of the present invention, and fig. 2 is a functional diagram of a configuration of the support device according to the present embodiment.

As shown in fig. 1, the system including the support device of the present embodiment includes, for example, a flight control device 1, an unmanned aerial vehicle 2, and a construction machine 3. In fig. 1, the flight control device 1 and the unmanned aerial vehicle 2 and the flight control device 1 and the construction machine 3 are connected via a network. In this case, the flight control device 1 may be constituted by a server (server computer, management device), for example. In this case, the contracted configuration of the network is arbitrary, and the network may include a wireless communication network, the internet, VPN (Virtual Private Network), WAN (Wide Area Network), a wired network, or any combination thereof, or the like.

The support device according to the present embodiment is a device for supporting the work of the construction machine 3 based on the image information transmitted from the imaging device 22 mounted on the unmanned aerial vehicle 2. In the present embodiment, the function of the support device is provided in the flight control device 1. Further, all functions of the support device may be provided in either one of the unmanned aerial vehicle 2 and the construction machine 3. The functions of the support device may be separately provided in any two or three of the flight control device 1, the unmanned aerial vehicle 2, and the construction machine 3.

Further, a device having some or all of the functions of the support device may be mounted on the construction machine 3.

As shown in fig. 2, the flight control device 1 constituting the support device according to the present embodiment includes a state information acquisition unit 11, a determination unit 12, a control unit 13, a calculation unit 14, a surrounding environment information acquisition unit 15, and an information output unit 16. The state information acquisition unit 11 acquires state information indicating the state of the work machine 3. The determination unit 12 determines a target position of the unmanned aerial vehicle 2 and a movement pattern (e.g., a route) in which the unmanned aerial vehicle 2 moves to the target position. The control unit 13 controls the movement of the unmanned aerial vehicle 2 to the target position based on the movement pattern determined by the determination unit 12. The calculation unit 14 calculates a parameter value indicating the possibility that the unmanned aerial vehicle 2 interferes with the construction machine 3 when the unmanned aerial vehicle 2 moves straight from the current position to the target position, based on the state information acquired by the state information acquisition unit 11. The surrounding environment information acquiring unit 15 acquires surrounding environment information indicating the surrounding environment of the work machine 3. The information output unit 16 outputs predetermined information when the determination unit 12 cannot determine the movement pattern.

The unmanned aerial vehicle 2 is, for example, a rotorcraft, and in this case, includes a plurality of rotatable blades, an electric motor (actuator) for rotating the plurality of blades, a battery for supplying power to the electric motor, and the like. Alternatively, the power supply line may be connected from the ground to the unmanned aerial vehicle 2 instead of or in addition to the battery.

The unmanned aerial vehicle 2 further comprises a control device 21 and a camera device 22.

The control device 21 controls the flight state (forward state, backward state, upward state, downward state, hover, etc.) of the unmanned aerial vehicle 2 based on the control information from the flight control device 1, and guides the unmanned aerial vehicle 2 to the target position. The target position is expressed in terms of latitude, accuracy, and altitude, for example. In addition, the control device 21 also controls the attitude of the unmanned aerial vehicle 2 at the target position.

The control device 21 acquires fuselage information indicating various states related to the fuselage of the unmanned aerial vehicle 2, and controls the flight state of the unmanned aerial vehicle 2 based on the fuselage information. The fuselage information includes position information of the unmanned aerial vehicle 2, attitude information of the unmanned aerial vehicle 2, and the like. The positional information of the unmanned aerial vehicle 2 is expressed in terms of, for example, latitude, longitude, and altitude. Such positional information of the unmanned aerial vehicle 2 may be acquired by a GPS sensor. The attitude information of the unmanned aerial vehicle 2 includes, for example, information about the heading axis, the yaw axis, and the pitch axis of the unmanned aerial vehicle 2, which are rotated about the respective axes. Such attitude information of the unmanned aerial vehicle 2 may be acquired by a sensor mounted on the unmanned aerial vehicle 2, for example, an inertial measurement unit (IMU: inertial Measurement Unit).

The control device 21 also has a transmission function of transmitting the image acquired by the imaging device 22 to the construction machine 3 or the flight control device 1 via a transmission unit, and a reception function of receiving control information transmitted from the flight control device 1 via a reception unit (not shown).

The camera 22 includes a camera mounted on the unmanned aerial vehicle 2. The type of the camera is arbitrary, and for example, a wide-angle camera may be used. The imaging device 22 acquires a front environmental image in front of the fuselage of the unmanned aerial vehicle 2 using an imaging element such as a CCD (charge-Coupled device) or CMOS (complementary metal oxide semiconductor). The imaging device 22 may acquire the front environmental image in real time, for example, and supply the image to the control device 21 in a stream format of a predetermined frame period.

The imaging device 22 preferably includes a gimbal (not shown). The gimbal functions to maintain a fixed direction (for example, a predetermined direction in a horizontal plane) of the optical axis of the imaging device 22 even when the attitude of the unmanned aerial vehicle 2 changes.

The construction machine 3 cooperates with the unmanned aerial vehicle 2 and performs a predetermined operation. The construction machine 3 is, for example, a construction machine provided with a crusher suitable for a dismantling operation or the like, and includes a crawler-type lower traveling body 31 and an upper revolving body 32 rotatably mounted on the lower traveling body 31. A cab (cab) 34 is provided on the front left side of the upper revolving unit 32. A working mechanism 35 is provided in the front center of the upper revolving structure 32. The construction machine 3 may be a construction machine having a crane function, or may be another construction machine such as a crawler crane.

The work mechanism 35 includes a boom 35a attached to the upper revolving structure 32 and capable of ascending and descending, an arm 35b rotatably coupled to a distal end of the boom 35a, and a breaker 35c attached to a distal end of the arm 35 b. As is well known, the boom 35a, the arm 35b, and the crusher 35c are driven by a hydraulic cylinder or the like provided as a part of the work mechanism 35.

The construction machine 3 may be provided with a apron for taking off and landing the unmanned aerial vehicle 2.

Fig. 3 is a diagram showing an example of a hardware configuration of a control system of the construction machine according to the present embodiment.

As shown in fig. 3, the work machine 3 includes a control device 40 and peripheral equipment 50.

The control device 40 includes CPU (Central Processing Unit), RAM (Random Access Memory), 42, ROM (Read Only Memory), support storage device 44, drive device (reading device) 45, communication interface 47, and wired transmitting/receiving section 48 and wireless transmitting/receiving section 49 connected to the communication interface 47, which are connected via bus B.

The support storage device 44 is, for example, a HDD (Hard Disk Drive), an SSI (solid state Drive), or the like (Solid State Drive), and is a storage device for storing data associated with application software or the like.

The wired transceiver 48 includes a transceiver capable of communicating using a wired network. The wired transceiver section 48 is connected to the peripheral device 50. A part or all of the peripheral devices 50 may be connected to the bus B or the wireless transceiver 49.

The wireless transceiver 49 is a transceiver capable of communicating using a wireless network. The wireless network includes the wireless communication network of the mobile phone, the internet, VPN, WAN, etc. The wireless transceiver 49 may include a Near Field Communication (NFC) unit, a Bluetooth (registered trademark) unit, a Wi-Fi (wireless fidelity, registered trademark) unit, an infrared transceiver, and the like. The wireless transceiver 49 can communicate with the flight control device 1 in the form of a server. The wireless transceiver 49 also receives the image acquired by the imaging device 22 and transmitted from the control device 21.

The peripheral device 50 is an electronic control-capable device, various sensors, an operation unit, and the like mounted on the work machine 3. The peripheral device 50 may include, for example, an image output device 51, a buzzer, a voice output device (not shown), a hydraulic pressure generating device (not shown) that operates the work mechanism 35, various sensors 52 that detect the operation states of various operation members, and the like, and an operation portion 53 that further takes over the operation. In addition, the hydraulic pressure generating device may be a hydraulic pump driven by the engine and/or the electric motor. When a hydraulic pump driven by an electric motor is used, the hydraulic pressure generating device may include an inverter (inverter) for driving the electric motor.

The various types of sensors 52 include a gyro sensor, a GPS (global positioning system) sensor, various angle sensors, and an acceleration sensor (inclination sensor). The GPS sensor acquires position information of the construction machine 3. The positional information of the construction machine 3 is expressed by latitude, longitude, and altitude. The GPS sensor includes a GPS receiver, and calculates latitude, longitude, and altitude by interfering with a side position or the like based on radio waves from satellites.

The various sensors 52 include various sensors that acquire parameters related to the posture of the work machine 3, and the various sensors acquire these parameters as posture information of the work machine 3. In this case, the various sensors that acquire parameters related to the posture are, for example, a boom angle sensor, an arm angle sensor, a bucket angle sensor, a body inclination sensor, and the like. The boom angle sensor is a sensor for acquiring a boom angle, and includes, for example, a rotation angle sensor for detecting a rotation angle of a boom foot pin, a stroke sensor for detecting a stroke amount of a boom cylinder, a tilt (acceleration) sensor for detecting a tilt angle of the boom 35a, and the like. The same applies to the arm angle sensor and the bucket angle sensor. The body inclination sensor is a sensor that obtains the inclination angle of the body, and detects the inclination angle of the upper revolving unit 32 with respect to the horizontal plane, for example.

The various sensors 52 also include various sensors that acquire parameters related to the rotation angle of the upper revolving structure 32 with respect to the lower traveling structure 31, which are posture information of the construction machine 3, and these sensors acquire rotation angle information indicating the rotation angle of the upper revolving structure 32. In this case, the various sensors for acquiring parameters related to the rotation angle of upper revolving structure 32 may be, for example, a geomagnetic sensor, a rotation angle sensor (for example, a resolver or the like) for detecting the rotation angle of a revolving mechanism for revolving upper revolving structure 32 with respect to lower traveling structure 31 about a revolving shaft, a gyro sensor, or the like.

The posture information includes fixed (known) parameters such as the boom length, the boom mount position, the boom length, the arm length, and the like of the construction machine 3 other than the parameters acquired by the various sensors 52, and parameters that change during the work such as the boom angle, and the arm angle acquired by the various sensors 52.

The positional information of the construction machine 3 and the posture information of the construction machine 3 correspond to the state information of the construction machine 3.

An image output device 51 is provided in the cab 34 so as to be visible to an operator of the work machine 3. The image output device 51 displays the image acquired by the imaging device 22 and received by the wireless transceiver 49. Thus, the operator of the work machine 3 can grasp, for example, a situation of the work site or the like that is not directly seen from the front environmental image on the image output device 51.

The structure of the image output apparatus 51 is arbitrary, and may be, for example, a liquid crystal display, an organic EL (electroluminescence) display, or the like. In addition, as another embodiment, the image output device 51 may be a portable device (for example, a tablet terminal or the like) that is brought into the cab 34 by an operator of the construction machine 3.

Instead of displaying the image output from the image output apparatus 51 on a display apparatus provided in the image output apparatus 51, another display apparatus may be used. In this case, the output object of the image output device 51 is arbitrary, and examples thereof include a display device such as a dashboard provided in the cab of the construction machine, a terminal device such as a tablet held by an operator, and a management screen managed by a field supervisor.

The operation unit 53 is provided in the cab 34, and receives an instruction operation by an operator of the work machine 3. The instruction operation may include an instruction operation to the flight control device 1, and the instruction operation to the flight control device 1 is transmitted to the flight control device 1 via the wireless transceiver 49.

The control device 40 may be connected to the storage medium 46. The storage medium 46 stores a predetermined program. The program stored in the storage medium 46 is installed to the support storage device 44 of the control device 40 via the drive device 45. The installed predetermined program may be executed by the CPU41 of the control device 40. For example, the storage medium 46 may be a storage medium that stores information optically, electrically, magnetically, such as a CD (compact disc) -ROM, a floppy disk, an optical disk, or the like, a semiconductor memory that stores information electrically, such as a ROM, a flash memory, or the like. In addition, the storage medium 46 does not contain a carrier wave.

Fig. 4 is a flowchart showing the operation of the flight control device 1 according to the present embodiment.

After the flight control of the unmanned aerial vehicle 2 is started, in step S102 of fig. 4, the flight control device 1 determines whether or not a new target position of the unmanned aerial vehicle 2 has been set, and when the determination is affirmative, the process proceeds to step S104. Here, when an instruction operation by the operator or the like to the operation unit 53, that is, an operation to instruct the change of the target position, is received, the determination at step S102 is affirmative.

The setting of the new target position described above includes setting of the new target position at the take-off of the unmanned aerial vehicle 2. The setting of the new target position includes setting of the changed target position when the target position is changed in the middle of shooting by the unmanned aerial vehicle 2. The method further includes setting a landing place at the time of landing of the unmanned aerial vehicle 2 as a target position.

In step S104, the flight control device 1 functions as the status information acquisition unit 11 to acquire status information indicating the status of the construction machine 3. As described above, the state information of the construction machine 3 corresponds to the position information of the construction machine 3 and the posture information of the construction machine 3.

In step S106, the flight control device 1 functions as the calculation unit 14 to acquire the current position of the unmanned aerial vehicle 2. The current position corresponds to, for example, an initial position of the unmanned aerial vehicle 2 before the unmanned aerial vehicle 2 starts shooting, and a target position before the unmanned aerial vehicle 2 is changed when the target position is changed in the middle of shooting. Alternatively, the current position corresponds to a last shooting position (target position) in the case where the unmanned aerial vehicle 2 ends shooting. The flight control device 1 may grasp the current position based on the body information transmitted from the control device 21.

In step S108, the flight control device 1 functions as the calculation unit 14, and calculates a value of a parameter (disturbance parameter) indicating a possibility that the unmanned aerial vehicle 2 and the construction machine 3 will interfere when the unmanned aerial vehicle 2 moves straight from the current position to the new target position, based on the state information of the construction machine 3 acquired in step S104. Here, for example, the flight control device 1 may calculate, as the above-described parameter, the shortest distance between the path of the unmanned aerial vehicle 2 in the case of moving straight from the current position to the new target position and the construction machine 35 (the distance between the above-described path and the construction machine 35 when the unmanned aerial vehicle 2 is closest to the construction machine 35).

Here, the position (posture) of the work machine 35 is calculated based on the state information of the work machine 3, that is, the position information of the work machine 3 and the posture information of the work machine 3. Therefore, for example, even if the image captured by the imaging device 22 is not subjected to image processing or the like, the position (posture) of the working mechanism 35 can be grasped in correspondence with the latitude, longitude, and altitude, for example, similarly to the flight path. Therefore, the load of the processing for calculating the above-described parameters can be reduced.

In step S110, the flight control device 1 functions as the calculation unit 14, determines whether or not the parameter value (shortest distance) calculated in step S108 is equal to or greater than a predetermined threshold, and if the determination is affirmative, the process proceeds to step S112, and if the determination is negative, the process proceeds to step S114.

In step S112, the flight control device 1 functions as the determination unit 12, and selects, as a temporary flight path, a path (a path of the first movement method (first candidate method)) in which the unmanned aerial vehicle 2 is linearly moved from the current position (also referred to as the original target position) to the new target position, and the process proceeds to step S116.

In step S114, the flight control device 1 functions as the determining unit 12, searches for a path along which the unmanned aerial vehicle 2 moves from the current position to the new target position, that is, a path along which interference between the unmanned aerial vehicle 2 and the working mechanism 35 is avoided, based on the state information acquired in step S104, and extracts the path. Further, the flight control device 1 selects a route determined to be the optimal route from the extracted flight routes as a temporary flight route (a route of the second movement method (the second candidate method)), and the process proceeds to step S116.

In step S116, the flight control device 1 functions as the surrounding environment information acquiring unit 15 to acquire surrounding environment information indicating the surrounding environment of the construction machine 3. Here, for example, the flight control device 1 may transmit the image acquired by the camera 22 transmitted from the control device 21 to the image processing device 10 (fig. 1), and acquire the surrounding environment information based on the image processing result of the image processing device 10. The surrounding environment information includes the position (coordinates) of an obstacle that exists around the construction machine 3 and that may interfere with the flight of the unmanned aerial vehicle 2. The installation position of the image processing device 10 is arbitrary, and may be installed in the flight control device 1, the unmanned aerial vehicle 2, or the construction machine 3, for example.

In order to acquire the surrounding environment information, a radar or sonar sensor or the like may be provided in the unmanned aerial vehicle 2 instead of the image processing apparatus 10 or in combination with the image processing apparatus 10. In this case, the position (coordinates) of the obstacle can be detected by analyzing the electric wave or the acoustic wave reflected by the obstacle in a processing device provided in, for example, the unmanned aerial vehicle 2, the flight control device 1, the construction machine 3, or the like.

In step S118, the flight control device 1 functions as the determination unit 12, determines whether or not an obstacle that would interfere with the flight of the unmanned aerial vehicle 2 exists on the temporary flight path selected in step S112 or step S114 based on the surrounding environment information acquired in step S116, and if the determination is affirmative, the processing proceeds to step S120, and if the determination is negative, the processing proceeds to step S124.

In step S120, the flight control device 1 performs the function of the determination unit 12, searches for a flight path that avoids interference between the unmanned aerial vehicle 2 and the construction machine 3 and also avoids interference between the unmanned aerial vehicle 2 and an obstacle, based on the state information acquired in step S104 and the surrounding environment information acquired in step S116, and extracts the flight path.

In step S122, the flight control device 1 performs the function of the determination unit 12, determines whether or not the flight path search in step S120 is performed, extracts a flight path in which interference between the unmanned aerial vehicle 2 and the construction machine 3 is avoided and interference between the unmanned aerial vehicle 2 and the obstacle is avoided, and if the determination is affirmative, the process proceeds to step S124, and if the determination is negative, the process proceeds to step S126.

In step S124, the flight control device 1 functions as the determination unit 12, and sets the temporary flight path selected in step S112 or step S114 or the flight path selected in the flight path extracted in step S120 as the final flight path, and the process proceeds to step S130.

On the other hand, in step S126, the flight control device 1 functions as the information output unit 16, outputs information indicating that there is no flight path to be set, and the process proceeds to step S102. The information output from the information output unit 16 may include information that the flight path cannot be set, for example, the flight path having a distance within a predetermined range from the construction machine 3 cannot be selected due to the presence of an obstacle. The information output from the information output unit 16 may be displayed on the image output device 51 or the like, or may include voice output information.

In step S130, the flight control device 1 functions as the control unit 13, and transmits control information for specifying the final flight path set in step S124 to the unmanned aerial vehicle 2, and the process proceeds to step S102. The transmitted information is received by the control device 21. The unmanned aerial vehicle 2 flies along the designated flight path under the control of the control device 21 to fly to a new target position.

In this way, in the present embodiment, before the target position is updated (before the determination in step S102 is affirmative), the unmanned aerial vehicle 2 remains at the target position before the update, and the imaging device 22 continues to perform the imaging at the target position. At the target position, the unmanned aerial vehicle 2 maintains a hover state, and the operator of the construction machine 3 can see the photographed image by photographing the condition of the work (e.g., disassembly work) from the appropriate photographing position by the photographing device 22. Therefore, the high-definition photographed image can be presented to the operator without lowering the image quality due to the shake of the unmanned aerial vehicle 2, and the work of the construction machine 3 can be efficiently supported.

In addition, when the unmanned aerial vehicle 2 can be linearly moved to the new target position, the flight path for linearly moving the unmanned aerial vehicle 2 is selected, and therefore, the minimization of the flight path and the minimization of the flight time can be achieved. At this time, since the unmanned aerial vehicle 2 is less likely to move unnecessarily, the image captured by the imaging device 22 is less likely to shake, and the same image sharpness can be maintained.

Further, when the linear movement to the new target position cannot be selected due to the disturbance of the working mechanism 35, the flight path bypassing the working mechanism 35 is automatically selected. Therefore, the unmanned aerial vehicle 2 can be moved to a new target position without burdening the operator.

In the present embodiment, a flight path avoiding an obstacle is automatically selected. Therefore, the unmanned aerial vehicle 2 can be moved to a new target position without burdening the operator.

Next, the current position (step S106) and the target position (step S108) described above are described.

First, at the time before the unmanned aerial vehicle 2 takes off, the current position is the initial position where the unmanned aerial vehicle 2 is set, and the new target position is the shooting position at the beginning.

When the imaging position is changed after the imaging device 22 starts imaging, the current position is the imaging position before the change, and the new target position is the imaging position after the change.

Further, for example, at the end of the work, the unmanned aerial vehicle 2 flies from the last shooting position to a prescribed position, for example, an initial position, and lands. In this case, the current position is the last shot position, and the new target position is the position where the unmanned aerial vehicle 2 lands, for example, the initial position.

A position at which the state of the work mechanism 35 can be imaged, or a position at which at least a part of the work object, for example, at least a part of a building as a disassembly work object can be imaged is defined as an imaging position. The position where both the state of the work mechanism 35 and the work object can be imaged may be defined as an imaging position.

The flight control device 1 can set a position where the imaging device 22 can capture images of the working mechanism 35 from the left side of the working mechanism 35 and a position where the imaging device 22 can capture images of the working mechanism 35 from the right side of the working mechanism 35 as imaging positions. One of the left side position and the right side position is selected as a target position (photographing position).

The combination of the left side position and the right side position can be based on the state (position, direction) of the work mechanism 35 when the left side position and the right side position are set. In this case, the flight control device 1 may set the left side position and the right side position to positions that are bilaterally symmetrical with respect to the working mechanism 35.

The combination of the left side position and the right side position may be based on the state (position, direction) of the lower traveling body 31 or the upper revolving unit 32. In this case, the flight control device 1 may set the left side position and the right side position to positions that are laterally symmetrical with respect to the lower traveling body 31 or the upper revolving body 32.

The combination of the left side position and the right side position may be set according to an instruction of the operator. In this case, the flight control device 1 receives an instruction from an operator and determines the imaging position as a determination unit. Other position decisions are also the same.

Fig. 5 is a top view of the left and right positions of the unmanned aerial vehicle 2. In fig. 5, the left side position 2L and the right side position 2R are set in front of the rotation axis of the boom 35a, that is, at the distal end side in the extending direction of the work machine 35 with respect to the base end portion of the boom 35a, and are symmetrical positions with respect to the work machine 35 at the start of the work.

The left side position and the right side position may be prepared as a plurality of position combinations. In this case, a group may be selected from a plurality of combinations of positions according to an instruction from an operator or according to the state of the work machine 3 (the position and direction of each part of the work machine 3).

The combination of the left side position and the right side position may be changed in time according to the state (position, direction) of the working mechanism 35 or according to the state (position, direction) of the lower traveling body 31 or the upper revolving structure 32. In this case, an appropriate imaging position corresponding to the state of the construction machine 3 can be ensured.

Any one of the left side position and the right side position is selected as the photographing position. For example, the operator may select an arbitrary position by self-instruction. In this case, the operator of the work machine 3 can select a valid image by grasping the condition of the work site or the like that is not directly seen, for example. For example, in order to obtain a more efficient image, the operator can switch the photographing position from the left side position to the right side position. The imaging position may be specified to the left side position or the right side position by the initial setting, and the imaging position specified by the initial setting may be switched according to an instruction of the operator.

The operator may select any one of the left side position and the right side position in time depending on the state (position and direction) of the working mechanism 35 or depending on the state (position and direction) of the lower traveling body 31 or the upper revolving unit 32.

The photographing position is not limited to the left side position and the right side position, but may be set to any position instead of or in addition to the left side position and the right side position. The number of imaging positions may be arbitrary, or three or more imaging positions may be provided, and the imaging positions may be appropriately selected according to an instruction from an operator or according to the state of the construction machine 3 (the position and direction of each part of the construction machine 3).

Next, a parameter value indicating the possibility of interference between the unmanned aerial vehicle 2 and the construction machine 3 will be described (step S108).

When calculating the parameter values, the flight control device 1 obtains the shortest distance (minimum distance) between the working mechanism 35 and the path in the case where the unmanned aerial vehicle 2 moves straight from the current position to the new target position. The flight control device 1 calculates the state of the work implement 35 based on the state information of the construction machine 3, that is, the position information and the attitude information of the construction machine 3. As described above, the attitude information of the construction machine 3 includes fixed (known) parameters such as the boom length, boom mount position, boom length, arm length, and the like of the construction machine 3, and parameters such as the boom angle, the arm angle, and the arm angle that change during the work.

If the shortest distance is smaller than the predetermined threshold value, the unmanned aerial vehicle 2 may come into contact with the working mechanism 35 in the middle of the linear movement from the current position to the new target position. Therefore, in this case (in the case where the determination at step S108 is negative), the flight control device 1 does not select a path for straight movement, but searches for a path for detouring away from the working mechanism 35 (step S114). On the other hand, when the shortest distance is equal to or greater than the predetermined threshold (when the determination at step S108 is affirmative), the unmanned aerial vehicle 2 is unlikely to contact the working mechanism 35 in the middle of the linear movement, and therefore the flight control device 1 selects the route that is linearly moved from the current position to the new target position as the temporary flight route (step S112).

In the present embodiment, the flight control device 1 calculates the shortest distance as the value of the parameter. However, the parameter may be a binary parameter indicating whether the linear movement is possible. For example, the flight control device 1 may calculate the value (one of the two values) of the parameter by using an algorithm for determining whether or not the unmanned aircraft 2 flying on the branch line path interferes with the construction machine 3, instead of calculating the shortest distance. In this case, in particular, in the case where the linear movement is possible, the time required for selecting the flight path can be shortened.

Next, a description will be given of a case where the straight-line travel path cannot be selected (in a case where the determination at step S110 is negative), a flight path (temporary flight path) that avoids the interference between the unmanned aerial vehicle 2 and the construction machine 3 is selected (step S114).

The flight path that avoids the interference between the unmanned aerial vehicle 2 and the construction machine 3 is extracted and selected as a path that can ensure that the flight path is a certain distance from the working mechanism 35. In this case, the shortest distance between the flight path and the working mechanism 35 may be set to be the same as or greater than the above-described predetermined threshold value, for example. The shortest distance between the unmanned aerial vehicle 2 and the working mechanism 35 may be set to different lower limits depending on different positions of the boom 35a, the arm 35b, and the like, and a flight path in which the shortest distance between each position and the unmanned aerial vehicle 2 is greater than the respective lower limits may be selected. For example, since the arm 35b (breaker 35 c) is generally moved faster than the boom 35a, the lower limit value of the shortest distance of the arm 35b (breaker 35 c) can be set to a value larger than the lower limit value of the shortest distance of the boom 35 a.

The trajectory of the flight path may be set as a flight path including, for example, a passing point. In this case, the flight control device 1 may set a path that is sequentially connected in a straight line between the current position, the route point, and the destination position. One or more passing points may be provided.

Fig. 6A is a diagram showing a flight path passing through two passing points. Fig. 6A shows a flight path of the unmanned aerial vehicle 2 from the left side position 2L to the right side position 2R through the passing point 201 and the passing point 202 in this order. In this case, the unmanned aerial vehicle 2 linearly ascends from the left side position 2L to the passing point 201, then linearly horizontally flies to the passing point 202, and then linearly descends to the right side position 2R, thereby avoiding interference with the arm 53 b.

Fig. 6B is a diagram showing a flight path passing through one passing point. Fig. 6B shows a flight path of the unmanned aerial vehicle 2 from the left side position 2L to the right side position 2R via the passing point 203. In this case, the unmanned aerial vehicle 2 flies straight horizontally from the left side position 2L to the passing point 203, and then flies straight horizontally to the right side position 2R, thereby avoiding interference with the boom 53 a.

The flight path may be a curved path, for example, an arc path, which can ensure a distance equal to or greater than a predetermined distance from the working mechanism 35. For example, the flight control device 1 may set an arc-shaped path avoiding the working mechanism 35, such as a path 101 shown in fig. 5. The path 101 shown in fig. 5 represents a path along which the unmanned aerial vehicle 2 flies horizontally from the left side position 2L to the right side position 2R.

Fig. 6C is an exemplary diagram showing a flight path set at the start of a job. In FIG. 6C, the current position is shown in the initial position ₃ 01 to a left position 2L as a target position. In this example, the unmanned aerial vehicle 2 ascends from the initial position 301 to the passing point 204 and then horizontally flies to the passing point 205. Then, the unmanned aerial vehicle 2 ascends from the passing point 205 to the left position 2L. In this case, if the unmanned aerial vehicle 2 moves straight from the initial position 301 to the left position 2L, there is a possibility that the unmanned aerial vehicle interferes with the boom 53a. However, the flight control device 1 can prevent the unmanned aerial vehicle 2 from touching the boom 53a by selecting the flight path 102 passing through the passing point 204 and the passing point 205. In this case, the selected flight path 102 is ensured to have a contact with the boom 53a for avoiding contactAnd a distance delta equal to or greater than a threshold value (a predetermined threshold value) of the shortest distance.

The flight control device 1 may select a plurality of flight paths as flight paths for avoiding interference between the unmanned aerial vehicle 2 and the construction machine 3, but preferably selects a path having a relatively short flight distance or a path having a relatively short travel time.

As one of the selection methods, the flight control device 1 may consider selecting the shortest flight path among the flight paths that avoid the interference between the unmanned aerial vehicle 2 and the construction machine 3. For example, in the case of setting a path connecting the current position, the route point, and the target position in order with a straight line, shortening of the flight distance can be achieved by increasing the number of route points. On the other hand, if the number of passing points is increased, the number of times of changing the direction or pausing at the passing points is increased, and the moving time becomes longer. Accordingly, the flight control device 1 preferably selects a flight path that meets the required conditions. For example, when importance is placed on the movement time, the flight control device 1 may select a flight path having the shortest movement time among flight paths in which interference between the unmanned aerial vehicle 2 and the construction machine 3 is avoided.

Next, a description will be given of a method of setting a flight path in a case where an obstacle that would interfere with the flight of the unmanned aerial vehicle 2 exists on the temporary flight path (in a case where the determination in step S118 is affirmative).

In this case, the flight control device 1 sets a flight path in which the shortest distance between the flight path and the obstacle is equal to or greater than a predetermined threshold value, among the flight paths in which the unmanned aerial vehicle 2 and the construction machine 3 interfere with each other. Thus, the possibility of the unmanned aerial vehicle 2 coming into contact with the obstacle can be eliminated. The flight path may be set to shorten the flight distance, in particular, to minimize the flight distance, or may be set to shorten the flight time, in particular, to minimize the flight time. The flight path may be a combination of straight paths connecting the passing points, or may be a curved line, for example, an arc-shaped curved line.

Next, a flight path in the case where the cable is connected to the unmanned aerial vehicle 2 will be described.

Fig. 7A and 7B are exemplary diagrams of flight paths in the case where the unmanned aerial vehicle 2 is connected with a cable. The cable 400 is, for example, a cable for supplying power to the unmanned aerial vehicle 2 or a cable for communicating between the unmanned aerial vehicle 2 and the construction machine 3. In addition, the case of using the cable 400 as a rope for tying the unmanned aerial vehicle 2 to limit the flight range thereof is also included.

The example of fig. 7A shows a flight path of the unmanned aerial vehicle 2 from the initial position 301 to the right position 2R, which is the shooting position at the beginning. In this case, the unmanned aerial vehicle 2 moves straight up from the initial position 301 to the passing point 207, and then moves straight horizontally to the right position 2R, and reaches the right position 2R. In this case, the passing point 207 is provided on the flight path, so that not only the unmanned aerial vehicle 2 does not come into contact with the boom 53a or the arm 53b, but also the cable 400 extending from the unmanned aerial vehicle 2 does not come into contact with the boom 53a or the arm 53b and become entangled.

The example of fig. 7B shows the flight path of the unmanned aerial vehicle 2 moving from the left side position 2L to the right side position 2R. In this case, the unmanned aerial vehicle 2 flies horizontally from the left side position 2L to the passing point 208, and then flies horizontally to the right side position 2R to reach the right side position 2R. In this case, the passing point 208 is set on the flight path, so that not only the unmanned aerial vehicle 2 does not come into contact with the boom 53a or the arm 53b, but also the cable 400 extending from the unmanned aerial vehicle 2 does not come into contact with the boom 53a or the arm 53b and become entangled.

Thus, when the unmanned aerial vehicle 2 is connected to the cable 400, if the unmanned aerial vehicle 2 moves straight from the current position to the target position, the cable 400 may still contact the working mechanism 35 even if the unmanned aerial vehicle 2 itself cannot interfere with the working mechanism 35, and at this time, a flight path is selected that bypasses from the current position to the target position to avoid contact.

In order to prevent the cable 400 from being excessively loosened, a mechanism such as a winder for winding the cable 400 with a constant tension may be provided. In this case, since the cable 400 has a shape close to a straight line, the risk of the cable 400 blown by the wind inadvertently contacting the working mechanism 35 can be eliminated.

As described above, in the present embodiment, before the target position is updated, the unmanned aerial vehicle 2 stays at the target position before the update, and continues to take an image at the target position with the imaging device 22. Therefore, the high-definition photographed image can be presented to the operator without lowering the image quality due to the shake of the unmanned aerial vehicle 2, and the work of the construction machine 3 can be efficiently supported.

In addition, when the unmanned aerial vehicle 2 can be linearly moved to the new target position, a flight path such as the linear movement of the unmanned aerial vehicle 2 is selected, and therefore, the minimization of the flight path and the minimization of the flight time can be achieved. Further, when the unmanned aerial vehicle 2 cannot select a straight line to move to a new target position due to interference with the working mechanism 35, the unmanned aerial vehicle 2 automatically selects a flight path bypassing the working mechanism 35. Therefore, the unmanned aerial vehicle 2 can be moved to a new target position without burdening the operator.

While the embodiments of the present invention have been described in detail, the present invention is not limited to the specific embodiments, and various modifications and changes can be made within the scope of the claims. In addition, all or a plurality of the constituent elements of the above-described embodiments may be combined.

For example, although the above-described embodiment has been described in which the support apparatus of the present invention is applied to support an actual operation of the construction machine 1, the support apparatus of the present invention may be applied to support inspection of the construction machine 1 itself, traveling of the construction machine 1, or other operations, respectively. For example, the support device of the present invention is applicable to a case of supporting an inspection operation of a relatively high object such as an attachment at the time of machine stop or a surrounding monitoring by traveling in which the construction machine 1 is moved. As described above, the above-described embodiments are also applicable to various supports of the construction machine 1.

In the above structure, the construction machine may further include: a lower traveling body; an upper revolving body mounted on the lower traveling body; and a work mechanism provided on the upper revolving unit, wherein the status information includes posture information indicating a posture of at least one of the construction machine and the work mechanism.

In the above configuration, the unmanned aerial vehicle may further include a calculation unit that calculates a value of a parameter indicating a possibility that the unmanned aerial vehicle interferes with the construction machine when the unmanned aerial vehicle moves straight from a current position to the target position, based on the state information, wherein the determination unit may determine the movement pattern based on the value of the parameter calculated by the calculation unit.

In the above configuration, the determination unit may determine the target position as a position at which the imaging device can capture the image of the work mechanism.

In the above configuration, the determination unit may determine the target position as a position at which the imaging device can image a part of the work mechanism or the work object.

In the above configuration, the target position may include a left position at which the imaging device can image the working mechanism from a left side of the working mechanism (more specifically, a left side in a left-right direction of the upper revolving unit) and a right position at which the imaging device can image the working mechanism from a right side of the working mechanism (more specifically, a right side in the left-right direction of the upper revolving unit), and the determining unit may determine the movement pattern when the target position changes between the left position and the right position. For example, when the unmanned aerial vehicle is located on the right side of the working mechanism and moves to the target position set on the left side of the working mechanism, or when the unmanned aerial vehicle is located on the left side of the working mechanism and moves to the target position set on the right side of the working mechanism, the determining unit determines the movement pattern based on the state information so as to avoid interference between the unmanned aerial vehicle and the working machine.

In the above configuration, when the determination unit determines the target position for a predetermined time, the determination unit may determine, as the movement method, one of a first candidate method in which the unmanned aerial vehicle moves straight from the original target position to the new target position and a second candidate method in which the unmanned aerial vehicle moves from the original target position to the new target position via a predetermined route point, by setting the determined target position as a new target position and setting a target position preceding the determined target position as an original target position.

In the above configuration, the second alternative method may include a linear movement of the unmanned aerial vehicle from the original target position to the route point and a linear movement of the unmanned aerial vehicle from the route point to the new target position.

In the above configuration, the determining unit may determine the route point based on the state information when the second candidate mode is selected.

In the above configuration, the determining unit may determine the route point so that a shortest distance between the unmanned aerial vehicle and the construction machine when the unmanned aerial vehicle moves according to the second candidate method is larger than a shortest distance between the unmanned aerial vehicle and the construction machine when the unmanned aerial vehicle moves according to the first candidate method.

In the above configuration, the movement method may further include a surrounding environment information acquiring unit that acquires surrounding environment information indicating a surrounding environment of the construction machine, wherein the determining unit determines the movement method based on the state information and the surrounding environment information.

In the above configuration, the information output unit may be further configured to output predetermined information when the determination unit cannot determine the movement pattern.

Claims

1. A support device for supporting a construction machine based on image information provided by an imaging device mounted on an unmanned aerial vehicle, the support device comprising:

a state information acquisition unit that acquires state information indicating a state of the construction machine;

a determining unit configured to determine a target position of the unmanned aerial vehicle and a movement pattern of the unmanned aerial vehicle to the target position; the method comprises the steps of,

a control unit configured to control movement of the unmanned aerial vehicle to the target position based on the movement pattern determined by the determination unit,

The determination unit determines the movement pattern based on the state information so as to avoid interference between the unmanned aerial vehicle and the construction machine.

2. The support apparatus according to claim 1, wherein,

the construction machine includes:

a lower traveling body;

an upper revolving body mounted on the lower traveling body; the method comprises the steps of,

a working mechanism provided on the upper revolving body, wherein,

the state information includes posture information indicating a posture of at least one of the work machine and the work machine.

3. The support apparatus according to claim 2, further comprising:

a calculation section that calculates a value of a parameter indicating a possibility that the unmanned aerial vehicle interferes with the construction machine in a case where the unmanned aerial vehicle moves straight from a current position to the target position, based on the state information,

the determination unit determines the movement pattern based on the value of the parameter calculated by the calculation unit.

4. The support device according to claim 2 or 3, wherein,

the determination unit determines the target position as a position at which the imaging device can capture the image of the work mechanism.

5. The support device according to claim 2 or 3, wherein,

the determination unit determines the target position as a position at which the imaging device can capture a part of the work mechanism or the work object.

6. The support apparatus according to claim 4, wherein,

the target position includes a left side position at which the photographing device can photograph the working mechanism from the left side of the working mechanism, and a right side position at which the photographing device can photograph the working mechanism from the right side of the working mechanism,

the determination unit determines the movement pattern when the target position changes between the left position and the right position.

7. The support device according to claim 1 to 3,

when the determining unit determines the target position at a predetermined time, the determining unit determines one of the first candidate system and the second candidate system as the moving system by setting the determined target position as a new target position and setting a target position preceding the determined target position as an original target position,

The first candidate method is a method in which the unmanned aerial vehicle moves linearly from the original target position to the new target position, and the second candidate method is a method in which the unmanned aerial vehicle moves from the original target position to the new target position through a predetermined passing point.

8. The support apparatus according to claim 7, wherein,

the second alternative method includes a linear movement of the unmanned aerial vehicle from the original target position to the passing point and a linear movement of the unmanned aerial vehicle from the passing point to the new target position.

9. The support apparatus according to claim 7, wherein,

the determination unit determines the route point based on the state information when the second candidate mode is selected.

10. The support apparatus according to claim 9, wherein,

the determination unit determines the route point so that a shortest distance between the unmanned aerial vehicle and the construction machine when the unmanned aerial vehicle moves based on the second candidate method is greater than a shortest distance between the unmanned aerial vehicle and the construction machine when the unmanned aerial vehicle moves based on the first candidate method.

11. A support device according to any one of claims 1 to 3, further comprising:

a surrounding environment information acquisition unit that acquires surrounding environment information indicating a surrounding environment of the construction machine,

the determination unit determines the movement method based on the state information and the surrounding environment information.

12. A support device according to any one of claims 1 to 3, further comprising:

and an information output unit configured to output predetermined information when the determination unit cannot determine the movement method.

13. A system, comprising:

the support apparatus according to any one of claims 1 to 12; the method comprises the steps of,

at least one of the construction machine and the unmanned aerial vehicle.