JP2022028520A - Work support device - Google Patents

Abstract

To suppress deterioration of visibility of an image captured by an imaging device caused by change in a state of a work machine.SOLUTION: A work support device controls a positional relation between a work machine and a flying body mounted with an imaging device. The work machine includes a lower structure, a revolving super structure mounted on the lower structure, and a display device for displaying an image captured by the imaging device. The work support device includes an information acquiring unit for acquiring information indicating a state of the work machine and a state of the flying body, and a flight control unit for controlling a flying state of the flying body on the basis of the information acquired by the information acquiring unit. Even when a turning angle of the revolving super structure with respect to the lower structure is changed, the flight control unit executes hovering maintaining control for maintaining the flying body in the prescribed flight state in correspondence with the prescribed zone when the turning angle is in the prescribed zone.SELECTED DRAWING: Figure 5

Description

The present disclosure relates to a work support device.

A technique is known in which a space that cannot be imaged by an image pickup device attached to an upper swivel body of an excavator is imaged by an image pickup device attached to an autonomous flying object (see, for example, Patent Document 1).

International Publication No. WO2017 / 131194

However, in the conventional technology as described above, since the position of the flying object is constantly changed according to the movement of the excavator, the image pickup device is caused by the shaking of the flying object (vibration caused by acceleration, etc.) at that time. The visibility of the captured image tends to decrease.

Therefore, in one aspect, it is an object of the present invention to suppress a decrease in visibility of an image captured by an image pickup device due to a change in the state of a work machine.

On one side,
It is a work support device that controls the positional relationship between the work machine and the flying object equipped with the image pickup device.
The work machine has a lower traveling body, an upper swivel body mounted on the lower traveling body, and a display device for displaying an image captured by the image pickup device.
The work support device is
An information acquisition unit that acquires information representing the state of the work machine and the state of the flying object, and
A flight control unit that controls the flight state of the flying object based on the information acquired by the information acquisition unit is provided.
Even when the turning angle of the upper turning body with respect to the lower traveling body changes, the flight control unit associates the flying body with the predetermined zone when the turning angle is in a predetermined zone. Provide a work support device that performs hovering maintenance control to maintain the flight condition of the aircraft.

On one aspect, according to the present invention, it is possible to suppress a decrease in visibility of an image captured by an image pickup device due to a change in the state of a work machine.

It is explanatory drawing about the structure of the work machine and the unmanned aerial vehicle by Example 1. FIG. It is a figure which shows an example of the hardware composition which concerns on the control system of a work machine. It is a figure explaining the function realized by various control devices. It is a figure which shows roughly the zone to which the turning angle of the upper turning body belongs. It is a schematic flowchart which shows an example of the process executed by a flight control device. It is a schematic flowchart which shows another example of the processing performed by a flight control device.

Hereinafter, each embodiment will be described in detail with reference to the accompanying drawings.

[Example 1]
FIG. 1 is an explanatory diagram regarding a configuration of a work machine 1 and an unmanned airplane 40 according to an embodiment. In addition to the work machine 1 and the unmanned aerial vehicle 40, FIG. 1 shows a flight control device 50 (an example of a work support device) and a remote control device 52.

The work machine 1 carries out a predetermined work in cooperation with the unmanned airplane 40. The work machine 1 is a construction machine provided with a crusher 145 suitable for, for example, dismantling work, and is a crawler type lower traveling body 110 and an upper portion rotatably mounted on the lower traveling body 110 via a turning mechanism 130. It includes a swivel body 120 and a working mechanism 140. A cab (driver's cab) 122 is provided on the front left side portion of the upper swivel body 120. A work mechanism 140 is provided at the front center portion of the upper swivel body 120, and a crusher 145 is provided at the tip of the work mechanism 140. The work machine 1 may be provided with a basement to which the unmanned aerial vehicle 40 arrives and departs.

The working mechanism 140 includes a boom 141 undulatingly mounted on the upper swing body 120, an arm 143 rotatably connected to the tip of the boom 141, and a crusher 145 attached to the tip of the arm 143. I have. The work mechanism 140 is equipped with a boom cylinder 142, an arm cylinder 144 and a bucket cylinder 146 formed by a telescopic hydraulic cylinder. Instead of the crusher 145, another tip attachment such as a bucket may be attached to the tip of the arm 143.

The boom cylinder 142 is interposed between the boom 141 and the upper swing body 120 so as to expand and contract by receiving the supply of hydraulic oil and rotate the boom 141 in the undulating direction. The arm cylinder 144 expands and contracts by receiving the supply of hydraulic oil, and is interposed between the arm 143 and the boom 141 so as to rotate the arm 143 about a horizontal axis with respect to the boom 141. The bucket cylinder 146 expands and contracts by receiving the supply of hydraulic oil and is interposed between the crusher 145 and the arm 143 so as to rotate the crusher 145 about the horizontal axis with respect to the arm 143. The crusher cylinder 147 is provided in the crusher 145 so as to expand and contract by receiving the supply of hydraulic oil to open and close the crusher 145.

The unmanned aerial vehicle 40 is a rotorcraft and includes a plurality of (for example, 4, 6 or 8) blades, a battery for supplying electric power to an electric motor (actuator) for rotating the plurality of blades, and the like. In addition to or in addition to such a battery, the unmanned aerial vehicle 40 may be connected to a power supply line from the ground.

The unmanned aerial vehicle 40 includes a control device 400 and an image pickup device 410.

The control device 400 receives various flight states (forward state, reverse state, ascending state, descending state) of the unmanned aerial vehicle 40 according to control information (command) from the flight control device 50 and operation information from the remote control device 52, which will be described later. , Hovering, etc.). Further, the control device 400 transmits the image (front environment image) acquired by the image pickup device 410 to the work machine 1.

The image pickup apparatus 410 includes a camera. The type of camera is arbitrary, and may be, for example, a wide-angle camera. The image pickup apparatus 410 may be detachably attached to the unmanned aerial vehicle 40, or may be firmly fixed to the unmanned aerial vehicle 40. The image pickup device 410 acquires a front environment image of the front of the unmanned aerial vehicle 40 by an image pickup element such as a CCD (charge-coupled device) or a CMOS (complementary metal oxidase semiconductor). The image pickup device 410 may, for example, acquire a front environment image in real time and supply it to the control device 400 in the form of a stream having a predetermined frame period.

The image pickup apparatus 410 preferably includes a gimbal (not shown). The gimbal functions to keep the optical axis of the image pickup apparatus 410 in a fixed direction (for example, a predetermined direction in a horizontal plane) even if the attitude of the unmanned aerial vehicle 40 changes.

The flight control device 50 executes various controls of the unmanned aerial vehicle 40. In one embodiment, the flight control device 50 is realized by a server (server computer), in which case the flight control device 50 is connected to the work machine 1 and the unmanned aerial vehicle 40 via a network (not shown). In this case, the network may include a wireless communication network, the Internet, a VPN (Virtual Private Network), a WAN (Wide Area Network), a wired network, or any combination thereof. In another embodiment, the flight control device 50 may be realized by the control device 10 of the work machine 1. Further, in another embodiment, the flight control device 50 may be realized by the control device 400 of the unmanned aerial vehicle 40. Alternatively, in another embodiment, the function of the flight control device 50 may be realized in cooperation with any two combinations or all of the server, the control device 10, and the control device 400. Details of the flight control device 50 will be described later.

The remote control device 52 is, for example, in the form of a remote controller, and may be operated by a user (for example, an operator of the work machine 1 or an operator different from the operator). If the user is the operator of the work machine 1, the remote control device 52 may be brought into the cab 122. The remote control device 52 can wirelessly communicate with the unmanned aerial vehicle 40, and transmits an operation signal according to the user's operation to the unmanned aerial vehicle 40. In this case, when the unmanned aerial vehicle 40 receives the operation information from the remote control device 52, the control device 400 of the unmanned aerial vehicle 40 realizes the movement (forward, backward, up / down, etc.) of the unmanned aerial vehicle 40 according to the operation information. In the modified example, the remote control device 52 may be omitted. Further, the remote control device 52 may be realized by a smartphone or the like.

FIG. 2 is a diagram showing an example of a hardware configuration related to the control system of the work machine 1.

As shown in FIG. 2, the work machine 1 includes a peripheral device 8 and a control device 10.

The peripheral device 8 is an electronically controllable device, various sensors, and the like mounted on the work machine 1. The peripheral device 8 is, for example, an operation of an image output device 80 (an example of a display device), a buzzer, an audio output device (not shown), a hydraulic pressure generator for operating a work mechanism 140 (not shown), and various operation members. Various sensors 82 and the like for detecting the state may be included. The hydraulic pressure generator may be a hydraulic pump driven by an engine and / or an electric motor. When utilizing a hydraulic pump driven by an electric motor, the hydraulic pressure generator may include an inverter for driving the electric motor. The various sensors 82 are gyro sensors, GPS (Global Positioning System) sensors, various angle sensors, acceleration sensors (tilt sensors), and hydraulic pressure at predetermined points on the hydraulic line (not shown) applied by the hydraulic pressure generator. It may include a hydraulic sensor or the like to detect.

The image output device 80 is provided in the cab 122 so that the operator of the work machine 1 can see it. The image output device 80 is optional, but may be, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. In the modified example, the image output device 80 may be a portable device (for example, a tablet terminal or the like) that can be brought into the cab 122 by the operator of the work machine 1.

The control device 10 includes a CPU (Central Processing Unit) 11, a RAM (Random Access Memory) 12, a ROM (Read Only Memory) 13, an auxiliary storage device 14, a drive device 15, and a communication interface 17 connected by a bus 19. , A wired transmission / reception unit 25 and a wireless transmission / reception unit 26 connected to the communication interface 17 are included.

The auxiliary storage device 14 is, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like, and is a storage device for storing data related to application software or the like.

The wired transmission / reception unit 25 includes a transmission / reception unit capable of communicating using a wired network. A peripheral device 8 is connected to the wired transmission / reception unit 25. However, a part or all of the peripheral device 8 may be connected to the bus 19 or may be connected to the wireless transmission / reception unit 26.

The wireless transmission / reception unit 26 is a transmission / reception unit capable of communicating using a wireless network. The wireless network may include a wireless communication network of a mobile phone, the Internet, a VPN, a WAN, and the like. Further, the wireless transmission / reception unit 26 includes a short-range wireless communication (NFC: Near Field Communication) unit, a Bluetooth (Bluetooth, registered trademark) communication unit, a Wi-Fi (Wireless-Fidelity, registered trademark) transmission / reception unit, an infrared transmission / reception unit, and the like. It may be included. For example, the wireless transmission / reception unit 26 can realize communication with the flight control device 50 in the form of a server.

The control device 10 may be connectable to the recording medium 16. The recording medium 16 stores a predetermined program. The program stored in the recording medium 16 is installed in the auxiliary storage device 14 or the like of the control device 10 via the drive device 15. The installed predetermined program can be executed by the CPU 11 of the control device 10. For example, the recording medium 16 is a recording medium such as a CD (Compact Disc) -ROM, a flexible disk, a magneto-optical disk, or the like that optically, electrically, or magnetically records information, a ROM, a flash memory, or the like. It may be a semiconductor memory or the like that electrically records. The recording medium 16 does not include a carrier wave.

Here, the control system of the work machine 1 has been described with reference to FIG. 2, but the unmanned aerial vehicle 40 may be substantially the same except for the configuration related to the peripheral device 8. For example, in the case of the control system of the unmanned aerial vehicle 40, the hardware configuration of the control device 400 may be the same as that of the control device 10. The peripheral device corresponding to the peripheral device 8 includes an image pickup device 410 (see FIG. 1) and various sensors.

Further, the hardware configuration of the flight control device 50 may be substantially the same as the hardware configuration of the control device 10 shown in FIG.

Next, the flight control device 50 will be described in detail together with the control device 10 and the control device 400 with reference to FIGS. 3 and later.

FIG. 3 is a diagram illustrating functions realized by the control device 10, the control device 400, and the flight control device 50.

As shown in FIG. 3, the control device 10 includes a position information acquisition unit 150, a posture information acquisition unit 151, a turning angle information acquisition unit 152, a communication processing unit 153, and an image output processing unit 154. Each functional unit such as the position information acquisition unit 150 can be realized by the CPU 11 shown in FIG. 2 executing a program in a storage device such as the ROM 13 shown in FIG.

The position information acquisition unit 150 acquires the position information of the work machine 1 from the GPS sensor (not shown) among the sensors 82. The position information of the work machine 1 is expressed by latitude, longitude, and altitude. The GPS sensor includes a GPS receiver, and calculates latitude, longitude, and altitude by interference positioning or the like based on radio waves from satellites.

The posture information acquisition unit 151 acquires the posture information of the work machine 1 based on various sensors among the sensors 82 that acquire the parameters related to the posture of the work machine 1. In this case, the various sensors that acquire the parameters related to the posture may be, for example, a boom angle sensor, an arm angle sensor, a bucket angle sensor, an airframe tilt sensor, or the like. The boom angle sensor is a sensor that acquires the boom angle, for example, a rotation angle sensor that detects the rotation angle of the boom foot pin, a stroke sensor that detects the stroke amount of the boom cylinder 142, and a tilt angle of the boom 141. Includes tilt (acceleration) sensors and the like. The same applies to the arm angle sensor and the bucket angle sensor. The airframe tilt sensor is a sensor that acquires the airframe tilt angle, and detects, for example, the tilt angle of the upper swivel body 120 with respect to the horizontal plane.

The turning angle information acquisition unit 152 acquires various parameters related to the turning angle of the upper turning body 120 with respect to the lower traveling body 110 (hereinafter, also simply referred to as “turning angle of the upper turning body 120”) among the sensors 82. Based on the sensor, the turning angle information representing the turning angle of the upper turning body 120 is acquired. In this case, the various sensors that acquire the parameters related to the turning angle of the upper turning body 120 are, for example, a geomagnetic sensor, a rotation angle sensor (for example, a resolver) that detects the turning angle around the turning axis of the turning mechanism 130, and a gyro sensor. And so on.

The communication processing unit 153 transmits various information acquired by the position information acquisition unit 150, the attitude information acquisition unit 151, and the turning angle information acquisition unit 152 to the flight control device 50. For example, the communication processing unit 153 may transmit the latest information to the flight control device 50 at predetermined intervals in response to a request from the flight control device 50.

Further, the communication processing unit 153 receives image data from the unmanned aerial vehicle 40. The image data is the data of the front environment image captured by the image pickup apparatus 410.

The image output processing unit 154 outputs a front environment image on the image output device 80 based on the image data acquired by the communication processing unit 153. Thereby, the operator of the work machine 1 can grasp, for example, the situation of the work site which cannot be seen directly from the front environment image on the image output device 80.

As shown in FIG. 3, the control device 400 includes an airframe information acquisition unit 401, a target flight state setting unit 402, an airframe control unit 403, and a communication processing unit 404. Each functional unit such as the machine information acquisition unit 401 can be realized by a CPU such as the CPU 11 shown in FIG. 2 executing a program in a storage device such as the ROM 13 shown in FIG.

The aircraft information acquisition unit 401 acquires aircraft information representing various states related to the aircraft of the unmanned aerial vehicle 40. The aircraft information may include position information of the unmanned aerial vehicle 40, attitude information of the unmanned aerial vehicle 40, and the like. The position information of the unmanned aerial vehicle 40 may be expressed by, for example, latitude, longitude, and altitude. The position information of such an unmanned aerial vehicle 40 can be acquired from the GPS sensor. The attitude information of the unmanned aerial vehicle 40 may include information on the rotation of the unmanned aerial vehicle 40 around each axis of the yaw axis, the roll axis, and the pitch axis. The attitude information of the unmanned aerial vehicle 40 can be acquired from a sensor such as an inertial measurement unit (IMU) mounted on the unmanned aerial vehicle 40.

The target flight state setting unit 402 sets the target flight state of the unmanned aerial vehicle 40 based on the control information (command) from the flight control device 50. The target flight state includes the target flight position and the target flight attitude. When the control information includes the target flight position and the target flight attitude, the target flight state setting unit 402 may use the target flight position and the target flight attitude as they are.

The target flight position may be expressed, for example, in latitude, longitude, and altitude.

In this embodiment, "hovering maintenance control" using a zone (described later) in which the turning angle of the upper turning body 120 is divided into a plurality of ranges, and the unmanned aerial vehicle 40 following the change in the turning angle of the upper turning body 120. "State-following control" that dynamically changes the target flight position is selectively executed. Hereinafter, the target flight state will be described mainly in the case of hovering maintenance control.

FIG. 4 is a diagram schematically showing a zone to which the turning angle of the upper turning body 120 belongs. In the example shown in FIG. 4, the turning angle of the upper turning body 120 is in the first zone Z1 when the upper turning body 120 faces the rear side of the lower traveling body 110. Further, when the upper turning body 120 faces to the left side of the lower traveling body 110, the turning angle of the upper turning body 120 is in the second zone Z2. Further, when the upper turning body 120 faces the front side of the lower traveling body 110, the turning angle of the upper turning body 120 is in the third zone Z3. Further, when the upper turning body 120 faces the right side of the lower traveling body 110, the turning angle of the upper turning body 120 is in the fourth zone Z4.

In the example of FIG. 4, the four zones (zones Z1 to Z4) are arranged in the installation surface (horizontal plane or substantially horizontal plane) of the lower traveling body 110, for example, in the circumferential direction around the turning axis of the upper swivel body 120. For example, they are provided continuously by 90 °, and the turning angle of the upper turning body 120 always belongs to any of four zones (zones Z1 to Z4).

When hovering maintenance control is selected, the target flight state (target flight position and target flight attitude) is defined corresponding to the corresponding predetermined zone. In this case, while the turning angle of the upper turning body 120 continuously belongs to the predetermined zone, the hovering of the unmanned aerial vehicle 40 is continued even if the upper turning body 120 turns. That is, the unmanned aerial vehicle 40 maintains the target flight state associated with the predetermined zone.

The zone setting method is not limited to the example shown in FIG. The zone may be fixedly defined, or the zone may be fluidly defined according to the turning angle of the upper swing body 120 during work. In the latter case, each zone can be arbitrarily set with reference to the reference angle as the turning angle of the upper turning body 120 at a specific time point. For example, a range in which the turning angle of the upper turning body 120 has a difference of a predetermined threshold value (absolute value) or less with respect to the reference angle can be defined as a zone corresponding to the reference angle. Note that FIG. 4 shows an example in which the first zone Z1 is defined with the reference angle X1 as the reference angle and the angle θ (for example, 45 °) as the predetermined threshold value.

When hovering maintenance control is selected, the latitude and longitude related to the target flight position are determined according to the zone, for example, around the front-rear axis of the lower traveling body 110 of the work machine 1 and around the turning axis of the upper turning body 120. The position on the axis swiveled by an angle of 1 may be set horizontally separated by a predetermined distance D1 from a predetermined portion of the work machine 1 (for example, the swivel axis of the upper swivel body 120). The predetermined distance D1 may be, for example, about 20 m. Further, the predetermined distance D1 may be made variable by the user. Further, the predetermined distance D1 may be automatically changed according to the work mode or the like. As for the latitude and longitude related to the target flight position, an appropriate latitude and longitude may be selected according to each zone to which the turning angle of the upper swivel body 120 belongs. In FIG. 4, the unmanned aerial vehicle 40 is drawn at the target flight position corresponding to the third zone Z3.

Further, the altitude related to the target flight position when hovering maintenance control is selected may be constant (for example, within the range of 30 m to 40 m). However, the altitude related to the target flight position may be variable by the user, or may be automatically changed according to the work mode or the like. Alternatively, the zone to which the turning angle of the upper turning body 120 belongs, or the altitude corresponding to the predetermined distance D1 may be selected.

The target flight attitude when hovering maintenance control is selected may be expressed, for example, by parameters related to rotation around each axis of the yaw axis, the roll axis, and the pitch axis. The target flight attitude may be set so that, for example, the front-rear axis of the unmanned aerial vehicle 40 intersects the turning axis of the upper swivel body 120, and the front-rear axis of the unmanned aerial vehicle 40 is located in the horizontal plane. Similarly, the parameters related to the target flight attitude may be variable by the user, or may be automatically changed according to each zone to which the turning angle of the upper turning body 120 belongs, the working mode, and the like.

The aircraft control unit 403 controls the unmanned aerial vehicle 40 so that the target flight state set by the target flight state setting unit 402 is realized based on the aircraft information acquired by the aircraft information acquisition unit 401. The control method of the unmanned aerial vehicle 40 is arbitrary, and may be realized by, for example, PID (Proportional Integral Differential) control or the like.

The communication processing unit 404 transmits the aircraft information and the like acquired by the aircraft information acquisition unit 401 to the flight control device 50. For example, the communication processing unit 404 may transmit the latest aircraft information to the flight control device 50 at predetermined intervals in response to a request from the flight control device 50.

Further, the communication processing unit 404 transmits the data of the front environment image captured by the image pickup device 410 to the work machine 1. For example, the communication processing unit 404 may transmit the data of the forward environment image to the flight control device 50 at predetermined intervals in response to the request from the flight control device 50.

As shown in FIG. 3, the flight control device 50 includes an information acquisition unit 510, a zone determination unit 512, and a flight control unit 514. Each functional unit such as the information acquisition unit 510 can be realized by a CPU such as the CPU 11 shown in FIG. 2 executing a program in a storage device such as the ROM 13 shown in FIG.

The information acquisition unit 510 acquires various information necessary for various controls of the flight control unit 514. In this embodiment, as an example, the information acquisition unit 510 acquires the position information, the attitude information, and the turning angle information of the work machine 1 and the aircraft information of the unmanned aerial vehicle 40. The position information, posture information, and turning angle information of the work machine 1 can be acquired by communication from the communication processing unit 153 of the control device 10 of the work machine 1. Further, the aircraft information of the unmanned aerial vehicle 40 can be acquired from the communication processing unit 404 of the control device 400 of the unmanned aerial vehicle 40.

The zone determination unit 512 grasps (determines) the zone to which the turning angle belongs based on the turning angle of the upper turning body 120. The zone determination unit 512 compares, for example, a predetermined zone determined according to the setting method as described above with the turning angle of the upper turning body 120, and whether or not the turning angle of the upper turning body 120 is in the predetermined zone. To grasp (determine) whether or not. Further, the zone determination unit 512 grasps, for example, whether or not the turning angle of the upper turning body 120 is in a specific zone (for example, any of the first to fourth zones in FIG. 4) (for example). judge.

The flight control unit 514 generates control information (command) to be transmitted to the unmanned aerial vehicle 40 based on various information acquired by the information acquisition unit 510. As described above, the control information is information for causing the target flight state setting unit 402 of the unmanned aerial vehicle 40 to set the target flight state.

The flight control unit 514 generates control information so that the hovering of the unmanned aerial vehicle 40 is started when the hovering start condition is satisfied. The hovering start condition may be satisfied, for example, when the position of the unmanned aerial vehicle 40 reaches the target flight position. Whether or not the position of the unmanned aerial vehicle 40 has reached the target flight position can be determined based on, for example, the aircraft information acquired by the information acquisition unit 510.

The flight control unit 514 may cause the control device 400 to start hovering, for example, by not changing the control information (by not changing the target flight position of the unmanned aerial vehicle 40 accordingly). Alternatively, the flight control unit 514 may cause the control device 400 to start hovering by transmitting a command for starting hovering (for example, control information indicating a mode such as a hovering mode) to the unmanned aerial vehicle 40.

As described above, even if the turning angle of the upper swivel body 120 changes while the unmanned aerial vehicle 40 is hovering, the flight control unit 514 will hover the unmanned aerial vehicle 40 if the turning angle belongs to a certain zone. Perform hovering maintenance control to maintain.

The flight control unit 514 may realize hovering maintenance control by not changing the control information (by not changing the target flight position of the unmanned aerial vehicle 40 accordingly). Alternatively, the flight control unit 514 may realize hovering maintenance control by transmitting a command for maintaining hovering (for example, control information indicating a mode such as hovering mode) to the unmanned aerial vehicle 40.

Hereinafter, as a comparison with such hovering maintenance control, the control that dynamically changes the target flight state of the unmanned aerial vehicle 40 by following the change in the turning angle of the upper turning body 120 is also referred to as the above-mentioned "state tracking control". Refer to. In the state tracking control, the target flight position of the unmanned aerial vehicle 40 is moved in the same direction as the turning direction of the upper turning body 120 in synchronization with the turning angle of the upper turning body 120. In addition, the target flight attitude of the unmanned aerial vehicle 40 is adjusted at the same time. In this case, while the upper swivel body 120 is turning, the unmanned aerial vehicle 40 also moves around the work machine 1 so as to rotate in the same direction as the turning direction of the upper swivel body 120, and the flight attitude changes. Also in the state tracking control, the target flight position may be set at a predetermined distance D1 from a predetermined portion of the work machine 1 (for example, the swivel axis of the upper swivel body 120). The altitude related to the target flight position can also be determined in the same manner as in the case of hovering maintenance control. Further, regarding the target flight attitude, as in the case of hovering maintenance control, for example, in the target flight attitude, the front-rear axis of the unmanned aerial vehicle 40 intersects the turning axis of the upper turning body 120, and the front-rear and rear of the unmanned aerial vehicle 40 body. The axis may be set to be located in the horizontal plane.

By the way, although the movement of the work machine 1 may vary depending on the work content, there are cases where the work involving the turning of the upper turning body 120 is performed while the lower traveling body 110 is fixed. Further, in this work, the work involving the turning of the upper turning body 120 may be repeated within the movement range in which the turning angle of the upper turning body 120 is relatively narrow. In such a case, when the state tracking control is executed, the control device 400 repeats the movement of the unmanned aerial vehicle 40 as the upper turning body 120 turns. Such movement tends to generate a relatively large acceleration on the body of the unmanned aerial vehicle 40. When a relatively large acceleration is generated in the body of the unmanned aerial vehicle 40, the imaging range of the imaging device 410 changes (sways), and the visibility of the front environment image tends to decrease.

On the other hand, according to the present embodiment, during the period when the turning angle of the upper turning body 120 is within the predetermined zone, the hovering of the unmanned aerial vehicle 40 is maintained even when the upper turning body 120 turns. As a result, the hovering period can be lengthened as compared with the case where the state tracking control is executed. As a result, it is possible to reduce the frequency of relatively large accelerations in the body of the unmanned aerial vehicle 40 and suppress the deterioration of the visibility of the front environment image.

Here, the zone setting method used in the hovering maintenance control is arbitrary, but the larger the range of the turning angle of the upper swivel body 120 given to each zone, the easier it is to maintain hovering. On the other hand, if the range of the turning angle of the upper turning body 120 given to the zone is excessively large, the usefulness of the forward environment image (the amount of useful information for the user obtained from the image) tends to decrease. Therefore, the range of each zone may be determined in consideration of these conflicts. In addition, each zone may be variable by the user, or may be automatically changed according to a work mode or the like.

By the way, in the case where the turning angle of the upper turning body 120 is frequently changed greatly and it is not necessary to turn the upper turning body 120 in small steps, the state tracking control is better than the hovering maintenance control. It is possible to reduce the frequency of relatively large accelerations in the body of the unmanned aerial vehicle 40. In this case, in the hovering maintenance control, the flight position and the flight attitude of the unmanned aerial vehicle 40 are changed when the upper swivel body 120 is turned over the zone, and the flight position and the flight attitude of the unmanned aerial vehicle 40 are changed accordingly. This is because a large acceleration is likely to occur.

Therefore, in this embodiment, the hovering maintenance control may be realized only when a specific work is executed by the work machine 1. In this case, the specific work may be such that the turning angle of the upper turning body 120 is within a relatively narrow range. The specific work may be a plurality of types of work and may be set in advance.

Further, in this case, the flight control unit 514 may determine whether or not the work machine 1 is executing a specific work based on the information acquired by the information acquisition unit 510. The flight control unit 514 may determine whether or not the work machine 1 is performing a specific work, for example, based on the following conditional elements.
Conditional factors (1) Frequency of turning motion (crossing the zone) that greatly changes the turning angle of the upper turning body 120.
Conditional element (2) Frequency of turning motion (within the zone) that changes the turning angle of the upper turning body 120 in small steps.
Conditional element (3) Frequency at which the acceleration generated in the body of the unmanned aerial vehicle 40 exceeds a predetermined threshold value in the execution state of the state tracking control.

In this case, by using the condition element (1), it is possible to reduce the possibility that the hovering maintenance control is executed in the work in which the upper swivel body 120 is frequently swiveled over the zone. Further, by using the condition element (2), hovering maintenance control can be effectively executed, for example, in a situation where the upper swivel body 120 repeatedly swivels in a predetermined zone. Further, by using the condition element (3), the hovering maintenance control can be effectively executed in a situation where a relatively large acceleration is likely to occur in the body of the unmanned aerial vehicle 40 according to the state tracking control.

Next, an operation example of the flight control device 50 will be described with reference to FIGS. 5 and later. In the subsequent processing flow chart (flow chart), the processing order of each step may be changed as long as the relationship between the input and the output of each step is not impaired.

FIG. 5 is a schematic flowchart showing an example of processing executed by the flight control device 50. The process shown in FIG. 5 may be repeatedly executed, for example, at predetermined intervals.

In step S30, the flight control device 50 acquires various information for control (for example, position information, attitude information, and turning angle information of the work machine 1 and aircraft information of the unmanned aerial vehicle 40). The method of acquiring various information for control is as described above.

In step S32, the flight control device 50 determines whether or not the execution condition of the hovering maintenance control is satisfied. The execution condition of the hovering maintenance control is arbitrary, but is satisfied, for example, when it is determined that the work machine 1 is executing a specific work. The specific work may be as described above. If the determination result is "YES", the process proceeds to step S34, and if not, the process proceeds to step S33A.

In step S33A, the flight control device 50 determines whether or not the hovering flag is “0”. The state in which the hovering flag is "0" corresponds to the non-execution state of the control for realizing or maintaining hovering. The initial value of the hovering flag is "0". If the determination result is "YES", the process proceeds to step S33C, and if not, the process proceeds to step S33B.

In step S33B, the flight control device 50 resets the hovering flag to “0”.

In step S33C, the flight control device 50 executes state tracking control. Specifically, the flight control device 50 determines a target flight state (including a target flight position and a target flight attitude) based on various information obtained in step S30, and control information (command) indicating the target flight state. To generate. Then, the flight control device 50 transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives the control information, the control device 400 controls the unmanned aerial vehicle 40 so that the target flight state related to the control information is realized.

In step S34, the flight control device 50 determines whether or not the hovering flag is “1”. The state in which the hovering flag is "1" corresponds to the execution state of the control for realizing or maintaining hovering. If the determination result is "YES", the process proceeds to step S37, and if not, the process proceeds to step S42.

In step S37, the flight control device 50 is based on the reference angle held in the flight control device 50 (the angle that serves as a reference for the turning angle of the upper turning body 120) and the turning angle information acquired in step 30. Therefore, it is determined whether or not the amount of change (difference) in the turning angle of the upper turning body 120 at the present time with respect to the reference angle is equal to or less than a predetermined threshold. If the determination result is "YES", the process proceeds to step S38, and if not, the process proceeds to step S40.

In step S38, the flight control device 50 generates control information for realizing hovering, and transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives such control information, the control device 400 controls the unmanned aerial vehicle 40 so that hovering related to the control information is realized or maintained. The control information for realizing hovering may be, for example, the same information as the control information of the previous cycle, or information indicating that fact.

In step S40, the flight control device 50 resets the hovering flag to “0”.

In step S42, the flight control device 50 sets a new zone corresponding to the turning angle based on the current turning angle of the upper turning body 120 indicated by the turning angle information obtained in step S30, and sets a new zone in the zone. Calculate the corresponding target flight position. That is, the flight control device 50 calculates the target flight position according to the current turning angle of the upper turning body 120, as in the case of the state tracking control. Here, for example, as a new zone, a zone in which the current turning angle of the upper turning body 120 is located at the center of the angle range of the zone (for example, a third zone with respect to the turning angle of the upper turning body 120 in FIG. 4). Zone Z3) is set. Further, as the target flight position, the position associated with the newly set zone (for example, the position of the unmanned aerial vehicle 40 associated with the third zone Z3 in FIG. 4) is calculated. As described above, in step S42, the zone is fluidly set based on the current turning angle of the upper turning body 120. For example, in FIG. 4, a zone in which the respective zones Z1 to Z4 move in the circumferential direction is set according to the turning angle of the upper turning body 120. Further, according to the range of the newly set zone, the target flight position corresponding to the zone is calculated fluidly according to the range of the zone.

In step S44, the flight control device 50 generates control information (command) including the calculated value of the target flight position calculated in step S42, and transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives the control information, the control device 400 controls the unmanned aerial vehicle 40 so that the target flight state related to the control information is realized.

In step S46, the flight control device 50 determines whether or not the unmanned aerial vehicle 40 has reached the target flight state (calculated value of the target flight state calculated in step S42 of the previous cycle) based on various information obtained in step S30. To judge. For example, if the state information of the unmanned aerial vehicle 40 substantially matches the calculated value of the target flight state calculated in step S42 of the previous cycle, the flight control device 50 determines that the unmanned aerial vehicle 40 has reached the target flight state. .. If the determination result is "YES", the process proceeds to step S48, and in other cases, the processing of the current cycle ends.

In step S48, the flight control device 50 generates control information for starting hovering, and transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives the control information, the control device 400 controls the unmanned aerial vehicle 40 so that hovering related to the control information is realized. The control information for starting hovering may be, for example, the same information as the control information of the previous cycle, or information indicating that fact.

In step S50, the flight control device 50 sets the hovering flag to “1” and sets the current turning angle of the upper turning body 120 indicated by the turning angle information obtained in step S30 to a new reference angle. do. The reference angle is held by the flight control device 50, and is updated when a new reference angle is set in step S50. The reference angle held by the flight control device 50 is used in step S37 described above.

According to the process shown in FIG. 5, when the execution condition of the hovering maintenance control is satisfied, the hovering maintenance control is executed from step S34 to step S38. According to the process shown in FIG. 5, when hovering is started, the turning angle of the upper turning body 120 at that time is set and updated to the reference angle. After that, hovering is continued while the amount of change in the relative positional relationship between the turning angles of the upper turning body 120 from the reference angle is equal to or less than a predetermined threshold value. Further, when the amount of change in the turning angle of the upper turning body 120 from the reference angle exceeds a predetermined threshold value, a new zone in a range corresponding to the turning angle of the upper turning body 120 at that time is set. Then, after the unmanned aerial vehicle 40 is moved so that a new target flight state corresponding to the new zone is achieved, hovering is started again, and the turning angle of the upper swivel body 120 at that time is set as the reference angle. Will be done. That is, when the amount of change in the turning angle of the upper turning body 120 from the reference angle exceeds a predetermined threshold value, the turning angle of the upper turning body 120 at that time is set to a new reference angle. After that, hovering is continued while the amount of change in the turning angle of the upper turning body 120 from the reference angle is equal to or less than a predetermined threshold value. In this way, since the hovering maintenance control is continued while updating the reference angle, it is possible to suppress the deterioration of the visibility of the front environment image acquired by the image pickup apparatus 410 during that time.

FIG. 6 is a schematic flowchart showing another example of the processing executed by the flight control device 50. The process shown in FIG. 6 may be repeatedly executed, for example, at predetermined intervals.

In step S130, the flight control device 50 acquires various information for control (for example, position information, attitude information, and turning angle information of the work machine 1 and aircraft information of the unmanned aerial vehicle 40). The method of acquiring various information for control is as described above.

In step S132, the flight control device 50 determines whether or not the execution condition of the hovering maintenance control is satisfied. The execution condition of the hovering maintenance control is arbitrary, but is satisfied, for example, when it is determined that the work machine 1 is executing a specific work. The specific work may be as described above. If the determination result is "YES", the process proceeds to step S134, and if not, the process proceeds to step S133A.

In step S133A, the flight control device 50 determines whether or not the hovering flag is “0”. The state in which the hovering flag is "0" corresponds to the non-execution state of the control for realizing or maintaining hovering. The initial value of the hovering flag is "0". If the determination result is "YES", the process proceeds to step S133C, and if not, the process proceeds to step S133B.

In step S133B, the flight control device 50 resets the hovering flag to “0”.

In step S133C, the flight control device 50 executes state tracking control. Specifically, the flight control device 50 determines the target flight state based on various information obtained in step S130, and generates control information (command) representing the target flight state. Then, the flight control device 50 transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives the control information, the control device 400 controls the unmanned aerial vehicle 40 so that the target flight state related to the control information is realized.

In step S134, the flight control device 50 determines whether or not the hovering flag is “1”. The state in which the hovering flag is "1" corresponds to the execution state of the control for realizing or maintaining hovering. If the determination result is "YES", the process proceeds to step S135, and if not, the process proceeds to step S142.

In step S135, the flight control device 50 acquires the current turning angle of the upper turning body 120 based on the turning angle information obtained in step S130.

In step S136, the flight control device 50 sets a zone corresponding to the current turning angle of the upper turning body 120. The set zone is held by the flight control device 50. Here, for example, in the example of FIG. 4, among the first zone Z1, the second zone Z2, the third zone Z3, and the fourth zone Z4, depending on the current turning angle of the upper swing body 120. One of is set.

In step S137, the flight control device 50 compares the zone corresponding to the hovering during execution with the zone set in step S136, and determines whether or not there is a change in the zone (both are different). If this determination is affirmed (there is a change), the process proceeds to step S138, and if this determination is denied (there is no change), the process proceeds to step S140. Here, for example, in the example of FIG. 4, the turning angle of the upper turning body 120 is from the third zone Z3 (the zone corresponding to the hovering during execution) to another zone (the first zone Z1 and the second zone Z2). Alternatively, when the user moves to the fourth zone Z4), the determination in step S137 is affirmed.

In step S138, the flight control device 50 generates control information for realizing hovering, and transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives such control information, the control device 400 controls the unmanned aerial vehicle 40 so that hovering related to the control information is realized or maintained. The control information for realizing hovering may be, for example, the same information as the control information of the previous cycle, or information indicating that fact.

In step S140, the flight control device 50 resets the hovering flag to “0”.

In step S142, the flight control device 50 calculates the target flight position corresponding to the zone set in step S136. Here, as the target flight state, the flight state associated with the newly set zone is calculated. In the example of FIG. 4, one of the three flight states associated with the first zone Z1, the second zone Z2, or the fourth zone Z4 is calculated as the target flight state.

In step S144, the flight control device 50 generates control information (command) including the calculated value of the target flight position calculated in step S142, and transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives the control information, the control device 400 controls the unmanned aerial vehicle 40 so that the target flight state related to the control information is realized.

In step S146, the flight control device 50 determines whether or not the unmanned aerial vehicle 40 has reached the target flight state (calculated value of the target flight position calculated in step S142 of the previous cycle) based on various information obtained in step S130. To judge. For example, if the state information of the unmanned aerial vehicle 40 substantially matches the calculated value of the target flight state calculated in step S142 of the previous cycle, the flight control device 50 determines that the unmanned aerial vehicle 40 has reached the target flight state. .. If the determination result is "YES", the process proceeds to step S148, and in other cases, the processing of the current cycle ends.

In step S148, the flight control device 50 generates control information for starting hovering, and transmits the generated control information to the unmanned aerial vehicle 40. In this case, when the control device 400 of the unmanned aerial vehicle 40 receives the control information, the control device 400 controls the unmanned aerial vehicle 40 so that hovering related to the control information is realized. The control information for starting hovering may be, for example, the same information as the control information of the previous cycle, or information indicating that fact.

In step S150, the flight control device 50 sets the hovering flag to “1”.

According to the process shown in FIG. 6, when the execution condition of the hovering maintenance control is satisfied, the hovering maintenance control is executed from step S134 to step S138. According to the process shown in FIG. 6, a plurality of zones are defined in advance, and the target flight state is associated with each zone. While the turning angle of the upper turning body 120 is in the predetermined zone, the hovering of the unmanned aerial vehicle 40 is continued in the corresponding target flight state. When the zone to which the turning angle of the upper swivel body 120 belongs changes, the target flight state changes according to the new zone, and the unmanned aircraft 40 hovering so that the unmanned aircraft 40 moves and maintains the new target flight state. It will be started. As described above, while the turning angle of the upper turning body 120 is in the predetermined zone, the hovering of the unmanned aerial vehicle 40 is continued, so that the deterioration of the visibility of the forward environment image acquired by the image pickup apparatus 410 can be suppressed during that time. ..

Although each embodiment has been described in detail above, the present invention is not limited to a specific embodiment, and various modifications and changes can be made within the scope of the claims. It is also possible to combine all or a plurality of the components of the above-described embodiment.

For example, in the above-described embodiment, the control device 10 of the work machine 1 may include a zone notification unit. In this case, the zone notification unit notifies the operator of the work machine 1 of the zone determined by the zone determination unit 512 (for example, any of the first to fourth zones in FIG. 4). As a result, the operator of the work machine 1 can easily grasp which zone the unmanned aerial vehicle 40 currently belongs to (that is, from which direction the unmanned aerial vehicle 40 supplies the forward environment image).
More specifically, for example, when the upper swivel body 120 turns 180 degrees, the input before and after the traveling lever (the lever operated to drive the lower traveling body 110) and the front and rear of the actual traveling are in the normal state. Since it is the opposite of (the state where the turning angle is 0 degrees), the operator of the work machine 1 is likely to make an erroneous operation. On the other hand, by providing the zone notification unit described above, the operator of the work machine 1 can easily grasp the direction of the upper swivel body 120 with respect to the lower traveling body 110, so that such inconvenience can be prevented. In such a modification, the zone notification unit may notify the changed zone when the zone changes. For example, the zone notification unit may notify the zone when the above-mentioned hovering flag is set to “1”. Alternatively, the zone notification unit may always superimpose and display the zone information on the front environment image.

1 Work machine 8 Peripheral equipment 10 Control device 40 Unmanned aerial vehicle 50, 50A Flight control device 52 Remote control device 80 Image output device 82 Sensors 110 Lower traveling body 120 Upper turning body 122 Cab (driver's cab)
130 Swivel mechanism 140 Work mechanism 141 Boom 142 Boom cylinder 143 Arm 144 Arm cylinder 145 Crusher 146 Bucket cylinder 147 Crusher cylinder 150 Position information acquisition unit 151 Attitude information acquisition unit 152 Swivel angle information acquisition unit 153 Communication processing unit 154 Image output Processing unit 400 Control device 401 Aircraft information acquisition unit 402 Target flight state setting unit 403 Aircraft control unit 404 Communication processing unit 410 Imaging device 510 Information acquisition unit 512 Zone determination unit 514 Flight control unit

Claims (6)

It is a work support device that controls the positional relationship between the work machine and the flying object equipped with the image pickup device.
The work machine has a lower traveling body, an upper swivel body mounted on the lower traveling body, and a display device for displaying an image captured by the image pickup device.
The work support device is
An information acquisition unit that acquires information representing the state of the work machine and the state of the flying object, and
A flight control unit that controls the flight state of the flying object based on the information acquired by the information acquisition unit is provided.
Even when the turning angle of the upper turning body with respect to the lower traveling body changes, the flight control unit associates the flying body with the predetermined zone when the turning angle is in a predetermined zone. A work support device that performs hovering maintenance control to maintain the flight condition of the aircraft. The work support device according to claim 1, wherein the flight control unit executes the hovering maintenance control when the work machine is performing a specific work. The work support device according to claim 2, wherein the flight control unit determines whether or not the work machine is performing a specific work based on the information acquired by the information acquisition unit. When the turning angle exceeds the predetermined zone, the flight control unit updates the zone and changes the flight state of the flying object to the target flight state associated with the updated zone. The work support device according to any one of claims 1 to 3, wherein the hovering maintenance control is executed again. The operation according to claim 4, wherein when the turning angle exceeds the predetermined zone, the flight control unit updates the zone by setting a new zone in a range corresponding to the turning angle at that time. Support device. When the turning angle exceeds the predetermined zone, the flight control unit selects one zone from a plurality of preset zones according to the turning angle at that time, thereby setting the zone. The work support device according to claim 4, which is to be updated.

Images