JP2021021253A - Work machine and work machine assistance server - Google Patents

Abstract

To provide a work machine capable of safely performing work through communication with an unmanned aircraft.SOLUTION: A work machine 10 includes at least one or more markers M at prescribed positions of a work mechanism 140 which an unmanned aircraft 40 can recognize as references. A work machine control device 20 calculates distances between the markers M and the unmanned aircraft 40 on the basis of imaged images of the markers M imaged by an imaging device 410 of the unmapped aircraft 40.SELECTED DRAWING: Figure 1

Description

The present invention relates to work machines such as excavators and construction demolition machines.

Conventionally, there is known a system in which a work machine such as an excavator is tracked from the sky by an air vehicle equipped with a camera, and an operator who remotely controls the air vehicle can check an image by the camera on a display device. Such a system is used for inspection of a space that cannot be imaged by a camera attached to the upper swing body of an excavator.

For example, in Patent Document 1 below, the position on the turning axis of the upper swivel body of the excavator is set as the target flight position of the flying body, and the position and orientation of the flying body are changed to follow the movement of the excavator ( Paragraphs 0060 to 0061, FIG. 6A1).

International Publication No. 2017/131194

However, the method of Patent Document 1 has a problem that when inspecting an excavator attachment or the like using an air vehicle, it is difficult to capture a working state while the excavator is in operation. In particular, since only the flying object is controlled, there is a risk of collision with the flying object when the excavator changes suddenly.

The present invention has been made in view of the above points, and an object of the present invention is to provide a work machine capable of communicating with an air vehicle (unmanned aerial vehicle) and performing work safely.

The present invention relates to a lower traveling body, an upper rotating body capable of turning with respect to the lower traveling body, a working mechanism extending from the upper rotating body, the lower traveling body, the upper swinging body, and the working mechanism. A work machine including a control device for controlling each operation mode and a communication device for communicating with an unmanned airplane, wherein the work machine is at least one or more unmanned at a predetermined position of the work mechanism. The control device has a marker that can be recognized by the airplane, and the control device calculates the distance between the marker and the unmanned airplane based on the captured image of the marker captured by the image pickup device of the unmanned airplane. It is characterized by.

The work machine of the present invention has a marker at a predetermined position of a work mechanism (arm, boom, etc.), and the unmanned aerial vehicle recognizes the marker during flight. Since the control device of the work machine calculates the distance between the marker and the unmanned aerial vehicle based on the image captured by the unmanned aerial vehicle, it is possible to instruct the unmanned aerial vehicle about the flight position etc. according to the distance. it can. Since the image pickup device can recognize the marker and confirm the working state, it is not necessary to separately mount a distance measurement sensor or the like for distance measurement, and the weight and cost of the unmanned aerial vehicle can be reduced.

Further, in the work machine of the present invention, it is preferable that the marker is provided at a position between the cylinder brackets for driving the work mechanism.

The position between the cylinder brackets of the working mechanism is such that the cylinder is not present on its surface. Therefore, by providing the marker at this position, the unmanned aerial vehicle can easily recognize the marker.

Further, in the work machine of the present invention, the work mechanism includes a boom whose base end side is connected to the upper swing body, an arm connected to the tip end side of the boom, and a tip end connected to the tip end side of the arm. It is preferably composed of an attachment, and the marker is preferably provided at least on either the tip end side of the arm or the proximal end side opposite to the tip end side.

According to this configuration, when a marker is provided on the tip end side (tip attachment side) of the arm, the airborne position of the unmanned aerial vehicle is close to the tip attachment, which is advantageous for checking the work status. In addition, when a marker is provided on the base end side of the arm (the side close to the boom or the upper swing body), the position of the unmanned aerial vehicle is less likely to change due to the rotation of the arm, so that the position of the unmanned aerial vehicle with respect to the work mechanism is stable. Can be made to.

Further, in the work machine of the present invention, it is preferable to provide a relay marker for guiding the unmanned aerial vehicle to the marker at a predetermined position of the work mechanism in which the marker is not provided.

For example, in order to start an unmanned aerial vehicle and move the marker to a position where it can be recognized, a relay marker is provided at a predetermined position of a working mechanism without the marker. As a result, the unmanned aerial vehicle can reach a position where the marker can be visually recognized without making a detour along the relay marker.

The work machine support server of the present invention includes a lower traveling body, an upper rotating body capable of turning with respect to the lower traveling body, a working mechanism extending from the upper rotating body, the lower traveling body, and the upper rotating body. A work machine support server having a communication function with each of a work machine provided with a control device for controlling each operation mode of the work mechanism and an unmanned airplane equipped with an image pickup device, at least the above-mentioned. It is characterized by including a distance calculation element for calculating the distance between the work machine and the unmanned airplane from an image captured by a marker provided at a predetermined position of the work mechanism based on communication with the unmanned airplane.

According to the work machine support server of the configuration, the distance calculation element receives the captured image of the marker from the unmanned aerial vehicle and calculates the distance between the marker and the unmanned aerial vehicle. This distance information is used for controlling the unmanned aerial vehicle to be separated from the work machine when there is a high possibility of contact between the work mechanism and the unmanned aerial vehicle, and the possibility of contact is reduced.

Explanatory drawing about composition of work machine and unmanned aerial vehicle as one Embodiment of this invention. Explanatory drawing about the structure of the operation mechanism of a work machine. Flowchart of processing performed by a work machine and an unmanned aerial vehicle. Explanatory drawing (side view) explaining the marker provided in the work machine. Explanatory drawing (top view) explaining the marker provided in the work machine. The explanatory view about the structure of the work machine support server as one Embodiment of this invention.

(Constitution)
The work machine 10 as an embodiment of the present invention shown in FIG. 1 carries out a predetermined work in cooperation with the unmanned aerial vehicle 40. The work machine 10 is, for example, a crawler excavator (construction machine), and includes a crawler-type lower traveling body 110 and an upper rotating body 120 mounted on the lower traveling body 110 so as to be swivelable via a swivel mechanism 130. I have. A cab (driver's cab) 122 is provided on the front left side of the upper swivel body 120. A work attachment as a work mechanism 140 is provided in the front central portion of the upper swivel body 120.

The work mechanism 140 is rotatably connected to the boom 141 undulatingly mounted on the upper swing body 120, the arm 143 rotatably connected to the tip of the boom 141, and the tip of the arm 143. It is equipped with a bucket 145 and. The work mechanism 140 is equipped with a boom cylinder 142, an arm cylinder 144, and a bucket cylinder 146, which are composed of a telescopic hydraulic cylinder. Instead of the bucket 145, another tip attachment such as a nibbler may be attached to the tip of the arm 143.

The boom cylinder 142 is interposed between the boom 141 and the upper swing body 120 so as to expand and contract by receiving the supply of hydraulic oil and rotate the boom 141 in the undulating direction. The arm cylinder 144 expands and contracts by receiving the supply of hydraulic oil, and is interposed between the arm 143 and the boom 141 so as to rotate the arm 143 about a horizontal axis with respect to the boom 141.

The bucket cylinder 146 expands and contracts by receiving the supply of hydraulic oil and is interposed between the bucket 145 and the arm 143 so as to rotate the bucket 145 about the horizontal axis with respect to the arm 143.

The work machine 10 includes a work machine control device 20, a wireless communication device 202, an input interface 210, and an output interface 220. The work machine control device 20 (“control device” of the present invention) is composed of an arithmetic processing unit (single-core processor or multi-core processor or a processor core constituting the same), and obtains necessary data and software from a storage device such as a memory. It reads and executes arithmetic processing according to the software for the data.

The work machine control device 20 includes a distance calculation element 21 and a contact avoidance operation control element 22. In the present embodiment, the distance calculation element 21 calculates the distance between the work mechanism 140 and the unmanned aerial vehicle 40 based on the relative position of the unmanned aerial vehicle 40 with respect to the designated position provided at the predetermined position of the work mechanism 140. ..

Further, when the contact avoidance operation control element 22 recognizes that the contact possibility is high due to the distance calculated by the distance calculation element 21, the evacuation command signal or the like is given to the unmanned airplane 40 so that the contact possibility is low. To send.

The operation mechanism 211 constituting the input interface 210 includes a traveling operation device, a turning operation device, a boom operation device, an arm operation device, and a bucket operation device. Each operating device has an operating lever that receives a rotation operation. The operation lever (travel lever) of the travel operation device is operated to move the lower traveling body 110. The travel lever may also serve as a travel pedal. For example, a traveling pedal fixed to the base or the lower end of the traveling lever may be provided.

The operation lever (swivel lever) of the swivel operation device is operated to move the hydraulic swivel motor constituting the swivel mechanism 130. The operating lever (boom lever) of the boom operating device is operated to move the boom cylinder 142. The operating lever (arm lever) of the arm operating device is operated to move the arm cylinder 144. The operating lever (bucket lever) of the bucket operating device is operated to move the bucket cylinder 146.

Each operating lever constituting the operating mechanism 211 is arranged around the seat S for the operator to sit on, for example, as shown in FIG. The seat S is in the form of a high back chair with armrests, but may be in any form in which the operator can sit, such as a low back chair without a headrest or a chair without a backrest.

A pair of left and right traveling levers 2110 corresponding to the left and right crawlers are arranged side by side in front of the seat S. One operating lever may also serve as a plurality of operating levers. For example, when the right operating lever 2111 provided in front of the right frame of the seat S shown in FIG. 2 functions as a boom lever when operated in the front-rear direction and is operated in the left-right direction. May function as a bucket lever.

Similarly, the left side operating lever 2112 provided in front of the left side frame of the seat S shown in FIG. 2 functions as an arm lever when operated in the front-rear direction and is operated in the left-right direction. In some cases, it may function as a swivel lever. The lever pattern may be arbitrarily changed according to the operation instruction of the operator.

The image output device 222 constituting the output interface 220 is arranged in front of the sheet S as shown in FIG. The image output device 222 may further include a speaker (audio output device).

The unmanned aerial vehicle 40 includes an airplane control device 400, a wireless communication device 402, and an image pickup device 410. The unmanned aerial vehicle 40 is a rotary wing machine, and includes a plurality of (for example, 4, 6 or 8 blades) blades, an electric motor (actuator) for rotating the plurality of blades, a battery for supplying electric power to the motor, and the like. ing.

Further, the unmanned aerial vehicle 40 can be operated through the remote control device 50. For example, the input interface 210 may constitute a remote control device for the unmanned aerial vehicle 40.

The unmanned aerial vehicle 40 mainly confirms the working status of the work machine 10 and inspects a specific part. The image (still image or moving image) captured by the image pickup device 410 is displayed on the image output device 222 of the work machine 10. For example, an environmental image including the bucket 145 when the work machine 10 is destroying the building is displayed on the image output device 222.

The captured image is transmitted from the wireless communication device 402 of the unmanned airplane 40 to the wireless communication device 202 of the work machine 10 (“communication device” of the present invention), and the work machine control device 20 connects the work machine 10 and the unmanned airplane 40. It is also used to calculate the distance.

In the present embodiment, a marker is provided on the back surface (the surface opposite to the ground) of the work mechanism 140 (boom 141, arm 143, etc.). The unmanned aerial vehicle 40 recognizes the marker and hovering in the vicinity of the marker while maintaining a constant interval. Then, the marker is imaged at a predetermined timing, and the captured image is transmitted to the work machine 10.

After that, the work machine control device 20 of the work machine 10 calculates the distance between the marker and the unmanned aerial vehicle 40 based on the transmitted captured image. If the distance between the work machine 10 and the unmanned aerial vehicle 40 is known, the work machine 10 sends a signal to the unmanned aerial vehicle 40 to evacuate the unmanned aerial vehicle 40 when it is determined that the mutual distance is shorter than a predetermined interval. Can be done. In this way, the distance information between the work machine 10 and the unmanned aerial vehicle 40 can be used for control that reduces the possibility of contact between the two.

It is also possible to mount a distance measuring sensor or a TOF sensor on the unmanned aerial vehicle 40 and estimate the relative position of the work machine 10 and the unmanned aerial vehicle 40 based on the distance measuring data. However, if the image pickup device 410 is provided, the distance between the unmanned aerial vehicles 40 can be acquired without separately providing these sensors, so that the unmanned aerial vehicle 40 has a simpler configuration, and is lightweight and cost-effective. Is also connected.

(function)
Next, with reference to FIG. 3, a flowchart regarding each process of the work machine 10 and the unmanned aerial vehicle 40 and transmission / reception of signals will be described.

First, in the unmanned aerial vehicle 40, the airplane control device 400 receives the flight command signal transmitted from the remote control device 50 through the wireless communication device 402 (STEP 402). The airplane control device 400 controls the rotational operation of the actuator (electric motor) and thus the plurality of blades powered by the actuator according to the flight command signal (STEP404). As a result, the unmanned aerial vehicle 40 can fly by the air flow generated by rotating a plurality of blades, rise or fall on the spot, or stay (hover) in the air.

In the unmanned aerial vehicle 40, the image pickup device 410 acquires a captured image including a specific part of the work mechanism 140, such as a bucket 145 as an attachment tip portion (STEP 406). For example, when the work machine 10 is destroying a building, the bucket 145 is imaged from diagonally above by the image pickup device 410 mounted on the unmanned aerial vehicle 40.

The airplane control device 400 transmits the captured image data representing the captured image to the work machine 10 through the wireless communication device 402 (STEP 408). This completes a series of processes for the unmanned aerial vehicle 40 and returns to STEP 402.

Next, in the work machine 10, the work machine control device 20 receives the captured image data through the wireless communication device 202 (STEP 202). The work machine control device 20 displays an environment image (all or part of the captured image itself, or a simulated environment image generated based on the captured image data) on the image output device 222 (STEP204). ). For example, an environmental image including the bucket 145 when the work machine 10 is destroying the building is displayed on the image output device 222.

After that, the work machine control device 20 recognizes the relative position of the unmanned aerial vehicle 40 with respect to the work mechanism 140 (STEP206). Specifically, a marker having a predetermined shape is attached to a designated portion of the boom 141, arm 143 or bucket 145 of the work machine 10. Then, by inquiring the image database based on the shape and size of the marker in the captured image, the relative position of the unmanned aerial vehicle 40 with respect to the designated location is estimated.

Further, by querying the image database based on the shape and size of the boom 141, the arm 143, and the bucket 145 in the captured image, the relative position of the unmanned airplane 40 with respect to each of the boom 141, the arm 143, and the bucket 145. May be estimated.

Next, in the work machine control device 20, the distance between the marker and the unmanned aerial vehicle 40 is calculated (STEP208). After that, in the work machine control device 20, it is determined whether or not the marker and the unmanned aerial vehicle 40 are separated by a predetermined interval or more (STEP210). For example, when the marker and the unmanned aerial vehicle 40 are shorter than a predetermined interval, there is a high possibility that the working mechanism 140 and the unmanned aerial vehicle 40 come into contact with each other.

When it is determined that the working mechanism 140 and the unmanned aerial vehicle 40 are separated by a predetermined distance or more (“YES” in STEP 210), a stagnation command signal is transmitted to the unmanned aerial vehicle 40 (STEP 212). The stagnation command signal is a signal that allows the unmanned aerial vehicle 40 to hover at that position.

On the other hand, when it is determined that the working mechanism 140 and the unmanned aerial vehicle 40 are not separated by a predetermined interval or more (“NO” in STEP 210), an evacuation command signal is transmitted to the unmanned aerial vehicle 40 (STEP 212). The evacuation command signal is a signal for urging the unmanned aerial vehicle 40 to evacuate because hovering at that position may cause contact. This completes a series of processes of the work machine 10, and returns to STEP 202.

Next, with reference to FIGS. 4 and 5, a marker provided on the work machine 10 for hovering positioning of the unmanned aerial vehicle 40 will be described.

FIG. 4 is a side view of the work machine 10 as viewed from the side. In the work machine 10, a work mechanism 140 such as a boom 141, an arm 143, and a bucket 145 operates during work. Then, the unmanned aerial vehicle 40 captures the state of work from the position above the work mechanism 140 by the image pickup device 410.

In the present embodiment, a marker M is provided as a reference for the position on the back surface of the rear end portion (base end side near the boom 141 and the upper swing body 120) of the arm 143. The marker M has a predetermined shape such as a round shape or a rectangular shape, and preferably has a color different from that of the arm 143.

When the marker M is provided at the rear end of the arm 143, the position of the unmanned aerial vehicle 40 with respect to the working mechanism 140 can be stabilized because the position of the unmanned aerial vehicle 40 does not change much due to the rotational operation of the arm 143. Since the unmanned aerial vehicle 40 receives a command signal from the work machine 10 and separates from the work machine 10 if necessary, it is possible to perform work such as inspection while avoiding contact with the work machine 10.

For example, when the marker M is provided at the tip of the arm 143 (the tip side near the bucket 145), the airborne position of the unmanned aerial vehicle 40 is close to the bucket 145, which is advantageous for checking the working status. The position of the marker M may be a plurality of positions (for example, the front end portion and the rear end portion of the arm 143) as long as they are on the back surface of the arm 143 visible from the unmanned aerial vehicle 40. Further, the marker M may be provided at the tip end portion of the boom 141 (the tip end side close to the arm 143).

FIG. 5 is a top view of the work machine 10 as viewed from above. As shown in the figure, a bucket cylinder 146 exists on the upper surface side of the arm 143. Therefore, the marker M may be arranged between the brackets of the bucket cylinder 146. As a result, the marker M is always recognizable from the image pickup device 410 of the unmanned aerial vehicle 40.

By hovering the unmanned aerial vehicle 40 at a position above the marker M, it is easy to image the entire work area of the work machine 10. Further, when it is desired to move in the direction of the bucket 145 to take an image, the moving distance can be short.

In addition, a marker may be provided on or near the swivel axis of the upper swivel body 120. The marker X on the turning axis may be used as a reference for the unmanned aerial vehicle 40 such as the marker M, but can be used as a mark when the unmanned aerial vehicle 40 is moved from the ground to the sky. By doing so, even if the work mechanism 140 rotates with the rotation of the upper swivel body 120, it does not come into contact with the unmanned aerial vehicle 40, and even if the work mechanism 140 stands up in the vertical direction, the unmanned aerial vehicle 40 It is unlikely to come into contact.

Further, in the present embodiment, the relay marker Y is provided between the rear end portion of the boom 141 (the base end side close to the upper swing body 120) and the central portion thereof (for example, between the brackets of the arm cylinder 144). After its activation, the unmanned aerial vehicle 40 recognizes the relay marker Y (for example, an arrow) and moves in a predetermined direction when moving from the ground to the sky. As a result, the unmanned aerial vehicle 40 can reach the marker M without making a detour.

The unmanned aerial vehicle 40 may move away from the sky position of the marker M depending on the work, but when the image pickup device 410 recognizes the relay marker Y, it can be relayed and returned to the sky position of the marker M. At the moment when the unmanned aerial vehicle 40 recognizes the relay marker Y, the aircraft is mainly ascending, so it is desirable to decelerate or stop the lower traveling body 110 and the working mechanism 140. As a result, contact between the work machine 10 and the unmanned aerial vehicle 40 can be avoided.

As described above, the work machine 10 can avoid contact with the unmanned aerial vehicle 40 by communicating with the unmanned aerial vehicle 40 to acquire information on the mutual distance and maintaining a predetermined interval or more. This time, an example of a crawler excavator has been described as a work machine, but the present invention is not limited to this, and the present invention can be applied to various work machines such as cranes and construction demolition machines.

For example, in the case of a construction demolition machine, the work is performed in a posture in which the arm connected to the demolition attachment at the tip is substantially horizontal to the ground during the work. Therefore, it is advisable to provide a reference marker near the end of the arm on the opposite side of the dismantling attachment. At this position, the part of the dismantled attachment that cannot be seen by the driver can be imaged, and the crushed material is less scattered. Further, since it is close to the turning center, it is difficult to come into contact with the unmanned aerial vehicle 40 even if the arm stands up.

Further, in the case of a construction demolition machine, a plurality of connected booms are often in a posture substantially perpendicular to the ground during work. Therefore, it is advisable to provide a relay marker on at least one or each boom. For example, when moving the unmanned aerial vehicle 40 from the ground to a marker on the arm, the relay marker may be followed and ascended.

(Other Embodiments of the present invention)
The work machine support server 30 as an embodiment of the present invention shown in FIG. 6 has a communication function with each of the work machine 10 and the unmanned aerial vehicle 40. The work machine support server 30 includes the above-mentioned distance calculation element 21 and a contact avoidance operation control element 22.

In the work machine support server 30 having the configuration, the distance calculation element 21 receives the captured image of the marker captured by the unmanned aerial vehicle 40, recognizes the relative position of the unmanned aerial vehicle 40 with respect to the marker, and calculates the distance between the two. For example, when the calculated distance is shorter than the predetermined interval, there is a high possibility that the work mechanism 140 and the unmanned aerial vehicle 40 come into contact with each other. Therefore, the contact avoidance operation control element 22 with respect to the unmanned aerial vehicle 40 (or the remote control device 50). Send a save command signal. Then, by pulling the unmanned aerial vehicle 40 away from the work machine 10, the contact possibility can be reduced.

The contact avoidance operation control element 22 may transmit a movement signal for contact avoidance to the work machine 10. Accordingly, the work machine control device 20 controls the operation mode of the work mechanism 140 and the like so as to reduce the contact possibility. That is, when it is recognized that the work mechanism 140 of the work machine 10 and the unmanned aerial vehicle 40 have a high contact possibility, the contact possibility can be reduced by controlling the operation of the work mechanism 140 or the like.

10 ... work machine, 20 ... work machine control device, 21 ... distance calculation element, 22 ... contact avoidance operation control element, 40 ... unmanned airplane, 202 ... wireless communication equipment, 210 ... input interface, 211 ... operation mechanism, 220 ... output Interface, 222 ... image output device, 140 ... work mechanism, 400 ... airplane control device, 402 ... wireless communication equipment, 410 ... imaging device.

Claims (5)

Each operation mode of the lower traveling body, the upper rotating body capable of turning with respect to the lower traveling body, the working mechanism extending from the upper rotating body, the lower traveling body, the upper turning body, and the working mechanism. It is a work machine equipped with a control device for controlling the air conditioner and a communication device for communicating with an unmanned airplane.
The work machine has at least one or more markers recognizable by the unmanned aerial vehicle at a predetermined position of the work mechanism.
The control device is a work machine that calculates a distance between the marker and the unmanned airplane based on an image captured by the marker taken by an image pickup device included in the unmanned airplane. In the work machine according to claim 1,
A work machine characterized in that the marker is provided at a position between cylinder brackets for driving the work mechanism. In the work machine according to claim 1,
The working mechanism is composed of a boom whose base end side is connected to the upper swing body, an arm connected to the tip end side of the boom, and a tip attachment connected to the tip end side of the arm.
A work machine characterized in that the marker is provided at least on either the tip end side of the arm or the proximal end side opposite to the tip end side. In the work machine according to any one of claims 1 to 3,
A work machine characterized in that a relay marker for guiding the unmanned aerial vehicle to the marker is provided at a predetermined position of the work mechanism in which the marker is not provided. Each operation mode of the lower traveling body, the upper rotating body capable of turning with respect to the lower traveling body, the working mechanism extending from the upper rotating body, the lower traveling body, the upper turning body, and the working mechanism. A work machine support server having a communication function with each of a work machine equipped with a control device for controlling the above and an unmanned airplane equipped with an image pickup device.
It is characterized by including a distance calculation element for calculating the distance between the work machine and the unmanned airplane from an image captured by a marker provided at a predetermined position of the work mechanism, at least based on communication with the unmanned airplane. Work machine support server.

Images