JP2023150081A - Work machine and information processing unit - Google Patents

Abstract

To provide a technology capable of estimating a travel area of a work machine.SOLUTION: A crawler crane 100 in accordance with an embodiment of the present invention is provided with: a distance sensor 45 for acquiring data on objects around the crawler crane 100; a feature quantity extraction unit 303 for acquiring feature quantities regarding unique features at positions corresponding to the ground around the crawler crane 100 based on an output of the distance sensor 45; and a travel area estimation unit 304 for estimating a travelable area of a work machine or an area around the crawler crane 100 to allow it to travel based on the feature quantities extracted by the feature quantity extraction unit 303.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to work machines and the like.

Techniques for recognizing continuous ground are known (for example, see Patent Document 1).

JP2020-94834A

By the way, at a work site, there are two areas: areas where work machines such as cranes and shovels can run or are allowed to run (hereinafter referred to as "travel areas") and areas where they cannot run or are prohibited (hereinafter referred to as "non-travel areas"). ”) may be stipulated. For example, at a work site on the ground, a rectangular iron plate may be laid on the ground surface in the driving area, and the ground surface such as dirt may be exposed in the non-driving area. In addition, at work sites on relatively upper floors of buildings with two or more floors under construction, steel plates are similarly laid in the driving area, and the concrete floor surface may be exposed in the non-travelling area. be.

However, in Patent Document 1, although it is possible to recognize a continuous ground, it is not possible to recognize whether the ground is a running area.

Therefore, in view of the above problems, it is an object of the present invention to provide a technique capable of estimating the travel area of a working machine.

To achieve the above object, in one embodiment of the present disclosure,
a sensor that acquires data about objects around the work machine;
an estimation unit that estimates a first area on the ground around the working machine in which the working machine can run or is allowed to run, based on the output of the sensor;
Working machinery is provided.

Additionally, in other embodiments of the present disclosure,
Obtaining the output of a sensor that obtains data regarding objects within a predetermined range, and estimating a first region in the predetermined range in which the work machine can run or is allowed to run based on the output of the sensor. do,
An information processing device is provided.

According to the embodiment described above, it is possible to estimate the travel area of the working machine.

FIG. 1 is a diagram showing an example of a working machine. FIG. 1 is a diagram showing an example of an operation support system. It is a block diagram showing an example of the hardware configuration of a crawler crane. FIG. 2 is a functional block diagram showing a first example of a functional configuration related to estimating a travel area of a crawler crane. 2 is a flowchart schematically showing a first example of a main routine process related to estimating a driving area. 3 is a flowchart schematically showing an example of a subroutine process related to estimating a driving area. FIG. 2 is a diagram showing an example of an image displayed on a display device and representing a running area around a crawler crane. FIG. 3 is a functional block diagram showing a second example of a functional configuration related to estimating a driving area. 12 is a flowchart schematically showing a second example of main routine processing related to estimating a driving area. It is a figure showing other examples of a work machine.

Embodiments will be described below with reference to the drawings.

[Overview of working machine]
An overview of the working machine according to this embodiment will be explained with reference to FIGS. 1 and 2.

FIG. 1 is a diagram showing an example of a working machine. Specifically, FIG. 1 is a side view showing an example of a crawler crane 100. FIG. 2 is a diagram showing an example of the operation support system SYS.

Hereinafter, the direction in which the boom 4 extends when viewed from the top of the crawler crane 100 is defined as "front", and the direction in the crawler crane 100 or the direction seen from the crawler crane 100 may be explained.

Crawler crane 100 is an example of a working machine. The crawler crane 100 includes a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1 via a rotating mechanism 2, a boom 4, a mast 5, a backstop 6, and a main hoisting rope. 7, a hook HK, a counterweight 9, and a cabin 10.

The lower traveling body 1 causes the crawler crane 100 to travel. The undercarriage body 1 includes, for example, a pair of left and right crawlers 1C. The crawler 1C includes one left and right crawler 1CL and the other left and right crawler 1CR. The crawler 1CL is hydraulically driven by a travel hydraulic motor 1ML (see FIG. 3). Similarly, the crawler 1CR is hydraulically driven by a traveling hydraulic motor 1MR (see FIG. 3). Thereby, the lower traveling body 1 can self-propel.

The upper rotating body 3 is rotatably mounted on the lower traveling body 1 via the rotating mechanism 2 . Specifically, the upper rotating structure 3 turns with respect to the lower traveling structure 1 by hydraulically driving the turning mechanism 2 by a turning hydraulic motor 2M (see FIG. 3).

The boom 4 is attached to the center of the front part of the upper revolving body 3 in the left-right direction so as to be able to rise and fall. A main hoisting rope 7 is suspended from the tip of the boom 4, and a hook HK is attached to the tip of the main hoisting rope 7. That is, a hook HK is attached to the tip of the boom 4 via the main winding rope 7.

The mast 5 is attached slightly behind the base end of the boom 4 of the upper revolving body 3 so as to be rotatable around a rotation axis parallel to the rotation axis of the boom 4 . The tip of the mast 5 is connected to the tip of the boom 4 via a pendant rope 5a. The boom 4 is hoisted via the mast 5 by winding and unwinding the boom hoisting rope 5b by the boom hoisting winch 5c which is hydraulically driven by the hoisting hydraulic motor 5M (see FIG. 3).

The backstop 6 is attached so that its base end can rotate around a rotation axis parallel to the rotation axis of the boom 4 at a portion of the upper rotating structure 3 that is rearward of the base end of the boom 4 . Further, the backstop 6 is attached such that its tip is rotatable around a rotation axis parallel to the rotation axis of the boom 4 at a rear portion between the base end and the tip of the boom 4 . The backstop 6 expands and contracts in accordance with the up-and-down motion of the boom 4, and has the function of supporting the boom 4 from behind, for example, when the boom 4 is in a substantially upright state.

The main hoisting rope 7 has its base end attached to a main hoisting winch 7a attached to the rear portion between the base end and the tip of the boom 4, and its tip end is attached to the hook HK. The main winding rope 7 is wound up and unwound by a main winding winch 7a which is hydraulically driven by a main winding hydraulic motor 7M (see FIG. 3), so that the hook HK can be moved up and down.

The hook HK is attached to the tip of the main winding rope 7 and is used to hang a suspended load.

The counterweight 9 is provided at the rear end of the revolving upper structure 3 and has the function of balancing the weight of the boom 4 and the suspended load.

The cabin 10 is attached to the front right side of the upper revolving body 3, for example. Inside the cabin 10, a cockpit, an operating device 26 (see FIG. 3) for operating various actuators, and the like are provided.

The crawler crane 100 operates an actuator (for example, a hydraulic actuator) in response to an operation by an operator riding in a cabin 10, and operates driven elements such as a lower traveling body 1, an upper rotating body 3, a boom 4, and a main hoisting rope 7. to drive.

Moreover, the crawler crane 100 may be configured to be operable by an operator in the cabin 10, or in addition, to be configured to be remotely operated from outside the crawler crane 100. When the crawler crane 100 is remotely operated, the interior of the cabin 10 may be unmanned. Moreover, when the crawler crane 100 is remotely operated, the cabin 10 may be omitted. The following description will proceed on the premise that the operator's operations include at least one of operations on the operating device 26 by the operator of the cabin 10 and remote operations by an external operator.

The remote operation includes, for example, a mode in which the crawler crane 100 is operated by input from a user (operator) regarding an actuator of the crawler crane 100 performed by a remote operation support device external to the crawler crane 100. The remote operation support device is, for example, the support device 200 described below. In this case, the crawler crane 100 transmits, for example, image information around the crawler crane 100 (hereinafter referred to as "surrounding image") based on the output of the imaging device 40 (described later) to the support device 200, and the image information is transmitted to the support device 200. may be displayed on a display device (hereinafter referred to as a "remote control display device") provided in the remote control display device. Further, various information images (information screens) displayed on the output device 50 (display device) in the cabin 10 of the crawler crane 100 may be similarly displayed on the remote control display device of the support device 200. As a result, the operator using the support device 200 can, for example, check the display content of the crawler crane 100 on the remote control display device, such as peripheral images and various information images showing the surroundings of the crawler crane 100. 100 can be controlled remotely. Then, the crawler crane 100 operates the actuators to operate the lower traveling body 1, the upper rotating body 3, the boom 4, and the main hoisting rope 7 in accordance with a remote control signal representing the content of the remote control received from the support device 200. may drive driven elements such as.

Furthermore, remote control includes, for example, a mode in which the crawler crane 100 is operated by a person (e.g., a worker) around the crawler crane 100 giving instructions to the crawler crane 100 by external voice input, gesture input, etc. It's fine. Specifically, the crawler crane 100 uses sounds uttered by surrounding workers and the like through an audio input device (for example, a microphone), an imaging device, etc. mounted on the crawler crane 100 (own machine). Recognize gestures, etc. Then, the crawler crane 100 operates the actuator according to the content of the instruction corresponding to the recognized voice or gesture, and drives the lower traveling body 1, the upper rotating body 3, the boom 4, the main hoisting rope 7, etc. May drive elements.

Furthermore, the crawler crane 100 may automatically operate the actuator without depending on an operator's operation. As a result, the crawler crane 100 has a function of automatically operating at least some of the driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, and the main hoisting rope 7, that is, the so-called "automatic operation function". Realizes "Machine Control (MC) function".

The automatic operation function may include a function of automatically operating a driven element (actuator) other than the driven element (actuator) to be operated in response to an operator's operation on the operating device 26 or remote control. That is, the automatic driving function may include a so-called "semi-automatic driving function" or an "operation support type MC function." Further, the automatic operation function may include a function of automatically operating at least a portion of the plurality of driven elements (hydraulic actuators) on the premise that there is no operator operation on the operating device 26 or remote control. That is, the automatic driving function may include a so-called "fully automatic driving function" or a "fully automatic MC function." When the fully automatic operation function of the crawler crane 100 is enabled, the interior of the cabin 10 may be unmanned. Moreover, when the crawler crane 100 operates with a fully automatic operation function, the cabin 10 may be omitted. Further, the semi-automatic driving function, fully automatic driving function, etc. may include a mode in which the operation details of a driven element (actuator) that is a target of automatic driving are automatically determined according to predefined rules. In addition, for semi-automatic operation functions, fully automatic operation functions, etc., the crawler crane 100 autonomously makes various judgments, and based on the judgment results, autonomously controls the driven elements (hydraulic actuators) that are subject to automatic operation. A manner in which the operation content is determined may also be included. That is, the semi-automatic driving function, fully automatic driving function, etc. may include a so-called "autonomous driving function."

Further, the work of the crawler crane 100 may be remotely monitored. In this case, a remote monitoring support device having the same functions as the remote operation support device may be provided. The remote monitoring support device is, for example, the support device 200 described below. Thereby, the supervisor who is the user of the support device 200 can monitor the work status of the crawler crane 100 while checking the peripheral image displayed on the display device of the support device. For example, if the supervisor determines that it is necessary from a safety perspective, the supervisor may intervene in the operator's operation of the crawler crane 100 and make an emergency stop by inputting a predetermined input using the input device of the remote monitoring support device. can be done.

Further, as shown in FIG. 2, the crawler crane 100 is communicably connected to the support device 200 through the communication line NW using the communication device 60 (see FIG. 3), and together with the support device 200, the crawler crane 100 is connected to the operation support system SYS. It may be a component of

The operation support system SYS uses the support device 200 to provide support regarding the operation of the crawler crane 100.

The support device 200 cooperates with the crawler crane 100 by communicating with the crawler crane 100, and provides support regarding the operation of the crawler crane 100.

The functions of the support device 200 are realized by arbitrary hardware or a combination of arbitrary hardware and software. For example, the support device 200 is mainly configured with a computer including a CPU (Central Processing Unit), a memory device, an auxiliary storage device, an interface device, an input device, and an output device. The memory device is, for example, SRAM (Static Random Access Memory) or DRAM (Dynamic Random Access Memory). Examples of the auxiliary storage device include a HDD (Hard Disc Drive), an SSD (Solid State Disc), an EEPROM (Electrically Erasable Programmable Read-Only Memory), and a flash memory. The interface device includes an external interface for connecting to an external recording medium and a communication interface for communicating with the outside, such as the crawler crane 100. The input device includes, for example, a lever-type operation input device. The output device includes a display device and a sound output device. The display device is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. The sound output device is, for example, a speaker.

The support device 200 is installed, for example, in a management office within the work site of the crawler crane 100, or in a management center that manages the operating status of the crawler crane 100, etc., located at a location different from the work site of the crawler crane 100. It is a server and a terminal device for management. The management terminal device may be a stationary terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile phone) such as a tablet terminal, smartphone, or laptop PC. terminal). In the latter case, workers at the work site, supervisors who supervise work, managers who manage the work site, and the like can carry the portable support device 200 and move around the work site. In the latter case, the operator can bring the portable support device 200 into the cabin of the crawler crane 100, for example.

For example, the support device 200 acquires data regarding the operating state from the crawler crane 100. Thereby, the support device 200 can grasp the operating state of the crawler crane 100 and monitor the presence or absence of an abnormality in the crawler crane 100. Further, the support device 200 can display data regarding the operating state of the crawler crane 100 through its own display device for the user to confirm.

Further, the support device 200 may transmit various data such as programs and reference data to the crawler crane 100 that are used in processing by the controller 30 and the like. Thereby, the crawler crane 100 can perform various processes related to the operation of the crawler crane 100 using various data downloaded from the support device 200.

Further, the support device 200 may support remote operation and remote monitoring of the crawler crane 100, as described above.

[Crawler crane hardware configuration]
Next, the hardware configuration of the crawler crane 100 will be described with reference to FIG. 3 in addition to FIGS. 1 and 2.

The crawler crane 100 includes a hydraulic drive system for hydraulically driving driven elements, an operation system for operating driven elements, a user interface system for exchanging information with users, a communication system for communicating with the outside, and a control system for various controls. Contains each component such as.

<Hydraulic drive system>
As shown in FIG. 3, the hydraulic drive system of the crawler crane 100 according to the present embodiment includes the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, the boom 4, the main hoisting rope 7, etc. includes a hydraulic actuator HA for hydraulically driving each of the driven elements. Further, the hydraulic drive system of the crawler crane 100 according to the present embodiment includes an engine 11, a regulator 13, a main pump 14, and a control valve 17.

The hydraulic actuator HA includes travel hydraulic motors 1ML and 1MR, a swing hydraulic motor 2M, a hoisting hydraulic motor 5M, a main winding hydraulic motor 7M, and the like.

Note that in the crawler crane 100, part or all of the hydraulic actuator HA may be replaced with an electric actuator.

The engine 11 is a prime mover and a main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, at the rear of the upper revolving structure 3. The engine 11 rotates at a predetermined target rotation speed under direct or indirect control by a controller 30, which will be described later, and drives the main pump 14 and the pilot pump 15.

The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30 . For example, the regulator 13 adjusts the angle of the swash plate (hereinafter referred to as "tilt angle") of the main pump 14 in accordance with a control command from the controller 30.

The main pump 14 supplies hydraulic oil to the control valve 17 through a high pressure hydraulic line. The main pump 14 is, for example, mounted at the rear of the upper revolving structure 3, like the engine 11. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, the stroke length of the piston is adjusted by adjusting the tilt angle of the swash plate by the regulator 13 under the control of the controller 30, and the stroke length of the piston is adjusted. The flow rate (discharge pressure) is controlled.

The control valve 17 drives the hydraulic actuator HA in accordance with the contents of an operator's operation on the operating device 26 or remote control, or an operation command regarding an automatic operation function output from the controller 30. The control valve 17 is mounted, for example, in the center of the upper revolving body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line, and controls the hydraulic oil supplied from the main pump 14 according to an operator's operation or an operation command output from the controller 30. , selectively supplying each hydraulic actuator. Specifically, the control valve 17 includes a plurality of control valves (also referred to as "direction switching valves") that control the flow rate and flow direction of the hydraulic oil supplied from the main pump 14 to each of the hydraulic actuators HA.

<Operation system>
As shown in FIG. 3, the operating system of the crawler crane 100 according to the present embodiment includes a pilot pump 15, an operating device 26, a hydraulic control valve 31, a shuttle valve 32, and a hydraulic control valve 33.

The pilot pump 15 supplies pilot pressure to various hydraulic devices via a pilot line 25. The pilot pump 15 is, for example, mounted at the rear of the upper revolving structure 3, like the engine 11. The pilot pump 15 is, for example, a fixed capacity hydraulic pump, and is driven by the engine 11 as described above.

Note that the pilot pump 15 may be omitted. In this case, the relatively high pressure hydraulic oil discharged from the main pump 14 is reduced in pressure by a predetermined pressure reducing valve, and then the relatively low pressure hydraulic oil is supplied as pilot pressure to various hydraulic devices.

The operating device 26 is provided near the cockpit of the cabin 10 and is used by an operator to operate various driven elements. In other words, the operating device 26 is used by an operator to operate the hydraulic actuator HA that drives each driven element. The operating device 26 includes a pedal device and a lever device for operating each driven element (hydraulic actuator HA).

For example, as shown in FIG. 3, the operating device 26 is of a hydraulic pilot type. Specifically, the operating device 26 uses hydraulic oil supplied from the pilot pump 15 through the pilot line 25 and a pilot line 25A branched from the pilot line 25, and applies pilot pressure according to the operation content to the secondary pilot line. Output to 27A. Pilot line 27A is connected to one inlet port of shuttle valve 32 and connected to control valve 17 via pilot line 27, which is connected to an outlet port of shuttle valve 32. As a result, pilot pressure can be input to the control valve 17 via the shuttle valve 32 in accordance with the operation contents regarding various driven elements (hydraulic actuators) in the operating device 26 . Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation performed on the operating device 26 by an operator or the like.

Moreover, the operating device 26 may be electrical. Specifically, the operating device 26 outputs an electrical signal (hereinafter referred to as an "operating signal") according to the content of the operation, and the operating signal is taken into the controller 30. Then, the controller 30 outputs a control command according to the content of the operation signal, that is, a control signal according to the content of the operation on the operating device 26 to the hydraulic control valve 31. As a result, pilot pressure corresponding to the operation details of the operating device 26 is inputted from the hydraulic control valve 31 to the control valve 17, and the control valve 17 drives each hydraulic actuator HA according to the operation details of the operating device 26. be able to.

Note that when the operating device 26 is of an electric type, the operating pressure sensor 29, shuttle valve 32, and hydraulic control valve 33, which will be described later, are omitted.

Furthermore, the control valves (directional switching valves) built into the control valve 17 and driving the respective hydraulic actuators may be of an electromagnetic solenoid type. In this case, the operation signal output from the operation device 26 may be directly input to the control valve 17, that is, to an electromagnetic solenoid type control valve.

Further, as described above, part or all of the hydraulic actuator HA may be replaced with an electric actuator. In this case, the controller 30 may output a control command according to the operation content of the operating device 26 or the remote control content specified by the remote control signal to the electric actuator or a driver driving the electric actuator.

Note that when the crawler crane 100 is remotely controlled or operates with an automatic operation function, the operating device 26 may be omitted.

It is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 and for each driving direction (for example, the raising direction and lowering direction of the boom 4) of the driven element (hydraulic actuator HA). That is, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA. The hydraulic control valve 31 is provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and is configured to be able to change its flow path area (that is, the cross-sectional area through which hydraulic oil can flow). good. Thereby, the hydraulic control valve 31 can output a predetermined pilot pressure to the secondary side pilot line 27B using the hydraulic oil of the pilot pump 15 supplied through the pilot line 25B. Therefore, as shown in FIG. 3, the hydraulic control valve 31 indirectly applies a predetermined pilot pressure according to a control signal from the controller 30 to the control valve through the shuttle valve 32 between the pilot line 27B and the pilot line 27. 17. Therefore, the controller 30 can supply pilot pressure from the hydraulic control valve 31 to the control valve 17 according to the operation details of the operating device 26, and can realize the operation of the crawler crane 100 based on the operator's operation.

Further, the controller 30 may, for example, control the hydraulic control valve 31 to realize an automatic driving function. Specifically, the controller 30 outputs a control signal corresponding to an operation command related to the automatic driving function to the hydraulic control valve 31 regardless of whether or not the operating device 26 is operated. Thereby, the controller 30 can cause the hydraulic control valve 31 to supply the control valve 17 with pilot pressure corresponding to the operation command related to the automatic operation function, and can realize the operation of the crawler crane 100 based on the automatic operation function.

Further, the controller 30 may control the hydraulic control valve 31 to realize remote control of the crawler crane 100, for example. Specifically, the controller 30 outputs to the hydraulic control valve 31, through the communication device 60, a control signal corresponding to the content of the remote control specified by the remote control signal received from the support device 200. Thereby, the controller 30 can cause the hydraulic control valve 31 to supply pilot pressure corresponding to the content of the remote control to the control valve 17, and realize the operation of the crawler crane 100 based on the operator's remote control.

The shuttle valve 32 has two inlet ports and one outlet port, and outputs hydraulic fluid having a higher pilot pressure of the pilot pressures input to the two inlet ports to the outlet port. The shuttle valve 32 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26. One of the two inlet ports of the shuttle valve 32 is connected to the pilot line 27A on the secondary side of the operating device 26 (specifically, the above-mentioned lever device or pedal device included in the operating device 26), and the other is It is connected to the pilot line 27B on the secondary side of the hydraulic control valve 31. The outlet port of shuttle valve 32 is connected to the pilot port of the corresponding control valve of control valve 17 through pilot line 27 . The corresponding control valve is a control valve that drives a hydraulic actuator that is operated by the above-mentioned lever device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, these shuttle valves 32 each control the higher of the pilot pressure in the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure on the pilot line 27B on the secondary side of the hydraulic control valve 31, respectively. It can act on the pilot port of the control valve. In other words, the controller 30 controls the corresponding control valve by causing the hydraulic control valve 31 to output a pilot pressure higher than the pilot pressure on the secondary side of the operating device 26, regardless of the operator's operation on the operating device 26. be able to. Therefore, the controller 30 controls the operation of the driven elements (the lower traveling structure 1, the upper revolving structure 3, the boom 4, and the main hoisting rope 7) regardless of the operation state of the operating device 26 by the operator, and performs the remote control function. and autonomous driving functions.

The hydraulic control valve 33 is provided in the pilot line 27A that connects the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured to be able to change its flow path area, for example. The hydraulic control valve 33 operates according to a control signal input from the controller 30. Thereby, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is being operated by the operator. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. Further, the controller 30 can reduce the pilot pressure output from the operating device 26 to be lower than the pilot pressure output from the hydraulic control valve 31, for example, even when the operating device 26 is being operated. I can do it. Therefore, by controlling the hydraulic control valve 31 and the hydraulic control valve 33, the controller 30 applies a desired pilot pressure to the pilot port of the control valve in the control valve 17, for example, regardless of the operation details of the operating device 26. It can be made to work reliably. Therefore, the controller 30 can more appropriately realize the automatic operation function and remote control function of the crawler crane 100 by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31, for example.

<User interface>
As shown in FIG. 3, the user interface system of the crawler crane 100 according to this embodiment includes an operating device 26, an output device 50, and an input device 52.

The output device 50 outputs various information to the user (operator) of the crawler crane 100 inside the cabin 10.

For example, the output device 50 may be installed in a location within the cabin 10 that is easily visible to the seated operator, such as an indoor lighting device or a display device 50A (see FIGS. 4 and 8) that outputs various information in a visual manner. including. The lighting equipment is, for example, a warning light (indicator lamp) or the like. The display device 50A is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. For example, the lighting equipment and display device 50A are provided inside the cabin 10 and output various information to an operator inside the cabin 10 in a visual manner. Furthermore, the lighting equipment and the display device 50A may be provided, for example, on the side surface of the upper revolving structure 3, and may output various information to workers and the like around the crawler crane 100 in a visual manner.

Further, the output device 50 may include a sound output device 50B (see FIG. 8) that outputs various information in an auditory manner. The sound output device 50B includes, for example, a buzzer, a speaker, and the like. For example, the sound output device 50B is provided inside or outside the cabin 10, and outputs various information in an auditory manner to the operator inside the cabin 10 and the people (workers, etc.) around the crawler crane 100. You may do so.

Further, the output device 50 may include a device that outputs various information using a tactile method such as vibration of the cockpit.

The input device 52 accepts various inputs from the user of the crawler crane 100, and signals corresponding to the accepted inputs are taken into the controller 30. The input device 52 is provided inside the cabin 10 , for example, and receives input from an operator inside the cabin 10 . Further, the input device 52 may be provided, for example, on the side surface of the revolving upper structure 3, and may receive input from a worker or the like in the vicinity of the crawler crane 100.

For example, the input device 52 includes a mechanical input device that receives mechanical input that involves contact from a user. The mechanical input device includes a touch panel mounted on the display device 50A, a touch pad installed around the display device 50A, a button switch, a lever, a toggle, a knob switch provided on the operating device 26 (lever device), etc. It's fine.

Furthermore, the input device 52 may include a voice input device that accepts voice input from a user. The audio input device includes, for example, a microphone.

Input device 52 may also include a gesture input device that accepts gesture input from the user. The gesture input device includes, for example, an imaging device that captures an image of a gesture performed by a user.

Furthermore, the input device 52 may include a biometric input device that receives biometric input from the user. The biometric input includes, for example, input of biometric information such as a user's fingerprint or iris.

<Communication system>
As shown in FIG. 3, the communication system of the crawler crane 100 according to this embodiment includes a communication device 60.

The communication device 60 is connected to the communication line NW and communicates with a device provided separately from the crawler crane 100 (for example, the support device 200). Devices provided separately from the crawler crane 100 may include devices external to the crawler crane 100 as well as portable terminal devices (for example, the support device 200) brought into the cabin 10 by the user of the crawler crane 100. . The communication device 60 may include, for example, a mobile communication module that complies with standards such as 4G ( ^4th Generation) and 5G ( ^5th Generation). Further, the communication device 60 may include, for example, a satellite communication module. Further, the communication device 60 may include, for example, a WiFi communication module, a Bluetooth (registered trademark) communication module, or the like.

<Control system>
As shown in FIG. 3, the control system of the crawler crane 100 according to this embodiment includes a controller 30. Further, the control system of the crawler crane 100 according to the present embodiment includes an operating pressure sensor 29, an imaging device 40, a distance sensor 45, and sensors S1 to S4.

The controller 30 performs various controls regarding the crawler crane 100.

The functions of the controller 30 may be realized by arbitrary hardware or a combination of arbitrary hardware and software. For example, as shown in FIG. 3, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D, which are connected via a bus B1.

The auxiliary storage device 30A is a non-volatile storage means, and stores installed programs as well as necessary files, data, and the like. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory.

For example, when there is an instruction to start a program, the memory device 30B loads the program in the auxiliary storage device 30A so that the CPU 30C can read the program. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

For example, the CPU 30C executes a program loaded into the memory device 30B, and implements various functions of the controller 30 according to instructions of the program.

The interface device 30D functions as a communication interface for connecting to a communication line inside the crawler crane 100, for example. The interface device 30D may include a plurality of different types of communication interfaces depending on the type of communication line to be connected.

Further, the interface device 30D functions as an external interface for reading data from and writing data to the recording medium. The recording medium is, for example, a dedicated tool that is connected to a connector installed inside the cabin 10 with a detachable cable. Further, the recording medium may be a general-purpose recording medium such as an SD memory card or a USB (Universal Serial Bus) memory. Thereby, programs for realizing various functions of the controller 30 can be provided by, for example, a portable recording medium and installed in the auxiliary storage device 30A of the controller 30. Further, the program may be downloaded from another computer outside the crawler crane 100 through the communication device 60 and installed in the auxiliary storage device 30A.

Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot type operating device 26, that is, the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. To detect. A detection signal of pilot pressure corresponding to the operating state of each driven element (hydraulic actuator HA) in the operating device 26 by the operating pressure sensor 29 is taken into the controller 30.

The imaging device 40 is attached to the upper part of the upper revolving structure 3 and acquires images of the vicinity of the crawler crane 100. The imaging device 40 also generates three-dimensional data (hereinafter simply referred to as "object's 3D data) may also be acquired (generated). The three-dimensional data of the object around the crawler crane 100 is, for example, coordinate information data of a point group representing the surface of the object, distance image data, and the like.

For example, as shown in FIG. 1, the imaging device 40 includes a camera that images the front of the upper rotating structure 3. Further, the imaging device 40 may include at least one of a camera that images the rear of the revolving upper structure 3, a camera that images the left side of the revolving upper structure 3, and a camera that images the right side of the revolving upper structure 3. As a result, the operator can visually check surrounding images such as images captured by the imaging device 40 and processed images generated based on the captured images through the display device 50A or the remote control display device, and see the front and left side of the upper rotating structure 3. It is possible to check the situation in at least one of the front, right, and rear directions. Further, for example, the operator can visually check peripheral images such as images captured by the imaging device 40 and processed images generated based on the captured images through the remote control display device, while checking the state of the suspended load. Crawler crane 100 can be remotely controlled.

The imaging device 40 is, for example, a monocular camera. The imaging device 40 also captures data regarding distance (depth) in addition to two-dimensional images, such as a stereo camera, a TOF (Time Of Flight) camera, etc. (hereinafter collectively referred to as a "3D camera"). It may be possible to obtain it.

Output data of the imaging device 40 (for example, image data, three-dimensional data of objects around the crawler crane 100, etc.) is taken into the controller 30 through a one-to-one communication line or an in-vehicle network. Thereby, for example, the controller 30 can monitor objects around the crawler crane 100 based on the output data of the imaging device 40. Further, for example, the controller 30 can grasp the surrounding environment of the crawler crane 100 based on the output data of the imaging device 40. Furthermore, for example, the controller 30 can determine the attitude state of the body of the crawler crane 100 (upper rotating body 3) based on the output data of the imaging device 40, with reference to objects around the crawler crane 100.

The distance sensor 45 is attached to the upper part of the upper revolving structure 3 and acquires data regarding the distance to the surroundings of the crawler crane 100 based on the crawler crane 100 or itself (distance sensor 45). Specifically, the distance sensor 45 outputs a predetermined detection wave (also referred to as an "irradiation wave") around the crawler crane 100 and receives the reflected wave, thereby detecting a predetermined detection wave such as a TOF (Time of Flight) method. In this method, data regarding the distance between the crawler crane 100 and surrounding objects is obtained. The predetermined detection wave is a laser (light), a millimeter wave, or the like. Further, the distance sensor may acquire (generate) three-dimensional data (for example, coordinate information data of a point group) of objects around the crawler crane 100 within the sensing range based on the acquired data. The distance sensor 45 can acquire coordinate information of an object based on the output direction of the detection wave and the distance to the object that reflected the detection wave. The coordinates of each point in the point group data are expressed, for example, on a moving coordinate system with a predetermined location of the crawler crane 100 as a reference. Furthermore, if information regarding the absolute position of the crawler crane 100 on a fixed coordinate system fixed to the ground can be obtained using a positioning device or the like, the coordinates of each point in the point cloud data are expressed on the fixed coordinate system. You can.

For example, as shown in FIG. 2, the distance sensor 45 is attached to the front of the upper revolving body 3 (cabin 10) of the crawler crane 100, and the distance sensor 45 is a distance at which data regarding the distance to an object in front of the crawler crane 100 can be acquired. Contains sensors. Further, the distance sensor 45 may include a distance sensor that is attached to the left side of the revolving upper structure 3 and can acquire data regarding the distance to an object on the left side of the crawler crane 100. Further, the distance sensor 45 may include a distance sensor that is attached to the right side of the revolving upper structure 3 and can acquire data regarding the distance to an object on the right side of the crawler crane 100. Further, the distance sensor 45 may include a distance sensor that is attached to the rear part of the revolving upper structure 3 and can acquire data regarding the distance to an object behind the crawler crane 100.

The distance sensor 45 is, for example, a LIDAR (Light Detection and Ranging) that outputs a laser (light) as a detection wave. Further, for example, the distance sensor may be, for example, a millimeter wave radar that outputs millimeter waves as detection waves.

The sensor S1 acquires detection data regarding the attitude angle of the boom 4 with respect to a predetermined reference (for example, a horizontal plane, a state of either end of the movable angle range of the boom 4, etc.). The sensor S1 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), and the like. The detection data of the sensor S1 is taken into the controller 30.

The sensor S2 acquires detection data regarding the tilted state of the body of the crawler crane 100 including the lower traveling body 1 and the upper rotating body 3. The sensor S2 is mounted on the revolving upper structure 3, for example, and acquires detection data regarding the inclination angle of the revolving upper structure 3 in the longitudinal direction and the left-right direction (hereinafter referred to as "front-rear inclination angle" and "lateral inclination angle"). The sensor S2 may include, for example, an acceleration sensor (tilt sensor), an angular velocity sensor, a six-axis sensor, an IMU, and the like. The detection data of the sensor S2 is taken into the controller 30.

The sensor S3 acquires detection data regarding the rotating state of the upper rotating body 3. The sensor S3 acquires, for example, detection data regarding the turning angle of the upper rotating structure 3 with respect to a predetermined reference (for example, a state in which the forward direction of the lower traveling structure 1 and the forward direction of the upper rotating structure 3 match). The sensor S3 includes, for example, a potentiometer, a rotary encoder, a resolver, and the like. The detection data of the sensor S3 is taken into the controller 30.

In addition, when it is possible to obtain detection data regarding the attitude state of the revolving upper structure 3 including not only the inclination angle of the upper revolving structure 3 but also the turning angle by the components of the sensor S2 (for example, a six-axis sensor, an IMU, etc.), Sensor S3 may be omitted.

Sensor S4 acquires detection data regarding the relative position of the suspended load with respect to boom 4. The sensor S4 is, for example, an image sensor (camera) or a distance sensor. The detection data of the sensor S4 is taken into the controller 30.

Further, instead of or in addition to the sensor S4, a sensor capable of detecting the winding state of the main winding rope 7 may be provided.

Furthermore, the crawler crane 100 may be further equipped with a positioning device capable of positioning the absolute position of the crawler crane 100. The positioning device is, for example, a GNSS (Global Navigation Satellite System) sensor. Thereby, the accuracy of estimating the posture state of the crawler crane 100 can be improved.

The controller 30 can grasp the position of the load suspended from the hook HK based on data from the sensors S1 to S4 and the like.

Note that depending on the specifications regarding the functions of the crawler crane 100, the sensors S1 to S4, etc. may be omitted.

[First example of function related to estimating driving area]
Next, referring to FIGS. 4 to 7 in addition to FIGS. 1 to 3, functions related to estimating an area (traveling area) in which crawler crane 100 can run or is permitted to travel on the ground around it will be described. The first example will be explained.

<Functional configuration>
FIG. 4 is a functional block diagram showing a first example of a functional configuration related to estimating the travel area of the crawler crane 100.

As shown in FIG. 4, the controller 30 includes a data importing section 301, a preprocessing section 302, a feature extraction section 303, a driving area estimating section 304, and a display processing section 305 as functional sections.

The data capture unit 301 captures output data from the distance sensor 45. The output data of the distance sensor 45 is, for example, point cloud data representing the surfaces of objects around the crawler crane 100. The point group data includes, for example, coordinate data of each point and data regarding the intensity of reflected waves (hereinafter referred to as "reflection intensity") corresponding to each point. The following description will proceed on the premise that the output data of the distance sensor 45 is point cloud data.

The preprocessing unit 302 performs known preprocessing on the output data (point cloud data) of the distance sensor 45. The preprocessing includes, for example, processing related to noise removal and processing related to downsampling.

The feature extraction unit 303 extracts feature quantities related to features specific to the travel area around the crawler crane 100 based on the output data (point cloud data) of the distance sensor 45 that has been preprocessed by the preprocessing unit 302 .

For example, in a travel area of a work site on the ground where the crawler crane 100 performs work, rectangular iron plates (padding iron plates) are laid in a row on the ground surface where earth and sand are exposed. For example, in a relatively upper floor travel area of a building with two or more floors under construction where the crawler crane 100 works, a rectangular iron plate (paved) is placed on the floor surface where concrete is exposed. Iron plates) are laid in rows. Therefore, the feature amount extraction unit 303 extracts feature amounts related to features specific to the iron plate. Specifically, the feature amount extraction unit 303 extracts (obtains) the vertical height value and reflection intensity value of each point of the point cloud data as the feature amount. This is because the ground surface on which the iron plates are laid is approximately horizontal. Further, the height of the steel plate in the vertical direction is lower than in the non-travelable area where there are obstacles such as people and triangular cones. In addition, the reflection intensity of the iron plate may differ to such an extent that it can be distinguished from, for example, an area in which the crawler crane 100 cannot run or is prohibited to run (non-travel area) where earth and sand on the ground surface are exposed. be. Specifically, when the distance sensor 45 is a LIDAR, the reflection intensity is affected by the color of the object; for example, the reflection intensity of a white object is high, and the reflection intensity of a black object is low. Therefore, the color of the iron plate and the exposed soil on the ground surface are usually completely different, and the characteristics of the iron plate laid in the driving area are reflected in the reflection intensity. Another reason is that the reflection intensity of the iron plate may be different enough to distinguish it from a non-driving area where there are obstacles such as people or triangular cones.

The traveling area estimating unit 304 estimates the area around the crawler crane 100 based on the feature values (height value and reflection intensity value) for each position around the crawler crane 100, that is, for each point data included in the point cloud data. Estimate the driving area and non-driving area.

The display processing unit 305 causes the display device 50A to display an image representing the driving area and non-driving area within the range shown in the peripheral image, in a form superimposed on the peripheral image, based on the estimation result of the driving area estimation unit 304. (For example, see FIG. 8). As described above, the peripheral image may be the captured image itself of the imaging device 40, or may be a processed image (for example, an overhead image) based on the captured image. Thereby, the operator of the cabin 10 can operate the crawler crane 100 while checking the running area and non-running area around the crawler crane 100 through the display device 50A. In addition, when remote control or remote monitoring of the crawler crane 100 is performed, the display processing unit 305 uses the communication device 60 to display a display including a surrounding image and an image representing the travel area within the range shown in the surrounding image. image data may be transmitted to the support device 200. Thereby, the support device 200 can display the same image as the display device 50A on its own display device. Therefore, the operator using the support device 200 can remotely control the travel of the crawler crane 100 while checking the travel area and non-travel area around the crawler crane 100 through the support device 200 (display device). . Additionally, the supervisor using the support device 200 can monitor the operation related to the travel of the crawler crane 100 while checking the travel area and non-travel area around the crawler crane 100 through the support device 200 (display device). can.

<Processing>
FIG. 5 is a flowchart schematically showing a first example of main routine processing regarding estimation of the driving area around the crawler crane 100. FIG. 6 is a flowchart illustrating an example of a subroutine process related to estimating the driving area around the crawler crane 100. Specifically, FIG. 6 is a flowchart of a subroutine corresponding to the process of step S108 (driving area estimation process) in FIG. FIG. 7 is a diagram illustrating an example of an image showing a running area around the crawler crane 100, which is displayed on the display device 50A.

The flowchart in FIG. 5 is repeatedly executed at predetermined intervals, for example, from when the crawler crane 100 starts operating (key switch on) to when the crawler crane 100 stops operating (key switch on). Further, the estimation function related to the driving area may be enabled or disabled by switching it between valid and invalid by a predetermined input from the input device 52. In this case, the flowchart of FIG. 5 shows that when the estimating function regarding the travel area is enabled, from the start of operation (key switch on) to the stop of operation (key switch on) of the crawler crane 100, each predetermined control period is , may be executed repeatedly. The same applies to the flowchart of FIG. 9, which will be described later.

As shown in FIG. 5, in step S102, the data acquisition unit 301 acquires point cloud data of the distance sensor 45.

When the process of step S102 is completed, the controller 30 proceeds to step S104.

In step S104, the preprocessing unit 302 performs predetermined preprocessing on the point cloud data acquired in step S102.

When the process of step S104 is completed, the controller 30 proceeds to step S106.

In step S106, the feature extracting unit 303 extracts height values and reflection intensity values as feature quantities for each point data included in the point cloud data, based on the point cloud data preprocessed in step S104. do

When the process of step S106 is completed, the controller 30 proceeds to step S108.

In step S108, the driving area estimating unit 304 moves to the subroutine of FIG. 8 and performs a process of estimating the driving area around the crawler crane 100 (driving area estimation process) based on the feature amount acquired in step S106. conduct.

As shown in FIG. 8, in step S202, the travel area estimating unit 304 sets a reference position on the ground around the crawler crane 100 for comparing the feature amount with the position to be determined. . The reference position is expressed on a moving coordinate system with a predetermined location of the crawler crane 100 as a reference. Furthermore, if information regarding the absolute position of the crawler crane 100 on a fixed coordinate system fixed to the ground can be acquired by a positioning device or the like, the reference position may be expressed on the fixed coordinate system. The reference position is set to a position corresponding to the ground that has been previously recognized as the driving area. For example, the initial reference position is set from a point on the ground contact surface of the undercarriage 1 of the crawler crane 100. Further, after the process of step S212, if the determination result of the previous point data is a driving area, the reference position is set to the position of the point data to be determined the previous time. On the other hand, after the process of step S212, if the determination result of the previous point data is a non-driving area, the reference position is maintained as the previous reference position.

When the process of step S202 is completed, the controller 30 proceeds to step S204.

In step S204, the travel area estimating unit 304 sets a position around the crawler crane 100 to be determined as a travel area or a non-travel area. Specifically, the driving area estimating unit 304 sets (selects) point data to be determined as a driving area or a non-driving area from among the point data included in the point cloud data. The position (point data) to be determined is selected by a known method from point data adjacent to the reference position in the point group data. However, if the judgment result of the previous point data is a non-driving area, the position (point data) to be judged is a known point data that is adjacent to the reference position and different from the position of the previous judgment target. It is set in the following way.

When the process of step S204 is completed, the controller 30 proceeds to step S206.

In step S206, the driving area estimating unit 304 determines whether the gradient of the feature amount between the determination target position (point data) and the reference position (point data) is less than or equal to a certain value. Specifically, the driving area estimating unit 304 determines that the slope of the height value between the determination target position (point data) and the reference position (point data) is equal to or less than a certain threshold Th1, and the reflection intensity is It is determined whether the gradient of the value of is less than or equal to a certain threshold. The threshold value Th1 and the threshold value Th2 are determined in advance through experiments, computer simulations, etc., as values that take into account an allowance for the maximum value of the gradient expected when both the determination target position and the reference position are in the driving area. Ru. If the slope of the feature amount between the determination target position (point data) and the reference position (point data) is below a certain level, the driving area estimating unit 304 proceeds to step S208; otherwise, the process proceeds to step S210. move on.

In step S208, the driving area estimating unit 304 determines that the position of the point data to be determined is in the driving area. As mentioned above, the running area is covered with iron plates (paved iron plates), and it is thought that the change in reflection intensity (gradient) is relatively small, and the change in height (gradient) is also relatively small. It is.

When the process of step S208 is completed, the controller 30 proceeds to step S212.

In step S210, the driving area estimation unit 304 determines that the position of the point data to be determined is in a non-driving area. In non-driving areas where soil and sand are exposed, the value of reflection intensity is relatively large compared to the iron plate of the adjacent driving area, and the change (gradient) of the reflection intensity between the positions of the adjacent driving area is This is because it is considered to be relatively large. In addition, the non-driving area where there are obstacles such as people and triangular cones has a vertical height higher than the steel plate of the adjacent driving area, and there is a change in height (gradient) between the adjacent driving area. This is because it is considered to be relatively large.

When the process of step S210 is completed, the controller 30 proceeds to step S212.

In step S212, the driving area estimating unit 304 determines whether the termination condition is satisfied. The termination condition is that point data to be determined next time cannot be set. For example, if all of the point cloud data is a driving area, the driving area estimating unit 304 appropriately selects adjacent point data as determination target data, and determines whether all point data is a driving area or a non-driving area. Judgments can be made. Then, the termination condition is satisfied when the determination for all point data is completed. On the other hand, if the point cloud data includes a non-driving area, by successively finding point data adjacent to the driving area that corresponds to the boundary between the driving area and the non-travelling area, point data of the non-travelling area corresponding to the boundary line is detected. A group is formed. As a result, the point data of other non-driving areas is not set as a determination target as a point adjacent to the point data of the driving area, and the termination condition is satisfied. If the end condition is satisfied, the driving area estimating unit 304 proceeds to step S214.

In step S214, the driving area estimating unit 304 determines whether there is any undetermined position (point data). If there is an undetermined position (point data), the driving area estimating unit 304 proceeds to step S216, and if there is not, it ends the process of the current flowchart (the process of step S108 in FIG. 5).

In step S216, the driving area estimation unit 304 determines the position of the undetermined point data to be a non-driving area.

When the process of step S216 is completed, the controller 30 ends the process of the current flowchart (the process of step S108 in FIG. 5).

Returning to FIG. 5, upon completion of the process in step S108, the controller 30 proceeds to step S110.

In step S110, the display processing unit 305 causes the display device 50A to display an image representing the driving area around the crawler crane 100, superimposed on the surrounding image, based on the result of the driving area estimation process in step S108 ( (See Figure 7).

For example, as shown in FIG. 7, a captured image 700 including the ground in front of the crawler crane 100, which is captured by the imaging device 40, is displayed on the display device 50A. The captured image 700 includes an image area corresponding to the traveling area where the iron plate IP is laid in front of the crawler crane 100, including the grounding position of the lower traveling body 1 of the crawler crane 100, and the image area on the far side thereof (the upper side in the figure). ) and an image area corresponding to a non-driving area where the ground surface is exposed.

A line-shaped image 701 representing the driving area is displayed on the display device 50A, superimposed on the captured image 700. An image 701 represents a boundary line between an image area corresponding to a running area and an image area corresponding to a non-running area in the captured image 700. Thereby, the user (operator) can easily grasp the outer edge of the driving area, that is, the boundary line between the driving area and the non-driving area. Therefore, it is possible to prevent the crawler crane 100 from approaching the non-travel area or actually entering the non-travel area.

Furthermore, an image 702 corresponding to the non-traveling area is displayed on the display device 50A in a manner superimposed on the captured image 700. In the image 702, an image area corresponding to the non-traveling area in the captured image 700 is grayed out. This allows the user (operator) to easily understand that the image area corresponding to image 702 is a non-traveling area. Therefore, it is possible to prevent the crawler crane 100 from approaching the non-travel area or actually entering the non-travel area.

The display device 50A also displays, superimposed on the captured image 700, the traveling trajectory of the undercarriage 1 (crawlers 1CL, 1CR) when the undercarriage 1 is assumed to travel in the current state of the crawler crane 100. A representative image 703 is displayed. Thereby, it is possible to easily grasp the relationship between the traveling trajectory, the traveling area, and the non-travelling area when the undercarriage body 1 is traveling in the current state of the crawler crane 100. Therefore, it is possible to more appropriately suppress a situation in which the crawler crane 100 approaches the non-travel area or actually enters the non-travel area.

Note that a screen similar to that shown in FIG. 7 may be similarly displayed on the support device 200 (display device) used by an operator who performs remote control.

Returning to FIG. 5, when the process of step S110 is completed, the controller 30 ends the process of the current flowchart.

In this way, the controller 30 can estimate the running area and non-running area on the ground around the crawler crane 100 based on the output of the distance sensor 45. Moreover, the controller 30 can visually grasp the estimated driving area and non-driving area through the display device 50A. Thereby, it is possible to suppress a situation in which the crawler crane 100 approaches or enters a non-traveling area.

[Second example of function related to estimating driving area]
Referring to FIGS. 8 and 9 in addition to FIGS. 1 to 3, the second function related to estimating the area on the ground around the crawler crane 100 in which the crawler crane 100 can run or is allowed to run (travel area) Let's discuss an example.

Hereinafter, parts that are different from the above-mentioned first example will be mainly explained, and explanations of contents that are the same as or corresponding to the above-mentioned first example may be simplified or omitted.

<Functional configuration>
FIG. 8 is a functional block diagram showing a second example of a functional configuration related to estimating the travel area of the crawler crane 100.

As shown in FIG. 8, the controller 30 includes, as functional units, a data import unit 301, a preprocessing unit 302, a feature extraction unit 303, a driving area estimation unit 304, a display processing unit 305, and an operation support control unit. 306.

The operation support control unit 306 controls the support for the travel operation of the crawler crane 100 by the operator based on the estimation result of the travel area estimation unit 304, that is, the estimation result of the travel area and non-travel area on the ground around the crawler crane 100. I do.

For example, when the distance between the crawler crane 100 and the non-driving area becomes equal to or less than the threshold value Th3, the operation support control unit 306 notifies the user (operator) of this through the display device 50A and the sound output device 50B. Thereby, the operator of the cabin 10 can recognize that the crawler crane 100 is getting too close to the non-traveling area or has entered the non-travelling area. Furthermore, when the distance between the crawler crane 100 and the non-driving area becomes equal to or less than the threshold value Th3, the operation support control unit 306 may transmit a signal indicating this to the support device 200 via the communication device 60. As a result, the operator using the support device 200 recognizes through the support device 200 (display device and sound output device) that the crawler crane is getting too close to the non-travel area or has entered the non-travel area. be able to.

Further, the operation support control unit 306 may change the mode of notification as the distance between the crawler crane 100 and the non-driving area becomes shorter. For example, when the distance between the crawler crane 100 and the non-driving area is less than or equal to the threshold Th4 (<Th3) than when the distance is less than or equal to the threshold Th4 (<Th3), the operation support control unit 306 The sound pressure and pitch of the notification sound from the sound output device 50B are increased.

Further, when the distance between the crawler crane 100 and the non-traveling area becomes equal to or less than the threshold value Th5, the operation support control unit 306 controls the hydraulic control valve 33, etc., to operate the lower traveling body 1 (crawler 1C) of the crawler crane 100. may be restricted. The function of restricting the travel operation of the undercarriage 1 includes, for example, a deceleration function that reduces the traveling speed of the undercarriage 1 in response to an operator's operation input. Further, the function of restricting the travel operation of the undercarriage body 1 may include a stop function that stops the undercarriage body 1 and maintains the stopped state regardless of an operator's operation input. Thereby, when the crawler crane 100 gets too close to the non-travel area, the traveling operation of the crawler crane 100 can be slowed down or stopped, and as a result, the crawler crane 100 can be prevented from entering the non-travel area. It is possible to prevent the situation from occurring.

Further, the operation support control unit 306 may change the manner in which the operation of the undercarriage 1 is restricted as the distance between the crawler crane 100 and the non-traveling area becomes shorter. For example, when the distance between the crawler crane 100 and the non-driving area is less than or equal to the threshold Th5 and greater than the threshold Th6 (<Th5), the operation support control unit 306 controls the operation when the distance is less than or equal to the threshold Th6. Increase the degree of restriction. Increasing the degree of restriction includes increasing the degree of deceleration of the deceleration function and shifting from the deceleration function to the stop function.

<Processing>
FIG. 9 is a flowchart schematically showing a second example of the main routine processing regarding estimation of the driving area around the crawler crane 100.

Note that the subroutine processing related to estimating the driving area around the crawler crane 100 is the same as the above-mentioned first example, so FIG. 6 is used and illustration thereof is omitted.

As shown in FIG. 9, the processing in steps S302 to S310 is the same as the processing in steps S102 to S110 in FIG. 5, so a description thereof will be omitted.

When the process of step S310 is completed, the controller 30 proceeds to step S312.

In step S312, the operation support control unit 306 determines whether the distance between the crawler crane 100 and the non-traveling area is equal to or less than a certain value. For example, in the case of the notification function, the operation support control unit 306 determines whether the distance between the crawler crane 100 and the non-driving area is less than or equal to the threshold Th3. Further, in the case of the function of restricting the travel operation of the undercarriage 1, the operation support control unit 306 may determine whether the distance between the crawler crane 100 and the non-traveling area is equal to or less than a threshold value Th5. If the distance between the crawler crane 100 and the non-traveling area is less than a certain value, the operation support control unit 306 proceeds to step S314; otherwise, it ends the current flowchart.

In step S314, the operation support control unit 306 performs control processing related to support for the travel operation of the crawler crane 100 (travel operation support control process). Specifically, the operation support control unit 306 may operate a notification function or a function to limit the running operation of the undercarriage body 1.

When the process of step S314 is completed, the controller 30 ends the process of the current flowchart.

In this manner, in this example, the controller 30 can provide support regarding the traveling operation of the undercarriage body 1 based on the estimation results of the traveling area and the non-travelling area.

[Other embodiments]
Next, another embodiment will be described with reference to FIG.

FIG. 10 is a diagram showing another example of the working machine. Specifically, FIG. 10 is a side view showing an example of the shovel 300.

In the embodiments described above, individual examples may be combined with each other, and modifications and changes may be made as appropriate.

For example, in the above-described embodiment, a part or all of the function related to estimating the running area around the crawler crane 100, that is, the functional unit related to estimating the running area around the crawler crane 100 of the controller 30, is transferred to the support device 200. may be done. For example, the functions of the preprocessing unit 302, the feature amount extraction unit 303, and the driving area estimation unit 304 may be transferred to the support device 200. Furthermore, in addition to the functions of the preprocessing unit 302 , the feature extraction unit 303 , and the driving area estimation unit 304 , the function of the display processing unit 305 may be transferred to the support device 200 . In this case, the controller 30 transmits the point cloud data taken in from the distance sensor 45 to the support device 200 through the communication device 60. In this case, instead of the distance sensor 45, output data of a distance sensor external to the crawler crane 100 (for example, a distance sensor fixed at the work site or a distance sensor mounted on a drone flying over the work site) may be used to estimate the travel area within a predetermined range of the work site. Thereby, for example, the support device 200 can generate a travel area map within the work site and distribute it to the crawler crane 100 in advance.

Moreover, in the above-mentioned embodiment and its modification, instead of or in addition to the crawler crane 100, the travel area of another work machine different from the crawler crane 100 may be estimated. For example, as shown in FIG. 10, the driving area around the excavator 300 at the work site may be estimated. This is because, at a work site in an urban area or a work site where the shovel 300 is used for crane work, iron plates may also be laid in the travel area of the shovel 300.

Furthermore, in the above-described embodiments and modifications thereof, a traveling area formed of a member other than an iron plate or a substance other than iron may be estimated. For example, at a work site in a mountainous area, the only driving area may be the ground that has been trodden with road rollers or the like. In this case, the difference in flatness between the running area and the non-running area may be used as a feature.

In addition, in the above-described embodiments and modifications thereof, instead of or in addition to the output data of the distance sensor, information is provided on the unique characteristics of the travel area based on image data (for example, image data of the imaging device 40). The driving area may be estimated by using the image processing technique described above.

Furthermore, in the above-described embodiments and modifications thereof, the driving area may be estimated using a trained model based on a distance sensor or image data. In this case, the trained model is machine trained using a teacher dataset that is a combination of point cloud data or image data and point data that corresponds to the driving area or data that represents the image area in the point cloud data or image data. This is the model that was used.

[Effect]
Next, the operation of the working machine and information processing device according to this embodiment will be explained.

In this embodiment, the working machine includes a sensor that acquires data regarding objects around the working machine, and based on the output of the sensor, the working machine is able to run or is allowed to run on the ground around the working machine. and an estimating section that estimates a first region (driving region). The working machine is, for example, a crawler crane 100 or a shovel 300. The sensor is, for example, the distance sensor 45 or the imaging device 40. The estimating unit is, for example, the driving area estimating unit 304.

Further, in the present embodiment, the information processing device acquires the output of a sensor that acquires data regarding objects within a predetermined range, and determines whether or not the work machine can travel within the predetermined range based on the output of the sensor. Estimate the first area (driving area) in which the vehicle is permitted. The information processing device is, for example, the controller 30 or the support device 200. The working machine is, for example, a crawler crane 100 or a shovel 300. The sensor is, for example, the distance sensor 45 or the imaging device 40.

Thereby, it is possible to estimate the travel area of the work site where the work machine performs work. Therefore, for example, it is possible to suppress a situation in which a working machine with an automatic driving function deviates from the driving range. Further, for example, information regarding the driving area can be provided to a user who remotely operates a working machine in a situation where it is relatively difficult to grasp the situation around the working machine. Therefore, it is possible to suppress a situation in which the remotely controlled work machine deviates from the travel area.

Further, in the present embodiment, the first region (driving region) may have unique characteristics that can be extracted from the output of the sensor. The estimation unit may estimate the first region around the work machine based on the output of the sensor and taking into consideration the unique characteristics.

Thereby, by considering the unique characteristics of the driving area, it is possible to appropriately estimate the driving area from the output of the sensor.

Further, in the present embodiment, the working machine may include an extraction unit that acquires a feature quantity related to a unique feature for each position corresponding to the ground around the working machine, based on the output of the sensor. The extraction unit is, for example, the feature amount extraction unit 303. The estimation unit may then estimate the first area around the work machine based on the feature amount.

Thereby, the driving area can be estimated more appropriately using the feature amount.

In addition, in this embodiment, the sensor outputs a detection wave around the work machine and receives the reflected wave, thereby providing data regarding the distance to objects around the work machine and the intensity of the reflected wave (reflection intensity). may be obtained. The detection wave is, for example, a laser (light) or a millimeter wave. Then, the extraction unit may extract the value of the reflection intensity as a feature amount for each position corresponding to the ground around the working machine based on the output of the sensor.

Thereby, it is possible to estimate a driving area (for example, a driving area where iron plates are laid) having unique characteristics regarding the reflection intensity using the reflection intensity as a feature quantity.

Further, in the present embodiment, the estimating unit may estimate the first area around the working machine based on a change in the intensity value due to movement of a position corresponding to the ground around the working machine.

This makes it possible to more appropriately estimate the driving area by using the fact that within a driving area that has unique characteristics regarding reflection intensity, changes in reflection intensity due to positional movement are relatively small.

Further, in this embodiment, the extraction unit may extract intensity values and height values as feature quantities for each position corresponding to the ground around the work machine based on the output of the sensor.

Thereby, by using the vertical height value as a feature quantity, it is possible to more appropriately estimate a flat driving area. In addition, by using the vertical height value as a feature, places where it is impossible to drive due to the presence of obstacles in places where it is possible to drive are estimated as driving areas. It is possible to prevent such situations.

Further, in the present embodiment, the estimating unit may estimate the first area around the working machine based on a change in the height value due to movement of a position corresponding to the ground around the working machine.

Thereby, the travel area can be estimated more appropriately by using the fact that within the travel area, changes in height due to positional movement are relatively small.

Further, in the present embodiment, the first area has a higher reflection intensity than the second area (non-traveling area) on the ground around the working machine where the working machine cannot run or is prohibited from running. You can.

Thereby, the driving area can be estimated using the reflection intensity as a feature quantity.

Further, in this embodiment, a metal plate is laid in the first area.

Thereby, the driving area can be estimated using the reflection intensity as a feature quantity.

Further, in this embodiment, the working machine may include an imaging device that images the surroundings of the working machine, and a display device. The imaging device is, for example, the imaging device 40. The display device is, for example, a display device 50A. Specifically, the display device displays a first image representing the surroundings of the working machine based on the captured image of the imaging device, and displays a second image representing the first area superimposed on the first image. The image may be displayed. The first image is, for example, a captured image 700. The second image is, for example, image 701.

This allows the user (operator) to recognize the running area around the work machine. Therefore, it is possible to appropriately prevent a situation in which the work machine deviates from the travel area.

Furthermore, in the present embodiment, the display device displays a third image superimposed on the first image, which represents the travel trajectory of the traveling body when the work machine is assumed to travel in the current state. You can. The third image is, for example, image 703.

This allows the user (operator) of the cabin 10 to recognize the relationship between the running track of the work machine and the outer edge of the running area, that is, the boundary with the non-running area. Therefore, it is possible to more appropriately prevent a situation in which the work machine deviates from the travel area.

Further, in this embodiment, a notification device may be provided to notify the operator when there is a possibility of deviation from the first area.

Thereby, it is possible to more appropriately suppress a situation in which the work machine deviates from the travel area.

Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and changes can be made within the scope of the gist described in the claims.

1 Undercarriage 1C, 1CL, 1CR Crawler 1ML, 1MR Travel hydraulic motor 2 Swing mechanism 2M Swing hydraulic motor 3 Upper swing structure 4 Boom 5 Mast 5a Pendant rope 5b Boom hoisting rope 5c Boom hoisting winch 5M Hoisting hydraulic motor 6 Backstop 7 Main hoisting rope 7a Main hoisting winch 7M Main hoisting hydraulic motor 9 Counterweight 10 Cabin 11 Engine 13 Regulator 14 Main pump 15 Pilot pump 17 Control valve 26 Operating device 29 Operating pressure sensor 30 Controller 31 Hydraulic control valve 32 Shuttle valve 33 Hydraulic control valve 40 Imaging device 45 Distance sensor 50 Output device 50A Display device 50B Sound output device 52 Input device 60 Communication device 100 Crawler crane 200 Support device 300 Excavator 301 Data acquisition unit 302 Preprocessing unit 303 Feature extraction unit 304 Traveling area estimation unit 305 Display Processing unit 306 Operation support control unit 700 Captured image 701 Image 702 Image 703 Image HA Hydraulic actuator HK Hook IP Iron plate NW Communication lines S1 to S4 Sensor SYS Operation support system

Claims (13)

a sensor that acquires data about objects around the work machine;
an estimation unit that estimates a first area on the ground around the working machine in which the working machine can run or is allowed to run, based on the output of the sensor;
working machine. the first region has unique features extractable from the output of the sensor;
The estimation unit estimates the first area around the working machine based on the output of the sensor and taking into account the unique characteristics.
The working machine according to claim 1. an extraction unit that acquires a feature amount related to the unique feature for each position corresponding to the ground around the working machine based on the output of the sensor,
The estimation unit estimates the first area around the working machine based on the feature amount.
A working machine according to claim 2. The sensor outputs a detection wave around the work machine and receives the reflected wave, thereby acquiring data regarding the distance to objects around the work machine and the intensity of the reflected wave,
The extraction unit extracts the intensity value as the feature amount for each position corresponding to the ground around the working machine based on the output of the sensor.
A working machine according to claim 3. The estimating unit estimates the first area around the working machine based on a change in the intensity value as a position corresponding to the ground around the working machine moves.
The working machine according to claim 4. The extraction unit extracts the intensity value and height value as the feature amount for each position corresponding to the ground around the working machine, based on the output of the sensor.
The working machine according to claim 4 or 5. The estimating unit estimates the first area around the working machine based on a change in the height value as a position corresponding to the ground around the working machine moves.
The working machine according to claim 6. The strength of the first region is higher than that of the second region on the ground around the work machine where the work machine cannot run or is prohibited from running.
A working machine according to any one of claims 4 to 7. A metal plate is laid in the first area,
A working machine according to claim 8. an imaging device that images the surroundings of the work machine;
A display device that displays a first image representing the surroundings of the working machine based on the captured image of the imaging device, and displays a second image representing the first area superimposed on the first image. and,
A working machine according to any one of claims 1 to 9. The display device superimposes and displays a third image representing a travel trajectory of the traveling body when the working machine is assumed to travel in the current state, superimposed on the first image.
The working machine according to claim 10. comprising a notification device that notifies an operator when there is a possibility of deviation from the first area;
A working machine according to any one of claims 1 to 11. Obtaining the output of a sensor that obtains data regarding objects within a predetermined range, and estimating a first region in the predetermined range in which the work machine can run or is allowed to run based on the output of the sensor. do,
Information processing device.

Images