JP2022081286A - Periphery monitoring device - Google Patents

Abstract

To provide a technology capable of more properly displaying information related to objects around a work machine toward a user.SOLUTION: A periphery monitoring device 150 comprises a display device 50 which displays information related to objects around a shovel 100 toward a user based on a data related to a periphery status of the shovel 100 obtained by a distance sensor 45. And, the display device 50 may not display the information related to the object even when there is a data showing that there is an object around the shovel 100 in the data obtained by the distance sensor 45.SELECTED DRAWING: Figure 4

Description

The present disclosure relates to a peripheral monitoring device for a work machine.

For example, there is known a technique of displaying information about an object around a work machine to a user such as an operator by using output data of a peripheral monitoring sensor such as a camera or a distance sensor (see Patent Document 1).

In Patent Document 1, the peripheral image of the excavator based on the image data of the camera is displayed on the display device in the cabin of the excavator as a work machine, and the marker is superimposed on the position of the object around the excavator in the peripheral image. Is displayed.

International Publication No. 2018/151280

However, the information about the object around the work machine recognized from the data of the peripheral monitoring sensor may include the information about the object which is relatively less important to the user. For example, if a construction management drone is flying well above the construction site above the work machine, it is unlikely that the drone will affect the work of the work machine, and information about such drones is available. , Relatively less important to the user. Therefore, if information about objects around the work machine is uniformly displayed to the user, even if the information is displayed with relatively low importance to the user, the work of the work machine can be performed by confirming the information. There is a possibility that it will be interrupted and the work efficiency will decrease.

Therefore, in view of the above problems, it is an object of the present invention to provide a technique capable of more appropriately displaying information about an object around a work machine to a user.

In order to achieve the above object, in one embodiment of the present disclosure,
It is equipped with a display unit that displays information about objects around the work machine to the user based on the data about the surrounding condition of the work machine acquired by the peripheral monitoring sensor.
Even if the display unit includes data indicating that an object exists in the vicinity of the work machine in the data acquired by the peripheral monitoring sensor, the display unit may not display information about the object.
Peripheral monitoring equipment is provided.

According to the above-described embodiment, information about an object around the work machine can be more appropriately displayed to the user.

It is a figure which shows an example of the excavator management system. It is a top view of the excavator. It is a block diagram which shows an example of the configuration of a peripheral monitoring device. It is a flowchart which shows the 1st example of the map generation processing by an environment map generation part roughly. It is a figure which shows an example of the posture state of a shovel. It is a figure which shows an example of the positional relationship between a shovel and a peripheral object. It is a figure which shows an example of the positional relationship between a shovel and a peripheral object. It is a figure which shows the 1st example of the environment map image displayed on the display device. It is a figure which shows the 2nd example of the environment map image displayed on the display device. It is a figure which shows other example of the positional relationship between a shovel and a peripheral object. It is a figure which shows the 3rd example of the environment map image displayed on the display device. It is a figure which shows the 4th example of the environment map image displayed on the display device. It is a flowchart which shows the 2nd example of the map generation processing by the environment map generation part roughly. It is a figure which shows an example of the relationship between the distance from an object and the occupancy probability. It is a figure which shows the 5th example of the environment map image displayed on the display device. It is a figure which shows the sixth example of the environment map image displayed on the display device. It is a figure which shows the 7th example of the environment map image displayed on the display device. It is a figure which shows the 8th example of the environment map image displayed on the display device. It is a figure which shows the 9th example of the environment map image displayed on the display device. It is a flowchart which shows the 3rd example of the map generation processing by the environment map generation part roughly. It is a flowchart which shows the 4th example of the map generation processing by the environment map generation part roughly. It is a figure which shows the tenth example of the environment map image displayed on the display device. It is a figure which shows the eleventh example of the environment map image displayed on the display device. It is a flowchart which shows the 5th example of the map generation processing by the environment map generation part roughly. It is a figure which shows still another example of the positional relationship between a shovel and a peripheral object. It is a figure which shows still another example of the positional relationship between a shovel and a peripheral object. It is a figure which shows the twelfth example of the environment map image displayed on the display device. It is a flowchart which shows the sixth example of the map generation processing by the environment map generation part roughly. It is a figure which shows the thirteenth example of the environment map image displayed on the display device.

Hereinafter, embodiments will be described with reference to the drawings.

[Excavator management system]
First, the excavator management system SYS according to the present embodiment will be described with reference to FIGS. 1 and 2.

FIG. 1 is a diagram showing an example of the excavator management system SYS according to the present embodiment. In FIG. 1, the excavator 100 is shown on the left side. Further, FIG. 2 is a top view of the excavator 100.

The excavator management system SYS includes an excavator 100 and a management device 200.

The excavator management system SYS monitors (manages) the operating status, operating status, etc. of the excavator 100 by using, for example, a management device 200 or the like. Further, the excavator management system SYS may support remote control, automatic operation, etc. of the excavator 100 by using, for example, a management device 200 or the like.

The excavator 100 included in the excavator management system SYS may be one or a plurality of excavators 100. Further, the excavator management system SYS may have one management device 200 or a plurality of management devices 200.

<Overview of excavator>
The excavator 100 (an example of a work machine) includes a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be able to turn via a turning mechanism 2, a boom 4, an arm 5, and a bucket 6. It includes an attachment including and a cabin 10.

The excavator 100 is driven by the lower traveling body 1 (that is, a pair of left and right crawlers 1C), the upper swivel body 3, the boom 4, the arm 5, the bucket 6 and the like according to the operation of the operator boarding the cabin 10. Make the element work.

Further, the excavator 100 may be configured to be operable by an operator boarding the cabin 10, or in addition, may be configured to be remotely controlled (remote operation) from the outside of the excavator 100. When the excavator 100 is remotely controlled, the inside of the cabin 10 may be unmanned. Hereinafter, the description will proceed on the premise that the operation of the operator includes at least one of the operation of the cabin 10 with respect to the operation device 26 and the remote operation of an external operator.

The remote control includes, for example, an embodiment in which the excavator 100 is operated by an operation input relating to the actuator of the excavator 100 performed by a predetermined external device. The predetermined external device is, for example, a management device 200. In this case, the excavator 100, for example, through the communication device 70 described later, provides image information (hereinafter, “peripheral image”) representing the surrounding state including the front of the excavator 100 based on the image captured image output by the image pickup device 40 described later. It may be sent to an external device. Then, the external device may display the image information (peripheral image) received by the display device (hereinafter, “remote control display device”) provided in the own device. Further, various information images (information screens) displayed on the display device 50 inside the cabin 10 of the excavator 100 may be similarly displayed on the remote control display device of the external device. As a result, the operator of the external device can remotely control the excavator 100 while checking the display contents such as the image information and the information screen showing the state of the surroundings of the excavator 100 displayed on the remote control display device, for example. can. Then, the excavator 100 operates the actuator in response to the remote control signal indicating the content of the remote control received from the external device by the communication device 70, and causes the lower traveling body 1 (crawler 1C), the upper turning body 3, and the boom. 4. Driven elements such as the arm 5, the bucket 6 and the like may be driven.

Further, the remote control may include, for example, an embodiment in which the excavator 100 is operated by an external voice input or a gesture input to the excavator 100 by a person (for example, a worker) around the excavator 100. Specifically, the shovel 100 is a voice uttered by a surrounding worker or the like through a voice input device (for example, a microphone) or a gesture input device (for example, an image pickup device) mounted on the shovel 100 (own machine). Recognize gestures performed by workers and workers. Then, the excavator 100 operates an actuator according to the recognized voice, gesture, or the like, and causes the lower traveling body 1 (left and right crawlers 1C), the upper turning body 3, the boom 4, the arm 5, the bucket 6, and the like. The driven element may be driven.

Further, the excavator 100 may automatically operate the actuator regardless of the contents of the operator's operation. As a result, the excavator 100 automatically operates at least a part of the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper swivel body 3, the boom 4, the arm 5, and the bucket 6 (so-called "" "Automatic driving function" or "MC (Machine Control) function") is realized.

The automatic operation function is a function (so-called "semi-automatic luck function") that automatically operates a driven element (actuator) other than the driven element (actuator) to be operated according to the operation of the operator with respect to the operation device 26 or remote control. Alternatively, "operation support type MC function") may be included. Further, the automatic driving function is a function (so-called "fully automatic driving function") in which at least a part of a plurality of driven elements (actuators) is automatically operated on the premise that there is no operation or remote control of the operator's operating device 26. "Fully automatic MC function") may be included. When the fully automatic driving function is enabled in the excavator 100, the inside of the cabin 10 may be unmanned. Further, the semi-automatic operation function, the fully automatic operation function, and the like may include an embodiment in which the operation content of the driven element (actuator) to be automatically operated is automatically determined according to a predetermined rule. Further, for the semi-automatic driving function, the fully automatic driving function, etc., the excavator 100 autonomously makes various judgments, and according to the judgment results, the operation contents of the driven element (actuator) to be autonomously operated. May include aspects in which the is determined (so-called "autonomous driving function").

The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C (crawler 1CL on the left side and crawler 1CR on the right side), and the crawlers 1CL and 1CR are hydraulically driven by the corresponding traveling hydraulic motors 1M to self-propell. do.

The upper swivel body 3 swivels with respect to the lower traveling body 1 by hydraulically driving the swivel mechanism 2 by the swivel hydraulic motor 2A.

The boom 4 is vertically attached to the center of the front portion of the upper swing body 3, the arm 5 is attached to the tip of the boom 4 so as to be vertically rotatable, and the bucket 6 is vertically rotated to the tip of the arm 5. Can be installed.

The bucket 6 is an example of an end attachment, and is attached to the tip of the arm 5 in an appropriately replaceable manner according to the work content of the excavator 100. That is, instead of the bucket 6, a bucket of a type different from that of the bucket 6, for example, a relatively large large bucket, a slope bucket, a dredging bucket, or the like may be attached to the tip of the arm 5. Further, an end attachment of a type other than the bucket, for example, a stirrer, a breaker, a crusher, or the like may be attached to the tip of the arm 5. Further, a spare attachment such as a quick coupling or a tilt rotator may be provided between the arm 5 and the end attachment.

The boom 4, arm 5, and bucket 6 are hydraulically driven by the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9, respectively.

The cabin 10 is a cockpit for the operator to board and operate the excavator 100, and is mounted on the front left side of the upper swivel body 3, for example.

The image pickup device 40 and the distance sensor 45 are mounted on the upper surface of the upper swivel body 3.

Further, the power source of the excavator 100 is mounted on the upper swing body 3. The power source of the excavator 100 includes, for example, an engine 11 (for example, a diesel engine or the like) operated by a predetermined fuel (for example, light oil). Further, the power source of the excavator 100 is operated by electric power supplied from an external power source connected by a power storage device (for example, a capacitor, a lithium ion battery, etc.) or a cable in place of the engine 11 or in addition to the engine 11. An electric motor or the like may be included.

Further, various hydraulic devices such as a main pump 14, a pilot pump 15, and a control valve 17 are mounted on the upper swing body 3.

The main pump 14 is driven by a power source such as an engine 11 or an electric motor, and supplies hydraulic oil to various hydraulic actuators under the control of the controller 30. The hydraulic actuator includes a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and the like, in addition to the above-mentioned traveling hydraulic motor 1M and swivel hydraulic motor 2A.

The pilot pump 15 is driven by a power source such as an engine 11 or an electric motor, and supplies hydraulic oil to various hydraulic pilot-type hydraulic devices (for example, an operating device 26, a control valve 17, etc.).

The pilot pump 15 may be omitted. In this case, the hydraulic oil discharged from the main pump 14 may be supplied to various hydraulic pilot type hydraulic devices after being depressurized.

The control valve 17 selectively supplies the hydraulic oil discharged from the main pump 14 to each hydraulic actuator according to the operating state of the driven element (hydraulic actuator), and the flow rate of the hydraulic oil supplied to the hydraulic actuator. And adjust the flow direction. For example, the control valve 17 may be composed of a plurality of control valves (direction switching valves) that control the direction and flow rate of the hydraulic oil supplied to each hydraulic actuator. The control valve 17 is, for example, a hydraulic drive type (hydraulic pilot type), and a pilot pressure corresponding to an operation content corresponding to an operation content or an automatic operation function of each hydraulic actuator is input. As a result, the control valve (direction switching valve) corresponding to each hydraulic actuator is hydraulically driven according to the input pilot pressure. Further, the control valve 17 may be, for example, an electromagnetic drive type such as an electromagnetic solenoid type, and an electric signal corresponding to an operation command corresponding to the operation content of the operation device 26 or the automatic operation function is input. As a result, the control valve (direction switching valve) corresponding to each hydraulic actuator is electromagnetically driven according to the input electric signal.

Inside the cabin 10, for example, an operation device 26, a controller 30, a display device 50, an input device 52, a sound output device 54, and the like are provided. Further, for example, a communication device 70 is provided on the upper surface of the cabin 10.

The communication device 70 may be provided inside the cabin 10 or may be provided at a portion of the upper swing body 3 different from the cabin 10.

The operating device 26 operates a driven element driven by an actuator (specifically, a hydraulic actuator) such as a lower traveling body 1, an upper swivel body 3, and an attachment (boom 4, arm 5, and bucket 6). Used for. In other words, the operating device 26 is a traveling hydraulic motor 1M, a swing hydraulic motor 2A, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder corresponding to each of the hydraulic actuators (crawler 1CL, 1CR) for driving the driven element. 9 etc.) is used to operate. The operating device 26 includes, for example, each driven element, that is, a lever device, a pedal device, and the like corresponding to each hydraulic actuator.

The operating device 26 is, for example, a hydraulic pilot type. In this case, the operating device 26 uses the hydraulic oil supplied from the pilot pump 15 to output a pilot pressure to the control valve 17 according to the operation content of each driven element (that is, each corresponding hydraulic actuator). do. The operation content includes, for example, an operation direction and an operation amount. As a result, the control valve 17 can realize the operation of each driven element (hydraulic actuator) according to the operation content of the operating device 26.

Further, the operating device 26 may be, for example, an electric type. In this case, the operation device 26 outputs an electric signal (hereinafter, “operation signal”) corresponding to the operation content of each driven element (that is, each corresponding hydraulic actuator) to the controller 30. Then, the controller 30 controls the operation hydraulic control valve (hereinafter, "operation hydraulic control valve") provided in the oil passage (pilot line) between the pilot pump 15 and the control valve 17 in response to the operation signal. Output the command. As a result, the hydraulic control valve for operation uses the hydraulic oil supplied from the pilot pump 15 to apply the pilot pressure to the control valve 17 according to the operation content of each driven element (hydraulic actuator) in the operating device 26. Can be made to. Therefore, the control valve 17 can realize the operation of each driven element (hydraulic actuator) according to the operation content of the operating device 26.

Further, the driven element of the excavator 100, that is, the corresponding hydraulic actuator may be remotely controlled as described above. For example, a signal representing the content of remote control (remote control signal) is transmitted from a predetermined external device to the excavator 100, and the controller 30 receives the remote control signal through the communication device 70. Then, the controller 30 responds to the operation hydraulic control valve according to the content of remote control defined by the remote control signal (for example, the driven element or hydraulic actuator to be operated, the operation direction, the operation amount, etc.). Output a control command. As a result, the hydraulic control valve for operation can apply the pilot pressure according to the content of remote control to the hydraulically driven control valve 17 by using the hydraulic oil supplied from the pilot pump 15. Therefore, the control valve 17 can realize the operation of each driven element (that is, each corresponding hydraulic actuator) according to the content of remote control.

In the excavator 100, a part or all of various hydraulic actuators may be replaced with an electric actuator. That is, the excavator 100 may be a hybrid excavator or an electric excavator. In this case, the controller 30 drives the electric actuator or the electric actuator to give a control command according to the operation content of the operation device 26, the content of the remote control defined by the remote control signal, the content of the operation command corresponding to the automatic operation function, and the like. It may be output to the driver etc.

Further, when the gate lock lever of the cabin 10 is in the upright state (hereinafter, "locked state"), the operation command corresponding to the operation, remote control, and automatic operation function for the operation device 26 is invalidated, and the excavator 100 is displaced. Do not work. On the other hand, when the gate lock lever is in the lowered state (hereinafter, "release state"), the operation command corresponding to the operation, remote control, and automatic operation function for the operation device 26 becomes effective, and the excavator 100 operates. As a result, for example, when the operator gets into the cockpit or gets out of the cabin 10 from the cockpit with the gate lock lever upright, the operator's body touches the operating device 26 and the excavator 100 Can be avoided. Further, for example, by lowering the gate lock lever, the operator can start the operation of the excavator 100.

For example, the operating state of the gate lock lever and the most upstream gate lock valve of the pilot line that supplies pilot pressure from the pilot pump 15 to various hydraulic devices (for example, the hydraulic pilot type operating device 26 and the hydraulic control valve for operation). It is linked with the open / closed state. Specifically, when the gate lock lever is in the released state, the gate lock valve is maintained in the open state (communication state), and the pilot pressure is supplied from the pilot pump 15 to the operating device 26 and the hydraulic control valve for operation. Therefore, the operating device 26 and the hydraulic control valve for operation can supply the pilot pressure according to the operation of the operator to the control valve 17 to operate the hydraulic actuator. On the other hand, when the gate lock lever is in the locked state, the gate lock valve is maintained in the closed state (closed state), and the supply of pilot pressure from the pilot pump 15 to the operating device 26 and the hydraulic control valve for operation is cut off. Therefore, the operation device 26 and the hydraulic control valve for operation cannot supply the pilot pressure corresponding to the operation of the operator to the control valve 17, and invalidate the operation command corresponding to the operation of the operator and the automatic operation function. Can be done.

Further, as described above, a part or all of the hydraulic actuator may be replaced with the electric actuator. In this case, for example, when the gate lock lever is in the locked state, the controller 30 may not output a control command corresponding to an operator's operation or an operation command of the automatic operation function to the electric actuator, the driver, or the like. As a result, it is possible to invalidate the operation command corresponding to the operator's operation or the automatic operation function according to the locked state of the gate lock lever.

The communication device 70 communicates with the outside of the excavator 100 (for example, the management device 200 or the terminal device 300) through the communication line NW.

The communication line NW includes, for example, a wide area network (WAN). The wide area network may include, for example, a mobile communication network having a base station as an end. Further, the wide area network may include, for example, a satellite communication network that uses a communication satellite. Further, the wide area network may include, for example, an Internet network. Further, the communication line NW includes, for example, an internal local network (LAN: Local Area Network) such as a facility where the management device 200 is installed. The local network may be wired, wireless, or both. Further, the communication line NW may include, for example, a short-range communication line according to a predetermined wireless communication standard such as WiFi or Bluetooth (registered trademark).

The excavator 100 communicates with the management device 200 by using, for example, the communication device 70. As a result, the shovel 100 can transmit data related to the shovel 100 (own machine) to the management device 200 and receive data related to the control of the shovel 100 (own machine).

<Overview of management equipment>
The management device 200 is provided outside the shovel 100, and manages, for example, the operating state and the operating state of the shovel 100. Further, the management device 200 may support the remote control of the excavator 100. Further, the management device 200 may, for example, transmit a control command regarding the automatic driving function to the shovel 100, or provide the user with information for monitoring the operating status of the shovel 100 operated by the automatic driving function. You may support the automatic operation of the excavator 100.

The management device 200 is, for example, a server device. The server device may be, for example, a cloud server installed in a management center or the like outside the work site where the excavator 100 works. Further, the server device may be, for example, an edge server installed in a temporary office in the work site of the excavator 100 or a communication facility (for example, a station building, a base station, etc.) relatively close to the work site. Further, the management device 200 may be, for example, a terminal device (user terminal) used by the user of the shovel management system SYS. The users of the excavator management system SYS are, for example, the operator of the excavator 100, the owner of the excavator 100, the manager of the excavator 100, the manager (supervisor) of the work site, the worker of the work site, the manager of the management center, and the management center. Workers and the like may be included. The user terminal may be, for example, a stationary terminal device (stationary terminal) arranged in a temporary office or the like in the work site of the excavator 100. Further, the user terminal may be a portable terminal device (portable terminal) that can be carried by the user. The stationary terminal may include, for example, a desktop computer terminal. Further, the mobile terminal may include, for example, a mobile phone, a smartphone, a tablet terminal, a laptop computer terminal, or the like. Further, the user terminal as the management device 200 is configured to be communicable with the shovel 100 via the management device 200 as the server device, and for monitoring and remote control of the shovel 100 via the management device 200 as the server device. It may be in the form of providing support or the like.

The management device 200 includes a control device 210, a communication device 220, an output device 230, and an input device 240.

The control device 210 controls the management device 200. The function of the control device 210 may be realized by any hardware, or a combination of any hardware and software. The control device 210 includes, for example, a memory device such as a CPU (Central Processing Unit) and a RAM (Random Access Memory), an auxiliary storage device such as a ROM (Read Only Memory), and an interface device for input / output to / from the outside. It is mainly composed of a computer.

The communication device 220 communicates with the outside of the management device 200 (for example, the shovel 100) or the like through the communication line NW.

The output device 230 outputs information to users such as the manager and workers of the management device 200. The output device 230 may include, for example, a display device, a lighting device, or the like that outputs visual information. The display device may include, for example, the above-mentioned remote control display device. The display device is, for example, a liquid crystal display or an organic EL (Electroluminescence) display that outputs image information. Further, the output device 230 may include, for example, a sound output device that outputs auditory information. The sound output device is, for example, a speaker, a buzzer, or the like.

The input device 240 receives various inputs from the management device user, and signals corresponding to the input contents are taken into the control device 210. The input device 240 may include, for example, the above-mentioned remote control operating device. The input device 240 includes, for example, an operation input device that receives an operation input from a user. The operation input device is, for example, a mouse, a keyboard, a touch panel, a button, a toggle, a lever, or the like. Further, the input device 240 may include, for example, a voice input device or a gesture input device that accepts voice input or gesture input from the user.

The control device 210 communicates with the excavator 100 by using the communication device 220. As a result, the management device 200 can receive, for example, various data transmitted (uploaded) from the excavator 100 and collect various data related to the excavator 100. Further, the control device 210 may use, for example, the communication device 220 to transmit data relating to the control of the excavator 100 to the excavator 100 and control the excavator 100 from the outside. Further, the control device 210 uses, for example, a communication device 220 to send a signal (remote control signal) representing the content of remote control received from the input device 240 (remote control operation device) to the shovel 100 to be remotely controlled. You may send it. Thereby, the management device 200 can support the remote control of the excavator 100.

[Peripheral monitoring device]
Next, in addition to FIGS. 1 and 2, the configuration of the peripheral monitoring device 150 mounted on the shovel 100 according to the present embodiment will be described with reference to FIG.

FIG. 3 is a block diagram showing an example of the configuration of the peripheral monitoring device 150 according to the present embodiment.

The peripheral monitoring device 150 monitors the surrounding conditions of the excavator 100 by using the image pickup device 40 and the distance sensor 45.

For example, the peripheral monitoring device 150 displays visual information indicating the situation around the shovel 100 on the display device 50. The visual information includes, for example, a peripheral image generated based on the image captured by the image pickup apparatus 40. Further, the visual information includes, for example, an environmental map showing the positions of objects around the shovel 100. The environmental map may be, for example, a three-dimensional environmental map, or a two-dimensional environmental map showing the positions of objects around the excavator 100 on a plane on which the excavator 100 (lower traveling body 1) is located. You may. The two-dimensional environment map may be, for example, an Occupancy Grid Map (OGM). The following is an explanation focusing on a two-dimensional environmental map.

Further, for example, the presence or absence of a monitored object (hereinafter, “monitoring object”) within a predetermined range (hereinafter, “monitoring area”) around the peripheral monitoring device 150 and the excavator 100 is monitored. The monitored object may include a worker working around the excavator 100, a supervisor at the work site, or the like. In addition, the monitoring object includes any object other than humans, such as materials temporarily placed at the work site, fixed non-moving obstacles such as temporary offices at the work site, and moving obstacles such as vehicles including trucks. (Ie, obstacles) can be included. Then, the peripheral monitoring device 150 may activate a predetermined safety function when a monitoring object is present within a predetermined range.

The safety function includes, for example, an alarm for at least one of the inside of the cabin 10, the outside of the cabin 10, and a remote operator or administrator of the excavator 100, and a notification for notifying the existence of a monitoring object in the monitoring area. Features may be included. As a result, a monitoring object exists in the monitoring area around the excavator 100 for the operator inside the cabin 10, the workers around the excavator 100, the operator and the manager who remotely control the excavator 100, and the like. You can call attention to that. Hereinafter, the notification function for the inside of the cabin 10, that is, the operator or the like may be referred to as an “internal notification function”. Further, the notification function to the outside of the cabin 10, that is, to the workers and the like around the excavator 100 may be referred to as an "external notification function". Further, the notification function for a remote operator, administrator, or the like of the cabin 10 may be referred to as a "remote notification function". Further, the safety function may include, for example, an operation limiting function for limiting the operation of the shovel 100 with respect to the operation of the operating device 26 or remote control.

The operation limiting function may include, for example, an operation deceleration function that slows down the operation speed of the excavator 100 for an operation command corresponding to the operation of the operation device 26, remote control, or the automatic operation function. Further, the operation restriction function includes an operation stop function for stopping the operation of the excavator 100 and maintaining the stopped state regardless of the operation of the operation device 26, the remote control, or the operation command corresponding to the automatic operation function. good.

As shown in FIG. 3, the peripheral monitoring device 150 includes a controller 30, an operation information output device 29, an image pickup device 40, a distance sensor 45, a display device 50, an input device 52, a sound output device 54, and the like. The hydraulic control valve 60 and the communication device 70 are included.

The controller 30 controls the functions of the peripheral monitoring device 150. The controller 30 is mounted in the cabin 10, for example, as described above.

The function of the controller 30 may be realized by any hardware, or a combination of any hardware and software. The controller 30 is mainly composed of a computer including, for example, a memory device (main storage device) such as a CPU and RAM, an auxiliary storage device such as a ROM, and an interface device for input / output with the outside. The controller 30 realizes various functions by, for example, loading a program installed in the auxiliary storage device into the memory device and causing the CPU to execute the program. The controller 30 includes a peripheral image generation unit 301, an environment map generation unit 302, a display processing unit 303, a peripheral monitoring data transmission unit 304, and a safety function control unit 305 as functional units.

A part or all of the functions of the controller 30 may be realized by another controller. That is, the function of the peripheral monitoring device 150 may be realized by being shared by a plurality of controllers. Further, the controller 30 may control the excavator 100 other than the function of the peripheral monitoring device 150. That is, the controller 30 may be a dedicated control device specialized for the function of the peripheral monitoring device 150, or is a general-purpose control device that controls various functions of the excavator 100 including the function of the peripheral monitoring device 150. You may. Further, a part or all of the functions of the controller 30 may be transferred to the outside of the excavator 100 such as the management device 200 (an example of the peripheral monitoring device). In this case, the excavator 100 may transmit the output data of the image pickup device 40 and the distance sensor 45 to an external device such as the management device 200. The operation of the shovel 100 may be controlled in real time in response to a control command from, for example, an external device such as the management device 200. For example, when very high-speed communication can be executed through the communication line NW such as a ^5G (5th Generation) mobile communication network, even if a signal is exchanged between the management device 200 and the excavator 100. This is because it is possible to provide a peripheral monitoring function without delay.

The operation information output device 29 is the operation content of the operation device 26, the content of the remote operation, or the content of the operation command corresponding to the automatic operation function, that is, the information regarding the operation content related to each driven element (hydraulic actuator) (hereinafter referred to as the operation content). , "Operation information") is output.

The operation information output device 29 may be, for example, a sensor (hereinafter, “operation information acquisition sensor”) that acquires information regarding the operation content of the operation device 26. The operation information acquisition sensor is, for example, a linear encoder that senses the operation direction and operation amount of the lever, pedal, or the like of the operation device 26. Further, the operation information acquisition sensor is, for example, a pressure sensor that senses the pilot pressure on the secondary side of the hydraulic pilot type operation device 26. Further, the operation information output device 29 may be, for example, an electric operation device 26. This is because the operation signal output from the electric operation device 26 corresponds to the operation information. When the excavator 100 is remotely controlled, the operation information output device 29 is, for example, a communication device 70 that receives a remote control signal from an external device. Further, when the excavator 100 is operated by the automatic operation function, the operation information output device 29 may be, for example, an arithmetic unit that outputs an operation command.

The image pickup apparatus 40 is attached to the upper part of the upper swivel body 3 and takes an image of the periphery of the excavator 100 extending from a region relatively close to the excavator 100 to a region relatively far from the excavator 100, and outputs image data (captured image). The image pickup apparatus 40 includes cameras 40F, 40B, 40L, and 40R. Hereinafter, the cameras 40F, 40B, 40L, 40R may be comprehensively referred to, or any one of the cameras 40F, 40B, 40L, 40R may be individually referred to as "camera 40X".

The camera 40F, the camera 40B, the camera 40L, and the camera 40R image the front, rear, left side, and right side of the upper swivel body 3, respectively.

The camera 40F is attached to the front portion of the upper swing body 3, for example, the front end portion of the upper surface of the cabin 10. The camera 40B is attached to the rear portion of the upper swivel body 3, for example, the rear end portion of the upper surface of the upper swivel body 3. Further, the camera 40L is attached to, for example, the left end portion of the upper surface of the upper swing body 3. Further, the camera 40L is attached to the left side portion of the upper swivel body 3, for example, the left end portion of the upper surface of the upper swivel body 3. The camera 40R is attached to the right side portion of the upper swivel body 3, for example, the right end portion of the upper surface of the upper swivel body 3.

The camera 40X is a monocular camera (that is, a wide-angle camera) having a very wide angle of view. Further, for example, the camera 40X may be a stereo camera, a depth camera, a three-dimensional camera, or the like.

The camera 40F captures an imaging range in front of the upper swivel body 3, for example, a horizontal imaging range extending from the front left to the front right. Further, the camera 40B captures an imaging range behind the upper swivel body 3, for example, a horizontal imaging range extending from the left rear to the right rear. Further, the camera 40L captures, for example, an image pickup range on the left side of the upper swivel body 3, for example, a horizontal image pickup range extending from the left front to the left rear of the upper swivel body 3. Further, the camera 40R captures, for example, an image pickup range on the right side of the upper swivel body 3, for example, a horizontal image pickup range extending from the front right to the rear right of the upper swivel body 3. Further, the camera 40X captures an image pickup range in the vertical direction including the distance from the ground in the vicinity of the excavator 100 to the distance of the excavator 100 in the upper part of the upper swivel body 3.

The camera 40X outputs a captured image at predetermined intervals (for example, 1/30 second) from the start (that is, the key switch ON) to the stop (that is, the key switch OFF) of the excavator 100, for example. The captured image output from the camera 40X is captured by the controller 30.

The number (4) of the cameras 40X mounted on the excavator 100 is an example, and if it is possible to acquire image data in a necessary range in the horizontal direction and the vertical direction around the excavator 100, it is 3 or less. It may be 5 or more. That is, an arbitrary number of one or a plurality of cameras 40X may be mounted on the excavator 100.

The distance sensor 45 (an example of a peripheral monitoring sensor) acquires data on the situation around the excavator 100. The distance sensor 45 includes distance sensors 45F, 45BL, 45BR, 45L, 45R. Hereinafter, the distance sensors 45F, 45BL, 45BR, 45L, 45R may be comprehensively referred to, or any one of the distance sensors 45F, 45BL, 45BR, 45L, 45R may be individually referred to as a “distance sensor 45X”.

The distance sensor 45F, the distance sensor 45BL, the distance sensor 45BR, the distance sensor 45L, and the distance sensor 45R acquire data regarding the front, left rear, right rear, left side, and right side conditions of the upper swivel body 3, respectively. ..

The distance sensor 45F is attached to the front portion of the upper swing body 3, for example, the front end portion of the upper surface of the cabin 10. The distance sensor 45BL is attached to the left rear portion of the upper swing body 3, for example, the left rear end portion of the upper surface of the cabin 10. The distance sensor 45BR is attached to the right rear portion of the upper swivel body 3, for example, the rear end portion to the right of the upper surface of the upper swivel body 3. The distance sensor 45L is attached to the left portion of the upper swivel body 3, for example, the left end portion of the upper surface of the upper swivel body 3. The distance sensor 45R is attached to the right portion of the upper swivel body 3, for example, the right end portion of the upper surface of the upper swivel body 3.

For example, the distance sensor 45X is LIDAR (Light Detection and Ranging). Further, for example, the distance sensor 45X may be, for example, a millimeter wave radar, an ultrasonic sensor, or the like. Hereinafter, the description will be mainly given to the case where the distance sensor 45X is a lidar.

The distance sensor 45BL is configured to be capable of irradiating an irradiation range on the left rear side of the upper swivel body 3, for example, a horizontal irradiation range extending from the left rear side to the rear side of the upper swivel body 3. Further, the distance sensor 45BR is configured to be capable of irradiating infrared rays to the irradiation range on the right rear side of the upper swivel body 3, for example, the horizontal irradiation range extending from the right rear side to the rear side of the upper swivel body 3. Further, the distance sensor 45L is configured to be capable of irradiating infrared rays to the irradiation range on the left side of the upper swing body 3, for example, the horizontal irradiation range extending from the left front to the left rear of the upper swing body 3. Further, the distance sensor 45R is configured to be capable of irradiating infrared rays to the irradiation range on the right side of the upper swing body 3, for example, the horizontal irradiation range extending from the front right to the rear right of the upper swing body 3.

For example, the distance sensor 45X irradiates infrared rays in a certain direction and receives reflected light (infrared rays) from an object in that direction, so that data representing the situation around the shovel 100, specifically, is received. Acquires data related to the reflected light (hereinafter referred to as "light receiving data"). The distance sensor 45X is, for example, a scanning type LIDAR, which is a three-dimensional laser scanner capable of scanning the irradiation direction of an infrared laser in the vertical direction and the horizontal direction. Further, the distance sensor 45X may be a flash type LIDAR that irradiates infrared rays from a light emitting module over a three-dimensional range and captures reflected light (infrared rays) with a three-dimensional distance image element.

The light receiving data includes data related to the time (TOF: Time Of Flight) from the infrared irradiation for each infrared irradiation direction to the reception of the reflected light (hereinafter, “TOF data”). Further, the light receiving data includes data relating to the intensity of the reflected light received for each direction of infrared irradiation (hereinafter, “light receiving intensity data”).

Further, the distance sensor 45X may output TOF data, light receiving intensity data, and the like for each irradiation direction. Further, the distance sensor 45X performs point cloud data and each point representing an object around the shovel 100 in a three-dimensional space based on the shovel 100 by performing known post-processing on the TOF data, the light receiving intensity data, and the like. Attribute data and the like may be output. The three-dimensional space with respect to the excavator 100 has, for example, the X-axis and the Y-axis on the ground plane and the Z-axis in the height direction with the position of the shovel 100 (for example, the ground contact center of the lower traveling body 1) as the origin. It may be the XYZ coordinate space defined by. The attribute data may be, for example, data representing the type of the object corresponding to each point. Hereinafter, the description of the distance sensor 45X will be mainly described in the case of outputting the post-processed data.

When the TOF data, the light receiving intensity data, and the like are output from the distance sensor 45X, the post-processing may be executed by the controller 30.

The distance sensor 45X outputs the post-processed data at predetermined intervals from the start to the stop of the excavator 100, respectively. The post-processed data output from the distance sensor 45X is taken into the controller 30.

The number (5) of the distance sensors 45X mounted on the excavator 100 is an example, and it is possible to acquire data on the situation in the required range in the horizontal direction and the vertical direction (height direction) around the excavator 100. If there are, it may be 4 or less, or 6 or more. That is, an arbitrary number of one or a plurality of distance sensors 45X may be mounted on the excavator 100.

The display device 50 (an example of the display unit) is provided around the cockpit in the cabin 10, specifically, at a position easily visible to the operator seated in the cockpit, and displays various image information to be notified to the operator. .. The display device 50 is, for example, a liquid crystal display or an organic EL display, and may be a touch panel type that also serves as an input device 52.

The input device 52 receives various inputs related to the peripheral monitoring device 150 from the operator and outputs them to the controller 30. The input device 52 includes, for example, an operation input device of any hardware such as a touch panel, a touch pad, a button, a toggle, and a rotary knob. Further, the input device 52 may include software operation input means that can be operated through hardware operation means, such as a virtual button icon on the operation screen displayed on the display device 50. Further, the input device 52 may include a voice input device or a gesture input device that accepts voice input or gesture input of a user such as an operator.

The sound output device 54 outputs sound toward at least one of the inside and the outside of the cabin 10. The sound output device 54 may include, for example, a speaker or a buzzer provided inside the cabin 10, and may output sound to the operator. Further, the sound output device 54 may include, for example, a horn, a travel alarm, or the like, and may output sound toward the outside of the cabin 10, specifically, the periphery of the shovel 100.

The hydraulic control valve 60 is used to realize the operation limiting function of the excavator 100. Specifically, the hydraulic control valve 60 can be operated by a control command from the controller 30, and adjusts the pilot pressure acting on the control valve 17. The hydraulic control valve 60 may be provided, for example, in the pilot line between the pilot pump 15 and the hydraulic pilot type operating device 26, that is, the pilot line on the primary side of the operating device 26. In this case, the hydraulic control valve 60 may be the gate lock valve described above. Further, the hydraulic control valve 60 may be provided, for example, in the pilot line between the operating device 26 and the control valve 17, that is, the pilot line on the secondary side of the operating device 26. Further, the hydraulic control valve 60 may be the above-mentioned hydraulic control valve for operation, for example, in the case of an electric operation device 26, in the case of remote control, in the case of an automatic operation function, and the like. The hydraulic control valve 60 is, for example, an electromagnetic proportional valve. Specifically, the hydraulic control valve 60 can adjust the pilot pressure acting on the control valve 17 under the control of the controller 30, regardless of the operation content of the operation device 26 or the remote control content. ..

The peripheral image generation unit 301 generates a peripheral image representing the situation around the excavator 100 to be displayed on the display device 50 based on the image data captured from the image pickup device 40 (camera 40X), and outputs the peripheral image to the display processing unit 303. .. The peripheral image generation unit 301 may generate a peripheral image and output it to the display processing unit 303, for example, when a predetermined input requesting display of the peripheral image from the user is received through the input device 52.

The peripheral image may be, for example, the captured image (image data) of the camera 40X itself (hereinafter, “through image”).

For example, the peripheral image generation unit 301 may output the image data of one camera 40X selected from all the cameras 40X as a peripheral image. Further, for example, the peripheral image generation unit 301 may arrange the image data of two or more cameras 40X selected from all the cameras 40X in a predetermined direction and output the image data as a peripheral image.

The peripheral image generation unit 301 may switch which camera 40X's image data is selected as the peripheral image among all the cameras 40X according to a predetermined input to the input device 52. As a result, the operator can display the through image in the direction he / she wants to see on the display device 50 by using the input device 52.

Further, the peripheral image may be a composite image in which image data of at least two cameras 40X are combined.

For example, the peripheral image generation unit 301 may generate a viewpoint conversion image viewed from a virtual viewpoint by performing a known viewpoint conversion process, a composition process, or the like based on the captured images of a plurality of cameras 40X as a composite image. .. For example, the virtual viewpoint may be a bird's-eye view seen from directly above the excavator 100.

The peripheral image generation unit 301 may switch between generating (outputting) a through image as a peripheral image or generating a composite image according to a predetermined input to the input device 52. Thereby, the operator can switch the peripheral image displayed on the display device 50 between the through image and the composite image by using the input device 52.

The environment map generation unit 302 (an example of the map generation unit) generates an environment map showing the positions of objects around the excavator 100 based on the output (post-processing data) of the distance sensor 45X. The environment map generation unit 302 generates, for example, a two-dimensional environment map (for example, OGM) as described above.

The environment map generation unit 302 may generate an environment map in place of the output data of the distance sensor 45X, or in addition, based on the output data (captured image) of the camera 40X (an example of a peripheral monitoring sensor). good.

The image data corresponding to the environmental map generated by the environmental map generation unit 302 (hereinafter, “environmental map image data”) may be displayed on the display device 50. The environment map generation unit 302 may output the environment map image data to the display processing unit 303, for example, when a predetermined input requesting the display of the environment map from the user is received through the input device 52. As a result, the peripheral monitoring device 150 can intuitively grasp the position of an object around the excavator 100 by the operator through an image corresponding to an environmental map. Further, in the two-dimensional environment map, information such as the position of an object in the three-dimensional space around the excavator 100 is collected on a two-dimensional plane. Therefore, the peripheral monitoring device 150 allows the operator to more intuitively grasp the position of an object around the excavator 100 through an image corresponding to a two-dimensional environmental map.

The display device 50 may display only one of the peripheral image and the environmental map image, or both may be displayed at the same time.

Further, the environmental map data generated by the environmental map generation unit 302 may be output to the safety function control unit 305.

The display processing unit 303 causes the display device 50 to display the peripheral image generated by the peripheral image generation unit 301 and the environmental map generated by the environmental map generation unit 302.

The function of the display processing unit 303 may be built in the display device 50.

The peripheral monitoring data transmission unit 304 transmits various data generated by the peripheral monitoring device 150 (controller 30) to an external device (for example, a management device 200 or the like) through the communication device 70.

The peripheral monitoring data transmission unit 304 transmits, for example, the peripheral image data generated (output) by the peripheral image generation unit 301 to the management device 200 or the like via the communication device 70. Thereby, the management device 200 or the like can display the peripheral image on the output device 230 (remote control display device) or the like, and support the remote control of the excavator 100 by the operator. Further, the management device 200 or the like can display a peripheral image on, for example, an output device 230 (display device) or the like, and can support the monitoring work of the excavator 100 in operation by the automatic operation function by the observer.

Further, the peripheral monitoring data transmission unit 304 transmits, for example, the environment map image data generated (output) by the environment map generation unit 302 to the management device 200 or the like via the communication device 70. Thereby, the management device 200 or the like can display the environment map image data on the output device 230 (display device for remote control) (an example of the display unit) or the like, and support the remote device of the excavator 100 by the operator. .. Further, the management device 200 or the like can display the environment map image data on the output device 230 (display device) or the like, and can support the monitoring work of the excavator 100 in operation by the automatic operation function by the observer.

In particular, an operator who remotely controls the excavator 100 and an observer who monitors automatic driving cannot directly visually recognize the situation around the excavator 100, and for example, grasp the situation around the excavator 100 from a peripheral image. There is a need. Therefore, a remote-controlled operator or observer simply checks the peripheral image, for example, whether or not there is a possibility that the shovel 100 and the object come into contact with each other, and the positional relationship between the object around the shovel 100 and the shovel 100. May be difficult to judge properly.

On the other hand, the management device 200 may display the environment map image data received from the excavator 100 on the output device 230 (display device) through the output device 230 or the like so that the operator or the observer of the remote control can visually recognize the environment map image data. can. Therefore, the remote-controlled operator or observer can intuitively grasp the position or the like of an object around the excavator 100.

The safety function control unit 305 controls the safety function and activates the safety function when the monitored object exists in the monitoring area. For example, the safety function control unit 305 may determine the existence or nonexistence of the monitored object in the monitoring area based on the environment map generated by the environment map generation unit. Further, the safety function control unit 305 may determine the existence or nonexistence of the monitored object in the monitored area based on, for example, the output of the image pickup device 40 or the distance sensor 45.

The safety function control unit 305 activates the notification function, for example, when a monitoring object exists in a predetermined range (hereinafter, “notification range”) included in the monitoring area. The notification range may be the same as the monitoring area, or may be set so that the outer edge thereof is relatively closer to the excavator 100 than the monitoring area.

The safety function control unit 305, for example, controls the sound output device 54 to activate a sound (that is, auditory method) notification function to at least one of the inside and the outside of the cabin 10. At this time, the safety function control unit 305 may change the pitch, sound pressure, timbre, sounding cycle when the sound is periodically blown, the content of the sound, etc. according to various conditions. good.

Further, the safety function control unit 305 activates, for example, a notification function for the inside of the cabin 10 by a visual method. Specifically, the safety function control unit 305 may display an image indicating that the monitored object is detected on the display device 50. Further, the safety function control unit 305 determines the position on the peripheral image or the environmental map image corresponding to the position of the monitored object on the peripheral image or the environmental map image displayed on the display device 50 and the position seen from the shovel 100 of the monitored object. You may emphasize it. More specifically, the safety function control unit 305 superimposes and displays a frame surrounding the monitored object displayed on the peripheral image or the environmental map image, or displays the peripheral image corresponding to the actual position of the detected monitored object. Or, a marker may be superimposed and displayed at a position on the environment map image. As a result, the display device 50 can realize a visual notification function for the operator. Further, the safety function control unit 305 may use a warning light, a lighting device, or the like inside the cabin 10 to notify the operator or the like inside the cabin 10 that the monitored object has been detected.

Further, the safety function control unit 305 controls the lighting device such as the headlight and the display device for the outside provided in the house part of the upper swivel body 3, for example, so that the operator and the supervisor around the excavator 100 can control the safety function control unit 305. A person or the like may be activated by a visual notification function. Further, the safety function control unit 305 may operate the notification function to the operator in the cabin 10 by a tactile method, for example, by controlling a vibration generator that vibrates the cockpit in which the operator sits. As a result, the peripheral monitoring device 150 causes the operator, the workers around the excavator 100, the supervisor, and the like to recognize that a monitoring object (for example, a person such as a worker) exists around the excavator 100. Can be done. Therefore, the peripheral monitoring device 150 can urge the operator to confirm the safety status around the excavator 100 and urge the workers in the monitoring area to evacuate from the monitoring area. ..

Further, the safety function control unit 305 may activate the remote notification function by transmitting a command signal indicating the operation of the notification function to the management device 200, for example, through the communication device 70. In this case, when the management device 200 (control device 210) receives the command signal from the excavator 100 by the communication device 220, the management device 200 may output an alarm by a visual method or an auditory method through the output device 230. As a result, the operator who remotely controls the excavator 100 through the management device 200, the observer who monitors the excavator 100 operated by the automatic operation function through the management device 200, and the like have entered the monitoring object within the notification range around the excavator 100. You can figure out that.

The remote notification function of the safety function control unit 305 may be transferred to an external device such as the management device 200. In this case, the management device 200 receives data for determining the presence / absence of the monitored object in the monitoring area from the excavator 100, and determines whether or not the monitored object has entered the notification range based on the received data. Then, the management device 200 activates the remote notification function when the monitoring object exists within the notification range.

Further, the safety function control unit 305 may change the notification mode (that is, the notification method) according to the positional relationship between the monitoring object detected within the notification range and the shovel 100.

For example, when the monitoring object within the notification range is located at a position relatively far from the shovel 100, the safety function control unit 305 gives an alarm with a relatively low degree of urgency to alert the operator or the like to the monitoring object (for example). Hereinafter, "attention level alarm") may be output. Hereinafter, the range of the notification range that is relatively far from the excavator 100, that is, the range corresponding to the caution level alarm may be referred to as the “attention notification range” for convenience. On the other hand, when the monitoring object within the notification range is relatively close to the excavator 100, the safety function control unit 305 notifies that the monitoring object is approaching the excavator 100 and the degree of danger is increasing, which is relatively urgent. A high-degree alarm (hereinafter referred to as “alert level alarm”) may be output. Hereinafter, the range in which the distance from the shovel 100 is relatively short in the notification range, that is, the range corresponding to the warning of the warning level may be referred to as the “warning notification range”.

In this case, the safety function control unit 305 may make the pitch, sound pressure, timbre, sounding cycle, etc. of the sound output from the sound output device 54 different between the caution level alarm and the alert level alarm. .. Further, the safety function control unit 305 has an image indicating that a surveillance object displayed on the surveillance image displayed on the display device 50 is detected between the caution level alarm and the alert level alarm, and an image showing that the surveillance object is detected. The color, shape, size, presence / absence of blinking, blinking cycle, etc. of the monitored object or the image that emphasizes the position of the monitored object (for example, a frame or a marker) may be different. As a result, the peripheral monitoring device 150 determines the urgency of the operator or the like, in other words, the monitoring object, due to the difference between the notification sound (alarm sound) output from the sound output device 54 and the notification image displayed on the display device 50. The degree of approach to the excavator 100 can be grasped.

After the operation of the notification function is started, the safety function control unit 305 receives a predetermined operation for canceling the operation of the notification function when the target monitoring object no longer exists within the notification range or through the input device 52. In some cases, the notification function may be stopped.

Further, the safety function control unit 305 activates the operation limiting function when, for example, a monitored object exists within a predetermined range (hereinafter, “operation limiting range”) included in the monitoring area. The operation restriction range may be the same as the monitoring area, or may be set so that the outer edge thereof is relatively closer to the excavator 100 than the monitoring area. Further, the operation restriction range may include an operation deceleration range in which the operation speed of the excavator 100 is made slower than usual with respect to the operation command corresponding to the operation of the operation device 26, the remote control, and the automatic operation function. Further, the operation restriction range includes an operation stop range for stopping the operation of the excavator 100 and maintaining the stopped state regardless of the operation command corresponding to the operation of the operation device 26, the remote control, and the automatic operation function. good. That is, the operation limiting range may include at least one of the operation deceleration range and the operation stop range. For example, when the operation limit range includes both the operation deceleration range and the operation stop range, the operation stop range is, for example, a range close to the shovel 100 in the operation limit range, and the operation deceleration range is the operation limit range. It is a range set outside the operation stop range of.

The safety function control unit 305 operates the operation limiting function that limits the operation of the excavator 100 by controlling the hydraulic control valve 60. In this case, the safety function control unit 305 may limit the operation of all driven elements (that is, the corresponding hydraulic actuators) or may limit the operation of some driven elements (hydraulic actuators). good. As a result, the peripheral monitoring device 150 can slow down or stop the operation of the shovel 100 when the monitoring object exists within the operation limiting range around the excavator 100. Therefore, the peripheral monitoring device 150 can suppress the occurrence of contact between the monitoring object around the excavator 100 and the excavator 100.

Further, the safety function control unit 305 cancels the operation of the operation restriction function when the target monitoring object no longer exists within the operation restriction range after the operation of the operation restriction function is started, or through the input device 52. When the operation is accepted, the operation restriction function is stopped. The operation for canceling the operation of the notification function for the input device 52 and the operation for canceling the operation of the operation limiting function may be the same or different.

[First example of how to generate an environmental map]
Next, a first example of a method for generating an environmental map will be described with reference to FIGS. 4 to 12.

<Map generation process>
First, a map generation processing flow corresponding to the environmental map generation method according to this example will be described.

FIG. 4 is a flowchart schematically showing a first example of the map generation process by the environment map generation unit 302.

The flowchart of FIG. 4 is executed at predetermined control cycles, for example, during operation from the start (for example, the key switch ON) to the stop (for example, the key switch OFF) of the excavator 100. Hereinafter, the same may apply to the flowcharts of FIGS. 13, 20, 21, 24, and 28, which will be described later.

As shown in FIG. 4, in step S102, the environment map generation unit 302 acquires three-dimensional point group data from the latest post-processing data captured from the distance sensor 45X.

When the process of step S102 is completed, the controller 30 proceeds to step S104.

In step S104, the environment map generation unit 302 acquires information regarding the height of the excavator 100 (hereinafter, “excavator height information”).

For example, FIG. 5 is a diagram showing an example of a posture state of the excavator 100.

The height of the excavator 100 changes depending on the posture state of the attachment. For example, as shown in FIG. 5, when the excavator 100 performs a crane operation of suspending a suspended load W to the tip of an attachment, the tip of the attachment (bucket 6) is raised to a relatively high position and the positions of the attachments are relative to each other. May be higher. Therefore, the excavator height information may be, for example, information regarding the maximum value of the height of the excavator 100 according to the posture state of the attachment. Further, the excavator height information may be information on the current height of the excavator 100 determined from the posture state of the current excavator 100. The posture state of the excavator 100 may be determined based on, for example, the output of the posture sensor that detects the posture angles of the boom 4, the arm 5, and the bucket 6. The attitude sensor may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a six-axis sensor, an inertial measurement unit (IMU), and the like. Further, the posture state of the excavator 100 may be determined based on, for example, data corresponding to the attachment included in the output data of the camera 40F or the distance sensor 45F.

Returning to FIG. 4, the controller 30 proceeds to step S106 when the process of step S104 is completed.

In step S106, the environment map generation unit 302 has a range higher than the height of the shovel 100 defined in the shovel height information in the Z-axis direction (that is, the height direction of the shovel 100) from the three-dimensional point group data. Delete the point data of. As a result, only the point cloud data in the range of the maximum height of the excavator 100 or the current height or less remains from the latest three-dimensional point cloud data around the excavator 100.

When the process of step S106 is completed, the controller 30 proceeds to step S108.

In step S108, the environment map generation unit 302 generates an environment map (for example, a two-dimensional OGM) by projecting the remaining point cloud data perpendicularly to the XY plane. For example, the environment map generation unit 302 defines a two-dimensional grid group in which each of the X-axis direction and the Y-axis direction is divided at equal intervals on the XY plane, and projects the remaining point group data vertically on the two-dimensional grid. do. Then, the environment map generation unit 302 identifies an occupied grid in which an object exists or an unoccupied grid in which an object does not exist for each grid of the two-dimensional grid group based on the number and density of points included in the grid. , A two-dimensional environment map (OGM) may be generated. As a result, the environmental map generation unit 302 can prevent the environmental map from reflecting the point data regarding the object in the range higher than the height of the excavator 100.

When the process of step S108 is completed, the controller 30 ends the current flowchart.

For example, when an environmental map image containing information about an object that is unlikely to come into contact with the excavator 100 (ie, an object that is higher than the height of the excavator 100) is displayed on the display device 50, the operator can display the object. May pay attention to. As a result, the work efficiency of the excavator 100 may decrease.

Further, for example, if the safety function is activated based on the data of the map image including the information about the object that is unlikely to come into contact with the shovel 100, the work of the shovel 100 is stopped and the work efficiency of the shovel 100 is lowered. There is a possibility that it will end up.

On the other hand, in this example, the peripheral monitoring device 150 does not reflect the information about the object that is unlikely to come into contact with the excavator 100 (that is, the object existing at a position higher than the height of the excavator 100) in the environmental map. Can be. Therefore, the peripheral monitoring device 150 can suppress a decrease in the work efficiency of the excavator 100.

<First specific example of environmental map>
Subsequently, a first specific example of the environmental map generated by the method for generating the environmental map according to this example will be described.

6 and 7 are diagrams showing an example of the positional relationship between the excavator 100 and surrounding objects. Specifically, FIGS. 6 and 7 are a top view and a side view showing a specific example of the positional relationship between the shovel 100 and the drone DRN flying in the work site. 8 and 9 are diagrams showing the first example and the second example of the environment map image displayed on the display device 50. Specifically, FIGS. 8 and 9 are specific examples (environmental map images 800, 900) of the environmental map image displayed on the display device 50 in the situations of FIGS. 6 and 7. In the environment map images 800 and 900, the unoccupied grid indicating the absence of an object is represented in white, and the occupied grid representing the presence of an object is represented in black.

As shown in FIGS. 6 and 7, in this example, the drone DRN is flying in the sky in front of the excavator 100 (upper swivel body 3) to the left.

For example, when the drone DRN is flying at the position indicated by the dotted line, the height position is lower than the current height of the excavator 100 (dashed line in the figure). Therefore, when the excavator 100 moves (runs) forward, there is a high possibility that the excavator 100 (attachment) will come into contact with the drone DRN.

In this case, in this example, the environment map generation unit 302 corresponds to the drone DRN by using the point cloud data in the range in the Z-axis direction below the current height and the maximum height of the excavator 100 as described above. The group data can be reflected in the environmental map.

Specifically, as shown in FIG. 8, the environment map image 800 is displayed on the display device 50.

An excavator image CG showing the position of the excavator 100 is displayed on the environment map image 800. The excavator image CG has an upwardly pointed shape in the drawing, and indicates that the upward direction is in front of the excavator 100 (upper swivel body 3). The same applies to the environmental map images 900, 1100, 1200, 1500, 1600, 1700, 1800, 1900, 2200, 2300, 2700, and 2900 described below.

The environmental map image 800 includes an occupied grid group 801 corresponding to the drone DRN.

The occupied grid group 801 is defined by four two-dimensional grids that are separated from the excavator image CG to some extent upward to the left. This allows the operator to know that there is an object (drone DRN) that may come into contact with the left front of the excavator 100.

In the same situation, the environment map image 800 may be displayed on the output device 230 (display device) of the management device 200 as described above. The same applies to the environmental map images 900, 1100, 1200, 1500, 1600, 1700, 1800, 1900, 2200, 2300, 2700, and 2900 described below.

On the other hand, when the drone DRN is flying at the position indicated by the solid line, the height position is higher than the current height of the excavator 100. Therefore, even if the excavator 100 moves (runs) forward, it is unlikely that it will come into contact with the drone DRN.

In this case, in this example, the environment map generation unit 302 corresponds to the drone DRN by using the point cloud data in the range in the Z-axis direction below the current height and the maximum height of the excavator 100 as described above. It is possible to prevent the group data from being reflected in the environmental map.

Specifically, as shown in FIG. 9, the environment map image 900 is displayed on the display device 50.

In the environment map image 900, each two-dimensional grid of the two-dimensional grid group 901 corresponding to the position on the XY plane of the drone DRN is set as an unoccupied grid indicating that no object exists. This allows the operator to continue the work of the excavator 100 without being aware of the presence of the drone DRN, which is unlikely to come into contact with it. Therefore, the peripheral monitoring device 150 can improve the work efficiency of the excavator 100.

The dotted frame surrounding the two-dimensional grid group 901 in the figure is drawn for the sake of explanation, and is not displayed on the screen of the actual display device 50 (environmental map image 900). Hereinafter, the same applies to the dotted frame surrounding the two-dimensional grid group 1203 of FIG. 12, the two-dimensional grid group 2301 of FIG. 23, and the two-dimensional grid group 2705 of FIG. 27, which will be described later.

<Second specific example of environmental map>
Next, a second specific example of the environmental map generated by the environmental map generation method according to this example will be described.

FIG. 10 is a diagram showing another example of the positional relationship between the excavator 100 and surrounding objects. Specifically, FIG. 10 is a perspective view showing the positional relationship between the excavator 100 and the gantry crane CRN installed at the work site. 11 and 12 are diagrams showing a third example and a fourth example of the environment map image displayed on the display device 50. Specifically, FIGS. 11 and 12 are specific examples (environmental map images 1100, 1200) of the environmental map image displayed on the display device 50 in the situation of FIG. 10. In the environmental map images 1100 and 1200, the unoccupied grid indicating the absence of an object is represented in white, and the occupied grid representing the presence of an object is represented in black.

As shown in FIG. 10, in this example, the gantry crane CRN exists in front of the excavator 100 (upper swing body 3).

The gantry crane CRN is configured to include two stanchions and a bridge connecting the upper ends of the two stanchions.

For example, if the current height or maximum height of the shovel 100 is higher than the lower end of the bridge, there is a high possibility that the attachment of the shovel 100 will come into contact with the bridge when passing between the two columns.

In this case, as described above, the environmental map generation unit 302 corresponds to the bridge portion of the portal crane by using only the point cloud data in the range in the Z-axis direction below the current height and the maximum height of the excavator 100. The point cloud data can be reflected in the environmental map.

Specifically, as shown in FIG. 11, the environment map image 1100 is displayed on the display device 50.

The environmental map image 1100 includes occupied grid groups 1101-1103 corresponding to the gantry crane CRN.

The occupied grid groups 1101, 1102 correspond to the two stanchions of the gantry crane, and are defined by four grids separated from the excavator image CG to the left and to the right to some extent upward.

The occupied grid group 1103 corresponds to the bridge portion of the portal crane CRN, and is defined in the range of a plurality of grids connecting the occupied grid groups 1101 and 1102.

As a result, the operator can grasp that there is a contactable object (a stanchion portion and a bridge portion of the gantry crane CRN) having a certain width in the left-right direction in front of the excavator 100.

On the other hand, for example, if the current height or maximum height of the excavator 100 is lower than the lower end of the bridge portion, there is a possibility that the attachment of the excavator 100 may come into contact with the bridge portion even if it passes between the two strut portions. low.

In this case, the environmental map generation unit 302 corresponds to the bridge portion of the portal crane CRN by using the point cloud data in the range in the Z-axis direction below the current height and the maximum height of the excavator 100 as described above. It is possible to prevent the point cloud data from being reflected in the environmental map.

Specifically, as shown in FIG. 12, the environment map image 1200 is displayed on the display device 50.

The environmental map image 1200 includes occupied grid groups 1201, 1202 corresponding to the gantry crane CRN.

The occupied grid groups 1201, 1202 correspond to the two stanchions of the portal crane, similar to the occupied grid groups 1101, 1102 of the environmental map image 1100, and the four grids separated to some extent to the left and to the right from the excavator image CG. Is stipulated in.

Each two-dimensional grid of the two-dimensional grid group 1203 corresponding to the position of the bridge portion of the portal crane CRN between the occupied grid groups 1201 and 1202 on the XY plane is set as an unoccupied grid indicating the absence of an object. To. As a result, the operator can continue the work of the excavator 100 (for example, the movement passing under the gantry crane CRN) without being aware of the gantry crane CRN bridge that is unlikely to come into contact with the operator. be able to. Therefore, the peripheral monitoring device 150 can improve the work efficiency of the excavator 100.

[Second example of how to generate an environmental map]
Next, a second example of a method for generating an environmental map will be described with reference to FIGS. 13 to 18.

<Map generation process>
First, a map generation processing flow corresponding to the environmental map generation method according to this example will be described.

FIG. 13 is a flowchart schematically showing a second example of the map generation process by the environment map generation unit 302.

As shown in FIG. 13, in step S202, the environment map generation unit 302 acquires three-dimensional point group data from the latest post-processing data captured from the distance sensor 45X.

When the process of step S202 is completed, the controller 30 proceeds to step S204.

In step S204, the environment map generation unit 302 calculates the occupancy probability for each grid in the three-dimensional grid group divided at equal intervals in the X-axis direction, the Y-axis direction, and the Z-axis direction, and calculates the occupancy probability of the object. Identify the presence or absence.

The occupancy probability represents the probability that an object exists at a position corresponding to a grid. The occupancy probability is defined based on the density of the point cloud data of the grid, and the higher the density, the higher the occupancy probability. For example, the environment map generation unit 302 determines that an object exists in a grid in which the occupancy probability of the calculation result is equal to or greater than a predetermined threshold value. On the other hand, the environment map generation unit 302 may determine that the object does not exist in the grid whose occupancy probability of the calculation result is less than a predetermined threshold value, and set the occupancy probability of the grid to zero.

When the process of step S204 is completed, the controller 30 proceeds to step S206.

In step S206, the environment map generation unit 302 assigns an occupancy probability to a grid around the object (specifically, a grid having a occupancy probability of zero) according to the distance from the object specified in step S204. ..

For example, FIG. 14 is a diagram showing an example of the relationship between the distance from the object and the occupancy probability, specifically, the relationship between the distance from the object in the target grid and the occupancy probability given to the grid.

As shown in FIG. 14, a grid relatively close to the object is given a relatively high occupancy probability, and the occupancy probability given decreases as the distance from the object to the grid increases.

Returning to FIG. 13, the controller 30 proceeds to step S208 when the process of step S206 is completed.

In step S208, the environment map generation unit 302 acquires excavator height information.

When the process of step S208 is completed, the controller 30 proceeds to step S210.

In step S210, the environment map generation unit 302 sets the maximum value of the occupancy probability of each three-dimensional lattice arranged in the Z-axis direction in the XY plane (two) within the range equal to or less than the height of the shovel 100 defined in the shovel height information. An environmental map is generated by projecting onto a (dimensional grid). Specifically, the environment map generation unit 302 sets the maximum value in the occupancy probability of each three-dimensional grid at the same XY position for the three-dimensional grid in the range in the Z-axis direction equal to or less than the height of the excavator 100 at the XY position. Set as the occupancy probability of the two-dimensional grid of. As a result, the environment map generation unit 302 can reflect the occupancy probability of the three-dimensional grid in the range below the height of the excavator 100 on the environment map, and occupy the three-dimensional grid in the range higher than the height of the excavator 100. It is possible to prevent the probability from being reflected in the environmental map.

When the process of step S210 is completed, the controller 30 ends the current flowchart.

For example, the accuracy of the environmental map is affected by the resolution of the distance sensor 45X, the processing performance of the controller 30, and the like. Therefore, for example, depending on the accuracy of the environmental map, the shovel 100 may come too close to the object, and when the shovel 100 reaches a position near the occupied grid, it may come into contact with the object corresponding to the occupied grid. be.

On the other hand, in the present embodiment, the information indicating the existence of the object can be reflected not only in the position where the object exists but also in the vicinity of the position where the object exists. Therefore, it is possible to suppress a situation in which the excavator 100 gets too close to the object.

<First specific example of environmental map>
Subsequently, a first specific example of the environmental map generated by the method for generating the environmental map according to this example will be described.

15 and 16 are diagrams showing a fifth example and a sixth example of the environment map image displayed on the display device 50. Specifically, FIGS. 15 and 16 are specific examples (environmental map images 1500, 1600) of the environmental map image displayed on the display device 50 in the situation of FIGS. 6 and 7 described above. In the environmental map images 1500 and 1600, the unoccupied grid indicating that the object does not exist is represented by white, and the occupied grid indicating the existence of the object is represented by black, and the object may exist. A two-dimensional lattice (hereinafter referred to as "quasi-occupied lattice") representing the above is represented by a satin finish. Hereinafter, the same applies to the cases of the environmental map images 1700, 1800, 1900, 2200, 2300, 2700, and 2900 described later.

In this example, the quasi-occupied grid is one type, but a plurality of types of quasi-occupied grids may be defined depending on the magnitude of the occupancy probability (that is, the magnitude of the distance to the object). Hereinafter, the same may apply to the cases of the environmental map images 1700, 1800, 1900, 2200, 2300, 2700, and 2900 described later.

For example, as shown in FIGS. 6 and 7, when the drone DRN is flying at the position indicated by the dotted line, when the excavator 100 moves (runs) forward as described above, the excavator 100 (attachment) becomes the drone DRN. There is a high possibility of contact. Further, if the excavator 100 gets too close to the drone DRN, due to an error between the actual position of the drone DRN and the occupied grid corresponding to the drone DRN on the environmental map, the excavator 100 reaches the shovel 100 before reaching the occupied grid. There is also the possibility of contact.

In this case, in this example, the environment map generation unit 302 reflects the occupancy probability of the three-dimensional grid corresponding to the drone DRN and the occupancy probability of the surrounding three-dimensional grids on the environment map (two-dimensional grid) as described above. be able to.

Specifically, as shown in FIG. 15, the environment map image 1500 is displayed on the display device 50.

The environmental map image 1500 includes an occupied grid group 1501 and a quasi-occupied grid group 1502 corresponding to the drone DRN.

The occupied grid group 1501 is defined as four two-dimensional grids that are separated from the excavator image CG to some extent upward to the left, as in the occupied grid group 801 of FIG. This allows the operator to know that there is an object (drone DRN) that may come into contact with the left front of the excavator 100.

The quasi-occupied lattice group 1502 is defined as a two-dimensional lattice group surrounding the occupied lattice group 1501. This allows the operator to recognize that there is an area around the object (drone DRN) in front of the excavator 100 to the left that may come into contact with the object. Therefore, the peripheral monitoring device 150 can suppress a situation in which the excavator 100 comes into contact with an object (drone DRN) too close to the object (drone DRN), and can improve safety.

By appropriately adjusting the method of assigning the occupancy probability according to the distance to the object (see FIG. 14), a two-dimensional lattice separated by two or more with respect to the occupancy grid group 1501 is set as a quasi-occupancy grid. May be done. Hereinafter, the same may apply to the cases of the environmental map images 1700, 1800, 1900, 2200, 2300, 2700, and 2900 described later.

On the other hand, as shown in FIGS. 6 and 7, when the drone DRN is flying at the position indicated by the solid line, even if the excavator 100 moves (runs) forward as described above, it may come into contact with the drone DRN. Is relatively low. However, the difference between the height position of the drone DRN and the height of the excavator 100 may be relatively small. In this case, depending on the accuracy of the environmental map, the height position of the actual drone DRN may be lower than the position indicated by the occupied grid, and the excavator 100 may not be able to pass directly under the drone DRN.

In this case, in this example, the environmental map generation unit 302 can reflect the occupancy probability of the three-dimensional grid corresponding to the periphery of the drone DRN on the environmental map (two-dimensional grid) as described above.

Specifically, as shown in FIG. 16, the environment map image 1600 is displayed on the display device 50.

The environmental map image 1600 includes a quasi-occupied grid group 1601 corresponding to the drone DRN.

The quasi-occupied lattice group 1601 is set by reflecting the occupancy probability of the three-dimensional lattice in the range below the height of the excavator 100, which is adjacent to the lower part of the drone DRN, in the two-dimensional lattice. The quasi-occupied lattice group 1601 indicates that an object (drone DRN) exists above the XY positions corresponding to these two-dimensional lattices, and the difference between the height position and the height of the excavator 100 is relatively small. Represents. As a result, the operator can grasp that an object (drone DRN) relatively close to the height of the excavator 100 exists above the XY position of the quasi-occupied grid group 1601. Therefore, the peripheral monitoring device 150 can suppress a situation in which the excavator 100 comes into contact with an object (drone DRN) too close to the object (drone DRN), and can improve safety.

When the difference between the height position of the drone DRN and the height of the excavator 100 is relatively large, the display device 50 displays an environment map image similar to the environment map image 900 of FIG. In this case, since the environment map image does not include the occupied grid or the quasi-occupied grid corresponding to the drone DRN, the operator can continue the work of the excavator 100 without being aware of the existence of the drone DRN.

<Second specific example of environmental map>
Next, a second specific example of the environmental map generated by the environmental map generation method according to this example will be described.

17 to 19 are diagrams showing the 7th to 9th examples of the environmental map image displayed on the display device 50. Specifically, FIGS. 17 to 19 are specific examples (environmental map images 1700, 1800, 1900) of the environmental map image displayed on the display device 50 in the situation of FIG. 10 described above.

For example, in the situation of FIG. 10, when the current height or the maximum height of the excavator 100 is higher than the lower end of the bridge portion of the gantry crane CRN, when passing between the two strut portions, the attachment of the excavator 100 is bridged. There is a high possibility that it will come into contact with the part. Further, in the situation of FIG. 10, if the excavator 100 gets too close to the strut portion of the portal crane CRN, the excavator 100 may come into contact with the strut portion.

In this case, in this example, as described above, the environment map generation unit 302 determines the occupancy probability of the three-dimensional grid corresponding to the portal crane CRN (post portion and bridge portion) and the occupancy probability of the three-dimensional grid around it as the environment. It can be reflected on the map (two-dimensional grid).

Specifically, as shown in FIG. 17, the environment map image 1700 is displayed on the display device 50.

The environmental map image 1700 includes occupied grid groups 1701-1703 and quasi-occupied grid groups 1704-1706 corresponding to the portal crane CRN.

The occupied grid groups 1701,1702 correspond to the stanchions of the portal crane CRN, similar to the occupied grid groups 1101, 1102 in FIG.

The occupied grid group 1703 corresponds to the bridge portion of the portal crane CRN, as in the occupied grid group 1103 of FIG.

As a result, the operator can grasp that there is a contactable object (a stanchion portion and a bridge portion of the gantry crane CRN) having a certain width in the left-right direction in front of the excavator 100.

The quasi-occupied lattice groups 1704 and 1705 are defined as two-dimensional lattice groups around the occupied lattice groups 1701 and 1702.

The quasi-occupied grid group 1706 is defined as a two-dimensional grid group around the occupied grid group 1703.

This allows the operator to recognize that there is an area around the object (the stanchion and bridge of the gantry crane CRN) in front of the excavator 100 that may come into contact with the object. Therefore, the peripheral monitoring device 150 can suppress a situation in which the excavator 100 comes into contact with an object (a column portion or a bridge portion of a portal crane CRN) too close to the object, and can improve safety.

On the other hand, for example, if the current height or maximum height of the excavator 100 is lower than the lower end of the bridge portion, there is a possibility that the attachment of the excavator 100 may come into contact with the bridge portion even if it passes between the two strut portions. low. However, the difference between the height position of the bridge and the height of the excavator 100 may be relatively small. In this case, depending on the accuracy of the environmental map, the actual height position of the bridge portion may be lower than the position indicated by the occupied grid, and the excavator 100 may not be able to pass directly under the bridge portion.

In this case, in this example, the environmental map generation unit 302 can reflect the occupancy probability of the three-dimensional grid corresponding to the periphery of the bridge portion of the portal crane CRN on the environmental map (two-dimensional grid) as described above. ..

Specifically, as shown in FIG. 18, the environment map image 1800 is displayed on the display device 50.

The environmental map image 1800 includes occupied grid groups 1801, 1802 and quasi-occupied grid groups 1803-1805 corresponding to the portal crane CRN.

The occupied grid groups 1801, 1802 correspond to the two strut portions of the portal crane CRN, similar to the occupied grid groups 1701, 1702 in FIG.

The quasi-occupied lattice group 1803, 1804 is defined as a two-dimensional lattice group around the occupied lattice group 1801, 1802.

The quasi-occupied lattice group 1805 is set by reflecting the occupancy probability of the three-dimensional lattice in the range below the height of the excavator 100, which is adjacent to the lower part of the bridge portion of the portal crane CRN, in the two-dimensional lattice. In the quasi-occupied grid group 1805, an object (bridge portion of the portal crane CRN) exists above the XY positions corresponding to these two-dimensional grids, and the difference between the height position and the height position of the excavator 100. Indicates that is relatively small. As a result, the operator can visually recognize the quasi-occupied grid group 1805, and the difference between the height of the excavator 100 and the height position of the lower end of the bridge portion of the gantry crane CRN in front is relatively small and cannot pass underneath. You can recognize that there is a possibility. Therefore, the peripheral monitoring device 150 can suppress a situation in which the excavator 100 tries to pass under the bridge portion of the portal crane CRN and comes into contact with the excavator 100, and can improve safety.

Further, as shown in FIG. 19, when the difference between the height position of the lower end of the bridge portion of the gantry crane CRN and the height of the excavator 100 is relatively large, the environmental map image 1900 is displayed on the display device 50. To.

The environmental map image 1900 includes an occupied grid group 1901,1902 and a quasi-occupied grid group 1903, 1904 corresponding to the portal crane CRN.

The occupied grid groups 1901, 1902 correspond to the two strut portions of the portal crane CRN, similar to the occupied grid groups 1701, 1702 and the like in FIG.

The quasi-occupied lattice group 1903, 1904 is defined as a two-dimensional lattice group around the occupied lattice group 1901, 1902.

The two-dimensional grid corresponding to the position of the bridge portion of the portal crane CRN between the quasi-occupied grid groups 1903 and 1904 on the XY plane is set to the unoccupied grid indicating the absence of an object. As a result, the operator can continue the work of the excavator 100 without being aware of the bridge portion of the gantry crane CRN, which is unlikely to come into contact with the crane. Therefore, the peripheral monitoring device 150 can improve the work efficiency of the excavator 100. Further, by visually recognizing the quasi-occupied grid groups 1903 and 1904, the operator can recognize that when passing under the gantry crane CRN, there is a possibility of contact if the support is too close. Therefore, the peripheral monitoring device 150 can suppress a situation in which the excavator 100 gets too close to the support column when passing under the portal crane CRN, and can improve the safety of the excavator 100.

[Third example of how to generate an environmental map]
Next, a third example of a method for generating an environmental map will be described with reference to FIG. 20.

FIG. 20 is a flowchart schematically showing a third example of the map generation process by the environment map generation unit 302.

Since steps S302 and S304 are the same as the processes of steps S202 and S204 in FIG. 13, the description thereof will be omitted.

When the process of step S304 is completed, the controller 30 proceeds to step S306.

In step S306, the environment map generation unit 302 acquires information regarding the posture state of the shovel 100 (attachment) (hereinafter, “excavator posture information”). The excavator posture information is, for example, information regarding the posture angles of the boom 4, the arm 5, and the bucket 6. As a result, the environment map generation unit 302 can grasp the outer shape of the excavator 100 including the attachment.

When the process of step S306 is completed, the controller 30 proceeds to step S308.

In step S308, the environment map generation unit 302 sets the shovel 100 when the shovel 100 is present at the XY position for each two-dimensional grid of the two-dimensional grid group projected on the XY plane, and step S304. Calculate the distance to the object of the three-dimensional lattice specified in. When there are a plurality of objects, the environment map generation unit 302 calculates the distance between the object having the shortest distance and the excavator 100. Specifically, the environment map generation unit 302 calculates the distance between the outer shape of the shovel 100 and the three-dimensional grid corresponding to the object based on the shovel posture information.

When the process of step S308 is completed, the controller 30 proceeds to step S310.

In step S310, the environment map generation unit 302 generates environment map information by determining the occupancy probability according to the distance between the excavator 100 and the object for each two-dimensional grid on the XY plane.

For example, the environment map generation unit 302 may determine the occupancy probability for each two-dimensional lattice of the XY plane by using the relationship of FIG. 14 as in the case of the second example described above.

As shown in FIG. 14, when the distance from the object to the excavator 100 is zero, the occupancy probability is set to the maximum (for example, "1"), and the occupancy probability given as the distance from the object to the excavator 100 increases. Becomes smaller. For example, the environment map generation unit 302 uses a two-dimensional grid having a maximum occupancy probability corresponding to zero distance from an object to the excavator 100 as an occupancy grid, and has a predetermined value whose occupancy probability is smaller than the maximum value and smaller than this maximum value. The above two-dimensional lattice may be used as a quasi-occupied lattice. Then, the environment map generation unit 302 may use the other two-dimensional grid as an unoccupied grid.

Returning to FIG. 20, when the processing of step S310 is completed, the controller 30 ends the processing of the current flowchart.

As described above, in this example, the peripheral monitoring device 150 reflects not only the height of the excavator 100 but also the external shape of the excavator 100 based on the posture state of the excavator 100 on the environmental map. As a result, the peripheral monitoring device 150 can generate a more accurate environmental map. Therefore, the peripheral monitoring device 150 can make the operator more appropriately recognize the positional relationship between the shovel 100 and the surrounding objects by displaying the environmental map on the display device 50. Further, the peripheral monitoring device 150 can operate the safety function in a form that more appropriately reflects the positional relationship between the shovel 100 and surrounding objects by using the environmental map.

[Fourth example of environmental map generation method]
Next, a fourth example of a method for generating an environmental map will be described with reference to FIGS. 21 to 23.

<Map generation process>
First, a map generation processing flow corresponding to the environmental map generation method according to this example will be described.

FIG. 21 is a flowchart schematically showing a fourth example of the map generation process by the environment map generation unit 302.

As shown in FIG. 21, steps S402 to S406 are the same as the processes of steps S202 to S206 in FIG. 13, and thus the description thereof will be omitted.

When the process of step S406 is completed, the controller 30 proceeds to step S408.

In step S408, the environment map generation unit 302 determines whether or not a moving object exists in the object specified in step S404. The environment map generation unit 302 may determine whether or not the object has moved by comparing the positions of the object specified in the previous flowchart and the object specified in the current flowchart, for example. The environment map generation unit 302 proceeds to step S410 if there is a moving object, and proceeds to step S412 if there is no moving object.

In step S410, the environment map generation unit 302 corrects the occupancy probability of the moving object and the three-dimensional lattice corresponding to the periphery of the moving object according to the moving direction of the object. For example, the environment map generation unit 302 moves the occupancy probability given to each of the moving object and the three-dimensional grid corresponding to the periphery of the object to another three-dimensional grid existing in the moving direction of the object. You may modify the form to be (granted). In this case, the distance between the three-dimensional grid before movement and the three-dimensional grid after movement may be determined according to the moving speed of the object. For example, the larger the moving speed of the object, the larger the interval may be set.

When the process of step S410 is completed, the controller 30 proceeds to step S412.

Since steps S421 and S414 are the same as the processes of steps S208 and S210 in FIG. 13, the description thereof will be omitted.

When the process of step S414 is completed, the controller 30 ends the process of the current flowchart.

As described above, in this example, when the object around the excavator 100 is moving, the peripheral monitoring device 150 can reflect the moving direction on the environment map. As a result, the peripheral monitoring device 150 can generate an environmental map assuming that the moving object exists at a position moved in the moving direction from the current position, that is, an environmental map assuming the future. Therefore, the peripheral monitoring device 150 displays an environmental map assuming the future on the display device 50, so that the peripheral object is more appropriate for the operator who sees the environmental map and reflects it in the future (future) driving behavior. Can provide information about. Further, the peripheral monitoring device 150 can activate the safety function at an earlier timing or avoid the operation of the unnecessary safety function by activating the safety function based on the environment map assuming the future. ..

<Specific example of environmental map>
Subsequently, a specific example of the environmental map generated by the environmental map generation method according to this example will be described.

22 and 23 are diagrams showing the tenth example and the eleventh example of the environment map image displayed on the display device 50. Specifically, FIGS. 22 and 23 are specific examples (environmental map images 2200 and 2300) of the environmental map image displayed on the display device 50 in the situation of FIGS. 6 and 7 described above.

For example, as shown in FIGS. 6 and 7, as described above, the drone DRN is at a position higher than the height of the excavator 100 when flying at the position of the solid line. However, if this drone DRN is flying while moving downward, it is highly likely that it will fall to a range below the height of the excavator 100. Therefore, for example, in a situation where the excavator 100 moves forward, when the excavator 100 reaches the position of the drone DRN, the drone DRN flying while moving downward reaches a range equal to or less than the height of the excavator 100, and the drone May come into contact with the DRN.

In this case, in this example, the environmental map generation unit 302 can generate an environmental map assuming the position of the drone DRN in the future by reflecting the moving direction of the drone DRN on the environmental map as described above.

Specifically, as shown in FIG. 22, the environment map image 2200 is displayed on the display device 50.

The environmental map image 2200 includes an occupied grid group 2201 and a quasi-occupied grid group 2202 corresponding to the drone DRN.

The occupied grid group 2201 exists at a position higher than the height of the excavator 100 and corresponds to a drone DRN moving downward.

The quasi-occupied grid group 2202 exists at a position higher than the height of the excavator 100, corresponds to a region adjacent to the front, back, left and right of the drone DRN moving downward, and is set as a two-dimensional grid surrounding the occupied grid group 2201. To.

As described above, the peripheral monitoring device 150 assumes the future that the drone DRN exists below the current position, and provides the operator with an environmental map image including the occupied grid group 2201 and the quasi-occupied grid group 2202 corresponding to the drone DRN. Can be provided. Therefore, for example, the peripheral monitoring device 150 uses the current position of the drone DRN as a determination material to operate the operator to drive the excavator 100 forward as it is, and suppresses a situation in which the excavator 100 comes into contact with the drone DRN. Can be done. Therefore, the peripheral monitoring device 150 can improve the safety of the excavator 100.

On the other hand, as shown in FIGS. 6 and 7, as described above, the drone DRN exists in a range equal to or less than the height of the excavator 100 when flying at the position of the dotted line. However, if the drone DRN is flying upwards, it is likely to rise to a range higher than the height of the excavator 100. Therefore, for example, in a situation where the excavator 100 moves forward, when the excavator 100 reaches the position of the drone DRN, the drone DRN that flies while moving upward is evacuated to a range higher than the height of the excavator 100. There is a high possibility that it is.

In this case, in this example, the environmental map generation unit 302 can generate an environmental map assuming the position of the drone DRN in the future by reflecting the moving direction of the drone DRN on the environmental map as described above.

Specifically, as shown in FIG. 23, the environment map image 2300 is displayed on the display device 50.

The occupied grid or quasi-occupied grid corresponding to the drone DRN is not set in the environment map image 2300, and the two-dimensional grid group 2301 corresponding to the position of the drone DRN is set to the unoccupied grid.

In this way, the peripheral monitoring device 150 assumes that the drone DRN moves (retracts) to a position higher than the height of the excavator 100, and provides the operator with an environmental map that does not include the occupied grid or quasi-occupied grid corresponding to the drone DRN. offer. As a result, the peripheral monitoring device 150 suppresses a situation in which the operator interrupts the work (for example, movement) by the excavator 100 and the work efficiency of the excavator 100 is lowered by using the current position of the drone DRN as a determination material. Can be done.

[Fifth example of how to generate an environmental map]
Next, a fifth example of a method for generating an environmental map will be described with reference to FIGS. 24 to 27.

<Map generation process>
First, a map generation processing flow corresponding to the environmental map generation method according to this example will be described.

FIG. 24 is a flowchart schematically showing a fifth example of the map generation process by the environment map generation unit 302.

As shown in FIG. 24, steps S502 and S504 are the same as the processes of steps S202 and S204 of FIG. 13 described above, and thus the description thereof will be omitted.

When the process of step S504 is completed, the controller 30 proceeds to step S506.

In step S506, the environment map generation unit 302 specifies the type of the object specified in step S504. The type of object is predetermined. The types of objects may include, for example, people, traffic cones, fences, utility poles, work machines, work vehicles, materials, buildings, sand mountains and the like. For example, the environment map generation unit 302 may specify the type of the object based on the attribute data of the point cloud data of the position corresponding to the object included in the post-processing data of the distance sensor 45X. Further, for example, the environment map generation unit 302 may specify the type of the object by applying a known image processing technique, a machine learning-based classifier, or the like to the image data of the camera 40X.

When the process of step S506 is completed, the controller 30 proceeds to step S508.

In step S508, the environment map generation unit 302 determines whether or not an object of a contactable type exists. The type of object that can be contacted corresponds to an object that is allowed to come into contact with the shovel 100 in advance, and is, for example, a sand mountain or the like. The environment map generation unit 302 proceeds to step S510 if an object of a contactable type exists, and proceeds to step S512 if it does not exist.

In step S510, the environment map generation unit 302 corrects the occupancy probability of the three-dimensional grid corresponding to the contactable type of object and the three-dimensional grid around it to zero.

In step S510, the environment map generation unit 302 reduces (reduces) the occupancy probability of the three-dimensional grid corresponding to an object of a contactable type and the three-dimensional grid around it to zero, instead of making it zero. There may be.

When the process of step S510 is completed, the controller 30 proceeds to step S512.

Since steps S512 to S516 are the same as the processes of steps S206 to S210 in FIG. 13, the description thereof will be omitted.

When the process of step S516 is completed, the controller 30 ends the process of the current flowchart.

As described above, in this example, when the object around the excavator 100 is an object that can come into contact with the excavator 100 (for example, a sandy mountain), the peripheral monitoring device 150 does not reflect the information about the object on the environmental map. can do. As a result, the peripheral monitoring device 150 can exclude information about an object that can come into contact with the excavator 100 from the environmental map displayed on the display device 50. Therefore, the peripheral monitoring device 150 can suppress a situation in which the work is interrupted for the operator to confirm the information about the object that can come into contact with the shovel 100, and the work efficiency of the shovel 100 is lowered. Further, the peripheral monitoring device 150 can operate the safety function based on the environmental map excluding the information about the object that can come into contact with the excavator 100. Therefore, the peripheral monitoring device 150 suppresses a situation in which the work of the shovel 100 is interrupted due to the operation of the safety function caused by the presence of an object that can come into contact with the shovel 100, and the work efficiency of the shovel 100 is lowered. be able to.

<Specific example of environmental map>
Subsequently, a specific example of the environmental map generated by the environmental map generation method according to this example will be described.

25 and 26 are diagrams showing still another example of the positional relationship between the excavator 100 and surrounding objects. Specifically, FIGS. 25 and 26 are upper surfaces showing specific examples of the positional relationship between the excavator 100 and the drone DRN flying in the work site, the worker WKR operating the drone DRN, and the sandy mountain SND in the work site. It is a figure and a side view. FIG. 27 is a diagram showing a twelfth example of an environment map image displayed on the display device 50. Specifically, FIG. 27 is a specific example (environmental map image 2700) of the environmental map image displayed on the display device 50 in the situations of FIGS. 25 and 26.

As shown in FIGS. 25 and 26, in this example, as in the case of FIGS. 6 and 7 described above, the position is below the height of the excavator 100 in the sky in front of the excavator 100 (upper swivel body 3) to the left. Drone DRN is flying. Further, in this example, there is a worker WKR who operates the drone DRN in front of the excavator 100 on the right side, and a sand mountain SND is in front of the excavator 100 on the left side.

Contact between the excavator 100 and the drone DRN and the worker WKR is not allowed as a matter of course, while contact between the excavator 100 and the sand mountain SND may be allowed.

In this case, in this example, as described above, the environmental map generation unit 302 can reflect the possibility of contact (permission or disapproval) with the excavator 100 on the environment map for each type of the object existing around the excavator 100.

Specifically, as shown in FIG. 27, the environment map image 2700 is displayed on the display device 50.

The environmental map image 2700 includes an occupied grid group 2701, an occupied grid 2702, and a quasi-occupied grid group 2703, 2704.

The occupied grid group 2701 corresponds to a drone DRN flying at a position below the height in front of the excavator 100 to the left.

The occupied grid 2702 corresponds to the worker WKR operating the drone DRN on the ground in front of the excavator 100 to the right.

The quasi-occupied grid group 2703 corresponds to a region adjacent to the front, back, left and right of the drone DRN, and is defined as a two-dimensional grid group surrounding the circumference of the occupied grid group 2701.

The quasi-occupied grid group 2704 corresponds to a region close to the front, back, left and right of the worker WKR, and is defined as a two-dimensional grid group surrounding the circumference of the occupied grid 2702.

Further, the two-dimensional lattice group 2705 corresponding to the sand mountain SND in the environment map image 2700 is defined as an unoccupied lattice group.

As described above, the peripheral monitoring device 150 reflects the information about the drone DRN and the worker WKR where the contact with the excavator 100 is not allowed, but does not reflect the information about the sandy mountain where the contact with the shovel 100 is allowed. To the operator. As a result, the peripheral monitoring device 150 visually recognizes, for example, an environmental map image containing information about the sandy mountain SND, and the operator interrupts the work for confirming the sandy mountain SND even though contact is permitted, and the excavator. It is possible to suppress a situation in which the work efficiency of 100 is lowered.

[6th example of how to generate an environmental map]
Next, a sixth example of a method for generating an environmental map will be described with reference to FIGS. 28 and 29.

<Map generation process>
First, a map generation processing flow corresponding to the environmental map generation method according to this example will be described.

FIG. 28 is a flowchart schematically showing a sixth example of the map generation process by the environment map generation unit 302.

As shown in FIG. 28, steps S602 and S604 are the same as the processes of steps S202 and S204 in FIG. 13, and therefore the description thereof will be omitted.

When the process of step S604 is completed, the controller 30 proceeds to step S606.

In step S606, the environment map generation unit 302 determines whether or not a registered object exists in the specified object. For example, the environment map generation unit 302 applies a known image processing technique based on the image data of the camera 40X, and recognizes the presence / absence of a registered marker attached to the registered object for registration. It may be determined whether the object is a completed object or an unregistered object. Further, for example, the environmental map generation unit 302 uses an RFID (Radio Frequency Identification) reader to read a response signal from an RFID tag mounted on a registered object to read a registered object and an unregistered object. And may be identified.

The environment map generation unit 302 proceeds to step S608 if the registered object exists in the specified object, and proceeds to step S610 if it does not exist.

In step S608, the environment map generation unit 302 reduces (reduces) the occupancy probability of the three-dimensional grid corresponding to the registered object.

When the process of step S608 is completed, the controller 30 proceeds to step S610.

Since steps S610 to S614 are the same as the processes of steps S206 to S210 in FIG. 13, the description thereof will be omitted.

When the process of step S614 is completed, the controller 30 ends the process of the current flowchart.

In this way, the peripheral monitoring device 150 reduces the importance of the information about the registered objects in the environmental map among the objects around the excavator 100, thereby making the importance of the information about the unregistered objects in the environmental map relative to each other. Raise the target.

For example, it is assumed in advance that the registered object exists in the same work site as the excavator 100. Therefore, although it is necessary to pay attention to its existence, the possibility of contact with the excavator 100 is relatively low.

On the other hand, the unregistered object is not supposed to exist in the same work site as the excavator 100 in advance. Therefore, there is a possibility of unexpected operation, and the possibility of contact with the shovel 100 is relatively high.

Therefore, the peripheral monitoring device 150 can call the operator to call attention to the unregistered object by increasing the importance of the information about the unregistered object in the environmental map. Further, the peripheral monitoring device 150 reduces the importance of the information about the registered object in the environmental map, thereby suppressing the decrease in work efficiency due to the operator paying more attention to the registered object than necessary. Can be done.

Instead of lowering the occupancy probability of the three-dimensional lattice of the registered object, the environment map generation unit 302 specifically determines the possibility of contact with the excavator 100 based on the information about the registered object. , The occupancy probability given to the two-dimensional lattice corresponding to the object may be determined. The information about the registered object may include, for example, information about the shape of the object, information about the movement path of the object, and the like.

<Specific example of environmental map>
Subsequently, a specific example of the environmental map generated by the environmental map generation method according to this example will be described.

FIG. 29 is a diagram showing a thirteenth example of an environment map image displayed on the display device 50. Specifically, FIG. 29 is a specific example (environmental map image 2900) of the environmental map image displayed on the display device 50 in the above-mentioned situations of FIGS. 6 and 7.

For example, as shown in FIGS. 6 and 7, as described above, the drone DRN is at a position lower than the height of the excavator 100 when flying at the dotted line position. However, when this drone DRN is a registered object, its movement route (flight route), its shape, and the like are known in advance, and it is highly possible that the operator has already recognized its existence. Therefore, the information about this drone DRN may be of relatively low importance to the operator.

In this case, in this example, the environmental map generation unit 302 can relatively reduce the importance of the information regarding the drone DRN in the environmental map as described above.

Specifically, as shown in FIG. 29, the environment map image 2900 is displayed on the display device 50.

The environmental map image 2900 includes a quasi-occupied grid group 2901 corresponding to the registered object drone DRN.

In this way, the peripheral monitoring device 150 generates and displays the environment map image 2900 that expresses the two-dimensional grid group at the position corresponding to the drone DRN, which is a registered object, as a quasi-occupied grid instead of an occupied grid. Display at 50. As a result, the peripheral monitoring device 150 can suppress a situation in which the operator pays too much attention to the existence of the registered drone DRN, resulting in a decrease in work efficiency.

[Action]
Next, the operation of the peripheral monitoring device 150 according to the present embodiment will be described.

In the present embodiment, the peripheral monitoring device 150 displays a display device 50 that displays information about objects around the excavator 100 to the user (operator) based on the data regarding the peripheral condition of the excavator 100 acquired by the distance sensor 45X. Be prepared. Then, even if the display device 50 includes data indicating that an object exists in the vicinity of the excavator 100 in the data acquired by the distance sensor 45X, the display device 50 may not display information about the object.

For example, as described above, if the drone DRN is flying at a position sufficiently higher than the height of the excavator 100, it is unlikely that the drone DRN will affect the work of the excavator 100. Therefore, the information about this drone DRN may be of relatively low importance to the user. Further, for example, as described above, when the bridge portion of the gantry crane CRN is located at a position sufficiently higher than the height of the excavator 100, when the excavator 100 passes around the gantry crane CRN, the bridge portion is formed. It is unlikely to affect the excavator 100. Therefore, the information about the bridge portion of this gantry crane CRN may be of relatively low importance to the user. Further, for example, as described above, even when the drone DRN is flying at a height position near the height of the excavator 100, when the drone DRN is moving upward, the drone DRN is higher than the height of the excavator 100. There is a high possibility that it will move to a sufficiently high position. Therefore, the information about this drone DRN may be of relatively low importance to the user. Further, for example, as described above, since the sand mountain SND can travel and pass as it is even if it comes into contact with the excavator 100, the information about the sand mountain SND is relatively important to the user. May be low. Therefore, if information about objects around the excavator 100 is uniformly displayed to the user, even if the information is displayed with relatively low importance to the user, the work of the excavator 100 can be performed by confirming the information. There is a possibility that it will be interrupted and the work efficiency will decrease.

On the other hand, in the present embodiment, even if the display device 50 has data indicating that an object exists in the vicinity of the excavator 100 in the data acquired by the distance sensor 45X, the information about the object is information. If the importance is low, the information can be hidden. Therefore, the peripheral monitoring device 150 can more appropriately display information about an object around the excavator 100 to the user. Therefore, the peripheral monitoring device 150 can suppress a decrease in the work efficiency of the excavator 100, for example.

Further, in the present embodiment, the display device 50 may display only the information about the object having a high possibility of coming into contact with the excavator 100 among the objects around the excavator 100 based on the data acquired by the distance sensor 45X. ..

As a result, the display device 50 can selectively display information about an object that is likely to come into contact with the shovel 100 as information that is highly important to the user. Therefore, the peripheral monitoring device 150 can both suppress the decrease in the work efficiency of the excavator 100 and improve the safety of the excavator 100.

Further, in the present embodiment, the display device 50 has a high possibility of coming into contact with the shovel 100 among the objects around the shovel 100 based on the data acquired by the distance sensor 45X, and the contact with the shovel 100 is high. Only information about unacceptable objects may be displayed.

As a result, even if the display device 50 is an object that is likely to come into contact with the shovel 100, if it is an object that does not cause any problem even if it comes into contact with the shovel 100, such as a sand mountain SND, the user can be notified of the object. It is possible not to disclose information about the object. Therefore, the peripheral monitoring device 150 can more appropriately balance the suppression of the decrease in the work efficiency of the excavator 100 and the improvement of the safety of the excavator 100.

Further, in the present embodiment, among the objects around the excavator 100, the display device 50 is an object within the space range that the excavator 100 can occupy with the movement of the excavator 100 (for example, a range equal to or less than the height of the excavator 100). You may want to display only information about. Further, the display device 50 may display only the information about the space range that the shovel 100 can occupy with the movement of the shovel 100 and the objects within a predetermined range adjacent to the space range.

As a result, the peripheral monitoring device 150 can specifically select an object that is likely to come into contact with the shovel 100, and display information about the object on the display device 50.

Further, in the present embodiment, the peripheral monitoring device 150 generates a two-dimensional environmental map corresponding to the plane on which the excavator 100 moves, based on the data of the three-dimensional space around the excavator 100 acquired by the distance sensor 45X. An environment map generation unit 302 is provided. Specifically, the environment map generation unit 302 may generate an environment map including only information about an object that is likely to come into contact with the excavator 100 among the objects around the excavator 100. Then, the display device 50 may display the environment map generated by the environment map generation unit 302.

Thereby, the peripheral monitoring device 150 can provide the user with information about an object that is likely to come into contact with the excavator 100 through the two-dimensional environment map. Therefore, the peripheral monitoring device 150 can allow the user to more intuitively understand the information about the object that is likely to come into contact with the shovel 100 around the shovel 100.

Further, in the present embodiment, the environmental map generation unit 302 ignores the data in the range other than the space range occupied by the shovel 100 due to the movement of the shovel 100 among the data acquired by the distance sensor 45X, and the environmental map. May be generated. Further, the environment map generation unit 302 includes data in a range other than the spatial range that can be occupied by the excavator 100 and a predetermined range adjacent to the spatial range due to the movement of the excavator 100 among the data acquired by the distance sensor 45X. May be ignored and an environmental map may be generated.

As a result, the peripheral monitoring device 150 can specifically generate an environmental map containing only information about an object that is likely to come into contact with the excavator 100.

Further, in the present embodiment, when the object around the shovel 100 is moving in the direction approaching the shovel 100, the environmental map generation unit 302 provides information about the object as an environmental map than when the object is not moving. An environmental map may be generated so that it can be easily included in.

As a result, the peripheral monitoring device 150 may shift to a state in which there is a high possibility of contacting the excavator 100 in the future, such as a drone DRN moving downward in a range higher than the height of the excavator 100. Information about an object can be included in the environmental map. Therefore, the peripheral monitoring device 150 can more appropriately display information about an object around the excavator 100 to the user.

Further, in the present embodiment, when the object around the shovel 100 is a registered object, the environmental map generation unit 302 includes information about the object in the environmental map more than when the object is an unregistered object. An environmental map may be generated so that it is difficult to get rid of.

As a result, the peripheral monitoring device 150 can reduce the importance of the information about the registered object in the environmental map. Therefore, the peripheral monitoring device 150 can suppress a decrease in work efficiency due to, for example, an operator paying more attention to a registered object than necessary.

[Transform / Change]
Although the embodiments have been described in detail above, the present disclosure is not limited to such a specific embodiment, and various modifications and changes can be made within the scope of the gist described in the claims.

For example, in the above embodiment, the peripheral monitoring device 150 is applied to the excavator 100, but may be applied to other working machines. For example, the other work machine may be a mobile crane or the like.

1 Lower traveling body 1C, 1CL, 1CR Crawler 3 Upper swivel body 4 Boom 5 Arm 6 Bucket 10 Cabin 30 Controller 40 Imaging device 40B, 40F, 40L, 40R Camera 45, 45BL, 45BR, 45L, 45R, 45F Distance sensor 50 Display Device (display)
52 Input device 54 Sound output device 60 Hydraulic control valve 70 Communication device 100 Excavator (working machine)
150 Peripheral monitoring device 200 Management device (peripheral monitoring device)
210 Control device 220 Communication device 230 Output device (display unit)
240 Input device 301 Peripheral image generation unit 302 Environmental map generation unit 303 Display processing unit 304 Peripheral monitoring data transmission unit 305 Safety function control unit SYS excavator management system

Claims (8)

It is equipped with a display unit that displays information about objects around the work machine to the user based on the data about the surrounding condition of the work machine acquired by the peripheral monitoring sensor.
Even if the display unit includes data indicating that an object exists in the vicinity of the work machine in the data acquired by the peripheral monitoring sensor, the display unit may not display information about the object.
Peripheral monitoring device. Based on the data acquired by the peripheral monitoring sensor, the display unit displays only information about the objects around the work machine that are likely to come into contact with the work machine.
The peripheral monitoring device according to claim 1. Based on the data acquired by the peripheral monitoring sensor, the display unit is an object that has a high possibility of coming into contact with the work machine among objects around the work machine and is not allowed to come into contact with the work machine. Show only information about,
The peripheral monitoring device according to claim 2. Among the objects around the work machine, the display unit is an object within a space range that can be occupied by the work machine due to the movement of the work machine, or a predetermined range adjacent to the space range and the space range. Display only information about objects within a range,
The peripheral monitoring device according to any one of claims 1 to 3. A map generation unit that generates a two-dimensional environment map corresponding to a plane on which the work machine moves is provided based on the data of the three-dimensional space around the work machine acquired by the peripheral monitoring sensor.
The map generator generates the environmental map containing only information about the objects around the work machine that are likely to come into contact with the work machine.
The display unit displays the environment map generated by the map generation unit.
The peripheral monitoring device according to claim 2 or 3. The map generation unit includes data acquired by the peripheral monitoring sensor in a range other than the space range that the work machine can occupy with the movement of the work machine, or the space range and the space range. The environmental map is generated by ignoring the data in the range other than the adjacent predetermined range.
The peripheral monitoring device according to claim 5. When the object around the work machine is moving in the direction approaching the work machine, the map generation unit is more likely to include information about the object in the environment map than when the object is not moving. To generate the environmental map,
The peripheral monitoring device according to claim 5 or 6. When the object around the work machine is a registered object, the map generation unit makes it difficult for the environment map to include information about the object as compared with the case where the object is an unregistered object. Generate the environmental map,
The peripheral monitoring device according to any one of claims 5 to 7.

Images