JP2022179081A - Remote operation support system and remote operation support device - Google Patents

Abstract

To provide a technique capable of further improving the safety of a work machine when it is remotely operated.SOLUTION: A remote operation support system SYS according to one embodiment of the present disclosure comprises: an imaging device 40 for obtaining data concerning an object in the vicinity of a work machine 100; a communication device 60 for communicating with a remote operation support device 200; a communication device 220 for communicating with the work machine 100; a remote operation device 240A for receiving an input for remotely operating the work machine 100; a controller 30 for detecting an object to be monitored in the vicinity of the work machine 100 on the basis of an output by the imaging device 40; and an output device 230 for notifying a user who uses the remote operation device 240A of the fact that the object to be monitored is detected in a notification range in the vicinity of the work machine 100. If a communication delay between the work machine 100 and the remove operation support device 200 is relatively large, the remote operation support system SYS deforms an outer edge of the notification range more outward than when the communication delay is relatively small.SELECTED DRAWING: Figure 8

Description

The present disclosure relates to a remote operation support system for work machines and the like.

Conventionally, a technology related to remote control of work machines is known (see Patent Document 1, for example).

In Patent Document 1, an imaging device and a distance sensor are mounted on a working machine (excavator), and an object to be monitored such as a person (hereinafter referred to as a "monitoring object") is detected within a predetermined range around the working machine from the outputs of these devices. Then, the operator or the like of the external device that performs the remote operation is notified.

WO2020/218455

However, communication delays can occur between the work machine and the external device. Therefore, if the communication delay becomes relatively large, it will not be possible for the operator of the external device, etc., to actually perform remote control after the imaging device, distance sensor, etc., has detected that the person to be monitored has entered the predetermined range. There is a possibility that the time lag until the notification is made becomes relatively large. As a result, when the operator of the external device or the like is notified, there is a possibility that the person or the like to be monitored and the working machine will be closer to each other. Therefore, there is room for improvement in terms of excavator safety.

Therefore, in view of the above problems, it is an object of the present invention to provide a technique capable of further improving the safety of a work machine when remote control is performed.

To achieve the above objectives, in one embodiment of the present disclosure,
an acquisition device provided on a work machine for acquiring data related to objects around the work machine;
a work machine communication device that is provided in the work machine and communicates with an external device that supports remote control of the work machine;
an external device communication device that is provided in the external device and communicates with the working machine;
a remote control device that is provided in the external device and receives an input for remotely controlling the work machine;
an object detection device provided in the work machine or the external device for detecting a predetermined object around the work machine based on the output of the acquisition device;
a notification device provided in the external device for notifying a user using the remote control device when the predetermined object is detected by the object detection device within a predetermined range with respect to the working machine; , and
when the communication delay between the work machine and the external device is relatively large, changing the outer edge of the predetermined range in a direction away from the work machine more than when the communication delay is relatively small;
A remote operation support system is provided.

Also, in other embodiments of the present disclosure,
a communication device that communicates with the work machine having an acquisition device that acquires data about objects in the vicinity of the work machine;
a remote control device that receives input for remotely controlling the work machine;
a notification device that, when a predetermined object is detected within a predetermined range based on the work machine based on the output of the acquisition device, notifies a user using the remote control device of the fact;
When the communication delay with the work machine is relatively large, the outer edge of the predetermined range is changed in a direction away from the work machine more than when the communication delay is relatively small.
A remote operation support device is provided.

According to the embodiment described above, it is possible to further improve the safety of the work machine when remote control is performed.

1 is a schematic diagram showing an example of a remote operation support system; FIG. It is a side view which shows an example of a shovel. It is a top view which shows an example of a shovel. It is a figure which shows an example of a crawler crane. It is a figure which shows an example of a structure of a remote operation assistance system. It is a figure which shows an example of a structure of a shovel. It is a figure which shows an example of a structure of a crawler crane. It is a flowchart which shows roughly the 1st example of the setting process of the alerting|reporting range by a control apparatus. It is a flowchart which shows roughly the 2nd example of the setting process of the alerting|reporting range by a control apparatus. It is a flowchart which shows roughly the 3rd example of the setting process of the alerting|reporting range by a control apparatus. It is a figure which shows an example of the peripheral image of the excavator displayed on a display apparatus. It is a figure which shows an example of the peripheral image of the crawler crane displayed on a display apparatus. FIG. 10 is a diagram showing another example of a peripheral image of the excavator displayed on the display device; It is a figure which shows the other example of the peripheral image of the crawler crane displayed on a display apparatus.

Embodiments will be described below with reference to the drawings.

[Overview of remote operation support system]
First, an outline of the remote operation support system SYS according to the present embodiment will be described with reference to FIGS. 1 to 4. FIG.

FIG. 1 is a schematic diagram showing an example of a remote operation support system SYS according to this embodiment. 2 to 4 are diagrams showing specific examples of work machine 100 included in remote operation support system SYS. Specifically, FIGS. 2 and 3 are a side view and a top view showing an example of a shovel 100A, and FIG. 4 is a diagram showing an example of a crawler crane 100B. Hereinafter, corresponding components (for example, components having the same function) of the excavator 100A and the crawler crane 100B will be described with the same reference numerals.

As shown in FIG. 1 , remote operation support system SYS includes work machine 100 and remote operation support device 200 .

The working machine 100 included in the remote operation support system SYS may be one or plural. Further, when a plurality of work machines 100 are included in remote operation support system SYS, the plurality of work machines 100 may all be of the same type, or at least some of them may be of different types. For example, the remote operation support system SYS may include a plurality of excavators 100A or crawler cranes 100B, or may include both excavators 100A and crawler cranes 100B.

Further, a plurality of remote operation support devices 200 may be included in the remote operation support system SYS. That is, the plurality of remote operation support devices 200 may distribute and perform the processing related to the remote operation support system SYS. For example, the plurality of remote operation support devices 200 communicate with each other with some of the work machines 100 in charge of all the work machines 100 included in the remote operation support system SYS. A process that targets work machine 100 may be executed.

Remote operation support system SYS supports remote operation of work machine 100 in remote operation support device 200 .

Further, as will be described later, when work machine 100 performs work by fully automatic operation, remote operation support system SYS supports remote monitoring of the work by fully automatic operation of work machine 100 in remote operation support device 200, for example. may In this case, the remote operation support system SYS may support the operation of the work machine 100 by the intervention of the supervisor for the fully automatic operation function in the remote operation support device 200 .

<Outline of working machine>
As shown in FIGS. 2 to 4, the work machine 100 includes a lower traveling body 1, an upper revolving body 3 rotatably mounted on the lower traveling body 1 via a revolving mechanism 2, and attached to the upper revolving body 3. and a cabin 10 on which an operator boards. Below, the front of the work machine 100 (upper revolving body 3) is the attachment to the upper revolving body 3 when the excavator 100A is viewed from directly above along the revolving shaft of the upper revolving body 3 in plan view (top view). Corresponds to the extending direction.

The work machine 100 may be a machine other than the excavator 100A and the crawler crane 100B. For example, the work machine 100 may be a forklift, a lifting magnet with a lifting magnet as an end attachment, a bulldozer, a wheel loader, an asphalt finisher, a forestry machine, or the like.

The work machine 100 can perform various works by operating at least one of the lower traveling body 1, the upper revolving body 3, and the attachment AT.

Moreover, the work machine 100 is equipped with the communication device 60, and can mutually communicate with the remote operation support device 200 through a predetermined communication line NW. As a result, work machine 100 can transmit (upload) various types of information to remote operation support device 200 and receive various signals (for example, information signals and control signals) from remote operation support device 200 . can.

The communication line NW includes, for example, a wide area network (WAN). A wide area network may include, for example, a mobile communication network terminating at a base station. The wide area network may also include, for example, a satellite communication network that uses communication satellites over work machine 100 . The wide area network may also include, for example, the Internet network. Also, the communication line NW may include, for example, a local network (LAN: Local Area Network) such as a facility where the remote operation support device 200 is installed. The local network may be a wireless line, a wired line, or a line containing both. Also, the communication line NW may include, for example, a short-range communication line based on a predetermined wireless communication method such as WiFi or Bluetooth (registered trademark).

Work machine 100 is configured to be remotely operated (remotely operated) from remote operation support device 200 . When work machine 100 is remotely controlled, the interior of cabin 10 may be unmanned.

Specifically, work machine 100 is operated by a user (operator)'s input relating to the actuators of work machine 100 performed by remote operation support device 200 . In this case, the work machine 100 captures an image of the surroundings of the work machine 100 (hereinafter referred to as a “peripheral image”) data to the external device. Then, the peripheral image is displayed on a display device 230A, which is provided in the remote operation support device 200 and will be described later. Various information images (information screens) displayed on output device 50 (display device) in cabin 10 of work machine 100 may also be displayed on display device 230A of remote operation support device 200 in the same manner. As a result, the operator can remotely operate work machine 100 while confirming display contents such as a peripheral image representing the state of the surroundings of work machine 100 and various information images displayed on display device 230A. . The work machine 100 operates the actuators according to the remote operation signal representing the content of the remote operation received from the remote operation support device 200, and controls the lower traveling body 1, the upper revolving body 3, the attachment AT, and the like. A driven element may be driven.

In addition, the work machine 100 operates actuators (for example, hydraulic actuators) in response to operations by an operator riding in the cabin 10 to drive driven elements such as the lower traveling body 1, the upper revolving body 3, and the attachment AT. You can The following description is based on the premise that the operator's operation includes both the operation of the operating device 26 by the operator of the cabin 10 and the remote operation by the outside operator.

In addition, work machine 100 may automatically operate the actuator regardless of the details of the operator's operation. As a result, the work machine 100 has a function of automatically operating at least a part of the driven elements such as the lower traveling body 1, the upper revolving body 3, and the attachment AT, that is, the so-called "automatic driving function" or "machine control (machine control)." Control: MC) function" is realized.

The automatic operation function includes a function to automatically operate a driven element (actuator) other than the driven element (actuator) to be operated according to the operation of the operator, that is, the so-called "semi-automatic operation function" or "operation support type 'MC function' may be included. In addition, the automatic operation function includes a function that automatically operates at least a part of a plurality of driven elements (hydraulic actuators) on the premise that there is no operator's operation, that is, a so-called "fully automatic operation function" or "fully automatic type". 'MC function' may be included. In the working machine 100, the interior of the cabin 10 may be in an unmanned state when the fully automatic operation function is enabled. In addition, the semi-automatic operation function, the fully automatic operation function, and the like may include a mode in which the operation contents of the driven elements (actuators) to be automatically operated are automatically determined according to predetermined rules. In addition, in the semi-automatic operation function, the fully automatic operation function, etc., the work machine 100 autonomously makes various judgments, and according to the judgment results, autonomously operates the driven elements (hydraulic actuators) to be automatically operated. A so-called "autonomous operation function", in which the operation content is determined, may be included.

Work machine 100 is, for example, excavator 100A.

As shown in FIGS. 2 and 3, the excavator 100A includes a lower traveling body 1, an upper revolving body 3 mounted on the lower traveling body 1 so as to be rotatable via a revolving mechanism 2, and attachments for performing various operations. An AT and a cabin 10 are provided.

The undercarriage 1 includes, for example, a pair of left and right crawlers 1C. The lower traveling body 1 causes the excavator 100A to travel by hydraulically driving the respective crawlers 1C by the left traveling hydraulic motor 1ML and the right traveling hydraulic motor 1MR.

The upper revolving structure 3 revolves with respect to the lower traveling structure 1 by hydraulically driving the revolving mechanism 2 with a revolving hydraulic motor 2M.

Attachment AT includes boom 4A, arm 5A, and bucket 6A as driven elements.

The boom 4A is attached to the center of the front part of the upper rotating body 3 so as to be able to be raised. An arm 5A is attached to the tip of the boom 4A so as to be vertically rotatable. possible to be installed.

Bucket 6A is an example of an end attachment. The bucket 6A is used, for example, for excavation work or the like. Further, another end attachment may be attached to the tip of the arm 5A instead of the bucket 6A, depending on the type of work and the like. Other end attachments may be other types of buckets such as, for example, large buckets, slope buckets, dredging buckets, and the like. Other end attachments may also be types of end attachments other than buckets, such as agitators, breakers, grapples, and the like. Further, a spare attachment such as a quick coupling or a tilt rotator may be interposed between the arm 5A and the end attachment.

Also, a hook HK for crane work may be attached to the bucket 6A. A base end of the hook HK is rotatably connected to a bucket pin that connects the arm 5A and the bucket 6A. Thereby, the hook HK is accommodated in the space formed between the two bucket links when work other than crane work (hanging work) such as excavation work is performed.

The boom 4A, arm 5A, and bucket 6A are hydraulically driven by boom cylinders 7A, arm cylinders 8A, and bucket cylinders 9A as hydraulic actuators, respectively.

In the excavator 100A, some or all of the various hydraulic actuators may be replaced with electric actuators. That is, the excavator 100A may be a hybrid excavator or an electric excavator.

The cabin 10 is a cockpit in which an operator boards, and is mounted on the front left side of the upper swing body 3, for example.

Incidentally, when the operator does not operate the excavator 100A, and the work machine 100 operates solely by remote control or fully automatic operation function, the cabin 10 may be omitted. Hereinafter, the same applies to the case of the crawler crane 100B.

Moreover, the work machine 100 is, for example, a crawler crane 100B.

As shown in FIG. 4, the crawler crane 100B includes a lower traveling body 1, an upper revolving body 3 which is rotatably mounted on the lower traveling body 1 via a revolving mechanism 2, an attachment AT, a mast 5B, and a backshoe. A top 6B, a hook HK, a counterweight 9B and a cabin 10 are provided.

The undercarriage 1 includes, for example, a pair of left and right crawlers 1C. The lower traveling body 1 causes the crawler crane 100B to travel by hydraulically driving the respective crawlers 1C by the left traveling hydraulic motor 1ML and the right traveling hydraulic motor 1MR (see FIG. 7).

The upper revolving structure 3 revolves with respect to the lower traveling structure 1 by hydraulically driving the revolving mechanism 2 with a revolving hydraulic motor 2M (see FIG. 7).

An attachment AT (an example of a working device) includes a boom 4B and a main hoisting rope 7B.

The boom 4B is attached to the center of the front portion of the upper swing body 3 so as to be able to rise and fall. A main hoisting rope 7B hangs down from the tip of the boom 4B, and a hook HK is attached to the tip of the main hoisting rope 7B. That is, a hook HK is attached to the tip of the boom 4B via the main hoisting rope 7B.

The main hoisting rope 7B has its base end attached to a main hoisting winch 7Ba attached to the rear surface portion between the base end and the tip of the boom 4B, and its tip is attached to the hook HK. The main hoisting rope 7B is wound up and unwound by a main hoisting winch 7Ba hydraulically driven by a main hoisting hydraulic motor 7M (see FIG. 7), whereby the hook HK can be moved up and down.

The mast 5B is attached slightly behind the base end of the boom 4B of the upper rotating body 3 so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4B. The tip of the mast 5B is connected to the tip of the boom 4B via a pendant rope 5Ba, and the boom hoisting rope 5Bb is reeled in by a boom hoisting winch 5Bc hydraulically driven by a hoisting hydraulic motor 5M (see FIG. 7). The extension raises and lowers the boom 4B via the mast 5B.

Incidentally, in the crawler crane 100B, like the excavator 100A, some or all of the various hydraulic actuators may be replaced with electric actuators.

The backstop 6B is attached to a portion of the upper rotating body 3 behind the base end of the boom 4B so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4B. It is attached to a rear surface portion between the base end and the tip end of the boom 4B so as to be rotatable around a rotation axis parallel to the rotation axis of the boom 4B. The backstop 6B expands and contracts according to the hoisting motion of the boom 4B, and has a function of supporting the boom 4B from behind when the boom 4B is in a substantially upright state, for example.

The hook HK is attached to the tip of the main hoisting rope 7B and used to suspend the load.

The counterweight 9B is provided at the rear end of the upper swing body 3 and has a function of balancing the weight of the boom 4B and the weight of the suspended load SL.

The cabin 10 is attached to the right front end portion of the upper swing body 3, for example. Inside the cabin 10, an operating device 26 (see FIG. 7) for operating a cockpit and various actuators is provided.

<Overview of remote operation support device>
Remote operation support device 200 (an example of an external device) supports remote operation of work machine 100 . In addition, the remote operation support device 200 may support remote monitoring of the work machine 100 that performs work in fully automatic operation.

The remote operation support device 200 may be, for example, an on-premises server or a cloud server installed in a management center or the like outside the work site where the work machine 100 works. Further, the remote operation support device 200 may be, for example, an edge server arranged in a work site where the work machine 100 performs work or in a place relatively close to the work site. Further, the remote operation support device 200 may be a stationary terminal device or a portable terminal device (portable terminal) arranged in a management office or the like in the work site of the working machine 100 . Stationary terminal devices may include, for example, desktop computer terminals. Portable terminal devices may include, for example, smartphones, tablet terminals, laptop computer terminals, and the like.

The remote operation support device 200 has, for example, a communication device 220 (see FIGS. 5 to 7), and communicates with the work machine 100 through the communication line NW as described above. As a result, remote operation support device 200 can receive various information uploaded from work machine 100 and transmit various signals to work machine 100 . Therefore, the user of the remote operation support device 200 can confirm various information about the working machine 100 through the output device 230 (see FIGS. 2 and 3). Further, the remote operation support device 200 can, for example, transmit an information signal to the work machine 100 to provide information necessary for work, or transmit a control signal to control the work machine 100 . Users of the remote operation support device 200 include, for example, the owner of the work machine 100, the manager of the work machine 100, the engineer of the manufacturer of the work machine 100, the operator of the work machine 100, the manager of the work site of the work machine 100, Supervisors, workers, etc. may be included.

For example, the remote operation support device 200 has a remote operation device 240A for inputting remote operation by an operator, and a display device 230A for displaying an image (periphery image) of the periphery of the work machine 100 and the like. A signal input from remote operation device 240A is transmitted from remote operation support device 200 to work machine 100 as a remote operation signal. As a result, the user (operator) of remote operation support device 200 can remotely operate work machine 100 using remote control device 240A while checking the surroundings of work machine 100 on display device 230A. .

For example, a user (monitor) of remote operation support device 200 can monitor the state of work of work machine 100 on display device 230A. Further, for example, the user (monitoring person) of the remote operation support device 200 uses the remote operation device 240A when an abnormality occurs in the work machine 100 or when the operation of the work machine 100 is inappropriate. , an intervention operation for the automatic driving function of the work machine 100 can be performed. Therefore, the observer can use the remote control device 240A to bring the work machine 100 to an emergency stop or remotely control the work machine 100 to perform an appropriate operation.

[Configuration of remote operation support system]
Next, the configuration of the remote operation support system SYS will be described with reference to FIGS. 5 to 7. FIG.

FIG. 5 is a block diagram showing an example of the configuration of the remote operation support system SYS according to this embodiment. FIG. 6 is a diagram showing an example of the configuration of the excavator 100A. Specifically, FIG. 6 is a diagram showing a specific example of the configuration of the excavator 100A corresponding to FIG. FIG. 7 is a diagram showing an example of the configuration of the crawler crane 100B. Specifically, FIG. 7 is a diagram showing a specific example of the configuration of the crawler crane 100B corresponding to FIG.

5 to 7, the path through which mechanical power is transmitted is double lines, the path through which high-pressure hydraulic oil that drives the hydraulic actuator flows is solid lines, the path through which pilot pressure is transmitted is dashed lines, and electrical signals are shown in The paths to be propagated are indicated by dotted lines respectively.

<Configuration of working machine>
The work machine 100 includes a hydraulic drive system for hydraulically driving the driven elements, an operation system for operating the driven elements, a user interface system for exchanging information with the user, a communication system for communication with the outside, and a control system for various controls. and so on.

<< Hydraulic drive system >>
As shown in FIG. 5, the hydraulic drive system of the work machine 100 according to the present embodiment includes driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper revolving body 3, and the attachment AT, as described above. Hydraulic actuators HA for hydraulically driving each are included. Further, the hydraulic drive system of work machine 100 according to the present embodiment includes engine 11 , regulator 13 , main pump 14 , and control valve 17 .

As shown in FIG. 6, when work machine 100 is excavator 100A, hydraulic actuator HA includes travel hydraulic motors 1ML and 1MR, turning hydraulic motor 2M, boom cylinder 7A, arm cylinder 8A, and bucket cylinder 9A. be

As shown in FIG. 7, when the work machine 100 is a crawler crane 100B, the hydraulic actuator HA includes traveling hydraulic motors 1ML and 1MR, a turning hydraulic motor 2M, a hoisting hydraulic motor 5M, a main hoisting hydraulic motor 7M, and the like. is included.

The engine 11 is a prime mover of the work machine 100 and a main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, on the rear portion of the upper revolving body 3 . The engine 11 rotates at a preset target speed under direct or indirect control by a controller 30 to be described later, and drives the main pump 14 and the pilot pump 15 .

In place of or in addition to the engine 11, another prime mover (for example, an electric motor) or the like may be mounted on the work machine 100. FIG.

The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30 . For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter referred to as “tilt angle”) according to a control command from the controller 30 .

The main pump 14 supplies hydraulic fluid to the control valve 17 through a high pressure hydraulic line. The main pump 14 is mounted, for example, on the rear portion of the upper rotating body 3, similar to the engine 11. As shown in FIG. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the regulator 13 adjusts the tilting angle of the swash plate, thereby adjusting the stroke length of the piston and discharging. The flow rate (discharge pressure) is controlled.

The control valve 17 is a hydraulic control device that controls the hydraulic actuator HA according to the content of the operator's operation on the operation device 26 or remote control, or an operation command relating to the automatic operation function output from the controller 30 . The operation command corresponding to the automatic driving function may be generated by the controller 30, or may be generated by another control device (arithmetic device) that performs control related to the automatic driving function. The control valve 17 is mounted, for example, in the central portion of the upper revolving body 3 . As described above, the control valve 17 is connected to the main pump 14 via the high-pressure hydraulic line, and supplies hydraulic oil supplied from the main pump 14 according to an operator's operation or an operation command corresponding to the automatic operation function. , selectively feeding the respective hydraulic actuators. Specifically, the control valve 17 includes a plurality of control valves (also referred to as “direction switching valves”) that control the flow rate and flow direction of hydraulic oil supplied from the main pump 14 to each hydraulic actuator HA.

<<Operating System>>
As shown in FIG. 5 , the operating system of work machine 100 according to the present embodiment includes pilot pump 15 , operating device 26 , hydraulic control valve 31 , shuttle valve 32 , and hydraulic control valve 33 .

Pilot pump 15 supplies pilot pressure to various hydraulic devices via pilot line 25 . The pilot pump 15 is mounted, for example, on the rear portion of the upper revolving body 3 in the same manner as the engine 11 . The pilot pump 15 is, for example, a fixed displacement hydraulic pump, and is driven by the engine 11 as described above.

Incidentally, the pilot pump 15 may be omitted. In this case, relatively high-pressure hydraulic fluid discharged from the main pump 14 is decompressed by a predetermined pressure reducing valve, and then relatively low-pressure hydraulic fluid is supplied as pilot pressure to various hydraulic devices.

The operating device 26 is provided near the cockpit of the cabin 10 and is used by the operator to operate various driven elements. In other words, the operating device 26 is used by the operator to operate the hydraulic actuators HA that drive the respective driven elements. The operating device 26 includes a pedal device and a lever device for operating each driven element (hydraulic actuator HA).

For example, as shown in FIG. 4, the operating device 26 is hydraulically piloted. Specifically, the operating device 26 utilizes the hydraulic oil supplied from the pilot pump 15 through the pilot line 25 and the pilot line 25A branched therefrom, and applies the pilot pressure according to the operation content to the secondary side pilot line. 27A. The pilot line 27A is connected to one inlet port of the shuttle valve 32 and connected to the control valve 17 via the pilot line 27 connected to the outlet port of the shuttle valve 32 . As a result, a pilot pressure can be input to the control valve 17 through the shuttle valve 32 according to the operation details of various driven elements (hydraulic actuators) in the operating device 26 . Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26 by an operator or the like.

Note that the operating device 26 may be of an electric type. In this case, the pilot line 27A, shuttle valve 32 and hydraulic control valve 33 are omitted. Specifically, the operation device 26 outputs an electric signal (hereinafter referred to as “operation signal”) corresponding to the content of the operation, and the operation signal is captured by the controller 30 . Then, the controller 30 outputs to the hydraulic control valve 31 a control command corresponding to the content of the operation signal, that is, a control signal corresponding to the content of the operation on the operating device 26 . As a result, a pilot pressure corresponding to the operation content of the operation device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 drives each hydraulic actuator HA according to the operation content of the operation device 26. be able to. Also, the control valves (direction switching valves) built in the control valve 17 that drive the respective hydraulic actuators may be of the electromagnetic solenoid type. In this case, the operation signal output from the operation device 26 may be directly input to the control valve 17, that is, to the electromagnetic solenoid type control valve. Further, the operation device 26 may be omitted when the work machine 100 is remotely controlled or operates by an automatic operation function.

The hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 . The hydraulic control valve 31 is provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and is configured such that its passage area (that is, the cross-sectional area through which hydraulic oil can flow) can be changed. good. As a result, the hydraulic control valve 31 can output a predetermined pilot pressure to the secondary side pilot line 27B using the hydraulic fluid of the pilot pump 15 supplied through the pilot line 25B. Therefore, as shown in FIG. 4, the hydraulic control valve 31 indirectly controls a predetermined pilot pressure according to the control signal from the controller 30 through the shuttle valve 32 between the pilot lines 27B and 27B. 17. Further, as shown in FIG. 5, the hydraulic control valve 31 can directly apply a predetermined pilot pressure to the control valve 17 according to the control signal from the controller 30 through the pilot line 27B and the pilot line 27. can. Therefore, the controller 30 can cause the control valve 17 to supply the pilot pressure corresponding to the operation content of the electric operation device 26 from the hydraulic control valve 31, and can realize the operation of the work machine 100 based on the operator's operation.

Also, the controller 30 may control the hydraulic control valve 31, for example, to implement an automatic operation function. Specifically, the controller 30 outputs a control signal corresponding to an operation command related to the automatic operation function to the hydraulic control valve 31 regardless of whether or not the operation device 26 is operated. As a result, the controller 30 can cause the hydraulic control valve 31 to supply the control valve 17 with the pilot pressure corresponding to the operation command related to the automatic operation function, thereby realizing the operation of the work machine 100 based on the automatic operation function.

Also, the controller 30 may control the hydraulic control valve 31 to realize remote control of the work machine 100, for example. Specifically, the controller 30 outputs, via the communication device 60 , a control signal corresponding to the details of the remote operation designated by the remote operation signal received from the remote operation support device 200 to the hydraulic control valve 31 . As a result, the controller 30 causes the hydraulic control valve 31 to supply the pilot pressure corresponding to the content of the remote control to the control valve 17, thereby realizing the operation of the work machine 100 based on the operator's remote control.

The shuttle valve 32 has two inlet ports and one outlet port, and outputs to the outlet port the hydraulic fluid having the higher pilot pressure among the pilot pressures input to the two inlet ports. The shuttle valve 32 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 . One of the two inlet ports of the shuttle valve 32 is connected to the pilot line 27A on the secondary side of the operating device 26 (specifically, the above-described lever device and pedal device included in the operating device 26), and the other is connected to the pilot line 27A. It is connected to the pilot line 27B on the secondary side of the hydraulic control valve 31 . The outlet port of shuttle valve 32 is connected through pilot line 27 to the corresponding control valve pilot port of control valve 17 . The corresponding control valve is a control valve that drives the hydraulic actuator that is the object of operation of the above-described lever device or pedal device that is connected to one inlet port of the shuttle valve 32 . Therefore, these shuttle valves 32 correspond to the higher one of the pilot pressure of the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure of the pilot line 27B on the secondary side of the hydraulic control valve 31. It can act on the pilot port of the control valve. That is, the controller 30 causes the hydraulic control valve 31 to output a pilot pressure higher than the pilot pressure on the secondary side of the operation device 26, thereby controlling the corresponding control valve regardless of the operator's operation of the operation device 26. be able to. Therefore, the controller 30 can control the operation of the driven elements (the lower traveling body 1, the upper revolving body 3, and the attachment AT) regardless of the operating state of the operating device 26 by the operator, and can realize the automatic driving function. .

The hydraulic control valve 33 is provided in a pilot line 27A that connects the operating device 26 and the shuttle valve 32 . The hydraulic control valve 33 is configured, for example, so that its flow passage area can be changed. The hydraulic control valve 33 operates according to control signals input from the controller 30 . Thereby, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is operated by the operator. Therefore, even when the operation device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operation device 26 . Further, for example, even when the operating device 26 is being operated, the controller 30 reduces the pilot pressure output from the operating device 26 to be lower than the pilot pressure output from the hydraulic control valve 31. can be done. Therefore, the controller 30 controls the hydraulic control valve 31 and the hydraulic control valve 33 to apply a desired pilot pressure to the pilot port of the control valve in the control valve 17, regardless of the operation content of the operating device 26, for example. can work reliably. Therefore, by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31 , the controller 30 can more appropriately realize the automatic operation function and the remote control function of the work machine 100 .

<<User interface system>>
As shown in FIG. 5 , the user interface system of work machine 100 according to the present embodiment includes operation device 26 , output device 50 , and input device 52 .

The output device 50 outputs various information to a user of the work machine 100 (for example, an operator of the cabin 10, a work vehicle around the work machine 100, etc.).

For example, the output device 50 includes a lighting device, a display device, and the like that output various information in a visual manner. The lighting equipment is, for example, a warning light or the like. The display device is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. The lighting equipment and the display device may be provided inside the cabin 10, for example, and output various kinds of information to an operator or the like inside the cabin 10 in a visual manner. Further, the lighting device and the display device may be provided, for example, on the side surface of the upper revolving structure 3, and may output various information visually to a worker or the like around the work machine 100. FIG.

Also, for example, the output device 50 includes a sound output device that outputs various information in an auditory manner. Sound output devices include, for example, buzzers and speakers. The sound output device may be provided inside the cabin 10, for example, and output various information to an operator or the like inside the cabin 10 in an audible manner.

Also, for example, the output device 50 may include a device that outputs various information in a tactile manner such as vibration of the cockpit.

Input device 52 receives various inputs from the user of work machine 100 , and signals corresponding to the received inputs are captured by controller 30 . The input device 52 is provided, for example, inside the cabin 10 and receives input from an operator or the like inside the cabin 10 . Further, the input device 52 may be provided, for example, on the side surface of the upper revolving body 3 or the like to receive input from a worker or the like around the working machine 100 .

For example, the input device 52 is an operation input device that receives operation input. The operation input device may include a touch panel mounted on the display device, a touch pad installed around the display device, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), and the like. .

Also, for example, the input device 52 may be a voice input device that receives a user's voice input. Audio input devices include, for example, microphones.

Also, for example, the input device 52 may be a gesture input device that receives gesture input from the user. The gesture input device includes, for example, an imaging device that captures an image of a user's gesture.

<< communication system >>
As shown in FIG. 5 , the communication system of work machine 100 according to the present embodiment includes communication device 60 .

The communication device 60 (an example of a work device communication device) is connected to the communication line NW and communicates with a device provided separately from the work machine 100 (for example, the remote operation support device 200). The devices provided separately from work machine 100 may include devices external to work machine 100 as well as portable terminal devices brought into cabin 10 by the user of work machine 100 . The communication device 60 may include, for example, a mobile communication module conforming to standards such as 4G ( ^4th Generation) and 5G ( ^5th Generation). Communication device 60 may also include, for example, a satellite communication module. The communication device 60 may also include, for example, a WiFi communication module, a Bluetooth (registered trademark) communication module, and the like.

<<Control System>>
As shown in FIG. 5 , the control system of work machine 100 according to the present embodiment includes controller 30 . Further, the control system of work machine 100 according to the present embodiment includes operation pressure sensor 29, imaging devices 40 and 42, and acquisition device SX.

A controller 30 (an example of an object detection device) performs various controls related to the work machine 100 . The functions of the controller 30 may be implemented by any hardware, or any combination of hardware and software. For example, the controller 30 includes a CPU (Central Processing Unit), a memory device such as RAM (Random Access Memory), a non-volatile auxiliary storage device such as ROM (Read Only Memory), and various input/output interface devices. It is composed mainly of computers.

The controller 30 includes an operation control unit 301, an image transmission unit 302, an object detection unit 303, and a safety control unit 304 as functional units. The functions of the operation control unit 301, the image transmission unit 302, the object detection unit 303, and the safety control unit 304 are implemented by, for example, loading a program installed in an auxiliary storage device into a memory device and executing it on the CPU. Realized.

Note that part of the functions of the controller 30 may be implemented by another controller (control device). In other words, the functions of the controller 30 may be distributed and implemented by a plurality of controllers.

The operation pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot type operation device 26, that is, the pilot pressure corresponding to the operation state of each driven element (hydraulic actuator) in the operation device 26. To detect. A pilot pressure detection signal corresponding to the operation state of each driven element (hydraulic actuator HA) in the operation device 26 by the operation pressure sensor 29 is taken into the controller 30 .

Incidentally, if the operating device 26 is of an electric type, the operating pressure sensor 29 is omitted. This is because the controller 30 can grasp the operating state of each driven element through the operating device 26 based on the operating signal received from the operating device 26 .

The imaging device 40 (an example of an acquisition device) acquires an image for interpolating a blind spot or a location that is difficult for the operator to visually recognize around the excavator 100A. The output of the imaging device 40 is taken into the controller 30 .

The imaging device 40 includes, for example, a monocular camera, a stereo camera, a depth camera, and the like. In addition, based on the captured image, the imaging device 40 acquires three-dimensional data (for example, point cloud data or surface data) representing the position and outline of objects around the work machine 100 within a predetermined imaging range (angle of view). You may

For example, as shown in FIGS. 2 and 3, the imaging device 40 of the excavator 100A includes a camera 40B for imaging the rear side of the upper revolving body 3, a camera 40L for imaging the left side of the upper revolving body 3, and a camera 40L for imaging the left side of the upper revolving body 3. includes a camera 40R for imaging the right side of the . As a result, the operator can visually recognize peripheral images, such as images captured by the imaging device 40 and processed images generated based on the captured images, through the output device 50 and the display device 230A, and view left and right directions of the upper rotating body 3. , and the state behind can be confirmed.

The two-dot chain line in FIG. 3 represents the angle of view (imaging range) of the cameras 40B, 40L, and 40R when viewed from above.

Further, for example, as shown in FIG. 4, the imaging device 40 of the crawler crane 100B is attached to, for example, the boom 4B, and captures an image of the surroundings of the hook HK and the load hung on the hook HK as viewed from above. get. As a result, the operator can visually recognize the image captured by the image capturing device 40 and the peripheral image such as the processed image generated based on the captured image through the output device 50 (display device) and the display device 230A, and the operator can view the peripheral image such as the processed image generated based on the captured image. It is possible to check the surroundings of the load from a top view.

4 may be provided in the excavator 100A, and the imaging device 40 (cameras 40B, 40L, 40R) in the form of FIGS. 2 and 3 may be provided in the crawler crane 100B. good too.

The imaging device 42 acquires an image of the surroundings of the work machine 100 so that the operator of remote control and the supervisor of remote monitoring can confirm the state of work by the attachment AT. The output of the imaging device 42 is taken into the controller 30 .

The imaging device 42 includes, for example, a monocular camera, a stereo camera, a depth camera, and the like. In addition, based on the captured image, the imaging device 42 acquires three-dimensional data (for example, point cloud data or surface data) representing the position and outline of objects around the work machine 100 within a predetermined imaging range (angle of view). You may

For example, as shown in FIGS. 2 and 3, the imaging device 42 of the excavator 100A is provided, for example, in the front part of the upper surface of the cabin 10 or the like, in the front part of the upper rotating body 3, and is configured to capture the tip of the attachment AT (the bucket 6A). An image of the state in front of the upper rotating body 3 is captured. Accordingly, the operator can confirm the operation of the attachment AT including the bucket 6A by visually recognizing peripheral images such as the image captured by the imaging device 42 and the processed image generated based on the imaging device 42 through the display device 230A. while the excavator 100A can be operated.

Further, for example, as shown in FIG. 4, the imaging device 42 of the crawler crane 100B is provided in the front part of the upper revolving body 3, such as the front part of the upper surface of the cabin 10, for example. Take an image of the front. Specifically, the image capturing device 42 includes a camera 42A that captures an image of a space in a relatively high height direction away from the ground obliquely forward of the upper revolving body 3, and a space near the ground obliquely forward and downward of the upper revolving body 3. and a camera 42B that captures a relatively low elevation range of . As a result, the operator can check the movement of the hook HK by viewing the image captured by the image capturing device 42 and the peripheral image such as the processed image generated based on the captured image through the display device 230A. can be operated.

The acquisition device SX acquires information about the state of the excavator 100A, the state of the surroundings of the excavator 100A, and the like. The output of acquisition device SX is taken into controller 30 .

As shown in FIG. 6, when the work machine 100 is an excavator 100A, the acquisition device SX includes a boom angle sensor S1, an arm angle sensor S2, a bucket angle sensor S3, a machine body attitude sensor S4, and a turning angle sensor S5. including.

Moreover, as shown in FIG. 7, when the work machine 100 is a crawler crane 100B, the acquisition device SX includes a machine body attitude sensor S4 and a turning angle sensor S5.

The boom angle sensor S1 acquires detection information regarding the attitude angle (hereinafter referred to as "boom angle") of the boom 4A with respect to a predetermined reference (for example, a horizontal plane or one of the two ends of the movable angle range of the boom 4A). The boom angle sensor S1 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a hexaaxial sensor, an IMU (Inertial Measurement Unit), and the like. Also, the boom angle sensor S1 may include a cylinder sensor capable of detecting the telescopic position of the boom cylinder 7A.

The arm angle sensor S2 detects the posture angle of the arm 5A (hereinafter referred to as "arm angle ”). Arm angle sensor S2 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a hexaaxial sensor, an IMU, or the like. Also, the arm angle sensor S2 may include a cylinder sensor capable of detecting the extension/retraction position of the arm cylinder 8A.

The bucket angle sensor S3 detects the posture angle of the bucket 6A (hereinafter referred to as "bucket angle ”). Bucket angle sensor S3 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a hexaaxial sensor, an IMU, and the like. Also, the bucket angle sensor S3 may include a cylinder sensor capable of detecting the expansion/contraction position of the bucket cylinder 9A.

The body posture sensor S<b>4 acquires detection information regarding the posture state of the body of the working machine 100 including the lower traveling body 1 and the upper swing body 3 . The machine body attitude sensor S4 is mounted on the upper revolving structure 3, for example, and detects information related to the tilt angle of the upper revolving structure 3 with respect to the horizontal plane and the attitude angle around the revolving axis (that is, the orientation of the upper revolving structure 3 with respect to the ground). to get The airframe attitude sensor S4 may include, for example, an acceleration sensor (tilt sensor), an angular velocity sensor, a hexaaxial sensor, an IMU, and the like.

The turning angle sensor S5 acquires detection information regarding the turning angle of the upper turning body 3 with respect to the lower traveling body 1 (that is, the orientation of the upper turning body 3). The turning angle sensor S5 includes, for example, a potentiometer, rotary encoder, resolver, and the like.

Also, for example, the acquisition device SX may include a positioning device capable of positioning the absolute position of its own machine. The positioning device is, for example, a GNSS (Global Navigation Satellite System) sensor. As a result, the estimation accuracy of the posture state of work machine 100 can be improved.

Further, for example, the acquisition device SX may include a distance sensor (an example of an acquisition device) that detects the distance to objects around work machine 100 . Distance sensors include, for example, LIDAR (Light Detecting and Ranging), millimeter wave radar, ultrasonic sensors, infrared sensors, distance image sensors, and the like. Accordingly, the controller 30 (object detection unit 303) can detect an object around the work machine 100 by using the output of the distance sensor instead of or in addition to the output of the imaging devices 40 and 42. .

The sensors S1 to S5 of the excavator 100A and the sensors S4 and S5 of the crawler crane 100B may be omitted. For example, information about the surroundings of the excavator 100A acquired by the imaging devices 40 and 42, distance sensors described later, etc. includes information about the positions and shapes of surrounding objects and attachments viewed from the machine body (upper revolving body 3). may be In this case, for example, the controller 30 can estimate the attitude state of the attachment AT and the machine body (upper rotating body 3) from the information depending on the accuracy required.

The operation control unit 301 controls the operation of the hydraulic actuator of the working machine 100 with the hydraulic control valve 31 as the control target.

The operation control unit 301 controls the operation of the hydraulic actuator HA (driven element) of the work machine 100 based on the operation of the electric operation device 26, for example, with the hydraulic control valve 31 as a control target. Specifically, the operation control unit 301 selectively controls the hydraulic control valves 31 corresponding to each of the plurality of hydraulic actuators HA (driven elements) according to the content of the operation signal output from the operation device 26. You can

Further, the operation control unit 301 performs control related to remote operation of the hydraulic actuator HA (driven element) of the work machine 100, for example, with the hydraulic control valve 31 as a control target. That is, the operation of the hydraulic actuator HA (driven element) of the work machine 100 includes remote operation of the hydraulic actuator HA from outside the work machine 100 (remote operation support device 200). Specifically, the operation control unit 301 performs hydraulic control corresponding to each of the plurality of hydraulic actuators HA (driven elements) according to the content of the remote operation signal received from the remote operation support device 200 through the communication device 60. Valve 31 may be selectively controlled.

Further, the operation control unit 301 controls the automatic operation function of the working machine 100, for example, with the hydraulic control valve 31 as a control target. Specifically, the operation of the hydraulic actuator HA of the work machine 100 may include an operation command of the hydraulic actuator HA of the work machine 100 that is output based on the automatic operation function. Specifically, the operation control unit 301 may selectively control the hydraulic control valves 31 corresponding to each of the plurality of hydraulic actuators HA (driven elements) according to the content of the operation command.

The image transmission unit 302 transmits the captured image of the imaging device 40 or the peripheral image such as the processed image generated based on the captured image of the imaging device 40 to the remote operation support device 200 via the communication device 60 .

For example, the image transmission unit 302 of the excavator 100A generates a processed image (composite image) obtained by synthesizing images captured by a plurality of cameras (at least two of the cameras 40B, 40L, and 40R) based on the images captured by the imaging device 40. may be generated and transmitted to the remote operation support device 200 .

Specifically, the image transmission unit 302 performs known viewpoint conversion processing, composition processing, and the like based on the images captured by the cameras 40B, 40L, and 40R, thereby generating a composite image (viewpoint conversion image) viewed from a virtual viewpoint. It may be generated and transmitted to the remote operation support device 200 .

Also, for example, the image transmission unit 302 of the crawler crane 100</b>B may generate a viewpoint conversion image based on the captured image of the imaging device 40 and transmit it to the remote operation support device 200 .

Similarly, the image transmission unit 302 transmits the captured image of the imaging device 42 or the peripheral image such as the processed image generated by the controller 30 based on the captured image of the imaging device 42 to the remote operation support device 200 through the communication device 60. Send.

For example, the image transmission unit 302 of the excavator 100</b>A may generate a viewpoint-converted image based on the captured image of the imaging device 42 and transmit it to the remote operation support device 200 . In addition, the image transmission unit 302 of the excavator 100A performs known viewpoint conversion processing, synthesis processing, and the like on the captured images of the imaging devices 40 and 42, so that the surroundings of the excavator 100A can be viewed from a virtual viewpoint in 360 degrees in the horizontal direction. A combined image may be generated and transmitted to the remote operation support device 200 .

The object detection unit 303 detects a predetermined object to be monitored (hereinafter referred to as “monitoring object”) around the work machine 100 based on the output of the imaging device 40 .

Objects to be monitored may include people such as workers working around work machine 100 and supervisors at work sites. Observed objects may also include any objects (obstacles) other than people at the work site. Obstacles other than people at the work site include, for example, specific terrain such as holes, ditches, and mountains of earth and sand, road cones, fences, utility poles, temporary materials, temporary offices at the work site, etc. non-moving) obstacles may be included. Obstacles other than people at the work site may include, for example, movable obstacles such as other work machines and work vehicles.

Based on the output of the imaging device 40, that is, the captured image captured by the imaging device 40, the object detection unit 303 detects a predetermined monitoring area around the working machine 100 (upper rotating body 3) (hereinafter, for convenience, "monitoring area"). area”), the object to be monitored is detected. The following description of object detection unit 303 will be made under the assumption that work machine 100 is positioned on a horizontal plane. Specifically, the object detection unit 303 of the excavator 100</b>A may detect the monitored object in the monitoring area covering the rear, left, and right sides of the upper swing body 3 . Also, the object detection unit 303 of the crawler crane 100B may detect a monitored object in a monitoring area around the load hung on the hook HK.

When the above-described distance sensor is mounted on the upper rotating body 3, the object detection unit 303 detects the monitored object based on the output of the distance sensor instead of or in addition to the output of the imaging device 40. good. Further, the object detection unit 303 may detect an object around the upper rotating body 3 based on the output of the imaging device 42 in addition to the output of the imaging device 40 . That is, the object detection unit 303 may detect the monitored object in the monitoring area including the imaging range of the imaging device 42 based on the outputs of the imaging devices 40 and 42 . Further, when work machine 100 is exclusively used by remote control, the function of object detection unit 303 may be transferred to remote operation support device 200 (for example, control device 210 (an example of an object detection device)). . In this case, the outputs of the imaging device 40 and the distance sensor are transmitted to the remote operation support device 200 through the communication device 60 .

The object detection unit 303 detects, for example, the horizontal direction as viewed from the work machine 100 (hereinafter simply referred to as the “horizontal direction”), that is, the plane on which the work machine 100 is working (the lower traveling body 1 is in contact with the ground) (hereinafter , conveniently the "work plane") within a monitored area extending in a direction along which the monitored object is sensed. Specifically, object detection unit 303 may detect a monitored object within a monitored area where distance D from a predetermined portion of work machine 100 is within threshold value Dth1. At this time, the threshold value Dth1 may be constant regardless of the direction viewed from the predetermined portion of work machine 100, or may vary depending on the direction viewed from the predetermined portion of work machine 100. FIG. For example, the object detection unit 303 of the excavator 100A detects a monitored object within a monitored area where the horizontal distance D from the upper swing structure 3 is within a threshold value Dth1 (for example, 10 meters). Further, the object detection unit 303 of the crawler crane 100B detects the monitored object within the monitored area where the horizontal distance from the hook HK (or the suspended load) is within the threshold value Dth1.

For example, the object detection unit 303 recognizes the monitored object in the captured image by arbitrarily applying a machine learning-based classifier including various known image processing methods and AI (Artificial Intelligence). . Further, when the object to be monitored is a person such as a worker, the object detection unit 303 may specify whether the recognized person corresponds to one of a plurality of registered workers registered in advance. .

Further, the object detection unit 303 applies various known techniques to determine the position of the recognized monitored object (for example, a person's foot position) (hereafter referred to as , “actual location”) may be determined (estimated).

For example, the object detection unit 303 of the excavator 100A detects the size of the recognized monitored object on the captured image (for example, the size in the height direction on the captured image). A position (hereinafter "horizontal position") may be estimated. This is because there is a correlation that the size of the recognized monitored object on the captured image decreases as the monitored object moves away from the excavator 100A (upper revolving body 3). Specifically, since the monitored object has an assumed size range (for example, an assumed range of human height), from the excavator 100A of the monitored object included in the assumed size range, A correlation between the viewed horizontal position and the size on the captured image can be defined in advance. Therefore, the object detection unit 303 stores, for example, a map, which is stored in advance in an internal memory such as an auxiliary storage device of the controller 30, representing the correlation between the size of the monitored object on the captured image and the horizontal position viewed from the excavator 100A. Based on the conversion formula or the like, the actual position (horizontal position from the upper rotating body 3) of the recognized monitored object can be estimated.

Further, for example, on the premise that the monitored object exists on the same plane as the work machine 100 (specifically, the undercarriage 1), the object detection unit 303 performs projective transformation (homogeneous transformation) of the captured image onto the plane. The actual position (for example, the position of the foot) may be estimated by using a graph or the like. In this case, a certain portion (for example, a certain pixel) that constitutes the captured image is associated with a certain position on the same plane as work machine 100 .

Further, the function of the object detection unit 303 may be switched between ON (enabled) and OFF (disabled) according to a predetermined operation by an operator or the like on the input device 52 .

The safety control unit 304 performs control related to functional safety of the working machine 100 . Specifically, the safety control unit 304 may activate the safety function when the object detection unit 303 detects the object to be monitored.

The safety function includes, for example, a notification function that outputs an alarm or the like to at least one of the inside of the cabin 10, the outside of the cabin 10, and the remote operator or manager of the working machine 100, and notifies the detection of the monitored object. may be included. As a result, an operator inside the cabin 10, a worker around the working machine 100, an operator or a supervisor who remotely operates or remotely monitors the working machine 100, etc., can monitor within a predetermined range around the working machine 100. It can call attention to the presence of the object. Hereinafter, the notification function to the inside (operator, etc.) of the cabin 10 is "internal notification function", the notification function to the outside (workers, etc.) of the work machine 100 is "external notification function", and the remote operation of the work machine 100 A notification function for an operator who performs remote monitoring or an observer is sometimes referred to as a "remote notification function" for distinction.

In addition, the safety function may include, for example, an operation restriction function that restricts the operation of the work machine 100 in response to an operation command corresponding to the operation of the operating device 26, remote operation, or automatic operation function. As a result, the operation of the work machine 100 can be forcibly restricted, and the possibility of the work machine 100 or the suspended load approaching or contacting surrounding objects can be reduced. The motion limiting function may include a motion deceleration function that slows down the motion speed of work machine 100 in response to an operation command corresponding to the operation of operation device 26, remote control, or automatic operation function. In addition, the operation restriction function includes an operation stop function for stopping the operation of the working machine 100 and maintaining the stopped state regardless of the operation command corresponding to the operation of the operation device 26, the remote operation, or the automatic operation function. may

The safety control unit 304 activates the notification function when, for example, the object detection unit 303 detects a monitoring target within a predetermined range (hereinafter referred to as “reporting range”) included in the monitoring area. The notification range is set to the same range as the monitoring area, or to a range in which the outer edge of the monitoring area is relatively closer to the upper rotating body 3 and the hook HK (or the suspended load) than the monitoring area. Specifically, safety control unit 304 may activate the notification function when a monitored object is detected within a notification range in which distance D from a predetermined portion of work machine 100 is within threshold value Dth2 (≦Dth1). . At this time, threshold value Dth2 may be constant regardless of the direction viewed from a predetermined portion of work machine 100, or may vary depending on the direction viewed from a predetermined portion of work machine 100. FIG.

The safety control unit 304, for example, controls the output device 50 (sound output device) to perform an internal notification function and an external notification function by sound (that is, auditory method) for at least one of the inside and outside of the cabin 10. activate. At this time, the safety control unit 304 may vary the pitch, sound pressure, tone color of the sound to be output, the sounding period when the sound is sounded periodically, the content of the sound, etc., according to various conditions. .

Safety control unit 304 also activates an internal notification function, for example, by visual means. Specifically, the safety control unit 304 may control the output device 50 (display device) inside the cabin 10 to display an image indicating that the monitored object is detected on the display device. In addition, the safety control unit 304 may emphasize the monitored object appearing in the peripheral image displayed on the display device inside the cabin 10 or the position on the peripheral image corresponding to the detected monitored object. More specifically, the safety control unit 304 superimposes a frame surrounding the detected monitored object on the peripheral image displayed on the display device, or superimposes a frame around the detected monitored object on the actual position of the detected monitored object. A marker may be superimposed and displayed at the corresponding position on the peripheral image. As a result, the output device 50 (display device) can realize a visual notification function for the operator. Further, the safety control unit 304 may use a warning light, a lighting device, or the like inside the cabin 10 to notify an operator or the like inside the cabin 10 that a monitored object has been detected.

In addition, the safety control unit 304 controls the output device 50 (for example, a lighting device such as a headlight or a display device) provided on the side surface of the house portion of the upper rotating body 3, for example, thereby providing a visual method. You may operate the external notification function by. The safety control unit 304 may also activate the internal notification function in a tactile manner, for example, by controlling a vibration generator that vibrates the cockpit on which the operator sits. As a result, the controller 30 makes the operator, workers and supervisors around the work machine 100 aware of the existence of the monitored object (for example, a person such as a worker) around the work machine 100. can be done. Therefore, the controller 30 can prompt the operator to check the safety situation around the work machine 100, and can prompt the worker or the like in the monitoring area to evacuate from the monitoring area.

Further, the safety control unit 304 may activate the remote notification function by transmitting a command signal indicating activation of the notification function to the remote operation support device 200 via the communication device 60, for example. In this case, when remote operation support device 200 (control device 210) receives a command signal from work machine 100 via communication device 220, it may output an alarm visually or audibly through output device 230. . As a result, an operator or an observer who performs remote operation or remote monitoring of work machine 100 through remote operation support device 200 can recognize that a monitored object has entered within the notification range around work machine 100 .

Note that the remote notification function of the safety control unit 304 may be transferred to the remote operation support device 200 (for example, the notification control unit 2103 described later). In this case, remote operation support device 200 receives information about the detection status of the monitored object by object detection unit 303 from work machine 100, and determines whether or not the monitored object enters the reporting range based on the received information. , activates the external notification function when the object to be monitored exists within the notification range.

In addition, the safety control unit 304 may change the notification mode (that is, the notification method) according to the positional relationship between the monitored object detected within the notification range and the working machine 100 .

For example, the safety control unit 304 determines whether the monitored object detected within the notification range by the object detection unit 303 is relative to a predetermined portion of the work machine 100 such as the upper revolving body 3 and the hook HK (or suspended load). If it exists at a distant position, a relatively low-urgency warning (hereinafter referred to as a "caution-level warning") that calls attention to the monitored object may be output. Hereinafter, for the sake of convenience, the range corresponding to the caution level warning, that is, the range in which the distance to the predetermined parts of the work machine 100 such as the upper revolving body 3 and the hook HK (or the suspended load) is relatively long in the notification range. may be referred to as the "warning range". On the other hand, the safety control unit 304 determines whether the monitored object detected within the notification range by the object detection unit 303 is relative to a predetermined portion of the work machine 100 such as the upper revolving body 3 and the hook HK (or suspended load). If the monitored object exists at a close position, a relatively high-urgency alarm (hereinafter referred to as "warning level alarm") is output to notify that the monitored object is approaching a predetermined portion of the working machine 100 and the degree of danger is increasing. you can Hereinafter, the range corresponding to the alert level warning will be referred to as the "warning It may be referred to as "notification range".

In this case, the safety control unit 304 differentiates the pitch, sound pressure, timbre, sounding period, etc. of the sound output from the output device (sound output device) between the caution level warning and the caution level warning. you can In addition, the safety control unit 304 displays an image indicating that a monitored object has been detected displayed on the output device 50 (display device), or an image displayed on the display device between the caution level warning and the caution level warning. The color, shape, size, presence/absence of blinking, blinking cycle, etc. of the monitored object on the surrounding image or the image (for example, frame, marker, etc.) that emphasizes the position of the monitored object may be varied. As a result, the controller 30 informs the operator or the like of the degree of urgency, in other words, the level of urgency of the work machine 100 as the object to be monitored, based on the difference between the notification sound (warning sound) output from the sound output device and the notification image displayed on the display device. It is possible to grasp the degree of proximity to a predetermined site.

The safety control unit 304 may stop the notification function when the monitored object detected by the object detection unit 303 is no longer detected within the monitoring area after the operation of the notification function is started. Further, the safety control unit 304 may stop the notification function when a predetermined operation for canceling the operation of the notification function is received through the input device 52 after the activation of the notification function is started.

Further, the safety control unit 304 activates the operation restriction function, for example, when the object detection unit 303 detects a monitored object within a predetermined range (hereinafter referred to as “operation restriction range”) included in the monitoring area. The motion restriction range is set to a range in which the outer edge is relatively closer to a predetermined portion of the work machine 100 such as the upper revolving body 3 and the hook HK (or the suspended load) than the notification range described above. As a result, for example, when the monitored object enters the notification range from the outside, the safety control unit 304 first activates the notification function, and then when the monitored object enters the inner operation restriction range, the operation restriction function is activated. can be activated. Therefore, the controller 30 can activate the notification function and the operation restriction function step by step in accordance with the inward movement of the monitored object within the monitored area. Specifically, safety control unit 304 may activate the notification function when a monitored object is detected within a notification range in which distance D from a predetermined portion of work machine 100 is within threshold value Dth3 (<Dth2). . At this time, threshold value Dth3 may be constant regardless of the direction viewed from a predetermined portion of work machine 100, or may vary depending on the direction viewed from a predetermined portion of work machine 100. FIG.

In addition, the operation limitation range includes an operation deceleration range in which the operating speed of the work machine 100 in response to an operation command corresponding to the operation of the operation device 26, remote operation, and automatic operation function is slower than normal, It includes at least one of an operation stop range in which the operation of work machine 100 is stopped and the stop state is maintained regardless of operation commands corresponding to operation and automatic operation functions. For example, when both the motion deceleration range and the motion stop range are included in the motion restriction range, the motion stop range is the predetermined range of the work machine 100 such as the upper revolving body 3 and the hook HK (or the suspended load) within the motion restriction range. It is a range close to the site of The motion deceleration range is a range set outside the motion stop range in the motion limit range.

Safety control unit 304 operates an operation restriction function that restricts the operation of work machine 100 by controlling hydraulic control valve 31 . In this case, the safety control unit 304 may limit the operation of all driven elements (that is, the corresponding hydraulic actuators), or may limit the operation of some driven elements (hydraulic actuators). . As a result, controller 30 can decelerate or stop the operation of work machine 100 when an object to be monitored exists around work machine 100 . Therefore, the controller 30 can suppress the occurrence of contact between the monitored objects around the working machine 100 and the working machine 100 or the suspended load. Further, the safety control unit 304 may operate an operation limiting function (operation stopping function) by controlling an electromagnetic switching valve (not shown) of the pilot line 25 to shut off the pilot line 25 .

Further, the safety control unit 304 may stop the operation restriction function when the monitored object detected by the object detection unit 303 is no longer detected after the activation of the operation restriction function. Further, the safety control unit 304 may stop the operation restriction function when a predetermined operation to cancel the operation of the operation restriction function is received through the input device 52 after the operation of the operation restriction function is started. The operation for deactivation of the notification function and the operation for deactivation of the motion restriction function on the input device 52 may be the same or different.

Further, the function of the safety control unit 304 may be switched between ON (enabled) and OFF (disabled) according to a predetermined operation of the input device 52 by an operator or the like.

<Configuration of remote operation support device>
As shown in FIG. 5 , remote operation support device 200 includes control device 210 , communication device 220 , output device 230 , and input device 240 .

The control device 210 performs various controls related to the remote operation support device 200 . The functions of the control device 210 are realized by arbitrary hardware, or a combination of arbitrary hardware and software. The control device 210 is mainly composed of a computer including, for example, a CPU, a memory device such as a RAM, a non-volatile auxiliary storage device such as a ROM, and various input/output interface devices.

The control device 210 includes a remote operation support unit 2101, a display control unit 2102, a notification control unit 2103, a communication status monitoring unit 2104, and an operation status monitoring unit 2105 as functional units. The functions of the remote operation support unit 2101, the display control unit 2102, the notification control unit 2103, the communication status monitoring unit 2104, and the operation status monitoring unit 2105, for example, load a program installed in the auxiliary storage device into the memory device and execute it on the CPU. This is achieved by executing

A communication device 220 (an example of a communication device for an external device) is connected to the communication line NW and communicates with the outside of the remote operation support device 200 (for example, the work machine 100).

The output device 230 (an example of a notification device) outputs various information to the user of the remote operation support device 200 . The output device 230 includes a display device 230A and a sound output device 230B.

Display device 230A, under the control of control device 210, outputs various information about remote operation support device 200 to the user of remote operation support device 200 in a visual manner. Specifically, display device 230A displays an information image regarding remote operation support device 200 . The display device 230A includes, for example, a liquid crystal display, an organic EL display, or the like.

Note that the output device 230 may include a lighting device that outputs various information to the user of the remote operation support device 200 in a visual manner. Illumination devices include, for example, warning lamps and the like.

The sound output device 230B includes a sound output device that outputs various information to the user of the remote operation support device 200 in an auditory manner. Sound output devices include, for example, buzzers and speakers.

The input device 240 receives input from the user of the remote operation support device 200 and outputs a signal representing the content of the input (for example, operation input, voice input, gesture input, etc.). A signal representing the content of the input is taken into the controller 210 . A user of remote operation support device 200 includes a manager of remote operation support device 200, an operator of work machine 100, a supervisor of work machine 100, and the like.

The input device 240 includes, for example, a remote control device 240A.

Remote control device 240A is used for remote control of work machine 100 by an operator and remote control of work machine 100 by intervention of an automatic operation function by a supervisor. The remote control device 240A receives inputs for remote control of a plurality of actuators (driven elements) from an operator or an observer, and outputs signals representing the content of the inputs to the control device 210 .

A remote operation support unit 2101 performs control related to remote operation of the work machine 100 . Specifically, remote operation support unit 2101 acquires an input signal related to the remote operation of work machine 100 received by remote operation device 240A, and uses communication device 220 to generate a remote control signal representing the details of the remote operation of work machine 100 . An operation signal may be transmitted to work machine 100 .

The display control unit 2102 controls the display device 230A to display various information images on the display device 230A.

The display control unit 2102 causes the display device 230A to display, for example, an image representing the surroundings of the work machine 100 uploaded from the work machine 100 (surrounding image).

Specifically, the display control unit 2102 causes the display device 230A to display a peripheral image corresponding to the imaging range of the imaging device 42 . As a result, the user (operator) of the remote operation support device 200 can remotely operate the work machine 100 while confirming the movement of the attachment AT by viewing the peripheral image corresponding to the imaging range of the imaging device 42. can be done. Further, the user (monitoring person) of the remote operation support device 200 monitors the work status of the work machine 100 while confirming the movement of the attachment AT by viewing the peripheral image corresponding to the imaging range of the imaging device 42. be able to.

Further, the display control unit 2102 causes the display device 230A to display a peripheral image corresponding to the imaging range of the imaging device 40 . As a result, the user of the remote operation support device 200 visually recognizes the peripheral image corresponding to the imaging range of the imaging device 40, thereby making it difficult to confirm only the peripheral image corresponding to the imaging range of the imaging device 42 of the work machine 100. You can check the surrounding situation. Therefore, remote operation support device 200 can improve the safety of work machine 100 and the work site where work machine 100 is used.

Note that the display control unit 2102 may itself generate a processed image as a peripheral image based on the captured image of the imaging device 40 uploaded from the work machine 100, and display it on the display device 230A.

Notification control unit 2103 notifies an operator or a supervisor of the surroundings of work machine 100 through output device 230 when a command signal indicating activation of the notification function (remote notification function) is received through communication device 220 . Notifies that a surveillance object has been detected within the range.

The notification control unit 2103, for example, controls the sound output device 230B to audibly notify the operator or the monitor that a monitored object has been detected in the notification range around the work machine 100. FIG. At this time, the notification control unit 2103 may vary the pitch, sound pressure, timbre of the sound to be output, the sounding cycle when the sound is periodically sounded, the content of the sound, etc., according to various conditions. .

In addition, the notification control unit 2103 notifies the operator or the monitor that a monitored object has been detected in the notification range around the work machine 100 by, for example, a visual method. Specifically, the notification control unit 2103 may control the display device 230A through the display control unit 2102 to display an image indicating that the monitored object is detected on the display device 230A. In addition, the notification control unit 2103 may emphasize, through the display control unit 2102, the monitored object appearing in the peripheral image displayed on the display device 230A or the position on the peripheral image corresponding to the detected monitored object. . More specifically, the notification control unit 2103 causes the display control unit 2102 to superimpose and display a frame surrounding the detected monitored object on the peripheral image displayed on the display device 230A, or display the detected monitored object. A marker may be superimposed and displayed at a position on the peripheral image corresponding to the actual position of the monitored object. Thereby, the display device 230A can realize a visual notification function for the operator and the monitor. In addition, the notification control unit 2103 may use a lighting device such as a warning light to notify the operator or the monitor that the monitored object is detected within the notification range.

Communication status monitoring unit 2104 monitors the communication status between work machine 100 and remote operation support device 200 .

Specifically, communication status monitoring unit 2104 monitors work machine 100 and remote operation support device 200 based on the presence or absence of a predetermined signal received from work machine 100 and the time required from transmission to reception of the predetermined signal. monitor the communication status between The state of communication between work machine 100 and remote operation support device 200 includes, for example, the state of communication disruption between work machine 100 and remote operation support device 200 . Further, the communication status between work machine 100 and remote operation support device 200 includes, for example, the occurrence status of communication delay (magnitude of communication delay) when a signal is transmitted from work machine 100 to remote operation support device 200. ) is included. The predetermined signal is, for example, a reply signal from work machine 100 to a signal requesting a response periodically transmitted from communication status monitoring unit 2104 to work machine 100 through communication device 220 . Further, the predetermined signal may be, for example, a signal corresponding to data (for example, peripheral images) periodically transmitted (uploaded) from work machine 100 . Accordingly, the communication status monitoring unit 2104 can monitor whether or not a communication interruption has occurred based on whether or not a predetermined signal has been received. Further, the communication status monitoring unit 2104 detects the magnitude of the communication delay based on the data (time stamp) representing the transmission timing included in the predetermined signal and the time data when the predetermined signal is actually received through the communication device 220. can be measured.

Operation state monitoring unit 2105 monitors the operation state of the actuator (driven element) of work machine 100 based on the input state received by remote control device 240A.

[Notification range setting process]
Next, referring to FIGS. 8 to 10, processing for setting the notification range by control device 210 will be described.

<First example of notification range setting processing>
FIG. 8 is a flow chart schematically showing a first example of a notification range setting process by control device 210 . This flowchart is repeated at predetermined intervals while the working machine 100 is in operation, that is, from starting the working machine 100 (for example, turning on the key switch) to stopping the working machine 100 (for example, turning off the key switch). executed. The same may be applied to the flow charts of FIGS. 9 and 10, which will be described later.

As shown in FIG. 8, in step S102, the communication status monitoring unit 2104 monitors the working machine 100 and the remote operation support device based on the reception status of the above-described predetermined signal, the time lag between transmission and reception of the predetermined signal, and the like. 200 is monitored.

When the process of step S102 is completed, the control device 210 proceeds to step S104.

Communication status monitoring unit 2104 determines whether or not the communication delay between work machine 100 and remote operation support device 200 exceeds a predetermined standard based on the monitoring result. If the communication delay between work machine 100 and remote operation support device 200 does not exceed the predetermined standard, communication status monitoring unit 2104 proceeds to step S106, and if it exceeds the predetermined standard, proceeds to step S108.

In step S106, the communication status monitoring unit 2104 sets the notification range of the notification function by the safety control unit 304 to the default state.

When the process of step S106 is completed, the control device 210 proceeds to step S110.

On the other hand, in step S108, the communication status monitoring unit 2104 sets the notification range of the notification function by the safety control unit 304 to be expanded outward from the default.

For example, when the communication delay between work machine 100 and remote operation support device 200 increases, the remote operation support device 200 notifies the monitoring object after the imaging device 40 or the like captures the monitored object in the notification range around work machine 100 . The time lag until the function operates increases. Therefore, when the notification function is activated in the remote operation support device 200 and the operator or the observer confirms the presence of the monitored object, the monitored object moves inside the notification range and approaches the work machine 100 and the suspended load. There is a possibility that As a result, the safety of work machine 100 and the work site may deteriorate.

On the other hand, when the communication delay between the work machine 100 and the remote operation support device 200 increases, for example, as in the case of automobiles (see, for example, Japanese Unexamined Patent Application Publication No. 2016-71585), the operation restriction function is automatically activated. It is also possible to ensure safety by

However, the main function of automobiles is to move at relatively high speeds, and it is essential to slow down and stop them in order to ensure safety. It is the main function and there may be cases where uniform deceleration or stopping is not required. Therefore, if the operation restriction function is uniformly activated, there is a possibility that the operation of the work machine 100 will be decelerated or stopped even in a situation where the safety has not deteriorated so much. As a result, there is a problem in terms of compatibility between safety of work machine 100 and workability of work machine 100 .

Further, for example, when the communication delay between work machine 100 and remote operation support device 200 increases, the operation restriction function is activated only when a monitored object is detected in the monitoring area around work machine 100. It is also possible to let

However, even if the monitored object is detected in the monitored area around work machine 100, for example, the monitored object is only moving near the outer edge of the monitored area. Such cases may exist. Therefore, activating the operation restriction function of work machine 100 only when a monitored object is detected in the monitoring area around work machine 100 may not be sufficient from the viewpoint of ensuring workability of work machine 100. have a nature.

In contrast to these, in this example, the control device 210 can expand the notification range outside the default. Therefore, even when there is a large communication delay between work machine 100 and remote operation support device 200, control device 210 can perform remote operation before the object to be monitored approaches a relatively close range such as the work machine and the suspended load. A notification function can be activated in the support device 200 . Therefore, control device 210 can improve the safety of work machine 100 while suppressing deterioration in workability (work efficiency) of work machine 100 .

For example, the communication status monitoring unit 2104 sets the state in which the outer edge of the notification range is expanded outward by a predetermined amount or ratio. At this time, the enlargement amount and enlargement rate of the outer edge of the notification range may be uniform, or may differ depending on the location, and there may be locations that are enlarged and locations that are not enlarged.

Further, for example, the communication status monitoring unit 2104 moves the outer edge of the notification range to the outside so that the amount of change (extension amount) increases as the magnitude of the communication delay between the work machine 100 and the remote operation support device 200 increases. You may set it to the state expanded to .

When the process of step S108 is completed, the control device 210 proceeds to step S110.

At step S110, communication status monitoring unit 2104 transmits to work machine 100 via communication device 220 a signal requesting that the notification range set at step S106 or step S108 be set. As a result, the controller 30 (safety control unit 304 ) sets the notification range according to the signal received from the remote operation support device 200 through the communication device 60 . As a result, control device 210 can reflect the settings in steps S106 and S108 in the actual control of the notification function in work machine 100 .

When the process of step S110 is completed, the control device 210 ends the process of this flowchart.

<Second example of notification range setting processing>
FIG. 9 is a flowchart schematically showing a second example of processing for setting the notification range by control device 210 .

As shown in FIG. 9, steps S202 and S204 are the same as steps S102 and S104 in FIG. 8, so description thereof will be omitted.

In step S204, if the communication delay between work machine 100 and remote operation support device 200 does not exceed the predetermined standard, communication status monitoring unit 2104 proceeds to step S206. Proceed to S208.

Since step S206 is the same as the processing of step S106 in FIG. 8, the description is omitted.

After completing the processing of step S206, the control device 210 proceeds to step S212.

On the other hand, in step S208, operation state monitoring unit 2105 monitors the operation state of work machine 100 based on the input state of remote control device 240A.

After completing the process of step S208, the control device 210 proceeds to step S210.

In step S210, the communication status monitoring unit 2104 sets the notification range of the notification function by the safety control unit 304 to a state expanded outside the default according to the operation state of the work machine 100 based on the monitoring result of step S202. do.

For example, when the traveling operation of the lower traveling body 1 is being performed through the remote control device 240A, the communication status monitoring unit 2104 changes the outer edge portion in the traveling direction of the work machine 100 out of the outer edges of the notification range to the outer edge in the other direction. It may be set to a state in which it expands outward relatively larger than the part. This is because when work machine 100 moves, the distance between work machine 100 and a monitored object existing in the moving direction (running direction) of work machine 100 becomes relatively small.

Further, for example, when the upper rotating body 3 is being rotated through the remote control device 240A, the communication status monitoring unit 2104 rotates around the rotating axis based on the orientation of the attachment AT in the outer edge of the notification range. The directional outer edge portion may be set in a state in which it expands outward relatively more than the outer edge portion in the direction opposite to the turning direction. This is because when the work machine 100 turns, the distance between the monitoring object present in the turning direction about the turning axis and the attachment AT becomes relatively small.

After completing the processing of step S210, the control device 210 proceeds to step S212.

At step S212, communication status monitoring unit 2104 transmits to work machine 100 via communication device 220 a signal requesting that the notification range set at step S206 or step S210 be set. As a result, the controller 30 (safety control unit 304 ) sets the notification range according to the signal received from the remote operation support device 200 through the communication device 60 . As a result, control device 210 can reflect the settings in steps S206 and S210 in the actual control of the notification function in work machine 100 .

<Third example of notification range setting processing>
FIG. 10 is a flow chart schematically showing a third example of the notification range setting process by control device 210 .

As shown in FIG. 10, steps S302 and S304 are the same as the processing of steps S102 and S104 in FIG. 8, so description thereof is omitted.

In step S304, if the communication delay between work machine 100 and remote operation support device 200 does not exceed the predetermined standard, communication status monitoring unit 2104 proceeds to step S306. Proceed to S308.

Since step S306 is the same as the processing of step S106 in FIG. 8, the description is omitted.

After completing the process of step S306, the control device 210 proceeds to step S314.

On the other hand, in step S<b>308 , communication status monitoring unit 2104 determines whether or not object detection unit 303 of work machine 100 has detected a monitored object. The communication status monitoring unit 2104 can determine whether or not the monitored object is detected based on the data representing the object detection status sequentially received from the work machine 100 via the communication device 220 . The communication status monitoring unit 2104 proceeds to step S310 if the object detection unit 303 has detected the monitored object, and proceeds to step S312 if the monitored object has not been detected.

In step S310, the communication status monitoring unit 2104 sets the outer edge of the notification range to a state expanded outside the default according to at least one of the detected position and movement status of the monitored object.

For example, the communication status monitoring unit 2104 may change the outer edge portion of the notification range in the direction in which the monitored object being detected is positioned more outward than the outer edge portions in other directions. This is because the object to be monitored may move toward the inside of the notification range.

In addition, for example, when the monitored object is moving toward the inside of the notification range, the communication status monitoring unit 2104 determines that the monitoring object is larger outside the notification range than when the monitored object is not moving toward the inside of the notification range. You can change it. This is because it is necessary to notify the operator or the observer of the existence of the monitored object moving toward the inside of the notification range at an earlier timing.

Further, for example, when the moving speed of the monitored object is relatively high, the communication status monitoring unit 2104 may change the outer edge of the notification range to the outside more than when the moving speed of the monitored object is relatively slow. This is because it is necessary to notify the operator or the observer at an early timing of the object to be monitored that moves faster inside the notification range.

After completing the process of step S310, the control device 210 proceeds to step S314.

On the other hand, in step S312, the communication status monitoring unit 2104 increases the outer edge of the notification range from the default so that the degree of change (amount of change or rate of change) is relatively smaller than when a monitored object is detected. set to expand outward.

After completing the process of step S312, the control device 210 proceeds to step S314.

At step S314, communication status monitoring unit 2104 transmits to work machine 100 via communication device 220 a signal requesting that the notification range set at step S306, step S310, or step S312 be set. As a result, the controller 30 (safety control unit 304 ) sets the notification range according to the signal received from the remote operation support device 200 through the communication device 60 . As a result, control device 210 can reflect the settings in steps S306, S310, and S312 in the actual control of the notification function in work machine 100. FIG.

[Specific example of monitoring screen]
Next, a specific example of a screen including a surrounding image of work machine 100 (hereinafter referred to as a "monitoring screen") displayed on display device 230A will be described with reference to FIGS. 11 to 14. FIG.

<Monitoring screen when communication delay is relatively small>
11 and 12 are diagrams showing an example of a monitoring screen including peripheral images of work machine 100 displayed on display device 230A. Specifically, FIGS. 11 and 12 show specific examples of monitoring screens when the communication status monitoring unit 2104 determines that the communication delay between the work machine 100 and the remote operation support device 200 is equal to or less than a predetermined standard. It is a figure which shows. More specifically, FIG. 11 is a diagram showing an example of a monitoring screen (monitoring screen 1100) including peripheral images of the excavator 100A corresponding to the imaging range of the imaging device 40 displayed on the display device 230A. FIG. 12 is a diagram showing an example of a monitoring screen (monitoring screen 1200) including peripheral images of the crawler crane 100B corresponding to the imaging range of the imaging device 40 displayed on the display device 230A.

In the case of this example, apart from the monitoring screens 1100 and 1200 in FIGS. 11 and 12, the captured image of the imaging device 42 and the processed image generated based on the captured image are displayed in another display area of the display device 230A, or , are displayed on other display devices 230A of the plurality of display devices 230A. The same applies to the case where monitoring screens 1300 and 1400 of FIGS. 13 and 14, which will be described later, are displayed on the display device 230A.

For example, as shown in FIG. 11, under the control of the display control unit 2102, the display device 230A displays an excavator image CG and a viewpoint-converted image EP as a peripheral image of the excavator 100A, which is arranged around the excavator image CG. A monitoring screen 1100 including is displayed. The excavator image CG and the viewpoint conversion image EP are arranged according to the positional relationship between the excavator 100A (upper revolving body 3) and the imaging range of the imaging device 40 (cameras 40B, 40L, 40R). As a result, the operator can appropriately grasp the positional relationship between the excavator 100A and objects around the excavator 100A appearing in the viewpoint conversion image EP through the monitoring screen 1100 .

In this example, the viewpoint-converted image EP includes a bird's-eye view image BVP in which a peripheral area adjacent to the excavator 100A is viewed from directly above, and a bird's-eye view image BVP arranged around the bird's-eye image BVP, which is viewed from the excavator 100A in the horizontal direction. is combined with the horizontal image HVP. The viewpoint-transformed image EP is obtained by projecting the captured images of the cameras 40B, 40L, and 40R onto the spatial model and then reprojecting the projected images projected onto the spatial model onto another two-dimensional plane. A space model is a projection target of a captured image in a virtual space, and is composed of one or a plurality of planes or curved surfaces including planes or curved surfaces other than the plane on which the captured image is positioned.

In this example, a line LN1 at a constant distance from the excavator 100A is superimposed and displayed on the viewpoint-converted image EP on the monitoring screen 1100 . Line LN1 may represent, for example, the outer edge of the reporting range of safety control unit 304 . As a result, the operator or the observer can appropriately grasp the distance relationship between the excavator 100A and the surrounding objects appearing in the viewpoint-converted image EP.

Further, in this example, the worker W appears in the viewpoint conversion image EP on the monitoring screen 1100 at a location corresponding to the rear of the excavator image CG. Thereby, the operator and the observer can recognize that the worker W is positioned (presence) behind the excavator 100A (upper revolving body 3).

In this example, the worker W is present outside the line LN1 corresponding to the outer edge of the notification range. Therefore, the worker W is not subject to the visual notification function, and an image emphasizing the presence of the worker W (for example, a frame FRM1 described later) is not displayed.

The display control unit 2102 may cause the display device 230A to display a monitor screen including an image captured by at least one of the cameras 40B, 40L, and 40R. In other words, the display control unit 2102 may cause all of the captured images of the cameras 40B, 40L, and 40R or the captured images of the two cameras to be displayed side by side on the display device 230A, or may display the captured images of any one of the cameras. It may be displayed on the display device 230A. Typically, the display control unit 2102 may cause the images captured by the cameras 40B and 40R to be displayed side by side on the display device 230A. In this case, the display control unit 2102 superimposes a line representing the outer edge of the monitoring area, the notification range, etc. on the monitoring screen including the captured images of the cameras 40B, 40L, and 40R of the display device 230A, as in the case of the viewpoint conversion image EP. can be displayed.

Further, for example, as shown in FIG. 12, the display device 230A displays a monitor screen 1200 including an image captured by the imaging device 40 as a peripheral image of the crawler crane 100B under the control of the display control unit 2102. FIG.

A monitoring screen 1200 (captured image of the imaging device 40) shows the hook HK at the tip of the main hoisting rope 7B and the suspended load SL slinged on the hook HK.

Further, in this example, the monitoring screen 1200 displays a superimposed line LN2 at a certain distance from the suspended load SL. A line LN2 represents, for example, the outer edge of the notification range of the safety control unit 304 or the like. As a result, the operator or the monitor can appropriately grasp the positional relationship between the suspended load SL and the surrounding objects displayed on the monitoring screen 1200 (image captured by the imaging device 40).

In this example, the monitor screen 1200 shows two obstacles OB around the suspended load SL. This allows the operator and the observer to recognize that two obstacles OB are present around the suspended load SL.

In this example, both of the two obstacles OB are present outside the line LN1 representing the outer edge of the notification range. Therefore, the two obstacles OB are not targets of the visual notification function, and an image emphasizing the existence of the obstacle OB (for example, a frame FRM2 described later) is not displayed.

The display control unit 2102 may cause the display device 230A to display, as the peripheral image, a viewpoint conversion image generated based on the captured image of the imaging device 40 of the crawler crane 100B.

<Monitoring screen when communication delay is relatively large>
13 and 14 are diagrams showing other examples of monitoring screens including peripheral images of work machine 100 displayed on display device 230A. Specifically, FIGS. 13 and 14 show specific monitoring screens when the communication status monitoring unit 2104 determines that the communication delay between the work machine 100 and the remote operation support device 200 exceeds a predetermined standard. FIG. 4 is a diagram showing an example; More specifically, FIG. 13 is a diagram showing another example of a monitoring screen (monitoring screen 1300) displayed on the display device 230A and including a peripheral image of the excavator 100A corresponding to the imaging range of the imaging device 40. As shown in FIG. . FIG. 14 is a diagram showing an example of a monitoring screen (monitoring screen 1400) including peripheral images of the crawler crane 100B corresponding to the imaging range of the imaging device 40 displayed on the display device 230A.

For example, as shown in FIG. 13, the display device 230A displays, under the control of the display control unit 2102, an excavator image CG and a perspective conversion image as a peripheral image of the excavator 100A, as in the case of the above example (FIG. 11). A monitoring screen 1300 including an image EP is displayed.

Further, in this example, a line LN1 corresponding to the outer edge of the notification range is superimposed and displayed on the viewpoint-converted image EP of the monitoring screen 1300, as in the example described above.

In this example, the line LN1 uniformly changes to the outside, that is, in the direction away from the shovel image CG, as compared to the above-described example. This is because the communication delay between the excavator 100A and the remote operation support device 200 exceeds the predetermined standard. As a result, control device 210 can make it possible to recognize, through display device 230A, that the outer edge of the notification range has changed to the outside compared to normal times (when the communication delay is relatively small). In addition, the control device 210 controls the communication delay between the excavator 100A and the remote operation support device 200 relative to the line LN1 moving outside on the display device 230A (monitoring screen 1300). can be made to grasp that it is getting bigger.

Further, in this example, the worker W appears in the viewpoint conversion image EP on the monitoring screen 1300 in a place corresponding to the rear of the excavator image CG, as in the case of the above example.

Further, in this example, a frame FRM1 surrounding the worker W is superimposed on the viewpoint conversion image EP of the monitoring screen 1300 and displayed. This is because the worker W came to be inside the line LN1 by changing the outer edge of the notification range to the outside. As a result, in a situation where the time lag from when the worker W is captured by the imaging device 40 to when the notification function is activated by the remote operation support device 200 is relatively large, the operator or the operator can monitor the presence of the worker W at an earlier timing. can be understood by the person.

It should be noted that the manner in which the line LN1, which corresponds to the outer edge of the notification range, changes with respect to the example described above may be varied according to the operating state of the excavator 100A, as described above. For example, when the excavator 100A is operated to travel toward the rear of the upper revolving body 3, the degree of change of the line LN1 with respect to the above-described example is greater than that of the excavator image CG on the left and right sides (straight line portions). The rear (lower) portion (curved portion) of the image CG may be made larger. Also, for example, when the upper swing body 3 of the excavator 100A is being swiveled rightward, the degree of change of the line LN1 from the above example is more pronounced in the right portion of the excavator image CG than in the left portion. may be increased. In addition, the mode of change of the line LN1 corresponding to the outer edge of the notification range with respect to the example described above may be changed according to the position and movement status of the monitored object (worker W) around the excavator 100A, as described above. . For example, the degree of change of the line LN1 with respect to the above example is that the rear (lower) portion of the excavator image CG where the worker W is present is greater than the left or right side of the excavator image CG where the worker W is not present. may also be increased. Further, for example, the degree of change of the line LN1 with respect to the above example is made greater when the worker W is moving toward the excavator 100A than when the worker W is not moving toward the excavator 100A. good. Also, for example, the degree of change of the line LN1 with respect to the above example may be made larger when the moving speed of the worker W is relatively high than when it is relatively low.

In this example, the monitor screen 1300 also displays an image MSG1 indicating a relatively large communication delay between the excavator 100A and the remote operation support device 200. FIG. As a result, the operator and/or the supervisor can understand that there is a relatively large communication delay between the excavator 100A and the remote operation support device 200, and that the line LN1 changes from normal due to the communication delay. It is possible to understand that

Further, for example, as shown in FIG. 14, the display device 230A displays an image of the imaging device 40 as a peripheral image of the crawler crane 100B under the control of the display control unit 2102, as in the case of the above example (FIG. 12). A monitoring screen 1400 including a captured image is displayed.

On the monitoring screen 1400 (captured image of the imaging device 40), the hook HK at the tip of the main hoisting rope 7B and the suspended load SL slinged on the hook HK are shown, as in the above example.

Further, in this example, a line LN2 representing the outer edge of the notification range is superimposed and displayed on the monitoring screen 1200, as in the case of the above example.

In this example, the line LN2 is changed to the outside, that is, in the direction away from the suspended load SL (hook HK) than in the above example. This is because the communication delay between the crawler crane 100B and the remote operation support device 200 exceeds the predetermined standard. As a result, control device 210 can make it possible to recognize, through display device 230A, that the outer edge of the notification range has changed to the outside compared to normal times (when the communication delay is relatively small). In addition, the control device 210 controls the communication delay between the excavator 100A and the remote operation support device 200 relative to the line LN2 on the display device 230A (monitoring screen 1400) through the movement of the line LN2 to the outside. can be made to grasp that it is getting bigger.

Also, in this example, two obstacles OB around the suspended load SL are displayed on the monitor screen 1400, as in the above example.

In this example, the monitor screen 1400 also displays a superimposed frame FRM2 surrounding each of the two obstacles OB. This is because the obstacle OB now exists inside the line LN2 as the outer edge of the notification range changes to the outside. As a result, in a situation where the time lag from when the obstacle OB is captured by the imaging device 40 to when the notification function is activated by the remote operation support device 200 is relatively large, the operator can detect the presence of the obstacle OB at an earlier timing. can be understood by the person.

It should be noted that the mode of change of the line LN2 corresponding to the outer edge of the notification range with respect to the above example may be changed according to the operating state of the crawler crane 100B as described above. For example, when the crawler crane 100B is operated to travel toward the front of the upper swing structure 3, the degree of change of the line LN2 with respect to the above example is the left side, right side, and rear side of the hook HK (suspended load SL). The front portion may be made larger than the portion of . Further, for example, when the upper rotating body 3 of the crawler crane 100B is being rotated to the right, the degree of change of the line LN2 from the above example is greater than the left portion of the hook HK (slung load SL). The right portion may be made larger. Further, the mode of change of the line LN2 corresponding to the outer edge of the notification range with respect to the example described above may be varied according to the position and movement of the monitored object (obstacle OB) around the crawler crane 100B as described above. good. For example, the degree of change of the line LN2 with respect to the above-described example is such that the hook HK (suspended load SL) where the obstacle OB exists is closer to the hook HK (suspended load SL) than the hook HK (suspended load SL) where the obstacle OB exists. SL) to the left or right. Further, for example, the degree of change of the line LN2 with respect to the above example is that when the obstacle OB is relatively moving toward the hook HK (suspended load SL), the obstacle OB moves toward the hook HK (suspended load SL). It may be made larger than it would have been without relative movement towards it. Further, for example, the degree of change of the line LN2 with respect to the above example is made larger when the relative moving speed of the obstacle OB with respect to the hook HK (suspended load SL) is relatively high than when it is relatively low. good too.

In this example, the monitoring screen 1400 also displays an image MSG2 representing a message to the effect that a relatively large communication delay has occurred between the crawler crane 100B and the remote operation support device 200. FIG. As a result, the operator and/or the observer can understand that there is a relatively large communication delay between the crawler crane 100B and the remote operation support device 200, and that the line LN2 changes from normal due to the communication delay. You can understand what you are doing.

[Action]
Next, the operation of the remote operation support system SYS according to this embodiment will be described.

In this embodiment, the imaging device 40 of the work machine 100 acquires data (image data) regarding objects around the work machine 100 . Also, the communication device 60 of the work machine 100 communicates with the remote operation support device 200 . Further, communication device 220 of remote operation support device 200 communicates with work machine 100 . Remote operation device 240</b>A of remote operation support device 200 receives input for remotely operating work machine 100 . Controller 30 of work machine 100 or control device 210 of remote operation support device 200 detects objects to be monitored around work machine 100 based on the output of imaging device 40 . Further, the output device 230 is provided in the remote operation support device 200, and under the control of the control device 210, when the monitored object is detected by the controller 30 or the control device 210 within the notification range around the working machine 100, the remote The user using the operation device 240A is notified of this fact. When the communication delay between work machine 100 and remote operation support device 200 is relatively large, remote operation support system SYS shifts the outer edge of the notification range outward more than when the communication delay is relatively small. .

As a result, the remote operation support system SYS can make the object to be monitored subject to the notification function at a somewhat earlier stage than it enters the area inside the notification range where safety may be lowered. Therefore, even in a situation where the time lag from when the monitored object is captured by the imaging device 40 to when the notification function is activated by the remote operation support device 200 is relatively large due to the communication delay, the remote operation support system SYS can keep the monitored object within the notification range. The presence of the object to be monitored can be notified to the observer or operator before entering the range where the safety inside can be relatively lowered. Therefore, the safety of work machine 100 can be improved. In addition, the operator or supervisor can check the monitoring screen or the like displayed on the display device 230A in response to the activation of the notification function, and can determine whether to continue or stop the operation of the work machine 100 by himself/herself. Therefore, when the communication delay between work machine 100 and remote operation support device 200 becomes relatively large, the operation restriction function is uniformly activated or the operation restriction range is expanded outward. A situation in which the workability of the machine 100 is greatly reduced can be suppressed. Therefore, the remote operation support system SYS can improve the safety of the work machine 100 while suppressing deterioration of workability (work efficiency) of the work machine 100 .

Further, in this embodiment, when the communication delay described above is relatively large, the remote operation support system SYS may change the outer edge of the notification range outward according to the user's input to the remote operation device 240A.

As a result, remote operation support system SYS, in a situation where the above-described communication delay is relatively large, adjusts the operation state of work machine 100 through remote operation device 240A to the outer edge of the notification range that does not affect safety. It can vary outwardly around a certain outer edge portion. Therefore, remote operation support system SYS can more appropriately achieve both safety of work machine 100 and workability of work machine 100 .

Further, in the present embodiment, when the above-described communication delay is relatively large, the remote operation support system SYS is configured to perform the travel operation of the work machine 100 via the remote control device 240A, as viewed from the work machine 100. The outer edge of the notification range in the direction in which the vehicle is traveling may be changed more outward than the outer edge of the notification range in other directions.

As a result, remote operation support system SYS specifically adjusts the outer edge portion of the notification range that affects the safety of work machine 100 according to the operation state of work machine 100 through remote operation device 240A. Can be varied outward to the center.

Further, in the present embodiment, when the above-described communication delay is relatively large, the remote operation support system SYS can rotate the work attachment around the turning axis while the work machine 100 is being turned through the remote control device 240A. The outer edge of the notification range in the turning direction based on the direction of is changed to be larger outside than the outer edge of the notification range in the direction opposite to the turning direction.

As a result, remote operation support system SYS specifically adjusts the outer edge portion of the notification range that affects the safety of work machine 100 according to the operation state of work machine 100 through remote operation device 240A. Can be varied outward to the center.

Further, in the present embodiment, when the communication delay described above is relatively large, the remote operation support system SYS adjusts the notification range according to at least one of the position and movement status of the monitored object existing around the work machine 100. Change the outer edge.

As a result, the remote operation support system SYS can change the outer edge portion of the notification range, which affects safety, toward the outside in accordance with the position and movement of the object to be monitored. Therefore, remote operation support system SYS can more appropriately achieve both safety of work machine 100 and workability of work machine 100 .

Further, in this embodiment, when the communication delay is relatively large, the remote operation support system SYS shifts the outer edge of the notification range in the direction in which the monitored object is positioned as viewed from the work machine 100 to the notification range in other directions. may be varied more outward than the outer edge of .

As a result, the remote operation support system SYS can specifically change the outer edge portion of the notification range, which affects safety, toward the outside in accordance with the position of the object to be monitored.

Further, in the present embodiment, when the communication delay is relatively large, the remote operation support system SYS is configured to move the monitored object toward the inside of the notification range while the monitored object is moving toward the inside of the notification range. The outer edge of the notification range may be changed more outward than when it is not moving.

As a result, the remote operation support system SYS can specifically change the outer edge portion of the notification range, which affects safety, toward the outside in accordance with the movement of the monitored object.

Further, in the present embodiment, when the communication delay is relatively large, the remote operation support system SYS has a predetermined delay time when the moving speed of the monitored object is relatively high compared to when the moving speed of the monitored object is relatively low. , the outer edge of the range may be changed significantly outward.

As a result, the remote operation support system SYS can specifically change the outer edge portion of the notification range, which affects safety, toward the outside in accordance with the movement of the monitored object.

Further, in the present embodiment, the display device 230A may display information regarding the notification range so that the display content changes according to changes in the outer edge of the notification range.

As a result, the remote operation support system SYS can allow the operator or the observer to grasp the change in the outer edge of the notification range due to the above-described increase in communication delay through the display device 230A.

Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and changes are possible within the scope of the claims.

30 controller (object detection device)
40 imaging device (acquisition device)
40B camera 40L camera 40R camera 42 imaging device 42A camera 42B camera 50 output device 52 input device 60 communication device (working device communication device)
100 Working machine 100A Excavator 100B Crawler crane 200 Remote control support device (external device)
210 control device (object detection device)
220 communication device (communication device for external device)
230 output device (notification device)
230A display device 230B sound output device 240 input device 240A remote operation device 2101 remote operation support unit 2102 display control unit 2103 notification control unit 2104 communication status monitoring unit 2105 operation state monitoring unit SYS remote operation support system

Claims (10)

an acquisition device provided on a work machine for acquiring data related to objects around the work machine;
a work machine communication device that is provided in the work machine and communicates with an external device that supports remote control of the work machine;
an external device communication device that is provided in the external device and communicates with the working machine;
a remote control device that is provided in the external device and receives an input for remotely controlling the work machine;
an object detection device provided in the work machine or the external device for detecting a predetermined object around the work machine based on the output of the acquisition device;
a notification device provided in the external device for notifying a user using the remote control device of the fact that the predetermined object is detected within a predetermined range around the work machine by the object detection device; with
when the communication delay between the work machine and the external device is relatively large, the outer edge of the predetermined range is shifted outward more than when the communication delay is relatively small;
Remote operation support system. when the communication delay is relatively large, changing the outer edge of the predetermined range outward according to the user's input to the remote control device;
The remote operation support system according to claim 1. When the communication delay is relatively large, when the work machine is being operated to travel through the remote control device, the outer edge of the predetermined range in the direction in which the work machine is traveling is shifted in another direction. change greatly outside the outer edge of the predetermined range of
The remote operation support system according to claim 2. When the communication delay is relatively large, the outer edge of the predetermined range in the turning direction around the turning axis based on the orientation of the work attachment when the working machine is being turned through the remote control device. is changed greatly outside the outer edge of the predetermined range in the direction opposite to the turning direction,
The remote operation support system according to claim 2 or 3. when the communication delay is relatively large, changing the outer edge of the predetermined range according to at least one of the position and movement status of the predetermined object existing around the work machine;
The remote operation support system according to any one of claims 1 to 4. When the communication delay is relatively large, the outer edge of the predetermined range in the direction in which the predetermined object is positioned as viewed from the working machine is changed greatly outside the outer edge of the predetermined range in other directions. ,
The remote operation support system according to claim 5. When the communication delay is relatively large, when the predetermined object is moving toward the inside of the predetermined range, it is faster than when the predetermined object is not moving toward the inside of the predetermined range. greatly changing the outer edge of the predetermined range outward;
The remote operation support system according to claim 5 or 6. When the communication delay is relatively large, when the movement speed of the predetermined object is relatively high, the outer edge of the predetermined range is made larger outside than when the movement speed of the predetermined object is relatively low. change,
The remote operation support system according to any one of claims 5 to 7. A display device that is provided in the external device and displays information about the predetermined range so that the display content changes according to a change in the outer edge of the predetermined range,
The remote operation support system according to any one of claims 1 to 8. a communication device that communicates with the work machine having an acquisition device that acquires data about objects in the vicinity of the work machine;
a remote control device that receives input for remotely controlling the work machine;
a notification device that, when a predetermined object is detected within a predetermined range around the work machine based on the output of the acquisition device, notifies a user using the remote control device of the fact;
When the communication delay with the work machine is relatively large, the outer edge of the predetermined range is changed in a direction away from the work machine more than when the communication delay is relatively small.
Remote operation support device.

Images