JP2023183992A - Support system, remote control support device, work machine, and program - Google Patents

Abstract

To provide a technology capable of presenting to a user an image representing a shape of a work object in a wider range around a work machine.SOLUTION: A remote control support system SYS comprises: an imaging device 45 attached to an attachment AT of a shovel 100 and acquiring image data related to a shape of a work object around the shovel 100; and a display device 208 displaying an image of an environmental map representing the shape of the work object around the shovel 100 based on data acquired by the imaging device 45 during an operation of the attachment AT.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to a support system and the like.

For example, a technique for displaying images to assist an operator in operating a work machine has been disclosed (see Patent Document 1).

In Patent Document 1, an image representing the shape of a work object (ground) around the working machine is presented to the operator based on the output of an imaging device and a distance detection device installed in the driver's seat of the upper revolving body of the working machine. There is.

Japanese Patent Application Publication No. 2021-038649

However, in Patent Document 1, an imaging device and a distance detection device for acquiring data regarding work objects around the working machine are attached to the operator's cab of the upper revolving structure, and the imaging range in the far and near direction when viewed from the working machine is is fixed. Therefore, for example, the range in which data regarding the work object around the work machine can be obtained is limited, and as a result, the range that can be displayed as an image representing the shape of the work target around the work machine may be limited. .

Therefore, in view of the above-mentioned problems, it is an object of the present invention to provide a technique that can present to a user an image representing the shape of a work target in a wider range around a work machine.

To achieve the above object, in one embodiment of the present disclosure,
an acquisition unit that is attached to an attachment of a working machine and acquires data regarding the shape of a work target around the working machine;
a display unit that displays an image of an environmental map representing the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data acquired by the acquisition unit during operation of the attachment; ,
Support systems are provided.

Additionally, in other embodiments of the present disclosure,
an operation unit that accepts remote control of the work machine;
A signal regarding the operation state of the operating unit by the operator is transmitted to the working machine, and data regarding the shape of the work object around the working machine, which is acquired by an acquisition unit attached to an attachment of the working machine, is sent to the working machine. a communication unit that receives data from the
An environment that represents the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data regarding the shape of the work target around the work machine, which is acquired by the acquisition unit when the attachment is operated. a display unit that displays an image of a map;
A remote operation support device is provided.

In still other embodiments of the present disclosure,
attachment and
an acquisition unit that is attached to the attachment and acquires data regarding the shape of a work target around the work machine;
a display unit that displays an image of an environmental map representing the shape of the work target based on data acquired by the acquisition unit when the attachment is operated;
Working machinery is provided.

In still other embodiments of the present disclosure,
In the information processing device,
The work when viewed from a predetermined viewpoint around the work machine, based on data regarding the shape of the work target around the work machine, which is acquired by an acquisition unit attached to the attachment of the work machine when the attachment is in operation. executing a display step of displaying an image of an environmental map representing the shape of the target on a display unit;
program will be provided.

According to the embodiment described above, it is possible to present to the user an image representing the shape of the work target in a wider range around the work machine.

FIG. 1 is a diagram showing an example of a remote operation support system. It is a top view showing an example of a shovel. FIG. 2 is a block diagram showing an example of the hardware configuration of an excavator. It is a diagram showing an example of the hardware configuration of a remote operation support device. FIG. 2 is a functional block diagram showing an example of a functional configuration of a remote operation support system. FIG. 2 is a diagram showing a first example of a display screen of an environmental map showing the shape of a work target around the excavator. FIG. 7 is a diagram illustrating a second example of a display screen of an environmental map showing the shape of a work target around the excavator. FIG. 7 is a diagram showing a third example of a display screen of an environmental map showing the shape of a work target around the excavator. It is a figure which shows the 4th example of the display screen of the environmental map showing the shape of the work target around a shovel.

Embodiments will be described below with reference to the drawings.

[Overview of remote operation support system]
First, an overview of the remote operation support system SYS according to this embodiment will be explained with reference to FIGS. 1 and 2.

FIG. 1 is a diagram showing an example of a remote operation support system SYS. In FIG. 1, the excavator 100 is shown in a left side view. FIG. 2 is a top view showing an example of the shovel 100. Hereinafter, the direction on the shovel 100 or the direction seen from the shovel 100 may be described by defining the direction in which the attachment AT extends (upward direction in FIG. 2) as seen from the top of the shovel 100 as "front".

As shown in FIG. 1, the remote operation support system SYS includes a shovel 100 and a remote operation support device 200.

The remote operation support system SYS uses the remote operation support device 200 to cooperate with the excavator 100 and provides support regarding remote operation of the excavator 100.

The number of excavators 100 included in the remote operation support system SYS may be one or more than one.

The excavator 100 is a work machine to be remotely controlled in the remote control support system SYS.

Note that the object to be remotely controlled may be another working machine (for example, a crawler crane, etc.) different from the excavator 100.

As shown in FIGS. 1 and 2, the excavator 100 includes a lower traveling body 1, an upper rotating body 3, an attachment AT including a boom 4, an arm 5, and a bucket 6, and a cabin 10.

The lower traveling body 1 causes the excavator 100 to travel using the crawler 1C. The crawler 1C includes a left crawler 1CL and a right crawler 1CR. The crawler 1CL is hydraulically driven by a travel hydraulic motor 1ML. Similarly, the crawler 1CL is hydraulically driven by a travel hydraulic motor 1MR. Thereby, the lower traveling body 1 can self-propel.

The upper rotating body 3 is rotatably mounted on the lower traveling body 1 via the rotating mechanism 2 . For example, the upper rotating structure 3 turns with respect to the lower traveling structure 1 by hydraulically driving the turning mechanism 2 by the turning hydraulic motor 2M.

The boom 4 is attached to the center of the front part of the upper revolving body 3 so as to be able to rise and fall about a rotation axis along the left-right direction. The arm 5 is attached to the tip of the boom 4 so as to be rotatable about a rotation axis extending in the left-right direction. The bucket 6 is attached to the tip of the arm 5 so as to be rotatable about a rotation axis extending in the left-right direction.

The bucket 6 is an example of an end attachment, and is used, for example, in excavation work.

The bucket 6 is attached to the tip of the arm 5 in such a manner that it can be replaced as appropriate depending on the work content of the shovel 100. That is, instead of the bucket 6, a bucket of a different type than the bucket 6, such as a relatively large bucket, a slope bucket, a dredging bucket, etc., may be attached to the tip of the arm 5. Further, an end attachment of a type other than the bucket, such as an agitator, a breaker, a crusher, etc., may be attached to the tip of the arm 5. Furthermore, a preliminary attachment such as a quick coupling or a tiltrotator may be provided between the arm 5 and the end attachment.

The boom 4, arm 5, and bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.

The cabin 10 is a control room where an operator boards and operates the shovel 100. The cabin 10 is mounted, for example, on the front left side of the upper revolving body 3.

For example, the excavator 100 is configured to be able to be remotely operated from outside the excavator 100. When the excavator 100 is remotely controlled, the interior of the cabin 10 may be unmanned.

For example, the remote control includes a mode in which the shovel 100 is operated by an operation input regarding the actuator of the shovel 100 performed by the remote control support device 200.

The remote operation support device 200 is provided, for example, in a management center that manages the work of the excavator 100 from the outside. Further, the remote operation support device 200 may be a portable operation terminal, in which case the operator can remotely control the excavator 100 while directly checking the working status of the excavator 100 from around the excavator 100. can.

For example, the excavator 100 transmits an image representing the surroundings including the front of the excavator 100 (hereinafter referred to as "surrounding image") based on a captured image output by the imaging device 40 (described later) to the remote operation support device through the communication device 60 (described later). 200. Then, the remote operation support device 200 may display the image (surrounding image) received from the excavator 100 on the display device. Further, an information screen presenting various information regarding the excavator 100 may be displayed on the display device of the remote operation support device 200. As a result, an operator using the remote operation support device 200 can operate the excavator 100 remotely while checking display contents such as an image or an information screen showing the surroundings of the excavator 100 displayed on the display device 208 (described later). can be operated. Then, the excavator 100 operates the actuators to operate the lower traveling structure 1 , the upper rotating structure 3 , and the boom 4 in response to a remote control signal indicating the content of the remote control, which is received from the remote control support device 200 through the communication device 60 . , arm 5, and bucket 6 may be driven.

Further, the remote control may include, for example, a mode in which the shovel 100 is operated by a person (for example, a worker) around the shovel 100 by external voice input or gesture input to the shovel 100. At this time, the operator who performs remote control using gestures from around the excavator 100 has a portable remote control support device 200 and checks the surrounding images and information screen displayed on the remote control support device 200 (display device 208). It may be possible. Specifically, the excavator 100 receives sounds uttered by surrounding workers, etc. through a voice input device (for example, a microphone), a gesture input device (for example, an imaging device), etc. mounted on the excavator 100. Recognizes gestures etc. performed by Then, the excavator 100 operates the actuator according to the content of the recognized voice or gesture, and moves the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, the boom 4, the arm 5, the bucket 6, etc. The driven element may also be driven.

Further, the work of the shovel 100 may be remotely monitored. In this case, a remote monitoring support device having the same functions as the remote operation support device 200 may be provided. Thereby, the supervisor who is the user of the remote monitoring support device can monitor the working status of the excavator 100 while checking the peripheral image displayed on the display device of the remote monitoring support device. For example, if the supervisor determines that it is necessary from a safety perspective, the supervisor may intervene in the operator's operation of the excavator 100 and bring it to an emergency stop by inputting a predetermined input using the input device of the remote monitoring support device. be able to.

The remote operation support device 200 cooperates with the shovel 100 by communicating with the shovel 100, and provides support regarding the operation of the shovel 100.

The remote operation support device 200 is installed, for example, in a management office within the work site of the excavator 100, or in a management center that manages the operating status of the excavator 100, etc., located at a location different from the work site of the excavator 100. It is a terminal device for servers and management. The management terminal device may be a stationary terminal device such as a desktop PC (Personal Computer), or a portable terminal device (mobile phone) such as a tablet terminal, smartphone, or laptop PC. terminal). In the latter case, workers at the work site, supervisors who supervise work, managers who manage the work site, and the like can carry the portable remote operation support device 200 and move around the work site. In the latter case, the operator can, for example, bring the portable remote operation support device 200 into the cabin of the excavator 100.

For example, the remote operation support device 200 acquires data regarding the operating state from the excavator 100. Thereby, the remote operation support device 200 can grasp the operating state of the excavator 100 and monitor the presence or absence of an abnormality in the excavator 100. Further, the remote operation support device 200 can display data regarding the operating state of the excavator 100 through a display device 208, which will be described later, for the user to confirm.

Further, the remote operation support device 200 transmits to the excavator 100, for example, various data such as programs and reference data used in processing by the controller 30, etc. of the excavator 100. Thereby, the excavator 100 can perform various processes related to the operation of the excavator 100 using various data downloaded from the remote operation support device 200.

In addition, the excavator 100 moves the lower traveling body 1 (that is, the pair of left and right crawlers 1CL, 1CR), the upper revolving body 3, the boom 4, the arm 5, the bucket 6, etc. The driven element may be configured to be operable. The following description will proceed on the premise that the operator's operations include at least one of an operator's operation on the operating device 26 by an operator in the cabin 10 and a remote control by an external operator.

[Hardware configuration of remote operation support system]
Next, the hardware configuration of the remote operation support system SYS will be described with reference to FIGS. 3 and 4 in addition to FIGS. 1 and 2.

<Excavator hardware configuration>
FIG. 3 is a block diagram showing an example of the hardware configuration of shovel 100.

In Figure 3, the path through which mechanical power is transmitted is a double line, the path through which high-pressure hydraulic fluid that drives the hydraulic actuator flows is a solid line, the path through which pilot pressure is transmitted is a broken line, and the path through which electrical signals are transmitted is shown. Each route is indicated by a dotted line.

The excavator 100 includes a hydraulic drive system for hydraulically driving the driven elements, an operation system for operating the driven elements, a user interface system for exchanging information with the user, a communication system for communicating with the outside, a control system for various controls, etc. Contains each component of.

≪Hydraulic drive system≫
As shown in FIG. 3, the hydraulic drive system of the excavator 100 uses hydraulic pressure to hydraulically drive each of the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, and the attachment AT, as described above. Includes actuator HA. Further, the hydraulic drive system of the excavator 100 according to the present embodiment includes an engine 11, a regulator 13, a main pump 14, and a control valve 17.

The hydraulic actuator HA includes travel hydraulic motors 1ML and 1MR, a swing hydraulic motor 2M, a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, and the like.

Note that in the excavator 100, part or all of the hydraulic actuator HA may be replaced with an electric actuator. In other words, the excavator 100 may be a hybrid excavator or an electric excavator.

The engine 11 is the prime mover of the excavator 100 and is the main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, at the rear of the upper revolving structure 3. The engine 11 rotates at a predetermined target rotation speed under direct or indirect control by a controller 30, which will be described later, and drives the main pump 14 and the pilot pump 15.

Note that in place of or in addition to the engine 11, another prime mover (for example, an electric motor) or the like may be mounted on the excavator 100.

The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30 . For example, the regulator 13 adjusts the angle of the swash plate (hereinafter referred to as "tilt angle") of the main pump 14 in accordance with a control command from the controller 30.

The main pump 14 supplies hydraulic oil to the control valve 17 through a high pressure hydraulic line. The main pump 14 is, for example, mounted at the rear of the upper revolving structure 3, like the engine 11. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, the stroke length of the piston is adjusted by adjusting the tilt angle of the swash plate by the regulator 13 under the control of the controller 30, and the stroke length of the piston is adjusted. The flow rate and discharge pressure are controlled.

The control valve 17 drives the hydraulic actuator HA in accordance with the contents of an operator's operation on the operating device 26 or remote control, or an operation command corresponding to an automatic driving function. The control valve 17 is mounted, for example, in the center of the upper revolving body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line, and controls the hydraulic fluid supplied from the main pump 14 according to an operator's operation or an operation command corresponding to an automatic operation function. , selectively supplying each hydraulic actuator. Specifically, the control valve 17 includes a plurality of control valves (also referred to as "direction switching valves") that control the flow rate and flow direction of the hydraulic oil supplied from the main pump 14 to each of the hydraulic actuators HA.

≪Operation system≫
As shown in FIG. 3, the operating system of excavator 100 includes a pilot pump 15, an operating device 26, a hydraulic control valve 31, a shuttle valve 32, and a hydraulic control valve 33.

The pilot pump 15 supplies pilot pressure to various hydraulic devices via a pilot line 25. The pilot pump 15 is, for example, mounted at the rear of the upper revolving structure 3, like the engine 11. The pilot pump 15 is, for example, a fixed capacity hydraulic pump, and is driven by the engine 11 as described above.

Note that the pilot pump 15 may be omitted. In this case, the relatively high pressure hydraulic oil discharged from the main pump 14 may be reduced in pressure by a predetermined pressure reducing valve, and then the relatively low pressure hydraulic oil may be supplied as pilot pressure to various hydraulic devices.

The operating device 26 is provided near the cockpit of the cabin 10 and is used by an operator to operate various driven elements. Specifically, the operating device 26 is used for an operator to operate the hydraulic actuator HA that drives each driven element, and as a result, the operator operates the driven element to be driven by the hydraulic actuator HA. can be realized. The operating device 26 includes a pedal device and a lever device for operating each driven element (hydraulic actuator HA).

For example, as shown in FIG. 3, the operating device 26 is of a hydraulic pilot type. Specifically, the operating device 26 utilizes hydraulic oil supplied from the pilot pump 15 through the pilot line 25 and a pilot line 25A branching from the pilot line 25, and applies pilot pressure according to the operation content to the pilot line 27A on the secondary side. Output to. Pilot line 27A is connected to one inlet port of shuttle valve 32 and connected to control valve 17 via pilot line 27, which is connected to an outlet port of shuttle valve 32. Thereby, a pilot pressure can be input to the control valve 17 via the shuttle valve 32 in accordance with the operation contents regarding various driven elements (hydraulic actuator HA) in the operating device 26. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation performed on the operating device 26 by an operator or the like.

Moreover, the operating device 26 may be electrical. In this case, the pilot line 27A, shuttle valve 32, and hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electrical signal (hereinafter referred to as an "operating signal") according to the content of the operation, and the operating signal is taken into the controller 30. Then, the controller 30 outputs a control command according to the content of the operation signal, that is, a control signal according to the content of the operation on the operating device 26 to the hydraulic control valve 31. As a result, pilot pressure corresponding to the operation details of the operating device 26 is inputted from the hydraulic control valve 31 to the control valve 17, and the control valve 17 drives each hydraulic actuator HA according to the operation details of the operating device 26. be able to.

Furthermore, the control valves (directional switching valves) built into the control valve 17 and driving the respective hydraulic actuators HA may be of an electromagnetic solenoid type. In this case, the operation signal output from the operation device 26 may be directly input to the control valve 17, that is, to an electromagnetic solenoid type control valve.

Further, as described above, part or all of the hydraulic actuator HA may be replaced with an electric actuator. In this case, the controller 30 may output a control command according to the operation content of the operating device 26 or the remote control content specified by the remote control signal to the electric actuator or a driver driving the electric actuator. Moreover, when the shovel 100 is remotely controlled, the operating device 26 may be omitted.

The hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 and for each drive direction of the driven element (hydraulic actuator HA) (for example, the raising direction and lowering direction of the boom 4). . That is, two hydraulic control valves 31 are provided for each double-acting hydraulic actuator HA. The hydraulic control valve 31 is provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and is configured to be able to change its flow path area (that is, the cross-sectional area through which hydraulic oil can flow). good. Thereby, the hydraulic control valve 31 can output a predetermined pilot pressure to the secondary side pilot line 27B using the hydraulic oil of the pilot pump 15 supplied through the pilot line 25B. Therefore, the hydraulic control valve 31 can indirectly apply a predetermined pilot pressure according to the control signal from the controller 30 to the control valve 17 through the shuttle valve 32 between the pilot line 27B and the pilot line 27. . Therefore, the controller 30 can control the hydraulic control valve 31 and realize remote control of the excavator 100. Specifically, the controller 30 outputs to the hydraulic control valve 31, through the communication device 60, a control signal corresponding to the content of the remote operation specified by the remote operation signal received from the remote operation support device 200. Thereby, the controller 30 can cause the hydraulic control valve 31 to supply pilot pressure corresponding to the content of the remote control to the control valve 17, and realize the operation of the shovel 100 based on the operator's remote control.

Further, when the operating device 26 is an electric type, the controller 30 causes the hydraulic control valve 31 to directly supply pilot pressure according to the operation details (operation signal) of the operating device 26 to the control valve 17, and The operation of the excavator 100 based on the above can be realized.

The shuttle valve 32 has two inlet ports and one outlet port, and outputs hydraulic fluid having a higher pilot pressure of the pilot pressures input to the two inlet ports to the outlet port. The shuttle valve 32 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26 and for each drive direction of the driven element (hydraulic actuator HA). One of the two inlet ports of the shuttle valve 32 is connected to the pilot line 27A on the secondary side of the operating device 26 (specifically, the above-mentioned lever device or pedal device included in the operating device 26), and the other is It is connected to the pilot line 27B on the secondary side of the hydraulic control valve 31. The outlet port of shuttle valve 32 is connected to the pilot port of the corresponding control valve of control valve 17 through pilot line 27 . The corresponding control valve is a control valve that drives a hydraulic actuator that is operated by the above-mentioned lever device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, these shuttle valves 32 each control the higher of the pilot pressure in the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure on the pilot line 27B on the secondary side of the hydraulic control valve 31, respectively. It can act on the pilot port of the control valve. In other words, the controller 30 controls the corresponding control valve by causing the hydraulic control valve 31 to output a pilot pressure higher than the pilot pressure on the secondary side of the operating device 26, regardless of the operator's operation on the operating device 26. be able to. Therefore, the controller 30 can control the operation of the driven elements (the lower traveling body 1, the upper rotating body 3, the attachment AT) and realize a remote control function, regardless of the operation state of the operating device 26 by the operator. .

The hydraulic control valve 33 is provided in the pilot line 27A that connects the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured to be able to change its flow path area, for example. The hydraulic control valve 33 operates according to a control signal input from the controller 30. Thereby, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is being operated by the operator. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. Further, the controller 30 can reduce the pilot pressure output from the operating device 26 to be lower than the pilot pressure output from the hydraulic control valve 31, for example, even when the operating device 26 is being operated. Can be done. Therefore, by controlling the hydraulic control valve 31 and the hydraulic control valve 33, the controller 30 applies a desired pilot pressure to the pilot port of the control valve in the control valve 17, for example, regardless of the operation details of the operating device 26. It can be made to work reliably. Therefore, the controller 30 can more appropriately realize the remote control function of the excavator 100 by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31, for example.

≪User interface system≫
As shown in FIG. 3, the user interface system of excavator 100 includes an operating device 26, an output device 50, and an input device 52.

The output device 50 outputs various information to the user of the excavator 100 (for example, the operator in the cabin 10 or an external remote control operator) and the people around the excavator 100 (for example, a worker or a driver of a work vehicle). Output.

For example, the output device 50 includes lighting equipment, a display device 50A, etc. that output various information in a visual manner. The lighting equipment is, for example, a warning light (indicator lamp) or the like. The display device 50A is, for example, a liquid crystal display or an organic EL (Electroluminescence) display. For example, as shown in FIG. 2, lighting equipment and a display device 50A may be provided inside the cabin 10 and output various information visually to an operator inside the cabin 10. Further, the lighting equipment and the display device 50A may be provided, for example, on the side surface of the revolving upper structure 3, and may output various information visually to workers and the like around the excavator 100.

Further, for example, the output device 50 includes a sound output device 50B that outputs various information in an auditory manner. The sound output device 50B includes, for example, a buzzer, a speaker, and the like. The sound output device 50B is provided, for example, in at least one of the interior and exterior of the cabin 10, and outputs various information in an auditory manner to the operator inside the cabin 10 and the people (workers, etc.) around the excavator 100. It's fine.

Further, for example, the output device 50 may include a device that outputs various information using a tactile method such as vibration of the cockpit.

The input device 52 accepts various inputs from the user of the excavator 100, and signals corresponding to the accepted inputs are taken into the controller 30. For example, as shown in FIG. 2, the input device 52 is provided inside the cabin 10 and receives input from an operator or the like inside the cabin 10. Further, the input device 52 may be provided, for example, on a side surface of the revolving upper structure 3, and may receive input from a worker or the like around the excavator 100.

For example, the input device 52 includes an operation input device that receives mechanical input from the user. The operation input device may include a touch panel mounted on the display device, a touch pad installed around the display device, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), etc. .

Furthermore, for example, the input device 52 may include a voice input device that accepts a user's voice input. The audio input device includes, for example, a microphone.

Furthermore, for example, the input device 52 may include a gesture input device that accepts gesture input from the user. The gesture input device includes, for example, an imaging device that captures an image of a gesture performed by a user.

Furthermore, for example, the input device 52 may include a biometric input device that receives biometric input from the user. The biometric input includes, for example, input of biometric information such as a user's fingerprint or iris.

≪Communication system≫
As shown in FIG. 3, the communication system of the excavator 100 according to this embodiment includes a communication device 60.

The communication device 60 is connected to an external communication line and communicates with a device provided separately from the excavator 100. Devices provided separately from the excavator 100 may include devices external to the excavator 100 as well as portable terminal devices (portable terminals) brought into the cabin 10 by the user of the excavator 100. The communication device 60 may include, for example, a mobile communication module that complies with standards such as 4G ( ^4th Generation) and 5G ( ^5th Generation). Further, the communication device 60 may include, for example, a satellite communication module. Further, the communication device 60 may include, for example, a WiFi communication module, a Bluetooth (registered trademark) communication module, or the like. Furthermore, the communication device 60 may include a plurality of communication devices depending on the communication lines to be connected.

For example, the communication device 60 communicates with external devices such as the remote operation support device 200 within the work site through a local communication line constructed at the work site. The local communication line is, for example, a local 5G (so-called local 5G) mobile communication line built at a work site or a WiFi 6 local network (LAN: Local Area Network).

Further, for example, the communication device 60 communicates with the remote operation support device 200 and the like located outside the work site through a wide area communication line that includes the work site, that is, a wide area network (WAN). The wide area network includes, for example, a wide area mobile communication network, a satellite communication network, an Internet network, and the like.

≪Control system≫
As shown in FIG. 3, the control system of excavator 100 includes a controller 30. Further, the control system of the excavator 100 according to the present embodiment includes an operating pressure sensor 29, imaging devices 40 and 45, a positioning device 70, and sensors S1 to S5.

The controller 30 performs various controls regarding the shovel 100.

The functions of the controller 30 may be realized by arbitrary hardware or a combination of arbitrary hardware and software. For example, as shown in FIG. 3, the controller 30 includes an auxiliary storage device 30A, a memory device 30B, a CPU (Central Processing Unit) 30C, and an interface device 30D, which are connected via a bus B1.

The auxiliary storage device 30A is a non-volatile storage means, and stores installed programs as well as necessary files, data, and the like. The auxiliary storage device 30A is, for example, an EEPROM (Electrically Erasable Programmable Read-Only Memory) or a flash memory.

For example, when there is an instruction to start a program, the memory device 30B loads the program in the auxiliary storage device 30A so that the CPU 30C can read the program. The memory device 30B is, for example, an SRAM (Static Random Access Memory).

For example, the CPU 30C executes a program loaded into the memory device 30B, and implements various functions of the controller 30 according to instructions of the program.

The interface device 30D functions as a communication interface for connecting to a communication line inside the excavator 100, for example. The interface device 30D may include a plurality of different types of communication interfaces depending on the type of communication line to be connected.

Further, the interface device 30D functions as an external interface for reading data from and writing data to the recording medium. The recording medium is, for example, a dedicated tool that is connected to a connector installed inside the cabin 10 with a detachable cable. Further, the recording medium may be a general-purpose recording medium such as an SD memory card or a USB (Universal Serial Bus) memory. Thereby, programs for realizing various functions of the controller 30 can be provided by, for example, a portable recording medium and installed in the auxiliary storage device 30A of the controller 30. Further, the program may be downloaded from another computer outside the excavator 100 through the communication device 60 and installed in the auxiliary storage device 30A.

Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot type operating device 26, that is, the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. To detect. A detection signal of pilot pressure corresponding to the operating state of each driven element (hydraulic actuator HA) in the operating device 26 by the operating pressure sensor 29 is taken into the controller 30.

Note that if the operating device 26 is an electric type, the operating pressure sensor 29 is omitted. This is because the controller 30 can grasp the operating state of each driven element through the operating device 26 based on the operating signal taken in from the operating device 26.

The imaging device 40 acquires images around the excavator 100. The imaging device 40 also generates three-dimensional data (hereinafter simply referred to as "the object's three-dimensional shape") representing the position and external shape of the object around the shovel 100 within the imaging range (angle of view) based on the acquired image and distance-related data described below. "original data") may be obtained (generated). The three-dimensional data of objects around the shovel 100 is, for example, coordinate information data of a point group representing the surface of the object, distance image data, and the like.

For example, as shown in FIG. 2, the imaging device 40 includes a camera 40F that images the front of the upper revolving structure 3, a camera 40B that images the rear of the upper revolving structure 3, and a camera 40L that images the left side of the upper revolving structure 3. , and a camera 40R that images the right side of the upper rotating body 3. Thereby, the imaging device 40 can image the entire circumference of the excavator 100, that is, the range covering the angular direction of 360 degrees, when the excavator 100 is viewed from above. In addition, the operator visually recognizes peripheral images such as captured images of the cameras 40B, 40L, and 40R and processed images generated based on the captured images through the display device 50A and the remote operation support device 200 (display device 208). The left, right, and rear sides of the rotating body 3 can be confirmed. In addition, the operator can check the operation of the attachment AT by visually checking peripheral images such as images captured by the camera 40F and processed images generated based on the captured images through the remote operation support device 200 (display device 208). However, the excavator 100 can be remotely controlled. Hereinafter, the cameras 40F, 40B, 40L, and 40R may be collectively or individually referred to as "camera 40X."

Camera 40X is, for example, a monocular camera. In addition, the camera 40X acquires data regarding distance (depth) in addition to two-dimensional images, such as a stereo camera, a TOF (Time Of Flight) camera, etc. (hereinafter collectively referred to as a "3D camera"). It may be possible.

Output data (for example, image data, three-dimensional data of objects around the excavator 100, etc.) from the imaging device 40 (camera 40X) is taken into the controller 30 through a one-to-one communication line or an in-vehicle network. Thereby, for example, the controller 30 can monitor objects around the excavator 100 based on the output data of the camera 40X. Further, for example, the controller 30 can determine the surrounding environment of the excavator 100 based on the output data of the camera 40X. Further, for example, the controller 30 can determine the posture state of the attachment AT shown in the captured image based on the output data of the camera 40X (camera 40F). Further, for example, the controller 30 can determine the attitude state of the body of the excavator 100 (the upper revolving body 3) based on the output data of the camera 40X, with reference to objects around the excavator 100.

Note that some or all of the cameras 40F, 40B, 40L, and 40R may be omitted. For example, camera 40F may be omitted. The captured image of the imaging device 45, which will be described later, shows the state in front of the excavator 100. For example, the operator visually recognizes the captured image of the imaging device 45 and peripheral images such as processed images generated based on the captured image. This is because the excavator 100 can be remotely controlled while checking the operation of the attachment AT. Further, instead of or in addition to the imaging device 40 (camera 40X), a distance sensor may be provided in the upper revolving body 3. The distance sensor is attached to the upper part of the upper revolving body 3, for example, and acquires data regarding the distance and direction of surrounding objects with respect to the shovel 100 as a reference. Further, the distance sensor may acquire (generate) three-dimensional data (for example, coordinate information data of a point group) of objects around the shovel 100 within the sensing range based on the acquired data. The distance sensor is, for example, LIDAR (Light Detection and Ranging). Further, for example, the distance sensor may be, for example, a millimeter wave radar, an ultrasonic sensor, an infrared sensor, or the like.

The imaging device 45 is attached to the attachment AT, and images the work object around the shovel 100 to obtain a captured image representing the shape of the work object. Thereby, the imaging device 45 changes the imaging range in the far and near direction as seen from the shovel 100 in accordance with the operation of the attachment AT. Therefore, the imaging device 45 can acquire images of the work target over a wide range in the far and near directions as seen from the shovel 100 in accordance with the operation of the attachment AT. The work target around the shovel 100 is, for example, the ground at a target location where excavation work, land leveling work, etc. are performed at the work site.

Also, like the imaging device 40, the imaging device 45 acquires three-dimensional data of objects (work target) around the excavator 100 within the imaging range (angle of view) based on the acquired image and distance-related data described below. generation).

For example, as shown in FIGS. 1 and 2, the imaging device 45 is attached to the side of the arm 5, and is installed with its lens facing the ground. Further, the imaging device 45 may be attached to the lower surface of the arm 5, with its lens facing the ground. The lower surface of the arm 5 means the downward end surface of the arm 5 in a fully open state. Thereby, the imaging device 45 can acquire an image of the ground (work target) below the arm 5. Moreover, the imaging device 45 may be attached to the side surface or the bottom surface of the boom 4, and may be installed with its lens facing the ground. The lower surface of the boom 4 means the downward end surface when the boom 4 is in its lowest position.

The attachment location of the imaging device 45 on the arm 5 is, for example, a location closer to the connection position between the arm 5 and the bucket 6 than the connection position between the arm 5 and the boom 4. As a result, the amount of change in the imaging range in the far and near direction as seen from the excavator 100, which changes according to the movement of the boom 4 and arm 5, becomes relatively large. An image of the target can be obtained. Further, the attachment location of the imaging device 45 on the arm 5 may be closer to the connection location between the arm 5 and the boom 4 than the connection location between the arm 5 and the bucket 6. Thereby, it is possible to suppress adhesion of dirt to the imaging device 45 and damage to the imaging device 45 during operation of the excavator 100.

The imaging device 45 is, for example, a monocular camera. Further, the imaging device 45 may be capable of acquiring data regarding distance (depth) in addition to a two-dimensional image, such as a 3D camera, for example.

Output data of the imaging device 45 (for example, image data, three-dimensional data of objects around the excavator 100, etc.) is taken into the controller 30 through a one-to-one communication line or an in-vehicle network.

Note that instead of or in addition to the imaging device 45, a distance sensor may be provided on the attachment AT. The distance sensor is attached, for example, to the upper part of the revolving upper structure 3, and acquires data regarding the distance and direction of surrounding objects based on itself or its attachment position to the attachment AT. Further, the distance sensor may acquire (generate) three-dimensional data (for example, coordinate information data of a point group) of an object within the sensing range based on the acquired data. The distance sensor is, for example, LIDAR. Further, for example, the distance sensor may be, for example, a millimeter wave radar, an ultrasonic sensor, an infrared sensor, or the like.

The positioning device 70 outputs position information data of the excavator 100. The positioning device 70 may include, for example, a GNSS (Global Navigation Satellite System) sensor. Thereby, the positioning device 70 can acquire data on the position information of the excavator 100 in global coordinates such as the world geodetic system, for example. Further, the positioning device 70 may include, for example, a direction detection device that detects the direction of the shovel 100 and a speed detection device that detects the moving speed of the shovel 100. Thereby, the positioning device 70 can grasp the movement history of the shovel 100 at the work site from the direction and movement speed of the shovel 100, and can acquire data on the relative position of the shovel 100 with respect to the movement start point. Furthermore, the positioning device 70 may include a communication device that can communicate with one or more communication devices (beacons) installed at the work site. Thereby, the positioning device 70 can acquire data on the relative position within the work site depending on the status of communication with the beacon installed at the work site, the status of communication strength, and the like. The output of the positioning device 70 is taken into the controller 30 through a one-to-one communication line, an in-vehicle network, or the like. Thereby, the controller 30 can grasp the absolute position of the shovel 100 and the relative position at the work site.

The sensor S1 is attached to the boom 4, and detects the attitude angle (hereinafter referred to as "boom angle") around the rotation axis of the base end of the boom 4, which corresponds to the connection part with the upper revolving structure 3. The sensor S1 includes, for example, a rotary potentiometer, a rotary encoder, an acceleration sensor, an angular acceleration sensor, a 6-axis sensor, an IMU (Inertial Measurement Unit), and the like. Hereinafter, the same may be applied to the sensor S2 and the sensor S4. Further, the sensor S1 may include a cylinder sensor that detects the extended/contracted position of the boom cylinder 7. Hereinafter, the same may be applied to the sensor S2. A detection signal of the boom angle by the sensor S1 is taken into the controller 30. Thereby, the controller 30 can grasp the attitude state of the boom 4.

The sensor S2 is attached to the arm 5, and detects the attitude angle (hereinafter referred to as "arm angle") around the rotation axis of the base end of the arm 5, which corresponds to the connection part with the boom 4. The arm angle detection signal from the sensor S2 is taken into the controller 30. Thereby, the controller 30 can grasp the posture state of the arm 5.

The sensor S3 is attached to the bucket 6 and detects the posture angle (hereinafter referred to as "arm angle") around the rotation axis of the proximal end of the bucket 6, which corresponds to the connection part with the arm 5. A detection signal of the arm angle by the sensor S3 is taken into the controller 30. Thereby, the controller 30 can grasp the attitude state of the bucket 6.

The sensor S4 detects the tilt state of the aircraft body (for example, the upper revolving body 3) with respect to a predetermined reference plane (for example, a horizontal plane). For example, the sensor S4 is attached to the revolving upper structure 3, and measures the inclination angle of the excavator 100 (i.e., the revolving upper structure 3) about two axes in the front-rear direction and the left-right direction (hereinafter, "front-rear inclination angle" and "lateral inclination angle"). ”) is detected. A detection signal corresponding to the inclination angle (front/rear inclination angle and left/right inclination angle) detected by the sensor S<b>4 is taken into the controller 30 . Thereby, the controller 30 can grasp the tilting state of the aircraft body (upper rotating body 3).

The sensor S5 is attached to the revolving upper structure 3 and outputs detection information regarding the turning state of the revolving upper structure 3. The sensor S5 detects, for example, the turning angular velocity and turning angle of the upper rotating body 3. The sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, and the like. Detection information regarding the turning state detected by the sensor S5 is taken into the controller 30. Thereby, the controller 30 can grasp the turning state such as the turning angle of the upper rotating body 3.

In addition, when the sensor S4 includes a gyro sensor, a 6-axis sensor, an IMU, etc. that can detect angular velocity around three axes, the turning state (for example, turning angular velocity) of the upper rotating structure 3 is detected based on the detection signal of the sensor S4. may be done. In this case, sensor S5 may be omitted. Further, if it is possible to grasp the attitude state of the upper rotating body 3, attachment AT, etc. based on the output of the imaging device 40 or the distance sensor, at least some of the sensors S1 to S5 may be omitted.

≪Hardware configuration of remote operation support device>
FIG. 4 is a block diagram showing an example of the hardware configuration of the remote operation support device 200.

The functions of the remote operation support device 200 are realized by arbitrary hardware or a combination of arbitrary hardware and software. For example, as shown in FIG. 5, the remote operation support device 200 includes an external interface 201, an auxiliary storage device 202, a memory device 203, a CPU 204, a high-speed arithmetic device 205, a communication interface 206, and an input device 207, which are connected via a bus B2. , a display device 208, and a sound output device 209.

The external interface 201 functions as an interface for reading data from and writing data to the recording medium 201A. The recording medium 201A includes, for example, a flexible disk, a CD (Compact Disc), a DVD (Digital Versatile Disc), a BD (Blu-ray (registered trademark) Disc), an SD memory card, a USB memory, and the like. Thereby, the remote operation support device 200 can read various data used in processing through the recording medium 201A, store it in the auxiliary storage device 202, and install programs that implement various functions.

Note that the remote operation support device 200 may obtain various data and programs used in processing from an external device through the communication interface 206.

The auxiliary storage device 202 stores various installed programs, as well as files, data, etc. necessary for various processes. The auxiliary storage device 202 includes, for example, an HDD (Hard Disc Drive), an SSD (Solid State Disc), a flash memory, and the like.

The memory device 203 reads the program from the auxiliary storage device 202 and stores it when there is an instruction to start the program. The memory device 203 includes, for example, DRAM (Dynamic Random Access Memory) and SRAM.

The CPU 204 executes various programs loaded into the memory device 203 from the auxiliary storage device 202, and implements various functions related to the remote operation support device 200 according to the programs.

The high-speed arithmetic unit 205 works in conjunction with the CPU 204 and performs arithmetic processing at a relatively high speed. The high-speed calculation device 205 includes, for example, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), and the like.

Note that the high-speed arithmetic unit 205 may be omitted depending on the required speed of arithmetic processing.

The communication interface 206 is used as an interface for communicably connecting to an external device. Thereby, the remote operation support device 200 can communicate with an external device such as the excavator 100, for example, through the communication interface 206. Furthermore, the communication interface 206 may have a plurality of types of communication interfaces depending on the communication method with the connected device.

The input device 207 receives various inputs from the user. The input device 207 includes a remote control operating device for remotely controlling the shovel 100.

The input device 207 includes, for example, an input device (hereinafter referred to as an "operation input device") that accepts mechanical operation input from a user. The operating device for remote control may be an operating input device. The operation input device includes, for example, a button, a toggle, a lever, and the like. Further, the operation input device includes, for example, a touch panel mounted on the display device 208, a touch pad provided separately from the display device 208, and the like.

Furthermore, the input device 207 includes, for example, a voice input device that can accept voice input from a user. The voice input device includes, for example, a microphone that can collect the user's voice.

Furthermore, the input device 207 includes, for example, a gesture input device that can accept gesture input from a user. The gesture input device includes, for example, a camera that can capture images of the user's gestures.

Furthermore, the input device 207 includes, for example, a biometric input device that can accept biometric input from a user. The biometric input device includes, for example, a camera that can acquire image data that includes information about a user's fingerprint or iris.

The display device 208 displays information screens and operation screens for the user. The display device 208 is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, or the like.

The sound output device 209 conveys various information to the user of the remote operation support device 200 through sound. The sound output device 209 is, for example, a buzzer, an alarm, a speaker, or the like.

[Functional configuration of remote operation support system]
Next, the functional configuration of the remote operation support system SYS will be explained with reference to FIGS. 5 to 9.

FIG. 5 is a functional block diagram showing an example of the functional configuration of the remote operation support system SYS. FIG. 6 is a diagram showing a first example (screen 600) of a display screen of an environmental map representing the shape of a work target around the shovel 100. FIG. 7 is a diagram showing a second example (screen 700) of a display screen of an environmental map showing the shape of a work target around the shovel 100. FIG. 8 is a diagram showing a third example (screen 800) of a display screen of an environmental map representing the shape of a work target around the shovel 100. FIG. 9 is a diagram showing a fourth example (screen 900) of a display screen of an environmental map representing the shape of a work target around the shovel 100.

As shown in FIG. 5, the controller 30 of the shovel 100 includes a shovel data transmitting section 301 and an operation control section 302 as functional sections.

The shovel data transmitter 301 uses the communication device 60 to transmit various data acquired by the shovel 100 (hereinafter referred to as "shovel data") to the remote operation support device 200. The shovel data transmitter 301 may automatically transmit the shovel data to the remote operation support device 200 at predetermined intervals, or may send the shovel data to the remote operation support device 200 in response to a request from the remote operation support device 200. You may also send it to

The shovel data to be transmitted includes, for example, data representing an object (work target) around the shovel 100 (hereinafter referred to as "work target data" for convenience), which is output from the imaging device 45. Thereby, the remote operation support device 200 can recognize the shape of the work target around the shovel 100. Further, the shovel data to be transmitted may include data on the position information of the shovel 100 (hereinafter referred to as "shovel position data") output from the positioning device 70. Thereby, the remote operation support device 200 can recognize the absolute position of the shovel 100 and the relative position at the work site. Further, the shovel data to be transmitted may include at least some output data of the sensors S1 to S5 (hereinafter referred to as "shovel posture data"). Thereby, the remote operation support device 200 can recognize the posture state of the driven element (hydraulic actuator HA) of the excavator 100 and its change. Further, the shovel data to be transmitted may include image data output from the imaging device 40, peripheral image data generated based on the output of the imaging device 40, and the like. Thereby, the remote operation support device 200 can recognize, for example, the presence or absence of an object to be monitored around the excavator 100 and its state.

The operation control unit 302 controls the operation of the excavator 100 corresponding to the remote control by the remote control support device 200 in response to a remote control signal received from the remote control support device 200 . Specifically, the operation control unit 302 controls the hydraulic control valve 31 to implement the content of remote control of the shovel 100 specified by the remote control signal.

The remote operation support device 200 includes a shovel data acquisition unit 2001, an environmental map generation unit 2002, a display processing unit 2003, a trajectory generation unit 2004, a trajectory data storage unit 2005, a simulator unit 2006, and a remote operation support device 200 as functional units. and an operation support section 2007.

The shovel data acquisition unit 2001 acquires shovel data received from the shovel 100.

The environmental map generation unit 2002 generates an environmental map (hereinafter simply referred to as “environmental map”) representing the shape of the work target around the shovel 100 based on the shovel data acquired by the shovel data acquisition unit 2001. Specifically, the environmental map generation unit 2002 may generate an environmental map in real time based on the latest shovel data acquired by the shovel data acquisition unit 2001. The environmental map may be generated in a global coordinate system such as a world geodetic system, or may be generated in a local coordinate system. For example, if the positioning device 70 is capable of acquiring position information data in the global coordinate system, the environmental map generation unit 2002 generates an environmental map in the global coordinate system using the position of the shovel 100 in the global coordinate system as a reference. Further, for example, if the positioning device 70 is capable of acquiring position information data in the local coordinate system, the environmental map generation unit 2002 generates an environmental map in the local coordinate system based on the position of the excavator 100 in the local coordinate system. do.

The environmental map generation unit 2002 generates an environmental map based on the work target data included in the shovel data. At this time, the environmental map generation unit 2002 specifies the imaging range of the imaging device 45 attached to the arm 5 as seen from the shovel 100 by specifying the posture state of the arm 5 based on the shovel posture data included in the shovel data. can do. Therefore, the environmental map generation unit 2002 can recognize the position of the work target with respect to the shovel 100 included in the work target data corresponding to the output data of the imaging device 45, and can generate an environmental map.

Furthermore, the environmental map generation unit 2002 may generate an environmental map based on work target data when the boom 4 and the arm 5 are operated. For example, the environmental map generation unit 2002 may generate an environmental map based on a plurality of work target data obtained at different timings when the boom 4 or the arm 5 is operated. The environmental map generation unit 2002 recognizes changes in the postures of the boom 4 and arm 5 based on the excavator posture data, thereby determining whether the work target data acquired at the same timing is data when the boom 4 or arm 5 is in operation. It is possible to determine whether it is present or not. As a result, the environmental map generation unit 2002 uses work target data of a wider range in the far and near directions as seen from the excavator 100, which is acquired by the imaging device 45 in accordance with the movements of the boom 4 and the arm 5. It is possible to generate an environmental map of a wider area around the area.

Furthermore, the environmental map generation unit 2002 may update the environmental map according to the operations of the boom 4 and arm 5 of the excavator 100. This is because the imaging range of the imaging device 45 changes depending on the operation of the boom 4 and arm 5 of the excavator 100, and the image captured by the imaging device 45 may include a work target area that is not included in the environmental map. It is. Further, the work progresses due to the movement of the attachment AT in accordance with the movement of the boom 4 and arm 5 of the excavator 100, and the shape of the work target (j ground) may change. For example, each time the boom 4 or arm 5 of the excavator 100 is operated, the environmental map generation unit 2002 updates the environmental map based on the data of the work being performed.

Furthermore, the environmental map generation unit 2002 may update the environmental map in accordance with the progress of work on the work target of the shovel 100. For example, the environmental map generation unit 2002 determines whether or not the bucket 6 is in contact with the work target based on the latest environmental map and shovel posture data, thereby determining the progress of work on the work target. , determine whether there is a change in the shape of the work target. Then, when determining that there is a change in the shape of the work target, the environmental map generation unit 2002 may update the environmental map based on the work target data when the boom 4 or arm 5 corresponding to the work is operated. .

Furthermore, the environmental map generation unit 2002 may update the environmental map according to the operation of the lower traveling structure 1 and the operation of the upper rotating structure 3. The imaging range of the imaging device 45 changes according to the operation of the lower traveling body 1 and the upper revolving structure 3 of the excavator 100, and the image captured by the imaging device 45 may include a work target area that is not included in the environmental map. This is because of its nature. Further, as the work progresses due to the operation of the lower traveling body 1 and the upper rotating body 3 of the shovel 100, the shape of the work target (ground) around the shovel 100 may change. For example, the environmental map generation unit 2002 updates the environmental map every time the lower traveling body 1 or the upper revolving body 3 of the excavator 100 is operated, based on the work target data during the operation.

Furthermore, the environmental map generation unit 2002 sends a command to the excavator 100 through the communication interface 206 and causes the excavator 100 to perform a predetermined operation of the attachment AT, thereby creating an environmental map based on the shovel data at the time of the predetermined operation. may be generated. For example, when the excavator 100 starts working, the environmental map generation unit 2002 sends a command to the excavator 100 to perform a predetermined operation via the communication interface 206 in response to a predetermined input from an operator via the input device 207. Send to 100. The environmental map generation unit 2002 then generates an environmental map based on the shovel data during a predetermined operation, which is received from the shovel 100 via the communication interface 206. Thereby, the remote operation support device 200 can generate (obtain) an environmental map representing the initial state of the work target around the shovel 100 when the shovel 100 starts working. Furthermore, if the operating speed of the boom 4 or the arm 5 becomes extremely high, it may not be possible to obtain work target data that can be used to update the environmental map. In this case as well, the environmental map generation unit 2002 generates work target data that can be used to update the environmental map by transmitting a command for causing the excavator 100 to perform a predetermined operation to the excavator 100 via the communication interface 206. can be obtained.

Furthermore, the environmental map generation unit 2002 may generate and update the environmental map by combining the data of the captured image of the imaging device 40 in addition to the work target data corresponding to the captured image of the imaging device 45.

The display processing unit 2003 displays the image of the environmental map generated by the environmental map generation unit 2002 on the display device 208. Thereby, the operator can remotely operate the shovel 100 while grasping the shape of the work target around the shovel 100 by visually checking the image of the environmental map.

Furthermore, when the environmental map generation unit 2002 updates the environmental map, the display processing unit 2003 updates the image of the environmental map on the display device 208. For example, the display processing unit 2003 automatically updates the image of the environmental map on the display device 208 in accordance with the environmental map update by the environmental map generation unit 2002. Furthermore, when the environmental map generation unit 2002 updates the environmental map and a predetermined input requesting update is received through the input device 207, the display processing unit 2003 displays the image of the environmental map on the display device 208. May be updated.

For example, as shown in FIG. 6, the display processing unit 2003 causes the display device 208 to display an image TG of the environmental map when viewed from a predetermined viewpoint around the excavator 100. Furthermore, the display processing unit 2003 displays an image CG schematically showing the shovel 100 together with an image TG of an environmental map when viewed from a predetermined viewpoint around the shovel 100. At this time, the relative position and size between the image CG of the shovel 100 and the image TG of the environmental map are based on the actual relative position and size between the shovel 100 and the shape of the work target (ground) around it. Adjustments will be made accordingly. Thereby, the operator can understand the shape of the work target around the shovel 100 while considering the positional relationship and size with respect to the shovel 100.

The image CG includes an image CG1 corresponding to the body of the excavator 100 (the lower traveling body 1 and the upper rotating body 3), and an image CG2 corresponding to the attachment AT of the excavator 100. As a result, the operator can select the next operation of the shovel 100 while comparing the shape of the work target around the shovel 100 and the current position of the bucket 6, and as a result, the operator can select the next operation of the shovel 100. Efficiency can be improved.

The viewpoint of the image TG of the environmental map may be fixed or may be varied according to a predetermined input from the operator to the input device 207. For example, if the input device 207 is a touch panel, moving a finger or the like in a single-touch state rotates the viewpoint of the image TG, and moving a finger or the like in a multi-touch state rotates the viewpoint of the image TG. Move in parallel up/down/left/right. For example, if the input device 207 is a mouse, moving the pointer (cursor) while left-clicking will rotate the viewpoint of the image TG, and moving the pointer while right-clicking will rotate the viewpoint of the image TG. moves in parallel. This allows the operator to check the shape of the work target around the shovel 100 from various viewpoints, improving convenience. Further, the operator can select a more appropriate operation of the shovel 100 for the work being performed, and as a result, the work efficiency of the shovel 100 can be improved.

In the image TG of the environmental map, height differences may be expressed by coloring. Thereby, the operator can more appropriately grasp the shape of the work target around the shovel 100 by taking into consideration not only the shape of the work target in the image TG of the environmental map but also the difference in coloring.

Further, as shown in FIG. 7, the display processing unit 2003 may cause the display device 208 to display an image TG of the environmental map as viewed from the cabin 10 of the excavator 100. Thereby, the operator can remotely control the shovel 100 while checking the shape of the work target of the shovel 100 as seen from the cabin 10 of the shovel 100. In this case, the display processing unit 2003 displays a state in which an image TG seen from a viewpoint around the excavator 100 is displayed and an image TG seen from the cabin 10 in accordance with a predetermined input received through the input device 207. You can switch between the two states.

The display processing unit 2003 may cause the display device 208 to display an image CG2 corresponding to the attachment AT of the excavator 100 together with an image TG of the environmental map as viewed from the cabin 10 of the excavator 100. The relative position and size between the image CG2 corresponding to the attachment AT and the image TG of the environmental map are adjusted to the actual relative position and size between the attachment AT and the shape of the work target (ground) around it. Adjustments will be made accordingly. As a result, the operator can select the next operation of the shovel 100 while comparing the shape of the work target around the shovel 100 and the current position of the bucket 6, and as a result, the operator can select the next operation of the shovel 100. Efficiency can be improved.

The trajectory generation unit 2004 generates work for the attachment AT, the relative position of which is defined with respect to the environmental map, in response to an input that specifies the position on the image of the environmental map displayed on the display device 208, which is received through the input device 207. Generate part trajectories. For example, the trajectory generation unit 2004 generates a trajectory of the work part of the attachment AT on the same coordinate space as the environmental map. The work site of the attachment AT is, for example, the toe or back surface of the bucket 6.

For example, if the input device 207 is a touch panel, the operator moves the finger or the like while touching the display position of the image CG2 corresponding to the work area of the attachment AT. Further, for example, when the input device 207 is a mouse, the operator places the pointer on the display position of the image CG2 corresponding to the working part of the attachment AT, right-clicks, and moves the pointer. Thereby, the operator can input an input specifying the position of the environmental map on the image TG through the input device 207, and the trajectory generating unit 2004 specifies the relative position with respect to the environmental map according to the input. The trajectory of the working part of the attachment AT can be generated.

In addition, the trajectory generation unit 2004 applies a known spline function to an input specifying a position on the image of the environmental map displayed on the display device 208, which is received through the input device 207. The trajectory of the working part may be generated. Thereby, the trajectory generation unit 2004 can generate a smoother trajectory of the working part of the attachment AT.

The display processing unit 2003 displays the trajectory of the working part of the attachment AT generated by the trajectory generation unit 2004, superimposed on the image of the environmental map. Thereby, the operator can confirm the generated trajectory of the attachment AT while comparing it with the image of the environmental map. For example, the display processing unit 2003 displays the trajectory of the attachment AT on the display device 208 only after the input specifying the position on the image of the environmental map is completed and the trajectory generation unit 2004 completes generation of the trajectory. . In addition, the display processing unit 2003 displays the temporary trajectory of the attachment AT in real time in accordance with the progress of the input specifying the position on the image of the environmental map, and officially displays the temporary trajectory after the trajectory generation unit 2004 completes trajectory generation. The display may be updated to a new trajectory.

For example, as shown in FIG. 6, in this example, the trajectory generation unit 2004 generates three trajectories of the work site of the attachment AT, and images OG1 to OG3 of the three trajectories are displayed on the screen 600. Further, in this example, images END1 to END3 imitating the buckets 6 are displayed on the screen 600 at positions corresponding to the end points of the three trajectories. Thereby, the operator can easily grasp the end point position of the generated trajectory.

Further, the trajectory generation unit 2004 may modify (edit) the generated trajectory of the work part of the attachment AT in response to an input specifying a position on the image of the environmental map, which is received through the input device 207. Thereby, the operator can generate a formal trajectory by, for example, generating a rough trajectory of the working part of the attachment AT and then editing the trajectory.

For example, if the input device 207 is a touch panel, the operator can modify the position of the constituent points of the trajectory corresponding to the touch position by touching and moving the part of the image of the generated trajectory that he or she wants to edit with his/her finger or the like. do. In addition, the operator can add a component point corresponding to that position to the generated trajectory by long-pressing or double-touching a position near the part to be edited in the image of the generated trajectory with a finger, etc. Good too. For example, if the input device 207 is a mouse, the operator can adjust the pointer position by placing the pointer on the part of the image of the generated trajectory that he or she wants to edit, then right-clicking and moving the pointer. Correct the positions of the constituent points of the trajectory. In addition, the operator can move the pointer to a position near the part of the image of the generated trajectory that he or she wants to edit, and then left-click or double-click to edit the component point corresponding to that position on the generated trajectory. may be added. Thereby, the trajectory generation unit 2004 can edit the generated trajectory in response to an input received through the input device 207 that designates modification or addition of constituent points of the generated trajectory.

For example, the trajectory generation unit 2004 generates a rough trajectory in response to an input received through the input device 207 that specifies a position on an image of an environmental map viewed from a certain viewpoint. Then, the trajectory generation unit 2004 generates a schematic trajectory in response to corrections or additional inputs to constituent points of the schematic trajectory on the image of the environment map seen from a different viewpoint than when generating the schematic trajectory. Edit the trajectory and generate the official trajectory. Thereby, the operator can more appropriately generate a three-dimensional trajectory of the attachment AT in a form that matches the shape of the work target while using images of the environment map seen from a plurality of different viewpoints. For example, the operator uses an overhead image of the environment map viewed from directly above to generate a schematic trajectory in plan view. Then, the operator may generate a formal trajectory by correcting the rough trajectory in the height direction (depth direction) using the image of the environmental map when viewed in the horizontal direction.

The trajectory data storage unit 2005 stores data on the trajectory of the work site of the attachment AT, which is generated by the trajectory generation unit 2004.

The simulator unit 2006 performs a simulation regarding the operation of the shovel 100 for moving the working part of the attachment AT along the trajectory generated by the trajectory generation unit 2004.

The simulation conditions may be settable on the screen on which the environmental map image is displayed, or may be set on a separate setting screen from the screen on which the environmental map image is displayed. The simulation conditions include conditions regarding the operating speed of the shovel 100. The condition regarding the operating speed of the shovel 100 can be set, for example, as the time required (execution time) to move the work part of the attachment AT from the starting point to the ending point along the target trajectory. Further, the conditions regarding the operating speed of the shovel 100 may be settable as the conditions for the moving speed of the working part of the attachment AT.

For example, as shown in FIG. 8, an input field IF1 is used to input the time required (execution time) to move the work part of the attachment AT from the start point to the end point along the trajectory generated by the trajectory generation unit 2004. will be provided. Thereby, the operator can set conditions regarding the operating speed of the shovel 100 by inputting the execution time through the input device 207.

Further, as shown in FIG. 9, an input field IF2 may be provided by a slider to specify the high or low operating speed of the shovel 100. Thereby, the operator can set conditions regarding the operating speed of the excavator 100 by moving the slider left and right through the input device 207.

The display processing unit 2003 superimposes a moving image representing the operation of the attachment AT when moving the work part along the trajectory generated by the trajectory generation unit 2004 on the image of the environmental map, according to the simulation result of the simulator unit 2006. It may also be displayed. Furthermore, the display processing unit 2003 may cause the display device 208 to display a moving image including the operation of the upper rotating body 3 in addition to the operation of the attachment AT, together with the image of the environmental map. Thereby, the operator can confirm the operation of the shovel 100 when moving the work area of the attachment AT along the trajectory already generated by the trajectory generation unit 2004. Therefore, after confirming that the operation of the shovel 100 is appropriate, the operator can remotely control the shovel 100 to actually move the working part of the attachment AT along the trajectory. Therefore, the working efficiency of the shovel 100 can be improved.

For example, as shown in FIGS. 6, 8, and 9, in this example, one frame of the moving image SIM representing the operation of the attachment AT when it is moved along the trajectory corresponding to the image OG1 is displayed. ing.

Further, when an error occurs in the simulation by the simulator unit 2006, the display processing unit 2003 may display a notification corresponding to the error on the display device 208.

For example, the simulator unit 2006 outputs an error if the trajectory to be simulated deviates from the range in which the work part of the attachment AT can move. The movable range of the working part of attachment AT means the range in which attachment AT can be moved by the operation of only the lower traveling body 1, the upper rotating body 3, and the upper rotating body 3 of the attachment AT and the attachment AT. . In this case, the display processing unit 2003 causes the display device 208 to display a notification urging correction of the target trajectory, for example. Additionally, the display processing unit 2003 may display an image representing the movable range of the work area of the attachment AT, superimposed on the image of the environmental map. This allows the operator to easily understand how and to what extent the target trajectory should be corrected.

Further, for example, the simulator unit 2006 outputs an error when the condition regarding the operating speed of the shovel 100 with respect to the trajectory to be simulated exceeds the upper limit of the operating speed of the shovel 100. In this case, the display processing unit 2003 causes the display device 208 to display a notification urging correction of the target trajectory, for example. Further, the display processing unit 2003 may cause the display device 208 to display a notification urging correction of conditions related to the operating speed of the shovel 100. Furthermore, the display processing unit 2003 may cause the display device 208 to display a notification prompting the user to correct at least one of the target trajectory and the conditions regarding the operating speed of the shovel 100.

The remote operation support unit 2007 generates a remote operation signal in response to an instruction for remote operation of the excavator 100 inputted by the operator through the input device 207, and transmits the signal to the excavator 100 through the communication interface 206.

For example, the remote operation support unit 2007 generates a remote operation signal corresponding to the content of the operation for each driven element (hydraulic actuator HA) of the excavator 100 with respect to the remote operation operating device included in the input device 207, and Send to. In this case, the operation control unit 302 of the excavator 100 controls the hydraulic control valve 31 according to the content of the operation for each hydraulic actuator HA, which is specified by the remote control signal received by the communication device 60. Thereby, the operator can cause the excavator 100 to perform a desired operation by operating each hydraulic actuator HA using the remote control operating device.

Further, the remote operation support unit 2007 generates a remote operation signal corresponding to an instruction input received by the input device 207 for moving the work part of the attachment AT along the trajectory generated by the trajectory generation unit 2004, Send to excavator 100. In this case, the remote control signal includes data on the trajectory of the object and data on conditions regarding the operating speed of the shovel 100. Then, the operation control unit 302 of the excavator 100 moves the working part of the attachment AT along the target trajectory under conditions regarding the operating speed of the excavator 100 specified by the remote control signal received by the communication device 60. The hydraulic control valve 31 is controlled as follows. As a result, the operator can instruct the excavator 100 to move the work part of the attachment AT along the target trajectory by simply selecting the trajectory generated by the trajectory generation unit 2004 and inputting instructions through the input device 207. You can make it happen. Therefore, for example, even when a relatively inexperienced operator remotely controls the shovel 100, the shovel 100 can be easily made to perform a desired operation, and as a result, the working efficiency of the shovel 100 can be improved. Can be done.

Note that some or all of the functions of the environmental map generation unit 2002, the trajectory generation unit 2004, and the simulator unit 2006 may be transferred to the excavator 100 (for example, the controller 30). In addition, functions similar to those of the remote operation support device 200 (environmental map generation unit 2002, display processing unit 2003, trajectory generation unit 2004, trajectory data storage unit 2005, and simulator unit 2006) are adopted in the excavator 100 (for example, the controller 30). may be done. As a result, even when an operator riding in the cabin 10 operates the shovel 100, the operator's operation of the shovel 100 can be supported in the same manner as described above. Further, when supporting the operation of the excavator 100 by an operator boarding the cabin 10, functions similar to those of the remote operation support device 200 may be adopted in an external information processing device that is communicably connected to the excavator 100. . In this case, the output device 50 and input device 52 of the shovel 100 function as an interface for supporting the operator's operations, and specific processing for supporting the operator's operations is performed by an external information processing device.

[Effect]
Next, the operation of the support system etc. according to this embodiment will be explained.

In this embodiment, the support system includes an acquisition unit and a display unit. The support system is, for example, the above-mentioned remote operation support system SYS. The acquisition unit is, for example, the above-described imaging device 45 or distance sensor. The display unit is, for example, the display device 208 or the display device 50A described above. Specifically, the acquisition unit is attached to an attachment of the working machine and acquires data regarding the shape of the work target around the working machine. The working machine is, for example, the above-mentioned shovel 100. The attachment is, for example, the above-mentioned attachment AT. The display unit displays an image of an environmental map representing the shape of the work target based on the data acquired by the acquisition unit when the attachment is operated.

Further, in this embodiment, the remote operation support device includes an operation section, a communication section, and a display section. The remote operation support device is, for example, the above-mentioned remote operation support device 200. The operation unit is, for example, a remote control operation device included in the input device 207 described above. The communication unit is, for example, the communication interface 206 described above. The display unit is, for example, the display device 208 described above. Specifically, the operation unit accepts remote control regarding the work machine. The working machine is, for example, the above-mentioned shovel 100. In addition, the communication unit transmits signals regarding the operation state of the operating unit by the operator to the work machine, and also transmits data regarding the shape of the work target around the work machine, which is acquired by the acquisition unit attached to the attachment of the work machine. Receive from machine. The attachment is, for example, the above-mentioned attachment AT. The acquisition unit is, for example, the above-described imaging device 45 or distance sensor. The display unit displays the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data regarding the shape of the work target around the work machine, which is acquired by the acquisition unit when the attachment is operated. Display an image of the environmental map.

Further, in this embodiment, the working machine includes an attachment, an acquisition section, and a display section. The attachment is, for example, the above-mentioned attachment AT. The working machine is, for example, the above-mentioned shovel 100. The acquisition unit is, for example, the above-described imaging device 45 or distance sensor. The display section is, for example, a display device 50A. Specifically, the acquisition unit is attached to the attachment and acquires data regarding the shape of the work target around the work machine. Further, the display unit displays an image of an environmental map representing the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data acquired by the acquisition unit when the attachment is operated.

Further, in this embodiment, the program causes the information processing device to execute the display step. The information processing device is, for example, the above-mentioned remote operation support device 200 or controller 30. The program is, for example, a program installed in the auxiliary storage device 202 described above. Specifically, in the display step, the data is displayed from a predetermined viewpoint around the work machine based on data regarding the shape of the work target around the work machine, which is acquired by an acquisition unit attached to the attachment of the work machine when the attachment is operated. An image of an environmental map representing the shape of the work target when viewed is displayed on the display unit. The working machine is, for example, the above-mentioned shovel 100. The attachment is, for example, the above-mentioned attachment AT. The acquisition unit is, for example, the above-described imaging device 45 or distance sensor. The display unit is, for example, the display device 208 or the display device 50A described above.

Thereby, it is possible to change the data acquisition range in the far and near directions of the work machine by the acquisition unit in accordance with the operation of the attachment. Therefore, it is possible to present to the user an image representing the shape of the work target in a wider range around the work machine.

Furthermore, in this embodiment, the display unit displays an image of an environmental map representing the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data acquired by the acquisition unit when the attachment is operated. You may.

This allows the user, for example, to operate the work machine while checking not only the shape of the work target as seen from the work machine (cockpit), but also the shape of the work target as seen from the surroundings of the work machine. Can be done. Therefore, the user's convenience can be improved, and the working efficiency of the working machine can be improved.

Further, in this embodiment, the remote operation support system or the like may include an input section. The input unit is, for example, the input device 207 or the input device 52 described above. Specifically, the input unit receives input from the user. The display section may change the predetermined viewpoint in response to a predetermined input from the input section.

Thereby, user convenience can be further improved, and the working efficiency of the working machine can be further improved.

Further, in this embodiment, the remote operation support system or the like may include an input section and a generation section. The input unit is, for example, the input device 207 or the input device 52 described above. The generation unit is, for example, the trajectory generation unit 2004 described above. Specifically, the input unit receives input from the user. Then, the generation unit generates a trajectory of the work part of the attachment in response to an input from the input unit specifying a position on the image of the environmental map representing the shape of the work target. The work area of the attachment is, for example, the toe or back surface of the bucket 6 described above.

Thereby, the operator can create a trajectory of the work part of the attachment through the input unit while associating the positional relationship with the shape of the work target around the work machine. Then, the operator can perform an operation to move the work part of the attachment along the created trajectory. Therefore, it is possible to improve convenience for the operator and to improve the working efficiency of the working machine.

Further, in this embodiment, the remote operation support system or the like may include a generation unit. The generation unit is, for example, the trajectory generation unit 2004 described above. Specifically, the generation unit generates the attachment in response to input from the input unit that specifies the position on the image of the environmental map representing the shape of the work target when viewed from a plurality of different predetermined viewpoints. Generate the trajectory of the work part.

This allows the operator to visually check the shape of the work target around the work machine from multiple viewpoints, and use the input unit to create a trajectory for the work part of the attachment by associating the positional relationship with that shape. be able to. Therefore, the operator can more appropriately create a three-dimensional trajectory for the attachment's work site, and perform an operation to move the attachment's work site along the created trajectory. Therefore, it is possible to improve convenience for the operator and to improve the working efficiency of the working machine.

Further, in this embodiment, the generation unit responds to an input from the input unit that specifies the position on the image of the environmental map representing the shape of the work target when viewed from the first viewpoint as the predetermined viewpoint. In addition to generating a rough trajectory of the work area of the attachment, the position on the image of the environmental map representing the shape of the work target when viewed from a second viewpoint different from the first viewpoint as a predetermined viewpoint. The trajectory of the working part of the attachment may be generated by editing a rough trajectory in response to an input from the input unit that specifies.

Thereby, the operator can more appropriately create a three-dimensional trajectory of the work area of the attachment using the input unit.

Further, in this embodiment, the generation unit generates a rough outline of the work area of the attachment in accordance with input from the input unit, which is specified while moving the position on the image of the environmental map representing the shape of the work target. You can generate trajectories. Then, the generation unit edits the constituent points of the rough trajectory in accordance with the input from the input unit, which specifies the position on the image of the environmental map representing the shape of the work target, thereby generating the attachment work. A trajectory of the part may also be generated.

Thereby, the operator can use the input unit to generate a rough trajectory and then modify the rough trajectory to create a trajectory for the work part of the attachment. Therefore, the operator can more appropriately create a three-dimensional trajectory of the work area of the attachment using the input unit.

Furthermore, in the present embodiment, the display unit displays the operation of the work machine when the work part of the attachment moves along the trajectory generated by the generation unit by superimposing it on the image of the environmental map representing the shape of the work target. A simulated moving image may also be displayed.

This allows the operator to check whether the operation of the work machine is appropriate when moving the work part of the attachment along the created target trajectory before actually operating the work machine. Can be done. Therefore, the operator's convenience can be improved, and the working efficiency of the working machine can be improved.

Furthermore, in the present embodiment, if the trajectory generated by the generation section deviates from the movable range of the attachment's working part, the display unit displays a notification prompting the user to correct the trajectory of the attachment's working part. Good too.

This allows the operator to correct the trajectory of the attachment's working part so that it falls within the movable range of the attachment's working part, and perform an operation to move the attachment's working part along the corrected trajectory. Can be done. Therefore, the operator's convenience can be improved, and the working efficiency of the working machine can be improved.

Further, in the present embodiment, the display section displays the trajectory of the attachment when the condition regarding the operating speed of the working machine with respect to the trajectory generated by the generating section, which is received through the input section, exceeds the upper limit of the operating speed of the working machine. A notification may be displayed to prompt modification of conditions related to operating speed of the working machine.

Thereby, the operator can correct the trajectory of the working part of the attachment or modify the conditions regarding the operating speed of the working machine so as not to exceed the upper limit of the operating speed of the working machine. Then, it is possible to perform an operation to move the working part of the attachment along the corrected trajectory, or to perform an operation to move the working part of the attachment under the corrected conditions regarding the operating speed of the working machine. Therefore, the operator's convenience can be improved, and the working efficiency of the working machine can be improved.

Further, in this embodiment, the display unit may update the image representing the environmental map according to the operation of the attachment.

As a result, the image of the environmental map displayed on the display unit can be updated in accordance with changes in the shape of the work target as the work progresses with the work machine.

Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and changes can be made within the scope of the gist described in the claims.

1 Lower traveling body 1C, 1CL, 1CR Crawler 2 Swing mechanism 3 Upper rotating body 4 Boom 5 Arm 6 Bucket 10 Cabin 26 Operating device 30 Controller 30A Auxiliary storage device 30B Memory device 30C CPU
30D Interface device 31 Hydraulic control valve 40 Imaging device 40B, 40F, 40L, 40R Camera 45 Imaging device 50 Output device 50A Display device 50B Sound output device 52 Input device 60 Communication device 70 Positioning device 100 Excavator 200 Remote operation support device 201 External interface 201A Recording medium 202 Auxiliary storage device 203 Memory device 204 CPU
205 High-speed calculation device 206 Communication interface 207 Input device 208 Display device 209 Sound output device 301 Shovel data transmission section 302 Operation control section 2001 Shovel data acquisition section 2002 Environmental map generation section 2003 Display processing section 2004 Trajectory generation section 2005 Trajectory data storage section 2006 Simulator section 2007 Remote operation support section AT Attachment HA Hydraulic actuators S1 to S5 Sensor SYS Remote operation support system

Claims (14)

an acquisition unit that is attached to an attachment of a working machine and acquires data regarding the shape of a work target around the working machine;
a display unit that displays an image of an environmental map representing the shape of the work target based on data acquired by the acquisition unit when the attachment is operated;
support system. The display unit displays an image of an environmental map representing the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data acquired by the acquisition unit when the attachment is operated.
The support system according to claim 1. Equipped with an input section that accepts input from the user,
The display unit changes the predetermined viewpoint in response to a predetermined input from the input unit.
The support system according to claim 2. an input section that accepts input from the user;
a generating unit that generates a trajectory of the work part of the attachment in response to an input from the input unit specifying a position on an image of an environmental map representing the shape of the work target;
The support system according to claim 1. The trajectory of the work part of the attachment is determined in response to an input from the input unit that specifies a position on an image of an environmental map representing the shape of the work target when viewed from a plurality of different predetermined viewpoints. comprising a generation unit that generates
The support system according to claim 3. The generation unit is configured to generate the generation unit in response to an input from the input unit that specifies a position on an image of an environmental map representing the shape of the work target when viewed from a first viewpoint as the predetermined viewpoint. A rough trajectory of the work area of the attachment is generated, and an image of an environmental map representing the shape of the work target when viewed from a second viewpoint different from the first viewpoint as the predetermined viewpoint is generated. generating a trajectory of the working part of the attachment by editing the rough trajectory in response to an input from the input unit specifying a position;
The support system according to claim 5. A rough trajectory of the work area of the attachment is generated in accordance with an input from the input unit, which is specified while moving the position on an image of an environmental map representing the shape of the work target, and The trajectory of the work area of the attachment is edited by editing the constituent points of the rough trajectory in accordance with the input from the input unit that specifies the position on the image of the environmental map representing the shape of the target. comprising a generation unit that generates
The support system according to claim 3. The display unit simulates the operation of the work machine when the work part of the attachment moves along the trajectory generated by the generation unit, superimposed on an image of an environmental map representing the shape of the work target. display video images,
The support system according to claim 4. The display unit displays a notification prompting correction of the trajectory of the working part of the attachment when the trajectory generated by the generating unit deviates from a movable range of the working part of the attachment.
The support system according to claim 4. The display unit is configured to display a trajectory of the attachment when a condition regarding the operating speed of the working machine with respect to the trajectory generated by the generating unit, which is received through the input unit, exceeds an upper limit of the operating speed of the working machine. displaying a notice prompting modification or modification of the conditions regarding the operating speed of the work machine;
The support system according to claim 4. The display unit updates an image representing the environmental map according to the operation of the attachment.
The support system according to claim 1. an operation unit that accepts remote control of the work machine;
A signal regarding the operation state of the operating unit by the operator is transmitted to the working machine, and data regarding the shape of the work object around the working machine, which is acquired by an acquisition unit attached to an attachment of the working machine, is sent to the working machine. a communication unit that receives data from the
An environment that represents the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data regarding the shape of the work target around the work machine, which is acquired by the acquisition unit when the attachment is operated. a display unit that displays an image of the map;
Remote operation support device. attachment and
an acquisition unit that is attached to the attachment and acquires data regarding the shape of a work target around the work machine;
a display unit that displays an image of an environmental map representing the shape of the work target when viewed from a predetermined viewpoint around the work machine, based on data acquired by the acquisition unit during operation of the attachment; ,
working machine. In the information processing device,
The work is performed when viewed from a predetermined viewpoint around the work machine, based on data regarding the shape of the work target around the work machine, which is acquired by an acquisition unit attached to the attachment of the work machine when the attachment is operated. executing a display step of displaying an image of an environmental map representing the shape of the target on the display unit;
program.

Images