JP2024068294A - Control system for work machines - Google Patents

Abstract

[Problem] To operate a work machine appropriately. [Solution] A work machine control system 1000 comprises a work machine 100, a remote control device 400 that remotely operates the work machine 100, and a terminal 300 that limits the operation of the work machine 100, separate from the remote control device 400. Even when remote operation is performed by the remote control device 400, this control system 1000 allows the terminal 300, which is configured separately from the remote control device 400, to limit operation B of the work machine 100, thereby operating the work machine 100 more appropriately from a safety standpoint. [Selected Figure] Figure 1

Description

The present invention relates to a control system for a work machine.

When working with work machines such as excavators, workers carry signal transmitters that emit signals to notify the occupants of the work machines and to control the machine's turning movements, in order to ensure safety (see, for example, Patent Document 1).

Japanese Patent Application Laid-Open No. 4-330123

However, the above-mentioned conventional technology is premised on the assistance of an operator who is on board and operating the work machine, and does not take into consideration ensuring the safety of the work machine that is remotely operated.
The present invention has been made in consideration of the above circumstances, and has an object to operate a remotely operated work machine more appropriately while taking into consideration the safety aspects.

The control system for a work machine according to the present invention comprises:
A work machine;
A remote control device that remotely controls the work machine;
a terminal for limiting the operation of the work machine, separate from the remote control device;
The configuration includes:

The present invention allows remotely operated work machines to be operated more appropriately.

1 is a block diagram showing a schematic configuration of a control system for a shovel according to an embodiment of the present invention. FIG. FIG. 3A is an explanatory diagram showing a coordinate system of a shovel, and FIG. 3B is an explanatory diagram showing a remote control coordinate system in a remote control device. FIG. 2 is a block diagram showing a schematic configuration of a portable terminal. FIG. 2 is a front view of the portable terminal displaying a standby screen. 13 is an example of an input screen displayed in the process of executing the operation restriction process. 13 is an example of an input screen displayed in the process of executing the operation restriction process. 13 is an example of an input screen displayed in the process of executing the operation restriction process. 13 is an example of a status notification screen displayed on an image display device of a remote control device. 11 is a flowchart showing processing from the start of operation restriction processing for one excavator to the release of the operation restriction.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
In this embodiment, a control system for a shovel as a work machine will be illustrated.
FIG. 1 is a block diagram showing a schematic configuration of a control system 1000 for a shovel.
As shown in this figure, the shovel 100 is capable of communicating with a management device 200, a remote control device 400, and a portable terminal 300 via a network NW. Furthermore, the shovel 100 can be operated from the remote control device 400 in an unmanned state.
The control system 1000 may be configured to include a plurality of excavators 100, remote control devices 400, and portable terminals 300.

[Excavator configuration]
FIG. 2 is a side view of the shovel 100 according to this embodiment.
As shown in the figure, the excavator 100 includes a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1 via a rotating mechanism 2, a boom 4, an arm 5, and a bucket 6 as attachments 11, and a cabin 10 as a driver's seat for an operator. The attachment 11 is not limited to this as long as it is provided with a working element (e.g., a bucket, a crusher, a crane device, etc.).

The lower traveling body 1 includes, for example, a pair of left and right crawlers, and each of the crawlers is hydraulically driven by a traveling hydraulic motor (not shown) to cause the excavator 100 to travel.
The upper rotating body 3 rotates relative to the lower traveling body 1 by being driven by a rotating hydraulic motor or an electric motor (neither of which are shown).

The boom 4 is pivotally attached to the center of the front of the upper rotating body 3 so as to be able to be raised and lowered, an arm 5 is pivotally attached to the tip of the boom 4 so as to be able to rotate up and down, and a bucket 6 is pivotally attached to the tip of the arm 5 so as to be able to rotate up and down. The boom 4, the arm 5, and the bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively.
The cabin 10 is mounted, for example, on the front left side of the upper rotating body 3. The excavator 100 operates actuators under the control of a controller 30 to drive driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6.

As shown in FIG. 1 , in addition to the above-mentioned components, the shovel 100 includes an imaging device 40, a distance sensor 41, a movement/posture state sensor 42, a position sensor 43, a direction sensor 44, an operation device 45, an image display device 50, an audio output device 60, a communication device 80, a controller 30, and a memory unit 46.
The shovel 100 can be operated by a remote control device 400, which will be described later. For this reason, the shovel 100 may be configured not to include the cabin 10, the operation device 45, the image display device 50, and the audio output device 60, and may be configured so as to be operable only remotely.

The imaging devices 40 capture images of the periphery of the shovel 100 and output the images to the controller 30. The imaging devices 40 include, for example, a front camera that captures images in front of the shovel 100, a rear camera that captures images behind the shovel 100, a left camera that captures images to the left, and a right camera that captures images to the right. Each imaging device 40 is installed with its optical axis facing diagonally downward, and has an imaging range (angle of view) in the vertical direction that includes the ground near the shovel 100 to the far side of the shovel 100.

The distance sensor 41 is a distance measuring means that measures the distance to objects around the shovel 100 and acquires the information (two-dimensional or three-dimensional distance information), and outputs the acquired information to the controller 30. The distance sensor 41 is provided, for example, in correspondence with the imaging device 40 so as to be able to measure in four directions, the front, rear, left, and right of the shovel 100.

The motion/posture state sensor 42 is a sensor that detects the motion state and posture state of the excavator 100, and outputs the detection result to the controller 30. The motion/posture state sensor 42 includes a boom angle sensor, an arm angle sensor, a bucket angle sensor, a three-axis inertial sensor (IMU: Inertial Measurement Unit), a swing angle sensor, and an acceleration sensor.
These sensors may be composed of sensors that acquire rotational information such as stroke sensors of cylinders on the boom, etc., or rotary encoders, or may be replaced by acceleration (which may also include speed and position) acquired by the IMU.
The boom angle sensor detects the rotation angle of the boom 4 based on a predetermined angle relative to the upper rotating body 3 (hereinafter referred to as the “boom angle”).
The arm angle sensor detects the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the “arm angle”).
The bucket angle sensor detects the rotation angle of the bucket 6 relative to the arm 5 (hereinafter referred to as the “bucket angle”).
The IMUs are attached to each of the boom 4 and the arm 5 and the upper rotating body 3 (at locations other than the attachment 11). The IMUs detect the acceleration of the boom 4, the arm 5 and the upper rotating body 3 along predetermined three axes, and the angular acceleration of the boom 4, the arm 5 and the upper rotating body 3 around the predetermined three axes. The IMU provided on the upper rotating body 3 also detects the inclination angle of the upper rotating body 3 with respect to an absolute horizontal plane.
The rotation angle sensor detects a rotation angle based on a predetermined angular direction of the upper rotating body 3. However, this is not limited to this, and the rotation angle may be detected based on a GNSS or IMU sensor provided on the upper rotating body 3.
The acceleration sensor is attached at a position away from the rotation axis of the upper rotating body 3, and detects the acceleration at that position of the upper rotating body 3. As a result, it can be determined whether the upper rotating body 3 is rotating or the lower traveling body 1 is traveling, etc., based on the detection result of the acceleration sensor.

The position sensor 43 functions as a position information acquisition unit that acquires position information of the work machine, and is equipped on the upper rotating body 3, for example.
The position sensor 43 is a sensor that acquires information on the position (current position) of the shovel 100, and is a GNSS (Global Navigation Satellite System) receiver in this embodiment. The position sensor 43 receives a signal including information on the position of the shovel 100 from a GNSS satellite, and outputs the acquired position information of the shovel 100 to the controller 30. Note that the position sensor 43 does not have to be a GNSS receiver as long as it can acquire information on the position of the shovel 100, and may be one that uses, for example, a satellite positioning system other than GNSS. Furthermore, the position sensor 43 may be provided in the lower traveling body 1.

The orientation sensor 44 is provided on the upper rotating body 3 and is a sensor that acquires information on the orientation (direction) in which the upper rotating body 3 is facing, such as a geomagnetic sensor. The orientation sensor 44 acquires orientation information of the upper rotating body 3 and outputs it to the controller 30. Note that the orientation sensor 44 is not particularly limited as long as it can acquire orientation information of the upper rotating body 3. For example, two GNSS receivers may be provided and orientation information may be acquired from the difference in their position information.

The operation device 45 is provided near the cockpit of the cabin 10, and is an operation means by which the operator operates each operation element (the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, the bucket 6, etc.). In other words, the operation device 45 is an operation means for operating each hydraulic actuator that drives each operation element. The operation device 45 includes, for example, levers, pedals, various buttons, etc., and outputs operation signals according to the contents of these operations to the controller 30.
The operation device 45 is also an operation means for operating the imaging device 40, the distance sensor 41, the motion/posture state sensor 42, the position sensor 43, the image display device 50, the audio output device 60, the communication equipment 80, etc., and outputs operation commands for each of these parts to the controller 30.

The image display device 50 is provided near the cockpit in the cabin 10, and displays various image information to be notified to the operator under the control of the controller 30. The image display device 50 is, for example, a display panel such as an LCD (liquid crystal display), an organic EL (electro luminescence) display, or an inorganic EL display, and may be a touch panel type that also serves as at least a part of the operation device 45.

The audio output device 60 is provided near the cockpit in the cabin 10, and outputs various audio information to notify the operator under the control of the controller 30. The audio output device 60 is, for example, a speaker or a buzzer.

The storage unit 46 is a storage element or storage device such as a non-volatile memory, HDD (Hard Disk Drive), or SSD (Solid State Drive) that stores various types of data and other information in a writable and readable manner.

The communication device 80 is a communication device that transmits and receives various information to and from remote external devices such as the management device 200, the remote control device 400, the portable terminal 300, and other excavators 100 through a specified communication network NW based on a specified wireless communication standard. The communication network NW may include, for example, a mobile communication network with a base station as its terminal, a satellite communication network that uses communication satellites in the sky, a short-range communication network that complies with protocols such as WiFi or Bluetooth (registered trademark), an Internet communication network, etc.

The controller 30 is a control device that controls the operation of each part of the shovel 100 to control the drive of the shovel 100. The controller 30 is mounted inside the cabin 10. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof. The controller 30 is mainly configured with a microcomputer including, for example, a CPU, RAM, ROM, I/O, etc. The controller 30 may also be configured to include, in addition to these, for example, an FPGA or an ASIC.

The controller 30 also includes a storage area defined in an internal memory such as an Electrically Erasable Programmable Read-Only Memory (EEPROM).
This memory area section stores various programs and various data for operating each part of the shovel 100, and also functions as a work area for the controller 30.

The controller 30 of the shovel 100 can communicate with an external management device 200, remote control device 400, and portable terminal 300 through a predetermined communication network NW. The controller 30 transmits and receives information to and from the management device 200 through communication. The controller 30 also controls the operation of each component of the shovel 100 according to commands from the remote control device 400 or portable terminal 300 through communication.

The controller 30 also generates real image data, which is a visual image seen by an operator sitting in the operator's seat of the cabin 10 of the excavator 100. The real image data is transmitted to the remote control device 400, but can also be displayed on the image display device 50.
It should be noted that the generation of actual image data may be performed by a configuration other than the shovel 100 , such as the management device 200 or the remote control device 400 , rather than the controller 30 .

Here, the coordinate system of the shovel 100 will be described with reference to Fig. 3(A). Hereinafter, the coordinate system of the shovel 100 will be referred to as the shovel coordinate system.
The excavator coordinate system is a three-dimensional XYZ Cartesian coordinate system with a reference point O1 on the upper rotating body 3 as the origin, and has an X axis extending parallel to the front-rear direction of the upper rotating body 3, a Y axis extending parallel to the left-right direction of the upper rotating body 3, and a Z axis perpendicular to the X and Y axes. In the example of Fig. 3(A), the origin O1 is on the rotation axis of the upper rotating body 3, and the height can be set to an arbitrary value. In the example of Fig. 3(A), when the excavator 100 is located on a horizontal plane, the X and Y axes are horizontal, and the Z axis is vertical.

The specified viewpoint E1 is a fixed position that is assumed to be the eye position of an operator sitting in the cockpit of the cabin 10 of the shovel 100.
The controller 30 generates real image data of the field of view from the viewpoint E1. The real image data is generated by a process of converting image data captured by the front camera of the imaging device 40 into image data viewed from the viewpoint E1.
In addition, when displaying a real image of the field of view from the viewpoint E1 over a wider range, image data captured by the left and right cameras may be used in addition to the image data captured by the front camera. In this case, the image data captured by each camera is converted into image data captured from the viewpoint E1, and then combined with the converted image data captured by the front camera to generate a single real image data.
Then, the controller 30 transmits the actual image data to the remote control device 400 via the communication device 80 and the network NW.

[Management device]
The management device 200 (an example of an information processing device) is located at a position geographically separated from the shovel 100. The management device 200 is, for example, a server device installed in a management center or the like that is provided outside the work site where the shovel 100 works, and configured mainly with one or more server computers or the like. In this case, the server device may be a company server operated by the business that operates the control system 1000 or an associated business related to the said business, or may be a rental server. In addition, this server device may be a so-called cloud server. In addition, the management device 200 may be a server device (a so-called edge server) that is located in a management office or the like in the work site of the shovel 100, or may be a fixed or portable general-purpose computer terminal.
In addition, the functions of the management device 200 may be provided by an external cloud connected via the network NW.

As described above, the management device 200 can communicate with the shovel 100, the remote control device 400, and the portable terminal 300 via the communication network NW. This allows the management device 200 to receive and store (accumulate) various information uploaded from the shovel 100 and logs related to the shovel 100. In other words, the management device 200 functions as a log recording device.

The management device 200 includes a control unit 210 and a storage unit 220. The control unit 210 loads a program that is pre-stored in the storage unit 220, and executes various processes in cooperation with the loaded program.
The storage unit 220 stores various data and logs 221 , and also functions as a work area for the control unit 210 .

The log 221 recorded by the management device 200 includes, for example, a unique identifier that identifies the shovel 100, the name of the operator (or the operator performing remote operation if remote operation is being performed), operation records when various operations of the shovel 100 are performed, detection information from various sensors possessed by the shovel 100, the time when the log was recorded, the current position of the shovel 100, etc.
In addition, the operation of the shovel 100 may be restricted by commands from the portable terminal 300, and the identifier of the shovel on which the operation restriction was implemented, the name of the operator (or the operator remotely operating the shovel if remote operation is being performed), the date and time of implementation, the position of the shovel 100 at the time of implementation, call records, etc. are also recorded as log 221.

[Remote Control Device]
The remote control device 400 is an operation input device that remotely controls the shovel 100, and one remote control device 400 corresponds to one shovel 100 and remotely controls it.
As shown in FIG. 1, the remote control device 400 includes a driver's seat DS in which an operator OP who performs remote operation sits, an operation device 404 for operating the shovel 100 and performing other operations, and an image display device 403 for displaying an image of the field of view from the cabin 10.
Furthermore, the remote control device 400 includes a controller 410 that performs various processes, a storage unit 402 , a communication device 401 , an audio output device 405 , and a microphone 406 .

The operating device 404 has the same configuration as the operating device 45 of the shovel 100, and is an operating means for operating the respective hydraulic actuators that drive the respective operating elements of the shovel 100.
An operation command for the shovel 100 output from the operating device 404 is transmitted to the controller 30 of the shovel 100 via the communication device 401 and the network NW. Then, the controller 30 executes operation control of each part of the shovel 100 based on the operation command from the operating device 404.

The image display device 403 is provided in front of the driver's seat DS and is a display panel such as an LCD, an organic EL display, or an inorganic EL display that displays an image of the field of view seen from the cabin 10 of the excavator 100.
The image display device can also be configured as a head-mounted display.
The storage unit 402 is a storage element or storage device such as a non-volatile memory, HDD, SSD, etc., that stores various types of data and other information in a writable and readable manner.
The communication equipment 401 is a communication device that transmits and receives various information to and from the management apparatus 200, the shovel 100, and the portable terminal 300 through a predetermined communication network NW based on a predetermined wireless communication standard.

The audio output device 405 is used to output various types of audio information to be notified to the operator OP, and for voice communication with a worker L carrying a portable terminal 300 (described later). The audio output device 405 is, for example, a speaker.
The microphone 406 detects the speech of the operator OP and is used for voice communication with the worker L who carries the portable terminal 300.

The controller 410 is a control device that controls the operation of each part of the shovel 100 through the controller 30 of the shovel 100 to control the drive of the shovel 100. The controller 410 is mainly configured with a microcomputer including, for example, a CPU, RAM, ROM, I/O, etc., and may also be configured to include, in addition to these, for example, an FPGA or an ASIC.

The controller 410 also displays the field of view image from the cabin 10 on the image display device 403 based on the actual image data received from the controller 30 of the excavator 100.

Here, the field of view image that the controller 410 causes the image display device 403 to display will be described.
FIG. 3B is an explanatory diagram showing a remote control coordinate system having a predetermined position on the remote control device 400 as an origin O2.
The remote control coordinate system is a three-dimensional UVW Cartesian coordinate system having a U axis extending parallel to the front-to-rear direction of the driver's seat DS, a V axis extending parallel to the left-to-right direction of the driver's seat DS, and a W axis perpendicular to the U axis and V axis.
The X-axis of the excavator coordinate system corresponds to the U-axis of the remote control coordinate system, the Y-axis corresponds to the V-axis of the remote control coordinate system, and the Z-axis corresponds to the W-axis of the remote control coordinate system.

In this embodiment, each position of the three-dimensional coordinates in the remote control coordinate system is previously associated with each position of the three-dimensional coordinates in the shovel coordinate system in a one-to-one relationship. Therefore, once the three-dimensional coordinates in the remote control coordinate system of viewpoint E1, which is the position of the eyes of the operator OP in the remote control device 400, are determined, the three-dimensional coordinates of viewpoint E1 in the shovel coordinate system are uniquely determined. Note that the position of the viewpoint E1 of the operator OP may be obtained by detecting the positions of the left and right eyes of the operator OP sitting in the driver's seat DS by capturing images with a camera and finding viewpoint E1 from the midpoint between them.

The controller 410 calculates the amount of deviation between the three-dimensional coordinates in the shovel coordinate system of the viewpoint E1 of the operator OP in the remote control device 400 and the three-dimensional coordinates of the viewpoint E1 set on the shovel 100 described above.
Then, the controller 410 performs a viewpoint conversion process so that the actual image data showing the field of view image from the viewpoint E1 of the shovel 100 becomes a field of view image seen from the viewpoint E1 of the operator OP of the remote control device 400.
Then, an image obtained by performing viewpoint conversion processing on the actual image data from the shovel 100 is displayed on the image display device 403 .

When a head-mounted display is used as the image display device, the driver's seat DS and the head-mounted display are equipped with a position sensor such as a GNSS receiver. The head-mounted display is also equipped with a direction sensor that detects its orientation, and an attitude sensor such as an IMU that detects its posture. This allows the viewpoint E1 of the operator OP sitting in the driver's seat DS to be detected from the position, orientation, and posture of the head-mounted display in the remote operation coordinate system, and the viewpoint E1 of the operator OP is converted into the shovel coordinate system. Then, a viewpoint conversion process is performed so that actual image data showing the field of view image from the viewpoint E1 of the shovel 100 becomes the field of view image as seen from the viewpoint E1 of the operator OP, and the image is displayed on the head-mounted display.

As a result, the operator OP of the remote control device 400 can visually perform remote control as if he or she were operating the excavator 100 from the cabin 10.

In addition, when the amount of deviation between the three-dimensional coordinates in the shovel coordinate system of the viewpoint E1 of the operator OP in the remote control device 400 and the three-dimensional coordinates of the viewpoint E1 set on the shovel 100 is small, the field of view image based on the actual image data from the shovel 100 may be displayed directly on the image display device 403.

[Portable devices]
FIG. 4 is a block diagram showing a schematic configuration of the portable terminal 300. As shown in FIG.
The portable terminal 300 functions as a terminal for restricting the operation of the shovel 100 in an emergency from outside the shovel 100 .
The portable terminal 300 can also send commands to multiple excavators 100 to individually restrict their operation.
The portable terminal 300 may be any terminal capable of transmitting an operation restriction command to the controller 30 of the shovel 100, and is preferably portable. For example, a smartphone, a tablet, a portable PC (personal computer), or the like may be used.
Here, a case where a smartphone is used as the portable terminal 300 will be illustrated as an example.

As shown in FIG. 4, the portable terminal 300 includes an image display device 301, an input device 302, a position sensor 303, a direction sensor 304, an attitude sensor 305, an audio output device 306, a microphone 307, a memory unit 308, a communication device 309, and a controller 310.

The image display device 301 has a display panel such as an LCD, an organic EL display, or an inorganic EL display, and displays an input screen on the display screen of the display panel.
The input device 302 is a transparent sensor provided on the surface of the image display device 301, which detects the contact or approach position of a finger or an input pen. The sensor may be of any type, such as a pressure-sensitive type, a capacitance type, or an electromagnetic induction type.

Under the control of the controller 310, the image display device 301 displays an input screen for performing various inputs by contacting or approaching a finger or input pen to the display surface. The input device 302 detects the contact or approach position of the finger or input pen on the input screen, and the controller 310 can recognize the content input by the worker L.

The position sensor 303 is a sensor that acquires information on the current position of the portable terminal 300, and is, for example, a GNSS receiver. The position sensor 303 receives a signal including information on the position of the portable terminal 300 from a GNSS satellite, and outputs the acquired position information of the portable terminal 300 to the controller 310. Note that the position sensor 303 does not have to be a GNSS receiver as long as it can acquire information on the position of the portable terminal 300, and may, for example, use a satellite positioning system other than GNSS.

The orientation sensor 304 is provided inside the housing of the portable terminal 300 and is a sensor that acquires information on the orientation (direction) in which the portable terminal 300 is facing, such as a geomagnetic sensor. The orientation sensor 304 acquires the orientation information of the portable terminal 300 and outputs it to the controller 310. Note that the orientation sensor 304 is not particularly limited as long as it is capable of acquiring the orientation information of the portable terminal 300.

The attitude sensor 305 is a sensor that detects the attitude state of the portable terminal 300 and outputs the detection result to the controller 310. The attitude sensor 305 includes a three-axis inertial sensor (IMU: Inertial Measurement Unit), a rotation angle sensor, and an acceleration sensor. These sensors may be replaced by acceleration (which may also include speed and position) acquired by the IMU.

The audio output device 306 is used when the worker L carrying the portable terminal 300 performs a voice call with the operator OP of the remote control device 400. The audio output device 306 is, for example, a speaker.
The microphone 307 detects the voice of the worker L. The microphone 307 is used when the worker L has a voice call with the operator OP.
The voice output device 306 and the microphone 307 correspond to a means for communicating intentions to the operator OP of the remote operation device 400. The means for communicating intentions is not limited to voice communication, and may be a means for transmitting and receiving text information. It may also be a means for minimally communicating intentions (for example, a lamp displaying yes and no). The same is true for the remote operation device 400 side.

The storage unit 308 is a storage element or storage device such as a non-volatile memory, HDD (Hard Disk Drive), SSD (Solid State Drive), etc., that stores various programs, data, and other information in a writable and readable manner.

The communication device 309 is a communication device that transmits and receives various information to and from remote external devices such as the management device 200, the remote control device 400, and the excavator 100 via the communication network NW based on a specific wireless communication standard.

The controller 310 is mainly configured with a microcomputer including, for example, a CPU, a RAM, a ROM, an I/O, etc. The controller 310 may also be configured to include, in addition to these, for example, an FPGA, an ASIC, etc.
The controller 310 also includes a storage area defined in an internal memory such as an EEPROM.

The controller 310 executes operation restriction processing for the shovel 100 in cooperation with each of the above-mentioned components.
The contents of the operation restriction process will be described with reference to Figures 5 to 10. Figure 5 is a front view of the portable terminal 300 displaying a standby screen G1 that is displayed before and after the operation restriction process is executed, Figures 6 to 8 show various input screens G2 to G4 that are displayed during the process of executing the operation restriction process, and Figure 10 is a flowchart of the operation restriction process for the excavator 100A, which will be described later.

When executing the operation restriction process, the controller 310 can send an operation restriction command to one shovel 100 selected from among multiple shovels 100 that can be connected via the communication network NW, or to all shovels 100 at once.
Here, an example will be given in which three shovels 100 are targets of the operation restriction process, but the number of targets is not limited to this. For convenience, the three shovels 100 may be described separately as shovels 100A, 100B, and 100C.

In addition, the operation restriction that the controller 30 of each shovel 100 is caused to execute in response to an operation restriction command has a plurality of patterns, which will be described later, and one pattern from among these can be selected and set in advance.
The operation restriction patterns include three types: immediate stop of the shovel 100, a speed restriction that restricts the operation speed, and a gradual stop by gradually decreasing the operation speed. Note that the patterns are not limited to these, and more operation restriction patterns may be prepared.
In addition, the target of the operation restriction can be selected from the case where all the operations of the shovel 100 are restricted, and the case where one or a combination of the operations of the traveling, the turning, and the attachment 11 are restricted. These selections can also be set in advance.

Before the operation restriction process is executed, the controller 310 causes the image display device 301 to normally display the standby screen G1.
This standby screen G1 displays execute buttons B1 to B3 for individually executing operational restrictions on multiple shovels 100A to 100C registered on the portable terminal 300, release buttons B4 to B6 for releasing the operational restrictions on each of the shovels 100A to 100C, a collective execute button B7 for collectively executing operational restrictions on all of the shovels 100A to 100C registered on the portable terminal 300, and arrows Y1 to Y3 indicating the location of each of the shovels 100A to 100C.

On the standby screen G1, the arrows Y1 to Y3 correspond to the shovels 100A to 100C individually, and can always point in the direction in which each of the shovels 100A to 100C is located as viewed from the portable terminal 300.
The arrows Y1 to Y3 are displayed adjacent to and above the execution buttons B1 to B3 of the corresponding excavators 100A to 100C, respectively.
The portable terminal 300 and each of the shovels 100A to 100C are each equipped with a position sensor 303, 43. Therefore, the controller 310 requests and acquires current position information detected by the position sensor 43 from the controller 30 of each of the shovels 100A to 100C. This enables the controller 310 to grasp the relative positional relationship between the portable terminal 300 and each of the shovels 100A to 100C.
Furthermore, the portable terminal 300 is equipped with an orientation sensor 304 and an attitude sensor 305, so that it can detect in which direction the display surface of the image display device 301 is currently facing and in which direction it is tilted.
Therefore, by projecting vectors directed from the portable terminal 300 toward each of the shovels 100A-100C onto the display surface of the image display device 301, each of the arrows Y1-Y3 can be displayed pointing in the direction in which the corresponding shovels 100A-100C are located.
When worker L visually checks the shovel 100 that he has determined requires operational restriction, he can identify which of arrows Y1 to Y3 is the arrow whose orientation matches the direction in which the shovel 100 is located.
Furthermore, it is possible to immediately determine which of the execution buttons B1 to B3 will execute the operation restriction of the excavator 100 that should be operation restricted.

When the input device 302 detects a touch operation (contact or approach of a finger or input pen) on any of the execution buttons B1 to B3 on the standby screen G1, the controller 310 moves to a preparation stage for sending an operation restriction command to the controller 30 of the selected shovel 100.
Furthermore, one more step is required from this preparation step until the operation restriction command is actually sent.

In addition, when a touch operation on the release buttons B4 to B6 is detected for any of the shovels 100 to which an operation restriction command has already been sent, the controller 310 moves to a preparation stage for sending a release command to the controller 30 of the corresponding shovel 100.
In this case too, one more process step is required from this preparation step until the release command is actually sent.
The standby screen G1 is provided with only the release buttons B4 to B6 for individually releasing the operation restrictions on each of the shovels 100A to 100C. However, the standby screen G1 may be provided with a collective release button for collectively releasing the operation restrictions on all of the registered shovels 100.

Here, the process flow from selecting the excavator 100A (machine A) on the standby screen G1, sending an operation restriction command, and then releasing the command will be described with reference to the flowchart in FIG. 10.

When a touch operation on the execute button B1 is detected by the input device 302, the controller 310 requests a call connection to the remote control device 400 of the excavator 100A (step S1).
On the other hand, if the line for making a call cannot be connected due to a communication line connection failure or the like on the remote operation device 400 side, the controller 310 advances the process to step S7.
When the line for making a call is connected, the controller 310 causes the image display device 301 to display the input screen G2 shown in Fig. 6 as a pop-up screen (step S3). The input screen G2 is a call connection status screen.

This input screen G2 displays a message indicating that a call with the operator OP who remotely operates the excavator 100A is possible, and a call end button B11 for inputting disconnection of the call line.
The worker L carrying the portable terminal 300 can, for example, explain to the operator OP how the decision to implement operational restrictions on the shovel 100A was reached through the audio output device 306 and the microphone 307, and can receive an explanation from the operator OP as to whether there is an abnormality in the shovel 100A and the cause of the abnormality, etc.

Next, the controller 310 repeatedly determines whether or not the call has been ended until a touch operation on the call end button B11 on the input screen G2 is detected (step S5).
Then, when a touch operation on the call end button B11 is detected, the controller 310 causes the image display device 301 to display the input screen G3 shown in FIG. 7 as a pop-up screen (step S7).
Furthermore, the input screen G3 is also displayed when the line to the excavator 100A cannot be connected in step S1. The input screen G3 is a final confirmation screen for the execution of operational restrictions on the excavator 100A.

This input screen G3 displays a message confirming the decision to send an operation restriction command to the shovel 100A, a confirmation button B21 (marked "YES") indicating the decision to send, and a cancel button B22 (marked "NO") to cancel the sending.
Thus, in the operation restriction process, in principle, an operation restriction command can be sent to the excavator 100A by going through the process of a call with the operator OP.
This prevents unnecessary operational restrictions from being enforced.

When a touch operation on the cancel button B22 on the input screen G3 is detected, the controller 310 ends the operation restriction process without executing the operation restriction.
On the other hand, when a touch operation on the confirmation button B21 is detected, the controller 310 transmits an operation restriction command to the controller 30 of the shovel 100A (step S9).

When the controller 30 of the shovel 100A receives an operation restriction command, it cancels the operation command currently being executed from the remote control device 400 or the like and follows the operation restriction command. That is, depending on the content of the operation restriction command, it executes either an immediate stop, a speed limit, or a gradual stop for the entire operation of the shovel 100A or the operation of the traveling, turning, or attachment 11.

Furthermore, the controller 310 transmits information such as the operation record of the execution of the operation restriction on the shovel 100A, the contents of the operation restriction, the identifier of the shovel 100A, the call record, and the current location of the shovel 100A to the management device 200 (step S11). The management device 200 receives this information and records it as a log 221 together with the information on the shovel 100A held by the management device 200 and the time of reception.

When an operation restriction command for the shovel 100A is transmitted, the controller 310 returns the image display device 301 to a display state of the standby screen G1. At this time, it is preferable that the execution button B1 of the standby screen G1 is displayed in a visually distinguishable manner, for example, in a different color from the other execution buttons B2 and B3 or in a flashing state, until the operation restriction is released, so that it is clear that the operation restriction of the shovel 100A is being executed.

Then, it is determined whether the operation restriction command for the excavator 100A has been released (step S13). That is, the controller 310 repeatedly determines whether the operation restriction command has been released based on the detection of a touch operation on the release button B4 on the standby screen G1.

Then, when a touch operation on the release button B4 is detected, the controller 310 causes the image display device 301 to display the input screen G4 shown in Fig. 8 as a pop-up screen. The input screen G4 is a confirmation screen for releasing the operational restriction on the excavator 100A.
This input screen G4 displays a message confirming the decision to release the operation restriction command for the shovel 100A, a confirmation button B31 (indicated as "YES") indicating the decision to release, a cancel button B32 (indicated as "NO") indicating the cancellation of the release, and a call button B33 for executing a call with the operator OP.

The controller 310 determines whether or not a touch operation has been performed on the cancel button B32 while the input screen G4 is being displayed (step S15).
Then, when a touch operation on the cancel button B32 is detected, the controller 310 returns the process to step S13 and causes the image display device 301 to return to the display state of the standby screen G1.

Furthermore, when a touch operation on the cancel button B32 is not detected, the controller 310 determines whether or not a touch operation on the call button B33 has been performed (step S17).
If a touch operation on the call button B33 is not detected, the process proceeds to step S25.

On the other hand, when a touch operation on the call button B33 is detected, the controller 310 requests a call connection to the remote control device 400 of the excavator 100A (step S19).
If the line for making the call cannot be connected in response to this request, the controller 310 advances the process to step S25.
When the line for making a call has been connected, the controller 310 causes the image display device 301 to display the input screen G2 (FIG. 6) which is a call connection status screen as a pop-up screen (step S21).

Through the audio output device 306 and the microphone 307, the worker L can confirm with the operator OP the status of recovery from the abnormality of the shovel 100A and whether or not to lift the operational restrictions on the shovel 100A.

Next, the controller 310 repeatedly determines whether or not the call has been ended until a touch operation on the call end button B11 on the input screen G2 is detected (step S23).
Then, when a touch operation on the call end button B11 is detected, the controller 310 returns the image display device 301 to a display state of the input screen G4 (FIG. 8), and the process proceeds to step S25.

In step S25, the controller 310 determines whether or not a touch operation has been performed on the confirmation button B31, which indicates that the cancellation has been confirmed.
Then, if a touch operation on the Confirm button B31 is not detected, the process returns to step S15, and it is determined again whether or not a touch operation on the Cancel button B32 has been detected.

On the other hand, when a touch operation on the confirmation button B31 is detected, the controller 310 sends an operation restriction release command to the controller 30 of the excavator 100A (step S27) and ends the operation restriction process.

As a result, the controller 30 of the shovel 100A is released from the operational restrictions, and resumes the operation of the shovel 100A in accordance with operational commands from the remote control device 400 or the like.
At this time, the controller 310 may transmit information such as the release of the operation restriction on the shovel 100A, the identifier of the shovel 100A, the call record, the current location of the shovel 100A, etc. to the management device 200. In this case as well, the management device 200 may record the information on the shovel 100A held by the management device 200 and the reception time as the log 221.

In addition, during the period from the transmission of the operation restriction command for the shovel 100A to the transmission of the operation restriction release command, the controller 310 periodically and repeatedly transmits status information to the remote control device 400 of the shovel 100A, indicating that the shovel 100A is in an operational restriction implementation state.
In response to this, when the controller 410 of the remote operation device 400 of the shovel 100A receives the status information from the controller 310, it causes the image display device 403 to display a status notification screen G5 shown in Fig. 9. The status notification screen G5 is a screen that notifies the operator OP that the operation restriction of the shovel 100A is being implemented. In other words, the controller 410 functions as a presentation means that utilizes the image display device 403 to present the implementation status of the restriction on the operation of the shovel 100.

This status notification screen G5 displays a message indicating that the operation of the shovel 100A is restricted, a message suggesting a call with the worker L on the portable terminal 300, a call button B41 (indicated as "YES") to instruct the user to make the call, and an end button B42 (indicated as "NO") to end the call.
When an operation on the call button B41 is detected, the controller 410 connects the line by requesting a call connection to the portable terminal 300. When an operation on the end button B42 is detected, the controller 410 ends the line connection with the portable terminal 300.

When a communication line connection failure or the like occurs, the remote control device 400 of the shovel 100A may not be able to normally remotely control the shovel 100A, which may cause abnormal behavior of the shovel 100A. In such a case, an operation restriction command from the portable terminal 300 may be issued to restrict the operation of the shovel 100A.
In that case, the operator OP of the remote control device 400 may be unable to make a call to the portable terminal 300 due to a communication line connection problem, and may temporarily be unable to recognize that the shovel 100A is in an operationally restricted state.
In response to this, the controller 310 of the portable terminal 300 can periodically and repeatedly transmit status information to the remote control device 400, thereby allowing the remote control device 400 to receive the status information when the communication line connection is temporarily restored.
Therefore, the operator OP of the remote control device 400 can quickly grasp the status of the excavator 100A, and recovery from operational restrictions can be smoothly carried out.

Although FIG. 10 shows the process from the start of the operation restriction process to the release of the operation restriction on the shovel 100A, the same process is performed on the other shovels 100B and 100C.
Furthermore, while an operation restriction process is being performed on one of the shovels 100A to 100C, there may be cases where it becomes necessary to perform an operation restriction process on the other shovels 100A to 100C.
Even in this case, the input screens G2 to G4 are displayed as pop-up screens that do not occupy the entire display range of the image display device 301, so that the standby screen G1 can be displayed by touching the outside of the input screens G2 to G4. Therefore, by touching the execute buttons B1 to B3, it is possible to quickly execute the operation restriction process in parallel for the other excavators 100A to 100C.

The batch execution button B7 on the standby screen G1 is intended to be used in cases of higher urgency than the other execution buttons B1 to B3.
Therefore, when a touch operation of the collective execution button B7 is detected, the controller 310 quickly transmits an operation restriction command without going through a telephone call process to all of the shovels 100. At the same time, the controller 310 transmits predetermined information of each shovel 100 to the management device 200 to cause recording of the log 221.

In addition, after the operation restriction on all excavators 100 is implemented using the batch execution button B7, it can be released individually using each of the release buttons B4 to B6 on the standby screen G1. The process after each of the release buttons B4 to B6 is touched is the same as the process from step S13 onwards in the flowchart of FIG. 10 described above.

Technical Effects of the Invention Embodiments
As described above, the shovel control system 1000 is equipped with the portable terminal 300 that limits the operation of the shovel 100, in addition to the remote operation device 400. Therefore, even in the case of a shovel 100 that is remotely operated, operation restrictions can be stably imposed, making it possible to operate the shovel 100 more appropriately with safety in mind.

In addition, the portable terminal 300 has release buttons B4 to B6 as a means for releasing the operation restriction state of the shovel 100, so that the operation restriction state of the shovel 100 that has been implemented can be quickly released, and work can be resumed easily and smoothly.

The portable terminal 300 can also limit the operation of the shovel 100 by limiting the operation speed or gradually reducing the operation speed to bring the shovel 100 to a gradual stop. This makes it possible to suppress instability in posture or operation that may occur if the shovel 100 is suddenly stopped.

In addition, since the control system 1000 has multiple shovels 100 and the portable terminal 300 can limit the operation of the multiple shovels 100, it is possible to operate each shovel 100 more appropriately while taking safety into consideration, even in a large-scale work site where many shovels 100 are working.

In addition, since the control system 1000 has a management device 200 that records logs related to the shovel 100, it is possible to keep a record of the work status, including the implementation of operational restrictions on the shovel 100. Therefore, it is possible to obtain materials that are useful for checking and analyzing the work status of the shovel 100.

The portable terminal 300 also includes a voice output device 306 and a microphone 307 as means for communicating with the operator OP of the remote control device 400 .
This allows the worker L carrying the portable terminal 300 and the operator OP of the remote control device 400 to explain the situation and exchange questions and answers, enabling each party to more accurately grasp the other's situation.

In addition, the portable terminal 300 performs an operation restriction process so that the operation of the shovel 100 can be restricted after the intention is communicated via the audio output device 306 and the microphone 307 .
For this reason, after mutual understanding of the situation through communication of intentions, the operation of the shovel 100 is restricted. This makes it possible to reduce the frequency of operation restrictions that are unnecessary in the circumstances, and to operate the shovel 100 more appropriately while taking safety into consideration, without impairing work efficiency.

The controller 410 of the remote control device 400 also causes the image display device 403 to function as a presentation means for presenting the implementation status of the operation restriction of the shovel 100 by the portable terminal 300. This allows the operator OP of the remote control device 400 to quickly understand the implementation status of the operation restriction of the shovel 100A, and makes it possible to smoothly recover from the operation restriction.

[others]
Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment.
For example, the control system 1000 is exemplified as a configuration in which the portable terminal 300 restricts the operation of a plurality of shovels 100, but the portable terminal 300 may be configured to restrict the operation of a single shovel 100.
In this case, the execution buttons B1 to B3 and the release buttons B4 to B6 on the standby screen G1 can each be limited to one, and the batch execution button B7 can be omitted.

In addition, in the operation restriction process executed by the controller 310, when operation restrictions are to be imposed individually on each of the shovels 100A to 100C, a telephone line with the operator OP is connected, and then, after leaving a state in which telephone communication is possible, an operation restriction command is sent.
However, when implementing operational restriction using the execution buttons B1 to B3, the steps of connecting the telephone line and making a call may be omitted and an operational restriction command may be sent immediately.
In this case, even in a case of high urgency, it is possible to immediately implement operational restrictions on the excavators 100A to 100C.

In addition, with regard to the operational restrictions of the excavator 100, it is possible to select and set in advance a number of patterns, such as immediate stop, speed limit, and slow stop, but this is not limiting. For example, a button for executing all patterns may be displayed on the standby screen G1, and the operational restrictions of each pattern may be selected and executed on the spot.

In addition, in the above control system 1000, a configuration in which the portable terminal 300 transmits an operation restriction command to the shovel 100 has been exemplified, but the present invention is not limited to this. For example, the portable terminal 300 may be configured to substantially restrict operation by disabling various operation commands transmitted from the remote control device 400 to the shovel 100.

In the control system 1000, the portable terminal 300 transmits operation restriction commands to a plurality of excavators 100 through the network NW.
However, communication between the portable terminal 300 and each shovel 100 is not limited to this. For example, communication between the portable terminal 300 and each shovel 100 may be performed using a directional light beam (laser light, LED light, etc.).
In this case, when sending an operation restriction command to any one of the multiple shovels 100, it is only necessary to communicate by directing the light beam toward the desired shovel 100, thereby reducing the effort required to select the desired shovel 100 by pressing a button, etc., and enabling quick processing.
Furthermore, there is no need to prepare a button for each shovel 100, and the configuration of the portable terminal 300 can be simplified.

The portable terminal may have an analog button or an analog switch as a means for executing the operation restriction. In this case, the number of analog buttons or switches may be minimized. For example, the portable terminal may be configured with a terminal having one execution button (or switch) and one release button (or switch) for the operation restriction for only one shovel 100.
Furthermore, the configuration may include only one button (or switch) that can be used as both the execute button (or switch) and the release button (or switch), and the functions of the execute button (or switch) and the release button (or switch) may be swapped each time the button (or switch) is used.

Furthermore, the control system according to the present invention is not limited to excavators, but can be applied to all types of work machines (including construction machines). For example, the control system according to the present invention can be applied to work machines such as wheel loaders, bulldozers, motor graders, skid steer loaders, compact track loaders, and self-propelled cranes. Furthermore, the control system may be configured to be used with a mixture of multiple types of work machines.
In addition, the details shown in the embodiment can be modified as appropriate without departing from the spirit of the invention.

30 Controller 100, 100A, 100B, 100C Shovel 200 Management device (log recording device)
221 Log 300 Portable terminal (terminal)
301 Image display device 302 Input device 306 Audio output device (means for communicating intentions)
307 Microphone (Means of Communication)
310 Controller 400 Remote operation device 403 Image display device 404 Operation device 405 Audio output device 406 Microphone 410 Controller (presentation means)
1000 Control system B1 to B3 Execute button B4 to B6 Release button (means for releasing)
B7 Batch execution button G1 Waiting screens G2 to G4 Input screen G5 Status notification screen L Worker OP Operator

Claims (8)

A work machine;
A remote control device that remotely controls the work machine;
a terminal for limiting the operation of the work machine, separate from the remote control device;
A work machine control system comprising: The work machine control system according to claim 1 , wherein the terminal has a means for releasing a restricted state of the operation of the work machine. The work machine control system according to claim 1 , wherein the terminal is configured to limit the operating speed of the work machine or gradually reduce the operating speed of the work machine to stop it. A plurality of the work machines are provided,
The work machine control system according to claim 1 , wherein the terminal limits operations of the plurality of work machines. The work machine control system according to claim 1 , further comprising a log recording device that records logs relating to the work machine. The control system for a work machine according to claim 1 , wherein the terminal includes a means for communicating with an operator of the remote control device. The control system for a work machine according to claim 6 , wherein the terminal is capable of restricting the operation of the work machine after an intention is transmitted by the intention transmission means. The work machine control system according to claim 1 , wherein the remote control device comprises a presentation means for presenting an implementation status of a restriction on the operation of the work machine by the terminal.

Images