US20230167627A1 - Work machine remote control system - Google Patents

Work machine remote control system Download PDF

Info

Publication number
US20230167627A1
US20230167627A1 US17/922,639 US202117922639A US2023167627A1 US 20230167627 A1 US20230167627 A1 US 20230167627A1 US 202117922639 A US202117922639 A US 202117922639A US 2023167627 A1 US2023167627 A1 US 2023167627A1
Authority
US
United States
Prior art keywords
working equipment
alarm
work machine
image
control unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US17/922,639
Other languages
English (en)
Inventor
Yasuhiro Ohyama
Jun Morinaga
Masanori Minagawa
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Komatsu Ltd
Original Assignee
Komatsu Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Komatsu Ltd filed Critical Komatsu Ltd
Assigned to KOMATSU LTD. reassignment KOMATSU LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MINAGAWA, MASANORI, MORINAGA, JUN, OHYAMA, YASUHIRO
Publication of US20230167627A1 publication Critical patent/US20230167627A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/20—Drives; Control devices
    • E02F9/2025—Particular purposes of control systems not otherwise provided for
    • E02F9/205—Remotely operated machines, e.g. unmanned vehicles
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices
    • E02F9/261—Surveying the work-site to be treated
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices
    • E02F9/264—Sensors and their calibration for indicating the position of the work tool
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F3/00—Dredgers; Soil-shifting machines
    • E02F3/04—Dredgers; Soil-shifting machines mechanically-driven
    • E02F3/28—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets
    • E02F3/34—Dredgers; Soil-shifting machines mechanically-driven with digging tools mounted on a dipper- or bucket-arm, i.e. there is either one arm or a pair of arms, e.g. dippers, buckets with bucket-arms, i.e. a pair of arms, e.g. manufacturing processes, form, geometry, material of bucket-arms directly pivoted on the frames of tractors or self-propelled machines
    • E02F3/345—Buckets emptying side-ways

Definitions

  • the present disclosure relates to a remote control system of a work machine.
  • Patent Literature 1 discloses a technique in which the imaging apparatus is disposed in a cab of a swinging platform to capture an image in front of the cab.
  • Patent Literature 1 JP 2019-068236 A
  • a work machine performs work using working equipment.
  • an impact may act on the working equipment. It is difficult for an operator in a remote place to recognize the impact acting on the working equipment. Therefore, there is a possibility that the operator in the remote place performs remote control that results in applying an excessive impact to the working equipment. When the excessive impact acts on the working equipment, deterioration of the working equipment may be accelerated.
  • a work machine remote control system comprises: a sensor data reception unit configured to receive detection data of an attitude of working equipment included in a work machine operated by an operation signal from a remote place; an alarm control unit configured to output an alarm control signal when it is determined, based on the detection data, that the working equipment is approaching or has reached an end position in a movable range; and an alarm device provided in the remote place and configured to output an alarm based on the alarm control signal from the alarm control unit.
  • FIG. 1 is a schematic diagram illustrating a work machine remote control system according to an embodiment.
  • FIG. 2 is a perspective view illustrating the work machine according to the embodiment.
  • FIG. 3 is a side view illustrating the work machine according to the embodiment.
  • FIG. 4 is a diagram illustrating a remote control room according to the embodiment.
  • FIG. 5 is a schematic diagram illustrating a hydraulic system of the work machine according to the embodiment.
  • FIG. 6 is a functional block diagram illustrating the work machine remote control system according to the embodiment.
  • FIG. 7 is a schematic diagram illustrating a movable range of components of working equipment according to the embodiment.
  • FIG. 8 is a diagram illustrating a process by a first image processing unit according to the embodiment.
  • FIG. 9 is a diagram illustrating a process by a display control unit according to the embodiment.
  • FIG. 10 is a diagram illustrating a process by an alarm control unit according to the embodiment.
  • FIG. 11 is a flowchart illustrating a remote control method of the work machine according to the embodiment.
  • FIG. 12 is a block diagram illustrating a computer system according to the embodiment.
  • FIG. 13 is a diagram illustrating an operation of the work machine according to the embodiment.
  • FIG. 14 is a flowchart illustrating the remote control method of the work machine according to the embodiment.
  • positional relationship of each part will be described using terms of “left”, “right”, “front”, “back”, “top”, and “bottom”. These terms indicate relative positions or directions with respect to a center of a swinging platform 3 of a work machine 1 .
  • FIG. 1 is a schematic diagram illustrating a remote control system 100 of the work machine 1 according to an embodiment.
  • the remote control system 100 performs remote control of the work machine 1 operating at a work site.
  • An example of the work site is a mine or a quarry.
  • the remote control system 100 is disposed in a remote control room 200 .
  • the remote control room 200 is installed in a remote place away from the work site.
  • the remote control system 100 includes a remote controller 40 , a display apparatus 50 , and a control apparatus 60 .
  • the remote controller 40 is disposed in the remote control room 200 .
  • the remote controller 40 is operated by an operator in the remote control room 200 .
  • the operator can operate the remote controller 40 in a state seated on an operating seat 45 .
  • the display apparatus 50 is disposed in the remote control room 200 .
  • the display apparatus 50 displays an image of the work site.
  • the operator in the remote control room 200 cannot directly visually recognize a situation in the work site.
  • the operator in the remote control room 200 can visually recognize the situation in the work site via the display apparatus 50 .
  • the operator operates the remote controller 40 while viewing the image of the work site displayed on the display apparatus 50 .
  • the work machine 1 is remotely controlled by the remote controller 40 .
  • the control apparatus 60 is disposed in the remote control room 200 .
  • the control apparatus 60 includes a computer system.
  • the work machine 1 is provided with a control apparatus 300 .
  • the control apparatus 300 includes a computer system.
  • the control apparatus 60 and the control apparatus 300 communicate with each other via a communication system 400 .
  • Examples of the communication system 400 include the Internet, a local area network (LAN), a mobile phone communication network, and a satellite communication network.
  • the communication system 400 may include a relay station that relays data to be communicated.
  • FIG. 2 is a perspective view illustrating the work machine 1 according to the embodiment.
  • FIG. 3 is a side view illustrating the work machine 1 according to the embodiment.
  • the work machine 1 is an excavator that is one type of loading machine.
  • the work machine 1 operates at the work site.
  • the work machine 1 operates, for example, in a loading area of the work site.
  • the work machine 1 performs excavation of a work target. Examples of the work target include earth and sand or ore.
  • a dump truck which is one type of a haulage vehicle, operates at the work site.
  • the work machine 1 performs loading work to load a cargo onto the dump truck.
  • As the cargo an excavated object excavated by excavation is exemplified.
  • the work machine 1 includes a traveling body 2 , a swinging platform 3 supported by the traveling body 2 , working equipment 4 mounted on the swinging platform 3 , a hydraulic cylinder 5 that drives the working equipment 4 , a position sensor 71 that detects a position of the work machine 1 , a vehicle attitude sensor 72 that detects an attitude of the swinging platform 3 , a working equipment attitude sensor 73 that detects an attitude of the working equipment 4 , and an imaging apparatus 30 .
  • the traveling body 2 travels in a state of supporting the swinging platform 3 .
  • the swinging platform 3 is a vehicle body of the work machine 1 .
  • Traveling body 2 is disposed below the swinging platform 3 .
  • the traveling body 2 rotatably supports the swinging platform 3 .
  • the traveling body 2 includes a drive wheel 2 A, a driven wheel 2 B, and a crawler 2 C supported by the drive wheel 2 A and the driven wheel 2 B.
  • Each of the drive wheel 2 A and the driven wheel 2 B rotates about a rotation axis DX.
  • a pair of drive wheels 2 A, a pair of driven wheels 2 B, and a pair of crawlers 2 C are provided.
  • the crawler 2 C is rotated by the rotation of the drive wheel 2 A. When the crawler 2 C rotates, the traveling body 2 travels.
  • the swinging platform 3 can swing about a swing axis RX in the state of being supported by the traveling body 2 .
  • the swing axis RX extends in a top-bottom direction.
  • the swinging platform 3 includes a cab 3 A, a lower deck 3 B, a step 3 C, and an upper deck 3 D.
  • the cab 3 A is an internal space of the swinging platform 3 where an operator can board.
  • the cab 3 A is disposed at a front and upper part of the swinging platform 3 .
  • the lower deck 3 B is disposed at a back and lower part of the swinging platform 3 .
  • the upper deck 3 D is disposed at the front and upper part of the swinging platform 3 .
  • Step 3 C connects the lower deck 3 B and the upper deck 3 D.
  • the upper deck 3 D is disposed around the cab 3 A. At least a part of the upper deck 3 D is disposed in front of the cab 3 A.
  • a fence-like handrail 3 E is arranged in each of the lower deck 3 B, the step 3 C, and the upper deck 3 D.
  • Each of the lower deck 3 B, the step 3 C, and the upper deck 3 D includes a passage through which the operator can pass.
  • the operator can board the cab 3 A by passing the lower deck 3 B, the step 3 C, and the upper deck 3 D.
  • the swinging platform 3 includes a ladder 3 F.
  • the ladder 3 F is connected to the upper deck 3 D.
  • the working equipment 4 is mounted on the front part of the swinging platform 3 .
  • the working equipment 4 is disposed in front of the swing axis RX.
  • the working equipment 4 is operable to extend forward.
  • the working equipment 4 includes a boom 4 A connected to the swinging platform 3 , an arm 4 B connected to the boom 4 A, and a bucket 4 C connected to the arm 4 B.
  • a proximal end part of the boom 4 A is connected to the front part of the swinging platform 3 via a pin.
  • the proximal end part of the arm 4 B is connected to a distal end part of the boom 4 A via a pin.
  • a proximal end part of the bucket 4 C is connected to a distal end part of the arm 4 B via a pin.
  • the bucket 4 C has a tip blade 4 D.
  • the bucket 4 C excavates the work target.
  • the boom 4 A is connected to the front part of the swinging platform 3 so as to be rotatable about a boom rotation axis AX.
  • the arm 4 B is connected to the boom 4 A so as to be rotatable about an arm rotation axis BX.
  • the bucket 4 C is connected to the arm 4 B so as to be rotatable about a bucket rotation axis CX.
  • the boom rotation axis AX, the arm rotation axis BX, and the bucket rotation axis CX are parallel. Each of the boom rotation axis AX, the arm rotation axis BX, and the bucket rotation axis CX extends in a vehicle width direction of the swinging platform 3 .
  • the work machine 1 is a loading excavator.
  • the loading excavator is an excavator in which the bucket 4 C is attached to the arm 4 B such that the tip blade 4 D of the bucket 4 C faces forward.
  • the hydraulic cylinder 5 includes a boom cylinder 5 A that drives the boom 4 A, an arm cylinder 5 B that drives the arm 4 B, and a bucket cylinder 5 C that drives the bucket 4 C.
  • a proximal end part of the boom cylinder 5 A is connected to the swinging platform 3 .
  • a distal end part of the boom cylinder 5 A is connected to the boom 4 A.
  • a proximal end part of the arm cylinder 5 B is connected to the boom 4 A.
  • a distal end part of the arm cylinder 5 B is connected to the arm 4 A.
  • a proximal end part of the bucket cylinder 5 C is connected to the boom 4 A.
  • a distal end part of the bucket cylinder 5 C is connected to the bucket 4 C.
  • the position sensor 71 detects the position of the work machine 1 .
  • the position sensor 71 detects an absolute position of the work machine 1 using a global navigation satellite system (GNSS).
  • GNSS global navigation satellite system
  • the position sensor 71 includes a GNSS receiver provided in the swinging platform 3 .
  • the vehicle attitude sensor 72 detects the attitude of the swinging platform 3 .
  • the attitude of the swinging platform 3 includes an inclination angle (roll, pitch) of the swinging platform 3 with respect to a horizontal plane.
  • the vehicle attitude sensor 72 includes an inertial measurement unit (IMU) provided in the swinging platform 3 .
  • IMU inertial measurement unit
  • the working equipment attitude sensor 73 detects the attitude of the working equipment 4 .
  • the attitude of the working equipment 4 includes an angle of the working equipment 4 .
  • the working equipment attitude sensor 73 includes a boom attitude sensor 73 A that detects an angle of the boom 4 A with respect to the swinging platform 3 , an arm attitude sensor 73 B that detects an angle of the arm 4 B with respect to the boom 4 A, and a bucket attitude sensor 73 C that detects an angle of the bucket 4 C with respect to the arm 4 B.
  • the working equipment attitude sensor 73 is a stroke sensor disposed in the hydraulic cylinder 5 .
  • the hydraulic cylinder 5 includes a cylinder tube, a piston that moves inside the cylinder tube, and a rod connected to the piston.
  • the stroke sensor detects a stroke length of the hydraulic cylinder 5 indicating a movement distance of the rod.
  • the stroke length refers to a movement distance of the rod from the stroke end of the hydraulic cylinder 5 .
  • the stroke end refers to an end position in a movable range of the rod. In other words, the stroke end refers to a position of the rod in a state that the hydraulic cylinder 5 is most contracted or a position of the rod in a state that the hydraulic cylinder 5 is most extended.
  • the boom attitude sensor 73 A is a stroke sensor disposed in the boom cylinder 5 A.
  • the boom attitude sensor 73 A detects a stroke length of the boom cylinder 5 A.
  • the arm attitude sensor 73 B is a stroke sensor disposed in the arm cylinder 5 B.
  • the arm attitude sensor 73 B detects a stroke length of the arm cylinder 5 B.
  • the bucket attitude sensor 73 C is a stroke sensor disposed in the bucket cylinder 5 C.
  • the bucket attitude sensor 73 C detects a stroke length of the bucket cylinder 5 C.
  • the imaging apparatus 30 captures the work site to acquire an image of the work site.
  • the imaging apparatus 30 is disposed in the swinging platform 3 .
  • the imaging apparatus 30 is fixed to the swinging platform 3 .
  • Examples of the image of the work site acquired by the imaging apparatus 30 include an image of the work target of the work machine 1 , an image of at least a part of the work machine 1 , an image of a structure existing at the work site, an image of a work machine different from the work machine 1 , and an image of a worker working at the work site.
  • the image of the work target of the work machine 1 includes an image of an excavation target of the working equipment 4 .
  • the imaging apparatus 30 includes an optical system and an image sensor that receives light passing through the optical system.
  • the image sensor includes a couple charged device (CCD) image sensor or a complementary metal oxide semiconductor (CMOS) image sensor.
  • CCD couple charged device
  • CMOS complementary metal oxide semiconductor
  • the imaging apparatus 30 captures an image in an imaging range M.
  • the imaging range M is set to include the work target of the work machine 1 .
  • the top-bottom direction is a direction parallel to the swing axis RX.
  • a left-right direction is a direction parallel to the boom rotation axis AX.
  • a front-back direction is a direction orthogonal to both the boom rotation axis AX and the swing axis RX.
  • a direction in which the swinging platform 3 exists with reference to a ground contact surface of the traveling body 2 is toward top, and a direction opposite to the top is bottom.
  • One of the left and right directions with reference to the swing axis RX is right, and a direction opposite to the right is left.
  • a direction in which the working equipment 4 exists with reference to the swing axis RX is front, and a direction opposite to the front is back.
  • the imaging apparatus 30 is disposed in the cab 3 A.
  • An optical axis OA of the optical system of the imaging apparatus 30 extends in the front-back direction.
  • an image in the imaging range M captured by the imaging apparatus 30 is referred to as an image P as appropriate.
  • FIG. 4 is a diagram illustrating the remote control room 200 according to the embodiment. As illustrated in FIG. 4 , the remote controller 40 and the display apparatus 50 are disposed in the remote control room 200 .
  • the remote controller 40 is operated by the operator seated on the operating seat 45 .
  • the operator sits on the operating seat 45 so as to face a display screen of the display apparatus 50 .
  • the operator operates the remote controller 40 while viewing the display screen of the display apparatus 50 .
  • An operation signal generated by operating the remote controller 40 is transmitted to the control apparatus 300 of the work machine 1 via the control apparatus 60 and the communication system 400 .
  • the control apparatus 300 operates the work machine 1 according to the operation signal acquired via the communication system 400 .
  • the work machine 1 is operated by the operation signal from the remote place of the work machine 1 .
  • An operation of the work machine 1 includes at least one of an operation of the traveling body 2 , an operation of the swinging platform 3 , and an operation of the working equipment 4 .
  • the operation of the traveling body 2 includes a forward operation and a backward operation of the traveling body 2 .
  • the operation of the swinging platform 3 includes a left swing operation and a right swing operation of the swinging platform 3 .
  • the operation of the working equipment 4 includes an elevating operation of the boom 4 A, a lowering operation of the boom 4 A, a dumping operation of the arm 4 B, an excavating operation of the arm 4 B, an excavating operation of the bucket 4 C, and a dumping operation of the bucket 4 C.
  • the remote controller 40 includes a left working lever 41 and a right working lever 42 operated for the operation of the swinging platform 3 and the working equipment 4 , and a left travel pedal 43 and a right travel pedal 44 operated for the operation of the traveling body 2 .
  • the left working lever 41 is disposed on a left side of the operating seat 45 .
  • the right working lever 42 is disposed on a right side of the operating seat 45 .
  • the arm 4 B performs the dumping operation or the excavating operation.
  • the swinging platform 3 performs the left swing operation or the right swing operation.
  • the bucket 4 C performs the excavating operation or the dumping operation.
  • the boom 4 A performs the lowering operation or the elevating operation.
  • the swinging platform 3 may perform the right swing operation or the left swing operation
  • the arm 4 B may perform the dumping operation or the excavating operation.
  • the operating direction of the left working lever 41 and the operating direction of the right working lever 42 are arbitrarily related to the operation of the working equipment 4 .
  • the left travel pedal 43 and the right travel pedal 44 are disposed on a front lower part of the operating seat 45 .
  • the left travel pedal 43 is disposed to the left of the right travel pedal 44 .
  • the crawler 2 C on the left side of the traveling body 2 moves forward or backward.
  • the right travel pedal 44 is operated, the crawler 2 C on the right side of the traveling body 2 moves forward or backward.
  • a first monitor device 501 that displays work machine operation data indicating an operation status of the work machine 1 and an operation switch 502 that is operated to activate an electric apparatus equipped in the work machine 1 are disposed.
  • the first monitor device 501 displays, for example, a remaining amount of fuel of an engine mounted on the work machine 1 , a temperature of coolant of the engine, a temperature of hydraulic oil for driving the hydraulic cylinder 5 , and a traveling speed of the traveling body 2 as the work machine operation data.
  • the operation switch 502 operates, for example, a headlight provided in the work machine 1 as the electric apparatus equipped in the work machine 1 .
  • the display apparatus 50 displays the image P transmitted from the work machine 1 .
  • the image P is transmitted to the control apparatus 60 of the remote control system 100 via the control apparatus 300 and the communication system 400 .
  • the control apparatus 60 causes the display apparatus 50 to display the image P acquired via the communication system 400 .
  • the display apparatus 50 includes a flat panel display such as a liquid crystal display (LCD) or an organic electroluminescence display (OELD).
  • the display apparatus 50 includes a plurality of flat panel displays arranged adjacent to each other.
  • the display apparatus 50 includes a central display 51 , a left display 52 disposed on a left side of the central display 51 , a right display 53 disposed on a right side of the central display 51 , an upper display 54 disposed above the central display 51 , and a lower display 55 disposed below the central display 51 .
  • the image P displayed on the display apparatus 50 is an image corresponding to a field of view of a front space of the operator when the operator is assumed to be seated on a driver seat provided in the cab 3 A of the work machine 1 .
  • the operator in the remote control room 200 can obtain a feeling of actually sitting on the driver's seat of the work machine 1 .
  • the operator in the remote control room 200 operates the remote controller 40 to operate the working equipment 4 and excavate the work target.
  • the excavated object excavated by the bucket 4 C of the working equipment 4 is loaded onto the dump truck as a cargo.
  • the dump truck is an unmanned dump truck that travels according to a control command transmitted from a control facility in the work site.
  • a second monitor device 503 is arranged to display dump truck operation data indicating an operation status of the unmanned dump truck at the work site.
  • a position sensor is disposed to detect position data of the unmanned dump truck.
  • the position sensor detects an absolute position of the unmanned dump truck using the global navigation satellite system (GNSS).
  • GNSS global navigation satellite system
  • the second monitor device 503 displays a position of each of the plurality of unmanned dump trucks operating at the work site as the dump truck operation data.
  • the operator can stop or start the unmanned dump truck by operating an input unit provided in the second monitor device 503 .
  • a third monitor device 504 is disposed to display guidance data of the working equipment 4 .
  • the guidance data a relative distance between a target design surface of the work target and the working equipment 4 , a shape of the work target, and an ore distribution of the work target are exemplified.
  • FIG. 5 is a schematic diagram illustrating a hydraulic system 20 of the work machine 1 according to the embodiment.
  • the hydraulic system 20 includes a hydraulic pump 21 , the hydraulic cylinder 5 that drives the working equipment 4 according to the hydraulic oil supplied from the hydraulic pump 21 , a pump flow path 22 connected to the hydraulic pump 21 , a flow rate control valve 23 that adjusts the flow rate of the hydraulic oil supplied to the hydraulic cylinder 5 via the pump flow path 22 , and a pressure sensor 74 that detects a pressure of the hydraulic oil in the hydraulic cylinder 5 .
  • the hydraulic pump 21 is driven by power transmitted from a power source of the work machine 1 .
  • a power source of the work machine 1 a diesel engine or an electric motor is exemplified.
  • the hydraulic pump 21 discharges the hydraulic oil.
  • the hydraulic pump 21 is a variable displacement hydraulic pump.
  • the hydraulic cylinder 5 operates the working equipment 4 according to the hydraulic oil supplied from the hydraulic pump 21 .
  • the working equipment 4 operates within a predetermined movable range.
  • the hydraulic cylinder 5 includes the boom cylinder 5 A that operates the boom 4 A, the arm cylinder 5 B that operates the arm 4 B, and the bucket cylinder 5 C that operates the bucket 4 C.
  • the hydraulic cylinder 5 includes a bottom chamber BR and a rod chamber RR.
  • the hydraulic cylinder 5 extends.
  • the hydraulic oil is supplied to the rod chamber RR, the hydraulic cylinder 5 contracts.
  • the flow rate control valve 23 adjusts the flow rate of the hydraulic oil supplied to the hydraulic cylinder 5 .
  • the flow rate control valve 23 includes a boom flow rate control valve 23 A that adjusts the flow rate of the hydraulic oil supplied to the boom cylinder 5 A, an arm flow rate control valve 23 B that adjusts the flow rate of the hydraulic oil supplied to the arm cylinder 5 B, and a bucket flow rate control valve 23 C that adjusts the flow rate of the hydraulic oil supplied to the bucket cylinder 5 C.
  • the flow rate control valve 23 includes a pump port Pa, a bottom port Pb, a rod port Pc, and a tank port Pd.
  • the pump port Pa is connected to the hydraulic pump 21 via a supply flow path 24 and the pump flow path 22 .
  • the bottom port Pb is connected to the bottom chamber BR of the hydraulic cylinder 5 via a bottom flow path 25 .
  • the rod port Pc is connected to the rod chamber RR of the hydraulic cylinder 5 via a rod flow path 26 .
  • the tank port Pd is connected to a tank 28 via a discharge flow path 27 .
  • the hydraulic oil discharged from the hydraulic pump 21 can flow through the pump flow path 22 and the supply flow path 24 and then flow into the flow rate control valve 23 from the pump port Pa.
  • the hydraulic oil flowing out from the bottom port Pb can flow into the bottom chamber BR of the hydraulic cylinder 5 after flowing through the bottom flow path 25 .
  • the hydraulic oil flowing out from the bottom chamber BR of the hydraulic cylinder 5 can flow through the bottom flow path 25 and then flow into the flow rate control valve 23 from the bottom port Pb.
  • the hydraulic oil flowing out from the rod port Pc can flow into the rod chamber RR of the hydraulic cylinder 5 after flowing through the rod flow path 26 .
  • the hydraulic oil flowing out from the rod chamber RR of the hydraulic cylinder 5 can flow through the rod flow path 26 and then flow into the flow rate control valve 23 from the rod port Pc.
  • the hydraulic oil flowing out from the tank port Pd flows through the discharge flow path 27 and then is discharged to the tank 28 .
  • the flow rate control valve 23 is a slide spool type flow rate control valve that switches the flow rate and direction of the hydraulic oil supplied to the hydraulic cylinder 5 by moving a rod-shaped spool.
  • the spool moves in an axial direction, the supply of the hydraulic oil to the bottom chamber BR and the supply of the hydraulic oil to the rod chamber RR are switched.
  • the flow rate of the hydraulic oil supplied to the hydraulic cylinder 5 is adjusted according to a movement amount of the spool.
  • the spool of the flow rate control valve 23 moves to a first operating position Q 1 at which the hydraulic oil is supplied to the bottom chamber BR of the hydraulic cylinder 5 , a second operating position Q 2 at which the hydraulic oil is supplied to the rod chamber RR of the hydraulic cylinder 5 , and a stop position Q 3 arranged between the first operating position Q 1 and the second operating position Q 2 and at which the hydraulic oil does not flow.
  • the hydraulic oil discharged from the hydraulic pump 21 flows through the pump flow path 22 and the supply flow path 24 , then flows into the flow rate control valve 23 from the pump port Pa, and flows out from the bottom port Pb.
  • the hydraulic oil flowing out from the bottom port Pb flows through the bottom flow path 25 and then flows into the bottom chamber BR of the hydraulic cylinder 5 .
  • the hydraulic cylinder 5 extends.
  • the hydraulic oil flows out from the rod chamber BR.
  • the hydraulic oil flowing out from the rod chamber BR of the hydraulic cylinder 5 flows through the rod flow path 26 , then flows into the flow rate control valve 23 from the rod port Pc, and flows out from the tank port Pd.
  • the hydraulic oil flowing out from the tank port Pd is discharged to the tank 28 via the discharge flow path 27 .
  • the hydraulic oil discharged from the hydraulic pump 21 flows through the pump flow path 22 and the supply flow path 24 , then flows into the flow rate control valve 23 from the pump port Pa, and flows out from the rod port Pc.
  • the hydraulic oil flowing out from the rod port Pc flows through the rod flow path 26 and then flows into the rod chamber RR of the hydraulic cylinder 5 .
  • the hydraulic cylinder 5 contracts.
  • the hydraulic oil flows out from the bottom chamber BR.
  • the hydraulic oil flowing out from the bottom chamber BR of the hydraulic cylinder 5 flows through the bottom flow path 25 , then flows into the flow rate control valve 23 from the bottom port Pb, and flows out from the tank port Pd.
  • the hydraulic oil flowing out from the tank port Pd is discharged to the tank 28 via the discharge flow path 27 .
  • the pressure sensor 74 detects the pressure of the hydraulic oil in the hydraulic cylinder 5 .
  • the pressure sensor 74 includes a boom pressure sensor 74 A that detects the pressure of the hydraulic oil in the boom cylinder 5 A, an arm pressure sensor 74 B that detects the pressure of the hydraulic oil in the arm cylinder 5 B, and a bucket pressure sensor 74 C that detects the pressure of the hydraulic oil in the bucket cylinder 5 C.
  • the pressure sensor 74 detects the pressure of the hydraulic oil supplied to the hydraulic cylinder 5 .
  • the pressure sensor 74 is provided in each of the bottom flow path 25 and the rod flow path 26 .
  • the pressure sensor 74 provided in the rod flow path 26 detects the pressure of the hydraulic oil supplied to the hydraulic cylinder 5 .
  • the pressure sensor 74 provided in the bottom flow path 25 detects the pressure of the hydraulic oil supplied to the hydraulic cylinder 5 .
  • the pressure sensor 74 may be provided in each of the bottom chamber BR and the rod chamber RR.
  • FIG. 6 is a functional block diagram illustrating the remote control system 100 of the work machine 1 according to the embodiment.
  • the remote control system 100 includes a communication device 6 disposed in a remote place, the control apparatus 60 connected to the communication device 6 , the remote controller 40 connected to the control apparatus 60 , and the display apparatus 50 connected to the control apparatus 60 .
  • the remote control system 100 includes a communication device 7 disposed in the work machine 1 , the control apparatus 300 connected to the communication device 7 , the imaging apparatus 30 connected to the control apparatus 300 , a sensor 70 connected to the control apparatus 300 , the traveling body 2 controlled by the control apparatus 300 , the swinging platform 3 controlled by the control apparatus 300 , and the hydraulic cylinder 5 controlled by the control apparatus 300 .
  • the sensor 70 includes the position sensor 71 , the vehicle attitude sensor 72 , the working equipment attitude sensor 73 , and the pressure sensor 74 .
  • the control apparatus 300 includes a traveling body control unit 301 , a swinging platform control unit 302 , a working equipment control unit 303 , a stroke end determination unit 304 , an image data transmission unit 305 , and a sensor data transmission unit 306 .
  • the traveling body control unit 301 receives an operation signal of the remote controller 40 transmitted from the control apparatus 60 .
  • the traveling body control unit 301 outputs a control signal for controlling the operation of the traveling body 2 according to the operation signal of the remote controller 40 .
  • the swinging platform control unit 302 receives the operation signal of the remote controller 40 transmitted from the control apparatus 60 .
  • the swinging platform control unit 302 outputs a control signal for controlling the operation of the swinging platform 3 according to the operation signal of the remote controller 40 .
  • the working equipment control unit 303 receives the operation signal of the remote controller 40 transmitted from the control apparatus 60 .
  • the working equipment control unit 303 outputs a control signal for controlling the operation of the working equipment 4 according to the operation signal of the remote controller 40 .
  • the control signal for controlling the operation of the working equipment 4 includes a control signal for controlling the operation of the hydraulic cylinder 5 .
  • the control signal for controlling the operation of the hydraulic cylinder 5 includes a control signal for controlling the flow rate control valve 23 .
  • the stroke end determination unit 304 calculates an angle of the working equipment 4 based on detection data of the working equipment attitude sensor 73 .
  • the angle of the working equipment 4 and the stroke length of the hydraulic cylinder 5 are correlated.
  • the stroke end determination unit 304 can calculate the angle of the working equipment 4 by performing arithmetic processing based on the detection data of the working equipment attitude sensor 73 .
  • the stroke end determination unit 304 can calculate the angle of the boom 4 A with respect to the swinging platform 3 by performing arithmetic processing based on detection data of the boom attitude sensor 73 A.
  • the stroke end determination unit 304 can calculate the angle of the arm 4 B with respect to the boom 4 A by performing arithmetic processing based on detection data of the arm attitude sensor 73 B.
  • the stroke end determination unit 304 can calculate an angle of the bucket 4 C with respect to the arm 4 B by performing arithmetic processing based on detection data of the bucket attitude sensor 73 C.
  • the stroke end determination unit 304 can calculate a cylinder position of the hydraulic cylinder 5 based on the detection data of the working equipment attitude sensor 73 .
  • the cylinder position refers to a relative position of the rod with respect to the stroke end of the hydraulic cylinder 5 .
  • the stroke end refers to an end position of the rod in the movable range.
  • the stroke end determination unit 304 can calculate a cylinder position of the boom cylinder 5 A by performing arithmetic processing based on the detection data of the boom attitude sensor 73 A.
  • the stroke end determination unit 304 can calculate a cylinder position of the arm cylinder 5 B by performing arithmetic processing based on the detection data of the arm attitude sensor 73 B.
  • the stroke end determination unit 304 can calculate a cylinder position of the bucket cylinder 5 C by performing arithmetic processing based on the detection data of the bucket attitude sensor 73 C.
  • the stroke end determination unit 304 can calculate the stroke length of the hydraulic cylinder 5 based on the detection data of the working equipment attitude sensor 73 .
  • the stroke length refers to the movement distance of the rod from the stroke end of the hydraulic cylinder 5 .
  • the stroke end determination unit 304 can calculate a cylinder speed of the hydraulic cylinder 5 based on the detection data of the working equipment attitude sensor 73 .
  • the cylinder speed refers to a speed of the rod with respect to the cylinder tube of the hydraulic cylinder 5 .
  • the stroke end determination unit 304 can calculate a cylinder speed of the boom cylinder 5 A by performing arithmetic processing based on the detection data of the boom attitude sensor 73 A.
  • the stroke end determination unit 304 can calculate a cylinder speed of the arm cylinder 5 B by performing arithmetic processing based on the detection data of the arm attitude sensor 73 B.
  • the stroke end determination unit 304 can calculate a cylinder speed of the bucket cylinder 5 C by performing arithmetic processing based on the detection data of the bucket attitude sensor 73 C.
  • the working equipment attitude sensor 73 can function as at least one of an angle sensor of the working equipment 4 , a cylinder position sensor of the hydraulic cylinder 5 , a stroke length sensor of the hydraulic cylinder 5 , and a cylinder speed sensor of the hydraulic cylinder 5 .
  • the working equipment attitude sensor 73 may include an angle sensor capable of detecting the angle of the working equipment 4 such as a potentiometer. Still more, the working equipment attitude sensor 73 may be an inertial measurement unit (IMU) provided in the working equipment 4 .
  • IMU inertial measurement unit
  • the stroke end determination unit 304 determines whether the working equipment 4 approaches the end position in the movable range based on the detection data of the working equipment attitude sensor 73 .
  • FIG. 7 is a schematic diagram illustrating the movable range of the working equipment 4 according to the embodiment.
  • the working equipment 4 includes a plurality of working equipment components that can be relatively moved.
  • the working equipment components of the working equipment 4 include the boom 4 A, the arm 4 B, and the bucket 4 C.
  • the movable range of the working equipment 4 is the movable range of the working equipment components.
  • the working equipment 4 can move within the movable range determined by the stroke of the hydraulic cylinder 5 .
  • the movable range of the working equipment 4 is determined based on the movable range of the rod of the hydraulic cylinder 5 .
  • the end position in the movable range of the working equipment 4 is defined based on the stroke end of the hydraulic cylinder 5 .
  • an end zone, an alarm zone, and an intermediate zone are defined in the movable range of the working equipment 4 .
  • the end zone is a part of the movable range including the end position.
  • the intermediate zone is a part of the movable range including a central position of the movable range.
  • the alarm zone is a part of the movable range between the end zone and the central position.
  • the alarm zone is a section between the end zone and the intermediate zone.
  • the alarm zone is defined so as to be adjacent to each of the end zone and the intermediate zone. As described later, when the working equipment 4 approaches the end position in the alarm zone, an alarm is output in the remote place.
  • the alarm zone may be included in the end zone.
  • the alarm zone may be omitted.
  • a ratio of a length of the end zone to the movable range can be arbitrarily set.
  • the end zone may have an arbitrary ratio from 1 [%] to 20 [%], inclusive, of the movable range.
  • a ratio of a length of the alarm zone to the movable range can be arbitrarily set.
  • the alarm zone may be an arbitrary ratio from 1 [%] to 20 [%], inclusive, of the movable range.
  • the length of the end zone is shorter than the length of the intermediate zone.
  • the stroke end determination unit 304 determines whether the working equipment 4 exists in the end zone including the end position in the movable range based on the detection data of the working equipment attitude sensor 73 .
  • the end position in the movable range of the working equipment 4 is defined based on the stroke end of the hydraulic cylinder 5 .
  • the end zone of the movable range of the working equipment 4 is defined based on the stroke length of the hydraulic cylinder 5 .
  • the stroke end determination unit 304 can determine whether the working equipment 4 exists in the end zone of the movable range based on the detection data of the working equipment attitude sensor 73 .
  • the stroke end determination unit 304 can determine whether or not the working equipment 4 exists in the alarm zone between the end zone and the central position of the movable range based on the detection data of the working equipment attitude sensor 73 .
  • the stroke end determination unit 304 can determine whether the working equipment 4 exists in the intermediate zone including the central position of the movable range based on the detection data of the working equipment attitude sensor 73 .
  • the working equipment control unit 303 performs cushion control.
  • the cushion control is control to decelerate the rod when the rod of the hydraulic cylinder 5 approaches the stroke end.
  • the working equipment control unit 303 decelerates the rod more than the cylinder speed set based on the operation signal generated by the remote controller 40 .
  • the speed of the rod decreases as a distance between the rod of the hydraulic cylinder 5 and the stroke end decreases.
  • the cushion control reduces impact when the rod of the hydraulic cylinder 5 reaches the stroke end.
  • the working equipment control unit 303 When the stroke end determination unit 304 determines that the working equipment 4 is close to the end position in the movable range, the working equipment control unit 303 outputs, to the flow rate control valve 23 , a control command for reducing the flow rate of the hydraulic oil supplied to the hydraulic cylinder 5 .
  • the cushion control to reduce the speed of the working equipment 4 approaching the end position is performed.
  • the speed of the working equipment 4 decreases as the distance between the working equipment 4 and the end position decreases.
  • the speed of the working equipment 4 may be gradually reduced as the distance between the working equipment 4 and the end position becomes shorter, or may be reduced stepwise. In the cushion control, when the working equipment 4 reaches the end position, the speed of the working equipment 4 may or may not be zero.
  • the cushion control is performed when the working equipment 4 is arranged in the end zone and the working equipment 4 is operated to approach the end position.
  • the working equipment control unit 303 outputs, to the flow rate control valve 23 , the control command for reducing the flow rate of the hydraulic oil supplied to the hydraulic cylinder 5 . Accordingly, when the working equipment 4 approaches the end position in the end zone, the cushion control for reducing the speed of the working equipment 4 is performed.
  • the working equipment control unit 303 when the stroke end determination unit 304 determines that the boom 4 A is operated to approach the end position in the end zone of the movable range, the working equipment control unit 303 outputs, to the boom flow rate control valve 23 A, a control command for reducing the flow rate of the hydraulic oil supplied to the boom cylinder 5 A.
  • the working equipment control unit 303 outputs, to the arm flow rate control valve 23 B, a control command for reducing the flow rate of the hydraulic oil supplied to the arm cylinder 5 B.
  • the working equipment control unit 303 When the stroke end determination unit 304 determines that the bucket 4 C is operated to approach the end position in the end zone of the movable range, the working equipment control unit 303 outputs, to the bucket flow rate control valve 23 C, a control command for reducing the flow rate of the hydraulic oil supplied to the bucket cylinder 5 C.
  • the image data transmission unit 305 transmits an image around the work machine 1 acquired by the imaging apparatus 30 to the control apparatus 60 .
  • the image data transmission unit 305 acquires the image P in the imaging range M from the imaging apparatus 30 .
  • the image data transmission unit 305 transmits the image P to the control apparatus 60 .
  • the sensor data transmission unit 306 transmits detection data of the sensor 70 mounted on the work machine 1 to the control apparatus 60 .
  • the sensor 70 includes the position sensor 71 that detects the position of the work machine 1 , the vehicle attitude sensor 72 that detects the attitude of the swinging platform 3 , the working equipment attitude sensor 73 that detects the attitude of the working equipment 4 , and the pressure sensor 74 that detects the pressure of the hydraulic oil in the hydraulic cylinder 5 .
  • the communication device 7 communicates with the communication device 6 via the communication system 400 .
  • the communication device 7 receives the operation signal of the remote controller 40 transmitted from the control apparatus 60 via the communication device 6 , and outputs the operation signal to the control apparatus 300 .
  • the communication device 7 transmits the image P in the imaging range M received from the image data transmission unit 305 to the communication device 6 in the remote place.
  • the communication device 7 includes an encoder that compresses image data of the image P.
  • the image P is transmitted from the communication device 7 to the communication device 6 in a compressed state.
  • the communication device 7 transmits the detection data of the position sensor 71 , the detection data of the vehicle attitude sensor 72 , the detection data of the working equipment attitude sensor 73 , and the detection data of the pressure sensor 74 received from the sensor data transmission unit 306 to the communication device 6 in the remote place.
  • the communication device 6 communicates with the communication device 7 via the communication system 400 .
  • the communication device 6 transmits the operation signal generated by operating the remote controller 40 to the communication device 7 .
  • the communication device 6 receives the image P transmitted from the control apparatus 300 via the communication device 7 , and outputs the image P to the control apparatus 60 .
  • the communication device 6 includes a decoder that restores the image data of the image P compressed.
  • the image P is output from the communication device 6 to the control apparatus 60 in a restored state.
  • the communication device 6 receives the detection data of the position sensor 71 , the detection data of the vehicle attitude sensor 72 , the detection data of the working equipment attitude sensor 73 , and the detection data of the pressure sensor 74 transmitted from the control apparatus 300 via the communication device 7 , and outputs them to the control apparatus 60 .
  • the control apparatus 60 includes an operation signal transmission unit 61 , an image data reception unit 62 , a sensor data reception unit 63 , an image processing unit 64 , a display control unit 65 , and an alarm control unit 66 .
  • the operation signal transmission unit 61 transmits an operation signal for performing the remote control of the work machine 1 .
  • the operation signal for performing the remote control of the work machine 1 is generated.
  • the operation signal transmission unit 61 transmits the operation signal of the remote controller 40 to the control apparatus 300 .
  • the image data reception unit 62 receives the image around the work machine 1 .
  • the image data reception unit 62 receives the image P as the image around the work machine 1 .
  • the image data reception unit 62 acquires the image P restored by the decoder of the communication device 6 .
  • the sensor data reception unit 63 receives the detection data of the sensor 70 .
  • the detection data of the sensor 70 includes detection data related to the swinging platform 3 and detection data related to the working equipment 4 .
  • the sensor data reception unit 63 receives detection data of the position of the work machine 1 detected by the position sensor 71 , detection data of the attitude of the swinging platform 3 detected by the vehicle attitude sensor 72 , detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 , and detection data of the pressure of the hydraulic oil in the hydraulic cylinder 5 detected by the pressure sensor 74 .
  • the image processing unit 64 divides the image P received by the image data reception unit 62 .
  • FIG. 8 is a diagram illustrating a process by the image processing unit 64 according to the embodiment.
  • the image P is acquired by the image data reception unit 62 .
  • the image P is an image of a front space SP of the swinging platform 3 .
  • a part of the working equipment 4 including the bucket 4 C is captured in the image P.
  • the work target in front of the swinging platform 3 is also captured in the image P.
  • the handrail 3 E of the upper deck 3 D is captured in the image P.
  • the image processing unit 64 divides the image P into a plurality of images.
  • the image processing unit 64 divides the image P into an image P 11 to be displayed on the central display 51 , an image P 12 to be displayed on the left display 52 , an image P 13 to be displayed on the right display 53 , an image P 14 to be displayed on the upper display 54 , and an image P 15 to be displayed on the lower display 55 .
  • the display control unit 65 causes the display apparatus 50 to display the image around the work machine 1 .
  • the display control unit 65 causes the display apparatus 50 to display the image P as the image around the work machine 1 .
  • FIG. 9 is a diagram illustrating a process by the display control unit 65 according to the embodiment.
  • the display control unit 65 causes the central display 51 to display the image P 11 that is a part of the image P.
  • the display control unit 65 causes the left display 52 to display the image P 12 that is a part of the image P.
  • the display control unit 65 causes the right display 53 to display the image P 13 that is a part of the image P.
  • the display control unit 65 causes the upper display 54 to display the image P 14 that is a part of the image P.
  • the display control unit 65 causes the lower display 55 to display the image P 15 that is a part of image P.
  • the display control unit 65 causes the display apparatus 50 to display a vehicle data image P 3 indicating the attitude of the swinging platform 3 , a working equipment data image P 4 indicating the attitude of the working equipment 4 , a load data image P 5 indicating a weight of the cargo to be loaded onto the dump truck, and a bucket data image P 6 indicating the position of the tip blade 4 D of the bucket 4 C.
  • the display control unit 65 calculates the inclination angle of the swinging platform 3 with respect to the horizontal plane based on the detection data of the vehicle attitude sensor 72 .
  • the display control unit 65 causes the display apparatus 50 to display a symbol image indicating the inclination angle of the swinging platform 3 as the vehicle data image P 3 .
  • the vehicle data image P 3 is displayed on the upper display 54 .
  • the display control unit 65 calculates the attitude of the working equipment 4 based on the detection data of the working equipment attitude sensor 73 .
  • the display control unit 65 causes the display apparatus 50 to display an animation image indicating the attitude of the working equipment 4 as the working equipment data image P 4 .
  • the working equipment data image P 4 is displayed on the right display 53 .
  • the display control unit 65 calculates the weight of the cargo to be loaded onto the dump truck based on the detection data of a weight sensor (not illustrated) that detects the weight of the cargo held by the bucket 4 C.
  • the display control unit 65 causes the display apparatus 50 to display an indicator image indicating the weight of the cargo as the load data image P 5 .
  • the load data image P 5 is displayed on the right display 53 .
  • the weight sensor that detects the weight of the cargo loaded on the dump truck may be provided on the dump truck, and detection data of the weight sensor may be transmitted to the control apparatus 60 .
  • the display control unit 65 calculates the position of the tip blade 4 D of the bucket 4 C in the top-bottom direction based on the detection data of the working equipment attitude sensor 73 .
  • the display control unit 65 causes the display apparatus 50 to display an indicator image indicating the position in the top-bottom direction of the tip blade 4 D of the bucket 4 C as the bucket data image P 6 .
  • the position in the top-bottom direction of the tip blade 4 D is a height position from ground GR.
  • the bucket data image P 6 is displayed on the right display 53 .
  • the alarm control unit 66 outputs an alarm control signal when determining that the working equipment 4 approaches the end position in the movable range of the working equipment 4 based on the detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 . In other words, the alarm control unit 66 outputs the alarm control signal when determining that the piston of the hydraulic cylinder 5 approaches the stroke end and the working equipment 4 is approaching the end position in the movable range.
  • the alarm control unit 66 outputs the alarm control signal when determining that the working equipment 4 is operated to approach the end position in the alarm zone based on the detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 .
  • the alarm control unit 66 starts outputting the alarm control signal when determining that the working equipment 4 has moved from the intermediate zone to the alarm zone.
  • the cushion control starts when the working equipment 4 moves from the alarm zone to the end zone.
  • the alarm control unit 66 starts outputting the alarm control signal before the cushion control starts.
  • the alarm control unit 66 outputs the alarm control signal when determining that the working equipment 4 is operated to approach the end position in the end zone based on the detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 . In the embodiment, the alarm control unit 66 continues to output the alarm control signal when determining that the working equipment 4 is moving toward the end position in each of the alarm zone and the end zone.
  • the stroke end determination unit 304 determines that the working equipment 4 has moved from the alarm zone to the end zone, the working equipment control unit 303 starts the cushion control.
  • the alarm control unit 66 outputs the alarm control signal when determining that the working equipment 4 has reached the end position based on the detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 .
  • the alarm control unit 66 outputs the alarm control signal when the working equipment 4 moves toward the end position in each of the alarm zone and the end zone, and when the working equipment 4 reaches the end position.
  • FIG. 10 is a diagram illustrating a process by the alarm control unit 66 according to the embodiment.
  • the alarm control unit 66 outputs the alarm control signal to the display apparatus 50 .
  • the display apparatus 50 is provided in the remote place from the work machine 1 , and outputs an alarm based on the alarm control signal from the alarm control unit 66 .
  • the display apparatus 50 functions as an alarm device.
  • the alarm control unit 66 blinks the working equipment data image P 4 as the alarm.
  • the alarm control unit 66 may blink the entire working equipment data image P 4 or may blink a background image of the working equipment data image P 4 .
  • the alarm control unit 66 may change a color of the working equipment 4 displayed on the display apparatus 50 or blink the image of the working equipment 4 as the alarm.
  • the color of the working equipment 4 may be changed by changing the color of the entire working equipment 4 or by changing the color of a part of the working equipment 4 .
  • the alarm control unit 66 may change the color of the arm 4 B.
  • the blinking of the image of the working equipment 4 may be blinking of the entire image of the working equipment 4 or blinking of a part of the image of the working equipment 4 .
  • the alarm control unit 66 may blink the image of the arm 4 B.
  • the alarm control unit 66 may cause the display apparatus 50 to display computer graphics (CG) as the alarm. For example, when the bucket 4 C is close to the end position, the alarm control unit 66 may cause the display apparatus 50 to superimpose computer graphics indicating the position and attitude of the bucket 4 C on the image P in the display.
  • CG computer graphics
  • FIG. 11 is a flowchart illustrating a remote control method of the work machine 1 according to the embodiment.
  • Step SB 1 the operation signal for performing the remote control of the work machine 1 is transmitted from the operation signal transmission unit 61 of the control apparatus 60 to the control apparatus 300 (Step SB 1 ).
  • the imaging apparatus 30 captures an image in the imaging range M.
  • the image data transmission unit 305 transmits the image P to the control apparatus 60 via the communication device 7 and the communication system 400 (Step SA 1 ).
  • the working equipment attitude sensor 73 detects the attitude of the working equipment 4 .
  • the sensor data transmission unit 306 transmits the detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 to the control apparatus 60 via the communication device 7 and the communication system 400 (Step SA 2 ).
  • the sensor data transmission unit 306 transmits not only the detection data of the working equipment attitude sensor 73 but also the detection data of the position sensor 71 , the detection data of the vehicle attitude sensor 72 , and the detection data of the pressure sensor 74 to the control apparatus 60 via the communication device 7 and the communication system 400 .
  • Step SA 2 may be performed before the process in Step SA 1 , or the process in Step SA 1 and the process in Step SA 2 may be performed in parallel.
  • the image data reception unit 62 receives the image P transmitted from the work machine 1 via the communication device 6 .
  • the image processing unit 64 divides the image P into the image P 11 , the image P 12 , an image P 13 , the image P 14 , and the image P 15 (Step SB 2 ).
  • the display control unit 65 causes the display apparatus 50 to display the image P (Step SB 3 ).
  • the stroke end determination unit 304 determines whether the working equipment 4 is approaching the end position in the end zone of the movable range based on the detection data of the working equipment attitude sensor 73 (Step SA 3 ).
  • Step SA 3 When it is determined in Step SA 3 that the working equipment 4 is moving away from the end position in the end zone, or when it is determined that the working equipment 4 is not arranged in the end zone (Step SA 3 : No), the control apparatus 300 returns to the process in Step SA 1 .
  • Step SA 3 When it is determined in Step SA 3 that the working equipment 4 is operated to approach the end position in the end zone (Step SA 3 : Yes), the working equipment control unit 303 performs the cushion control (Step SA 4 ).
  • the working equipment control unit 303 outputs, to the flow rate control valve 23 , the control command for reducing the flow rate of the hydraulic oil supplied to the hydraulic cylinder 5 .
  • the speed of the working equipment 4 operated to approach the end position in the end zone decreases.
  • the alarm control unit 66 determines whether the working equipment 4 is approaching the end position in the alarm zone of the movable range based on the detection data of the working equipment attitude sensor 73 (Step SB 4 ).
  • Step SB 4 when it is determined that the working equipment 4 is moving away from the end position in the alarm zone, or when it is determined that the working equipment 4 is not arranged in each of the alarm zone and the end zone (Step SB 4 : No), the control apparatus 60 returns to the process in Step SB 1 .
  • Step SB 4 when it is determined that the working equipment 4 is operated to approach the end position in the alarm zone (Step SB 4 : Yes), the alarm control unit 66 outputs the alarm control signal (Step SB 5 ).
  • Step SA 3 may be performed by the control apparatus 60 in the remote control room 200 .
  • the determination in Step SB 4 may be performed by the control apparatus 300 of the work machine 1 .
  • the alarm control unit 66 causes the display apparatus 50 to output the alarm.
  • the operator in the remote place can recognize that the working equipment 4 is operated to approach the end position by looking at the alarm output from the display apparatus 50 .
  • the movable range of the working equipment 4 is determined based on the movable range of the rod of the hydraulic cylinder 5 .
  • the end position in the movable range of the working equipment 4 is defined based on the stroke end of the hydraulic cylinder 5 , and when the hydraulic cylinder 5 reaches the stroke end, the working equipment 4 is determined to reach the end position in the movable range.
  • the working equipment components may reach the end position in the movable range, due to mechanical restriction of the working equipment 4 , before the hydraulic cylinder 5 reaches the stroke end.
  • the end positions of the working equipment components may be set based on a mechanical structure of the working equipment 4 .
  • the end zone and the alarm zone may be set based on the mechanical structure of the working equipment 4 .
  • the stroke end determination unit 304 can determine whether or not the third working equipment component is approaching the end position in the movable range based on the detection data of the working equipment attitude sensor 73 and the correlation obtained in advance.
  • FIG. 12 is a block diagram illustrating a computer system 1000 according to the embodiment.
  • the computer system 1000 includes a processor 1001 such as a central processing unit (CPU), a main memory 1002 including a nonvolatile memory such as a read only memory (ROM) and a volatile memory such as a random access memory (RAM), a storage 1003 , and an interface 1004 including an input/output circuit.
  • the function of the control apparatus 60 and the function of the control apparatus 300 described above are stored in the storage 1003 as a computer program.
  • the processor 1001 reads the computer program from the storage 1003 , develops the computer program in the main memory 1002 , and executes the above-described processes according to the program. Note that the computer program may be distributed to the computer system 1000 via a network.
  • the computer program or the computer system 1000 arranged in the remote place can execute receiving the detection data of the attitude of the working equipment 4 and outputting the alarm control signal to the alarm device when determining that the working equipment 4 approaches or reaches the end position in the movable range based on the detection data according to the above-described embodiment.
  • the work machine 1 includes the communication device 6 that transmits the detection data of the working equipment attitude sensor 73 to the remote place.
  • the operator in the remote place can recognize that an impact may act on the working equipment 4 based on the detection data of the working equipment attitude sensor 73 .
  • the impact may act on the working equipment 4 .
  • the operator actually boards the work machine 1 it is possible to feel that the impact acting on the working equipment 4 .
  • the operator is in the remote place it is not possible to feel the impact acting on the working equipment 4 . Therefore, there is a possibility that the operator in the remote place further performs the remote control in which an excessive impact acts on the working equipment 4 .
  • the excessive impact acts on the working equipment 4 deterioration of the working equipment 4 may be accelerated.
  • the cushion control when the working equipment 4 reaches the end position many times, the working equipment 4 may deteriorate.
  • the detection data of the working equipment attitude sensor 73 is transmitted from the work machine 1 to the remote place.
  • the alarm is output in the remote place.
  • the operator in the remote place can recognize that the working equipment 4 is approaching the end position. Therefore, the operator in the remote place can perform the remote control to avoid applying the excessive impact to the working equipment 4 .
  • the operator in the remote place can operate the remote controller 40 such that the working equipment 4 does not reach the end position.
  • the cushion control is performed when the working equipment 4 moves from the alarm zone to the end zone.
  • the impact acting on the working equipment 4 is reduced.
  • the working equipment 4 reaches the end position against the intention of the operator in the remote place due to a communication delay of the image data acquired by the imaging apparatus 30 .
  • the possibility that the communication delay of image data acquired by the imaging apparatus 30 occurs is higher than a possibility that a communication delay of the operation signal of the remote controller 40 occurs.
  • the cushion control is performed based on the detection data of the working equipment attitude sensor 73 . As a result, even when the communication delay of the image data occurs, an impact on the working equipment 4 is suppressed.
  • the alarm control unit 66 starts to output the alarm control signal
  • the display apparatus 50 starts to output the alarm.
  • the display apparatus 50 starts to output the alarm before the cushion control of the working equipment 4 is performed.
  • the alarm control unit 66 continues to output the alarm control signal, and thus the display apparatus 50 continues to output the alarm.
  • the operator in the remote place can recognize that the working equipment 4 is approaching the end position in the end zone.
  • the alarm control unit 66 continues to output the alarm control signal, and thus the display apparatus 50 continues to output the alarm.
  • the operator in the remote place can recognize that the working equipment 4 has reached the end position.
  • the stroke end determination unit 304 determines whether the working equipment 4 exists in the end zone of the movable range based on the detection data of the working equipment attitude sensor 73 .
  • the stroke end determination unit 304 may determine whether the working equipment 4 exists in the end zone of the movable range based on an output of a proximity switch.
  • the proximity switch is a switch that operates when the working equipment 4 moves to the end position in the movable range.
  • the alarm control unit 66 causes the display apparatus 50 to output the alarm.
  • the alarm control unit 66 may cause at least a part of the first monitor device 501 , the second monitor device 503 , and the third monitor device 504 to output the alarm.
  • an alarm message such as “pay attention to lever operation” or “approaching stroke end” may be displayed.
  • the alarm control unit 66 may cause the sound output device or the buzzer to output an alarm sound as the alarm.
  • the alarm control unit 66 may vibrate the operating seat 45 as the alarm.
  • the alarm device may not be the display apparatus 50 , and may be at least one of the first monitor device 501 , the second monitor device 503 , the third monitor device 504 , the sound output device, the buzzer, and the vibration generator.
  • the alarm control unit 66 outputs the alarm before the working equipment 4 reaches the end position.
  • the alarm control unit 66 may output the alarm when determining that the working equipment 4 has reached the end position in the movable range based on the detection data of the working equipment attitude sensor 73 .
  • the alarm control unit 66 may output the alarm after the working equipment 4 that is operated to approach the end position has reached the end position.
  • the computer program or the computer system 1000 arranged in the remote place described with reference to FIG. 12 can execute receiving the detection data of the attitude of the working equipment 4 and outputting the alarm when it is determined that the working equipment 4 has reached the end position in the movable range based on the detection data.
  • the output of the alarm may be omitted.
  • the cushion control may be performed in a state that no alarm is output.
  • the cushion control may not be performed.
  • the cushion control may not be performed.
  • the cushion control may not be performed.
  • FIG. 13 is a diagram illustrating the operation of the work machine 1 according to the embodiment.
  • an impact may act on the working equipment 4 .
  • the cushion control is not performed, there is a possibility that an impact acts on the working equipment 4 by the working equipment 4 reaching the end position.
  • the alarm control unit 66 outputs the alarm control signal in a case where it is determined that a level of the impact acting on the working equipment 4 is equal to or greater than a threshold based on the detection data related to the working equipment 4 received by the sensor data reception unit 63 .
  • the threshold related to the level of impact is a predetermined value and is held in the alarm control unit 66 .
  • the working equipment 4 vibrates with an amplitude of a predetermined value or higher.
  • the attitude of the working equipment 4 changes such that the working equipment 4 reciprocates at the amplitude equal to or larger than a first threshold.
  • the alarm control unit 66 can determine whether the level of the impact is equal to or greater than the threshold based on the detection data of the attitude of the working equipment 4 detected by the working equipment attitude sensor 73 .
  • a pressure of the hydraulic oil in the hydraulic cylinder 5 rapidly changes. More specifically, when the impact acts on the working equipment 4 , the pressure of the hydraulic oil in the hydraulic cylinder 5 increases by a change amount equal to or greater than a second threshold.
  • the alarm control unit 66 can determine whether the level of the impact is equal to or greater than the threshold based on the detection data of the pressure of the hydraulic oil detected by the pressure sensor 74 .
  • the swinging platform 3 vibrates in a pitch direction with an amplitude or acceleration equal to or greater than a predetermined value.
  • the alarm control unit 66 can determine whether the level of the impact is equal to or greater than the threshold based on the detection data of the vehicle attitude sensor 72 including the inertial measurement unit (IMU) provided in the swinging platform 3 .
  • IMU inertial measurement unit
  • the working equipment 4 vibrates at an acceleration equal to or greater than a predetermined value.
  • the alarm control unit 66 can determine whether the level of the impact is equal to or greater than the threshold based on the detection data of the acceleration sensor.
  • FIG. 14 is a flowchart illustrating a remote control method of the work machine 1 according to the embodiment.
  • the alarm control unit 66 determines whether the level of the impact is equal to or greater than the threshold based on the detection data of the pressure of the hydraulic oil detected by the pressure sensor 74 .
  • the alarm control unit 66 can determine whether the level of the impact is equal to or greater than the threshold based on at least one of the detection data of the working equipment attitude sensor 73 , the detection data of the vehicle attitude sensor 72 , and the detection data of the acceleration sensor.
  • the operation signal for performing the remote control of the work machine 1 is transmitted from the operation signal transmission unit 61 of the control apparatus 60 to the control apparatus 300 (Step SB 11 ).
  • the image P is transmitted from the work machine 1 to the control apparatus 60 (Step SA 11 ).
  • the detection data of the sensor 70 is transmitted from the work machine 1 to the control apparatus 60 .
  • the sensor data transmission unit 306 transmits at least the detection data of the pressure of the hydraulic oil in the hydraulic cylinder 5 detected by the pressure sensor 74 to the control apparatus 60 via the communication device 7 and the communication system 400 (Step SA 12 ).
  • Step SA 12 may be performed before the process in Step SA 11 , or the process in Step SA 11 and the process in Step SA 12 may be performed in parallel.
  • the image data reception unit 62 receives the image P transmitted from the work machine 1 via the communication device 6 .
  • Image processing unit 64 divides the image P into the image P 11 , the image P 12 , the image P 13 , the image P 14 , and the image P 15 (Step SB 12 ).
  • the display control unit 65 causes the display apparatus 50 to display the image P (Step SB 13 ).
  • the alarm control unit 66 determines whether the level of the impact acting on the working equipment 4 is equal to or greater than the threshold based on the detection data of the pressure sensor 74 (Step SB 14 ).
  • Step SB 14 when it is determined that the level of the impact acting on the working equipment 4 is less than the threshold (Step SB 14 : No), the control apparatus 60 returns to the process in Step SB 12 .
  • Step SB 14 when it is determined that the level of the impact acting on the working equipment 4 is equal to or greater than the threshold value (Step SB 14 : Yes), the alarm control unit 66 outputs the alarm control signal (Step SB 15 ).
  • Step SB 14 may be performed by the control apparatus 300 of the work machine 1 .
  • the computer program or the computer system 1000 arranged in the remote place described with reference to FIG. 12 can execute receiving the detection data related to the working equipment 4 and outputting the alarm control signal to the alarm device in a case where it is determined that the level of the impact acting on the working equipment 4 is equal to or greater than the threshold based on the detection data.
  • the control apparatus 60 of the remote control room 200 functions as the alarm control unit 66 .
  • the control apparatus 300 of the work machine 1 may function as the alarm control unit 66 .
  • the alarm control signal output from the alarm control unit 66 provided in the work machine 1 may be transmitted to the remote control room 200 via the communication system 400 , and the alarm device provided in the remote control room 200 may output the alarm based on the alarm control signal from the alarm control unit 66 .
  • the work machine 1 is a loading excavator.
  • the work machine 1 may be a backhoe.
  • the work machine 1 may be a bulldozer or a wheel loader as long as the work machine is equipped with the working equipment.

Landscapes

  • Engineering & Computer Science (AREA)
  • Mining & Mineral Resources (AREA)
  • Civil Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Structural Engineering (AREA)
  • Operation Control Of Excavators (AREA)
  • Component Parts Of Construction Machinery (AREA)
US17/922,639 2020-05-27 2021-04-30 Work machine remote control system Pending US20230167627A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2020092563A JP7507605B2 (ja) 2020-05-27 2020-05-27 作業機械の遠隔操作システム
JP2020-092563 2020-05-27
PCT/JP2021/017252 WO2021241141A1 (ja) 2020-05-27 2021-04-30 作業機械の遠隔操作システム

Publications (1)

Publication Number Publication Date
US20230167627A1 true US20230167627A1 (en) 2023-06-01

Family

ID=78744469

Family Applications (1)

Application Number Title Priority Date Filing Date
US17/922,639 Pending US20230167627A1 (en) 2020-05-27 2021-04-30 Work machine remote control system

Country Status (5)

Country Link
US (1) US20230167627A1 (ja)
JP (1) JP7507605B2 (ja)
AU (1) AU2021278492A1 (ja)
CA (1) CA3182563A1 (ja)
WO (1) WO2021241141A1 (ja)

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH05196004A (ja) * 1992-01-20 1993-08-06 Komatsu Ltd 作業機シリンダの自動クッション制御装置
JPH11158930A (ja) * 1997-11-28 1999-06-15 Hitachi Constr Mach Co Ltd 多関節建設機械の操作装置
JP4286494B2 (ja) 2002-06-19 2009-07-01 株式会社タダノ 作業機の安全装置
JP2008144378A (ja) 2006-12-06 2008-06-26 Shin Caterpillar Mitsubishi Ltd 遠隔操縦作業機の制御装置
JP7183744B2 (ja) 2018-11-30 2022-12-06 コベルコ建機株式会社 建設機械の遠隔操作装置

Also Published As

Publication number Publication date
CA3182563A1 (en) 2021-12-02
AU2021278492A1 (en) 2022-12-08
JP2021188310A (ja) 2021-12-13
WO2021241141A1 (ja) 2021-12-02
JP7507605B2 (ja) 2024-06-28

Similar Documents

Publication Publication Date Title
KR101815268B1 (ko) 건설 기계의 표시 시스템 및 그 제어 방법
CN111315935B (zh) 挖土机
KR20210106409A (ko) 쇼벨
KR102608801B1 (ko) 쇼벨
US20210010239A1 (en) Work machine and information processing apparatus
KR102659076B1 (ko) 쇼벨
CN118007731A (zh) 挖土机及挖土机的管理系统
CN112840283A (zh) 工程机械的远程操作装置
CA3029812A1 (en) Image display system of work machine, remote operation system of work machine, work machine, and method for displaying image of work machine
CN111788358B (zh) 挖土机
US20230078047A1 (en) Excavator and system for excavator
KR20230162605A (ko) 쇼벨의 표시장치, 쇼벨
US20240018750A1 (en) Display device for shovel, shovel, and assist device for shovel
JP7261111B2 (ja) 作業機械、および作業機械の制御方法
US20230167627A1 (en) Work machine remote control system
AU2021222454B2 (en) Remote operation system for work machine
JP2023041850A (ja) ショベル及びショベル用のシステム
US20240026637A1 (en) Work vehicle control system, work vehicle control method, and work vehicle
US20230050071A1 (en) Work machine and remote control system for work machine
JP7516068B2 (ja) 作業機械の遠隔操作システム
CN113454292A (zh) 挖土机
JP2021055433A (ja) ショベル
JP2023122359A (ja) 遠隔操作システム
JP2022137769A (ja) ショベル、情報処理装置
CN116964281A (zh) 挖土机、挖土机的支援系统

Legal Events

Date Code Title Description
AS Assignment

Owner name: KOMATSU LTD., JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:OHYAMA, YASUHIRO;MORINAGA, JUN;MINAGAWA, MASANORI;REEL/FRAME:061613/0226

Effective date: 20220907

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION