EP3851593B1 - Teleoperation device for construction machinery - Google Patents

Teleoperation device for construction machinery Download PDF

Info

Publication number
EP3851593B1
EP3851593B1 EP19888518.8A EP19888518A EP3851593B1 EP 3851593 B1 EP3851593 B1 EP 3851593B1 EP 19888518 A EP19888518 A EP 19888518A EP 3851593 B1 EP3851593 B1 EP 3851593B1
Authority
EP
European Patent Office
Prior art keywords
vibration
attachment
operator
remote
construction machine
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP19888518.8A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP3851593A4 (en
EP3851593A1 (en
Inventor
Daisuke Noda
Koji Yamashita
Yoichiro Yamazaki
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Kobelco Construction Machinery Co Ltd
Original Assignee
Kobelco Construction Machinery Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Kobelco Construction Machinery Co Ltd filed Critical Kobelco Construction Machinery Co Ltd
Publication of EP3851593A1 publication Critical patent/EP3851593A1/en
Publication of EP3851593A4 publication Critical patent/EP3851593A4/en
Application granted granted Critical
Publication of EP3851593B1 publication Critical patent/EP3851593B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/20—Drives; Control devices
    • E02F9/2025—Particular purposes of control systems not otherwise provided for
    • E02F9/205—Remotely operated machines, e.g. unmanned vehicles
    • G—PHYSICS
    • G08—SIGNALLING
    • G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
    • G08C17/00—Arrangements for transmitting signals characterised by the use of a wireless electrical link
    • G08C17/02—Arrangements for transmitting signals characterised by the use of a wireless electrical link using a radio link
    • E—FIXED CONSTRUCTIONS
    • E02—HYDRAULIC ENGINEERING; FOUNDATIONS; SOIL SHIFTING
    • E02F—DREDGING; SOIL-SHIFTING
    • E02F9/00—Component parts of dredgers or soil-shifting machines, not restricted to one of the kinds covered by groups E02F3/00 - E02F7/00
    • E02F9/26—Indicating devices
    • E02F9/264—Sensors and their calibration for indicating the position of the work tool
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/0038—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0011—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement
    • G05D1/005—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots associated with a remote control arrangement by providing the operator with signals other than visual, e.g. acoustic, haptic
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/22—Command input arrangements
    • G05D1/221—Remote-control arrangements
    • G05D1/222—Remote-control arrangements operated by humans
    • G05D1/224—Output arrangements on the remote controller, e.g. displays, haptics or speakers
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/20—Control system inputs
    • G05D1/22—Command input arrangements
    • G05D1/221—Remote-control arrangements
    • G05D1/222—Remote-control arrangements operated by humans
    • G05D1/224—Output arrangements on the remote controller, e.g. displays, haptics or speakers
    • G05D1/2244—Optic
    • G05D1/2247—Optic providing the operator with simple or augmented images from one or more cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q9/00—Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04Q—SELECTING
    • H04Q2209/00—Arrangements in telecontrol or telemetry systems
    • H04Q2209/80—Arrangements in the sub-station, i.e. sensing device
    • H04Q2209/82—Arrangements in the sub-station, i.e. sensing device where the sensing device takes the initiative of sending data
    • H04Q2209/823—Arrangements in the sub-station, i.e. sensing device where the sensing device takes the initiative of sending data where the data is sent when the measured values exceed a threshold, e.g. sending an alarm

Definitions

  • the present invention relates to a remote operation device for remotely operating a construction machine from a remote place distant from the construction machine.
  • Such information the operator refers to includes, for example, information on vibrations that are caused when a bucket of the attachment comes in contact with earth and sand, which is an excavation target, during the excavation work, and information on vibrations that are caused when a crusher of the attachment comes in contact with a construction, which is a crushing target, during the demolition work.
  • Patent Literature 1 discloses a remote operation system in which a work machine is provided with a vibration detection sensor and the vibration detection sensor detects vibrations caused by work that work unit performs.
  • a vibration detection sensor detects vibrations caused by work that work unit performs.
  • an operator's seat is provided with a vibration generation device.
  • a detection signal detected by the vibration detection sensor is wirelessly transmitted to this vibration generation device.
  • the vibration generation device Upon receiving the incoming detection signal from the vibration detection sensor, the vibration generation device applies a vibration to the operator's seat.
  • Patent Literature 2 discloses a remote operation excavator in which a work force (an excavating force and a torsion) of a hydraulic excavator is detected by a work force detector and the detected work force is transmitted to a remote operation device in a remote place, via an antenna.
  • a controller converts the excavating force and the torsion into two sinusoidal amplitudes and outputs the sinusoidal amplitudes to a vibration synthesizer, which constructs sine waves from the incoming sinusoidal amplitudes and outputs the sine waves to a vibration generation device.
  • the vibration generation device then vertically vibrates a seat, in which an operator is sitting, according to the sine wave based on the excavating force while vibrating the seat in the direction of its rotation according to the sine wave based on the torsion.
  • vibrations caused on an attachment during the excavation work or demolition work using a construction machine include not only the vibration that is caused when the attachment comes in contact with a work target but also a vibration that originates from a machine body, such as a lower travelling body and an upper slewing body, that is, a vibration that is caused when the attachment makes no contact with the work target.
  • a vibration that originates from a machine body such as a lower travelling body and an upper slewing body
  • Vibrations applied to the operator's seat in the remote place by the remote operation devices according to Patent Literatures 1 and 2 therefore, include various types of vibrations different from the vibration that is caused when the attachment comes in contact with the work target. It is thus difficult for the operator in the remote place to determine whether the attachment has come in contact with the work target, based on the vibrations applied to the operator's seat in the remote place.
  • Document KR 2010 0127963 A discloses a system and a method for preventing an unmanned excavator from being overturned, that informs a user of an overturn possibility by calculating a stability level value.
  • the system for preventing an unmanned excavator from being overturned comprises an unmanned excavator and a regulator.
  • the unmanned excavator has a sensor module.
  • the sensor module detects and transmits state information with position and status information.
  • the regulator remotely controls the unmanned excavator through wireless communication. When a stability level value of the unmanned excavator is more than a critical level value with an overturn possibility, the regulator warns a user by outputting vibration.
  • An object of the present invention is to provide a remote operation device for a construction machine, the remote operation device capable of determining whether a vibration caused on an attachment when remotely operating the construction machine is a vibration having a high possibility of the attachment coming in contact with a different object, and when determining that such a vibration has been caused, selectively transmitting the vibration to an operator in a remote place.
  • FIG. 1 is a side view of a hydraulic excavator 100 that is an example of a construction machine remotely operated by a remote operation device 101 according to an embodiment of the present invention.
  • FIG. 2 is a perspective view of an operator's seat 31, remote operation levers 32, and transmission devices 71 and 72 of the remote operation device 101 according to the embodiment.
  • FIG. 3 is a block diagram showing a functional configuration of the remote operation device 101.
  • the hydraulic excavator 100 (construction machine) and the remote operation device 101 make up a remote operation system.
  • FIGS. 1 and 2 directions are indicated as “up”, “down”, “left”, “right”, “front”, and “rear”. These directions are indicated to facilitate description of respective structures of the remote operation device 101 and the hydraulic excavator 100 according to the embodiment of the present invention, and do not set a limit to movement directions of the hydraulic excavator 100 or forms of using the same.
  • the hydraulic excavator 100 includes a lower travelling body 1, an upper slewing body 2 slewably mounted on the lower travelling body 1, and an attachment 6 attached to the upper slewing body 2.
  • the upper slewing body 2 has a slewing frame 2a coupled to the lower travelling body 1, and a cab 7 mounted on the slewing frame 2a.
  • the attachment 6 includes a boom 3 coupled to a front end of the slewing frame 2a so as to be able to rise and fall, an arm 4 turnably coupled to a tip of the boom 3, and a tip attachment 5 turnably coupled to a tip of the arm 4.
  • the tip attachment 5 is a bucket 5.
  • the cab 7 is mounted on a front part of the slewing frame 2a, where the cab 7 is located adjacent to the boom 3 in the left-to-right direction of the slewing frame 2a.
  • the cab 7 is an operator's cab in which an operator operates the hydraulic excavator.
  • the hydraulic excavator 100 further includes a plurality of hydraulic actuators 3a, 4a, and 5a that cause the attachment 6 to move, a slewing motor (not illustrated) that causes the upper slewing body 2 to slew, and a travelling motor (not illustrated) that causes the lower travelling body 1 to travel.
  • the plurality of hydraulic actuators include a boom cylinder 3a that causes the boom 3 to move, an arm cylinder 4a that causes the arm 4 to move, and a bucket cylinder 5a that causes the bucket 5 to move.
  • the remote operation device 101 is a device for remotely operating the hydraulic excavator 100 from a remote place distant from the hydraulic excavator 100.
  • the remote operation device 101 includes a camera 11 (image acquiring device), a display device 21, the operator's seat 31, a pair of the remote operation levers 32 and 32, a pair of travelling pedals 33 and 33, a pair of travelling levers 33A and 33A, a vibration detector 41, a controller 50, communication devices 61 and 62, a vibration generation device 71 (an example of a transmission device), and a sound generation device 72 (another example of the transmission device).
  • a machine controller 50A disposed on the hydraulic excavator 100 and a remote place controller 50B disposed in a remote place make up the controller 50.
  • the camera 11, the vibration detector 41, the machine controller 50A, and the communication device 61 are disposed on the hydraulic excavator 100 of FIG. 1 or in the vicinity thereof.
  • the display device 21, the operator's seat 31, the pair of remote operation levers 32 and 32, the pair of travelling pedals 33 and 33, the pair of travelling levers 33A and 33A, the remote place controller 50B, the vibration generation device 71, the sound generation device 72, and the communication device 62 are disposed in a remote place shown in FIG. 2 , which is distant from the hydraulic excavator 100.
  • the camera 11 is a device that can capture images. Specifically, the camera 11 can capture moving images.
  • the camera 11 has a given viewing angle (e.g., a viewing angle indicated by a two-dot chain line in FIG. 1 ) and is configured to capture an image in a range defined by the given viewing angle.
  • Information on an image acquired by the camera 11 is input to the remote place controller 50B disposed in the remote place, via the communication devices 61 and 62 shown in FIG. 3 .
  • the camera 11 acquires an image (work image) that is an image of a work area of the hydraulic excavator 100 and that corresponds to a visual field of the operator sitting in a seat in the operator's cab, i.e., the cab 7.
  • the camera 11 is set, for example, at a position corresponding in height to the eyes of the operator sitting in the seat in the operator's cab, i.e., the cab 7, and has a forwardly expanding visual field so as to be able to capture an image of a principle part of the attachment 6, such as the arm 4 and the bucket 5.
  • the display device 21 is a device that in the remote place, displays an image acquired by the camera 11.
  • the display device 21 receives information on an image (image signal), which is input to the remote place controller 50B via the communication devices 61 and 62, and displays the image.
  • the display device 21 may be provided as a display, such as a liquid crystal display and an organic electroluminescence display. In such a case, the display device 21 is disposed at a position that allows the operator sitting in the operator's seat 31 to observe an image displayed on the display device 21.
  • the display device 21 is not limited to such a display as described above, and may be provided, for example, as a projector or the like (not illustrated) that projects an image onto a screen or the like or as a head mount display (not illustrated) fitted on the operator's head.
  • the operator's seat 31 is a seat in which the operator sits in the remote place.
  • the operator's seat 31 includes a seat 34, a backrest 35, a headrest 36, and left and right armrests 37 and 37.
  • the seat 34 supports the lower half of the operator's body, specifically, the buttocks and a part of the legs (thighs).
  • the backrest 35 supports the upper half of the operator's body, specifically, the back.
  • the headrest 36 supports the operator's head, specifically, the back of the head.
  • the left and right armrests 37 and 37 support the operator's forearms when the operator, who is sitting in the operator's seat 31, operates the remote operation levers 32 and 32.
  • the left and right armrests 37 and 37 each have a longitudinally elongated shape so as to be able to support the operator's forearm.
  • the left and right armrests 37 and 37 are arranged respectively on the left side and the right side of the seat 34.
  • Attachment operations include a boom rising and falling operation for causing the boom 3 to make a rising and falling movement relative to the upper slewing body 2, an arm turning operation for causing the arm 4 to make a turning movement relative to the boom 3, a bucket turning operation for causing the bucket 5 to make a turning movement relative to the arm 4, and an attachment slewing operation for causing the upper slewing body 2 to make a slewing movement relative to the lower travelling body 1, thereby causing the attachment 6 to slew.
  • the pair of travelling pedals 33 and 33 are pedals for causing the lower travelling body 1 to travel.
  • the pair of travelling levers 33A and 33A are levers for causing the lower travelling body 1 to travel.
  • the operator sitting in the operator's seat 31 steps on the left and right travelling pedals 33 and 33 located in front of the operator's seat 31, or operates the left and right travelling levers 33A and 33A, thereby causing the lower travelling body 1 to travel.
  • This means that an operation for causing the lower travelling body 1 to travel is carried out as an operation of stepping on the travelling pedals 33 and 33 and as an operation of manually manipulating the travelling levers 33A and 33A as well.
  • each remote operation lever 32 When an operation is applied to each remote operation lever 32 by the operator, each remote operation lever 32 generates an operation signal corresponding to a state of the remote operation lever 32 (operation state), the state being determined according to an operation amount, an operation direction, and the like of the operation, and inputs the operation signal to the remote place controller 50B. Specifically, the operation state of each remote operation lever 32, the operation state being determined according to the operation amount, the operation direction, and the like, is converted to an electric signal (operation signal), which is input to the remote place controller 50B. Likewise, when an operation is applied to each travelling pedal 33 or each travelling lever 33A by the operator, each travelling pedal 33 or each travelling lever 33A generates an operation signal corresponding to an operation amount of the operation, and inputs the operation signal to the remote place controller 50B.
  • Each remote operation lever 32 includes a lever body to which an operation is applied by the operator, and an operation signal generating part that generates an operation signal corresponding to an operation amount, an operation direction, and the like of the operation and that inputs the operation
  • the operation signal input to the remote place controller 50B is then input to the machine controller 50A of the hydraulic excavator 100 via the communication devices 61 and 62.
  • the machine controller 50A of the hydraulic excavator 100 properly carries out signal processing, such as calculations, based on the incoming operation signal, to generate an instruction signal corresponding to the operation signal.
  • the instruction signal is input to control valves and the like for actuating the boom cylinder 3a, the arm cylinder 4a, the bucket cylinder 5a, the slewing motor, the travelling motor, and the like.
  • the operator can cause the hydraulic excavator 100 to make various movements, which include, specifically, the slewing movement of the attachment, the rising and falling movement of the boom, the turning movement of the arm, the turning movement of the bucket, and the travelling movement of the lower travelling body 1.
  • the vibration detector 41 only needs to be a detector capable of detecting a vibration caused on the attachment 6. As such a detector, the vibration detector 41 is not limited in its specific configuration.
  • the vibration detector 41 comprises, for example, an acceleration sensor 41X that detects a vibration component in an X direction, an acceleration sensor 41Y that detects a vibration component in a Y direction, and an acceleration sensor 41Z that detects a vibration component in a Z direction.
  • These acceleration sensors 41X, 41Y, and 41Z are arranged such that their directions of detection of accelerations are perpendicular to each other.
  • Data of vibration components detected respectively by the acceleration sensors 41X, 41Y, and 41Z includes pieces of data of vibration amplitudes, vibration frequencies, and the like.
  • the communication device 61 (transmission and reception device) is disposed on the hydraulic excavator 100 or in the vicinity thereof. Signals output from the camera 11, the vibration detector 41, and the like are input to the communication device 61 via the machine controller 50A.
  • the communication device 61 is configured to transmit these signals to the communication device 62 (transmission and reception device) disposed in the remote place, and receive signals transmitted from the communication device 62.
  • the communication device 62 (transmission and reception device) is disposed in the remote place.
  • the communication device 62 is configured to receive signals transmitted from the communication device 61 and input the signals to the remote place controller 50B.
  • the communication device 62 is configured to receive signals output from the remote place controller 50B and transmit the signals to the communication device 61 disposed on the hydraulic excavator 100 or in the vicinity thereof.
  • the communication devices 61 and 62 are configured to transmit and receive signals to and from each other through wireless communication.
  • the communication devices 61 and 62 are not limited thereto and may be configured to transmit and receive signals through wire communication.
  • the vibration generation device 71 is configured to generate a vibration that allows the operator to perceive vibration information through the operator's cutaneous sensation or the like.
  • Examples of the vibration generation device 71 include a vibration generation device having a motor (not illustrated) and a weight attached to a shaft of the motor such that the barycenter of the weight is offset to the shaft.
  • the vibration generation device 71 is not limited thereto. As shown in FIG. 2 , for example, the vibration generation device 71 is disposed on at least one member selected out of the seat 34, the backrest 35, the headrest 36, the left and right armrests 37 and 37, the pair of remote operation levers 32 and 32, and a floor plate 38 (floor member).
  • the vibration generation device 71 may be configured to vibrate in directions corresponding to the three directions of the vibration components detected by the acceleration sensors 41X, 41Y, and 41Z of the vibration detector 41.
  • an example of the vibration generation device 71 is cited as a vibration generation device including a front-to-rear vibration generating part that generates a vibration in the front-to-rear direction, a left-to-right vibration generating part that generates a vibration in the left-to-right direction, and a vertical vibration generating part that generates a vibration in the vertical direction.
  • the vibration generation device 71 of such a configuration can selectively generate a vibration in a direction corresponding to a direction of a maximum vibration component that will be described later, the direction being one of the front-to-rear direction, the left-to-right direction, and the vertical direction.
  • the sound generation device 72 is configured to generate a sound that allows the operator to perceive the vibration information through the operator's auditory sense.
  • the sound generation device 72 has, for example, an alarm buzzer 72 or a speaker 72, as shown in FIG. 2 .
  • the controller 50 is configured by, for example, a computer or the like. As shown in FIG. 3 , according to this embodiment, the controller 50 includes the machine controller 50A disposed on the hydraulic excavator 100 and the remote place controller 50B disposed in the remote place.
  • the remote place controller 50B has a coordinate system conversion section 52, a vibration determination section 53, and a transmission control section 54, as functional sections.
  • the coordinate system conversion section 52 is configured to convert vibration detection information, which is detected by the vibration detector 41 disposed on the bucket 5 of the attachment 6 and is defined on a first coordinate system with the bucket 5 as a reference, into vibration conversion information, which is defined on a second coordinate system with the operator's seat 31 in the remote place as a reference.
  • each of the directions in the first coordinate system with the bucket 5 as a reference changes depending on a movement of the bucket 5.
  • the vibration detection information defined on the first coordinate system is converted into the vibration conversion information defined on the second coordinate system.
  • the coordinate system conversion section 52 converts the coordinate system, for example, in the following manner.
  • the coordinate system conversion section 52 of the machine controller 50A calculates an angle of the bucket 5 with respect to the cab 7, and converts the directions on a vibration detected by the vibration detector 41 disposed on the bucket 5 (the directions in the first coordinate system) into the directions on the vibration with respect to the cab 7.
  • the converted directions on the vibration correspond to the directions in the coordinate system in the remote place.
  • the angle of the bucket 5 with respect to the cab 7 can be calculated based on posture data acquired by various types of known posture detectors capable of detecting a posture of the attachment 6. A specific example of such a posture detector is as follows.
  • the posture detector includes, for example, a boom angle sensor (not illustrated) capable of detecting an angle of the boom 3 (boom angle) with respect to the upper slewing body 2, an arm angle sensor (not illustrated) capable of detecting an angle of the arm 4 (arm angle) with respect to the boom 3, a bucket angle sensor (not illustrated) capable of detecting an angle of the bucket 5 (bucket angle) with respect to the arm 4, and a slewing angle sensor (not illustrated) capable of detecting an angle of the upper slewing body 2 (slewing angle) with respect to the lower travelling body 1.
  • a boom angle sensor capable of detecting an angle of the boom 3 (boom angle) with respect to the upper slewing body 2
  • an arm angle sensor capable of detecting an angle of the arm 4 (arm angle) with respect to the boom 3
  • a bucket angle sensor capable of detecting an angle of the bucket 5 (bucket angle) with respect to the arm 4
  • a slewing angle sensor capable of detecting an angle of the upper sle
  • the X-axis direction is defined as the front-to-rear direction, the Y-axis direction as the left-to-right direction, and the Z-axis direction as the vertical direction.
  • definition of directions on the second coordinate system is not limited thereto.
  • the vibration determination section 53 compares amplitudes of three vibration components detected by the acceleration sensors 41X, 41Y, and 41Z of the vibration detector 41, and identifies a maximum vibration component largest in amplitude among these vibration components.
  • the vibration determination section 53 also determines whether the amplitude of the maximum vibration component is equal to or larger than a preset amplitude threshold.
  • the amplitude threshold can be set based on, for example, data that can be acquired by experiments, simulations, or the like, specifically, based on amplitude data on a vibration that is caused when the bucket 5 comes in contact with a wall surface W or a ground G.
  • a vibration is caused on the bucket 5, as shown in FIGS. 4 and 5 .
  • the vibration of the bucket 5 is, for example, detected as three vibration components, by the acceleration sensors 41X, 41Y, and 41Z of the vibration detector 41, as shown in FIG. 6 .
  • These three vibration components are a vibration component in the X direction, a vibration component in the Y direction, and a vibration component in the Z direction.
  • Signals indicative of the vibration components detected by the vibration detector 41 are input to the remote place controller 50B via the communication devices 61 and 62.
  • the vibration determination section 53 Based on the incoming signals, the vibration determination section 53 identifies the maximum vibration component largest in amplitude (the vibration component in the X direction in the example of FIG. 6 ) among the vibration component in the X direction, the vibration component in the Y direction, and the vibration component in the Z direction. The vibration determination section 53 then determines whether the amplitude of the vibration component in the X direction, which is the maximum vibration component, is equal to or larger than the amplitude threshold.
  • the transmission control section 54 is configured to control operations of the transmission devices 71, 72 to allow the vibration information to be transmitted to the operator only when a preset vibration determination condition is met.
  • the vibration determination condition is set as a condition based on which the transmission control section 54 determines whether or not to transmit the vibration information on the vibration of the bucket 5, the vibration being detected by the vibration detector 41, to the operator in the remote place.
  • the vibration determination condition includes at least a condition (amplitude condition) that the amplitude of the maximum vibration component is equal to or larger than the preset amplitude threshold.
  • the vibration determination condition may also include a preset frequency condition on a frequency of the vibration of the bucket 5.
  • the frequency condition includes a condition that whether the frequency of the vibration of the bucket 5 is within a preset frequency range.
  • Data of the vibration components detected respectively by the acceleration sensors 41X, 41Y, and 41Z includes data on vibration frequencies. Therefore, the controller 50 can determine whether the frequency condition is met, based on data on frequencies of the vibration components.
  • the vibration determination condition may also include a condition (operation condition) that the attachment operation has been applied to the remote operation lever 32.
  • the controller 50 determines whether the operation condition included in the vibration determination condition is met, in the following manner. As described above, when the operator carries out an operation to the remote operation lever 32, the remote operation lever 32 generates an operation signal corresponding to a state of the remote operation lever 32 (operation state), the state being determined according to an operation amount, an operation direction, and the like of the operation, and inputs the operation signal to the remote place controller 50B. Therefore, the transmission control section 54 can determine whether the attachment operation has been applied to the remote operation lever 32, based on the operation signal input to the remote place controller 50B.
  • the transmission control section 54 may determine that the attachment operation has been applied to the remote operation lever 32 when finding the operation amount of the operation applied to the remote operation lever 32 to be equal to or larger than a preset reference value, and may determine that the attachment operation has not been applied to the remote operation lever 32 when finding the operation amount to be smaller than the reference value. Determination on whether the operation condition is met may be made by the machine controller 50A.
  • the vibration determination condition may include only the amplitude condition. Nevertheless, the vibration determination condition may further include at least one of the frequency condition and the operation condition, in addition to the amplitude condition. Further, the vibration determination condition may also include a condition different from the frequency condition and the operation condition, in addition to the amplitude condition.
  • FIG. 7 is a flowchart showing an example of a calculation control processing carried out by the machine controller 50A disposed on the hydraulic excavator 100, the machine controller 50A being included in the controller 50 of the remote operation device 101.
  • FIG. 8 is a flowchart showing an example of a calculation control processing carried out by the remote place controller 50B disposed in the remote place, the remote place controller 50B being included in the controller 50 of the remote operation device 101.
  • the machine controller 50A determines whether a vibration of the bucket 5 of the attachment 6 has been detected, based on information on the vibration of the bucket 5, the information being sent from the vibration detector 41 to the machine controller 50A (step S1).
  • the machine controller 50A transmits information on the bucket 5, such as vibration information and movement information on the bucket 5 of the attachment 6, to the remote place controller 50B via the communication devices 61 and 62 (step S2).
  • the information on the bucket 5 includes information on the vibration of the bucket 5, the vibration being detected by the vibration detector 41 (vibration detection information defined on the first coordinate system).
  • the remote place controller 50B determines whether it has received the information on the bucket 5, such as the vibration information (step S11).
  • the coordinate system conversion section 52 converts vibration detection information, which is detected by the vibration detector 41 disposed on the bucket 5 and is defined on the first coordinate system, into vibration conversion information, which is defined on the second coordinate system with the operator's seat 31 in the remote place as a reference (step S12).
  • the vibration determination section 53 identifies a maximum vibration component largest in amplitude among vibration components in three directions (the X direction, Y direction, and Z direction) that are detected by the acceleration sensors 41X, 41Y, and 41Z of the vibration detector 41, based on information on these vibration components (step S13).
  • the maximum vibration component is the vibration component in the X direction.
  • the vibration determination section 53 determines whether the amplitude of the maximum vibration component is equal to or larger than a preset amplitude threshold (step S14).
  • the vibration determination section 53 determines whether a frequency of the maximum vibration component is within a preset frequency range (step S15). When the frequency of the maximum vibration component is within the preset frequency range (YES in step S15), the transmission control section 54 controls at least one of the vibration generation device 71 and the sound generation device 72 so that the vibration information is transmitted to the operator (step S16).
  • the above frequency range may be set as a frequency range equal to or larger than a preset threshold or as a frequency range between a preset lower limit value and a preset upper limit value.
  • the frequency range is thus set in the above manner. Including such a frequency condition in the vibration determination condition prevents a case where a vibration with a frequency belonging to a specific frequency range (e.g. a frequency range associated with a frequency of a vibration caused by the sudden stop) is transmitted to the operator. As a result, a deterioration in the operability of the attachment is suppressed.
  • the transmission control section 54 may control at least one of the vibration generation device 71 and the sound generation device 72 so that the vibration information is transmitted to the operator when the frequency of the maximum vibration component is out of the preset frequency range.
  • the vibration generation device 71 is configured to be capable of vibrating in a plurality of directions.
  • the transmission control section 54 can control the vibration generation device 71 so as to cause it to selectively generate a vibration in a direction corresponding to a direction of the maximum vibration component.
  • the X direction is defined as the front-to-rear direction
  • the Y direction as the left-to-right direction
  • the Z direction as the vertical direction
  • Vibration detection information detected by the acceleration sensors 4lX, 41Y, and 41Z is, for example, data defined on the first coordinate system shown in FIG. 4 .
  • this vibration detection information is converted by the coordinate system conversion section 52 into, for example, coordinate system conversion information defined on the second coordinate system shown in FIG. 4 .
  • the coordinate system conversion information resulting from the conversion includes, for example, data of the three vibration components shown in FIG. 6 .
  • the transmission control section 54 controls the vibration generation device 71 so as to cause it to selectively generate only the vibration in the X direction corresponding to the direction of the maximum vibration component, that is, the vibration in the front-to-rear direction.
  • the operator is able to specifically understand a vibration actually caused on the bucket 5.
  • step S15 may be omitted from the flowchart.
  • the vibration determination condition may include a different condition set before step S16 of the flowchart of FIG. 8 .
  • the vibration determination condition may include a condition (operation condition) that the attachment operation has been applied to the remote operation lever 32.
  • an operation signal from each of the remote operation levers 32 and 32 is input to the controller 50 (the machine controller 50A or the remote place controller 50B).
  • the controller 50 determines whether the attachment operation has been applied to each of the remote operation levers 32 and 32.
  • a specific operation included in the attachment operations may be set in advance and stored in a memory (not illustrated) of the controller 50, in which case the vibration determination condition includes a condition (specific operation condition) that the specific operation has been applied to the remote operation lever 32.
  • the specific operation include the arm turning operation, which is one of a plurality of operations included in the attachment operations.
  • examples of the specific operation include an arm pushing operation of causing the arm 4 to move forward, which is an example of the arm turning operation.
  • the arm pushing operation corresponds to an operation of moving the bucket 5 in a forward direction D to bring the bucket 5 into contact with the wall surface W of the building, as shown in FIG. 4 , or to an operation of moving the bucket 5 in a downward direction D to bring the bucket 5 into contact with earth and sand of the ground, as shown in FIG. 5 .
  • the vibration determination condition By including the condition that the specific operation has been applied to the remote operation levers 32 and 32, in the vibration determination condition, a machine vibration generated as a vibration having a higher possibility of the bucket 5 having come in contact with a different object, such as a work target, can be selectively transmitted to the operator.
  • the vibration determination condition includes the condition (amplitude condition) that the amplitude of the maximum vibration component is equal to or larger than the amplitude threshold, and only when the vibration determination condition is met, transmitting the vibration information to the operator is allowed. Therefore, this prevents a case where the vibration information is transmitted to the operator when the bucket 5 is not in contact with a different object, such as a work target. As a result, a vibration having a high possibility of the bucket 5 coming in contact with the different object can be selectively transmitted to the operator. This allows the operator to carry out the attachment operation more exactly.
  • Determination on the amplitude condition that is, determination on the amplitude condition that the amplitude of the maximum vibration component largest in amplitude among the plurality of vibration components detected by the vibration detector 41 is equal to or larger than the amplitude threshold can be made by, for example, comparing detection values from the plurality of acceleration sensors relatively with each other and with the amplitude threshold, which requires a relatively simple configuration and simple control. Complication of the structure and control of the remote operation device 101, therefore, can be suppressed.
  • the vibration detector 41 is made up of the acceleration sensors 41X, 41Y, and 41Z, which can detect vibration components in three directions perpendicular to each other (a first direction, a second direction, and a third direction), respectively.
  • the first direction as the front-to-rear direction
  • the second direction as the left-to-right direction
  • the third direction as the vertical direction
  • various vibration components classified in the above manner that is, the upward vibration component, the rearward vibration component, the leftward vibration component, and the rightward vibration component are detected through highly sensitive detection performance.
  • the vibration determination condition may further include a condition that the attachment operation has been applied to the remote operation lever 32.
  • a vibration having a high possibility of the attachment coming in contact with a different object, such as a work target can be selectively transmitted to the operator in the remote place.
  • the present invention is not limited to the above-described embodiment.
  • the present invention includes, for example, the following aspects.
  • the remote operation device is applied not only to the hydraulic excavator 100 described exemplarily in the embodiment but can also be applied widely to other construction machines, such as a crane and a bulldozer.
  • the tip attachment is not limited to the bucket 5 described exemplarily in the embodiment.
  • the tip attachment may be, for example, a grapple that grasps and carries scraps in a scrapyard or the like, a crusher (disintegrator) for carrying out demolition work of demolishing a concrete building or the like, a breaker used to drill bedrocks, break rocks into pieces, and crush concrete, or a fork that holds an object to be transferred.
  • the transmission device only needs to be a device capable of transmitting vibration information on the vibration of the attachment to the operator in the remote place.
  • the transmission device is not limited to the vibration generation device 71 and the sound generation device 72 that are described exemplarily in the embodiment, but may be, for example, a light generation device, a vibration information display device, or the like.
  • the light generation device has, for example, an indicator lamp, a rotary lamp, and a signal lamp (which are not illustrated).
  • the vibration information display device has, for example, a display (not illustrated) that displays characters, figures, and the like that allow the operator to visually recognize the vibration information represented by the characters and figures.
  • the vibration detector only needs to be capable of detecting the vibration of the attachment.
  • the vibration detector disposed on the bucket 5 making up the tip of the attachment 6 is described exemplarily in the embodiment, the vibration detector may not be disposed on the bucket 5. It may be disposed on the arm 4, the boom 3, or the upper slewing body 2.
  • a plurality of vibration detectors may be disposed on a plurality of parts respectively.
  • the coordinate system conversion section 52, the vibration determination section 53, and the transmission control section 54 may be included not in the remote place controller 50B but in the machine controller 50A.
  • the controller 50 may be configured by only one of the machine controller 50A and the remote place controller 50B.
  • the inventors have focused on a fact that when the attachment comes in contact with a different object during work using the construction machine, such as excavation work and demolition work, it tends to cause a vibration with a distinctive feature that the amplitude of a vibration component in a specific direction is significantly larger than the amplitude of a vibration component in a different direction.
  • the size of the amplitude of a maximum vibration component largest in amplitude among a plurality of vibration components in a plurality of directions different from each other serves as important criteria for determining whether the attachment has come in contact with the different object.
  • a condition on the size of the amplitude of the maximum vibration component extracted from the plurality of vibration components is included in a vibration determination condition for determining whether the vibration caused on the attachment is a vibration caused by the attachment's coming in contact with the different object. This makes it possible to determine whether the vibration caused on the attachment is a vibration having a high possibility of the attachment coming in contact with the different object, and when it is determined that such a vibration has been caused, makes it possible also to selectively transmit the vibration to an operator in a remote place.
  • the remote operation device for the construction machine has been devised from such a point of view as described above.
  • the remote operation device is a device for remotely operating a construction machine having an attachment from a remote place distant from the construction machine.
  • the remote operation device includes: a vibration detector disposable on the construction machine and configured to detect a plurality of vibration components in a plurality of directions different from each other, the vibration components being included in a vibration caused on the attachment; a first communication device disposable on the construction machine and a second communication device disposable in the remote place with respect to the construction machine, the first and second communication devices being configured to transmit and to receive signals from each other; a transmission device configured to transmit vibration information on the vibration of the attachment received from the first communicaion device via the second communication device, to an operator in a remote place; and a transmission control section configured to control an operation of the transmission device.
  • a vibration determination condition for determining whether the vibration of the attachment is a vibration caused by the attachment's coming in contact with a different object is set in advance.
  • the vibration determination condition includes a condition that an amplitude of a maximum vibration component largest in amplitude among the plurality of vibration components detected by the vibration detector is equal to or larger than a preset amplitude threshold.
  • the transmission control section is configured to control an operation of the transmission device to allow the vibration information to be transmitted to the operator only when the vibration determination condition is met.
  • the vibration determination condition includes the condition (amplitude condition) that the amplitude of the maximum vibration component is equal to or larger than the amplitude threshold, and only when the vibration determination condition is met, transmitting the vibration information to the operator is allowed.
  • the vibration caused on the attachment is filtered, using the amplitude condition, to allow extraction of a vibration having a high possibility of the attachment coming in contact with a different object.
  • the vibration determination condition is not met, transmission of the vibration information to the operator is suppressed.
  • a vibration having a high possibility of the attachment coming in contact with the different object can be selectively transmitted to the operator.
  • the plurality of directions include three directions perpendicular to each other.
  • a vibration caused on the attachment mainly includes a lot of upward vibration components.
  • a work target during the demolition work or excavation work is a wall surface of a building in front of the construction machine, on the other hand, it is assumed that the attachment moves forward to come in contact with the wall surface.
  • a vibration caused on the attachment mainly includes a lot of rearward vibration components.
  • the attachment makes a slewing movement as a result of a slewing movement made by the upper slewing body and comes in contact with, for example, the wall surface of the building
  • the attachment is moving rightward or leftward in most cases.
  • a vibration caused on the attachment therefore, mainly includes a lot of leftward or rightward vibration components. Vibrations caused on the attachment by its various movements are roughly classified in the above manner.
  • the vibration detector can detect three vibration components in the three directions (the first direction, the second direction, and the third direction) perpendicular to each other, for example, by defining the first direction as the front-to-rear direction, the second direction as the left-to-right direction, and the third direction as the vertical direction, various vibration components classified in the above manner, that is, the upward vibration component, the rearward vibration component, the leftward vibration component, and the rightward vibration component are highly sensitively detected.
  • the remote operation device for the construction machine may further include a remote operation lever to which an attachment operation is applied by the operator in the remote place, the attachment operation being an operation for causing the attachment to move, in which the vibration determination condition may further include a condition that the attachment operation has been applied to the remote operation lever.
  • the vibration detector is disposable on the attachment and the vibration of the attachment being detected by the vibration detector, be defined on a first coordinate system with the attachment as a reference, and that the remote operation device further include an operator's seat in which an operator sits in the remote place, and a coordinate system conversion section that converts vibration detection information detected by the vibration detector and defined on the first coordinate system, into vibration conversion information defined on a second coordinate system with the operator's seat in the remote place as a reference.
  • each of the directions in the first coordinate system with the attachment as a reference changes depending on a movement of the attachment.
  • the vibration detection information defined on the first coordinate system is converted into the vibration conversion information defined on the second coordinate system.
  • the vibration determination condition may further include a preset frequency condition on a frequency of the vibration of the attachment, and the frequency condition may include a condition of determining whether the frequency of the vibration of the attachment is included in a preset frequency range. For example, when the attachment moving in the air stops suddenly without coming in contact with a different object, it is assumed that a vibration is caused on the attachment due to impact resulting from sudden stoppage of the attachment. However, it should be noted that the frequency of a vibration that is caused on the attachment due to such sudden stoppage is different from the frequency of a vibration that is caused on the attachment when the attachment comes in contact with the different object.
  • the frequency of the vibration caused on the attachment may serve as criteria for determining whether the attachment has come in contact with the different object.
  • the vibration determination condition includes the frequency condition. This makes it possible to extract a vibration having a higher possibility of the attachment coming in contact with the different object, and makes it possible also to selectively transmit the extracted vibration to the operator.
  • the transmission device may include a vibration generation device configured to vibrate in directions corresponding to the plurality of directions, and the transmission control section may control the vibration generation device to cause the vibration generation device to vibrate in a direction corresponding to a direction of the maximum vibration component.
  • a vibration of the attachment having come in contact with a different object includes vibration components among which, for example, a vibration component in the rearward direction (front-to-rear direction) is the largest in amplitude, the vibration component in the rearward direction (front-to-rear direction) is transmitted to the operator in the remote place.
  • the vibration component in the rearward direction front-to-rear direction
  • the operator is able to specifically understand a vibration actually caused on the attachment.
  • the remote operation device when remotely operating the construction machine, is capable of determining whether a vibration caused on the attachment is a vibration having a high possibility of the attachment coming in contact with a different object, and when determining that such a vibration has been caused, selectively transmitting the vibration to the operator.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Structural Engineering (AREA)
  • General Engineering & Computer Science (AREA)
  • Civil Engineering (AREA)
  • Mining & Mineral Resources (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Acoustics & Sound (AREA)
  • Human Computer Interaction (AREA)
  • Operation Control Of Excavators (AREA)
  • Selective Calling Equipment (AREA)
EP19888518.8A 2018-11-30 2019-11-01 Teleoperation device for construction machinery Active EP3851593B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2018224603A JP7176377B2 (ja) 2018-11-30 2018-11-30 建設機械の遠隔操作装置
PCT/JP2019/043083 WO2020110605A1 (ja) 2018-11-30 2019-11-01 建設機械の遠隔操作装置

Publications (3)

Publication Number Publication Date
EP3851593A1 EP3851593A1 (en) 2021-07-21
EP3851593A4 EP3851593A4 (en) 2022-01-12
EP3851593B1 true EP3851593B1 (en) 2023-09-13

Family

ID=70852368

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19888518.8A Active EP3851593B1 (en) 2018-11-30 2019-11-01 Teleoperation device for construction machinery

Country Status (5)

Country Link
US (1) US11952747B2 (ja)
EP (1) EP3851593B1 (ja)
JP (1) JP7176377B2 (ja)
CN (1) CN112823226B (ja)
WO (1) WO2020110605A1 (ja)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021228958A1 (en) 2020-05-13 2021-11-18 Energy Pool Developpement Method, system and device for power balancing in a power grid

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH09217382A (ja) 1996-02-09 1997-08-19 Hitachi Constr Mach Co Ltd 遠隔操縦掘削機
CA2390363C (en) * 1997-10-29 2003-11-18 Shin Caterpillar Mitsubishi Ltd. Remote radio operating system, and remote operating apparatus, mobile relay station and radio mobile working machine
JP3184972B2 (ja) * 1999-12-17 2001-07-09 株式会社タカハシワークス 遠隔操作によるツインアーム作業機
JP2003278159A (ja) * 2002-03-27 2003-10-02 Port & Airport Research Institute 遠隔操作による施工方法及びシステム
KR101090183B1 (ko) * 2009-05-27 2011-12-06 전자부품연구원 무인 굴삭기의 전도 방지 시스템 및 방법
JP5950607B2 (ja) 2012-02-15 2016-07-13 株式会社フジタ 遠隔操作システム
US8706363B2 (en) * 2012-07-30 2014-04-22 Caterpillar Inc. System and method for adjusting a boundary for a machine
KR102547626B1 (ko) * 2015-09-16 2023-06-23 스미도모쥬기가이고교 가부시키가이샤 쇼벨
SE541823C2 (en) 2016-06-09 2019-12-27 Husqvarna Ab Improved arrangement and method for operating a hydraulic cylinder
CN106643881A (zh) * 2016-11-24 2017-05-10 蓬溪斌鹏科技有限公司 一种用于施工现场的施工机械监测系统
GB2558266A (en) * 2016-12-23 2018-07-11 Caterpillar Inc Work tool positioning system
GB2558251B (en) * 2016-12-23 2019-12-04 Caterpillar Sarl A method of operating a work machine
JP7119285B2 (ja) * 2017-02-27 2022-08-17 株式会社Ihi 遠隔操縦システム
JP7127313B2 (ja) * 2018-03-19 2022-08-30 コベルコ建機株式会社 建設機械

Also Published As

Publication number Publication date
US11952747B2 (en) 2024-04-09
CN112823226B (zh) 2023-01-10
CN112823226A (zh) 2021-05-18
US20220002968A1 (en) 2022-01-06
JP7176377B2 (ja) 2022-11-22
JP2020084703A (ja) 2020-06-04
WO2020110605A1 (ja) 2020-06-04
EP3851593A1 (en) 2021-07-21

Similar Documents

Publication Publication Date Title
JP7151392B2 (ja) 建設機械の遠隔操作装置
WO2019244574A1 (ja) 掘削機、情報処理装置
JP5706051B1 (ja) 作業車両
KR101090183B1 (ko) 무인 굴삭기의 전도 방지 시스템 및 방법
CN111868335B (zh) 远程操作系统以及主操作装置
WO2020095945A1 (ja) ショベル、情報処理装置、情報処理方法、情報処理プログラム、端末装置、表示方法、表示プログラム
JP7071203B2 (ja) 作業機械
WO2020170687A1 (ja) 安全装置及び建設機械
WO2021024553A1 (ja) 作業機械
EP3851593B1 (en) Teleoperation device for construction machinery
JP7183744B2 (ja) 建設機械の遠隔操作装置
EP3923594B1 (en) Remote operation system
CN113614320A (zh) 作业机械用周边监视装置
WO2019181817A1 (ja) 建設機械
US20230128501A1 (en) Remote operation assistance device and remote operation assistance method
WO2019181814A1 (ja) 建設機械
WO2021246190A1 (ja) 実機状態監視システムおよび実機状態監視方法
US20220389682A1 (en) Overturning-risk presentation device and overturning-risk presentation method
WO2020170527A1 (ja) 作業機械用周辺監視装置
US20220356680A1 (en) Operation area presentation device and operation area presentation method

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20210414

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

A4 Supplementary search report drawn up and despatched

Effective date: 20211209

RIC1 Information provided on ipc code assigned before grant

Ipc: H04Q 9/00 20060101ALI20211203BHEP

Ipc: G08C 17/02 20060101ALI20211203BHEP

Ipc: E02F 9/20 20060101AFI20211203BHEP

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20230504

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602019037623

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231213

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231214

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1611434

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240113

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240115

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230913