EP2723622B1 - Einrichtung und verfahren zum steuern eines schienenfahrzeugs - Google Patents

Einrichtung und verfahren zum steuern eines schienenfahrzeugs Download PDF

Info

Publication number
EP2723622B1
EP2723622B1 EP12750541.0A EP12750541A EP2723622B1 EP 2723622 B1 EP2723622 B1 EP 2723622B1 EP 12750541 A EP12750541 A EP 12750541A EP 2723622 B1 EP2723622 B1 EP 2723622B1
Authority
EP
European Patent Office
Prior art keywords
image
rail vehicle
control command
control
cab
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP12750541.0A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2723622A1 (de
Inventor
Karsten Rahn
Ulrich Bock
Christian Bode
Winfried ERNST
Claus FÜHNER
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Siemens AG
Original Assignee
Siemens AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Siemens AG filed Critical Siemens AG
Publication of EP2723622A1 publication Critical patent/EP2723622A1/de
Application granted granted Critical
Publication of EP2723622B1 publication Critical patent/EP2723622B1/de
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B61—RAILWAYS
    • B61L—GUIDING RAILWAY TRAFFIC; ENSURING THE SAFETY OF RAILWAY TRAFFIC
    • B61L3/00—Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal
    • B61L3/02—Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control
    • B61L3/06—Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control controlling by electromagnetic or particle radiation, e.g. by light beam
    • B61L3/065—Devices along the route for controlling devices on the vehicle or train, e.g. to release brake or to operate a warning signal at selected places along the route, e.g. intermittent control simultaneous mechanical and electrical control controlling by electromagnetic or particle radiation, e.g. by light beam controlling optically

Definitions

  • the invention relates to a device and a method for controlling a rail vehicle.
  • Rail vehicles are controlled today predominantly by drivers who are in a cab of the rail vehicle and control provided for controls, for example in the form of buttons, levers or switches operate.
  • rail vehicles such as the shunting locomotive of Deutsche Bahn V6, which are remotely controlled by radio from an operator not located in the cab, but on or near the rail track near the vehicle using a control panel.
  • the invention has for its object to provide a device for controlling a rail vehicle, which is cost-effective in terms of their structure and allows a simple rail vehicle control.
  • a device for controlling a rail vehicle characterized by an image capture device for capturing an image or an image sequence, and an image recognition device in communication with the image capture device that is suitable for receiving a control command or control command movement in the image capture device Recognize image or in the image captured by the image capture device image sequence and to generate a control command or the control command movement control signal for controlling the rail vehicle, wherein the image recognition device is adapted to detect an operating movement shown in the image sequence of an operator and to the control movement corresponding control signal produce.
  • a significant advantage of the device according to the invention is that it - with very few components - in comparison to today's usual driver's cabs, which include a variety of controls;
  • the device according to the invention requires only an image capture device and an image recognition device. Additional controls, such as levers, switches or buttons, which must be cumbersome wired, are not required in the inventive device for controlling the rail vehicle.
  • the rail vehicle can be controlled in a particularly simple and therefore advantageous manner in that the image recognition device is suitable for detecting an operating movement (eg, hand movement) of an operator shown in the image sequence and for generating a control signal corresponding to the operating movement.
  • the rail vehicle can thus be controlled, for example, solely by gestures of an operator.
  • the image capture device optionally a control of Rail vehicle from the interior of the vehicle, so for example, from within a cab of the rail vehicle, or alternatively allowed from outside the rail vehicle.
  • the image capture device is merely to be aligned such that it shows the cab at least in sections. If the rail vehicle instead should be controlled from outside, for example from the track, then the image capture device is to be directed to the surroundings of the rail vehicle.
  • the image capture device is arranged and aligned on the rail vehicle, that it detects a surrounding portion of the environment of the rail vehicle, and the image recognition device is suitable, a present in the recorded environment portion control command or a Recognize existing in the recorded environment section control command movement. Due to the arrangement and orientation of the image capture device, it is thus possible to generate control commands for controlling the rail vehicle without a driver having to be in the driver's cab.
  • a control from outside is particularly advantageous for broken or defective rail vehicles; Ascending a vehicle driver into a broken or defective rail vehicle usually requires previous precautionary measures (eg securing the rail vehicle against rolling away, shutting off the power), which may be very complicated. If now a broken-down or defective rail vehicle is controlled from the outside, the safety measures mentioned are unnecessary.
  • the image capture device can be arranged in or on the driver's cab so that it detects a section of the driver's cab of the rail vehicle.
  • the image recognition device is suitable Recognize existing in the recorded portion of the cab control command or existing in the recorded portion of the cab control command movement.
  • a control of the rail vehicle in the cab without using the usual buttons, switches or levers is possible.
  • Such components can be dispensed without replacement when using the described embodiment of the measuring device, which can save costs and especially space.
  • the image recognition device is suitable for recognizing in the image or the image sequence a control plate with a control symbol located thereon and for generating a control signal corresponding to the control symbol.
  • the control symbol is preferably an optical control symbol, for example a time-invariable (static) control symbol or a time-varying (dynamic) control symbol.
  • the control plate, on which the control symbol is located can be stationary or movable. For example, the control plate is moved by an operator, so that a picture sequence is produced, which shows the movement of the movable control plate.
  • the image recognition device is suitable for detecting a one-dimensional or two-dimensional barcode located on the control plate and for generating a control signal corresponding to the one-dimensional or two-dimensional barcode.
  • barcodes allow a very high information density to be accommodated on a very small area, so that even very complex control commands can be transmitted by means of a barcode.
  • the image recognition device may be suitable for recognizing a pictogram on the control plate which defines, for example, a travel command (eg stop, forward travel, reverse travel) and to generate a control signal corresponding to the pictogram.
  • a travel command eg stop, forward travel, reverse travel
  • the image capture device preferably comprises a camera which operates in the region of the optically visible light.
  • the image capture device may be adapted to detect infrared light and to generate image signals describing a two-dimensional image of the environmental section or the driver's cab.
  • the image capture device may include an ultrasound transmitting and receiving device capable of emitting ultrasonic signals, measuring reflected ultrasonic signals, and generating image signals describing an image of the environmental section or the driver's cab.
  • the image acquisition device processes at least two different signals, for example a combination of optically visible light and infrared light, a combination of optically visible light and ultrasound signals or a combination of infrared light and ultrasound signals , Particularly preferably, the image capture device will receive optically visible light, infrared light and ultrasonic signals.
  • the image recognition device is preferably configured such that it evaluates the images or image sequences recorded by the image capture device separately according to their source (for example, according to optically visible light, infrared light, ultrasound) and generates a control signal for controlling the rail vehicle only if the images in the images or picture sequences of the different ones Channels identified control command or control command movements correspond to each other or at least not contradict each other.
  • their source for example, according to optically visible light, infrared light, ultrasound
  • the image recognition device comprises a computing device (computer) and a memory in which an evaluation program for controlling the computing device is included.
  • the evaluation program When executed by the computing device, the evaluation program will evaluate the image recorded by the image capture device or the image sequence captured by the image capture device with an image or image sequence recognition process and recognize whether a control command or control command movement is contained in the image or image sequence, and after recognizing a generate such a control command character or such a control command movement a control signal or the control command movement corresponding control signal for controlling the rail vehicle.
  • the image recognition device is suitable for face recognition and generates a control signal for controlling the rail vehicle only if a control command or a control command movement originates from an authorized person whose face has been recognized.
  • an authentication can also be effected by means of a key, chip card, fingerprint reader, or password input.
  • the invention also relates to a rail vehicle having a device as described above.
  • a rail vehicle having a device as described above.
  • a display device which displays information for controlling the rail vehicle.
  • the display device is connected to the image recognition device and the image recognition device displays on the display device the respectively currently recognized control command symbol or the respectively currently recognized control command movement.
  • the invention also relates to a method for controlling a rail vehicle.
  • an image or an image sequence is recorded with an image capture device and it is checked whether a control command or control command movement is shown in the image or the image sequence and - in the case of recognition of a control command or control command movement - a recognized control command or the control command movement detected corresponding control signal is generated which describes a control command for controlling the rail vehicle.
  • an operating movement of an operator (100) shown in the image sequence is detected and a control signal (ST, ST ') corresponding to the operating movement is generated.
  • the image capturing device 20 has, for example, three image capture modules 21, 22 and 23, for example in the form of a camera (see reference numeral 21) for capturing or detecting visible light, an infrared camera (see reference numeral 22) for capturing or capturing infrared light and an ultrasonic module (see reference numeral 23) with an ultrasonic transmitting and ultrasonic receiving device.
  • the image capture device 20 Based on the information from the image acquisition modules, the image capture device 20 generates image signals B, which are transmitted to a computing device 40 (computer) of the image recognition device 30.
  • the computing device 40 is connected to a memory 50 in which a computer program P for controlling the computing device 40 is stored. If the computing device 40 executes the computer program P, then it will evaluate the image signals B of the image capture device 20 and check whether in the of the In the image capture device 20 recorded image or in the image captured by the image capture device 20 image sequence control command or a control command movement are recognizable, which are predefined in the computer program P and stored in the memory 50.
  • the image signals B of the image capture device 20 may be unprocessed or already preprocessed, for example prefiltered or pre-interpreted.
  • the computing device 40 If the computing device 40 detects a predefined control command signal or a predefined control command movement, then it generates a corresponding control signal ST with which the rail vehicle 10 is controlled.
  • the control signal ST is transmitted directly or indirectly via a train control system (not shown) to a drive 70 of the rail vehicle 10 in order to control the travel movement of the rail vehicle 10.
  • FIG. 1 one also sees an operator 100 pivoting her hand 110 in a circular manner.
  • This circular pivoting movement is to transmit, for example, the control command "stop".
  • the image capture device 20 detects the surrounding section U with the operator 100 and the circular motion and transmits a corresponding image signal B to the computing device 40 of the image recognition device 30, the computing device 40 will detect the circular movement of the operator 100 and determine the corresponding control command "Stop".
  • the corresponding stop command is transmitted in the form of the control signal ST to the drive 70, so that the rail vehicle 10 comes to a stop.
  • the computer program P may be programmed in a manner analogous to image recognition methods known today in the field of game consoles with a view to the recognition of control commands or commands.
  • image recognition methods known today in the field of game consoles with a view to the recognition of control commands or commands.
  • Microsoft company known that movements of persons detected and corresponding control signals can be generated.
  • the image acquisition device 20 has three image acquisition modules 21, 22 and 23.
  • the image capture device 20 may also comprise only a single or only two of the three image acquisition modules mentioned.
  • FIG. 2 shows by way of example the rail vehicle 10 according to FIG. 1 if there is an operator 100 in the vicinity U, who transmits another travel command, namely "forward travel".
  • This travel command can be displayed, for example, by raising and lowering the hand 110 of the operator 100.
  • a control signal ST ' is transmitted to the drive 70 for starting the rail vehicle 10.
  • the driver's cab 130 of the rail vehicle 10 may be unoccupied when transmitting the control commands from outside the rail vehicle 10.
  • FIG. 3 1 shows an exemplary embodiment of a rail vehicle in which the image capture device 20 is not aligned with the surroundings U of the rail vehicle 10, but is located inside the driver's cab 130 of the rail vehicle 10 and shows at least a section of the driver's cab 130.
  • a display device (display) 140 which can communicate with the computing device 40 via a control line (not shown), can be recognized.
  • the computing device 40 may display information that may be useful for controlling the rail vehicle (eg, information about the current driving condition, condition of the track system, etc.).
  • command elements can be displayed on the display device 140, which can be "moved" by predetermined movements (eg, hand movements) of the operator 100 located in the driver's cab 130 in order to control the rail vehicle 10.
  • the display device 140 may also be present a projection device which projects the information to be displayed, the detected control command and / or the command elements on the windscreen or another surface of the rail vehicle.
  • the computing device 40 may display on the display device 140 the respectively recognized control command ST for controlling the rail vehicle in order to indicate to the operator 100 located in the driver's cab 130 which operating command has currently been recognized in each case.
  • FIG. 4 shows the rail vehicle 10 again with an operator 100 located in the cab 130, but this instead of a stop command transmits a "ride" command.
  • the operator 100 raises and lowers her hand, which is detected by the image capture device 20 and transmitted to the computing device 40 in the form of image signals B. becomes.
  • the computing device 40 detects the corresponding gesture of the operator 100 and generates a control signal ST 'for the drive 70 to set the rail vehicle 10 in motion.
  • FIG. 5 shows an embodiment of a rail vehicle 10, in which two image sensing devices are present, one of which is denoted by the reference numeral 20 and the other by the reference numeral 20 '.
  • the one image capture device 20 is aligned with the surroundings U of the rail vehicle 10 and enables a detection of control command or control command movements (see image signal B), which are generated outside the rail vehicle in the environment U.
  • the image capture device 20 ' is located inside the driver's cab 130 and shows with image signals B' at least a portion of the cab, so that control commands or commands can also be generated, detected and implemented within the driver's cab 130.
  • I refer to the above observations in connection with Figures 1-4 referred, which apply here accordingly.
  • FIG. 6 shows an exemplary embodiment of a rail vehicle 10, which comprises an adjustable image capture device 20.
  • the image capture device 20 can be aligned with the driver's cab 130 (eg by swiveling, see arrow) in order to detect an operator 100 located in the driver's cab 130.
  • the image capture device 20 can also be oriented outward (eg by pivoting, see arrow) in order to be able to detect an operator 100 located outside the rail vehicle.
  • FIG. 7 shows an example of an operator 100 held in front of the image capture device 20 control plate 300, on which a one-dimensional barcode 310 is shown as a control symbol. Will the control plate 300 in the area brought the image capture device 20, so this corresponding image signals B is transmitted to the computing device 40. The computing device 40 will recognize the barcode 310 and generate a corresponding control command in the form of the control signal ST for the drive 70.
  • FIG. 8 1 shows an exemplary embodiment for controlling the rail vehicle 10, in which an operator 100 uses a control screen 400 with a two-dimensional bar code 410 for controlling the rail vehicle 10.
  • the control screen 400 As soon as the control screen 400 is brought into the area of the image capture device 20, it will generate image signals B which reach the computing device 40 of the image recognition device 30.
  • the computing device 40 will recognize the control symbol and generate a control signal ST corresponding to the control symbol for the drive 70.

Landscapes

  • Physics & Mathematics (AREA)
  • Electromagnetism (AREA)
  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Electric Propulsion And Braking For Vehicles (AREA)
  • Image Processing (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
EP12750541.0A 2011-08-08 2012-07-27 Einrichtung und verfahren zum steuern eines schienenfahrzeugs Not-in-force EP2723622B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE201110080592 DE102011080592A1 (de) 2011-08-08 2011-08-08 Einrichtung und Verfahren zum Steuern eines Schienenfahrzeugs
PCT/EP2012/064792 WO2013020837A1 (de) 2011-08-08 2012-07-27 Einrichtung und verfahren zum steuern eines schienenfahrzeugs

Publications (2)

Publication Number Publication Date
EP2723622A1 EP2723622A1 (de) 2014-04-30
EP2723622B1 true EP2723622B1 (de) 2016-04-27

Family

ID=46724345

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12750541.0A Not-in-force EP2723622B1 (de) 2011-08-08 2012-07-27 Einrichtung und verfahren zum steuern eines schienenfahrzeugs

Country Status (4)

Country Link
EP (1) EP2723622B1 (es)
DE (1) DE102011080592A1 (es)
ES (1) ES2585088T3 (es)
WO (1) WO2013020837A1 (es)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102012219901A1 (de) * 2012-10-31 2014-04-30 Siemens Aktiengesellschaft Verfahren und Vorrichtung zur Erhöhung der Haltegenauigkeit eines bewegten Objektes
DE102014211851A1 (de) * 2014-06-20 2015-12-24 Robert Bosch Gmbh Verfahren und Vorrichtung zum Ausgeben eines akustischen Warnsignals eines Schienenfahrzeugs und Warnsystem für ein Schienenfahrzeug
DE102018210242A1 (de) * 2018-06-22 2019-12-24 Siemens Aktiengesellschaft Anordnung zur Sicherung eines Schienenfahrzeugs gegen Handlungen von unbefugten Personen
CN115520247B (zh) * 2022-08-19 2024-04-12 昆明聚讯科技有限公司 一种铁路运输作业记录管理平台及方法

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19538022C1 (de) * 1995-10-12 1997-01-09 Schaltbau Ag Vorrichtung zur Steuerung eines Schienenfahrzeugs
US5978718A (en) * 1997-07-22 1999-11-02 Westinghouse Air Brake Company Rail vision system
JP3752246B2 (ja) * 2003-08-11 2006-03-08 学校法人慶應義塾 ハンドパターンスイッチ装置
DE102007038820A1 (de) * 2007-08-16 2009-02-26 Deutsches Zentrum für Luft- und Raumfahrt e.V. Rangierüberwachungsvorrichtung
US8233662B2 (en) * 2008-07-31 2012-07-31 General Electric Company Method and system for detecting signal color from a moving video platform
US8712610B2 (en) * 2008-09-18 2014-04-29 General Electric Company System and method for determining a characterisitic of an object adjacent to a route
US8073581B2 (en) * 2008-11-21 2011-12-06 Lockheed Martin Corporation Efficient data acquisition for track databases

Also Published As

Publication number Publication date
DE102011080592A1 (de) 2013-02-14
ES2585088T3 (es) 2016-10-03
EP2723622A1 (de) 2014-04-30
WO2013020837A1 (de) 2013-02-14

Similar Documents

Publication Publication Date Title
EP3194241B1 (de) Türsystem mit sensoreinheit und kommunikationselement
DE102013012466B4 (de) Bediensystem und Verfahren zum Bedienen einer fahrzeugseitigen Vorrichtung
DE102014113567B4 (de) Türsystem mit Sensoreinheit zur berührungslosen Fahrgastraumüberwachung
EP2726960A1 (de) Vorrichtung und verfahren zur berührungslosen erfassung von gegenständen und/oder personen und von diesen ausgeführten gesten und/oder bedienvorgängen
EP2723622B1 (de) Einrichtung und verfahren zum steuern eines schienenfahrzeugs
DE102016211494A1 (de) Steuerungseinrichtung für ein Kraftfahrzeug
WO2009071345A1 (de) Zentrales steuergerät für mehrere in einem kraftfahrzeug vorgesehene assistenzsysteme und kraftfahrzeug
DE102013213161A1 (de) Vorrichtung zum Erkennen von Gegenständen oder Lebewesen innerhalb eines Gefahrenbereichs unterhalb eines Fahrzeuges
EP3024713B1 (de) Verfahren sowie vorrichtung zum erkennen einer positionsänderung eines zumindest teilweisen abgeschalteten fahrzeugs
EP2921325A1 (de) Verfahren und vorrichtung zum fernsteuern eines kraftfahrzeugs, elektronisches kraftfahrzeugsteuersystem und kraftfahrzeug damit
EP3517398B1 (de) Verfahren zur innenraumzustandsüberwachung, sowie fahrzeug mit einer innenraumzustandsüberwachungseinrichtung
DE112019002604T5 (de) Virtuelle Bildanzeigevorrichtung
EP3254172B1 (de) Bestimmung einer position eines fahrzeugfremden objekts in einem fahrzeug
EP3230828A1 (de) Erfassungsvorrichtung zum erkennen einer geste und/oder einer blickrichtung eines insassen eines kraftfahrzeugs durch synchrone ansteuerung von leuchteinheiten
DE102012021422A1 (de) Vorrichtung und Verfahren zur Steuerung von Maschinen mit mindestens einem beweglichen Element, insbesondere Kranen
DE102014219538A1 (de) Verfahren zum Betreiben von mobilen Plattformen und mobile Plattform
EP3479884A1 (de) Fahrgeschäft, insbesondere achterbahn, sowie verfahren und computerprogramm zum betreiben eines derartigen fahrgeschäfts
DE102012203197A1 (de) Verfahren und System zum Steuern eines Prozesses
DE102017129214B4 (de) System und Verfahren zur Verbesserung eines Blickfelds
WO2018157887A1 (de) System und verfahren zur anzeige benutzerspezifischer informationen an einem bahnsteigtürsystem
DE102018213554A1 (de) Verfahren und Anzeigeeinrichtung zur Visualisierung einer Anordnung sowie Wirkungsweise von Umfeldsensoren eines Kraftfahrzeugs
DE102022121885B4 (de) Kameraanordnung für ein Kraftfahrzeug sowie Interaktionseinrichtung, Monitorsystem und Kraftfahrzeug
DE102016202526A1 (de) Verfahren und Vorrichtung zur Erkennung einer Bediengeste eines Nutzers, insbesondere in einem Kraftfahrzeug
DE102007059807A1 (de) Modelleisenbahnsystem
DE102011004462A1 (de) Anordnung sowie Verfahren zum Erzeugen eines Weichenstellbefehls für eine elektrisch ortsgestellte Weiche

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140123

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

RIN1 Information on inventor provided before grant (corrected)

Inventor name: FUEHNER, CLAUS

Inventor name: BOCK, ULRICH

Inventor name: RAHN, KARSTEN

Inventor name: BODE, CHRISTIAN

Inventor name: ERNST, WINFRIED

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20151111

RIN1 Information on inventor provided before grant (corrected)

Inventor name: FUEHNER, CLAUS

Inventor name: ERNST, WINFRIED

Inventor name: BOCK, ULRICH

Inventor name: RAHN, KARSTEN

Inventor name: BODE, CHRISTIAN

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 794408

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160515

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: GERMAN

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 502012006940

Country of ref document: DE

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2585088

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20161003

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160727

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160829

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160728

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160731

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 502012006940

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

26N No opposition filed

Effective date: 20170130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160731

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160731

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160727

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160727

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20170713

Year of fee payment: 6

Ref country code: DE

Payment date: 20170919

Year of fee payment: 6

Ref country code: FR

Payment date: 20170713

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: AT

Payment date: 20170606

Year of fee payment: 6

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20171023

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120727

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160427

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 502012006940

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: MM01

Ref document number: 794408

Country of ref document: AT

Kind code of ref document: T

Effective date: 20180727

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20180727

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180727

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190201

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180727

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180731

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20190917

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180728