EP1526925A2 - Vorrichtung zum sortieren von postgut umfassend einen farblesekopf mit zwei kameras - Google Patents

Vorrichtung zum sortieren von postgut umfassend einen farblesekopf mit zwei kameras

Info

Publication number
EP1526925A2
EP1526925A2 EP03753634A EP03753634A EP1526925A2 EP 1526925 A2 EP1526925 A2 EP 1526925A2 EP 03753634 A EP03753634 A EP 03753634A EP 03753634 A EP03753634 A EP 03753634A EP 1526925 A2 EP1526925 A2 EP 1526925A2
Authority
EP
European Patent Office
Prior art keywords
read head
camera
postal
image
cameras
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP03753634A
Other languages
English (en)
French (fr)
Other versions
EP1526925B1 (de
Inventor
Cyrille Prudhomme
Claude Mitte
Agnès Résidence Quinson GAMEZ CUATZIN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Solystic SAS
Original Assignee
Solystic SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Solystic SAS filed Critical Solystic SAS
Publication of EP1526925A2 publication Critical patent/EP1526925A2/de
Application granted granted Critical
Publication of EP1526925B1 publication Critical patent/EP1526925B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B07SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07CPOSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C3/00Sorting according to destination
    • B07C3/10Apparatus characterised by the means used for detection ofthe destination
    • B07C3/14Apparatus characterised by the means used for detection ofthe destination using light-responsive detecting means

Definitions

  • Postal sorting installation including a color reading head with two cameras
  • the invention relates to a read head for acquiring digital images of postal objects in a postal sorting installation, said read head comprising a high resolution multi-level gray camera.
  • the invention is more particularly intended for a postal sorting installation in which postal objects are processed by video coding so that an operator enters the recipient address from the digital image of the object which is displayed on a screen from his video coding station.
  • Digital images are generally transmitted in real time from a computerized management system of the sorting installation to the video coding station through a computer network. Given the processing rates, the digital images must have a low weight, otherwise it is not possible to video-code postal objects in real time.
  • the weight of images is limited by limiting their resolution and / or their color palette.
  • the images acquired are high resolution multilevel gray images because the size of high resolution color images is not compatible with the transfer rates of current computer networks.
  • the images displayed on the screen of the video coding station are currently in multi-gray levels, which reduces the operator's efficiency and his working comfort.
  • Document US-5912698 discloses a read head for the acquisition of digital images of postal objects in a postal sorting installation, comprising a high-resolution multi-level gray camera and a low-resolution color acquisition camera. The two cameras are arranged in the read head so as to provide two digital images acquired separately and superimposed on each postal object.
  • the object of the invention is to propose a read head as defined above, with which the superimposability of the digital images acquired separately is improved.
  • the subject of the invention is a read head for acquiring digital images of postal objects in a postal sorting installation, said read head comprising a high-resolution multi-level gray camera and a security camera. low resolution color acquisition, said cameras being arranged in said read head so as to simultaneously supply two digital images acquired separately and superimposable of each postal object, an observation window through which the two cameras each capture an image of each object
  • FIBILIE M Bf.ttPi.ACEfl.EMT ⁇ E ⁇ OE 21 ⁇ postal characterized in that it comprises an optical system for separating the light flux coming from the window towards each camera so that the two cameras simultaneously capture the same object points with the same angle of view.
  • a high-resolution multi-level gray image and a low-resolution color image are acquired simultaneously with the same viewing angle for each postal object.
  • the two images are transmitted to the video coding station where a high resolution color image of the postal object is displayed after combining the two images received.
  • the invention thus makes it possible to propose a high resolution color image by limiting the amount of additional data compared to the high resolution multilevel gray grayscale usually carried out.
  • the high resolution color image can also be used to improve the optical recognition of characters by facilitating the recognition of address blocks in the image of a postal object.
  • This high resolution color image can also be used to facilitate automatic recognition of postage marks.
  • the cameras are arranged at a right angle in the read head, and a separating blade is disposed substantially at forty-five degrees relative to each camera, so that the two cameras capture the same point object.
  • the two images can thus be acquired simultaneously by the same read head and through the same observation window, which improves their superimposability.
  • Figure 1 is a schematic representation of a read head according to the invention
  • Figure 2 is a flow diagram representation of the combination of two images to form a high resolution color image.
  • a read head 1 comprises a housing 2 which is a dark room with an observation window 2 'in front of which parade postal objects to be photographed.
  • a multi-level gray digital camera 3 high resolution is fixed to this box by being placed facing the observation window 2 'to acquire a digital image of a postal object visible in this window.
  • the read head 1 comprises another camera 4 which is a low resolution color camera.
  • This low-resolution color camera 4 is arranged so as to simultaneously acquire another digital image of the postal envelope superimposable on the image acquired by the multi-level gray camera 3.
  • the two cameras 3 and 4 can, for example, face the observation window 2 ′ to provide substantially superimposable images.
  • a digital processing of one of the two images may be implemented to correct this offset.
  • the acquisition of two superimposable images in the read head with the same viewing angle according to the invention can be ensured by using an optical system placed between the cameras and the observation window, this system being intended to separate into two the luminous flux coming from the window.
  • the optical separation system used could for example be a separating cube or else a fiber anamorphic agent.
  • the optical separation system chosen is a separating plate 5 arranged substantially at forty-five degrees relative to the plane of the observation window.
  • the luminous flux F1 coming from the observation window is separated by this plate into two fluxes F2 and F3 oriented at right angles to one another. More particularly, a first half F2 of the light flux crosses the separating plate 5 to reach the objective of the high-resolution multilevel gray camera 3 which is placed facing the observation window while being oriented at forty-five degrees relative to the separating blade.
  • a second half F3 of the light flux is reflected by the separating plate 5 so as to be oriented parallel to the plane of the window 2 'towards the low-resolution color camera 4 which is arranged at forty-five degrees relative to the separating plate.
  • the camera 3 faces the observation window 2 ′ and the camera 4 is arranged parallel to the plane of this observation window, the separating blade 5 being disposed at forty-five degrees relative to the plane of the window so as to form an angle of forty-five degrees with the axis of each of the cameras 3 and 4.
  • the mechanical and optical imperfections of the target point that constitutes the postal object thus strictly affect the multilevel gray camera and the color camera.
  • the superimposability of the images from these cameras is thus improved, and the two images can be acquired simultaneously.
  • Such a separating blade which is known per se is generally made of treated glass, and has the advantage of a very low cost compared to other possible optical systems. In the example illustrated in FIG.
  • the intensity of the light fluxes F2 and F3 received by each camera is substantially equal to half the intensity of the light flux F1 leaving the observation window.
  • This reduction in light intensity can advantageously be compensated for by a larger aperture of the diaphragm of each camera.
  • Another way to compensate for this reduction may be to increase the lighting of the postal item.
  • the resolution of the color camera is much lower than the resolution of the multi-level gray camera.
  • the amount of data to be transmitted to form a high resolution color image on the video coding screen is significantly reduced.
  • the resolution of the color camera in number of points per unit of length
  • the weight of the image from the low resolution color camera is worth 18% of the weight of the high-resolution multi-level gray image.
  • the amount of additional data to form a high resolution color image on the video coding screen is small compared to high resolution multi-level grayscale acquisition.
  • the two images are superimposable, to within a factor of size, which means that they represent the same part of the postal object at different resolutions.
  • one point in the low resolution color image corresponds to sixteen points in the high resolution multi-level gray image.
  • FIG 2 a flowchart representative of the processing of combining two images acquired by the read head to display a high resolution color image on the video coding station is shown very schematically.
  • This treatment could for example be implemented by computerized means at the level of a video coding station, or at the level of the computerized management system of the postal sorting installation.
  • the data supplied by the read head 1 includes, for each object, two superimposable images which are subjected to a processing unit which may be a video coding station or else the computerized management system of the postal sorting installation.
  • a low resolution color image is acquired by camera 4 in RGB (Red Green Blue) format which corresponds to the RGB-BR block of FIG. 2.
  • RGB Red Green Blue
  • a multilevel gray image is acquired by camera 3 which corresponds to the block MNG-HR in Figure 2.
  • Two options can then be considered.
  • the RGB-BR image is transmitted to a processing unit in which it is converted into a so-called TSL (Tint Saturation Luminance) representation.
  • TSL Tin Saturation Luminance
  • the luminance layer of this image is not transmitted, which further reduces the amount of data of the color image.
  • the converted color image whose luminance layer is not preserved is represented by the TS-BR block.
  • the luminance component (L) of a point in a TSL representation corresponds to the value of the shade of gray in a multilevel gray image.
  • the multi-level gray image is directly a luminance image, which is represented by the L-HR block.
  • the two images are then combined to form a high resolution image in TSL format, which is represented by the TSL-HR block.
  • this high resolution color image is constructed by combining the luminance layer from the high resolution multi-level gray image, with the T and S layers (hue and saturation) which come from the low resolution image from camera 4.
  • This combination is made possible by the fact that the two images are superimposable. More particularly, it is known that the eye is very sensitive to luminance while it is less sensitive to saturation and hue in an image. Consequently, the subsampling (by acquisition in low resolution) of the layers of saturation and hue is without great loss for the human eye.

Landscapes

  • Sorting Of Articles (AREA)
  • Character Input (AREA)
  • Discharge Of Articles From Conveyors (AREA)
  • Facsimile Heads (AREA)
  • Character Discrimination (AREA)
EP03753634A 2002-07-11 2003-07-10 Vorrichtung zum sortieren von postgut umfassend einen farblesekopf mit zwei kameras Expired - Lifetime EP1526925B1 (de)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
FR0208740 2002-07-11
FR0208740A FR2842383B1 (fr) 2002-07-11 2002-07-11 Installation de tri postal comprenant une tete de lecture couleur a deux cameras
PCT/FR2003/002182 WO2004007101A2 (fr) 2002-07-11 2003-07-10 Installation de tri postal comprenant une tete de lecture couleur a deux cameras

Publications (2)

Publication Number Publication Date
EP1526925A2 true EP1526925A2 (de) 2005-05-04
EP1526925B1 EP1526925B1 (de) 2006-11-15

Family

ID=29763746

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03753634A Expired - Lifetime EP1526925B1 (de) 2002-07-11 2003-07-10 Vorrichtung zum sortieren von postgut umfassend einen farblesekopf mit zwei kameras

Country Status (10)

Country Link
US (1) US7388600B2 (de)
EP (1) EP1526925B1 (de)
JP (1) JP4459809B2 (de)
AT (1) ATE345177T1 (de)
AU (1) AU2003271800A1 (de)
CA (1) CA2487374C (de)
DE (1) DE60309716T2 (de)
ES (1) ES2271632T3 (de)
FR (1) FR2842383B1 (de)
WO (1) WO2004007101A2 (de)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7809158B2 (en) * 2005-05-02 2010-10-05 Siemens Industry, Inc. Method and apparatus for detecting doubles in a singulated stream of flat articles
KR101531709B1 (ko) 2008-10-17 2015-07-06 삼성전자 주식회사 고감도 컬러 영상을 제공하기 위한 영상 처리 장치 및 방법
US10453046B2 (en) * 2014-06-13 2019-10-22 Conduent Business Services, Llc Store shelf imaging system
US10176452B2 (en) 2014-06-13 2019-01-08 Conduent Business Services Llc Store shelf imaging system and method
US9542746B2 (en) 2014-06-13 2017-01-10 Xerox Corporation Method and system for spatial characterization of an imaging system
US9659204B2 (en) 2014-06-13 2017-05-23 Conduent Business Services, Llc Image processing methods and systems for barcode and/or product label recognition
US9928438B2 (en) 2016-03-10 2018-03-27 Conduent Business Services, Llc High accuracy localization system and method for retail store profiling via product image recognition and its corresponding dimension database
US10002344B2 (en) 2016-10-17 2018-06-19 Conduent Business Services, Llc System and method for retail store promotional price tag detection
US10289990B2 (en) 2016-10-17 2019-05-14 Conduent Business Services, Llc Store shelf imaging system and method
US10019803B2 (en) 2016-10-17 2018-07-10 Conduent Business Services, Llc Store shelf imaging system and method using a vertical LIDAR
US10210603B2 (en) 2016-10-17 2019-02-19 Conduent Business Services Llc Store shelf imaging system and method
US20200137380A1 (en) * 2018-10-31 2020-04-30 Intel Corporation Multi-plane display image synthesis mechanism

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2089332A1 (en) * 1992-03-12 1993-09-13 Robert Bishop Method of and apparatus for object or surface inspection employing multicolor reflection discrimination
DE19532842C1 (de) * 1995-09-05 1996-12-19 Ibm Bildaufnahmesystem
US6303889B1 (en) * 2000-02-15 2001-10-16 Opex Corporation Method and apparatus for sorting documents into a pre-defined sequence
FR2812226B1 (fr) * 2000-07-25 2002-12-13 Mannesmann Dematic Postal Automation Sa Procede de traitement d'objets postaux de grande taille dans une installation de tri

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See references of WO2004007101A2 *

Also Published As

Publication number Publication date
EP1526925B1 (de) 2006-11-15
DE60309716T2 (de) 2007-09-20
DE60309716D1 (de) 2006-12-28
US7388600B2 (en) 2008-06-17
ES2271632T3 (es) 2007-04-16
CA2487374C (fr) 2008-12-23
WO2004007101A3 (fr) 2004-04-08
AU2003271800A1 (en) 2004-02-02
JP2005537123A (ja) 2005-12-08
JP4459809B2 (ja) 2010-04-28
FR2842383B1 (fr) 2004-12-03
WO2004007101A2 (fr) 2004-01-22
US20040233278A1 (en) 2004-11-25
ATE345177T1 (de) 2006-12-15
FR2842383A1 (fr) 2004-01-16
CA2487374A1 (fr) 2004-01-22

Similar Documents

Publication Publication Date Title
EP1526925B1 (de) Vorrichtung zum sortieren von postgut umfassend einen farblesekopf mit zwei kameras
TWI434574B (zh) 成像設備
US20240163409A1 (en) Color night vision cameras, systems, and methods thereof
EP2225608B1 (de) Vorrichtung zur beurteilung einer reifenfläche
EP0383646B1 (de) Hochauflösende Farbsichtanzeigevorrichtung
FR2524235A1 (fr) Systeme de transformation spatiale comprenant un generateur de signal repere
FR2686697A1 (fr) Dispositif de detection de defauts dans des pieces bicouches, notamment dans des cellules solaires.
Lovell et al. Stability of the color-opponent signals under changes of illuminant in natural scenes
WO2008068257A1 (fr) Dispositif d'affichage d'images par projection a deux etages de modulation
EP2011329A2 (de) Verfahren zum verarbeiten eines relativbeleuchtungsphänomens an einem digitalbild und assoziiertes verarbeitungssystem
FR2748621A1 (fr) Lecteur optique de caracteres a faisceau divise
CN105301741A (zh) 一种网络视频监控用高清透雾变焦镜头
Baltsavias et al. Scanners-A survey of current technology and future needs
EP0023861A1 (de) Kartographische Indikationsvorrichtung, insbesondere für die Flugnavigation
CN108093230A (zh) 一种摄像装置及电子设备
FR2675595A1 (fr) Melangeur optique sans perturbations de chemins optiques notamment pour visuel de casque.
EP4115327A1 (de) Verfahren zur unterstützung der detektion von elementen und zugehörige vorrichtung und plattform
CN107146207B (zh) 一种参数优化的低照度图像增强方法
FR3093886A1 (fr) Méthode de masquage d’objets dans un flux vidéo
EP0359634B1 (de) Periskop zum Tag- und Nachtbetrieb für Panzerfahrzeuge
CN218630380U (zh) 一种彩色线结构光成像系统
EP0703481A1 (de) Tag-Nacht Episkop
Toet Colorizing grayscale intensified nightvision images
Aneshansley et al. A multivision linear filter for capturing multispectral images
Govier Advancements in Short Wave Infrared Sensors

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050211

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: CH

Ref legal event code: NV

Representative=s name: SPIERENBURG & PARTNER AG, PATENT- UND MARKENANWAEL

REF Corresponds to:

Ref document number: 60309716

Country of ref document: DE

Date of ref document: 20061228

Kind code of ref document: P

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20061213

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070215

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070215

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070416

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2271632

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070731

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070710

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070516

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20190719

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20190822

Year of fee payment: 17

Ref country code: DE

Payment date: 20190719

Year of fee payment: 17

Ref country code: FR

Payment date: 20190723

Year of fee payment: 17

Ref country code: IT

Payment date: 20190730

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20190718

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20190719

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20190719

Year of fee payment: 17

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60309716

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20200801

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200710

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200710

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210202

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20211228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200710