EP1526925B1 - Installation de tri postal comprenant une tete de lecture couleur a deux cameras - Google Patents

Installation de tri postal comprenant une tete de lecture couleur a deux cameras Download PDF

Info

Publication number
EP1526925B1
EP1526925B1 EP03753634A EP03753634A EP1526925B1 EP 1526925 B1 EP1526925 B1 EP 1526925B1 EP 03753634 A EP03753634 A EP 03753634A EP 03753634 A EP03753634 A EP 03753634A EP 1526925 B1 EP1526925 B1 EP 1526925B1
Authority
EP
European Patent Office
Prior art keywords
read head
postal
cameras
camera
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP03753634A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1526925A2 (fr
Inventor
Cyrille Prudhomme
Claude Mitte
Agnès Résidence Quinson GAMEZ CUATZIN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Solystic SAS
Original Assignee
Solystic SAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Solystic SAS filed Critical Solystic SAS
Publication of EP1526925A2 publication Critical patent/EP1526925A2/fr
Application granted granted Critical
Publication of EP1526925B1 publication Critical patent/EP1526925B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B07—SEPARATING SOLIDS FROM SOLIDS; SORTING
    • B07C—POSTAL SORTING; SORTING INDIVIDUAL ARTICLES, OR BULK MATERIAL FIT TO BE SORTED PIECE-MEAL, e.g. BY PICKING
    • B07C3/00—Sorting according to destination
    • B07C3/10—Apparatus characterised by the means used for detection ofthe destination
    • B07C3/14—Apparatus characterised by the means used for detection ofthe destination using light-responsive detecting means

Definitions

  • the invention relates to a read head for acquiring digital images of postal objects in a postal sorting facility, said read head comprising a high resolution multilevel gray camera.
  • the invention is more particularly intended for a postal sorting facility in which postal objects are processed by video coding for an operator to enter the recipient address from the digital image of the object which is displayed on a screen from his video coding station.
  • the digital images are generally transmitted in real time from a computerized management system of the sorting facility to the video coding station through a computer network. Given the processing rates, the digital images must have a low weight otherwise it is not possible to videocode the postal objects in real time.
  • the weight of the images is restricted by limiting their resolution and / or their color palette.
  • the images acquired are high resolution gray multilevel images because the high resolution color image size is not compatible with the transfer rates of the current computer networks.
  • the images displayed on the screen of the video coding station are currently in multilevel gray, which reduces the efficiency of the operator and his comfort of work.
  • a read head for acquiring digital images of postal objects in a postal sorting facility, including a high resolution gray multilevel camera and a low resolution color acquisition camera.
  • the two cameras are arranged in the read head so as to provide two separately acquired and superimposable digital images of each mailpiece.
  • the object of the invention is to propose a reading head as defined above, with which the superposability of the digital images acquired separately is improved.
  • the subject of the invention is a reading head for acquiring digital images of postal objects in a postal sorting installation, said reading head comprising a high resolution gray multilevel camera and a video camera. low-resolution color acquisition, said cameras being arranged in said reading head so as to simultaneously provide two separately acquired and superimposable digital images of each postal object, an observation window through which the two cameras each capture an image of each object postal, characterized in that it comprises an optical system for separating the luminous flux from the window to each camera so that the two cameras simultaneously capture the same object points with the same angle of view.
  • a high resolution gray multilevel image and a low resolution color image are acquired simultaneously with the same angle of view for each postal object. Both images are transmitted to the video coding station where a high resolution color image of the postal object is displayed after combining the two received images.
  • the invention thus makes it possible to propose a high-resolution color image by limiting the amount of additional data compared with the conventional high-resolution multilevel acquisition of gray.
  • the high-resolution color image may also be used to improve the optical character recognition by facilitating the recognition of address blocks in the image of a postal object.
  • This high resolution color image can also be used to facilitate automatic recognition of postage marks.
  • the cameras are arranged at right angles in the read head, and a separating plate is disposed substantially at forty-five degrees with respect to each camera, so that the two cameras seize one and the same. point object.
  • the two images can thus be acquired simultaneously by the same reading head and through the same observation window, which improves their superimposability.
  • a read head 1 comprises a housing 2 which is a dark room with an observation window 2 'in front of which travel postal objects to photograph.
  • a multilevel gray digital camera 3 high resolution is fixed to this housing being disposed facing the observation window 2 'to acquire a digital image of a postal object visible in this window.
  • the read head 1 comprises another camera 4 which is a low resolution color camera.
  • This low-resolution color camera 4 is arranged so as to simultaneously acquire another digital image of the postal fold superimposed on the image acquired by the multilevel gray camera 3.
  • These two images, which are acquired separately, are intended to be combined, for example at the the video coding station to form a high resolution color image of the postal item, which will be detailed below.
  • the two cameras 3 and 4 may for example face the observation window 2 'to provide substantially superimposable images. In the case where the two acquired images have too large a shift, digital processing of one of the two images can be implemented to correct this shift.
  • the acquisition of two superimposable images in the read head with the same angle of view according to the invention can be ensured by using an optical system placed between the cameras and the observation window, this system being intended to separate in two the luminous flux coming from the window.
  • the optical separation system used may for example be a separator cube or a fiber anamorphic.
  • the selected optical separation system is a separator plate 5 disposed substantially forty-five degrees from the plane of the viewing window.
  • the luminous flux F1 coming from the observation window is separated by this plate into two flows F2 and F3 oriented at right angles to one another. More particularly, a first half F2 of the luminous flux passes through the separating blade 5 to reach the objective of the high resolution multilevel gray camera 3 which is placed facing the observation window while being oriented at forty-five degrees relative to the separating blade.
  • a second half F3 of the luminous flux is reflected by the separating plate 5 to be oriented parallel to the plane of the window 2 'to the low-resolution color camera 4 which is arranged at forty-five degrees with respect to the separating plate. As shown schematically in FIG.
  • the camera 3 faces the observation window 2 'and the camera 4 is arranged parallel to the plane of this observation window, the separating plate 5 being arranged at forty-five degrees with respect to the plane of the window so as to form an angle of forty-five degrees with the axis of each of the cameras 3 and 4.
  • the mechanical and optical imperfections of the target point that is the postal object thus affect in a strictly identical manner the multilevel camera of gray and the color camera. The superimposability of images from these cameras is thus improved, and the two images can be acquired simultaneously.
  • Such a separator blade which is known per se is generally made of treated glass, and has the advantage of a very low cost compared to other optical systems possible.
  • the intensity of the luminous fluxes F 2 and F 3 received by each camera is substantially half the intensity of the luminous flux F1 leaving the observation window.
  • This lowering of the luminous intensity can advantageously be compensated by a larger opening of the diaphragm of each camera. Another way to compensate for this lowering may be to increase the illumination of the postal item.
  • the resolution of the color camera is much lower than the resolution of the multilevel camera of gray.
  • the amount of data to be transmitted to form a high resolution color image on the video encoding screen is significantly reduced.
  • the resolution of the color camera in number of points per unit length
  • the weight of the image from the low resolution color camera is worth 18% of the weight of the multilevel image of high resolution gray.
  • the amount of additional data to form a high resolution color image on the video coding screen is small compared to the high resolution multilevel acquisition of gray.
  • the two images are superimposable, to a size factor, which means that they represent the same part of the postal object at different resolutions.
  • a dot in the low-resolution color image corresponds to sixteen points in the multilevel high-resolution gray image.
  • a representative flow chart of the combination processing of two images acquired by the read head to display a high resolution color image on the video coding station is shown very schematically.
  • This treatment can for example be implemented by computerized means at a video coding station, or at the level of the computerized management system of the postal sorting facility.
  • the data provided by the read head 1 comprises for each object two superimposable images which are submitted to a processing unit which may be a video coding station or the computerized management system of the postal sorting facility.
  • a low resolution color image is acquired by the camera 4 in the RGB (Red Green Blue) format, which corresponds to the RGB-BR block of FIG. 2.
  • a multilevel gray image is acquired by the camera 3, which corresponds to the block MNG-HR in Figure 2. Two options may then be considered.
  • the RGB-BR image is transmitted to a processing unit in which it is converted into a representation called TSL (Hue Saturation Luminance).
  • TSL Human Saturation Luminance
  • the luminance layer of this image is not transmitted, which further reduces the amount of data in the color image.
  • the converted color image whose luminance layer is not conserved is represented by the TS-BR block.
  • the luminance component (L) of a point in a representation TSL corresponds to the value of the shade of gray in a multilevel image of gray.
  • the multilevel gray image is directly a luminance image, which is represented by the L-HR block.
  • the two images are then combined to form a high resolution image in the TSL format, which is represented by the TSL-HR block.
  • this high-resolution color image is constructed by combining the luminance layer from the multilevel gray-scale high resolution image with the T and S (hue and saturation) layers that are derived from the low-resolution image from the camera 4. This combination is made possible by the fact that the two images are superimposable.
  • the eye is very sensitive to luminance while it is less sensitive to saturation and hue in an image. Therefore, the subsampling (by low resolution acquisition) of the saturation and hue layers is without much loss to the human eye.

Landscapes

  • Sorting Of Articles (AREA)
  • Character Input (AREA)
  • Character Discrimination (AREA)
  • Discharge Of Articles From Conveyors (AREA)
  • Facsimile Heads (AREA)
EP03753634A 2002-07-11 2003-07-10 Installation de tri postal comprenant une tete de lecture couleur a deux cameras Expired - Lifetime EP1526925B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
FR0208740A FR2842383B1 (fr) 2002-07-11 2002-07-11 Installation de tri postal comprenant une tete de lecture couleur a deux cameras
FR0208740 2002-07-11
PCT/FR2003/002182 WO2004007101A2 (fr) 2002-07-11 2003-07-10 Installation de tri postal comprenant une tete de lecture couleur a deux cameras

Publications (2)

Publication Number Publication Date
EP1526925A2 EP1526925A2 (fr) 2005-05-04
EP1526925B1 true EP1526925B1 (fr) 2006-11-15

Family

ID=29763746

Family Applications (1)

Application Number Title Priority Date Filing Date
EP03753634A Expired - Lifetime EP1526925B1 (fr) 2002-07-11 2003-07-10 Installation de tri postal comprenant une tete de lecture couleur a deux cameras

Country Status (10)

Country Link
US (1) US7388600B2 (ja)
EP (1) EP1526925B1 (ja)
JP (1) JP4459809B2 (ja)
AT (1) ATE345177T1 (ja)
AU (1) AU2003271800A1 (ja)
CA (1) CA2487374C (ja)
DE (1) DE60309716T2 (ja)
ES (1) ES2271632T3 (ja)
FR (1) FR2842383B1 (ja)
WO (1) WO2004007101A2 (ja)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7809158B2 (en) * 2005-05-02 2010-10-05 Siemens Industry, Inc. Method and apparatus for detecting doubles in a singulated stream of flat articles
KR101531709B1 (ko) 2008-10-17 2015-07-06 삼성전자 주식회사 고감도 컬러 영상을 제공하기 위한 영상 처리 장치 및 방법
US10176452B2 (en) 2014-06-13 2019-01-08 Conduent Business Services Llc Store shelf imaging system and method
US9542746B2 (en) 2014-06-13 2017-01-10 Xerox Corporation Method and system for spatial characterization of an imaging system
US10453046B2 (en) * 2014-06-13 2019-10-22 Conduent Business Services, Llc Store shelf imaging system
US9659204B2 (en) 2014-06-13 2017-05-23 Conduent Business Services, Llc Image processing methods and systems for barcode and/or product label recognition
US9928438B2 (en) 2016-03-10 2018-03-27 Conduent Business Services, Llc High accuracy localization system and method for retail store profiling via product image recognition and its corresponding dimension database
US10289990B2 (en) 2016-10-17 2019-05-14 Conduent Business Services, Llc Store shelf imaging system and method
US10210603B2 (en) 2016-10-17 2019-02-19 Conduent Business Services Llc Store shelf imaging system and method
US10002344B2 (en) 2016-10-17 2018-06-19 Conduent Business Services, Llc System and method for retail store promotional price tag detection
US10019803B2 (en) 2016-10-17 2018-07-10 Conduent Business Services, Llc Store shelf imaging system and method using a vertical LIDAR
US20200137380A1 (en) * 2018-10-31 2020-04-30 Intel Corporation Multi-plane display image synthesis mechanism

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2089332A1 (en) * 1992-03-12 1993-09-13 Robert Bishop Method of and apparatus for object or surface inspection employing multicolor reflection discrimination
DE19532842C1 (de) * 1995-09-05 1996-12-19 Ibm Bildaufnahmesystem
US6303889B1 (en) * 2000-02-15 2001-10-16 Opex Corporation Method and apparatus for sorting documents into a pre-defined sequence
FR2812226B1 (fr) * 2000-07-25 2002-12-13 Mannesmann Dematic Postal Automation Sa Procede de traitement d'objets postaux de grande taille dans une installation de tri

Also Published As

Publication number Publication date
WO2004007101A3 (fr) 2004-04-08
ATE345177T1 (de) 2006-12-15
WO2004007101A2 (fr) 2004-01-22
JP2005537123A (ja) 2005-12-08
US7388600B2 (en) 2008-06-17
AU2003271800A1 (en) 2004-02-02
US20040233278A1 (en) 2004-11-25
DE60309716D1 (de) 2006-12-28
CA2487374A1 (fr) 2004-01-22
DE60309716T2 (de) 2007-09-20
EP1526925A2 (fr) 2005-05-04
FR2842383A1 (fr) 2004-01-16
JP4459809B2 (ja) 2010-04-28
CA2487374C (fr) 2008-12-23
FR2842383B1 (fr) 2004-12-03
ES2271632T3 (es) 2007-04-16

Similar Documents

Publication Publication Date Title
EP1526925B1 (fr) Installation de tri postal comprenant une tete de lecture couleur a deux cameras
EP2864741B1 (en) Pathology slide scanners for fluorescence and brightfield imaging and method of operation
US20240163409A1 (en) Color night vision cameras, systems, and methods thereof
EP2225608B1 (fr) Dispositif d'evaluation de la surface d'un pneumatique
TWI434574B (zh) 成像設備
AU2011307151B2 (en) Digital multi-spectral camera system having at least two independent digital cameras
CN110365878B (zh) 一种摄像装置和方法
CN102124392A (zh) 照相机
CN209746285U (zh) 棱镜分光相机装置
Lovell et al. Stability of the color-opponent signals under changes of illuminant in natural scenes
EP2011329A2 (fr) Procédé de traitement d'un phénomène d'éclairement relatif sur une image numérique et système de traitement associé
Martinez High resolution digital imaging of paintings: the vasari project
FR2748621A1 (fr) Lecteur optique de caracteres a faisceau divise
CN108093230A (zh) 一种摄像装置及电子设备
FR2749944A1 (fr) Appareil de lecture d'images
FR3093886A1 (fr) Méthode de masquage d’objets dans un flux vidéo
CN218630380U (zh) 一种彩色线结构光成像系统
FR2612030A1 (fr) Systeme de prise de vue d'image video a trois capteurs solides fonctionnant en bispectral ou en haute resolution
Zinnecker et al. Near Infrared Images of Star-Forming Regions.
Govier Advancements in Short Wave Infrared Sensors
Aneshansley et al. A multivision linear filter for capturing multispectral images
Simon Choosing the Right Lens–When Round Must Become Square: Five factors to consider when choosing the right lens for a camera or imaging system
HU231356B1 (hu) Kombinált világos látóterű és fáziskontrasztos mikroszkópiás rendszer, valamint azzal ellátott képfeldolgozó berendezés
CN113728618A (zh) 一种摄像装置
CN118264914A (en) Image pickup device

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20050211

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL LT LV MK

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PT RO SE SI SK TR

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

Ref country code: IE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

Free format text: NOT ENGLISH

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: CH

Ref legal event code: NV

Representative=s name: SPIERENBURG & PARTNER AG, PATENT- UND MARKENANWAEL

REF Corresponds to:

Ref document number: 60309716

Country of ref document: DE

Date of ref document: 20061228

Kind code of ref document: P

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Free format text: LANGUAGE OF EP DOCUMENT: FRENCH

GBT Gb: translation of ep patent filed (gb section 77(6)(a)/1977)

Effective date: 20061213

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070215

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070215

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070215

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070416

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2271632

Country of ref document: ES

Kind code of ref document: T3

REG Reference to a national code

Ref country code: IE

Ref legal event code: FD4D

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20070817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070731

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070216

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20070710

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20070516

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20061115

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 14

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 15

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 16

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20190719

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20190822

Year of fee payment: 17

Ref country code: DE

Payment date: 20190719

Year of fee payment: 17

Ref country code: FR

Payment date: 20190723

Year of fee payment: 17

Ref country code: IT

Payment date: 20190730

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: BE

Payment date: 20190718

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20190719

Year of fee payment: 17

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20190719

Year of fee payment: 17

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 60309716

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: NL

Ref legal event code: MM

Effective date: 20200801

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20200710

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200710

Ref country code: NL

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200801

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210202

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200731

REG Reference to a national code

Ref country code: ES

Ref legal event code: FD2A

Effective date: 20211228

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200711

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200710