EP1713259B1 - Photographing system - Google Patents

Photographing system Download PDF

Info

Publication number
EP1713259B1
EP1713259B1 EP06251975.6A EP06251975A EP1713259B1 EP 1713259 B1 EP1713259 B1 EP 1713259B1 EP 06251975 A EP06251975 A EP 06251975A EP 1713259 B1 EP1713259 B1 EP 1713259B1
Authority
EP
European Patent Office
Prior art keywords
frequency
image blur
blur correction
vibration
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
EP06251975.6A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP1713259A2 (en
EP1713259A3 (en
Inventor
Kenichi c/o Canon Kabushiki Kaisha Kubo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Publication of EP1713259A2 publication Critical patent/EP1713259A2/en
Publication of EP1713259A3 publication Critical patent/EP1713259A3/en
Application granted granted Critical
Publication of EP1713259B1 publication Critical patent/EP1713259B1/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/681—Motion detection
    • H04N23/6815—Motion detection by distinguishing pan or tilt from motion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/68—Control of cameras or camera modules for stable pick-up of the scene, e.g. compensating for camera body vibrations
    • H04N23/682—Vibration or motion blur correction
    • H04N23/685—Vibration or motion blur correction performed by mechanical compensation
    • H04N23/687—Vibration or motion blur correction performed by mechanical compensation by shifting the lens or sensor position
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B2205/00—Adjustment of optical system relative to image or object surface other than for focusing
    • G03B2205/0007—Movement of one or more optical elements for control of motion blur
    • G—PHYSICS
    • G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
    • G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
    • G03B2217/00—Details of cameras or camera bodies; Accessories therefor
    • G03B2217/005—Blur detection

Definitions

  • the present invention relates to a photographing system provided with a lens apparatus having an image blur correction function.
  • a frequency of vibration applied to a television camera lens apparatus ranges from 1 Hz to 15 Hz.
  • a vibration sensor which is used for detecting vibration applied to a lens apparatus, transmits a signal containing a signal component that is output in accordance with the vibration and a low frequency noise component of approximately 0.1 Hz.
  • a high pass filter is provided in a control unit of a vibration proof lens group.
  • the high pass filter fails to remove low frequency noise components sufficiently. This causes the vibration proof lens group to be driven by the noise components that pass through the high pass filter, resulting in an unintended blurred image of a subject. This is well known as a drift phenomenon that causes a subject to be viewed moving slowly within a frame.
  • Japanese Patent Laid-Open No. 1992-56831 discloses a technique for eliminating or reducing the effect of low frequency noise by inactivating an image blur correction function when a vibration sensor output satisfies a predetermined condition.
  • vibration sensors output an analogue signal
  • low frequency noise components vary largely depending on the individual vibration sensors. This causes a significant difficulty in setting a condition by which a signal component is determined to be or not to be a low frequency noise component.
  • a condition for distinguishing a low frequency noise component a small amplitude value may be set, for example. In this case, however, it may not be possible to detect every low frequency noise component, leaving the low frequency noise problem unsolved.
  • a vibration component to be corrected can be misrecognized as a noise component, which reduces the efficiency of a vibration proof function.
  • EP1102107 provides an alternative solution, namely a vibration isolator determines a speed of a vibration applied to a camera with a vibration speed sensor, and differentiates and integrates the speed, and corrects the integrated value to zero when the differentiated value is zero, and determines the position of the correcting optical system according to the integrated value.
  • an oscillation center of the correcting optical system is appropriately set at the origin, and an unwanted influence due to noise components in an output signal from the vibration speed sensor can be eliminated. Therefore, the vibration isolator can appropriately prevent an image blur.
  • FIG. 1 is a block diagram illustrating a general configuration of a photographing system provided with a lens apparatus 20 and a platform 30.
  • the lens apparatus 20 has an image blur correction function.
  • the platform 30 has detecting means for detecting a panning operation and a tilting operation of the lens apparatus 20.
  • the lens apparatus 20 also has the following components including: a vibration sensor 1 for detecting vibration applied to the lens apparatus 20; a high pass filter for removing a low frequency noise component contained in an output signal of the vibration sensor 1; an arithmetic circuit 3 for amplifying an output of the high pass filter 2 and converting an angular velocity signal from the vibration sensor 1 into a signal representing an angle; an A/D converter 4 for feeding an output signal received from the arithmetic circuit 3 to a CPU 8; an image blur correction lens group 5 for correcting image blur on an image plane by shifting a portion of the lens group in a direction perpendicular to an optical axis; an actuator 6 for driving the image blur correction lens group 5; a position detector 7 for detecting a position of the image blur correction lens group 5; a CPU 8 for calculating a control signal of the image blur correction lens group 5 on the basis of an output of the A/D converter 4; a D/A converter 9 for converting a signal calculated by the CPU 8 into an analogue signal; a
  • the flowchart of Fig. 2 illustrates a processing sequence of the CPU 8 of the lens apparatus 20.
  • Operations of the platform 30 include a panning operation associated with movement in a horizontal direction and a tilting operation associated with movement in a vertical direction.
  • the image blur correction lens group 5 is driven in horizontal and/or vertical directions. In this embodiment, for simplicity, only a panning operation of the platform 30 and a horizontal driving of the image blur correction lens group 5 will be described.
  • the CPU 8 initializes a register, memory, etc. in the CPU 8, at STEP S1.
  • This counter prevents an image blur correction flag (to be hereinafter described) from being unstable, such as repeatedly shifting between a set state and a clear state.
  • current output data of the panning detecting means 11 is read and stored in the buffer (PanData).
  • StepEP S13 If, in STEP S13, the image blur correction flag is found to be set, the processing proceeds to STEP S15.
  • STEP S15 vibration components to be corrected are extracted from the outputs of the vibration sensor 1 and the panning detecting means 11 so that the image blur correction control data (Control) is calculated.
  • This calculated image blur correction control data (Control) is output to the D/A converter, at STEP S16.
  • an angular velocity signal of the vibration sensor 1 is converted into a signal representing an angle, using the arithmetic circuit 3 constituted by hardware.
  • this conversion operation may also be implemented by software.
  • the vibration sensor 1 may not only be an angular velocity sensor but also be an acceleration sensor such as a linear acceleration sensor, etc.
  • the position detecting means according to this embodiment is configured to detect panning and tilting operations of a lens apparatus.
  • the position detecting means may also be a rotary encoder or a potentiometer for detecting the position of a focus lens or a zoom lens contained in a lens apparatus.
  • FIG. 3 the second embodiment according to the present invention will be described. Since this embodiment has the same general configuration as the first embodiment, the description thereof will be omitted.
  • the flowchart of Fig. 3 illustrates a processing procedure of the CPU 8 according to this embodiment.
  • the processing of STEP S101 through STEP S105 is the same as that of STEP S1 through STEP S5 of Fig. 1 , and thus the description thereof will also be omitted.
  • the values of two buffers (PanData, PanBuffer) are compared. If the absolute value of a difference between PanData and PanBuffer is smaller than a predetermined value A, the processing proceeds to STEP S107. If the absolute value of the difference between PanData and PanBuffer is greater than or equal to the predetermined value A, the processing proceeds to STEP Sill.
  • FIG. 4 illustrates a processing procedure of the CPU 8 according to this embodiment.
  • An output of the vibration sensor 1 (hereinafter referred to as a vibration sensor output) is read through the A/D converter and set as SensorData, at STEP S203.
  • an output of the panning detecting means 11 (hereinafter referred to as a panning detecting means output) is read and set as PanData.
  • the panning detecting means output is input to a frequency estimation function, so that an output signal frequency is calculated.
  • the calculated frequency is set as PanFreq.
  • This frequency estimation function outputs the highest frequency among frequencies (frequency component) contained in input data.
  • the frequency estimation function is implemented by software in this embodiment, it may also be implemented by hardware.
  • a difference signal (Sabun) corresponding to a difference between the vibration sensor output (SensorData) and the panning detecting means output (PanData) is calculated and stored in a buffer.
  • the panning detecting means 11 maintains an output of a constant value.
  • the vibration sensor outputs a low frequency signal which is a noise component, in spite of the absence of vibration. Therefore, a calculation of a difference between the vibration sensor output and the panning detecting means output yields a value corresponding to a noise component of the output signal from the vibration sensor 1. If the panning detecting means output is synchronized with neither the vibration sensor output nor the difference signal between the vibration sensor output and the panning detecting means output, the vibration sensor output can be determined to be noise.
  • a difference signal between a vibration sensor output and a panning detecting means output is monitored. This enables it to be determined whether an output signal from the vibration sensor 1 is a vibration component or a noise component more accurately, compared with a case where only a vibration sensor that produces an output containing a noise component is used.
  • the difference signal (Sabun) is input to a frequency estimation function, and the calculated frequency is set as SabunFreq.
  • the frequency of the output signal of the panning detecting means 11 and a lowest frequency fa of an image blur correction frequency band are compared.
  • This lowest frequency fa of an image blur correction frequency band is the lowest frequency in the band of frequencies for which image blur correction is performed. Thus, on a signal having a frequency lower than fa, no image blur correction is performed.
  • the frequency of the panning detecting means output signal (PanFreq) is lower than fa
  • the difference signal frequency (SabunFreq) and a noise determination frequency (f_noise) are compared, at STEP S209.
  • This noise determination frequency (f_noise) is the highest frequency of a low frequency noise component output from the vibration sensor 1.
  • the difference signal frequency (SabunFreq) is found to be higher than or equal to the noise determination frequency (f_noise) in STEP S209, the difference signal is determined to be a vibration component to be corrected.
  • the panning detecting means output signal (PanFreq) is found to be higher than or equal to the lowest frequency fa of the image blur correction frequency band, the panning detecting means output signal is determined to be a vibration component to be corrected.
  • the lowest frequency fa of the image blur correction frequency band is set to be lower than the noise determination frequency (f_noise), as shown in Fig. 5 .
  • the lowest frequency fa of the image blur correction frequency band is set to be the lowest frequency of a vibration component that can be detected by the panning detecting means 11. This enables image blur correction to be performed on any vibration component.
  • the noise determination frequency (f_noise) is set to be the highest frequency of a noise component that can be output by the vibration sensor 1, a drift phenomenon due to low frequency noise can be eliminated or reduced.
  • image blur correction control data (Control) is calculated on the basis of the vibration sensor output signal read through the high pass filter 2, arithmetic circuit 3, and A/D converter 4, at STEP S215.
  • the calculated data is output to the D/A converter 9, at STEP S216.
  • the processing of STEP S203 through STEP S216 will be repeated until the lens apparatus 20 is turned off.
  • the photographing system can also prevent an image blur correction flag from being unstable, such as repeatedly shifting between a set state and a clear state.
  • a counter may be employed for measuring a time period during which a frequency of the panning detecting means output signal (PanFreq) stays lower than the lowest frequency fa of an image blur correction frequency band.
  • Another counter may also be used for measuring a time period during which a difference signal frequency (SabunFreq) stays lower than a noise determination frequency (f_noise).
  • FIG. 6 the fourth embodiment according to the present invention will be described. Since the first embodiment and this embodiment have the same general configuration, the description thereof will be omitted.
  • the flowchart of Fig. 6 illustrates a processing procedure of the CPU 8.
  • the processing of STEP S301 through STEP S303 is the same as that of STEP S201 through STEP S203 of Fig. 4 , and thus the description thereof will be omitted.
  • Step S304 an output of the vibration sensor 1 (SensorData) is input to a frequency estimation function, and the calculated frequency of the vibration sensor output signal is set as SensorFreq.
  • This frequency estimation function is configured to output the highest frequency among frequencies (frequency component) contained in input data.
  • the frequency estimation function can be implemented either by software or by hardware to yield the same result.
  • an output of the panning detecting means 11 is read and set as PanData.
  • the processing of STEP S306 through STEP S307 is the same as that of STEP S206 through STEP S207, and thus the description thereof will be omitted.
  • the frequency of the vibration sensor output signal (SensorFreq) and a lowest frequency fa of an image blur correction frequency band are compared.
  • the lowest frequency fa of the image blur correction frequency band is the lowest frequency in the band of frequencies for which image blur correction is performed. Therefore, image blur correction is not performed on signals whose frequencies are lower than fa, such as low frequency noise components output from the vibration sensor 1.
  • the processing proceeds to STEP S309.
  • a difference signal frequency (SabunFreq) and a noise determination frequency (f_noise) are compared.
  • This noise determination frequency (f_noise) is the highest frequency of a low frequency noise component output from the vibration sensor 1.
  • An output of a frequency higher than the f_noise is a vibration component which is detected by the vibration sensor 1 and is to be corrected.
  • the difference signal frequency (SabunFreq) is lower than the noise determination frequency (f_noise)
  • the difference signal is a noise component of the output signal of the vibration sensor 1.
  • the difference signal frequency (SabunFreq) is found to be higher than or equal to the noise determination frequency (f_noise)
  • the difference signal is determined to be a vibration component corresponding to vibration which has been applied to the lens apparatus 20 and is to be corrected.
  • the image blur correction flag is set, at STEP S311, and the processing proceeds to STEP S313.
  • the vibration sensor output signal is determined to be a vibration component to be corrected.
EP06251975.6A 2005-04-14 2006-04-07 Photographing system Expired - Fee Related EP1713259B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2005117134A JP4689328B2 (ja) 2005-04-14 2005-04-14 撮影システム

Publications (3)

Publication Number Publication Date
EP1713259A2 EP1713259A2 (en) 2006-10-18
EP1713259A3 EP1713259A3 (en) 2012-03-28
EP1713259B1 true EP1713259B1 (en) 2018-02-14

Family

ID=36764669

Family Applications (1)

Application Number Title Priority Date Filing Date
EP06251975.6A Expired - Fee Related EP1713259B1 (en) 2005-04-14 2006-04-07 Photographing system

Country Status (3)

Country Link
US (2) US7613387B2 (ja)
EP (1) EP1713259B1 (ja)
JP (1) JP4689328B2 (ja)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4689328B2 (ja) * 2005-04-14 2011-05-25 キヤノン株式会社 撮影システム
JP5366379B2 (ja) * 2007-07-31 2013-12-11 キヤノン株式会社 撮影システム
JP2009044520A (ja) * 2007-08-09 2009-02-26 Sanyo Electric Co Ltd 防振制御回路
US20090059016A1 (en) 2007-08-29 2009-03-05 Chikatsu Moriya Image stabilization apparatus for camera
JP2009151028A (ja) * 2007-12-19 2009-07-09 Sanyo Electric Co Ltd 振動補正制御回路およびそれを搭載する撮像装置
JP5137556B2 (ja) * 2007-12-19 2013-02-06 オンセミコンダクター・トレーディング・リミテッド 振動補正制御回路およびそれを搭載する撮像装置
JP5253020B2 (ja) 2008-07-04 2013-07-31 キヤノン株式会社 像振れ補正装置およびその制御方法
JP5374217B2 (ja) 2009-04-22 2013-12-25 キヤノン株式会社 画像処理装置およびその方法
JP5616210B2 (ja) * 2010-12-09 2014-10-29 株式会社ミツトヨ 形状測定システム及び形状測定方法
GB2508471B (en) * 2012-11-29 2015-10-07 Cooke Optics Ltd Camera lens assembly

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2836053B2 (ja) * 1990-06-22 1998-12-14 キヤノン株式会社 像ぶれ補正装置及び該装置に適用される装置
JPH0566452A (ja) * 1991-09-10 1993-03-19 Canon Inc 防振機能付カメラ
US5592335A (en) * 1993-03-23 1997-01-07 Nikon Corporation Reduced diameter camera lens barrel with lens shift device and improved focus lens drive linkage
US6233009B1 (en) * 1993-09-08 2001-05-15 Canon Kabushiki Kaisha Image-shake correcting device for detecting vibration frequency and for changing vibration characteristics
JP4258857B2 (ja) * 1998-06-02 2009-04-30 株式会社ニコン 振動検出装置
JP2000039641A (ja) * 1998-07-24 2000-02-08 Fuji Photo Optical Co Ltd カメラの防振レンズ装置
DE60037710T2 (de) * 1999-11-16 2008-06-05 Fujinon Corp. Schwingungsisolator
JP2002049068A (ja) * 2000-08-04 2002-02-15 Canon Inc 防振ズームレンズ装置及びカメラシステム
JP2002218308A (ja) * 2001-01-16 2002-08-02 Canon Inc 撮像装置および撮像方法
JP2004184887A (ja) * 2002-12-06 2004-07-02 Fuji Photo Optical Co Ltd 像振れ補正装置
JP2004214729A (ja) * 2002-12-26 2004-07-29 Canon Inc 撮影装置及びその制御方法、プログラム
JP2004301939A (ja) * 2003-03-28 2004-10-28 Sony Corp カメラシステム、カメラ及び交換レンズ
JP4869572B2 (ja) * 2004-08-09 2012-02-08 株式会社ニコン 撮像装置
JP4689328B2 (ja) * 2005-04-14 2011-05-25 キヤノン株式会社 撮影システム

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
US7899314B2 (en) 2011-03-01
US7613387B2 (en) 2009-11-03
US20090324209A1 (en) 2009-12-31
US20060233539A1 (en) 2006-10-19
EP1713259A2 (en) 2006-10-18
JP4689328B2 (ja) 2011-05-25
EP1713259A3 (en) 2012-03-28
JP2006293218A (ja) 2006-10-26

Similar Documents

Publication Publication Date Title
EP1713259B1 (en) Photographing system
US8150250B2 (en) Camera body and camera system including the same
JP6362556B2 (ja) 制御装置、撮像装置、制御方法、プログラム、および、記憶媒体
US8159541B2 (en) Image stabilization method and apparatus
JP2010054986A (ja) 像ブレ補正機能を有する光学機器及びその制御方法
CN103176332A (zh) 图像稳定设备及其控制方法、光学设备和摄像设备
US20050201741A1 (en) Image blur correcting device
US20110158619A1 (en) Lens apparatus
JP4683268B2 (ja) 像振れ補正装置
JP3154776B2 (ja) 像振れ防止装置
JP2004295027A (ja) ブレ補正装置
KR100215448B1 (ko) 캠코더의 손떨림/이동속도 디스플레이장치
JP2000039640A (ja) 撮像装置及び揺れ補正装置及び揺れ補正方法
JPH05199449A (ja) 防振カメラ
JP2017219635A (ja) 画像処理装置及び方法、撮像装置、及び撮像システム
EP2393279B1 (en) Optical apparatus having image shake correcting function
US20090059016A1 (en) Image stabilization apparatus for camera
CA2177544C (en) Hand deviation correction apparatus
JP4305723B2 (ja) 像振れ補正装置
EP3739868B1 (en) Control apparatus, lens apparatus, imaging apparatus, control method, and program
JP2014041230A (ja) 像ブレ補正機能を有する光学機器及びその制御方法
JPH05232562A (ja) カメラの流し撮り装置
EP1508827A4 (en) PICTURE DEVICE, SIGNAL PROCESSING METHOD, RECORDING MEDIUM AND PROGRAM
JP2011170383A (ja) レンズ装置、及びそれを有する撮像装置
JPH11275449A (ja) ビデオカメラ

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU LV MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR MK YU

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 5/232 20060101AFI20120220BHEP

17P Request for examination filed

Effective date: 20120928

AKX Designation fees paid

Designated state(s): DE FR GB NL

17Q First examination report despatched

Effective date: 20150717

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20170425

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAL Information related to payment of fee for publishing/printing deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR3

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20170920

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB NL

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602006054687

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20180214

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20180214

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602006054687

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20181115

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20180514

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180514

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180414

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20200629

Year of fee payment: 15

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602006054687

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20211103