EP2757541A4 - Three-dimensional object detection device - Google Patents

Three-dimensional object detection device

Info

Publication number
EP2757541A4
EP2757541A4 EP12832527.1A EP12832527A EP2757541A4 EP 2757541 A4 EP2757541 A4 EP 2757541A4 EP 12832527 A EP12832527 A EP 12832527A EP 2757541 A4 EP2757541 A4 EP 2757541A4
Authority
EP
European Patent Office
Prior art keywords
detection device
object detection
dimensional object
dimensional
detection
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP12832527.1A
Other languages
German (de)
French (fr)
Other versions
EP2757541B1 (en
EP2757541A1 (en
Inventor
Chikao Tsuchiya
Yasuhisa Hayakawa
Shinya Tanaka
Osamu Fukata
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nissan Motor Co Ltd
Original Assignee
Nissan Motor Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nissan Motor Co Ltd filed Critical Nissan Motor Co Ltd
Publication of EP2757541A1 publication Critical patent/EP2757541A1/en
Publication of EP2757541A4 publication Critical patent/EP2757541A4/en
Application granted granted Critical
Publication of EP2757541B1 publication Critical patent/EP2757541B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/588—Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/136—Segmentation; Edge detection involving thresholding
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/165—Anti-collision systems for passive traffic, e.g. including static obstacles, trees
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/167—Driving aids for lane monitoring, lane changing, e.g. blind spot detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/806—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for aiding parking
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8066—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring rearward traffic
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20224—Image subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Image Processing (AREA)
  • Traffic Control Systems (AREA)
  • Mechanical Engineering (AREA)
EP12832527.1A 2011-09-12 2012-07-27 Three-dimensional object detection device Active EP2757541B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2011198007 2011-09-12
PCT/JP2012/069091 WO2013038818A1 (en) 2011-09-12 2012-07-27 Three-dimensional object detection device

Publications (3)

Publication Number Publication Date
EP2757541A1 EP2757541A1 (en) 2014-07-23
EP2757541A4 true EP2757541A4 (en) 2015-06-17
EP2757541B1 EP2757541B1 (en) 2020-09-09

Family

ID=47883057

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12832527.1A Active EP2757541B1 (en) 2011-09-12 2012-07-27 Three-dimensional object detection device

Country Status (9)

Country Link
US (1) US9349057B2 (en)
EP (1) EP2757541B1 (en)
JP (1) JP5614504B2 (en)
CN (1) CN103797529B (en)
BR (1) BR112014002366B1 (en)
MX (1) MX2014001500A (en)
MY (1) MY171030A (en)
RU (1) RU2544775C1 (en)
WO (1) WO2013038818A1 (en)

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5413516B2 (en) * 2010-08-19 2014-02-12 日産自動車株式会社 Three-dimensional object detection apparatus and three-dimensional object detection method
CN104246821B (en) 2012-04-16 2016-08-17 日产自动车株式会社 Three-dimensional body detection device and three-dimensional body detection method
US10678259B1 (en) * 2012-09-13 2020-06-09 Waymo Llc Use of a reference image to detect a road obstacle
WO2014061123A1 (en) * 2012-10-17 2014-04-24 富士通株式会社 Image processing device, image processing program and image processing method
JP6150164B2 (en) * 2013-07-01 2017-06-21 株式会社リコー Information detection apparatus, mobile device control system, mobile object, and information detection program
IL227860B (en) * 2013-08-08 2019-05-30 Israel Aerospace Ind Ltd Classification of environment elements
KR101491305B1 (en) * 2013-08-27 2015-02-06 현대자동차주식회사 Apparatus and method for detecting obstacle
KR101439052B1 (en) * 2013-09-05 2014-09-05 현대자동차주식회사 Apparatus and method for detecting obstacle
US9978146B2 (en) * 2014-01-27 2018-05-22 Xylon d.o.o. Data-processing system and method for calibration of a vehicle surround view system
DE102014110663A1 (en) * 2014-07-29 2016-02-04 Connaught Electronics Ltd. Method for operating a camera system of a motor vehicle, driver assistance system and motor vehicle
CN104299244B (en) * 2014-09-26 2017-07-25 东软集团股份有限公司 Obstacle detection method and device based on monocular camera
JP6532229B2 (en) * 2014-12-18 2019-06-19 株式会社デンソーテン Object detection apparatus, object detection system, object detection method and program
WO2016113904A1 (en) * 2015-01-16 2016-07-21 株式会社日立製作所 Three-dimensional-information-calculating device, method for calculating three-dimensional information, and autonomous mobile device
JP6623729B2 (en) * 2015-12-04 2019-12-25 株式会社ソシオネクスト Ranging systems, moving objects and parts
JP6625446B2 (en) * 2016-03-02 2019-12-25 株式会社神戸製鋼所 Disturbance removal device
US10614321B2 (en) * 2016-03-24 2020-04-07 Nissan Motor Co., Ltd. Travel lane detection method and travel lane detection device
WO2017209313A1 (en) * 2016-05-30 2017-12-07 엘지전자 주식회사 Vehicle display device and vehicle
JP2018022343A (en) * 2016-08-03 2018-02-08 株式会社東芝 Image processing system and image processing method
US10183677B2 (en) * 2016-09-20 2019-01-22 Ford Global Technologies, Llc Ice and snow detection systems and methods
US10163015B2 (en) 2016-11-16 2018-12-25 Ford Global Technologies, Llc Detecting foliage using range data
JP7122806B2 (en) * 2017-02-24 2022-08-22 コニカミノルタ株式会社 OBJECT APPROACH DETECTION DEVICE AND OBJECT APPROACH DETECTION METHOD
US20190039515A1 (en) * 2017-08-01 2019-02-07 Nanning Fugui Precision Industrial Co., Ltd. System and method for warning against vehicular collisions when driving
JP7315303B2 (en) * 2018-01-26 2023-07-26 株式会社アイシン Image processing device
WO2019202627A1 (en) * 2018-04-16 2019-10-24 三菱電機株式会社 Obstacle detection device, automatic braking device using obstacle detection device, obstacle detection method, and automatic braking method using obstacle detection method
CN109064445B (en) * 2018-06-28 2022-01-04 中国农业科学院特产研究所 Animal quantity statistical method and system and storage medium
CN109785352B (en) * 2018-12-21 2022-11-22 广东工业大学 Intelligent efficient airborne radar point cloud analysis method
CN112017239B (en) * 2019-05-31 2022-12-20 北京市商汤科技开发有限公司 Method for determining orientation of target object, intelligent driving control method, device and equipment
JP7325296B2 (en) * 2019-10-25 2023-08-14 日産自動車株式会社 Object recognition method and object recognition system

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1964719A2 (en) * 2007-02-28 2008-09-03 Sanyo Electric Co., Ltd. Apparatus and method for monitoring a vehicle's surroundings
US20090268027A1 (en) * 2008-04-23 2009-10-29 Sanyo Electric Co., Ltd. Driving Assistance System And Vehicle

Family Cites Families (30)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE19921449C1 (en) * 1999-05-08 2001-01-25 Daimler Chrysler Ag Guide assistance when changing the lane of a motor vehicle
US6801244B2 (en) * 2000-02-29 2004-10-05 Kabushiki Kaisha Toshiba Obstacle detection apparatus and method
JP4156214B2 (en) * 2001-06-13 2008-09-24 株式会社デンソー Vehicle periphery image processing apparatus and recording medium
JP3871614B2 (en) * 2002-06-12 2007-01-24 松下電器産業株式会社 Driving assistance device
JP4259368B2 (en) * 2004-03-26 2009-04-30 三菱自動車工業株式会社 Nose view monitor device
JP2006072830A (en) * 2004-09-03 2006-03-16 Aisin Aw Co Ltd Operation supporting system and operation supporting module
JP4596978B2 (en) * 2005-03-09 2010-12-15 三洋電機株式会社 Driving support system
JP4661339B2 (en) 2005-05-11 2011-03-30 マツダ株式会社 Moving object detection device for vehicle
JP4934308B2 (en) * 2005-10-17 2012-05-16 三洋電機株式会社 Driving support system
JP2007221199A (en) * 2006-02-14 2007-08-30 Auto Network Gijutsu Kenkyusho:Kk On-vehicle camera display device and image processing unit
JP4930046B2 (en) * 2006-12-26 2012-05-09 日産自動車株式会社 Road surface discrimination method and road surface discrimination device
JP4924896B2 (en) * 2007-07-05 2012-04-25 アイシン精機株式会社 Vehicle periphery monitoring device
RU2352480C1 (en) 2007-07-24 2009-04-20 Открытое акционерное общество "Горизонт" Target detection and identification ground wheeled complex
JP5054612B2 (en) * 2008-05-15 2012-10-24 株式会社日立製作所 Approaching object detection device and approaching object detection method
JP5068779B2 (en) 2009-02-27 2012-11-07 現代自動車株式会社 Vehicle surroundings overhead image display apparatus and method
US20110001819A1 (en) * 2009-07-02 2011-01-06 Sanyo Electric Co., Ltd. Image Processing Apparatus
DE102010015079A1 (en) * 2010-04-15 2011-10-20 Valeo Schalter Und Sensoren Gmbh A method of displaying an image on a display device in a vehicle. Driver assistance system and vehicle
RU97092U1 (en) * 2010-04-21 2010-08-27 Руслан Владимирович Ситников VEHICLE SPACE ENVIRONMENT MONITORING SYSTEM
DE102010040803A1 (en) * 2010-09-15 2012-03-15 Continental Teves Ag & Co. Ohg Visual driver information and warning system for a driver of a motor vehicle
RU2549595C2 (en) * 2011-02-21 2015-04-27 Ниссан Мотор Ко., Лтд. System for detection of intermittent stationary objects and method to this end
US9290204B2 (en) * 2011-04-19 2016-03-22 Ford Global Technologies, Llc Hitch angle monitoring system and method
JP5664787B2 (en) * 2011-08-02 2015-02-04 日産自動車株式会社 Moving body detection apparatus and moving body detection method
WO2013125403A1 (en) * 2012-02-23 2013-08-29 日産自動車株式会社 Three-dimensional object detection device
JP5733467B2 (en) * 2012-03-02 2015-06-10 日産自動車株式会社 Three-dimensional object detection device
CN104246821B (en) * 2012-04-16 2016-08-17 日产自动车株式会社 Three-dimensional body detection device and three-dimensional body detection method
CN104509102B (en) * 2012-07-27 2017-12-29 日产自动车株式会社 Three-dimensional body detection means and detection device for foreign matter
RU2571368C1 (en) * 2012-07-27 2015-12-20 Ниссан Мотор Ко., Лтд. Device for detecting three-dimensional objects, method of detecting three-dimensional objects
US9591274B2 (en) * 2012-07-27 2017-03-07 Nissan Motor Co., Ltd. Three-dimensional object detection device, and three-dimensional object detection method
US9286800B2 (en) * 2012-12-30 2016-03-15 Robert Gordon Guidance assist vehicle module
US9129528B2 (en) * 2013-02-04 2015-09-08 Ford Global Technologies Trailer active back-up assist with lane width HMI

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1964719A2 (en) * 2007-02-28 2008-09-03 Sanyo Electric Co., Ltd. Apparatus and method for monitoring a vehicle's surroundings
US20090268027A1 (en) * 2008-04-23 2009-10-29 Sanyo Electric Co., Ltd. Driving Assistance System And Vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO2013038818A1 *

Also Published As

Publication number Publication date
CN103797529B (en) 2016-11-09
RU2544775C1 (en) 2015-03-20
US9349057B2 (en) 2016-05-24
MY171030A (en) 2019-09-23
JP5614504B2 (en) 2014-10-29
MX2014001500A (en) 2014-05-12
US20140168440A1 (en) 2014-06-19
CN103797529A (en) 2014-05-14
WO2013038818A1 (en) 2013-03-21
JPWO2013038818A1 (en) 2015-03-26
BR112014002366B1 (en) 2021-10-05
EP2757541B1 (en) 2020-09-09
EP2757541A1 (en) 2014-07-23
BR112014002366A2 (en) 2019-12-17

Similar Documents

Publication Publication Date Title
EP2757541A4 (en) Three-dimensional object detection device
EP2819090A4 (en) Three-dimensional object detection device
TWI563424B (en) Position detecting device
EP2755192A4 (en) Visually-distracted-driving detection device
EP2687055A4 (en) Improved device location detection
GB2503351B (en) Underwater Detection Device
EP2806287A4 (en) Object detection device
EP2860581A4 (en) Position detection device
EP2792528A4 (en) Position detection device
EP2821980A4 (en) Three-dimensional object detection device
EP2700538A4 (en) Obstacle detection device
GB201204792D0 (en) Detection techniques
GB201205160D0 (en) State detection
EP2879113A4 (en) Three-dimensional object detection device, three-dimensional object detection method
EP2843615A4 (en) Device for detecting three-dimensional object and method for detecting three-dimensional object
EP2827106A4 (en) Angle detection device
EP2787496A4 (en) Object detection device
EP2685271A4 (en) Magnetic detection device
GB2502467B (en) Object carrying device
EP2797320A4 (en) Object detection device
EP2879384B8 (en) Three-dimensional object detection device
EP2821958A4 (en) Three-dimensional object detection device
EP2772636A4 (en) Catalyst-degradation detection device
EP2667181A4 (en) Sensing device
EP2755184A4 (en) Object detection device

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20140408

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAX Request for extension of the european patent (deleted)
RA4 Supplementary search report drawn up and despatched (corrected)

Effective date: 20150518

RIC1 Information provided on ipc code assigned before grant

Ipc: G06T 7/00 20060101ALI20150511BHEP

Ipc: B60R 1/00 20060101ALI20150511BHEP

Ipc: G08G 1/16 20060101ALI20150511BHEP

Ipc: G06K 9/00 20060101AFI20150511BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20181002

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602012072277

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G08G0001160000

Ipc: G06K0009000000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G08G 1/16 20060101ALI20200602BHEP

Ipc: G06T 7/136 20170101ALI20200602BHEP

Ipc: B60R 1/00 20060101ALI20200602BHEP

Ipc: G06K 9/00 20060101AFI20200602BHEP

Ipc: G06T 7/12 20170101ALI20200602BHEP

INTG Intention to grant announced

Effective date: 20200618

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1312496

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200915

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012072277

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201210

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201209

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20201209

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1312496

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200909

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200909

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210111

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20210109

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012072277

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20210610

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602012072277

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G06K0009000000

Ipc: G06V0010000000

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20210731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210731

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210727

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210727

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20210731

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120727

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230621

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230620

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230620

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200909