EP3358547A4 - Dispositif de traitement d'informations, procédé de traitement d'informations et programme - Google Patents

Dispositif de traitement d'informations, procédé de traitement d'informations et programme Download PDF

Info

Publication number
EP3358547A4
EP3358547A4 EP16850568.3A EP16850568A EP3358547A4 EP 3358547 A4 EP3358547 A4 EP 3358547A4 EP 16850568 A EP16850568 A EP 16850568A EP 3358547 A4 EP3358547 A4 EP 3358547A4
Authority
EP
European Patent Office
Prior art keywords
information processing
program
processing device
processing method
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP16850568.3A
Other languages
German (de)
English (en)
Other versions
EP3358547A1 (fr
EP3358547B1 (fr
Inventor
Suguru Aoki
Taketo AKAMA
Hideki Oyaizu
Yasutaka Hirasawa
Yuhi Kondo
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Group Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of EP3358547A1 publication Critical patent/EP3358547A1/fr
Publication of EP3358547A4 publication Critical patent/EP3358547A4/fr
Application granted granted Critical
Publication of EP3358547B1 publication Critical patent/EP3358547B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/095—Predicting travel path or likelihood of collision
    • B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B60W40/064—Degree of grip
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/23—Recognition of whole body movements, e.g. for sport training
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00—Input parameters relating to infrastructure
    • B60W2552/40—Coefficient of friction
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/40—Dynamic objects, e.g. animals, windblown objects
    • B60W2554/402—Type
    • B60W2554/4026—Cycles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/40—Dynamic objects, e.g. animals, windblown objects
    • B60W2554/402—Type
    • B60W2554/4029—Pedestrians
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Geometry (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Traffic Control Systems (AREA)
  • Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
EP16850568.3A 2015-09-29 2016-08-26 Dispositif de traitement d'informations, procédé de traitement d'informations et programme Active EP3358547B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2015191190 2015-09-29
PCT/JP2016/003885 WO2017056385A1 (fr) 2015-09-29 2016-08-26 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Publications (3)

Publication Number Publication Date
EP3358547A1 EP3358547A1 (fr) 2018-08-08
EP3358547A4 true EP3358547A4 (fr) 2019-05-08
EP3358547B1 EP3358547B1 (fr) 2024-07-24

Family

ID=58422943

Family Applications (1)

Application Number Title Priority Date Filing Date
EP16850568.3A Active EP3358547B1 (fr) 2015-09-29 2016-08-26 Dispositif de traitement d'informations, procédé de traitement d'informations et programme

Country Status (3)

Country Link
US (1) US10692379B2 (fr)
EP (1) EP3358547B1 (fr)
WO (1) WO2017056385A1 (fr)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190277947A1 (en) * 2018-03-12 2019-09-12 Panasonic Intellectual Property Management Co., Ltd. Tracking apparatus and tracking method
EP3912149A4 (fr) * 2019-01-17 2022-10-19 Jungo Connectivity Ltd. Procédé et système de surveillance d'une personne à l'aide de lumière infrarouge et de lumière visible
US11851060B2 (en) 2020-03-15 2023-12-26 Ron Zass Controlling vehicles in response to windows
JP7348874B2 (ja) * 2020-04-06 2023-09-21 トヨタ自動車株式会社 傾斜角検出装置及び制御装置
US11783710B2 (en) * 2020-06-24 2023-10-10 Humanising Autonomy Limited Appearance and movement based model for determining risk of micro mobility users
JP7380449B2 (ja) * 2020-06-30 2023-11-15 トヨタ自動車株式会社 判定装置及びプログラム

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060291694A1 (en) * 2005-06-24 2006-12-28 Objectvideo, Inc. Detection of change in posture in video
US20080097699A1 (en) * 2004-12-28 2008-04-24 Kabushiki Kaisha Toyota Chuo Kenkyusho Vehicle motion control device
US20130164722A1 (en) * 2010-08-31 2013-06-27 The University Of Tokyo Behavior analysis device
EP2759998A1 (fr) * 2011-09-20 2014-07-30 Toyota Jidosha Kabushiki Kaisha Dispositif de prédiction de l'action de piétons et procédé de prédiction de l'action de piétons

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3582570B2 (ja) 1998-02-10 2004-10-27 三菱自動車工業株式会社 路面状態判別装置
JP2005242914A (ja) * 2004-02-27 2005-09-08 Sumitomo Electric Ind Ltd 人間認識システム及び画像処理装置
WO2007088859A1 (fr) * 2006-02-01 2007-08-09 Pioneer Corporation Dispositif et procede de determination de l'etat de la surface d'une route, dispositif et procede de recherche de trajet, procede de parametrage de position, programme et support d'enregistrement
JP4173896B2 (ja) 2006-04-03 2008-10-29 本田技研工業株式会社 車両周辺監視装置
US7671725B2 (en) * 2006-03-24 2010-03-02 Honda Motor Co., Ltd. Vehicle surroundings monitoring apparatus, vehicle surroundings monitoring method, and vehicle surroundings monitoring program
JP2009122854A (ja) * 2007-11-13 2009-06-04 Aisin Aw Co Ltd 注意喚起システム及びプログラム
JP2010072839A (ja) * 2008-09-17 2010-04-02 Toyota Motor Corp 衝突予測装置
JP5648655B2 (ja) * 2012-04-27 2015-01-07 株式会社デンソー 対象物識別装置
JP5983243B2 (ja) * 2012-09-26 2016-08-31 株式会社デンソー 検出装置
WO2016092650A1 (fr) * 2014-12-10 2016-06-16 三菱電機株式会社 Dispositif de traitement d'images, système d'affichage d'intérieur de véhicule, dispositif d'affichage, procédé de traitement d'images, et programme de traitement d'images
JP6397934B2 (ja) * 2014-12-19 2018-09-26 株式会社日立製作所 走行制御装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080097699A1 (en) * 2004-12-28 2008-04-24 Kabushiki Kaisha Toyota Chuo Kenkyusho Vehicle motion control device
US20060291694A1 (en) * 2005-06-24 2006-12-28 Objectvideo, Inc. Detection of change in posture in video
US20130164722A1 (en) * 2010-08-31 2013-06-27 The University Of Tokyo Behavior analysis device
EP2759998A1 (fr) * 2011-09-20 2014-07-30 Toyota Jidosha Kabushiki Kaisha Dispositif de prédiction de l'action de piétons et procédé de prédiction de l'action de piétons

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
JOHANNES KOOIJMAN: "Bicycle Rider Control. Observations, modeling & Experiments.", 18 September 2012 (2012-09-18), Netherlands, XP055575426, ISBN: 978-90-40-72578-4, Retrieved from the Internet <URL:http://bicycle.tudelft.nl/stablebicycle/kooijman2012bicycle.pdf> [retrieved on 20190328] *
See also references of WO2017056385A1 *

Also Published As

Publication number Publication date
EP3358547A1 (fr) 2018-08-08
US20180268700A1 (en) 2018-09-20
WO2017056385A1 (fr) 2017-04-06
US10692379B2 (en) 2020-06-23
EP3358547B1 (fr) 2024-07-24

Similar Documents

Publication Publication Date Title
EP3358546A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3343339A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3340205A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3260990A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3264258A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme
EP3343535A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme
EP3267390A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3293955A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme de traitement d&#39;informations
EP3276622A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3255880A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3327661A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3309739A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3243554A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme
EP3258684A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3273712A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3282687A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme de traitement d&#39;informations
EP3309738A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3260951A4 (fr) Dispositif, procédé et programme de traitement d&#39;informations
EP3243557A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3279782A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme
EP3270346A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3244321A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme
EP3343948A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme
EP3296944A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations, et programme
EP3340188A4 (fr) Dispositif de traitement d&#39;informations, procédé de traitement d&#39;informations et programme

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20180319

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20190405

RIC1 Information provided on ipc code assigned before grant

Ipc: G06K 9/00 20060101AFI20190401BHEP

Ipc: G08G 1/16 20060101ALN20190401BHEP

Ipc: G06T 7/60 20170101ALN20190401BHEP

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20201203

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: SONY GROUP CORPORATION

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602016088578

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G08G0001160000

Ipc: G06V0020580000

Ref country code: DE

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G08G0001160000

Ipc: G06V0020580000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G08G 1/16 20060101ALN20240123BHEP

Ipc: G06T 7/60 20170101ALN20240123BHEP

Ipc: G06V 40/10 20220101ALI20240123BHEP

Ipc: G06V 20/58 20220101AFI20240123BHEP

RIC1 Information provided on ipc code assigned before grant

Ipc: G08G 1/16 20060101ALN20240206BHEP

Ipc: G06T 7/60 20170101ALN20240206BHEP

Ipc: G06V 40/10 20220101ALI20240206BHEP

Ipc: G06V 20/58 20220101AFI20240206BHEP

INTG Intention to grant announced

Effective date: 20240219

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20240529

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

Ref country code: DE

Ref legal event code: R096

Ref document number: 602016088578

Country of ref document: DE

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240819

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20240820

Year of fee payment: 9