EP4200674C0 - Erkennung kritischer komponenten unter verwendung von tiefenlernen und aufmerksamkeit - Google Patents

Erkennung kritischer komponenten unter verwendung von tiefenlernen und aufmerksamkeit

Info

Publication number
EP4200674C0
EP4200674C0 EP21873333.5A EP21873333A EP4200674C0 EP 4200674 C0 EP4200674 C0 EP 4200674C0 EP 21873333 A EP21873333 A EP 21873333A EP 4200674 C0 EP4200674 C0 EP 4200674C0
Authority
EP
European Patent Office
Prior art keywords
attention
deep learning
component detection
critical component
critical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP21873333.5A
Other languages
English (en)
French (fr)
Other versions
EP4200674A1 (de
EP4200674B1 (de
EP4200674A4 (de
Inventor
Julianna Ianni
Rajath Elias Soans
Kameswari Devi Ayyagari
Saul Kohn
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Proscia Inc
Original Assignee
Proscia Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Proscia Inc filed Critical Proscia Inc
Publication of EP4200674A1 publication Critical patent/EP4200674A1/de
Publication of EP4200674A4 publication Critical patent/EP4200674A4/de
Application granted granted Critical
Publication of EP4200674C0 publication Critical patent/EP4200674C0/de
Publication of EP4200674B1 publication Critical patent/EP4200674B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0895—Weakly supervised learning, e.g. semi-supervised or self-supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/774—Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10088—Magnetic resonance imaging [MRI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20021—Dividing image into blocks, subimages or windows
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30088—Skin; Dermal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30096—Tumor; Lesion

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Medical Informatics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Databases & Information Systems (AREA)
  • Quality & Reliability (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Image Analysis (AREA)
  • Confectionery (AREA)
  • Solid-Sorbent Or Filter-Aiding Compositions (AREA)
  • Image Processing (AREA)
EP21873333.5A 2020-09-23 2021-09-22 Erkennung kritischer komponenten unter verwendung von tiefenlernen und aufmerksamkeit Active EP4200674B1 (de)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202063082125P 2020-09-23 2020-09-23
PCT/US2021/051506 WO2022066736A1 (en) 2020-09-23 2021-09-22 Critical component detection using deep learning and attention

Publications (4)

Publication Number Publication Date
EP4200674A1 EP4200674A1 (de) 2023-06-28
EP4200674A4 EP4200674A4 (de) 2024-03-20
EP4200674C0 true EP4200674C0 (de) 2025-06-18
EP4200674B1 EP4200674B1 (de) 2025-06-18

Family

ID=80846858

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21873333.5A Active EP4200674B1 (de) 2020-09-23 2021-09-22 Erkennung kritischer komponenten unter verwendung von tiefenlernen und aufmerksamkeit

Country Status (7)

Country Link
US (1) US11861881B2 (de)
EP (1) EP4200674B1 (de)
AU (1) AU2021349226C1 (de)
CA (1) CA3196713C (de)
ES (1) ES3040224T3 (de)
IL (1) IL301593B2 (de)
WO (1) WO2022066736A1 (de)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11636924B2 (en) 2016-09-30 2023-04-25 Molecular Devices, Llc Computer device for detecting an optimal candidate compound and methods thereof
US11462032B2 (en) 2019-09-23 2022-10-04 Proscia Inc. Stain normalization for automated whole-slide image classification
US11663838B2 (en) * 2020-10-29 2023-05-30 PAIGE.AI, Inc. Systems and methods for processing images to determine image-based computational biomarkers from liquid specimens
US20240005701A1 (en) * 2021-09-23 2024-01-04 Intel Corporation Methods and apparatus for team classification in sports analysis
CN114782796B (zh) * 2022-06-17 2023-05-02 武汉北大高科软件股份有限公司 一种物品图像防伪的智能验证方法和装置
CN115272419B (zh) * 2022-09-27 2022-12-09 南昌工程学院 基于混合卷积与自注意力的聚合网络目标跟踪方法与系统
CN116012394B (zh) * 2023-01-13 2026-03-06 杭州可帮医学检验实验室有限公司 一种病理切片图像去笔迹的方法、设备及存储介质
CN119068180A (zh) * 2023-05-26 2024-12-03 香港理工大学 自动图像分割系统和方法
CN117216010B (zh) * 2023-08-17 2025-10-10 山东省计算中心（国家超级计算济南中心） 融合元数据和标签相关性的政策文件分类方法及系统
CN119558358A (zh) * 2024-11-12 2025-03-04 电子科技大学(深圳)高等研究院 基于注意力的多任务可解释图神经网络的阿兹海默病分析方法

Family Cites Families (35)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9965705B2 (en) * 2015-11-03 2018-05-08 Baidu Usa Llc Systems and methods for attention-based configurable convolutional neural networks (ABC-CNN) for visual question answering
US10970887B2 (en) 2016-06-24 2021-04-06 Rensselaer Polytechnic Institute Tomographic image reconstruction via machine learning
WO2018152248A1 (en) 2017-02-14 2018-08-23 Dignity Health Systems, methods, and media for selectively presenting images captured by confocal laser endomicroscopy
CN110651301A (zh) 2017-05-24 2020-01-03 黑拉有限责任两合公司 用于自动为夜视图像着色的方法和系统
CN108230294B (zh) * 2017-06-14 2020-09-29 北京市商汤科技开发有限公司 图像检测方法、装置、电子设备和存储介质
CA3081643A1 (en) 2017-11-06 2019-05-09 University Health Network Platform, device and process for annotation and classification of tissue specimens using convolutional neural network
EP3709877A4 (de) * 2017-11-17 2021-08-25 Mayo Foundation for Medical Education and Research Verfahren zur verwendung von maschinenlern- und mechanistischen modellen zur biologischen merkmalsabbildung mit multiparametrischer mrt
WO2019108888A1 (en) 2017-11-30 2019-06-06 The Research Foundation For The State University Of New York SYSTEM AND METHOD TO QUANTIFY TUMOR-INFILTRATING LYMPHOCYTES (TILs) FOR CLINICAL PATHOLOGY ANALYSIS
US10460150B2 (en) 2018-03-16 2019-10-29 Proscia Inc. Deep learning automated dermatopathology
CN120014640A (zh) 2018-03-30 2025-05-16 加利福尼亚大学董事会 生成数字染色显微图像的方法
US10991150B2 (en) 2018-05-09 2021-04-27 Massachusetts Institute Of Technology View generation from a single image using fully convolutional neural networks
US11348661B2 (en) * 2018-05-14 2022-05-31 Tempus Labs, Inc. Predicting total nucleic acid yield and dissection boundaries for histology slides
WO2020014477A1 (en) 2018-07-11 2020-01-16 The University Of North Carolina At Chapel Hill Methods, systems, and computer readable media for image analysis with deep learning to predict breast cancer classes
US10748281B2 (en) * 2018-07-21 2020-08-18 International Business Machines Corporation Negative sample enhanced object detection machine
US10304193B1 (en) 2018-08-17 2019-05-28 12 Sigma Technologies Image segmentation and object detection using fully convolutional neural network
US11508168B2 (en) 2018-10-15 2022-11-22 Upmc Systems and methods for specimen interpretation
EP3909016A1 (de) 2019-01-13 2021-11-17 Lightlab Imaging, Inc. Systeme und verfahren zur klassifizierung von arterienbildbereichen und merkmalen davon
US10453197B1 (en) * 2019-02-18 2019-10-22 Inception Institute of Artificial Intelligence, Ltd. Object counting and instance segmentation using neural network architectures with image-level supervision
WO2020182710A1 (en) 2019-03-12 2020-09-17 F. Hoffmann-La Roche Ag Multiple instance learner for prognostic tissue pattern identification
EP3938951A1 (de) 2019-03-15 2022-01-19 Spintellx, Inc. Erklärbare ai (xai)-plattform für rechnerische pathologie
US12026875B2 (en) 2019-03-28 2024-07-02 Hoffmann-La Roche, Inc. Machine learning using distance-based similarity labels
WO2020232363A1 (en) 2019-05-16 2020-11-19 PAIGE.AI, Inc. Systems and methods for processing images to classify the processed images for digital pathology
EP3739504A1 (de) * 2019-05-16 2020-11-18 Basf Se System und verfahren zur unterstützung der erkennung von pflanzenkrankheiten
WO2020243193A1 (en) 2019-05-28 2020-12-03 PAIGE.AI, Inc. Systems and methods for processing images to prepare slides for processed images for digital pathology
US12094105B2 (en) 2019-06-10 2024-09-17 Datma, Inc. System and method for automatic labeling of pathology images
CN110309832A (zh) * 2019-06-19 2019-10-08 创新奇智(成都)科技有限公司 一种基于图像的物体分类方法、系统及电子设备
US20210027098A1 (en) 2019-07-22 2021-01-28 Shenzhen Malong Technologies Co., Ltd. Weakly Supervised Image Segmentation Via Curriculum Learning
US11462032B2 (en) 2019-09-23 2022-10-04 Proscia Inc. Stain normalization for automated whole-slide image classification
CN111026915B (zh) * 2019-11-25 2023-09-15 Oppo广东移动通信有限公司 视频分类方法、视频分类装置、存储介质与电子设备
CN111126488B (zh) * 2019-12-24 2023-08-18 威创集团股份有限公司 一种基于双重注意力的图像识别方法
CA3162251A1 (en) 2020-01-06 2021-07-15 Jillian SUE Systems and methods for analyzing electronic images for quality control
US12373980B2 (en) * 2020-01-31 2025-07-29 Servicenow, Inc. Method and system for localization with supervision
US11308619B2 (en) * 2020-07-17 2022-04-19 International Business Machines Corporation Evaluating a mammogram using a plurality of prior mammograms and deep learning algorithms
AU2021345220A1 (en) 2020-09-18 2023-04-27 Proscia Inc. Training end-to-end weakly supervised networks at the specimen (supra-image) level
WO2022066725A1 (en) 2020-09-23 2022-03-31 Proscia Inc. Training end-to-end weakly supervised networks in a multi-task fashion at the specimen (supra-image) level

Also Published As

Publication number Publication date
US20230245431A1 (en) 2023-08-03
AU2021349226B2 (en) 2023-05-11
CA3196713A1 (en) 2022-03-31
IL301593B1 (en) 2024-03-01
IL301593B2 (en) 2024-07-01
EP4200674A1 (de) 2023-06-28
ES3040224T3 (en) 2025-10-29
US11861881B2 (en) 2024-01-02
WO2022066736A1 (en) 2022-03-31
IL301593A (en) 2023-05-01
EP4200674B1 (de) 2025-06-18
AU2021349226A9 (en) 2024-05-02
CA3196713C (en) 2023-11-14
AU2021349226A1 (en) 2023-05-04
EP4200674A4 (de) 2024-03-20
AU2021349226C1 (en) 2023-08-24

Similar Documents

Publication Publication Date Title
EP4200674C0 (de) Erkennung kritischer komponenten unter verwendung von tiefenlernen und aufmerksamkeit
EP3712812C0 (de) Erkennen von maschinengeschriebenen und handschriftlichen zeichen unter verwendung von end-to-end-tiefenlernen
EP4412883A4 (de) Kamerakalibrierung zur fahrspurerkennung und entfernungsschätzung unter verwendung von einzelansichtsgeometrie und tiefenlernen
EP3803341A4 (de) Automatisierte detektion und charakterisierung von mikroobjekten in mikrofluidischen vorrichtungen
EP3753953A4 (de) Cd47-einzeldomänen-antikörper und verwendung davon
EP3763088C0 (de) Erkennung und wiederherstellung von strahlfehlern
EP3948476A4 (de) Autonomes erfassen und platzieren von unbekannten objekten
EP3814860C0 (de) Maschinenlernanalyse von rohrleitungen und instrumentdiagrammen
EP4133422A4 (de) Patientenüberwachung unter verwendung von edge-servern mit tiefenlernbeschleuniger und direktzugriffsspeicher
EP3616062A4 (de) Anomalie- und kausalitätsdetektion in computing-umgebungen unter verwendung von kontrafaktischer verarbeitung
IL271091B (en) Deep learning-based techniques for pre-training deep convolutional neural networks
EP4274229C0 (de) Interprädiktionsverfahren und -vorrichtung auf der basis von dmvr und bdof
EP3245614C0 (de) Objekterfassung unter verwendung von standortdaten und skalenraumdarstellungen von bilddaten
EP3731878C0 (de) Dekontaminationsvorrichtung und verfahren unter verwendung von ultraschallkavitation
EP3617946A4 (de) Kontexterfassungsverfahren und vorrichtung auf der basis von sprachinteraktion
EP3555284A4 (de) Zusammensetzungen und verfahren unter verwendung von cas9 mit verbesserter abstandserfassungsfunktion
EP3594854C0 (de) Verfahren und system zum clustern von benutzern mit kognitiver stressmeldung zur klassifizierung von stressniveaus
EP3602065A4 (de) Spektroskopische verfahren zur detektion und charakterisierung von mikroorganismen
EP3635114C0 (de) Herstellung und verwendung von guide-nukleinsäuren
EP4202483A4 (de) Detektionsverfahren mit lidar und lidar
EP3554515A4 (de) Antikörper und verfahren zur abreicherung von regulatorischen b10-zellen und verwendung in kombination mit immuncheckpoint-inhibitoren
EP4122712C0 (de) Detektion von bedruckbaren gegenständen
EP3500333C0 (de) Verwaltung von satzerkennung und authentifizierung unter verwendung von nockenprofilen
EP3579844A4 (de) Biologische gerüste, produkte mit biologischen gerüsten und verfahren zur verwendung davon
EP3486654A4 (de) Detektionsverfahren mit verwendung von immunchromatographie

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20230322

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
REG Reference to a national code

Ref legal event code: R079

Free format text: PREVIOUS MAIN CLASS: G05B0013020000

Ipc: G06T0007000000

Ref country code: DE

Ref legal event code: R079

Ref document number: 602021032603

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G05B0013020000

Ipc: G06T0007000000

A4 Supplementary search report drawn up and despatched

Effective date: 20240215

RIC1 Information provided on ipc code assigned before grant

Ipc: G06N 3/084 20230101ALI20240209BHEP

Ipc: G06N 3/045 20230101ALI20240209BHEP

Ipc: G06T 7/60 20170101ALI20240209BHEP

Ipc: G06T 5/50 20060101ALI20240209BHEP

Ipc: G06T 1/00 20060101ALI20240209BHEP

Ipc: G05B 19/02 20060101ALI20240209BHEP

Ipc: G05B 13/04 20060101ALI20240209BHEP

Ipc: G05B 13/02 20060101ALI20240209BHEP

Ipc: G06T 7/00 20170101AFI20240209BHEP

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20250117

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602021032603

Country of ref document: DE

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

U01 Request for unitary effect filed

Effective date: 20250714

U07 Unitary effect registered

Designated state(s): AT BE BG DE DK EE FI FR IT LT LU LV MT NL PT RO SE SI

Effective date: 20250728

REG Reference to a national code

Ref country code: CH

Ref legal event code: U11

Free format text: ST27 STATUS EVENT CODE: U-0-0-U10-U11 (AS PROVIDED BY THE NATIONAL OFFICE)

Effective date: 20251001

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250918

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250919

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20250929

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250918

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IE

Payment date: 20250929

Year of fee payment: 5

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 3040224

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20251029

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20251018

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250618

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: CH

Payment date: 20251001

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250618

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20250618

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20251001

Year of fee payment: 5