EP3566435A4 - Procédé de traitement super-résolution pour image animée et appareil de traitement d'image associé - Google Patents

Procédé de traitement super-résolution pour image animée et appareil de traitement d'image associé Download PDF

Info

Publication number
EP3566435A4
EP3566435A4 EP18801442.7A EP18801442A EP3566435A4 EP 3566435 A4 EP3566435 A4 EP 3566435A4 EP 18801442 A EP18801442 A EP 18801442A EP 3566435 A4 EP3566435 A4 EP 3566435A4
Authority
EP
European Patent Office
Prior art keywords
super
image
apparatus therefor
processing apparatus
processing method
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP18801442.7A
Other languages
German (de)
English (en)
Other versions
EP3566435A1 (fr
EP3566435B1 (fr
Inventor
Parichay KAPOOR
Ji-Hun Oh
Kwang-hoon SON
Young-Chul Cho
Yong-sup PARK
Hyun-Jung Kim
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Electronics Co Ltd
Original Assignee
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co Ltd filed Critical Samsung Electronics Co Ltd
Priority claimed from PCT/KR2018/005633 external-priority patent/WO2018212599A1/fr
Publication of EP3566435A1 publication Critical patent/EP3566435A1/fr
Publication of EP3566435A4 publication Critical patent/EP3566435A4/fr
Application granted granted Critical
Publication of EP3566435B1 publication Critical patent/EP3566435B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/59—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial sub-sampling or interpolation, e.g. alteration of picture size or resolution
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440263—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the spatial resolution, e.g. for displaying on a connected PDA
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/10—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from different wavelengths
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/95—Computational photography systems, e.g. light-field imaging systems
    • H04N23/951—Computational photography systems, e.g. light-field imaging systems by using two or more images to influence resolution, frame rate or aspect ratio
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Computer Graphics (AREA)
  • Image Analysis (AREA)
EP18801442.7A 2017-05-17 2018-05-16 Procédé de traitement super-résolution pour image animée et appareil de traitement d'image associé Active EP3566435B1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
KR20170061195 2017-05-17
KR1020180012941A KR102511059B1 (ko) 2017-05-17 2018-02-01 동영상의 초해상 처리 방법 및 이를 위한 영상 처리 장치
PCT/KR2018/005633 WO2018212599A1 (fr) 2017-05-17 2018-05-16 Procédé de traitement super-résolution pour image animée et appareil de traitement d'image associé

Publications (3)

Publication Number Publication Date
EP3566435A1 EP3566435A1 (fr) 2019-11-13
EP3566435A4 true EP3566435A4 (fr) 2019-11-13
EP3566435B1 EP3566435B1 (fr) 2022-10-26

Family

ID=64603497

Family Applications (1)

Application Number Title Priority Date Filing Date
EP18801442.7A Active EP3566435B1 (fr) 2017-05-17 2018-05-16 Procédé de traitement super-résolution pour image animée et appareil de traitement d'image associé

Country Status (3)

Country Link
EP (1) EP3566435B1 (fr)
KR (1) KR102511059B1 (fr)
CN (1) CN110622502B (fr)

Families Citing this family (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102525578B1 (ko) 2018-10-19 2023-04-26 삼성전자주식회사 부호화 방법 및 그 장치, 복호화 방법 및 그 장치
KR102222871B1 (ko) 2019-02-22 2021-03-04 삼성전자주식회사 디스플레이 장치 및 이의 영상 표시 방법
KR20200114436A (ko) 2019-03-28 2020-10-07 국방과학연구소 스케일러블 영상 부호화를 수행하는 장치 및 방법
KR20190117416A (ko) 2019-09-26 2019-10-16 엘지전자 주식회사 동영상 프레임 해상도를 향상시키기 위한 방법 및 장치
KR20190119550A (ko) 2019-10-02 2019-10-22 엘지전자 주식회사 영상의 해상도를 향상시키기 위한 방법 및 장치
KR102095443B1 (ko) * 2019-10-17 2020-05-26 엘아이지넥스원 주식회사 구조텐서를 이용한 딥러닝 기반의 영상 개선 방법 및 장치
KR102436512B1 (ko) * 2019-10-29 2022-08-25 삼성전자주식회사 부호화 방법 및 그 장치, 복호화 방법 및 그 장치
KR102157610B1 (ko) * 2019-10-29 2020-09-18 세종대학교산학협력단 초고해상도 디지털 이미지 생성을 통한 자동화 구조물 손상 검출 시스템 및 그 방법
CN110827200B (zh) * 2019-11-04 2023-04-07 Oppo广东移动通信有限公司 一种图像超分重建方法、图像超分重建装置及移动终端
KR20210056733A (ko) * 2019-11-11 2021-05-20 삼성전자주식회사 스트리밍 데이터를 제공하는 전자 장치 및 그 작동 방법
KR102214502B1 (ko) * 2019-12-13 2021-02-09 한국과학기술원 이미지 합동 처리 방법 및 장치
CN111127325B (zh) * 2019-12-31 2020-11-24 珠海大横琴科技发展有限公司 基于循环神经网络的卫星视频超分辨率重建方法及系统
KR102198480B1 (ko) * 2020-02-28 2021-01-05 연세대학교 산학협력단 재귀 그래프 모델링을 통한 비디오 요약 생성 장치 및 방법
CN111246250B (zh) * 2020-03-13 2022-07-01 广州市百果园信息技术有限公司 视频处理方法、装置、设备及存储介质
JP7293157B2 (ja) * 2020-03-17 2023-06-19 株式会社東芝 画像処理装置
KR20230036675A (ko) 2021-09-08 2023-03-15 정수아 Cnn 영상처리기법을 이용한 의류 원단의 불량검출 방법
KR102565421B1 (ko) 2021-11-03 2023-08-11 김지현 딥러닝 기반 알고리즘을 통한 영상의 초해상화 처리 장치 및 방법
KR20230089882A (ko) 2021-12-14 2023-06-21 인하대학교 산학협력단 휴대폰 동작 변화 기반 동영상 해상도 자동 조절 시스템
KR102598334B1 (ko) 2021-12-24 2023-11-02 한양대학교 산학협력단 윤곽선 정보를 이용한 딥러닝 기반 초해상도 동영상 생성 시스템 및 방법
KR20230160096A (ko) * 2022-05-16 2023-11-23 주식회사 실리콘아츠 에지 선명도의 조정이 가능한 초해상화 이미지 생성 장치 및 방법

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016132152A1 (fr) * 2015-02-19 2016-08-25 Magic Pony Technology Limited Interpolation de données visuelles

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120113119A1 (en) * 2010-11-05 2012-05-10 Nova Research, Inc. Super resolution infrared imaging system
CN105072373B (zh) * 2015-08-28 2018-03-27 中国科学院自动化研究所 基于双向循环卷积网络的视频超分辨率方法和系统

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016132152A1 (fr) * 2015-02-19 2016-08-25 Magic Pony Technology Limited Interpolation de données visuelles

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
CHAO DONG ET AL: "Image Super-Resolution Using Deep Convolutional Networks", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 38, no. 2, 1 February 2016 (2016-02-01), USA, pages 295 - 307, XP055559563, ISSN: 0162-8828, DOI: 10.1109/TPAMI.2015.2439281 *
JUN GUO ET AL: "Building an End-to-End Spatial-Temporal Convolutional Network for Video Super-Resolution", PROCEEDINGS OF THE THIRTY-FIRST AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, 4 February 2017 (2017-02-04), pages 4053 - 4060, XP055618668, Retrieved from the Internet <URL:https://www.aaai.org/ocs/index.php/AAAI/AAAI17/paper/viewFile/14733/14280> [retrieved on 20190905] *
LORENZO BARALDI ET AL: "Hierarchical Boundary-Aware Neural Encoder for Video Captioning", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 28 November 2016 (2016-11-28), XP081399245, DOI: 10.1109/CVPR.2017.339 *
MANDERS JEROEN: "Radboud University Nijmegen Optimizing Convolutional Neural Networks for Fast Training on a Small Dataset", 11 May 2016 (2016-05-11), XP055618108, Retrieved from the Internet <URL:https://theses.ubn.ru.nl/bitstream/handle/123456789/1880/Manders,_T.M.C._1.pdf?sequence=1> [retrieved on 20190903] *
MARKOU M ET AL: "Novelty detection: a review-part 2: - neural network based approaches", SIGNAL PROCESSING, ELSEVIER SCIENCE PUBLISHERS B.V. AMSTERDAM, NL, vol. 83, no. 12, 1 December 2003 (2003-12-01), pages 2499 - 2521, XP004467983, ISSN: 0165-1684, DOI: 10.1016/J.SIGPRO.2003.07.019 *
See also references of WO2018212599A1 *

Also Published As

Publication number Publication date
KR20180126362A (ko) 2018-11-27
CN110622502A (zh) 2019-12-27
CN110622502B (zh) 2021-07-13
EP3566435A1 (fr) 2019-11-13
KR102511059B1 (ko) 2023-03-17
EP3566435B1 (fr) 2022-10-26

Similar Documents

Publication Publication Date Title
EP3566435A4 (fr) Procédé de traitement super-résolution pour image animée et appareil de traitement d&#39;image associé
EP3599760A4 (fr) Procédé et appareil de traitement d&#39;image
EP3734552A4 (fr) Procédé de traitement d&#39;image et dispositif associé
EP3446282A4 (fr) Dispositif et procédé de traitement d&#39;image
EP3627499A4 (fr) Appareil et procédé de traitement d&#39;images
EP3416372A4 (fr) Procédé et appareil de traitement d&#39;image à 360 degrés
EP3844970A4 (fr) Appareil de traitement d&#39;image, et procédé de traitement d&#39;image associé
EP3686845A4 (fr) Procédé, dispositif et appareil de traitement d&#39;image
EP3565249A4 (fr) Dispositif et procédé de traitement d&#39;image
EP3440829A4 (fr) Appareil et procédé de traitement d&#39;image
EP3704668A4 (fr) Procédé et appareil de traitement d&#39;image
EP3522538A4 (fr) Procédé de traitement d&#39;image et appareil associé
EP3525164A4 (fr) Appareil de traitement d&#39;image et procédé de traitement d&#39;image
EP3827412A4 (fr) Procédé et appareil de traitement d&#39;image
EP3584760A4 (fr) Procédé et appareil de traitement d&#39;image
EP3676800A4 (fr) Procédé et appareil de traitement d&#39;image
EP3493156A4 (fr) Appareil de traitement d&#39;images et procédé de traitement d&#39;images
EP3780624A4 (fr) Traitement d&#39;images et dispositif pour sa mise en oeuvre
EP3767949A4 (fr) Dispositif et procédé de traitement d&#39;image
EP3886037A4 (fr) Appareil et procédé de traitement d&#39;image pour transformation de style
EP3857506A4 (fr) Appareil de traitement d&#39;image et procédé de traitement d&#39;image associé
EP3790272A4 (fr) Appareil de traitement d&#39;image et son procédé de fonctionnement
EP3827582A4 (fr) Appareil de traitement d&#39;images et procédé d&#39;extraction de caractéristiques
EP3833029A4 (fr) Dispositif et procédé de traitement d&#39;image
EP3520072A4 (fr) Appareil électronique et son procédé de traitement d&#39;image

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190809

A4 Supplementary search report drawn up and despatched

Effective date: 20190913

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20200610

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20220603

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602018042325

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1527933

Country of ref document: AT

Kind code of ref document: T

Effective date: 20221115

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20221026

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1527933

Country of ref document: AT

Kind code of ref document: T

Effective date: 20221026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230227

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230126

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230226

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230127

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602018042325

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20230420

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20230727

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20230420

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20230531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221026

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230516

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230531

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230531

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230516

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20230516