EP3748629C0 - Identification method for voice keywords, computer-readable storage medium, and computer device - Google Patents

Identification method for voice keywords, computer-readable storage medium, and computer device

Info

Publication number
EP3748629C0
EP3748629C0 EP19747243.4A EP19747243A EP3748629C0 EP 3748629 C0 EP3748629 C0 EP 3748629C0 EP 19747243 A EP19747243 A EP 19747243A EP 3748629 C0 EP3748629 C0 EP 3748629C0
Authority
EP
European Patent Office
Prior art keywords
computer
storage medium
readable storage
identification method
computer device
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP19747243.4A
Other languages
German (de)
French (fr)
Other versions
EP3748629A4 (en
EP3748629B1 (en
EP3748629A1 (en
Inventor
Jun Wang
Dan Su
Dong Yu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Publication of EP3748629A1 publication Critical patent/EP3748629A1/en
Publication of EP3748629A4 publication Critical patent/EP3748629A4/en
Application granted granted Critical
Publication of EP3748629C0 publication Critical patent/EP3748629C0/en
Publication of EP3748629B1 publication Critical patent/EP3748629B1/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/083—Recognition networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/04—Segmentation; Word boundary detection
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/04—Segmentation; Word boundary detection
    • G10L15/05—Word boundary detection
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/16—Speech classification or search using artificial neural networks
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/28—Constructional details of speech recognition systems
    • G10L15/32—Multiple recognisers used in sequence or in parallel; Score combination systems therefor, e.g. voting systems
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/063—Training
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L2015/088—Word spotting
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Machine Translation (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Telephonic Communication Services (AREA)
EP19747243.4A 2018-01-31 2019-01-22 Identification method for voice keywords, computer-readable storage medium, and computer device Active EP3748629B1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810096472.XA CN108305617B (en) 2018-01-31 2018-01-31 Method and device for recognizing voice keywords
PCT/CN2019/072590 WO2019149108A1 (en) 2018-01-31 2019-01-22 Identification method and device for voice keywords, computer-readable storage medium, and computer device

Publications (4)

Publication Number Publication Date
EP3748629A1 EP3748629A1 (en) 2020-12-09
EP3748629A4 EP3748629A4 (en) 2021-10-27
EP3748629C0 true EP3748629C0 (en) 2023-09-06
EP3748629B1 EP3748629B1 (en) 2023-09-06

Family

ID=62850811

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19747243.4A Active EP3748629B1 (en) 2018-01-31 2019-01-22 Identification method for voice keywords, computer-readable storage medium, and computer device

Country Status (5)

Country Link
US (1) US11222623B2 (en)
EP (1) EP3748629B1 (en)
JP (1) JP7005099B2 (en)
CN (3) CN108305617B (en)
WO (1) WO2019149108A1 (en)

Families Citing this family (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108305617B (en) 2018-01-31 2020-09-08 腾讯科技（深圳）有限公司 Method and device for recognizing voice keywords
JP6911785B2 (en) * 2018-02-02 2021-07-28 日本電信電話株式会社 Judgment device, judgment method and judgment program
CN112513983A (en) 2018-06-21 2021-03-16 奇跃公司 Wearable system speech processing
CN110752973B (en) * 2018-07-24 2020-12-25 Tcl科技集团股份有限公司 Terminal equipment control method and device and terminal equipment
CN109065046A (en) * 2018-08-30 2018-12-21 出门问问信息科技有限公司 Method, apparatus, electronic equipment and the computer readable storage medium that voice wakes up
CN113748462A (en) 2019-03-01 2021-12-03 奇跃公司 Determining input for a speech processing engine
GB201904185D0 (en) 2019-03-26 2019-05-08 Sita Information Networking Computing Uk Ltd Item classification system, device and method therefor
US11043218B1 (en) * 2019-06-26 2021-06-22 Amazon Technologies, Inc. Wakeword and acoustic event detection
CN110335592B (en) * 2019-06-28 2022-06-03 腾讯科技（深圳）有限公司 Speech phoneme recognition method and device, storage medium and electronic device
CN110334244B (en) * 2019-07-11 2020-06-09 出门问问信息科技有限公司 Data processing method and device and electronic equipment
US11328740B2 (en) 2019-08-07 2022-05-10 Magic Leap, Inc. Voice onset detection
CN110364143B (en) * 2019-08-14 2022-01-28 腾讯科技（深圳）有限公司 Voice awakening method and device and intelligent electronic equipment
CN110570861B (en) * 2019-09-24 2022-02-25 Oppo广东移动通信有限公司 Method and device for voice wake-up, terminal equipment and readable storage medium
CN111081241B (en) * 2019-11-20 2023-04-07 Oppo广东移动通信有限公司 Data detection method and device for device false wake-up, mobile terminal and storage medium
CN110992929A (en) * 2019-11-26 2020-04-10 苏宁云计算有限公司 Voice keyword detection method, device and system based on neural network
CN111477223A (en) * 2020-03-04 2020-07-31 深圳市佳士科技股份有限公司 Welding machine control method and device, terminal equipment and computer readable storage medium
CN111445899B (en) * 2020-03-09 2023-08-01 咪咕文化科技有限公司 Speech emotion recognition method, device and storage medium
US11917384B2 (en) 2020-03-27 2024-02-27 Magic Leap, Inc. Method of waking a device using spoken voice commands
CN111508493B (en) * 2020-04-20 2022-11-15 Oppo广东移动通信有限公司 Voice wake-up method and device, electronic equipment and storage medium
CN111768764B (en) * 2020-06-23 2024-01-19 北京猎户星空科技有限公司 Voice data processing method and device, electronic equipment and medium
CN111833856B (en) * 2020-07-15 2023-10-24 厦门熙重电子科技有限公司 Voice key information calibration method based on deep learning
CN111798840B (en) * 2020-07-16 2023-08-08 中移在线服务有限公司 Voice keyword recognition method and device
CN111951807A (en) * 2020-08-21 2020-11-17 上海依图网络科技有限公司 Voice content detection method, apparatus, medium, and system thereof
CN112435691B (en) * 2020-10-12 2024-03-12 珠海亿智电子科技有限公司 Online voice endpoint detection post-processing method, device, equipment and storage medium
CN112634870B (en) * 2020-12-11 2023-05-30 平安科技（深圳）有限公司 Keyword detection method, device, equipment and storage medium
CN112883375A (en) * 2021-02-03 2021-06-01 深信服科技股份有限公司 Malicious file identification method, device, equipment and storage medium
CN114937450B (en) * 2021-02-05 2024-07-09 清华大学 Voice keyword recognition method and system
CN115148197A (en) * 2021-03-31 2022-10-04 华为技术有限公司 Voice wake-up method, device, storage medium and system
CN113192501B (en) * 2021-04-12 2022-04-22 青岛信芯微电子科技股份有限公司 Instruction word recognition method and device
CN113838467B (en) * 2021-08-02 2023-11-14 北京百度网讯科技有限公司 Voice processing method and device and electronic equipment
CN113724698B (en) * 2021-09-01 2024-01-30 马上消费金融股份有限公司 Training method, device, equipment and storage medium of voice recognition model
EP4156179A1 (en) * 2021-09-23 2023-03-29 Siemens Healthcare GmbH Voice control of a medical device
CN114141239A (en) * 2021-11-29 2022-03-04 江南大学 Voice short instruction identification method and system based on lightweight deep learning
CN116030792B (en) * 2023-03-30 2023-07-25 深圳市玮欧科技有限公司 Method, apparatus, electronic device and readable medium for converting voice tone

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8639507B2 (en) * 2007-12-25 2014-01-28 Nec Corporation Voice recognition system, voice recognition method, and program for voice recognition
KR20120072145A (en) * 2010-12-23 2012-07-03 한국전자통신연구원 Method and apparatus for recognizing speech
CN102831891B (en) * 2011-06-13 2014-11-05 富士通株式会社 Processing method and system for voice data
CN102915729B (en) * 2011-08-01 2014-11-26 佳能株式会社 Speech keyword spotting system and system and method of creating dictionary for the speech keyword spotting system
CN102982024B (en) * 2011-09-02 2016-03-23 北京百度网讯科技有限公司 A kind of search need recognition methods and device
CN103177721B (en) * 2011-12-26 2015-08-19 中国电信股份有限公司 Audio recognition method and system
US10304465B2 (en) * 2012-10-30 2019-05-28 Google Technology Holdings LLC Voice control user interface for low power mode
US9390708B1 (en) * 2013-05-28 2016-07-12 Amazon Technologies, Inc. Low latency and memory efficient keywork spotting
CN104143329B (en) 2013-08-19 2015-10-21 腾讯科技（深圳）有限公司 Carry out method and the device of voice keyword retrieval
CN103943107B (en) * 2014-04-03 2017-04-05 北京大学深圳研究生院 A kind of audio frequency and video keyword recognition method based on Decision-level fusion
US9484022B2 (en) * 2014-05-23 2016-11-01 Google Inc. Training multiple neural networks with different accuracy
US9508340B2 (en) * 2014-12-22 2016-11-29 Google Inc. User specified keyword spotting using long short term memory neural network feature extractor
JP6604013B2 (en) * 2015-03-23 2019-11-13 カシオ計算機株式会社 Speech recognition apparatus, speech recognition method and program
KR102386854B1 (en) * 2015-08-20 2022-04-13 삼성전자주식회사 Apparatus and method for speech recognition based on unified model
US20170061959A1 (en) * 2015-09-01 2017-03-02 Disney Enterprises, Inc. Systems and Methods For Detecting Keywords in Multi-Speaker Environments
CN105679310A (en) * 2015-11-17 2016-06-15 乐视致新电子科技（天津）有限公司 Method and system for speech recognition
CN106856092B (en) * 2015-12-09 2019-11-15 中国科学院声学研究所 Chinese speech keyword retrieval method based on feedforward neural network language model
CN105679316A (en) * 2015-12-29 2016-06-15 深圳微服机器人科技有限公司 Voice keyword identification method and apparatus based on deep neural network
CN106940998B (en) * 2015-12-31 2021-04-16 阿里巴巴集团控股有限公司 Execution method and device for setting operation
US10373612B2 (en) * 2016-03-21 2019-08-06 Amazon Technologies, Inc. Anchored speech detection and speech recognition
US9799325B1 (en) * 2016-04-14 2017-10-24 Xerox Corporation Methods and systems for identifying keywords in speech signal
CN106328147B (en) * 2016-08-31 2022-02-01 中国科学技术大学 Speech recognition method and device
US10311863B2 (en) * 2016-09-02 2019-06-04 Disney Enterprises, Inc. Classifying segments of speech based on acoustic features and context
CN106448663B (en) * 2016-10-17 2020-10-23 海信集团有限公司 Voice awakening method and voice interaction device
CN106547742B (en) * 2016-11-30 2019-05-03 百度在线网络技术（北京）有限公司 Semantic parsing result treating method and apparatus based on artificial intelligence
JP6968012B2 (en) 2017-03-30 2021-11-17 株式会社バルカー Laminates, their manufacturing methods, and gate seals
CN107221326B (en) * 2017-05-16 2021-05-28 百度在线网络技术（北京）有限公司 Voice awakening method and device based on artificial intelligence and computer equipment
CN107123417B (en) * 2017-05-16 2020-06-09 上海交通大学 Customized voice awakening optimization method and system based on discriminant training
CN110349572B (en) * 2017-05-27 2021-10-22 腾讯科技（深圳）有限公司 Voice keyword recognition method and device, terminal and server
CN107274888B (en) * 2017-06-14 2020-09-15 大连海事大学 Emotional voice recognition method based on octave signal strength and differentiated feature subset
CN107622770B (en) * 2017-09-30 2021-03-16 百度在线网络技术（北京）有限公司 Voice wake-up method and device
CN108305617B (en) * 2018-01-31 2020-09-08 腾讯科技（深圳）有限公司 Method and device for recognizing voice keywords

Also Published As

Publication number Publication date
WO2019149108A1 (en) 2019-08-08
EP3748629A4 (en) 2021-10-27
JP7005099B2 (en) 2022-01-21
CN110444195A (en) 2019-11-12
JP2021512362A (en) 2021-05-13
CN110444193A (en) 2019-11-12
CN108305617A (en) 2018-07-20
EP3748629B1 (en) 2023-09-06
US20200286465A1 (en) 2020-09-10
CN110444193B (en) 2021-12-14
CN110444195B (en) 2021-12-14
EP3748629A1 (en) 2020-12-09
US11222623B2 (en) 2022-01-11
CN108305617B (en) 2020-09-08

Similar Documents

Publication Publication Date Title
EP3748629C0 (en) Identification method for voice keywords, computer-readable storage medium, and computer device
SG11202001627XA (en) Speech recognition method, apparatus, and computer readable storage medium
EP3770905C0 (en) Speech recognition method, apparatus and device, and storage medium
SG11202100900QA (en) Text-based speech synthesis method and device, computer device, and non-transitory computer-readable storage medium
EP3757937A4 (en) Image recognition method, storage medium and computer device
EP3648099A4 (en) Voice recognition method, device, apparatus, and storage medium
EP3848730A4 (en) Positioning method, apparatus and device, and computer-readable storage medium
EP3758364A4 (en) Dynamic emoticon-generating method, computer-readable storage medium and computer device
SG11202101614VA (en) Association recommendation method and device, computer equipment and storage medium
EP3477495A4 (en) Apparatus and method for extracting user keyword, and computer-readable storage medium
EP3591930A4 (en) Information storage method, device, and computer-readable storage medium
EP3584786A4 (en) Voice recognition method, electronic device, and computer storage medium
EP3518122A4 (en) Keyword extraction method, computer device, and storage medium
EP3828885C0 (en) Voice denoising method and apparatus, computing device and computer readable storage medium
EP3605407A4 (en) Information processing device, information processing method, and computer-readable storage medium
EP3370188A4 (en) Facial verification method, device, and computer storage medium
EP3605400A4 (en) Information processing device, information processing method, and computer-readable storage medium
SG11202001873SA (en) Semantic recognition method, electronic device , and computer-readable storage medium
EP3796219A4 (en) Object recognition method and device, and storage medium
SG11202009691TA (en) Method and device for liveness detection, and storage medium
SG11202107392TA (en) Application starting method and apparatus, computer device and storage medium
EP3673399A4 (en) Method for recognizing fingerprint, and electronic device and storage medium therefor
SG11202101248SA (en) Information processing method and device based on blockchain, and computer-readable storage medium
EP3594940A4 (en) Training method for voice data set, computer device and computer readable storage medium
EP3570220A4 (en) Information processing method, information processing device, and computer-readable storage medium

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20200831

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)
A4 Supplementary search report drawn up and despatched

Effective date: 20210929

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 15/06 20130101ALN20210923BHEP

Ipc: G10L 15/26 20060101ALI20210923BHEP

Ipc: G10L 15/16 20060101ALI20210923BHEP

Ipc: G10L 15/04 20130101ALI20210923BHEP

Ipc: G10L 15/02 20060101ALI20210923BHEP

Ipc: G10L 15/08 20060101ALI20210923BHEP

Ipc: G10L 15/32 20130101AFI20210923BHEP

RAP3 Party data changed (applicant data changed or rights of an application transferred)

Owner name: TENCENT TECHNOLOGY (SHENZHEN) COMPANY LIMITED

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602019036822

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: G10L0015020000

Ipc: G10L0015320000

Ref legal event code: R079

Ipc: G10L0015320000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G10L 15/06 20130101ALN20230320BHEP

Ipc: G10L 15/26 20060101ALI20230320BHEP

Ipc: G10L 15/16 20060101ALI20230320BHEP

Ipc: G10L 15/04 20130101ALI20230320BHEP

Ipc: G10L 15/02 20060101ALI20230320BHEP

Ipc: G10L 15/08 20060101ALI20230320BHEP

Ipc: G10L 15/32 20130101AFI20230320BHEP

INTG Intention to grant announced

Effective date: 20230404

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602019036822

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

U01 Request for unitary effect filed

Effective date: 20230921

U07 Unitary effect registered

Designated state(s): AT BE BG DE DK EE FI FR IT LT LU LV MT NL PT SE SI

Effective date: 20231002

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231207

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231206

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20231207

U20 Renewal fee paid [unitary effect]

Year of fee payment: 6

Effective date: 20240119

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240106

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20240106

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20240124

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602019036822

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20240607

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230906

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20240131