KR102887749B1 - 차량 오디오 시스템을 위한 공간 음성 분리 기능을 갖는 보이스 더킹 - Google Patents

차량 오디오 시스템을 위한 공간 음성 분리 기능을 갖는 보이스 더킹

Info

Publication number
KR102887749B1
KR102887749B1 KR1020200188246A KR20200188246A KR102887749B1 KR 102887749 B1 KR102887749 B1 KR 102887749B1 KR 1020200188246 A KR1020200188246 A KR 1020200188246A KR 20200188246 A KR20200188246 A KR 20200188246A KR 102887749 B1 KR102887749 B1 KR 102887749B1
Authority
KR
South Korea
Prior art keywords
vehicle
audio signals
incoming audio
human
microphone array
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
KR1020200188246A
Other languages
English (en)
Korean (ko)
Other versions
KR20210086556A (ko
Inventor
크리스토퍼 마이클 트리스탠
라일리 윈턴
크리스토퍼 루드비히
Original Assignee
하만인터내셔날인더스트리스인코포레이티드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 하만인터내셔날인더스트리스인코포레이티드 filed Critical 하만인터내셔날인더스트리스인코포레이티드
Publication of KR20210086556A publication Critical patent/KR20210086556A/ko
Application granted granted Critical
Publication of KR102887749B1 publication Critical patent/KR102887749B1/ko
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers
    • H04R3/005—Circuits for transducers for combining the signals of two or more microphones
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/02—Arrangements for holding or mounting articles, not otherwise provided for for radio sets, television sets, telephones, or the like; Arrangement of controls thereof
    • B60R11/0247—Arrangements for holding or mounting articles, not otherwise provided for for radio sets, television sets, telephones, or the like; Arrangement of controls thereof for microphones or earphones
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G10L25/84—Detection of presence or absence of voice signals for discriminating voice from noise
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R1/00—Details of transducers, loudspeakers or microphones
    • H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
    • H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
    • H04R1/40—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers
    • H04R1/406—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only by combining a number of identical transducers microphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers
    • H04R3/12—Circuits for transducers for distributing signals to two or more loudspeakers
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/225—Feedback of the input speech
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics
    • G10L2015/228—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics of application context
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
    • G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
    • G10L21/0208—Noise filtering
    • G10L21/0216—Noise filtering characterised by the method used for estimating noise
    • G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
    • G10L2021/02166—Microphone arrays; Beamforming
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G10L2025/783—Detection of presence or absence of voice signals based on threshold decision
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
    • G10L25/78—Detection of presence or absence of voice signals
    • G10L25/81—Detection of presence or absence of voice signals for discriminating voice from music
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
    • H04R2201/401—2D or 3D arrays of transducers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2201/00—Details of transducers, loudspeakers or microphones covered by H04R1/00 but not provided for in any of its subgroups
    • H04R2201/40—Details of arrangements for obtaining desired directional characteristic by combining a number of identical transducers covered by H04R1/40 but not provided for in any of its subgroups
    • H04R2201/405—Non-uniform arrays of transducers or a plurality of uniform arrays with different transducer spacing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2420/00—Details of connection covered by H04R, not provided for in its groups
    • H04R2420/01—Input selection or mixing for amplifiers or loudspeakers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2430/00—Signal processing covered by H04R, not provided for in its groups
    • H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R2499/00—Aspects covered by H04R or H04S not otherwise provided for in their subgroups
    • H04R2499/10—General applications
    • H04R2499/13—Acoustic transducers and sound field adaptation in vehicles
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers
    • H04R3/02—Circuits for transducers for preventing acoustic reaction, i.e. acoustic oscillatory feedback
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R3/00—Circuits for transducers
    • H04R3/04—Circuits for transducers for correcting frequency response
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; ELECTRIC HEARING AIDS; PUBLIC ADDRESS SYSTEMS
    • H04R5/00—Stereophonic arrangements
    • H04R5/02—Spatial or constructional arrangements of loudspeakers

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Otolaryngology (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Mechanical Engineering (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
  • Soundproofing, Sound Blocking, And Sound Damping (AREA)
KR1020200188246A 2019-12-30 2020-12-30 차량 오디오 시스템을 위한 공간 음성 분리 기능을 갖는 보이스 더킹 Active KR102887749B1 (ko)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201962954991P 2019-12-30 2019-12-30
US62/954,991 2019-12-30
US17/130,175 2020-12-22
US17/130,175 US11729549B2 (en) 2019-12-30 2020-12-22 Voice ducking with spatial speech separation for vehicle audio system

Publications (2)

Publication Number Publication Date
KR20210086556A KR20210086556A (ko) 2021-07-08
KR102887749B1 true KR102887749B1 (ko) 2025-11-17

Family

ID=73857051

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020200188246A Active KR102887749B1 (ko) 2019-12-30 2020-12-30 차량 오디오 시스템을 위한 공간 음성 분리 기능을 갖는 보이스 더킹

Country Status (5)

Country Link
US (1) US11729549B2 (https=)
EP (1) EP3846495A1 (https=)
JP (1) JP7667657B2 (https=)
KR (1) KR102887749B1 (https=)
CN (1) CN113129929A (https=)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11170752B1 (en) * 2020-04-29 2021-11-09 Gulfstream Aerospace Corporation Phased array speaker and microphone system for cockpit communication
WO2022065891A1 (ko) * 2020-09-25 2022-03-31 주식회사 아모센스 음성 처리 장치 및 이의 작동 방법
US11290814B1 (en) * 2020-12-15 2022-03-29 Valeo North America, Inc. Method, apparatus, and computer-readable storage medium for modulating an audio output of a microphone array
EP4156181A1 (en) 2021-09-24 2023-03-29 Elektrobit Automotive GmbH Controlling playback of audio data
EP4195201A1 (en) * 2021-12-07 2023-06-14 Nokia Technologies Oy Bone conduction confirmation
EP4202922A1 (en) * 2021-12-23 2023-06-28 GN Audio A/S Audio device and method for speaker extraction
US12095828B2 (en) 2021-12-30 2024-09-17 Harman International Industries, Incorporated In-vehicle communications and media mixing
CN114430498B (zh) * 2021-12-31 2023-07-18 惠州华阳通用智慧车载系统开发有限公司 一种车机双区音频系统及控制方法
CN114422932A (zh) * 2021-12-31 2022-04-29 广州小鹏汽车科技有限公司 声场调节方法、系统、装置、车辆及存储介质
US12598424B2 (en) * 2022-10-06 2026-04-07 Panasonic Automotive Systems America, LLC. Zoned audio duck for in car conversation
JP2024127573A (ja) * 2023-03-09 2024-09-20 本田技研工業株式会社 制御装置、コンテンツ視聴制御システム、移動体、制御方法及びプログラム
US20250280231A1 (en) * 2024-03-01 2025-09-04 Valeo Telematik Und Akustik Gmbh Systems and methods for microphone sensor systems for a vehicle exterior

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180020313A1 (en) 2016-03-03 2018-01-18 Google Inc. Systems and Methods for Spatial Audio Adjustment
US20180308504A1 (en) 2017-02-13 2018-10-25 Bose Corporation Audio systems and method for perturbing signal compensation
WO2019130282A1 (en) 2017-12-29 2019-07-04 Harman International Industries, Incorporated Acoustical in-cabin noise cancellation system for far-end telecommunications

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ATE532324T1 (de) * 2007-07-16 2011-11-15 Nuance Communications Inc Verfahren und system zur verarbeitung von tonsignalen in einem multimediasystem eines fahrzeugs
EP2211564B1 (en) * 2009-01-23 2014-09-10 Harman Becker Automotive Systems GmbH Passenger compartment communication system
US8219394B2 (en) * 2010-01-20 2012-07-10 Microsoft Corporation Adaptive ambient sound suppression and speech tracking
CN102164328B (zh) * 2010-12-29 2013-12-11 中国科学院声学研究所 一种用于家庭环境的基于传声器阵列的音频输入系统
US9508344B2 (en) 2014-10-15 2016-11-29 Delphi Technologies, Inc. Automatic volume control based on speech recognition
US11348595B2 (en) * 2017-01-04 2022-05-31 Blackberry Limited Voice interface and vocal entertainment system
WO2018130282A1 (en) 2017-01-12 2018-07-19 Universita' Degli Studi Di Padova Star-like (guanidyl)x-oligosaccharidic compounds and conjugates or complexes thereof
CA3084890A1 (en) * 2017-12-07 2019-06-13 Hed Technologies Sarl Voice aware audio system and method

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180020313A1 (en) 2016-03-03 2018-01-18 Google Inc. Systems and Methods for Spatial Audio Adjustment
US20180308504A1 (en) 2017-02-13 2018-10-25 Bose Corporation Audio systems and method for perturbing signal compensation
WO2019130282A1 (en) 2017-12-29 2019-07-04 Harman International Industries, Incorporated Acoustical in-cabin noise cancellation system for far-end telecommunications

Also Published As

Publication number Publication date
JP2021110948A (ja) 2021-08-02
US20210204059A1 (en) 2021-07-01
JP7667657B2 (ja) 2025-04-23
US11729549B2 (en) 2023-08-15
CN113129929A (zh) 2021-07-16
EP3846495A1 (en) 2021-07-07
KR20210086556A (ko) 2021-07-08

Similar Documents

Publication Publication Date Title
KR102887749B1 (ko) 차량 오디오 시스템을 위한 공간 음성 분리 기능을 갖는 보이스 더킹
US11146887B2 (en) Acoustical in-cabin noise cancellation system for far-end telecommunications
CN111902866B (zh) 头戴式耳机中的双耳自适应噪声消除系统中的回声控制
US20200372926A1 (en) Acoustical in-cabin noise cancellation system for far-end telecommunications
EP2211564B1 (en) Passenger compartment communication system
JP6266849B1 (ja) 共有された音響空間における強められた会話のコミュニケーションに関するフィードバックキャンセレーション
EP1850640B1 (en) Vehicle communication system
CN108141663B (zh) 集音装置以及集音装置的控制方法
JP2021081746A (ja) ノイズ低減のためのオーディオ信号処理
JP2010531125A (ja) 指向性をもって音を放射するシステムおよび方法
EP3133831A1 (en) System and method for in-car communication
US11968517B2 (en) Systems and methods for providing augmented audio
US10299027B2 (en) Headset with reduction of ambient noise
CN105304089A (zh) 虚拟的遮蔽方法
CN108353229A (zh) 车辆中的音频信号处理
US20180122395A1 (en) Virtual Duplex Operation

Legal Events

Date Code Title Description
PA0109 Patent application

St.27 status event code: A-0-1-A10-A12-nap-PA0109

PG1501 Laying open of application

St.27 status event code: A-1-1-Q10-Q12-nap-PG1501

R18-X000 Changes to party contact information recorded

St.27 status event code: A-3-3-R10-R18-oth-X000

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

PA0201 Request for examination

St.27 status event code: A-1-2-D10-D11-exm-PA0201

E902 Notification of reason for refusal
PE0902 Notice of grounds for rejection

St.27 status event code: A-1-2-D10-D21-exm-PE0902

E13 Pre-grant limitation requested

Free format text: ST27 STATUS EVENT CODE: A-2-3-E10-E13-LIM-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

E13-X000 Pre-grant limitation requested

St.27 status event code: A-2-3-E10-E13-lim-X000

P11 Amendment of application requested

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P11-NAP-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

P11-X000 Amendment of application requested

St.27 status event code: A-2-2-P10-P11-nap-X000

P13 Application amended

Free format text: ST27 STATUS EVENT CODE: A-2-2-P10-P13-NAP-X000 (AS PROVIDED BY THE NATIONAL OFFICE)

P13-X000 Application amended

St.27 status event code: A-2-2-P10-P13-nap-X000

D22 Grant of ip right intended

Free format text: ST27 STATUS EVENT CODE: A-1-2-D10-D22-EXM-PE0701 (AS PROVIDED BY THE NATIONAL OFFICE)

PE0701 Decision of registration

St.27 status event code: A-1-2-D10-D22-exm-PE0701

F11 Ip right granted following substantive examination

Free format text: ST27 STATUS EVENT CODE: A-2-4-F10-F11-EXM-PR0701 (AS PROVIDED BY THE NATIONAL OFFICE)

PR0701 Registration of establishment

St.27 status event code: A-2-4-F10-F11-exm-PR0701

PR1002 Payment of registration fee

St.27 status event code: A-2-2-U10-U11-oth-PR1002

Fee payment year number: 1

U11 Full renewal or maintenance fee paid

Free format text: ST27 STATUS EVENT CODE: A-2-2-U10-U11-OTH-PR1002 (AS PROVIDED BY THE NATIONAL OFFICE)

Year of fee payment: 1

PG1601 Publication of registration

St.27 status event code: A-4-4-Q10-Q13-nap-PG1601

Q13 Ip right document published

Free format text: ST27 STATUS EVENT CODE: A-4-4-Q10-Q13-NAP-PG1601 (AS PROVIDED BY THE NATIONAL OFFICE)