EP3046341B1 - Adaptives verfahren gemäss der benutzerpräsenz - Google Patents

Adaptives verfahren gemäss der benutzerpräsenz Download PDF

Info

Publication number
EP3046341B1
EP3046341B1 EP16020006.9A EP16020006A EP3046341B1 EP 3046341 B1 EP3046341 B1 EP 3046341B1 EP 16020006 A EP16020006 A EP 16020006A EP 3046341 B1 EP3046341 B1 EP 3046341B1
Authority
EP
European Patent Office
Prior art keywords
sound
user
zone
audio
val
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP16020006.9A
Other languages
English (en)
French (fr)
Other versions
EP3046341A1 (de
Inventor
Søren Borup Jensen
Søren BECH
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bang and Olufsen AS
Original Assignee
Bang and Olufsen AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Bang and Olufsen AS filed Critical Bang and Olufsen AS
Publication of EP3046341A1 publication Critical patent/EP3046341A1/de
Application granted granted Critical
Publication of EP3046341B1 publication Critical patent/EP3046341B1/de
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303—Tracking of listener position or orientation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R2227/00—Details of public address [PA] systems covered by H04R27/00 but not provided for in any of its subgroups
    • H04R2227/005—Audio distribution systems for home, i.e. multi-room use
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
    • H04R27/00—Public address systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/308—Electronic adaptation dependent on speaker or headphone connection

Definitions

  • the present invention relates to a method for dynamic configuring and reconfiguring a multimedia reproduction system comprising two or more sound channels.
  • the premises of the reconfiguring based on the position of one or more listeners and the one or more sound channels in the reproduction system.
  • the configuring includes portable devices for audio/video sources and audio/video rendering equipment.
  • the feature obtained is a sound distribution into individual sound zones, with optimized sound quality as perceived by a user/listener.
  • the method includes a feature for adaption according to perceptive aspects per user and his/her position in the domain, this known as "object based rendering per user".
  • object based rendering per user This is an advanced configuration tasks for rendering of object-based audio material in domestic environments, where both the reproduction system and the listener position are dynamic.
  • the position of the rendering devices and the user(s) in a domain might be determined via orecision GPS means.
  • the invention includes digital signal processing, sound transducers, filters and amplifier configurations is to be applied in multichannel sound systems, surround sound systems and traditional stereophonic system.
  • the system applied in any domain like a home, in a vehicle, in a boat, in an airplane, in an office or in any other private or public domain.
  • EP 2 161 950 A2 there is disclosed a method and system for dynamic reconfiguring an audio reproduction system comprising two or more sound channels.
  • Reconfiguring is based on the position of one or more listeners.
  • sound can be distributed into individual sound zones.
  • the reconfiguring is based on digital signal processing that configures the sound channels by adjusting equalization (frequency response), delay and gain of the channels according to the position of the individual listeners.
  • equalization frequency response
  • gain of the channels according to the position of the individual listeners.
  • the method and system of this document does not disclose application of the actual physical position of the loudspeakers used in order to arrive at an optimal sound reproduction for the individual listeners.
  • AES Convention Paper 9097 describes a metadata scheme that allows for enhanced audio rendering techniques such as content-dependent processing, automatic scene scaling and enhanced level control.
  • a metadata processing logic is proposed that prepares rendering and playout and allows for user interaction with the audio content of an object-based scene.
  • the invention id defined by method claim 1.
  • a multichannel sound system where different sound settings enabled via digital signal processing means controlling the individual sound channels parameters e.g. the equalization (filters), the delay, the gain (amplification).
  • the premise for the control based upon the listener position in the listening room/space.
  • control and sound setting may work in a contextual mode of operation to accommodate:
  • the system includes:
  • the fundamental goal of the invention is to enhance the perceived sound quality seen from a listener prospective. This obtained by adjusting the gain, the delay and the equalization parameters of the individual sound channels in a multichannel sound system. This feature is regardless of the physical location of the loudspeakers transducers.
  • the invention discloses a system concept having the functional features as "Multimedia - Multi room/domain - Multi user" System Concept.
  • the primary keywords of the system concept Automatically adaptive according to: a) installed equipment, b) user presence and c) user carried equipment:
  • the video rendering is via a screen (not a TV ) and based on digital input from the internet.
  • the audio rendering is typically via active loudspeaker and wirelessly via digital networks.
  • the wireless distribution is within domains, in a zone-based concept that might include two individual sound zones in the same room.
  • a use case example is a tablet applied to browse, preview, select, and providing content:
  • the system includes a feature for adaption according to perceptive aspects per user and his/her position in the domain, this known as "object based rendering per user".
  • object based rendering per user This is an advanced configuration tasks for rendering of object-based audio material in domestic environments, where both the reproduction system and the listener position are dynamic.
  • the position of the rendering devices and the user(s) in a domain might be determined via precision GPS means.
  • the reconfiguring process validates the position of the one or more users, and the validation priorities the position of the one or more listeners:
  • the reconfiguring process includes one or more algorithms to provide the calculation of each of the values of the sound channel adjustment-variables.
  • the reconfiguring process provides a table with relations enabled for access by the digital signal processor to provide each of the values of the sound channel adjustment-variables.
  • the saved attributes and key parameters are loaded into the reconfiguring means supported by electronically means connected wirelessly or is connected wired to the audio reproduction system.
  • the reconfiguring process provides the settings of the sound parameters; gain, equalization and delay for one sound channel applied in one sound field zone related to the physical position of first group of people including one or more listeners.
  • the reconfiguring process provides the settings of the sound parameters; gain, equalization and delay for one sound channel, to be applied in one sound field zone related to the physical position of one or more other groups of people including one or more listeners.
  • a listener position is detected via standard sensor means like switches or infrared detectors or strain gauges or temperature sensitive detectors, or indoor GPS.
  • the configuring process executes automatically controlled by the audio amplifier means, including a digital signal processor, which drives the loudspeaker system.
  • the reconfiguring means are embedded into the controller of an audio reproduction system and/or distributed in one or all of the rendering devices, e.g. sound transducers, displays, and TV's.
  • the reconfiguring is controlled via a table mapping the mode of operation to adjustment parameters for each speaker in every channel or just relevant channels.
  • the adjustment parameters are e.g. but not limited to: equalization, delay and gain.
  • the table may be represented as one or more data set as most appropriate to the digital controller unit.
  • one data set may contain the relations among:
  • Another data set may contain the functional/mode related information like:
  • This table concept is a data driven control systems, and enables for easy updates of the functional behaviour of a specific system simply by loading alternative data set into the controller.
  • the invention includes a constraint solver, which comprises a table with digital data representing at least the constraints of the listener positions and equipment positions and related acoustical adjustment, attributes and corresponding variable values.
  • the constraint solver processing enables an arbitrary access mode to information with no order of sequences required.
  • the configuration domain table is organized as relations among variables in the general mathematical notation of 'Disjunctive Form':
  • AttribVariable 1.1 may be defining a listener position, AttribVariable 1.1 a speaker transducer unit, AttribVariable 1.3 a speaker system/subsystem and AttribVariable 1.n gain value for the transducer unit.
  • AttribVariable 2.n may be a reference to another table.
  • the product configuration function proceeds by finding the result of the interrogation in the set of allowed and possible combinations in one or more Configuration Constraint Tables. According to definitions made in the configuration constraint tables, the result might be:
  • the constraint solver evaluates alternative configurations.
  • the alternatives being one or more of the defined set of legal combination in the constraint table.
  • Table entries in a constraint table may combine into legal/illegal combination such that all the logical operators known from the Boolean algebra will be included as required.
  • the reconfiguring means is embedded into an audio reproduction system.
  • the system becomes the controller of the reconfiguring process that provides a reconfiguration of the system itself.
  • a loudspeaker transducer unit is the fundamental means that transforms the electrical signal to the sound waves produced by movements of the membrane of the transducer unit.
  • the characteristics of each loudspeaker transducer unit are determined by measurement and/or from specifications delivered by the supplier of the unit.
  • an audio reproduction system comprising active sound transducers including an amplifier is provided for each transducer unit.
  • This type of amplifier is e.g. the technology ICEpower from Bang & Olufsen DK.
  • an equalizer is provided per amplifier.
  • the means provides a frequency dependent amplification to control the overall gain, which may be regulated up or down as required.
  • Means for down regulation may be as simple as adjustment of a resistive means serial connected to the loudspeaker module.
  • the sound delay among channels must be controlled.
  • the delay is controlled individually per sound channel.
  • Figure 2 displays specific alternative configurations including:
  • Figure 3 displays how a media file is provided via loudspeaker means (306,308) to a user.
  • A/V media file(s) with related Meta Data is provided to a user e.g. as sound (301,302).
  • Physical constraints define the data about the position in a room of the loudspeaker means (303).
  • Psycho acoustical constraints define perceptual model data applied as correction values to optimize the user listening experience (304), see also Figure 4 .
  • a first user P1 is in one room and the FIR setting (305) are according to loudspeaker (306) position and user position in that room.
  • a second user Q1 is in another room and the FIR setting (307) are according to loudspeaker (308) position and user position in that room. The second user moves to another position Q2 in the second room and accordingly the FIR settings (307) are adjusted.
  • Figure 4 illustrates how to optimize listeners experience according to given physical constraints and perceptual constraints when rendering a media file (400) in two zones.
  • the perceptual constraints are considered to be a perceptual model (409) being the criteria to be fulfilled to optimize the listener experience for each of the user in each of the two sound zones.
  • P1 ⁇ P2 the perceptual model applied in the control process as a correction ⁇ value having
  • the correction function relates the physical aspects and the perceptual aspect as determined via experiments.
  • a sound zone may be characterized as a "personal bubble".
  • One or more bubbles can be configured in a room, typically two bubbles one per two users.
  • a loudspeaker configuration can be a fixed number of loudspeaker transducers placed around in a room, or it might be a variable number of transducers. Some of these fixed located in a room and others being brought into or out from the room as user(s) are entering/leaving that room.
  • FIG 4 two sound zones / bubbles are displayed, one as setup X (401), rendering for user 1 in bubble S1 (403), and having a second setup Y (402), rendering for user 2 in bubble S2 (404) for another user.
  • the current number and physical position of loudspeaker transducers available for the sound field control system (113) is continuously monitored and determined.
  • the reconfiguring process executes automatically, when an audio file (sound object) actually being rendered relates to object based audio metadata that includes information about the position of the sound object versus a period of time, said information being relative correction values +, 0, -, for a set of FIR parameters being addressed.
  • Figure 5 displays a preferred embodiment the configure means is imbedded into the controller of the audio sound system, and/or distributed partly in rendering devices.
  • the audio sound system (500) is the master having the digital signal processor means that initiates, controls and applies the reconfiguring process.
  • Alternative CS table definitions (504) may be loaded into the digital signal processor from external means e.g. a laptop/PC, in addition to actual settings for the system (502).
  • the application interfaces with the constraint solver is via an input/output variable list of variables, which are referred in the constraint definitions (504).
  • the invention is used to automatically adjusting an audio sound system to a maximum level of quality taking into account the position of individual listeners in dedicated zone of sound fields.
  • the invention includes sound channel configurations applied in surround sound systems and traditional stereophonic system, and to be applied in any domain like a home, in a vehicle, in an airplane in a boat, in an office or in any other public domain and alike.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Stereophonic System (AREA)

Claims (10)

  1. Ein Verfahren zur automatischen Konfiguration und Neukonfiguration eines Multimedie Reproduktions Systems in einem oder mehrere Räume (201, 202, 203), welsche Räume mit Audio-oder Video oder beide Arten von Rendering-Geräten (105, 106, 107) versorgen sind, und wohin die Räumein einer oder mehreren einzelnen klangzonen (204, 205) aufgeteilt sind, so dass zwei oder mehreren Benutzer (207, 208) gleichzeitig die gleiche Multimedia-Datei hören können oder verschiedene Multimedia-Dateien in jeder der klangzonen hören können, in denen die Benutzer sich befinden, wobei das Verfahren umfasst:
    (i) Bestimmung der physikalischen Position der einen oder mehreren Zuhörer,
    (ii) Bestimmung der physikalischen Position des einen oder mehreren Lautsprecher Wandlers (306, 308),
    (iii) Anwendung der physikalischen Position von den ein oder anderen Lautsprecher Wandlern und der körperlichen Position des einen oder mehreren Zuhöhrer als Informationen, um eine Reihe von vordefinierten Klangparametern auszuwählen, wobei die Reihe von Parametern der FIR-Parametereinstellungen (305, 307) pro Lautsprecher Wandler (306, 308) umfasst,
    wobei das Verfahren dadurch gekennzeichnet ist, dass:
    (iv) die Anwendung von Kontextinformationen, die sich auf eine aktuelle Mediendatei beziehen, wobei die Kontextinformationen objektbasierte Audio-Metadaten sind;
    (v) die Anwendung von psycho-akustischer Informationen im Zusammenhang mit der perceptuellen Erlebniss eines Nutzers von Klangqualität, wobei die Psycho-akustische Informationen von einem Wahrnehmungsmodell (409) zur Verfügung gestellt werden dass wird verwendet, um die Kriterien festzulegen, die erfüllt werden müssen, um das Hörereigniss jeder Höhrer zu optimieren verwendet wird;
    (vi)die Anwendung der oben genannten Parameter in vollem Umfang oder teilweise als Restriktionen in einem Restriktions Löser (503), der bei der Ausführung eine oder mehrere zulässige Kombinationen von Parametern unter allen definierten Sätzen der zulässigen Parameter Kombinationen finden; und;
    (vii)der Parameter pro Tonkanal und/oder pro Lautsprecher Wandler entsprechend zu Bereitstellen.
  2. Ein Verfahren nach Anspruch 1, bei der der neukonfigurierende Prozess automatisch durchgeführt wird, wenn sich ein Benutzer (207, 208) von einer Position in einer Klangzone in eine andere Position in der gleichen Klangzone bewegt.
  3. Ein Verfahren nach Anspruch 2, bei der der neukonfigurierende Prozess automatisch durchgeführt wird, wenn sich ein Audio-Rendering-Gerät von einer Position in einer Klangzone in eine andere Position in der gleichen Klangzone bewegt.
  4. Ein Verfahren nach Anspruch 3, bei der der neukonfigurierende Prozess automatisch durchgeführt wird, wenn eine tatsächlich reproduzierte Audiodatei (Schallobject Object) sich auf objektbasierte audiometadaten bezieht, dieInformationen über die Position des Klang Objekts versus einen Zeitraum enthalten, wobei die genannten Informationen relative Korrekturwerte +, 0, -, für eine Reihe von FIR-Parametern (305,307), die addressiert werden, sind.
  5. Ein Verfahren nach Anspruch 4, bei der eine stille Zone in einem der Räume/Domains konfiguriert ist, wobei dei Ruhezone, die relativ zu einem aktiv gerenderten Klang ist, in einer Zone im selben Raum/Domäne wie die Ruhezone.
  6. Ein Verfahren nach Anspruch 5, bei der der neukonfigurierende Prozess automatisch durchgeführt wird, wenn ein Quellgerät (209) eines Benutzers sich von einer Position in einer Klangzone in eine andere Position in der gleichen Klangzone bewegt.
  7. Ein Verfahren nach Anspruch 6, bei der der neukonfigurierende Prozess automatisch durchgeführt wird, wenn ein Benutzer von einer Position in einer Klangzone zu einer Position in einer anderen Klangzone wechselt.
  8. Ein Verfahren nach Anspruch 7, bei der der neukonfigurierende Prozess automatisch durchgeführt wird, wenn ein Benutzer von einer Position in einer Klangzone zu einer Position in einer anderen Klangzone wechselt, die von einem anderen aktiven Benutzer besetzt ist.
  9. Ein Verfahren nach einem der vorhergehenden Ansprüche, bei der einem oder mehreren der aktiven Geräte: Controller, Audio-Rendering-Geräte, Video-Rendering-Geräte, die ein Audio-/Video-Reproduktionssystem darstellen, einen Konfigurator enthalten.
  10. Ein Verfahren nach einem der vorhergehenden Ansprüche, bei der die gespeicherten Attribute und Schlüsselparameter in den Konfigurator geladen werden, unterstützt von elektronischen Mitteln , die drahtlos oder verdrahtet mit dem Audiowiedergabesystem verbunden sind.
EP16020006.9A 2015-01-14 2016-01-06 Adaptives verfahren gemäss der benutzerpräsenz Active EP3046341B1 (de)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
DKPA201500024A DK178752B1 (en) 2015-01-14 2015-01-14 Adaptive System According to User Presence

Publications (2)

Publication Number Publication Date
EP3046341A1 EP3046341A1 (de) 2016-07-20
EP3046341B1 true EP3046341B1 (de) 2019-03-06

Family

ID=55070831

Family Applications (1)

Application Number Title Priority Date Filing Date
EP16020006.9A Active EP3046341B1 (de) 2015-01-14 2016-01-06 Adaptives verfahren gemäss der benutzerpräsenz

Country Status (2)

Country Link
EP (1) EP3046341B1 (de)
DK (1) DK178752B1 (de)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9980076B1 (en) 2017-02-21 2018-05-22 At&T Intellectual Property I, L.P. Audio adjustment and profile system
US10516961B2 (en) 2017-03-17 2019-12-24 Nokia Technologies Oy Preferential rendering of multi-user free-viewpoint audio for improved coverage of interest
US10735885B1 (en) 2019-10-11 2020-08-04 Bose Corporation Managing image audio sources in a virtual acoustic environment
US20240114309A1 (en) * 2020-12-03 2024-04-04 Dolby Laboratories Licensing Corporation Progressive calculation and application of rendering configurations for dynamic applications
GB2616073A (en) * 2022-02-28 2023-08-30 Audioscenic Ltd Loudspeaker control

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2161950B1 (de) * 2008-09-08 2019-01-23 Harman Becker Gépkocsirendszer Gyártó Korlátolt Felelösségü Társaság Konfigurierung eines Schallfeldes
US8128342B2 (en) * 2008-10-09 2012-03-06 Manufacturing Resources International, Inc. Multidirectional multisound information system
US9277322B2 (en) * 2012-03-02 2016-03-01 Bang & Olufsen A/S System for optimizing the perceived sound quality in virtual sound zones
US20130294618A1 (en) * 2012-05-06 2013-11-07 Mikhail LYUBACHEV Sound reproducing intellectual system and method of control thereof
US9532153B2 (en) * 2012-08-29 2016-12-27 Bang & Olufsen A/S Method and a system of providing information to a user
US9179232B2 (en) * 2012-09-17 2015-11-03 Nokia Technologies Oy Method and apparatus for associating audio objects with content and geo-location
CN104982043A (zh) * 2013-02-05 2015-10-14 皇家飞利浦有限公司 音频设备及其方法
EP2806664B1 (de) * 2013-05-24 2020-02-26 Harman Becker Automotive Systems GmbH Tonsystem zur Herstellung einer Tonzone

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
DK201500024A1 (en) 2016-08-01
DK178752B1 (en) 2017-01-02
EP3046341A1 (de) 2016-07-20

Similar Documents

Publication Publication Date Title
EP3046341B1 (de) Adaptives verfahren gemäss der benutzerpräsenz
US10536123B2 (en) Volume interactions for connected playback devices
EP2161950B1 (de) Konfigurierung eines Schallfeldes
US11747863B2 (en) Wireless antenna sensing in a playback device
CN107852562B (zh) 校正状态变量
EP2867895B1 (de) Modifizierung von auf näherungserkennung reagierenden ton
JP6161791B2 (ja) メディア再生システム用のプライベートキュー
EP3095027B1 (de) Softwareanwendung und zonen
JP6185146B2 (ja) メディア再生システムにおけるプレイリストの更新
JP6022580B2 (ja) マルチゾーンメディア再生システムのゾーンを管理する方法及び装置
KR20130048794A (ko) 마스터 볼륨 제어 및 개별 볼륨 제어의 동적 조절
US10846045B2 (en) Content based dynamic audio settings
US20240040313A1 (en) Wearable Audio Device Within a Distributed Audio Playback System
CN112005492B (zh) 用于动态声音均衡的方法
US11601757B2 (en) Audio input prioritization
US11395087B2 (en) Level-based audio-object interactions
GB2550877A (en) Object-based audio rendering
CN111095191A (zh) 显示装置及其控制方法
DK178063B1 (en) Dynamic Configuring of a Multichannel Sound System
CN113645560A (zh) 用于控制音箱组的方法、装置和音箱设备
US20240111483A1 (en) Dynamic Volume Control

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20170119

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 27/00 20060101ALN20180910BHEP

Ipc: H04S 7/00 20060101AFI20180910BHEP

RIC1 Information provided on ipc code assigned before grant

Ipc: H04R 27/00 20060101ALN20180920BHEP

Ipc: H04S 7/00 20060101AFI20180920BHEP

INTG Intention to grant announced

Effective date: 20181009

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1106218

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190315

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602016010531

Country of ref document: DE

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: NV

Representative=s name: INDUSTRIAL PROPERTY SERVICES GMBH, CH

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20190306

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190606

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190607

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190606

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1106218

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190706

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602016010531

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190706

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

26N No opposition filed

Effective date: 20191209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200106

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200131

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200106

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190306

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230703

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240123

Year of fee payment: 9

Ref country code: CH

Payment date: 20240202

Year of fee payment: 9

Ref country code: GB

Payment date: 20240122

Year of fee payment: 9