EP3149969A1 - Ermittlung und nutzung hörraumoptimierter übertragungsfunktionen - Google Patents
Ermittlung und nutzung hörraumoptimierter übertragungsfunktionenInfo
- Publication number
- EP3149969A1 EP3149969A1 EP15724972.3A EP15724972A EP3149969A1 EP 3149969 A1 EP3149969 A1 EP 3149969A1 EP 15724972 A EP15724972 A EP 15724972A EP 3149969 A1 EP3149969 A1 EP 3149969A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- room
- listening
- optimized
- transfer functions
- space
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R3/00—Circuits for transducers, loudspeakers or microphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/305—Electronic adaptation of stereophonic audio signals to reverberation of the listening space
- H04S7/306—For headphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/301—Automatic calibration of stereophonic sound system, e.g. with test microphone
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
- H04S7/30—Control circuits for electronic adaptation of the sound field
- H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
- H04S7/303—Tracking of listener position or orientation
- H04S7/304—For headphones
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
- H04S2400/11—Positioning of individual sound objects, e.g. moving airplane, within a sound field
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
Definitions
- Embodiments of the present invention relate to a device for determining transmission functions optimized for a listening room, to a corresponding method and to a device for spatial reproduction of an audio signal using corresponding methods Further embodiments relate to a system comprising the two devices, and to a computer method for carrying out the mentioned methods.
- the perceptual quality in the presentation of a spatial auditory scene depends crucially on the acoustically artistic design of the content of the presentation, the playback system and the room acoustics of the listening room or listening room.
- a major goal in the development of audio playback systems is the generation of auditory events that are considered to be plausible by the listener. This plays a special role in the reproduction of picture-sound content, for example.
- various perceptual quality features such as localizability, perception of distance, perception of space and sound aspects of the image must meet expectations. Ideally, therefore, the perception of the reproduced corresponds to the real situation in space.
- speaker-based audio playback systems two- or multi-channel audio is played back in the listening room.
- This audio material can come from a channel-based mix that already has ready-made speaker signals.
- the speaker signals can also be generated by an object-based sound reproduction method.
- the speaker reproduction signals are generated.
- This produces phantom sources that are usually located on the connection axes between the speakers.
- these phantom sound sources can be perceived by the listener in different directions and distances.
- the room acoustics themselves have a decisive influence on the euphony of the reproduced auditory scene.
- speaker systems are not practical in all listening situations. Furthermore, it is not possible everywhere to install speakers. Examples of such situations may include music listening on mobile devices, use in changing rooms, user acceptance, or the acoustic harassment of others.
- loudspeakers proximity transducers, such as those described in US Pat. In-ears or headphones that are "worn" directly to or in the immediate vicinity of the ear used.
- the classic stereo reproduction via transducers which are each equipped with one acoustic driver per side or ear, for example, generate the listener's perception that the imaging phantom sound sources are located in the head on the connecting axis between the two ears. There is a so-called "in-head localization.”
- a plausibly acting external perception (externality) of the phantom sound sources does not come about.
- the phantom sound sources thus generated usually have neither a user-decodable direction (-sinformation) nor distance ( -sinformation), which would be present in the listening room, for example, when playing the same acoustic scene via a speaker system (eg 2.0 or 5.1).
- Binaural synthesis utilizes so-called head-related transfer function (HRTF) for left and right ear, and these outer ear transfer functions include a plurality of outer ear transfer functions associated with respective direction vectors for virtual sound sources, corresponding to each ear
- HRTF head-related transfer function
- the binaural synthesis makes use of the fact that interaural features are decisively responsible for the realization of a directional perception of a sound source, whereby these interplay are responsible for the fact that an auditory scene is spatially or emulated.
- Aural features in the outer ear transmission functions reflect, so if an audio signal from a defined direction to be perceived, this signal filtered with the left or right ear HRTFs belonging to this direction. With the help of the binaural synthesis, it is thus possible to reproduce both a realistic Jardinkiangsze- ne, for example, stored as a multi-channel audio through the headphones.
- To virtually simulate a speaker setup use the directional HRTF pairs for each speaker to be simulated.
- the direction-dependent acoustic transmission functions of the listening area (room-related-transfer-function, RRTF, room-related transfer function) must also be emulated. These are combined with the HRTFs and yield the binaural room impulse responses (BRIRs, binaural room-impuls-respons).
- the BRIRs can be applied to the acoustic signal as a filter.
- the object of the present invention is to provide an improved spatial reproduction by means of short-range sound transducers, in particular with respect to the conformity of acoustic synthesizing and the consumer's expectation horizon.
- Embodiments of the present invention provide a (portable) device for determining "room-acoustically optimized transmission functions for a listening room.”
- the room-optimized transmission functions are used for audio-optimized post-processing of audio signals in spatial reproduction, based on the outer ear transmission functions (HRTFs)
- HRTFs outer ear transmission functions
- the use of these two transfer functions which can also be referred to as a binaural spatial room impulse response in combination, results in a realistic room sound simulation, which is based on the audio space-optimized transmission functions the space corresponds to the characteristics given by the multi-channel (stereo) signal, but taking into account the tion horizon, which is anticipated in particular by the room acoustics, is improved.
- the present invention provides a further (portable) device for spatial reproduction of an audio signal by means of a binaural Nah Kunststoffsschallwandlers, in which the spatial reproduction is emulated using knownêtohrübertragungsfunktionen and with the aid of a Hörraumoptim studying transfer functions, so that in the Playback of audio content to the acoustic signals emanating from the near-range sound transducer imprints the listening room characteristic.
- the present invention thus provides the prerequisites for considering cognitive effects in the reproduction of multichannel stereo.
- hearing-room-optimized transmission functions for the respective listening room in which, for example, an auditory scene should be reproduced by means of a headphone (generally by means of a binaural close-range sound transducer) are determined.
- the determination of the hearing-space-optimized transfer function corresponds in principle to the derivation of a room-acoustic filter on the basis of the determined or measured room acoustics with the objective of reproducing the acoustic properties of the real space synthetically.
- the auditory scene can then be reproduced in accordance with a second aspect of the invention, both with the aid of the HRTFs and with the aid of the hearing room-optimized transfer functions as a room sound simulation.
- the spatiality is generated by means of HRTFs, while the adjustment of the spatiality to the current listening room situation is achieved by means of hearing room-optimized transmission functions.
- the hearing-space-optimized transmission functions perform an adaptation or post-processing of the HRTFs or the signals processed by the HRTFs.
- the hearing-space-optimized transmission functions namely the metrological determination with the aid of a test sound source and a microphone, so that the room acoustics can be analyzed over a test track in the listening room in order to obtain an acoustic model of the room.
- a second variant can Also naturally occurring noises, such as a voice, are used as test signals.
- the second variant offers the particular advantage that virtually every electrical terminal with a microphone, such as a mobile phone or a smartphone, on which the functionality described above is implemented, sufficient to determine the room acoustics.
- the analysis of the listening room or the determination of the acoustic spatial model can be based on geometric models.
- the background to this is that the room acoustics or acoustic perception changes accordingly, depending on whether the listening position is closer to the wall or in which direction the listener is looking.
- a plurality of direction-dependent and / or position-dependent transfer functions be deposited within the Hörraumopti- m striv transfer functions, which are selected here, for example, depending on the position of the listener in the listening room or from the viewpoint of the listener.
- the spatial reproduction apparatus may also comprise a position determining device, such as e.g. include a GPS.
- the audio material in addition to or parallel to the Ab surgicalraum suspectizing the audio material to be reproduced the corresponding characteristic of a virtual speaker setup, which corresponds for example to the real speaker setup in the listening room or is freely configured.
- Further exemplary embodiments relate to the corresponding methods for determining the audio-space-optimized transmission functions and for reproducing multi-channel stereo audio signals (or object-based audio signals or WFS audio signals) using the audio-space-optimized transmission functions.
- FIG. 1 shows a schematic block diagram of a device for determining hearing-room-optimized transmission functions for a listening room;
- FIG. a schematic flow diagram of a method in the determination of hearing room optimized transfer functions;
- a schematic block diagram of a system for detecting and using hearing room optimized transmission functions is a schematic block diagram of a device for determining hearing-room-optimized transmission functions for a listening room.
- the binaural synthesis is based on filtering an audio signal before output via a sound transducer (preferably directly on one of the ears) with a specific filter function or HRTF, the filter characteristic differing per directional vector or virtual sound source so as to produce surround sound, e.g. when using a headphone, to emulate.
- the filter functions / HRTFs are modeled on the natural damaging mechanisms of human hearing. This makes it possible to edit the audio signal in the analog or digital domain so or to impose an acoustic characteristic on it, as if this is sent from any position in space.
- the mechanisms in the localization of sound are:
- acoustic characteristics such as differences in transit time between left / right and (frequency-dependent) level differences between left / right are decisive.
- runtime differences can be distinguished in particular between phase delay at low frequencies and group delay at high frequencies.
- runtime differences can be simulated via any stereo driver via signal processing.
- the determination of the direction of incidence in the medial plane is based in particular on the fact that the auricle and / or the auditory canal entrance performs a direction-selective filtering of the acoustic signal.
- This filtering is frequency selective, so that an audio signal can be filtered in advance with such a frequency filter to simulate a particular direction of arrival or to emulate a spatiality.
- the determination of the distance of a sound source from the listener is based on different mechanisms.
- the main mechanisms are volume, frequency-selective filtering of the traveled sound path, sound reflection and initial time gap. Much of the above factors are person-specific. Person-individual variables can be, for example, the distance between the ears and the shape of the auricle, which has an effect on the lateral and medial localization.
- HRTFs outer ear transmission functions
- HRTFs outer ear transmission functions
- the background to this is that the above three factors for localization for indoor use are distorted to the extent that the sound emitted by a sound source reaches the listener not only directly but also in a reflected form (eg via walls) Change in the acoustic perception has consequences. In rooms, it comes to direct shali and (later arriving) reflected sound, these sounds for the listener, for example, based on maturity for certain frequency groups and / or position of the secondary sound source in space are differentiable. These (Hall) parameters also depend on the room size and nature (e.g., attenuation, shape) so that a listener can estimate the room size and texture.
- the room acoustics can also be binaurally emulated.
- the RRTF extends the HRTF to the binaural room impulse response (BRIR), which simulates the listener with certain acoustic room conditions in the case of headphone reproduction.
- BRIR binaural room impulse response
- cognitive effects also play a major role for the listener.
- Studies on such cognitive effects have shown that the relevance of parameters such as the degree of agreement between the listening room and the space to be synthesized, the emergence of a plausible auditory illusion are high.
- the expert speaks in the case of a small divergence between listening room and room to be reproduced by low externality of the auditory event.
- the binaural synthesis is to be expanded in such a way that the binaural simulation of an auditory scene can be adapted to the context of its use.
- the simulation is adapted to the listening conditions, such as current room acoustics (attenuation) and the geometry of the interception room.
- the perception of distance, the perception of spatiality and the perception of direction can be varied in such a way that they relate to the current situation. hearing room seem plausible.
- Variation parameters are, for example, the HRTF or RRTF features, such as time differences, level differences, frequency-selective filtering or initial time gap.
- the adaptation takes place, for example, in such a way that a room size with a certain Hall behavior (reverberation behavior or reflection behavior) is emulated or distances between the listener and the sound source, for example, are limited to a maximum value.
- Another factor influencing the surround sound behavior is the position of the user in the listening room, since it is crucial in terms of reverberation and reflection whether the user is centrally located in the room or in the vicinity of a wall.
- This behavior can also be emulated by adjusting the HRTF or RRTF parameters.
- the following section explains how or by which means the adaptation of the HRTF or RRTF parameters is carried out in order to improve the plausibility of the acoustic simulation on-site.
- the concept of auralization of room acoustics in the basic structure comprises two components, which are represented on the one hand by two independent devices and on the other by two corresponding methods.
- 1 a and 1 b the first component, namely the acquisition of hearing-room-optimized transmission functions TF is explained, before referring to FIGS. 2a and 2b, the use of the hearing-room-optimized transmission functions TF will be explained.
- the device 10 for determining transmission functions TF optimized for a monitoring room 12 (transfer function).
- the device 10 includes an interface, e.g. As here illustrated, a microphone interface (see reference numeral 14) illustrates auditory space related data. Since the hearing-space-optimized transfer functions TF, on the basis of which the listening space characteristic is to be impressed on an acoustic material by means of binaural synthesis, is typically designed such that already existing HRTFs are adapted, the device 10 can determine the transfer functions TF taking into account the HRTFs to be used. In this respect, the device 10 optionally includes a further interface for reading in or forwarding HRTFs.
- the detection of the prevailing room acoustic conditions of the listening room is metrologically possible.
- the room acoustics of the listening room 12 are measured by means of an acoustic measuring method with the aid of the device 10.
- a test signal transmitted via an optional loudspeaker (not shown), is used.
- the reproduction of the test signal or the control of the loudspeaker can in this case take place via the device 10, if the device 10 for this purpose comprises a loudspeaker interface (not shown) or the loudspeaker itself.
- the measurement signal radiated into the room 12 via the loudspeaker is recorded by means of the microphone 14, so that the room acoustics can be determined based on the signal change over the measuring path (between loudspeaker microphone), so that at least one hearing room optimized transfer function TF eg for a spatial direction or a plurality of hearing-room-optimized transmission functions TF can be derived. From the measured transfer function from one direction relevant room acoustic parameters are derived for the listening room. These are used to generate the hearing-room-optimized transfer functions TF for the other required directions.
- the discrete first reflections can be adapted to other spatial directions and distance of the virtual sound source positions to be imaged.
- the information relevant for directional perception is available in the HRTFs.
- the determination of the room acoustics can be estimated by using acoustic signals that are already behaving through the listening room 12. Examples of such signals are the ambient sounds that are present anyway, as well as a voice signal of a user.
- the algorithms used for this purpose are derived from algorithms for removing reverberation from a speech signal. The background to this is that an estimation of the space transfer function lying on the signal to be contained is typically carried out in the reverberation algorithms. To date, these algorithms are used to determine a filter which, when applied to the original signal, results in the most likely unattenuated signal. In the application of the analysis of room acoustics, the filter function is not determined, but only an estimation method is used to evaluate the characteristics of the interception to recognize space. In this procedure, again, the microphone 14, which is coupled to the device 10, is used.
- the room acoustics can be simulated based on geometric spatial data. This approach is based on the fact that geometric data (e.g., edge dimensions, free path length) of a room 12 make it possible to estimate the room acoustics.
- the room acoustics of room 12 can either be simulated directly or approximated based on room acoustic filter database comprising comparative acoustic models.
- methods such as the acoustic ray tracing or the mirror sound source method in conjunction with a diffuse sound model can be mentioned. The two methods mentioned are based on geometric models of the listening room.
- the above-explained interface for recording hearing-room-related data of the device 10 does not necessarily have to be a microphone interface, but can also generally be referred to as a data interface which serves for reading geometry data. Furthermore, it is also possible that further data on the room acoustics are read in addition by means of the interface, which include, for example, information about an existing in the listening room speaker setup.
- the data can be retrieved from a geometry database, such as e.g. Google Maps are taken in-house.
- These databases typically include geometric models, e.g. Vector models of spatial geometries, from which primarily the distances, but also reflection characteristics can be determined.
- an image database can also be used as input, in which case the geometric parameters are subsequently determined by means of image recognition in an intermediate step.
- the hearing-space-optimized transfer functions TF are derived in a subsequent step for at least one, preferably for a plurality of rooms.
- the derivation of the auditory room opti- mated transfer functions TF which is comparable in terms of their parameters with the RRTFs, corresponds in principle to the determination of a filter function (per spatial direction), by means of which the acoustic behavior in the room, eg in the sound propagation in a particular spatial direction, can be reproduced.
- the hearing room-specific transmission functions TF per room typically comprise a multiplicity of transmission functions by means of which the outer ear transmission functions (assigned to individual room angles) can be adapted accordingly (comparable to the procedure for processing the room impulse response).
- the number of hearing-space-optimized transmission functions TF therefore typically depends on the number of outer-clock transmission functions which occur as a functional group and a multiplicity, namely for left / right and for the relevant directions.
- the exact number of outer ear transmission functions in the HRTF model will vary according to the desired spatial resolution capability, and may vary considerably due to the existence of H RTF models in which a large number of directional vectors are determined by interpolation.
- the device for determining the hearing-space-optimized transfer function TF uses the HRTF model.
- the determined auditory-space-optimized transfer functions TF are stored, for example, in a room-acoustic filter database.
- a multitude of hearing-room-optimized transmission function groups can also be determined and stored per monitoring room, which takes into account that the listening room functions or the acoustic behavior in the listening room differs depending on the position of the listener.
- a separate hearing room-optimized transmission characteristic can be determined, the determination of which can be based on one and the same acoustic model of the listening room 12.
- the analysis of the listening room is advantageously carried out only once.
- different space-optimized transmission functions flocks can also be determined per spatial direction into which the user is viewing.
- the device 10 described above can be designed differently.
- the device 10 is designed as a mobile device, in which case the sensor 14, such as the microphone or the camera, can be integrated accordingly.
- the analysis unit 10 can be implemented, for example, as hardware or software-based.
- embodiments of the device 10 include an internal or cloud-connected CPU or other logic that is configured to perform the determination of auditory-space optimized transmission functions TF and / or listening room analysis.
- FIG. 1b shows a flow diagram 100 of the method for determining the transfer room-optimized transfer functions TF.
- the method 100 comprises the central step 1 10 of determining the hearing-room-optimized transmission functions TF.
- step 1 10 is based on the analysis of the room acoustics 120 (compare step 120 "Analyzing Room Acoustics") and optionally also on existing HRTF functions Starting from the step 1 10, another optional step, namely the Storing the transfer functions TF. This step is designated by reference numeral 130.
- FIG. 2 a shows a device for spatial reproduction 20 with the aid of a binaural local area sound transducer 22.
- the functionality of the device 20 is explained inter alia with the aid of the flowchart from FIG. 2 b, which illustrates the method 200 of the reproduction.
- the device 20 is designed to 24, such as reproducing a multi-channel stereo audio signal (or an object-based audio signal or an audio signal based on a Wave Field Synthesis Algorithm (WFS)) and simultaneously emulating surround sound (see step 210).
- WFS Wave Field Synthesis Algorithm
- the reproduction apparatus 20 carries out a processing of the audio signal with the aid of HRTFs and with the aid of the hearing room-optimized transmission functions TF.
- the device 20 may comprise an HRTF / TF memory or is connected, for example, to a database on which the HRTFs as well as the hearing-room-optimized transmission functions TF determined in accordance with the above methods are stored.
- combining before the signal processing of the audio signal, combining (see step 220) the HRTF with the TF or adjusting the HRTF based on the TF.
- the result of this combining is a BRIR 'transfer function comparable to the BRIR (spatial impulse response), which is then used to process the audio signal 24 to emulate the surround sound (see step 210).
- This processing is basically equivalent to applying a BRIR 'based filter to the audio signal.
- the synthesized space (at least approximately) matches the user's expectation horizon, which increases the plausibility of the scene.
- the device 20 may also include the position determination unit, such as a GPS receiver, by means of which the current position of the listener can be determined. Starting from the determined position, the monitoring room can now be determined and the hearing room-optimized transmission functions TF associated with the monitoring room can be loaded (and, if necessary, updated during a room change).
- the position determination unit such as a GPS receiver
- the monitoring room can now be determined and the hearing room-optimized transmission functions TF associated with the monitoring room can be loaded (and, if necessary, updated during a room change).
- this position determination unit can also be extended by an orientation determination unit so that the viewing direction of the listener can also be determined and the TFs are correspondingly loaded depending on the particular viewing direction in order to cope with the direction-dependent monitoring room acoustics.
- an extendedskysbeispiei of Fig. 3 will now be explained. 3 shows a schematic representation of the signal flow when listening to adapted room acoustic simulations for use with binaural synthesis from a system 10 + 20 comprising the device for determining the TFs and the device for reproducing the audio signals using the TFs.
- Such a system 10 + 20 may, for example, be implemented as a mobile terminal (e.g., a smartphone) on which the file to be played is also stored.
- the system 10 + 20 is in principle a combination of the device 10 of FIG. 1 a and the device 20 of FIG. 1 b, wherein the individual components are subdivided differently for function-oriented explanation.
- the system 10 + 20 comprises a functional unit for the auralization of the listening room 20a and a functional unit for the binaural synthesis 20b.
- the system 10 + 20 includes a function block 10a for modeling the room acoustics and a function block 10b for modeling the transmission behavior.
- the modeling of the room acoustics in turn is based on a detection of the listening room, which is performed with the function block 10c for detecting the room acoustics.
- the system 10 + 20 includes two memories, one for storing scene position data 30a and one for storing HRTF data 30b.
- the functionality of the system 10 + 20 is explained below on the basis of the information flow during reproduction, it being assumed that the interception space is known to the system 10 + 20 or has already been determined by means of a position determination method (see above).
- the audio data is supplied in a first step to the signal processing unit 20a, which applies the previously modeled space transfer function TF to the signal 24 and dies out.
- the modeling of the space transfer function TF takes place in a signal processing block 10a, this modeling being superimposed by the modeling transfer behavior (see function block 10b), as will be explained below.
- This second (optional) function block 10b models a virtual speaker setup in the respective listening room.
- the user can be emulated an acoustic behavior as if the audio file to be played on a particular speaker setup (2.0, 5.1, 9.2) is played.
- the loudspeaker position is fixedly connected to the listening room and the respective loudspeakers are also assigned a specific transmission behavior, eg defined by the frequency response and directional characteristic or different level behavior.
- a specific transmission behavior eg defined by the frequency response and directional characteristic or different level behavior.
- special sound source types eg a mirror sound source, in the room.
- the speaker setup is modeled based on the scene position data that includes information about the position, distance, or type of the virtual speaker. This scene position data may correspond to a real existing speaker setup, or based on virtual speaker setup, and is typically customizable by the user.
- the reverberated signals are fed to the binaural synthesis 20b which impress the direction of the virtual loudspeakers onto the audio material associated with the loudspeaker through a set of directional HRTF filters (see Fig. 30b).
- the binaural synthesis system can optionally evaluate the head rotation of the listener. The result is a headphone signal, which can be adjusted by appropriate equalization to a particular headphone, with the acoustic signal behaving as if it were being delivered with a specific speaker setup in the respective listening room.
- the system 10 + 20 may be implemented, for example, as a mobile terminal or components of a home theater system.
- the device 20 of FIG. 2a may also be configured to emulate a particular loudspeaker setup or the reproduction of an audio signal for a particular loudspeaker setup based on scene position data.
- the device 10 may be adapted to the scene position data of a speaker setup in the listening room 12 (eg via a acoustic measurement), so that this speaker setup can be emulated with the device 20.
- aspects have been described in the context of a device, it will be understood that these aspects also constitute a description of the corresponding method, so that a block or a component of a device is also to be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device.
- Some or all of the method steps may be performed by a hardware device (or using a hardware device). Apparatus), such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some or more of the most important method steps may be performed by such an apparatus.
- a signal coded according to the invention such as an audio signal or a video signal or a transport stream signal, may be stored on a digital storage medium or may be stored on a transmission medium such as a wireless transmission medium or a wired transmission medium, e.g. the internet
- the encoded audio signal of the present invention may be stored on a digital storage medium, or may be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.
- embodiments of the invention may be implemented in hardware or in software.
- the implementation may be performed using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or FLASH memory, a hard disk, or other magnetic disk or optical memory are stored on the electronically readable control signals, which can cooperate with a programmable computer system or cooperate such that the respective method is performed. Therefore, the digital storage medium can be computer readable.
- some embodiments according to the invention include a data carrier having electronically readable control signals capable of interacting with a programmable computer system to perform one of the methods described herein.
- embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is operable to perform one of the methods when the computer program product runs on a computer.
- the program code can also be stored, for example, on a machine-readable carrier.
- Other embodiments include the computer program for performing any of the methods described herein, wherein the computer program is stored on a machine-readable medium.
- an embodiment of the method according to the invention is thus a computer program which has a program code for performing one of the methods described herein when the computer program runs on a computer.
- a further embodiment of the method according to the invention is thus a data medium (or a digital storage medium or a computer-readable medium) on which the computer program is recorded for performing one of the methods described herein.
- a further exemplary embodiment of the method according to the invention is thus a data stream or a sequence of signals which represents or represents the computer program for performing one of the methods described herein.
- the data stream or the sequence of signals may be configured, for example, to be transferred via a data communication connection, for example via the Internet.
- Another embodiment includes a processing device, such as a computer or a programmable logic device, that is configured or adapted to perform one of the methods described herein.
- Another embodiment includes a computer on which the computer program is installed to perform one of the methods described herein.
- Another embodiment according to the invention comprises a device or system adapted to transmit a computer program for performing at least one of the methods described herein to a receiver.
- the transmission can be done for example electronically or optically.
- the receiver may be, for example, a computer, a mobile device, a storage device or a similar device.
- the device or system may include a file server for transmitting the computer program to the recipient.
- a programmable logic device eg, a field programmable gate array, an FPGA
- a field programmable gate rarray may cooperate with a microprocessor to perform one of the methods described herein.
- the methods are performed by any hardware device. This may be a universal hardware such as a computer processor (CPU) or hardware specific to the process, such as an ASIC.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Stereophonic System (AREA)
- Soundproofing, Sound Blocking, And Sound Damping (AREA)
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102014210215.4A DE102014210215A1 (de) | 2014-05-28 | 2014-05-28 | Ermittlung und Nutzung hörraumoptimierter Übertragungsfunktionen |
PCT/EP2015/060792 WO2015180973A1 (de) | 2014-05-28 | 2015-05-15 | Ermittlung und nutzung hörraumoptimierter übertragungsfunktionen |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3149969A1 true EP3149969A1 (de) | 2017-04-05 |
EP3149969B1 EP3149969B1 (de) | 2019-09-18 |
Family
ID=53268781
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP15724972.3A Active EP3149969B1 (de) | 2014-05-28 | 2015-05-15 | Ermittlung und nutzung hörraumoptimierter übertragungsfunktionen |
Country Status (7)
Country | Link |
---|---|
US (1) | US10003906B2 (de) |
EP (1) | EP3149969B1 (de) |
JP (1) | JP6446068B2 (de) |
KR (1) | KR102008771B1 (de) |
CN (1) | CN106576203B (de) |
DE (1) | DE102014210215A1 (de) |
WO (1) | WO2015180973A1 (de) |
Families Citing this family (31)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2546504B (en) | 2016-01-19 | 2020-03-25 | Facebook Inc | Audio system and method |
US9591427B1 (en) * | 2016-02-20 | 2017-03-07 | Philip Scott Lyren | Capturing audio impulse responses of a person with a smartphone |
US10187740B2 (en) * | 2016-09-23 | 2019-01-22 | Apple Inc. | Producing headphone driver signals in a digital audio signal processing binaural rendering environment |
US11197119B1 (en) | 2017-05-31 | 2021-12-07 | Apple Inc. | Acoustically effective room volume |
CN109286889A (zh) * | 2017-07-21 | 2019-01-29 | 华为技术有限公司 | 一种音频处理方法及装置、终端设备 |
DK3454578T3 (da) * | 2017-09-06 | 2021-01-04 | Sennheiser Communications As | Kommunikationssystem til kommunikation af audiosignaler mellem en flerhed af kommunikationsanordninger i et virtuelt lydmiljø |
US10390171B2 (en) * | 2018-01-07 | 2019-08-20 | Creative Technology Ltd | Method for generating customized spatial audio with head tracking |
ES2954317T3 (es) * | 2018-03-28 | 2023-11-21 | Fund Eurecat | Técnica de reverberación para audio 3D |
US10764703B2 (en) | 2018-03-28 | 2020-09-01 | Sony Corporation | Acoustic metamaterial device, method and computer program |
US11617050B2 (en) | 2018-04-04 | 2023-03-28 | Bose Corporation | Systems and methods for sound source virtualization |
US10966046B2 (en) * | 2018-12-07 | 2021-03-30 | Creative Technology Ltd | Spatial repositioning of multiple audio streams |
US11418903B2 (en) | 2018-12-07 | 2022-08-16 | Creative Technology Ltd | Spatial repositioning of multiple audio streams |
US11113092B2 (en) | 2019-02-08 | 2021-09-07 | Sony Corporation | Global HRTF repository |
WO2020189263A1 (ja) * | 2019-03-19 | 2020-09-24 | ソニー株式会社 | 音響処理装置、音響処理方法、および音響処理プログラム |
US11451907B2 (en) | 2019-05-29 | 2022-09-20 | Sony Corporation | Techniques combining plural head-related transfer function (HRTF) spheres to place audio objects |
US11347832B2 (en) | 2019-06-13 | 2022-05-31 | Sony Corporation | Head related transfer function (HRTF) as biometric authentication |
US11146908B2 (en) | 2019-10-24 | 2021-10-12 | Sony Corporation | Generating personalized end user head-related transfer function (HRTF) from generic HRTF |
US11330371B2 (en) | 2019-11-07 | 2022-05-10 | Sony Group Corporation | Audio control based on room correction and head related transfer function |
US11070930B2 (en) * | 2019-11-12 | 2021-07-20 | Sony Corporation | Generating personalized end user room-related transfer function (RRTF) |
WO2021106613A1 (ja) * | 2019-11-29 | 2021-06-03 | ソニーグループ株式会社 | 信号処理装置および方法、並びにプログラム |
CN111031467A (zh) * | 2019-12-27 | 2020-04-17 | 中航华东光电(上海)有限公司 | 一种hrir前后方位增强方法 |
CN111372167B (zh) * | 2020-02-24 | 2021-10-26 | Oppo广东移动通信有限公司 | 音效优化方法及装置、电子设备、存储介质 |
JP7463796B2 (ja) | 2020-03-25 | 2024-04-09 | ヤマハ株式会社 | デバイスシステム、音質制御方法および音質制御プログラム |
US11356795B2 (en) | 2020-06-17 | 2022-06-07 | Bose Corporation | Spatialized audio relative to a peripheral device |
EP3945729A1 (de) * | 2020-07-31 | 2022-02-02 | FRAUNHOFER-GESELLSCHAFT zur Förderung der angewandten Forschung e.V. | System und verfahren zur kopfhörerentzerrung und raumanpassung zur binauralen wiedergabe bei augmented reality |
US11982738B2 (en) | 2020-09-16 | 2024-05-14 | Bose Corporation | Methods and systems for determining position and orientation of a device using acoustic beacons |
US11700497B2 (en) | 2020-10-30 | 2023-07-11 | Bose Corporation | Systems and methods for providing augmented audio |
US11696084B2 (en) | 2020-10-30 | 2023-07-04 | Bose Corporation | Systems and methods for providing augmented audio |
CN112584277B (zh) * | 2020-12-08 | 2022-04-22 | 北京声加科技有限公司 | 一种室内音频均衡的方法 |
WO2022231338A1 (en) * | 2021-04-28 | 2022-11-03 | Samsung Electronics Co., Ltd. | A portable receiving device of calibrating device configurations using room geometry and a system including the same |
KR102652559B1 (ko) * | 2021-11-24 | 2024-04-01 | 주식회사 디지소닉 | 음향실 및 이를 이용한 brir 획득 방법 |
Family Cites Families (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE10249416B4 (de) * | 2002-10-23 | 2009-07-30 | Siemens Audiologische Technik Gmbh | Verfahren zum Einstellen und zum Betrieb eines Hörhilfegerätes sowie Hörhilfegerät |
JP2005223713A (ja) * | 2004-02-06 | 2005-08-18 | Sony Corp | 音響再生装置、音響再生方法 |
GB0419346D0 (en) * | 2004-09-01 | 2004-09-29 | Smyth Stephen M F | Method and apparatus for improved headphone virtualisation |
US20080273708A1 (en) * | 2007-05-03 | 2008-11-06 | Telefonaktiebolaget L M Ericsson (Publ) | Early Reflection Method for Enhanced Externalization |
EP2356825A4 (de) * | 2008-10-20 | 2014-08-06 | Genaudio Inc | Audiospatialisierung und umgebungssimulation |
WO2012093352A1 (en) * | 2011-01-05 | 2012-07-12 | Koninklijke Philips Electronics N.V. | An audio system and method of operation therefor |
CN103563401B (zh) * | 2011-06-09 | 2016-05-25 | 索尼爱立信移动通讯有限公司 | 减少头部相关传递函数数据量 |
WO2013058728A1 (en) * | 2011-10-17 | 2013-04-25 | Nuance Communications, Inc. | Speech signal enhancement using visual information |
-
2014
- 2014-05-28 DE DE102014210215.4A patent/DE102014210215A1/de not_active Withdrawn
-
2015
- 2015-05-15 KR KR1020167036695A patent/KR102008771B1/ko active IP Right Grant
- 2015-05-15 CN CN201580028158.2A patent/CN106576203B/zh active Active
- 2015-05-15 EP EP15724972.3A patent/EP3149969B1/de active Active
- 2015-05-15 WO PCT/EP2015/060792 patent/WO2015180973A1/de active Application Filing
- 2015-05-15 JP JP2016569427A patent/JP6446068B2/ja active Active
-
2016
- 2016-11-28 US US15/362,017 patent/US10003906B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
CN106576203A (zh) | 2017-04-19 |
JP6446068B2 (ja) | 2018-12-26 |
US20170078820A1 (en) | 2017-03-16 |
CN106576203B (zh) | 2020-02-07 |
KR20170013931A (ko) | 2017-02-07 |
DE102014210215A1 (de) | 2015-12-03 |
US10003906B2 (en) | 2018-06-19 |
KR102008771B1 (ko) | 2019-08-09 |
EP3149969B1 (de) | 2019-09-18 |
WO2015180973A1 (de) | 2015-12-03 |
JP2017522771A (ja) | 2017-08-10 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3149969B1 (de) | Ermittlung und nutzung hörraumoptimierter übertragungsfunktionen | |
DE60304358T2 (de) | Verfahren zur verarbeitung von audiodateien und erfassungsvorrichtung zur anwendung davon | |
CN110192396A (zh) | 用于基于头部跟踪数据确定和/或使用音频滤波器的方法和系统 | |
CN106797525A (zh) | 用于生成和回放音频信号的方法和设备 | |
KR20200047414A (ko) | 헤드셋을 통한 공간 오디오 렌더링을 위한 룸 특성 수정 시스템 및 방법 | |
DE102019107302A1 (de) | Verfahren zum Erzeugen und Wiedergeben einer binauralen Aufnahme | |
CN112005559B (zh) | 改进环绕声的定位的方法 | |
EP3225039B1 (de) | System und verfahren zur erzeugung von kopfexternalisiertem 3d-audio durch kopfhörer | |
WO2015036271A2 (de) | Vorrichtung und verfahren zur dekorrelation von lautsprechersignalen | |
DE102021103210A1 (de) | Surround-Sound-Wiedergabe basierend auf Raumakustik | |
Rasumow et al. | Perceptual evaluation of individualized binaural reproduction using a virtual artificial head | |
DE102011003450A1 (de) | Erzeugung von benutzerangepassten Signalverarbeitungsparametern | |
Brandenburg et al. | Auditory illusion through headphones: History, challenges and new solutions | |
DE112021001695T5 (de) | Schallverarbeitungsvorrichtung, schallverarbeitungsverfahren und schallverarbeitungsprogramm | |
US20190394583A1 (en) | Method of audio reproduction in a hearing device and hearing device | |
EP2503799B1 (de) | Verfahren und System zur Berechnung synthetischer Außenohrübertragungsfunktionen durch virtuelle lokale Schallfeldsynthese | |
US20230104111A1 (en) | Determining a virtual listening environment | |
Aspöck et al. | Dynamic real-time auralization for experiments on the perception of hearing impaired subjects | |
Urbanietz | Advances in binaural technology for dynamic virtual environments | |
CN117793609A (zh) | 一种声场渲染方法和装置 | |
Werner | Auditory illusion through headphones: History, Challenges and new Solutions | |
Netcom et al. | SIMULATION OF REALISTIC BACKGROUND NOISE USING MULTIPLE LOUDSPEAKERS | |
Hrauda | Virtual binaural acoustics in VLC player: HRTFs and efficient rendering |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20161123 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
17Q | First examination report despatched |
Effective date: 20180201 |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1236308 Country of ref document: HK |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: BRANDENBURG, KARLHEINZ Inventor name: WERNER, STEPHAN Inventor name: SLADECZEK, CHRISTOPH |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20190401 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502015010407 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1182731 Country of ref document: AT Kind code of ref document: T Effective date: 20191015 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20190918 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20191218 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20191218 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20191219 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200120 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200224 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502015010407 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG2D | Information on lapse in contracting state deleted |
Ref country code: IS |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20200119 |
|
26N | No opposition filed |
Effective date: 20200619 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200531 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200531 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20200531 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200531 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 1182731 Country of ref document: AT Kind code of ref document: T Effective date: 20200515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20200515 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190918 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230524 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230517 Year of fee payment: 9 Ref country code: DE Payment date: 20230519 Year of fee payment: 9 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230522 Year of fee payment: 9 |