EP3069530A1 - Method and device for compressing and decompressing sound field data of an area - Google Patents
Method and device for compressing and decompressing sound field data of an areaInfo
- Publication number
- EP3069530A1 EP3069530A1 EP14793183.6A EP14793183A EP3069530A1 EP 3069530 A1 EP3069530 A1 EP 3069530A1 EP 14793183 A EP14793183 A EP 14793183A EP 3069530 A1 EP3069530 A1 EP 3069530A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- harmonic components
- order
- sound field
- field data
- designed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 41
- 230000004044 response Effects 0.000 claims description 64
- 230000003595 spectral effect Effects 0.000 claims description 29
- 230000008447 perception Effects 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 13
- 230000006870 function Effects 0.000 claims description 13
- 230000015572 biosynthetic process Effects 0.000 claims description 12
- 238000003786 synthesis reaction Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 10
- 230000005540 biological transmission Effects 0.000 claims description 9
- 238000003860 storage Methods 0.000 claims description 9
- 230000009466 transformation Effects 0.000 claims description 7
- 238000012546 transfer Methods 0.000 claims description 3
- 238000001914 filtration Methods 0.000 claims 1
- 238000000354 decomposition reaction Methods 0.000 description 20
- 238000007906 compression Methods 0.000 description 10
- 230000006835 compression Effects 0.000 description 10
- 230000002123 temporal effect Effects 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 238000004458 analytical method Methods 0.000 description 7
- 230000009467 reduction Effects 0.000 description 7
- 238000009877 rendering Methods 0.000 description 7
- 238000009826 distribution Methods 0.000 description 5
- 230000001419 dependent effect Effects 0.000 description 4
- 238000011045 prefiltration Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000001228 spectrum Methods 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 238000005259 measurement Methods 0.000 description 3
- 238000000926 separation method Methods 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013144 data compression Methods 0.000 description 2
- 230000006837 decompression Effects 0.000 description 2
- 230000004069 differentiation Effects 0.000 description 2
- 238000005265 energy consumption Methods 0.000 description 2
- 230000002349 favourable effect Effects 0.000 description 2
- 241000283690 Bos taurus Species 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 238000012952 Resampling Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 238000011038 discontinuous diafiltration by volume reduction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000004886 head movement Effects 0.000 description 1
- 238000001093 holography Methods 0.000 description 1
- 238000011835 investigation Methods 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 230000005291 magnetic effect Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004091 panning Methods 0.000 description 1
- 238000002360 preparation method Methods 0.000 description 1
- 230000008569 process Effects 0.000 description 1
- 230000005855 radiation Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000036962 time dependent Effects 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S3/00—Systems employing more than two channels, e.g. quadraphonic
- H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/008—Multichannel audio signal coding or decoding using interchannel correlation to reduce redundancy, e.g. joint-stereo, intensity-coding or matrixing
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
- G10L19/0208—Subband vocoders
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0212—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using orthogonal transformation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/02—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders
- G10L19/0204—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis using spectral analysis, e.g. transform vocoders or subband vocoders using subband decomposition
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04S—STEREOPHONIC SYSTEMS
- H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
- H04S2420/11—Application of ambisonics in stereophonic audio systems
Definitions
- the present invention relates to audio technology, and more particularly to the compression of spatial sound field data.
- the acoustic description of rooms is of great interest for the control of display devices in the form of e.g. a headphone, a speaker assembly with e.g. two to an average number of loudspeakers, such as 10 loudspeakers or loudspeaker arrangements with a large number of loudspeakers, such as those used in Wave Field Synthesis (WFS).
- WFS Wave Field Synthesis
- One approach is e.g. in creating different channels for different speakers at predefined speaker positions, as is the case with MPEG surround, for example.
- a listener located in the reproduction room at a certain and, ideally, the middle position obtains a sense of space for the reproduced sound field.
- An alternative spatial description is to describe a space through its impulse response. For example, if a sound source is positioned anywhere in a room or area, that space or area may be measured with a circular array of microphones in the case of a two-dimensional area or with a ball-microphone array in the case of a three-dimensional area. For example, if a ball-microphone array with a high number of microphones is considered, such as 350 microphones, then a survey of the room will be done as follows. At a certain position inside or outside the microphone array, a pulse is generated. Then the response to this pulse, ie the impulse response, is measured by each microphone. Depending on how strong the reverb characteristics are, a longer or shorter impulse response is then measured.
- impulse responses can last over 10 s.
- Such a set of eg 350 impulse responses thus describes the sound characteristic of this space for the particular position of a sound source at which the impulse has been generated.
- this set of impulse responses represents sound field data of the area for exactly the one case where a source is positioned at the position where the impulse was generated.
- the procedure described In order to measure the room further, ie to capture the sound properties of the room when a source is positioned at another room, the procedure described must be repeated for each additional position eg outside the array (but also inside the array).
- the temporal length of the impulse responses can be quite considerable, and since a more detailed representation of the sound properties of the room may be desired in terms of not only four but even more positions, there is a huge amount of impulse response data, especially if taken into account - it is considered that the impulse responses can certainly take lengths over 10 s.
- a harmonic decoder scheme for low bit rates is described in [8].
- the encoder generates a composite audio information signal representing the sound field to be reproduced describes and a directional vector or steering control signal.
- the spectrum is divided into subbands. For control, the dominant direction is evaluated in each subband.
- [9] describes a spatial audio coding framework in the frequency domain. Time-frequency-dependent direction vectors describe the input audio scene.
- [10] describes a parametric, channel-based audio coding method in the time and frequency domain.
- a binaural-cue-coding (BCC) is described which uses one or more object-based cue codes. These include the direction, width and envelopment of an auditory scene.
- BCC binaural-cue-coding
- [12] refers to the processing of sphere array data for playback using Ambisonics. Thereby the distortions of the system are to be affected by measurement errors, e.g. Noise, be equalized.
- a channel-based coding method is described, which also refers to positions of the loudspeakers, as well as individual audio objects.
- a matrix-based coding method is presented, which allows the real-time transmission of Higher Order Ambisonics sound fields with orders greater than 3.
- [15] describes a method for encoding spatial audio that is independent of the rendering system.
- the input material is subdivided into two groups, of which the first group contains the audio, which requires high localizability, while the second group is described with sufficiently low Ambisonics orders for the localization.
- the signal is encoded into a set of mono channels with metadata.
- the metadata includes time information as to when the corresponding channel is to be played back and directional information at each moment.
- the audio channels are decoded for conventional panning algorithms, where the playback system must be known.
- the audio in the second group is encoded into channels of different Ambisonics orders. In decoding, the playback system uses corresponding Ambisonics orders.
- Corteel E and Rosenthal M (201 1) Method and device for enhanced sound field reproduction of spatially encoded audio input signals, EP 2609759 A1
- the object of the present invention is to provide a more efficient concept for handling such.
- a device for compressing sound field data of a region comprises a splitter for splitting the sound field data into a first part and a second part, and a downstream converter for converting the first part and the second part into harmonic components, wherein the conversion takes place so that the second Number is converted into one or more harmonic components with a second order, and that the first fraction is converted into harmonic components with a first order, with the first order being higher than the second order, to obtain the compressed sound field data.
- an implementation of the sound field data such as, for example, the amount of impulse responses, is carried out in harmonic components, whereby even this conversion can lead to a considerable saving of data.
- Harmonic components such as are obtainable by means of a spatial spectral transformation, describe a sound field much more compact than impulse responses.
- the order of harmonic components is readily controllable.
- the har- monic zero-order component is just a (non-directional) mono signal. It does not yet allow a sound field direction description.
- the additional first order harmonic components already allow a relatively coarse directional representation analogous to beamforming.
- the harmonic components of second order allow an even more accurate sound field description with even more directional information.
- the number of components is 2n + 1, where n is the order. For the zeroth order there is thus only a single harmonic component. There are already three harmonic components for implementation up to the first order. For example, for a fifth-order conversion, there are already 11 harmonic components, and it has been found that, for example, for 350 impulse responses, an order equal to 14 is sufficient. In other words, this means that 29 harmonic components describe space as well as 350 impulse responses. Already this conversion from a value of 350 input channels to 29 output channels brings a compression gain. Moreover, according to the invention, a conversion of different parts of the sound field data, such as impulse responses with different orders, is carried out since it has been found that not all parts have to be described with the same accuracy / order.
- the directional perception of the human ear is mainly derived from the early reflections, while the late / diffuse reflections in a typical impulse response to directional perception contribute little or nothing.
- the first portion will be the early portion of the impulse responses, which is translated at a higher order to the harmonic component range, while the late diffused portion is converted to a lower order and sometimes even zero order becomes.
- the directional perception of human hearing is frequency-dependent. At low frequencies, the directional perception of the human ear is relatively weak.
- the sound field data can be broken down into individual subband sound field data by means of a filter bank, and these subband sound field data are then decomposed with different orders, again the first portion having subband sound field data at higher frequencies, while the second portion having subband sound field data at lower frequencies , where very low frequencies can again be represented even with an order equal to zero, ie only with a single harmonic component.
- the advantageous characteristics of temporal and frequency processing are combined.
- the early portion which is anyway implemented with a higher order, can be decomposed into spectral components, for which then again orders adapted to the individual bands can be obtained.
- QMF quadrature mirror filter bank
- the effort to convert the subband sound field data in the harmonic component area is additionally reduced.
- the differentiation of different parts of the sound field data with respect to the order to be calculated provides a considerable reduction of the computational effort, since the calculation of the harmonic components, such as the cylindrical harmonic components or the spherical harmonic components, depends strongly on the order up to which order the harmonic components are to be calculated.
- a calculation of the harmonic components to the second order requires much less computational effort and thus computation time or battery performance, especially in mobile devices as a calculation of the harmonic components to order 14, for example.
- the converter is thus designed to implement the component, that is to say the first component of the sound field data, which is more important for a directional perception of the human ear, with a higher order than the second part, which is less important for the directional perception of a sound source than the first part.
- the present invention can be used not only for a temporal decomposition of the sound field data into shares or for a spectral decomposition of the sound field data into shares, but also for an alternative, for.
- B. spatial decomposition of the shares for example, if it is considered that the direction perception of the human ear for sound in different azimuth or elevation angles is different.
- the sound field data is impulse responses or other sound field annotations in which each description is assigned a certain azimuth / elevation angle
- the sound field data may be from azimuth / elevation angles, where the direction perception of the human ear is stronger, compressed with a higher order than a spatial portion of the sound field data from another direction.
- the individual harmonics can be "thinned out", ie in the example with order 14, in which there are 29.
- Modes Depending on the human direction perception, individual modes are saved which map the sound field for unimportant sound incidence directions - There is some uncertainty here, because you do not know in which direction the head is aligned to the array sphere, but if you represent HRTFs by means of spherical harmonics, this uncertainty is resolved.
- decompositions of the sound field data in addition to decompositions in temporal, spectral or spatial direction can also be used, such as a decomposition of the sound field data into a first and a second portion in volume classes, etc.
- the description of acoustic problems occurs in the cylindrical or spherical coordinate system, ie by means of complete sets of orthogonal eigenfunctions, the so-called cylindrical or spherical harmonic components.
- cylindrical or spherical harmonic components With higher spatial accuracy of the description of the sound field increase the amount of data and the computing time in the processing or manipulation of the data.
- high accuracies are required, which adds to the problems of long computation times, which are particularly detrimental to real-time systems, the large amount of data, which makes the transmission more spatial Sound field data difficult, and high energy consumption by intensive computational effort, especially in mobile devices leads.
- the reduced computing power and the reduced storage volume automatically reduce the energy consumption, which is particularly incurred when using sound field data in mobile devices.
- the spatial sound field description is optimized in the cylindrical or spherical harmonic region based on the spatial perception of humans.
- a combination of time- and frequency-dependent calculation of the order of spherical harmonics as a function of the spatial perception of human hearing leads to a considerable effort reduction without reducing the subjective quality of sound field perception.
- the objective quality is reduced since the present invention represents lossy compression.
- this lossy compression is not critical, especially since the ultimate receiver is the human ear, and therefore it is irrelevant even for transparent reproduction, whether or not sound field components, which are not perceived by the human ear anyway, are present in the reproduced sound field.
- FIG. 1a is a block diagram of an apparatus for compressing sound field data according to an embodiment
- Fig. 1b is a block diagram of an apparatus for decompressing compressed sound field data of a region
- FIG. 1 c is a block diagram of a device for compression with temporal decomposition
- Fig. 1d is a block diagram of an embodiment of a device for decompressing in the case of a temporal decomposition
- Fig. 1e is an alternative to Fig. 1d device for decompressing
- Fig. 1f an example of the application of the invention with temporal and spectral
- Fig. 2a is a block diagram of a device for compressing with spectral
- Fig. 2b shows an example of a sub-sampled filter bank and a subsequent one
- Fig. 2c shows a device for decompressing the example of the spectral decomposition shown in Fig. 2a;
- Fig. 2d shows an alternative implementation of the decompressor for the spectral
- decomposition an overview block diagram with a special analysis / synthesis encoder according to another embodiment of the present invention; a more detailed representation of an embodiment with temporal and spectral decomposition; a schematic representation of an impulse response; a block diagram of a converter from the time or spectral range in the harmonic component variable-order area; and an illustration of an exemplary harmonic component region converter into the time domain or spectral domain followed by auralization.
- FIG. 1 a shows a block diagram of a device or a method for compressing sound field data of a region, as they are input at an input 10 into a splitter 100.
- the splitter 100 is designed to divide the sound field data into a first portion 101 and a second portion 102.
- a converter is provided which has the two functionalities designated 140 or 180.
- the converter is configured to convert the first portion 101, as shown at 140, and to convert the second portion 102, as shown at 180.
- the converter converts the first portion 101 into one or more harmonic components 141 having a first order
- the converter 180 converts the second portion 102 into one or more harmonic components 182 having a second order.
- the first order that is, the order underlying the harmonic components 141 is higher than the second order, which means, in other words, that the higher order converter 140 outputs more harmonic components 141 than the lower order converter 180.
- the order n t by which the converter 140 is driven is thus greater than the order n 2 , with which the converter 180 is driven.
- the converters 140, 180 may be controllable converters. Alternatively, however, the order may be fixed and thus permanently programmed so that the inputs designated ni and n 2 are not present in this embodiment.
- Fig. 1b shows an apparatus for decompressing compressed sound field data 20 having first harmonic components of a first order and one or more second harmonic components of a second order, such as those outputted from Fig. 1a at 141, 182.
- the decompressed sound field data need not necessarily be "raw" harmonic components 141, 142.
- a lossless entropy coder such as a Huffman coder or an arithmetic coder, could be provided to reduce the number of
- the data stream 20 fed to an input interface 200 would consist of entropy-coded harmonic components and, if necessary, page information, as further described with reference to FIGS
- a respective entropy decoder would be provided at the output of the input interface 200, which is adapted to the entropy encoder on the encoder side, that is to say with reference to Fig.
- Both groups of harmonic components are fed to a decoder / combiner 240.
- the block 240 is configured to decompress the compressed sound field data 201, 202 using a combination of the first portion and the second portion and using a translation from a harmonic component representation into a time domain representation, and finally to provide the decompressed representation of the sound field as shown at 240.
- the decoder 240 which may be formed, for example, as a signal processor, is thus designed, on the one hand, to implement a conversion into the time domain from the spherical-harmonic component region and, on the other hand, to perform a combination. However, the order between conversion and combination may be different, as illustrated with respect to Figs. 1d, 1e, or 2c, 2d for different examples.
- FIG. 1 c shows a device for compressing sound field data of a region according to an exemplary embodiment, in which the splitter 100 is designed as a temporal splitter 100 a.
- the scheduler 100a is one implementation splitter 100 of Fig. 1a is arranged to divide the sound field data into a first portion comprising first reflections in the area and a second portion comprising second reflections in the area, the second reflections being later in time when the first reflections occur.
- the first portion 101 output from the block 100a thus represents the impulse response portion 310 of Fig. 4
- the second fraction represents the portion 320 of the impulse response of Fig. 4.
- the timing of the division may be, for example, 100 ms.
- the division is preferably made where the discrete reflections change into diffuse reflections. This can be a different time, depending on the room, and concepts exist to create the best layout here.
- the division into an early and a late portion can also be carried out depending on an available data rate, in such a way that the distribution time is made smaller and smaller the less bit rate there is. This is favorable in terms of the bit rate, because then as large a proportion of the impulse response with a low order is converted into the harmonic component range.
- the converter which is represented by blocks 140 and 180 in FIG. 1c, is thus designed to convert the first component 101 and the second component 102 into harmonic components, wherein the converter in particular converts the second component into one or more harmonic components Converts components 182 with a second order and converts the first portion 101 into harmonic components 141 having a first order, the first order being higher than the second order, to finally obtain the compressed sound field received from an output interface 190 finally for purposes of Transmission and / or storage can be issued.
- Fig. 1d shows an implementation of the decompressor for the example of the time distribution.
- the decompressor is configured to perform the compressed sound field data using a combination of the first portion 201 with the first reflections and the second portion 202 with the late reflections and a conversion from the harmonic component range to the time domain.
- Fig. 1d shows an implementation in which the combination takes place after the conversion.
- Fig. 1e shows an alternative implementation in which the combination takes place before the conversion.
- the converter 241 is configured to convert harmonic components of the high order into the time domain while the converter 242 is configured to form the harmonic components of the low order to implement in the time domain.
- the output of converter 241 provides something corresponding to region 210, while converter 242 provides something corresponding to region 320, but due to the lossy compression, the portions at the output of bridge 241, 242 do not identical to the sections 310, 320 are.
- an at least perceptual similarity or identity of the portion will exist at the output of block 241 to portion 310 of Figure 4, while the portion at the output of block 242 corresponding to the late portion 320 of the impulse response will have significant differences and thus only approximates the course of the impulse response.
- these deviations are not critical for human directional perception, because human directional perception is hardly or not based on the late component or diffuse reflections of the impulse response anyway.
- FIG. 1 e shows an alternative implementation in which the decoder first comprises the combiner 245 and then the converter 244.
- the individual harmonic components are added together in the exemplary embodiment shown in FIG. 1e, whereupon the result of the addition is converted in order finally to obtain a time domain representation.
- a combination will not consist in an addition, but in a serialization, in that the output of the block 241 will be located earlier in a decompressed impulse response than the output of the block 242 again to receive an impulse response corresponding to Fig. 4, which can then be used for further purposes, such as an auralization so in a preparation of sound signals with the desired spatial impression.
- Fig. 2a shows an alternative implementation of the present invention in which a division in the frequency domain is made.
- the splitter 100 of FIG. 1a is implemented as a filter bank in the embodiment of FIG. 2a in order to filter at least part of the sound field data in order to obtain sound field data in different filter bank channels 101, 102.
- the filter bank in one embodiment in which the timing of FIG. 1a is not implemented, receives both the early and late portions, while in an alternative embodiment, only the early portion of the sound field data is fed to the filter bank, while the latter Part is no longer spectrally decomposed.
- Subordinate to the analysis filter bank 100b is the converter, which may be formed of sub-converters 140a, 140b, 10c.
- the converter 140a, 140b, 140c is configured to convert the sound field data in different filter bank channels using different orders for different filter bank channels in order to obtain one or more harmonic components for each filter bank channel.
- the converter is configured to perform a first-order conversion for a first filterbank channel having a first center frequency, and to perform a second-order conversion for a second filterbank channel having a second center frequency, the first order being higher than the second order and wherein the first center frequency, ie, f n , is higher than the second center frequency to finally obtain the compressed sound field representation.
- a lower order may be used for the lowest frequency band than for a middle frequency band.
- the highest frequency band does not necessarily have to be converted with a higher order than, for example, a middle channel.
- the highest order may be used in the areas where the direction perception is highest, as in the other areas, which may include, for example, a certain high frequency range, the order is lower, because in these areas also the directional Perception of human hearing is lower.
- Fig. 2b shows a more detailed implementation of the analysis filter bank 100b.
- this comprises a bandpass filter and furthermore has downstream decimators 100c for each filterbank channel.
- each decimator can be decimated by a factor of 1/64, so that the total number of digital samples at the output of the decimators over all channels adds up to the number of samples of one Blocks of sound field data in the time domain, which has been decomposed by the filter bank.
- An exemplary filter bank may be a real or complex QMF filter bank.
- Each subband signal preferably the early portions of the impulse responses, is then converted to harmonic components by means of the converters 140a to 140c in analogy to FIG. 2a to finally obtain a description for cylindrical or preferably spherical harmonic components for various subband signals of the sound field description different subband signals have different orders, that is, a different number of harmonic components.
- FIG. 2 c and FIG. 2 d again show different implementations of the decompressor, as shown in FIG. 1 b, ie a different sequence of the combination and subsequent conversion in FIG. 2 c or the implementation carried out initially and the subsequent combination as described in FIG Fig. 2d is shown.
- 1 b again comprises a combiner 245 which performs an addition of the different harmonic components from the various subbands in order then to obtain an overall representation of the harmonic components. which are then converted with the converter 244 in the time domain.
- the inputs to combiner 245 are in the harmonic component spectral region, while the output of combiner 345 is a harmonic component region representation, which is then converted by converter 244 to the time domain.
- the individual harmonic components for each subband are first converted into the spectral domain by different converters 241a, 241b, 241c, so that the output signals of the blocks 241a, 241b, 241c are the output signals of the blocks 140a , 140b, 140c of Fig. 2a or Fig.
- synthesis filter bank which in the case of a downsampling on the encoder side (block 100c of FIG. 2b) can also have a high-touch function, that is to say an upsampling function.
- the synthesis filter bank then represents the combiner function of the decoder 240 of FIG. 1b. At the output of the synthesis filter bank there is thus the decompressed sound field representation that can be used for auralization, as will be shown.
- Fig. 1f shows an example of the decomposition of impulse responses into harmonic components of different orders.
- the late sections are not spectrally decomposed but implemented in total with the zeroth order.
- the early sections of the impulse responses are spectrally decomposed. For example, the lowest band is processed with the first order, while the next band is already processed with the fifth order and the last band, because it is most important for direction / space perception, with the highest order, in this example with order 14, is processed.
- Figure 3a shows the entire encoder / decoder scheme or the entire compressor / decompressor scheme of the present invention.
- the compressor includes not only the functionalities of Fig. 1a, denoted by 1 or PENC, but also a decoder PDEC2, which may be formed as in Fig. 1b.
- the compressor also comprises a controller CTRL4, which is designed to compare decompressed sound field data obtained by the decoder 2 with original sound field data taking into account a psychoacoustic model, such as the model PEAQ, which has been standardized by the ITU.
- the controller 4 generates optimized parameters for the division, such as the time distribution, the frequency distribution in the filter bank or optimized parameters for the orders in the individual converters for the different parts of the sound field data, if these converters are designed to be controllable.
- Control parameters such as split information, filter bank parameters, or orders, may then be transmitted along with a bitstream having the harmonic components to a decoder, represented at 2 in Figure 3a.
- the compressor 11 thus consists of the control block CTRL4 for the codec control as well as a parameter encoder PENC1 and the parameter decoder PDEC2.
- the inputs 10 are data from microphone array measurements.
- the control block 4 initializes the encoder 1 and provides all the parameters for encoding the array data.
- PENC block 1 the data is processed in the time and frequency domain in accordance with the described methodology of the sub-band division and provided for data transmission.
- Fig. 3b shows the scheme of data en- and decoding.
- the input data 10 are first decomposed by the splitter 100a into an early 101 and a late sound field 102.
- This spherical harmonic decomposition is a preferred embodiment, but can be performed by any sound field decomposition that generates harmonic components.
- the frequency range in the reconstruction block 245, which is also referred to as a combiner is reconstructed and recombined with the late sound field in the further combiner 243 after it has been computed with a low-order listener fit.
- the control block CTRL4 of Fig. 3a includes a room acoustic analysis module and a psychoacoustic module.
- the control block analyzes both the input data 10 and the output data of the decoder 2 of FIG. 3a in order to provide the coding parameters, which are also referred to as page information 300 in FIG. 3a, or which are provided directly in the compressor 11 to the encoder PENC1. adaptively adapt.
- room acoustic parameters are extracted which specify the initial parameters of the coding with the parameters of the array configuration used. These include both the time of separation between early and late sound field, also referred to as "mixing time", and the parameters for the filter bank, such as corresponding orders of spherical harmonics.
- the output which may be in the form of binaural impulse responses, for example, as output from combiner 243, is fed into a psychoacoustic module with an auditory model that evaluates the quality and adjusts the encoding parameters accordingly.
- the concept can also work with static parameters.
- the invention is advantageous in that data and computational complexity in the processing and transmission of circular and Kugelarray poems be reduced depending on the human ear. It is also advantageous that the data thus processed can be integrated into existing compression methods and thus allow additional data reduction. This is advantageous in band-limited transmission systems, such as for mobile terminals.
- the present invention can be used in many fields, and in particular in the areas where the acoustic sound field is represented by means of cylindrical or spherical harmonics. This is done, for example, in sound field analysis by means of circular or spherical arrays. If the analyzed sound field is to be auralized, the concept of the present invention can be used. In devices for simulating rooms, the databases used to store existing spaces. Here, the inventive concept allows a space-saving and high-quality storage. There are reproduction methods based on spherical surface functions, such as Higher Order Ambisonics or binaural synthesis.
- the present invention provides a reduction of computation time and data overhead. This can be of particular advantage, in particular with regard to data transmission, for example in the case of teleconferencing systems.
- FIG 5 shows an implementation of a converter 140 or 180 with adjustable order or with at least different order, which can also be fixed.
- the converter includes a time-frequency transform block 502 and a downstream space transform block 504.
- the space transform block 504 is configured to operate in accordance with computation rule 508.
- n is the order.
- the calculation rule 508 is solved only once, if the order is zero, depending on the order, or is solved more often if the order is e.g. to order 5 or in the example described above to order 14.
- the time-frequency transform element 502 is configured to transform the impulse responses on the input lines 101, 102 into the frequency domain, preferably employing the fast Fourier transform. Furthermore, then only the half-page spectrum is forwarded to reduce the computational effort.
- a spatial Fourier transform is performed in the block space transformation 504 as described in the Fourier Acoustics, Sound Radiation and Nearfield Acoustical Holography, Academic Press, 1999 by Earl G. Williams.
- the spatial transformation 504 is optimized for sound field analysis while providing high numerical accuracy and fast computation speed.
- FIG. 6 shows the preferred implementation of a harmonic component range converter into the time domain, where as an alternative a planar wave processor and beamforming 602 is illustrated as an alternative to an inverse space transform implementation 604.
- the outputs of both blocks 602, 604 may alternatively be fed to a block 606 for generating impulse responses.
- Inverse space transformation 604 is configured to undo the Hin transformation in block 504.
- the decomposition into plane waves and the beamforming in block 606 results in a large amount of decomposition direction. can be processed uniformly, which is favorable for rapid processing in particular for visualization or auralization.
- the block 602 receives radial filter coefficients and, depending on the implementation, additional beamforming or beamforming coefficients. These can either have a constant orientation or be frequency-dependent.
- Alternative inputs to block 602 may be modal radial filters, and in particular spherical arrays or different configurations, such as an omnidirectional open-sphere microphone, an open sphere with cardioid microphones, and a rigid sphere with omnidirectional microphones.
- the impulse response generation block 606 generates impulse responses or time domain signals from either block 602 or block 604. This block recombines the previously omitted negative portions of the spectrum, performs a fast inverse Fourier transform, and allows resampling. Sample rate conversion to the original sample rate if the input signal was down-sampled at one location. Furthermore, a window option can be used.
- the block 606 may further be configured to output the complete set of decompressed impulse responses, such as the lossy impulse responses, in which case the block 608 would again output, for example, 350 impulse responses.
- it is preferred to output only the impulse responses ultimately required for rendering which may be accomplished by a block 608 that provides selection or interpolation for a particular rendering scenario. For example, if stereo reproduction is desired, as shown in block 616, then depending on the placement of the two stereo speakers, the impulse response selected from the 350 retrieved impulse responses, for example, corresponds to the spatial direction of the corresponding stereo speaker. With this impulse response, a prefilter of the corresponding loudspeaker is then set, such that the prefilter has a filter characteristic which corresponds to this impulse response.
- an audio signal to be reproduced is fed to the two speakers via the corresponding pre-filters and reproduced to finally produce the desired spatial impression for a stereo auralization. If among the available impulse responses there is not an impulse response in a particular direction in which a loudspeaker is located in the actual rendering scenario, preferably the two or three nearest impulse impulses are used and interpolation is performed.
- the reflections of a source in wave field synthesis playback 612 are represented by four impulse responses at certain positions for the early reflections and eight impulse responses at certain positions for the late reflections.
- Selection block 608 then selected the 12 impulse responses for the 12 virtual positions. Subsequently, these impulse responses are fed along with the associated positions in a wavefield synthesis renderer, which may be located at block 612, and the wavefield synthesis renderer then uses these impulse responses to compute the loudspeaker signals for the speakers actually present to make them then map the corresponding virtual sources. This calculates a separate pre-filter for each loudspeaker in the Wave Field Synthesis Playback System, which is then used to filter an audio signal to be ultimately reproduced before it is output from the loudspeaker to achieve adequate high quality spatial effects.
- An alternative implementation of the present invention is a generation of a headphone signal, ie in a binaural application, in which the spatial impression of the area is to be generated via the headphone reproduction.
- any other sound field data for example sound field data by magnitude and vector, ie also with respect to eg sound pressure and sound velocity at certain positions in space, can also be used. These sound field data can also make more important and less important contributions to the human cattle tion perception and implemented into harmonic components.
- the sound field data may also include any type of impulse responses, such as Head Elated Transfer Functions (HRTF) functions or Binaural Room Impulse Responses (BRIR) functions or impulse responses, each one discrete point to a predetermined position in the area.
- HRTF Head Elated Transfer Functions
- BRIR Binaural Room Impulse Responses
- a space is scanned with a ball array.
- the sound field is a set of impulse responses.
- the sound field is split into its early and late parts.
- both parts are decomposed into their spherical or cylindrical harmonic components. Since the relative direction information is present in the early sound field, a higher order of the spherical harmonic is calculated here than in the late sound field, which is sufficient for a low order.
- the early part is relatively short, for example 100 ms, and is represented accurately, that is, with many harmonic components, while the late part is, for example, 100 ms to 2 s or 10 s long. However, this late part is represented with less or only a single harmonic component.
- a further data reduction results from the splitting of the early sound field into individual bands before the representation as spherical harmonics.
- the early sound field is decomposed into its spectral components by means of a filter bank.
- a data reduction is achieved which significantly accelerates the calculation of the harmonic components.
- a sufficiently early order is used perceptually as a function of human directional perception.
- aspects have been described in the context of a device, it should be understood that these aspects also constitute a description of the corresponding method such that a block or device of a device may also be described as a corresponding method step or feature of a method step. is standing. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device.
- Some or all of the method steps may be performed by a hardware device (or using a Hardware apparatus), such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some or more of the most important method steps may be performed by such an apparatus.
- embodiments of the invention may be implemented in hardware or in software.
- the implementation may be performed using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or FLASH memory, a hard disk, or other magnetics Viewing or optical storage are carried on the electronically readable control signals are stored, which can cooperate with a programmable computer system or cooperate such that the respective method is performed. Therefore, the digital storage medium can be computer readable.
- some embodiments according to the invention include a data carrier having electronically readable control signals capable of interacting with a programmable computer system such that one of the methods described herein is performed.
- embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is operable to perform one of the methods when the computer program product runs on a computer.
- the program code can also be stored, for example, on a machine-readable carrier.
- an embodiment of the method according to the invention is thus a computer program which has a program code for performing one of the methods described herein when the computer program runs on a computer.
- a further embodiment of the inventive method is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program is recorded for carrying out one of the methods described herein.
- a further embodiment of the method according to the invention is thus a data stream or a sequence of signals, which represent the computer program for performing one of the methods described herein.
- the data stream or the sequence of signals may be configured, for example, to be transferred via a data communication connection, for example via the Internet.
- Another embodiment includes a processing device, such as a computer or programmable logic device, configured or adapted to perform any of the methods described herein.
- a processing device such as a computer or programmable logic device, configured or adapted to perform any of the methods described herein.
- Another embodiment includes a computer on which the computer program is installed to perform one of the methods described herein.
- Another embodiment according to the invention comprises a device or system adapted to transmit a computer program for performing at least one of the methods described herein to a receiver.
- the transmission can be done for example electronically or optically.
- the receiver may be, for example, a computer, a mobile device, a storage device or a similar device.
- the device or system may include a file server for transmitting the computer program to the recipient.
- a programmable logic device eg, a field programmable gate array, an FPGA
- a field programmable gate array may include a Microprocessor cooperate to perform any of the methods described herein.
- the methods are performed by any hardware device. This may be a universal hardware such as a computer processor (CPU) or hardware specific to the process, such as an ASIC.
Abstract
Description
Claims
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE201310223201 DE102013223201B3 (en) | 2013-11-14 | 2013-11-14 | Method and device for compressing and decompressing sound field data of a region |
PCT/EP2014/073808 WO2015071148A1 (en) | 2013-11-14 | 2014-11-05 | Method and device for compressing and decompressing sound field data of an area |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3069530A1 true EP3069530A1 (en) | 2016-09-21 |
EP3069530B1 EP3069530B1 (en) | 2019-02-20 |
Family
ID=51846694
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14793183.6A Not-in-force EP3069530B1 (en) | 2013-11-14 | 2014-11-05 | Method and device for compressing and decompressing sound field data of an area |
Country Status (6)
Country | Link |
---|---|
US (1) | US20160255452A1 (en) |
EP (1) | EP3069530B1 (en) |
JP (1) | JP6329629B2 (en) |
CN (1) | CN105766002B (en) |
DE (1) | DE102013223201B3 (en) |
WO (1) | WO2015071148A1 (en) |
Families Citing this family (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP2960903A1 (en) | 2014-06-27 | 2015-12-30 | Thomson Licensing | Method and apparatus for determining for the compression of an HOA data frame representation a lowest integer number of bits required for representing non-differential gain values |
BR112018013526A2 (en) * | 2016-01-08 | 2018-12-04 | Sony Corporation | apparatus and method for audio processing, and, program |
DE102016125886B4 (en) * | 2016-12-29 | 2019-08-29 | Symonics GmbH | Apparatus and method for efficient calculation of auralization |
US10614788B2 (en) * | 2017-03-15 | 2020-04-07 | Synaptics Incorporated | Two channel headset-based own voice enhancement |
US10764684B1 (en) * | 2017-09-29 | 2020-09-01 | Katherine A. Franco | Binaural audio using an arbitrarily shaped microphone array |
EP3525482B1 (en) | 2018-02-09 | 2023-07-12 | Dolby Laboratories Licensing Corporation | Microphone array for capturing audio sound field |
EP3547305B1 (en) * | 2018-03-28 | 2023-06-14 | Fundació Eurecat | Reverberation technique for audio 3d |
EP3782152A2 (en) * | 2018-04-16 | 2021-02-24 | Dolby Laboratories Licensing Corporation | Methods, apparatus and systems for encoding and decoding of directional sound sources |
GB2574873A (en) * | 2018-06-21 | 2019-12-25 | Nokia Technologies Oy | Determination of spatial audio parameter encoding and associated decoding |
GB201818959D0 (en) * | 2018-11-21 | 2019-01-09 | Nokia Technologies Oy | Ambience audio representation and associated rendering |
EP3683794B1 (en) * | 2019-01-15 | 2021-07-28 | Nokia Technologies Oy | Audio processing |
CN110265042B (en) * | 2019-05-31 | 2021-07-23 | 歌尔科技有限公司 | Sound signal processing method, device and equipment |
WO2021154211A1 (en) * | 2020-01-28 | 2021-08-05 | Hewlett-Packard Development Company, L.P. | Multi-channel decomposition and harmonic synthesis |
Family Cites Families (25)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS54149684A (en) * | 1978-05-15 | 1979-11-24 | Sanyo Electric Co Ltd | Reverberation meter |
JP3295139B2 (en) * | 1992-09-28 | 2002-06-24 | 日本放送協会 | Reverberation device |
US5440639A (en) * | 1992-10-14 | 1995-08-08 | Yamaha Corporation | Sound localization control apparatus |
JP2002510921A (en) * | 1998-03-31 | 2002-04-09 | レイク テクノロジー リミティド | Formulation of complex room impulse response from 3D audio information |
US6016473A (en) * | 1998-04-07 | 2000-01-18 | Dolby; Ray M. | Low bit-rate spatial coding method and system |
FR2851879A1 (en) * | 2003-02-27 | 2004-09-03 | France Telecom | PROCESS FOR PROCESSING COMPRESSED SOUND DATA FOR SPATIALIZATION. |
KR101215868B1 (en) * | 2004-11-30 | 2012-12-31 | 에이저 시스템즈 엘엘시 | A method for encoding and decoding audio channels, and an apparatus for encoding and decoding audio channels |
TWI396188B (en) * | 2005-08-02 | 2013-05-11 | Dolby Lab Licensing Corp | Controlling spatial audio coding parameters as a function of auditory events |
US8379868B2 (en) * | 2006-05-17 | 2013-02-19 | Creative Technology Ltd | Spatial audio coding based on universal spatial cues |
DE102006050068B4 (en) * | 2006-10-24 | 2010-11-11 | Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. | Apparatus and method for generating an environmental signal from an audio signal, apparatus and method for deriving a multi-channel audio signal from an audio signal and computer program |
JP4277234B2 (en) * | 2007-03-13 | 2009-06-10 | ソニー株式会社 | Data restoration apparatus, data restoration method, and data restoration program |
CN103561378B (en) * | 2008-07-31 | 2015-12-23 | 弗劳恩霍夫应用研究促进协会 | The signal of binaural signal generates |
US9703756B2 (en) * | 2008-09-05 | 2017-07-11 | Adobe Systems Incorporated | Method and apparatus for converting spherical harmonics representations of functions into multi-resolution representations |
EP2205007B1 (en) * | 2008-12-30 | 2019-01-09 | Dolby International AB | Method and apparatus for three-dimensional acoustic field encoding and optimal reconstruction |
JP5168208B2 (en) * | 2009-03-30 | 2013-03-21 | ヤマハ株式会社 | Audio signal processing device and speaker device |
KR101613684B1 (en) * | 2009-12-09 | 2016-04-19 | 삼성전자주식회사 | Apparatus for enhancing bass band signal and method thereof |
US9047876B2 (en) * | 2010-03-30 | 2015-06-02 | Panasonic Intellectual Property Managment Co., Ltd. | Audio device |
EP2375779A3 (en) * | 2010-03-31 | 2012-01-18 | Fraunhofer-Gesellschaft zur Förderung der Angewandten Forschung e.V. | Apparatus and method for measuring a plurality of loudspeakers and microphone array |
JP4886881B2 (en) * | 2010-06-30 | 2012-02-29 | 株式会社東芝 | Acoustic correction device, acoustic output device, and acoustic correction method |
EP2609759B1 (en) * | 2010-08-27 | 2022-05-18 | Sennheiser Electronic GmbH & Co. KG | Method and device for enhanced sound field reproduction of spatially encoded audio input signals |
EP2451196A1 (en) * | 2010-11-05 | 2012-05-09 | Thomson Licensing | Method and apparatus for generating and for decoding sound field data including ambisonics sound field data of an order higher than three |
EP2592845A1 (en) * | 2011-11-11 | 2013-05-15 | Thomson Licensing | Method and Apparatus for processing signals of a spherical microphone array on a rigid sphere used for generating an Ambisonics representation of the sound field |
EP2782094A1 (en) * | 2013-03-22 | 2014-09-24 | Thomson Licensing | Method and apparatus for enhancing directivity of a 1st order Ambisonics signal |
US9674632B2 (en) * | 2013-05-29 | 2017-06-06 | Qualcomm Incorporated | Filtering with binaural room impulse responses |
KR101815082B1 (en) * | 2013-09-17 | 2018-01-04 | 주식회사 윌러스표준기술연구소 | Method and apparatus for processing multimedia signals |
-
2013
- 2013-11-14 DE DE201310223201 patent/DE102013223201B3/en not_active Expired - Fee Related
-
2014
- 2014-11-05 JP JP2016530874A patent/JP6329629B2/en not_active Expired - Fee Related
- 2014-11-05 EP EP14793183.6A patent/EP3069530B1/en not_active Not-in-force
- 2014-11-05 CN CN201480061929.3A patent/CN105766002B/en not_active Expired - Fee Related
- 2014-11-05 WO PCT/EP2014/073808 patent/WO2015071148A1/en active Application Filing
-
2016
- 2016-05-13 US US15/154,189 patent/US20160255452A1/en not_active Abandoned
Also Published As
Publication number | Publication date |
---|---|
WO2015071148A1 (en) | 2015-05-21 |
US20160255452A1 (en) | 2016-09-01 |
EP3069530B1 (en) | 2019-02-20 |
CN105766002A (en) | 2016-07-13 |
JP2017500782A (en) | 2017-01-05 |
CN105766002B (en) | 2018-04-20 |
DE102013223201B3 (en) | 2015-05-13 |
JP6329629B2 (en) | 2018-05-23 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3069530B1 (en) | Method and device for compressing and decompressing sound field data of an area | |
US11096000B2 (en) | Method and apparatus for processing multimedia signals | |
US10692508B2 (en) | Method for generating filter for audio signal and parameterizing device therefor | |
EP1854334B1 (en) | Device and method for generating an encoded stereo signal of an audio piece or audio data stream | |
DE60311794T2 (en) | SIGNAL SYNTHESIS | |
KR101456640B1 (en) | An Apparatus for Determining a Spatial Output Multi-Channel Audio Signal | |
DE602004001868T2 (en) | METHOD FOR PROCESSING COMPRESSED AUDIO DATA FOR SPATIAL PLAYBACK | |
EP1687809A1 (en) | Device and method for reconstruction a multichannel audio signal and for generating a parameter data record therefor | |
WO2015036271A2 (en) | Device and method for the decorrelation of loudspeaker signals | |
DE112015003108T5 (en) | Operation of the multi-channel audio signal systems | |
EP1471770B1 (en) | Method for generating an approximated partial transfer function | |
EP2357854B1 (en) | Method and device for generating individually adjustable binaural audio signals | |
DE102011003450A1 (en) | Generation of user-adapted signal processing parameters | |
EP2503799B1 (en) | Method and system for calculating synthetic head related transfer functions by means of virtual local sound field synthesis | |
DE102023209048A1 (en) | METHOD AND SYSTEM FOR SHIPPING VOLUME ADJUSTMENTS OF AUDIO COMPONENTS | |
DE102017121876A1 (en) | METHOD AND DEVICE FOR FORMATTING A MULTI-CHANNEL AUDIO SIGNAL |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20160512 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: SLADECZEK, CHRISTOPH Inventor name: NOWAK, JOHANNES |
|
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: EXAMINATION IS IN PROGRESS |
|
REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1228151 Country of ref document: HK |
|
17Q | First examination report despatched |
Effective date: 20171004 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: GRANT OF PATENT IS INTENDED |
|
INTG | Intention to grant announced |
Effective date: 20180820 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE PATENT HAS BEEN GRANTED |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R108 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1099813 Country of ref document: AT Kind code of ref document: T Effective date: 20190315 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20190220 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190520 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190620 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190521 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190620 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190520 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20191121 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20191121 Year of fee payment: 6 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20191126 Year of fee payment: 6 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191130 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191130 Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191105 |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20191130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191105 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191130 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 1099813 Country of ref document: AT Kind code of ref document: T Effective date: 20191105 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20191105 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
GBPC | Gb: european patent ceased through non-payment of renewal fee |
Effective date: 20201105 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20141105 Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: FR Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20201130 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: GB Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20201105 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190220 |