EP3005732A1 - Device and method for spatially selective audio playback - Google Patents
Device and method for spatially selective audio playbackInfo
- Publication number
- EP3005732A1 EP3005732A1 EP14727481.5A EP14727481A EP3005732A1 EP 3005732 A1 EP3005732 A1 EP 3005732A1 EP 14727481 A EP14727481 A EP 14727481A EP 3005732 A1 EP3005732 A1 EP 3005732A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- audio signal
- output
- audio
- masking threshold
- version
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims description 31
- 230000005236 sound signal Effects 0.000 claims abstract description 147
- 230000000873 masking effect Effects 0.000 claims abstract description 67
- 238000004590 computer program Methods 0.000 claims description 12
- 230000008859 change Effects 0.000 claims description 11
- 238000007493 shaping process Methods 0.000 claims description 10
- 238000000926 separation method Methods 0.000 abstract description 11
- 230000002452 interceptive effect Effects 0.000 abstract description 7
- 238000012545 processing Methods 0.000 description 20
- 230000009467 reduction Effects 0.000 description 10
- 238000012986 modification Methods 0.000 description 9
- 230000004048 modification Effects 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000000694 effects Effects 0.000 description 5
- 238000001914 filtration Methods 0.000 description 5
- 230000001629 suppression Effects 0.000 description 5
- 230000001934 delay Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004088 simulation Methods 0.000 description 4
- 230000001419 dependent effect Effects 0.000 description 3
- 238000005562 fading Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000005855 radiation Effects 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 230000006978 adaptation Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 2
- 230000006866 deterioration Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000000527 sonication Methods 0.000 description 2
- 230000002123 temporal effect Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000006399 behavior Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000009499 grossing Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 230000003362 replicative effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R1/00—Details of transducers, loudspeakers or microphones
- H04R1/20—Arrangements for obtaining desired frequency or directional characteristics
- H04R1/32—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only
- H04R1/323—Arrangements for obtaining desired frequency or directional characteristics for obtaining desired directional characteristic only for loudspeakers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R27/00—Public address systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/02—Spatial or constructional arrangements of loudspeakers
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2203/00—Details of circuits for transducers, loudspeakers or microphones covered by H04R3/00 but not provided for in any of its subgroups
- H04R2203/12—Beamforming aspects for stereophonic sound reproduction with loudspeaker arrays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2227/00—Details of public address [PA] systems covered by H04R27/00 but not provided for in any of its subgroups
- H04R2227/001—Adaptation of signal processing in PA systems in dependence of presence of noise
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/01—Aspects of volume control, not necessarily automatic, in sound systems
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2430/00—Signal processing covered by H04R, not provided for in its groups
- H04R2430/20—Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic
Definitions
- the present invention relates to a space-selective audio reproduction, e.g. of different audio signals to different listeners or listener groups located in different locations of one play.
- the reproduction of audio signals over several speakers, typically organized as an array, is a common practice.
- an individual modification such as, the application of a delay and a change in the amplitude, generally also described with filtering
- the shape of the sound field radiated by a loudspeaker can be purposefully influenced, such as e.g. for the purpose of deliberately sounding certain areas.
- beamforming This technique also allows multiple audio signals to be played simultaneously with different directional characteristics by producing individual filtered speaker signals for all signals, which are summed up loudly before playback.
- the object of the present invention is to provide such a concept.
- the core idea of the present invention is to have recognized that a better separation of a first audio signal in a first area of a PA area of a plurality of loudspeakers can be achieved by the version of the first area resulting from the space-selective reproduction of the audio signals Calculating a masking threshold depending on the version of the audio signal to be separated from the one or more other audio signals in that area and depending on a comparison of the masking threshold with the version of the one or more other, ie interfering , Audio signals, the output of the audio signals for space-selective playback is influenced to the outputs of the plurality of speakers.
- the calculation or estimation of the audio signals in this first area can also be illustrated as a simulation of the sound propagation in this first area and thus the element for its execution as a calculator or simulator.
- the separation of the audio signals to the first area of the PA area already enabled by the room-selective reproduction can thus be improved by evaluating the masking threshold by calculating or simulating the versions of the audio signals resulting from the room-selective reproduction.
- the influencing of the space-selective reproduction for avoiding or reducing the "violation" of the masking at the first area of the public address area can be carried out in different ways, such as by frequency-selective reduction of the respective disturbance. other audio signal in frequency ranges at which the respective simulated other audio signal exceeds the masking threshold.
- Fig. 1 shows a block diagram of a device for space-selective reproduction
- Fig. 2 shows a sketch for illustrating possible measures of the adapter of Fig. 1;
- Fig. 3 illustrates a sketch for illustrating an additional or alternative measure of the Fig. 1 adapter
- Fig. 4 is a block diagram of a conventional space selective reproduction apparatus.
- Fig. 5 shows a block diagram of an implementation variant of the embodiment of Fig. 1 with starting point.
- the device 10 comprises an input 12 for at least a first audio signal 14T and a second audio signal 14 2 and an output 16 for a plurality of loudspeakers 18.
- a beam forming processor 20 of the device 10 is connected between the input 12 on the one hand and the output 16 on the other hand and adapted to output the first and second audio signals 14i and 14 2 for space-selective reproduction via the output 16 to the loudspeaker 18.
- the loudspeakers 18 are capable of sonicating a PA area 22, such as an area surrounded or directed by the loudspeakers at their intended speaker locations, or generally an area which is sonicated by at least one of the loudspeakers 18 ,
- the sonication area may be a fictitious space relative to the configurations of fictitious speaker positions of the speakers 18, such as a virtual one PA area without reflective surfaces, or around a real PA area, which reflection effects, such as on walls or the like, may have.
- Space-selective reproduction of the audio signals 14 and 14 2 on the loudspeakers 18 is intended to mean that the audio signals are not simply output to the loudspeakers 18 in the form of identical copies in superimposed form, but that, as described in the description of the present application, by means of, for example, speaker-specific delays and / or amplitude modifications, or generally filtered by means of a loudspeaker-individual filtering, via the loudspeakers 18 and indeed differently for the loudspeakers and 14 2 so that there is at least a first area 24 of the PA area which is less or not irradiated by the second audio signal 14 2 compared to the first audio signal 14 1 .
- the first audio signal 14 ! sonicates this area 26 less or not at all compared to the second audio signal 14 2 due to the space-selective reproduction via the speakers 18.
- the juxtaposition of more than two superimposed reproduced audio signals is possible. Under optimal conditions, it could be that the separation of the first audio signal 14-t on the first area 24 from the other audio signal 14 2 goes so far that a listener in this area 24 does not hear the other audio signal 14 2 .
- room selectivity over the reproduction by the loudspeakers is limited, which may arise from reflections that occur in reality or simply from a limited overall extension of the distribution of the positions of the loudspeakers.
- the other elements included in the device 10 are to improve "room selectivity in this sense. The details will be discussed below.
- the audio signals 14- and 14 2 can be present at the input 12 in any form, such as analog or digital, in separate or in m / s-coded form or in a parameterized downmix-comprising form
- any form such as analog or digital
- Loudspeaker individual loudspeaker signals for the loudspeakers 18 can be output via the output 16 separately, in analogue or digital form, compressed or uncompressed, already reinforced, merely pre-reinforced or unamplified
- the loudspeaker signals it would be possible for the loudspeaker signals to be output in compressed form in a downmix together with spatial cues parameters, such as in MPEG surround or SAOC encoded form.
- the beamforming processor 20 initially processes the incoming audio signals 14 and 14 2 completely separate from each other to produce a set of loudspeaker signals for the loudspeakers 18 to each of them so that each loudspeaker signal for the respective audio signal has a particular loudspeaker position for the respective loudspeaker position individual speaker, such as delay and / or amplitude modification has experienced. Only at the end, for example, the thus obtained from the individual speaker signals loudspeaker signal sets per channel or speakers are superimposed with each other. This is also illustrated once more in the following figures.
- the term "space selectivity" is intended to be natural also broad enough to refer to merely "angle selectivity" in the sense that the audio signal individual processing within the beamforming processor 20 results in the audio signals 14 ! and 14 2 are emitted in different solid angle ranges as viewed from the loudspeakers 18.
- Such an angular selectivity can be interpreted as influencing the radiation in the far field of the loudspeaker setup.
- a small distance to the speaker setup in relation to the size of the speaker setup, ie in the geometric near field
- beamforming processor 20 may be fixed or optimized for space selective playback.
- the spatial selectivity of the reproduction of the beamforming processor 20 may be constant. It may be optimized in advance to the area 24 or the areas 24 and 26, ie to the effect that in the area 24 only the first audio signal 14, and, if provided, in the area 26 only the second audio signal 14 2 from a listener in the respective Area is audible.
- the optimization then defines the aforementioned delays, amplitude modifications and / or filters, such as FIR filters, for the individual channels or speakers 18, and the beamforming processor 20 may be hard-wired, for example, or fixed in software or programmable Hardware implemented to accomplish the space-selective playback via the output 16 to the speakers 18.
- the beamforming processor is also adjustable in terms of speaker-individual processing (delay, amplitude modulation or filtering) for one or more of the audio signals 1 1, 14 2 .
- the beamforming processor 20 can be adjusted or influenced at the output 16 with regard to its space-selective reproduction of the audio signals 14, 14 2 , as will be described in more detail below.
- this setting can also be achieved by audio-signal-specific, frequency-selective modification / influencing of individual or all audio signals that acts on all loudspeakers / channels, as will also be described below. It is the aforementioned controllability of the beamforming processor 20 that uses the components of the device 10 described below to enhance the separation of the first audio signal 14 in the region 24 from the other audio signal 14 2 .
- the device 10 comprises, in addition to the components described so far, a calculator 28, a masking threshold calculator 30 and an adjuster 32.
- the calculator 28 is also connected to the input 12 and is designed to use a propagation model for the audio signals 14 and 14 2, respectively to calculate the version of the respective audio signal 14 ⁇ or 14 2 resulting from the room-selective reproduction in the first area 24, ie the version 34, the audio signal 14- reproduced at the location 24, as well as the version 34 2 of the audio signal 14 reproduced at the location 24 2 .
- the masking threshold calculator 30 receives the socket 34, and is configured to calculate a masking threshold 36 as a function thereof, and the adapter 32 receives the socket 34 2 of the other audio signal and optionally also the socket 34, the first audio signal 14, and is optionally designed to, depending on a comparison of the masking threshold 36 with the socket of the second audio signal 34 2, influence the output of the first and second audio signals for space-selective reproduction via the output 16 to the loudspeakers 18 by suitably controlling the beamforming processor 20, such as it is indicated by an arrow 38.
- an output of the adder 32 is connected to a control input of the beamforming processor 20.
- Calculators 28, masking threshold calculators 30, and fitters 32 may each be implemented in software, programmable hardware, or in hardware.
- the calculator 28 may use propagation models that are also for optimization the internal, channel / speaker-individual processing of the audio signals 14 1 t 14 2 within the beamforming processor 20 could have been used.
- the calculator 28 calculates or estimates, for example, as will be described in more detail below, the location 24 by the first audio signal 14 ! and the second audio signal 14 2 generated sound events. For example, it may take account of the channel / loudspeaker-individual processing of the audio signals 14 ⁇ 14 2 within the beamforming processor 20 and the positions of the loudspeakers 18 and optionally further parameters such as emission characteristics and / or orientation of the loudspeakers 18.
- the calculator 28 calculates the sound events, for example, measured or represented in sound pressure, amplitude or the like, and possibly frequency-dependent, ie for different frequencies.
- the calculator 28 may perform the simulation in a constant / fixed manner.
- the consideration of the adaptation to the channel / loudspeaker individual processing of the processor 20 is then based on the suitable design of the propagation model, which the calculator 28 uses to calculate the sockets 34T, 34 2 .
- the propagation model can thus also take into account the parameters just mentioned.
- the versions 34 and 34 2 can be output by the calculator 28 again in any form, ie analog or digital, compressed or uncompressed, in the time domain or in the frequency domain or the like.
- the masking threshold calculator 30 calculates a masking threshold depending on the version 34 ⁇ ie the audible version of the audio signal 14 ⁇ at the location 24. As indicated by a dashed arrow 40, the masking threshold calculator in addition to the version 34 ! also use a background audio signal (eg, noise or driving noise) for masking threshold calculation.
- a background audio signal eg, noise or driving noise
- the calculation takes into account temporal and / or spectral auditory masking effects.
- the calculated masking threshold thus indicates, depending on the frequency, how much the socket 34i of the audio signal 14-i at the location 24 is able to make other audio signals inaudible to a listener at the location 24 by covering the latter.
- the masking threshold calculator 30 may be configured to calculate the masking threshold at a frequency resolution that becomes increasingly coarse as the frequency increases, ie, as the frequency bands increase in frequency as the frequency increases, such as in a Bark frequency resolution.
- the adjuster 32 compares the masking threshold 36 with the socket 34 2 of the second audio signal 14 2 and thus determines, for example, whether the second audio signal 14 2 is audible to a person at the location 24, ie if the second audio signal at any frequency is the masking threshold exceeds. If so, the adjuster 32 takes countermeasures and appropriately controls the beamforming processor 20. Several examples of such controls have been previously indicated. Referring to the following figures, this is illustrated once again.
- Fig. 2 shows, for example, in a diagram plotted on the frequency f, the masking threshold 36, the socket 34 ⁇ and the socket 34 2 in a virtual, the hearing-force-measuring scale.
- a frequency range 42 in which currently the interfering audio signal 14 2 or the version 34 2 resulting at the location 24 according to the simulation exceeds the masking threshold 36, is illustrated by way of example.
- a possible countermeasure would be that the adjuster 32 controls the beamforming processor 20 in such a way that the second audio signal 34 2 is reduced in this frequency range 42, as indicated by an arrow 44. Additionally or alternatively, the adjuster 32 could control the beamforming processor 20 in such a way that the first audio signal 14 !
- the beamforming processor 20 executes, for example, the gain 46 and / or reduction 44 in advance on the respective incoming audio signal 14i or 14 2 and only then the channel / speaker individual processing of the same preprocessed. Audio signals for room-selective playback.
- the adjuster 32 may be formed to vary the beam shaping itself depending on the aforementioned comparison with the masking threshold 36. To illustrate this, reference is made to FIG. 3.
- beamforming processor 20 may have a plurality of channel / speaker options or modes of individual beamforming processing of audio signals 14 ⁇ and 14 2 , which are exemplarily indicated at 48 1 -48 N by different modes.
- the beam-forming processing according to 48 t - could be, for example, an optimal processing for space-selective reproduction, that is, optionally in place and frequency to a best suppression of the audio signal 14 2 and 34 2 at the location 24 lead.
- the other modes 24 2 - 48 N may possibly lead to similarly good separations or even to equally good or even optimal according to other or differently weighted criteria.
- All modes 48i - 48N may comprise, for example, differences in the quality of the suppression for different frequency ranges and in this case could, for example, the adjuster 32 depending on the comparison with the masking threshold 36 and a layer of an interval 42, is present in which a violation of the masking threshold 36
- an arrow 50 should indicate, for example, the selection of a currently selected 48 t -48 N mode and a double arrow 52 should indicate the change in the beamforming processor 20 with a loudspeaker / channel individual fading between one with the last one and the one in the beamforming processor 20 em associated with the new mode received loudspeaker signal.
- the device 10 of FIG. 1 is thus able to improve the suppression of another audio signal 14 2 at a location 24 of the PA area of the loudspeaker setup 18 with respect to a constant, optimized beamforming separation.
- Various measures are possible in order to avoid a possible deterioration of the audio quality of the first and / or the second audio signal at the location 24 and / or location 26 by the masking threshold-controlled modification.
- the extent of gain 46 and / or reduction 44 can be limited both in terms of its absolute value, ie the strength of the gain 46 and / or magnitude of the reduction 44, but also the change of this severity in time and / or frequency. In the case of using the possibility according to FIG.
- fading could be used to change from one mode to the other mode.
- a delay may also be provided to make processing delay adjustment to the processing delay caused by the series of computational processing 28, masking threshold calculator 30 and adjuster 32.
- Such additional delay in the beamforming processor 20 path from processing in the path along the calculator 28, masking threshold calculator 30, and aligner 32 could also be used to eliminate the aforementioned fading transitions between different beamforming modes 48 ! - 48 N easier.
- audio signals 14i and 14 2 can be provided. This is indicated by a dashed arrow 54 in FIG. 1.
- Additional audio signals 54 would be treated, for example, as the audio signal 14 2 , ie as audio signals whose reproduction at location 24 should be inaudible to a listener at this location 24.
- the above exemplary embodiment makes it possible to improve the perceived quality of a spatial-related reproduction by incorporating psychoacoustic effects. It exploits the fact that an audio Signal can prevent the audibility of components of another, quieter signal. This effect is called masking. For example, this plays a central role in lossy audio coding.
- masking In psychoacoustics, a distinction is made between masking in the time domain and in the frequency domain. During masking in the time domain, a loud signal, the so-called masker, masks other components that appear shortly after or within narrow limits before this sound event. In the frequency domain masking, a component of a particular frequency masks other components of similar frequency and amplitude.
- the threshold to which masking occurs depends on the frequency and absolute level of the marker and the distance between the frequencies of the masker and other signal.
- the masking thresholds and thus the decision as to whether a signal component is masked can be determined by psychoacoustic models. Such psychoacoustic models may be used by masking threshold calculator 30.
- FIG. Fig. 4 shows how two audio signals S ⁇ t) and S 2 (t) via two beamforming filter sets 6O 1 and 60 2 , a summation stage 62 and a speaker array of speakers 18 are processed so that these signals in the areas Zi and Z 2 , ie the audio signal S t) mainly in the area Z, and the audio signal S 2 (t) primarily in the area Z 2 .
- the components 60 ⁇ 60 2 and 62 form a simple beamforming processor 64 which, for example, operates constantly and is optimized to perform the aforementioned separation.
- the beamformer 60i subjects the incoming audio signal Si (t) to beamforming to generate a set of loudspeaker signals for that signal and does the same as the beamformer 60 2 for the second audio signal S 2 (t).
- Both beamformers 60 1i 2 output their loudspeaker signal sets to the summer 62, which individually adds the same loudspeaker signals to the channel / loudspeaker and feeds them to the loudspeakers 18.
- FIG. 5 now shows how the setup of FIG. 4 according to the embodiment of FIG. 1 can be improved.
- the apparatus of Fig. 5 is indicated at 10 and otherwise the reference numerals of Fig. 1 have been adopted to indicate corresponding parts to each other in Fig. 1.
- the beamforming processor 20 of FIG. 5 is modified from the starting point of FIG. 4 by way of example only by inserting a level adjuster 66 into the signal path of the interfering audio signal S 2 , here exemplarily on the input side of the beamformer 60 2 , although one for all channels / loudspeakers 18 equal level adjustment by level adjuster 66 would also be possible.
- the level adjuster 66 is controlled by the adjuster 32 to perform the reduction 44 illustrated above with reference to FIG. FIG. 5 also shows that it is possible to carry out the signal separation of other audio signals carried out in FIG.
- the calculator 28 simulates the respective audible version for both audio signals 60 S- and S 2 at both locations, namely locations Z1 and Z2. Therefore, in Fig. 5, a propagation model user 68! which applies the corresponding propagation models to the audio signal Si, and a propagation model user 68 2 who undertakes the same for the audio signal S 2 .
- the masking threshold calculator 30 For the respective version, for which the respective audio signal is provided in the respective location, ie the audible version of the audio signal S 2 at the location Z 2 and the audible version of the signal Si at the location, the masking threshold calculator 30 performs a masking threshold calculation in each case and outputs the result. ie the respective masking threshold for the location Zt and Z 2 , ie the masking by the signal Si at the location Z-, or the masking by the audio signal S 2 at the location Z 2 , to the control data adaptation or the adjuster 32 further, which in addition retains the respective disturbing listening versions, ie the audible version of the signal S 2 at the location Z and the audible version of the signal Si at the location Z 2 .
- the masking thresholds of the audibility of the signal S 2 in zone Zi are determined.
- the signals resulting from the signals Si (t) and S 2 (t) in the zone Zi are determined, such as the magnitudes in the frequency domain.
- a propagation model is calculated or used, which includes the transfer function of the loudspeaker array of loudspeakers 18.
- the signals are referred to as Si (t, Z,) and S 2 (t, Zi).
- the masking thresholds for the audibility of the signal S 2 (t, Z,) are determined using the masker S, (t, Zi).
- Thresholds are determined in a component change values for the magnitudes of the audio signal S t (t) (for certain frequency ranges).
- other psychoacoustically motivated parameters can be included, such as maximally permitted changes of the signal Si (t) in order to limit the effects of the adjustments by the adjuster 32 on the reproduction of Si (t) in Z 1 .
- the temporal course of the magnitude change is limited in order to avoid sudden, potentially disturbing changes.
- the parameters of this timing can also be determined by psychoacoustic parameters.
- the adjuster 32 Since the adjuster 32 knows the result of all comparisons, ie, the result of the comparison of the masking threshold in Z 2 with Si at location Z 2 and the result of the comparison of the masking threshold in Zi with S 2 at location ZL, the adjuster is able to derive therefrom For all places or areas Z 1 2 a reduction in the influence of the respective disturbing signal, ie S 2 in Z ⁇ and S ! in Z 2 , to calculate the desired signal, ie S 2 in Z 2 and Si in Z. It may be that the adjuster 32 has to compromise on this, since the disturbances in the individual areas require measures that mean a deterioration in the other area or in the other areas. This trade-off could be influenced by having the matcher 32 prioritize the ranges and associated desired signals so that the negative impact of higher priority signals at their respective destination is realized by other higher priority signals than lower priority signals.
- the number of audio signals may also be higher.
- the signal flow of the concept or algorithm is thus represented in FIG. 5 in such a way that from the signals Si (t) and S 2 (t) by means of an acoustic propagation model the acoustical propagation model table event, such as the sound pressure, the magnitude, etc., in the zone Z «is determined.
- This propagation model is usually a function of frequency and produces a discrete set of values, each associated with a frequency.
- the transfer function of the beam former 60 ! to a point, such as the center of zone Z- used as a propagation model.
- other models such as.
- a weighted average of the magnitude transfer function to a dot grid may be used.
- the kernel characteristic of the Propagation Model is that it translates an input signal S ( (t) to a measure which is the strength of the sound incidence originating from that signal in zone describes, for each of the considered frequency bands.
- the division of the audio frequency range into frequency bands can be done differently, but useful are psycho-acoustic properties oriented divisions, such as Constant-Q or Bark scale.
- the output values of the psychoacoustic model may be output at a lower frequency than the audio sample rate. This can be done, for example, by sub-sampling or by moving averaging with, for example, decimation.
- 5 are still raw control data which describe a desired level change in the individual frequency bands. These data are also defined over a grid of frequency bands and are usually present at a lower than the audio sampling rate. The raw control data is reworked in the matcher. In this module optional upper and lower limits for the level change of individual frequency ranges can be specified. On the other hand, the timing of the changes can be adjusted, such as by delaying and smoothing the level changes.
- the matched control signals of the adaptor are used in the level adjuster to frequency-level level the signal Si (t) before filtering with the loudspeaker-specific beamforming filters in the beamformer 60 2 .
- the level adjuster 66 thus acts as a multi-band equalizer.
- a function similar to a multiband compressor, or more generally multiband dynamics control, is achieved in conjunction with the timing of the adaptor, but unlike normal use, these units here use a different signal to control the gain values.
- the signal S 2 (t) may also be adaptively changed to reduce the interference from S 2 (t) to the zone Z ⁇ . This is also possible to reduce the crosstalk simultaneously.
- this possibility exists independently of the details of FIG. 5 more generally also for the example of FIG. 1.
- a reference signal 40 for background noise such as general background noise level, interior noise in automotive applications or the like, can be used in addition.
- This signal 40 can be used as an additional input to the masking threshold calculation, as described above.
- the reference signal 40 is preferably a measured or sensible estimated value for the background noise signal in the "sound zones" 24 or 26 or Zi in Z 2 .
- the above exemplary embodiments thus describe a concept for space-selective reproduction with loudspeaker arrays by psychoacoustic environmental effects or the spatial reproduction of audio signals via a plurality of loudspeakers, which can be arranged, for example, as an array.
- loudspeakers which can be arranged, for example, as an array.
- this has been accomplished by a combination of beamforming algorithms with a psychoacoustic model that modifies the audio signals to reduce the audibility of the interfering signals through psychoacoustic masking by the wanted signal.
- aspects have been described in the context of a device, it will be understood that these aspects also constitute a description of the corresponding method, so that a block or a component of a device is also to be understood as a corresponding method step or as a feature of a method step. Similarly, aspects described in connection with or as a method step also represent a description of a corresponding block or detail or feature of a corresponding device.
- Some or all of the method steps may be performed by a hardware device (or using a Hardware apparatus), such as a microprocessor, a programmable computer or an electronic circuit. For some Embodiments, some or more of the most important method steps may be performed by such an apparatus.
- embodiments of the invention may be implemented in hardware or in software.
- the implementation may be performed using a digital storage medium, such as a floppy disk, a DVD, a Blu-ray Disc, a CD, a ROM, a PROM, an EPROM, an EEPROM or FLASH memory, a hard disk, or other magnetic disk or optical memory are stored on the electronically readable control signals, which can cooperate with a programmable computer system or cooperate such that the respective method is performed. Therefore, the digital storage medium can be computer readable.
- some embodiments according to the invention include a data carrier having electronically readable control signals capable of interacting with a programmable computer system such that one of the methods described herein is performed.
- embodiments of the present invention may be implemented as a computer program product having a program code, wherein the program code is operable to perform one of the methods when the computer program product runs on a computer.
- the program code can also be stored, for example, on a machine-readable carrier.
- an embodiment of the method according to the invention is thus a computer program which has a program code for performing one of the methods described herein when the computer program runs on a computer.
- a further embodiment of the method according to the invention is thus a data carrier (or a digital storage medium or a computer-readable medium) on which the computer program is recorded for performing any of the methods described herein.
- a further exemplary embodiment of the method according to the invention is thus a data stream or a sequence of signals which represents or represents the computer program for performing one of the methods described herein.
- the data stream or the sequence of signals may be configured, for example, to be transferred via a data communication connection, for example via the Internet.
- Another embodiment includes a processing device, such as a computer or a programmable logic device, that is configured or adapted to perform one of the methods described herein.
- a processing device such as a computer or a programmable logic device
- Another embodiment includes a computer on which the computer program is installed to perform one of the methods described herein.
- Another embodiment according to the invention comprises a device or system adapted to transmit a computer program for performing at least one of the methods described herein to a receiver.
- the transmission can be done for example electronically or optically.
- the receiver may be, for example, a computer, a mobile device, a storage device or a similar device.
- the device or system may include a file server for transmitting the computer program to the recipient.
- a programmable logic device eg, a field programmable gate array, an FPGA
- a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein.
- the methods are performed by any hardware device. This may be a universal hardware such as a computer processor (CPU) or hardware specific to the process, such as an ASIC.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- Otolaryngology (AREA)
- Circuit For Audible Band Transducer (AREA)
- Stereophonic System (AREA)
- Obtaining Desirable Characteristics In Audible-Bandwidth Transducers (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102013210184 | 2013-05-31 | ||
DE102013217367.9A DE102013217367A1 (en) | 2013-05-31 | 2013-08-30 | DEVICE AND METHOD FOR RAUMELECTIVE AUDIO REPRODUCTION |
PCT/EP2014/061188 WO2014191526A1 (en) | 2013-05-31 | 2014-05-28 | Device and method for spatially selective audio playback |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3005732A1 true EP3005732A1 (en) | 2016-04-13 |
EP3005732B1 EP3005732B1 (en) | 2017-06-21 |
Family
ID=51899430
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP14727481.5A Active EP3005732B1 (en) | 2013-05-31 | 2014-05-28 | Device and method for spatially selective audio playback |
Country Status (7)
Country | Link |
---|---|
US (1) | US9813804B2 (en) |
EP (1) | EP3005732B1 (en) |
JP (1) | JP6301453B2 (en) |
KR (1) | KR101877323B1 (en) |
CN (1) | CN105247892B (en) |
DE (1) | DE102013217367A1 (en) |
WO (1) | WO2014191526A1 (en) |
Families Citing this family (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP3467818B1 (en) * | 2016-05-30 | 2020-04-22 | Sony Corporation | Locally attenuated sound field forming device, corresponding method and computer program |
EP3264734B1 (en) * | 2016-06-30 | 2022-03-02 | Nokia Technologies Oy | Controlling audio signal parameters |
US10019981B1 (en) * | 2017-06-02 | 2018-07-10 | Apple Inc. | Active reverberation augmentation |
US10531196B2 (en) * | 2017-06-02 | 2020-01-07 | Apple Inc. | Spatially ducking audio produced through a beamforming loudspeaker array |
US11516614B2 (en) | 2018-04-13 | 2022-11-29 | Huawei Technologies Co., Ltd. | Generating sound zones using variable span filters |
US11089403B1 (en) | 2018-08-31 | 2021-08-10 | Dream Incorporated | Directivity control system |
JP2022516192A (en) * | 2019-01-06 | 2022-02-24 | シレンティウム リミテッド | Sound control devices, systems, and methods |
US20230136085A1 (en) * | 2019-02-19 | 2023-05-04 | Akita Prefectural University | Acoustic signal encoding method, acoustic signal decoding method, program, encoding device, acoustic system, and decoding device |
EP4005247A1 (en) * | 2019-07-30 | 2022-06-01 | Dolby Laboratories Licensing Corporation | Coordination of audio devices |
US11968268B2 (en) | 2019-07-30 | 2024-04-23 | Dolby Laboratories Licensing Corporation | Coordination of audio devices |
US20220272454A1 (en) * | 2019-07-30 | 2022-08-25 | Dolby Laboratories Licensing Corporation | Managing playback of multiple streams of audio over multiple speakers |
CN114391262B (en) | 2019-07-30 | 2023-10-03 | 杜比实验室特许公司 | Dynamic processing across devices with different playback capabilities |
US11871184B2 (en) | 2020-01-07 | 2024-01-09 | Ramtrip Ventures, Llc | Hearing improvement system |
WO2021245876A1 (en) * | 2020-06-04 | 2021-12-09 | 日本電信電話株式会社 | Speaker callibration method, device, and program |
KR102347626B1 (en) | 2020-07-01 | 2022-01-06 | 한국과학기술원 | Method and apparatus for generating personal sound zone according to distance |
WO2023280357A1 (en) * | 2021-07-09 | 2023-01-12 | Soundfocus Aps | Method and loudspeaker system for processing an input audio signal |
CN114882721B (en) * | 2022-05-27 | 2023-05-09 | 中国第一汽车股份有限公司 | Vehicle navigation information playing method and device, electronic equipment and storage medium |
Family Cites Families (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3473517B2 (en) * | 1999-09-24 | 2003-12-08 | ヤマハ株式会社 | Directional loudspeaker |
DE60036958T2 (en) * | 1999-09-29 | 2008-08-14 | 1...Ltd. | METHOD AND DEVICE FOR ORIENTING SOUND WITH A GROUP OF EMISSION WANDERS |
JP4349123B2 (en) * | 2003-12-25 | 2009-10-21 | ヤマハ株式会社 | Audio output device |
GB0405346D0 (en) * | 2004-03-08 | 2004-04-21 | 1 Ltd | Method of creating a sound field |
TWI475896B (en) * | 2008-09-25 | 2015-03-01 | Dolby Lab Licensing Corp | Binaural filters for monophonic compatibility and loudspeaker compatibility |
US8218783B2 (en) * | 2008-12-23 | 2012-07-10 | Bose Corporation | Masking based gain control |
WO2011039413A1 (en) * | 2009-09-30 | 2011-04-07 | Nokia Corporation | An apparatus |
US8965546B2 (en) * | 2010-07-26 | 2015-02-24 | Qualcomm Incorporated | Systems, methods, and apparatus for enhanced acoustic imaging |
KR101782050B1 (en) * | 2010-09-17 | 2017-09-28 | 삼성전자주식회사 | Apparatus and method for enhancing audio quality using non-uniform configuration of microphones |
JP5838740B2 (en) * | 2011-11-09 | 2016-01-06 | ソニー株式会社 | Acoustic signal processing apparatus, acoustic signal processing method, and program |
US20130259254A1 (en) * | 2012-03-28 | 2013-10-03 | Qualcomm Incorporated | Systems, methods, and apparatus for producing a directional sound field |
US20140006017A1 (en) * | 2012-06-29 | 2014-01-02 | Qualcomm Incorporated | Systems, methods, apparatus, and computer-readable media for generating obfuscated speech signal |
EP3040984B1 (en) * | 2015-01-02 | 2022-07-13 | Harman Becker Automotive Systems GmbH | Sound zone arrangment with zonewise speech suppresion |
-
2013
- 2013-08-30 DE DE102013217367.9A patent/DE102013217367A1/en not_active Withdrawn
-
2014
- 2014-05-28 KR KR1020157034882A patent/KR101877323B1/en active IP Right Grant
- 2014-05-28 EP EP14727481.5A patent/EP3005732B1/en active Active
- 2014-05-28 CN CN201480031334.3A patent/CN105247892B/en not_active Expired - Fee Related
- 2014-05-28 JP JP2016516172A patent/JP6301453B2/en active Active
- 2014-05-28 WO PCT/EP2014/061188 patent/WO2014191526A1/en active Application Filing
-
2015
- 2015-11-30 US US14/954,913 patent/US9813804B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
JP2016524862A (en) | 2016-08-18 |
EP3005732B1 (en) | 2017-06-21 |
US20160088388A1 (en) | 2016-03-24 |
DE102013217367A1 (en) | 2014-12-04 |
KR20160007584A (en) | 2016-01-20 |
KR101877323B1 (en) | 2018-08-09 |
CN105247892B (en) | 2019-02-22 |
US9813804B2 (en) | 2017-11-07 |
JP6301453B2 (en) | 2018-03-28 |
CN105247892A (en) | 2016-01-13 |
WO2014191526A1 (en) | 2014-12-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3005732B1 (en) | Device and method for spatially selective audio playback | |
EP1977626B1 (en) | Method for recording and reproducing a sound source with time-variable directional characteristics | |
EP1013141B1 (en) | Method and device for reproducing a stereophonic audiosignal | |
WO2005046194A1 (en) | Device and method for processing an input signal | |
EP3005737A2 (en) | Mixing desk, sound signal generator, method and computer program for providing a sound signal | |
EP1972181B1 (en) | Device and method for simulating wfs systems and compensating sound-influencing wfs characteristics | |
EP2754151B2 (en) | Device, method and electro-acoustic system for prolonging a reverberation period | |
DE112012006457T5 (en) | Frequency characteristic modification device | |
WO2006133812A1 (en) | Device and method for generating a loudspeaker signal based on a randomly occurring audio source | |
WO2022218822A1 (en) | Device and method for generating a first control signal and a second control signal using linearisation and/or bandwidth expansion | |
DE102011086728A1 (en) | Hearing apparatus with a device for reducing a microphone noise and method for reducing a microphone noise | |
DE69309679T2 (en) | STEREOPHONIC SOUND PLAYER WITH SEVERAL SPEAKERS FOR EACH CHANNEL | |
DE102017201195A1 (en) | Method for operating a binaural hearing aid system | |
EP3314915A1 (en) | Method for sound reproduction in reflection environments, in particular in listening rooms | |
WO1995030323A1 (en) | Process and device for compensating for acoustic distortion | |
DE102007014816B4 (en) | Communication system and method for operating a communication system in a vehicle | |
WO2006116883A1 (en) | Method for compensating for changes in reproduced audio signals and a corresponding device | |
DE10215775A1 (en) | Method for spatial representation of sources of sound uses an array of loudspeakers fitted alongside each other and operated with coherent signals. | |
EP2737726A1 (en) | Method for processing an audio signal, audio playback system, and processing unit for processing audio signals | |
WO2022183231A1 (en) | Method for producing audio signal filters for audio signals in order to generate virtual sound sources | |
DE102016121989B4 (en) | SOUND SYSTEM, METHOD FOR OPTIMIZING A ROOM SOUND, AND METHOD FOR STORING AND / OR PLAYBACKING ACOUSTIC SIGNALS | |
EP1900250B1 (en) | Electroacoustic method | |
DE102018216604A1 (en) | System for transmitting sound into and out of the head of a listener using a virtual acoustic system | |
CH709271A2 (en) | Encoding and decoding of a low-frequency channel in an audio multi-channel signal. | |
DE102022134717A1 (en) | Audio system with seat-centered speaker arrangement |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20151112 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: SPORER, THOMAS Inventor name: FRANCK, ANDREAS Inventor name: SLADECZEK, CHRISTOPH |
|
DAX | Request for extension of the european patent (deleted) | ||
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04R 1/32 20060101ALI20161222BHEP Ipc: H04R 27/00 20060101AFI20161222BHEP |
|
INTG | Intention to grant announced |
Effective date: 20170110 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 903902 Country of ref document: AT Kind code of ref document: T Effective date: 20170715 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502014004339 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20170621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170921 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170922 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170921 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20171021 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502014004339 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
REG | Reference to a national code |
Ref country code: FR Ref legal event code: PLFP Year of fee payment: 5 |
|
26N | No opposition filed |
Effective date: 20180322 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20180531 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LI Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180531 Ref country code: CH Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180531 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180528 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180528 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180531 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20170621 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20140528 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20170621 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 903902 Country of ref document: AT Kind code of ref document: T Effective date: 20190528 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20190528 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230524 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20230517 Year of fee payment: 10 Ref country code: DE Payment date: 20230519 Year of fee payment: 10 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20230522 Year of fee payment: 10 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R081 Ref document number: 502014004339 Country of ref document: DE Owner name: BRANDENBURG LABS GMBH, DE Free format text: FORMER OWNER: FRAUNHOFER-GESELLSCHAFT ZUR FOERDERUNG DER ANGEWANDTEN FORSCHUNG E.V., 80686 MUENCHEN, DE |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: 732E Free format text: REGISTERED BETWEEN 20240104 AND 20240110 |