EP3197181B1 - Method for reducing latency of a filter bank for filtering an audio signal and method for low latency operation of a hearing system - Google Patents
Method for reducing latency of a filter bank for filtering an audio signal and method for low latency operation of a hearing system Download PDFInfo
- Publication number
- EP3197181B1 EP3197181B1 EP16204529.8A EP16204529A EP3197181B1 EP 3197181 B1 EP3197181 B1 EP 3197181B1 EP 16204529 A EP16204529 A EP 16204529A EP 3197181 B1 EP3197181 B1 EP 3197181B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- signal
- audio signal
- output
- prediction period
- block
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000005236 sound signal Effects 0.000 title claims description 98
- 238000000034 method Methods 0.000 title claims description 45
- 238000001914 filtration Methods 0.000 title claims description 20
- 230000006870 function Effects 0.000 claims description 50
- 230000005540 biological transmission Effects 0.000 claims description 39
- 230000009466 transformation Effects 0.000 description 16
- 230000000875 corresponding effect Effects 0.000 description 12
- 230000008569 process Effects 0.000 description 10
- 238000007781 pre-processing Methods 0.000 description 9
- 230000000694 effects Effects 0.000 description 7
- 230000001419 dependent effect Effects 0.000 description 6
- 239000000523 sample Substances 0.000 description 6
- 230000003595 spectral effect Effects 0.000 description 6
- 230000002123 temporal effect Effects 0.000 description 6
- 230000008901 benefit Effects 0.000 description 5
- 230000007774 longterm Effects 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 238000007906 compression Methods 0.000 description 4
- 230000006835 compression Effects 0.000 description 4
- 238000013144 data compression Methods 0.000 description 4
- 230000010355 oscillation Effects 0.000 description 4
- 230000004044 response Effects 0.000 description 4
- 230000008054 signal transmission Effects 0.000 description 4
- 210000003454 tympanic membrane Anatomy 0.000 description 4
- 230000006978 adaptation Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 230000001629 suppression Effects 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000018109 developmental process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 208000016354 hearing loss disease Diseases 0.000 description 2
- 230000035807 sensation Effects 0.000 description 2
- 238000001228 spectrum Methods 0.000 description 2
- 206010011878 Deafness Diseases 0.000 description 1
- 206010048865 Hypoacusis Diseases 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000003321 amplification Effects 0.000 description 1
- 230000002238 attenuated effect Effects 0.000 description 1
- 230000008033 biological extinction Effects 0.000 description 1
- 230000001427 coherent effect Effects 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 230000009365 direct transmission Effects 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 210000000613 ear canal Anatomy 0.000 description 1
- 230000010370 hearing loss Effects 0.000 description 1
- 231100000888 hearing loss Toxicity 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000003199 nucleic acid amplification method Methods 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 239000013074 reference sample Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/35—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using translation techniques
- H04R25/353—Frequency, e.g. frequency shift or compression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
- H04R25/505—Customised settings for obtaining desired overall acoustical characteristics using digital signal processing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/55—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using an external connection, either wireless or wired
- H04R25/552—Binaural
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R5/00—Stereophonic arrangements
- H04R5/04—Circuit arrangements, e.g. for selective connection of amplifier inputs/outputs to loudspeakers, for loudspeaker detection, or for adaptation of settings to personal preferences or hearing impairments
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L19/00—Speech or audio signals analysis-synthesis techniques for redundancy reduction, e.g. in vocoders; Coding or decoding of speech or audio signals, using source filter models or psychoacoustic analysis
- G10L19/0017—Lossless audio signal coding; Perfect reconstruction of coded audio signal by transmission of coding error
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
- H04R2225/43—Signal processing in hearing aids to enhance the speech intelligibility
Definitions
- the invention relates to a method for reducing the latency of a filter bank for filtering an audio signal, wherein a plurality of signal blocks in the time domain is formed from the audio signal, wherein in each case a filter function is specified for at least a plurality of the signal blocks, the signal Block is filtered with the given filter function transformed into the frequency domain, and thereby a transformed signal block is formed, and signal components of the transformed signal block are output for further processing.
- the invention further relates to a method for low-latency operation of a hearing system, wherein a first audio signal is generated from a sound signal by a first input transducer, wherein the first audio signal is filtered in a signal processing unit by means of a first filter bank, wherein signal components of the filtered first audio signal in the signal processing unit further processed and for generating an output signal, and wherein an output sound signal is generated from the output signal by an output transducer.
- an audio signal generated by a microphone is usually transformed from the time domain into the frequency domain after digitization, ie after the digitization the audio signal initially exists in the form of time-resolved samples which, if necessary, become individual signal blocks (so-called “signal blocks”).
- Frames ") are decomposed by a Fourier transform such as FFT into individual spectral signal components of the generated audio signal.
- FFT Fourier transform
- This has the advantage that frequency-selective algorithms such as noise reduction, directional microphone or dynamic compression can be applied.
- the mentioned transformation has the disadvantage that an audio signal converted back into the time domain after a corresponding frequency-selective processing has a delay with respect to the input signal, which is typically of the order of magnitude of several ms. This delay, also known as latency, is greater the higher the resolution in the frequency domain is chosen.
- an open adaptation of the hearing device is often chosen, in which the sound output from a speaker of the hearing aid output signal via a sound tube with Schirmchen or a listener with Schirmchen in the ear canal is passed to the eardrum.
- the eardrum itself thus comes a mixture of a frequency selective attenuated direct sound of the environment as well as the output sound signal generated by the hearing aid.
- different mixing ratios are therefore found as a function of the frequency.
- the described problems with comb filter effects are not tied to a binaural hearing system, but can also occur in a monaural hearing system with only one hearing aid, in which a direct sound of the environment and an output sound signal of a hearing device overlaid with temporal displacement reach the eardrum of the user ,
- the temporal offset is primarily due to the internal latency of the hearing system for signal processing and in particular in the filtering.
- a method for filtering an input signal by means of a desired impulse response in which the impulse response in the time domain is decomposed into individual segments which are transformed into the frequency domain and from which respective coefficient blocks are formed for the filtering of the individual time-delayed frames in the frequency domain ,
- the frames thus filtered with the coefficient blocks are summed with their corresponding time delay, and from this a signal in the time domain is generated by inverse transformation, of which in a predetermined time Way still individual signal components are discarded to obtain the final, filtered output signal.
- the US Pat. No. 7,251,271 B1 refers to a method to avoid so-called aliasing effects when filtering a discrete input signal with a discrete impulse response. These can occur during the transformation of the individual frames of the input signal from the time domain into the frequency domain and the inverse transformation of the product of impulse response and frequency spectrum of the input signal into the time domain. To avoid the aliasing effects, individual frames are extended before the respective transformation by adding zeros in order to correspond to the respective filter length.
- the invention is therefore based on the object of specifying a method for as low-latency spectral filtering of an audio signal with the highest possible spectral resolution.
- the invention is further based on the object of specifying a method for low-latency operation of a hearing system.
- the first object is achieved according to the invention by a method for reducing the latency of a filter bank for filtering an audio signal, wherein a plurality of signal blocks in the time domain is formed from the audio signal.
- a filter function is specified, at least a subinterval of the signal block is given as a prediction period, signal components of the signal block are estimated in the at least one subinterval for the prediction period, and from the for the prediction period of estimated signal components and the signal components of the signal block outside the prediction period, a predicted signal block is generated.
- the predicted signal block is filtered with the predetermined filter function transformed into the frequency domain, and thereby a transformed signal block is formed, and signal components of the transformed signal block are output for further processing.
- the second object is achieved by a method for low-latency operation of a hearing system, wherein from a sound signal through a first input transducer, a first audio signal is generated, wherein the first audio signal is transmitted directly to a signal processing unit, and in the signal processing unit directly by means of a first filter bank according to the above-described method for reducing the latency of a filter bank for filtering an audio signal is filtered, wherein signal portions of the filtered first audio signal in the signal processing unit further processed and used to generate an output signal, and wherein from the output signal directly by an output transducer, an output sound signal is generated.
- a signal block (“frame") in the time domain is formed from the audio signal by the audio signal is converted by time and amplitude discretization in a plurality of each successive time points associated amplitude ("samples"), and a plurality of consecutive samples is combined into a signal block.
- the further processing of the signal components of the transformed signal block comprises in particular a frequency band-dependent amplification, a frequency band-dependent directional characteristic, a frequency band-dependent noise suppression and a backward transformation of frequency band-dependent treated signal components in the time domain.
- the estimation of the signal components for the prediction period of a respective signal block preferably takes place via a prediction algorithm, such as by a linear prediction filter.
- a prediction algorithm such as by a linear prediction filter.
- an adaptive adaptation of time-correlated coefficients used for the estimation is possible such that an estimation coefficient, which is assigned as a coordinate in the signal block in each case to a sample with a specific time delay, depending on the error between an estimated sample and a real from the Audio signal obtained sample is corrected, whereby the corrector is renewed at periodic intervals.
- a signal component estimated for a signal block becomes is also used for a signal block which follows later, if the period corresponding to the signal component still falls within the prediction period of the signal block following later.
- the prediction period preferably comprises the respectively first and / or the last sample of a signal block.
- the period lying outside the prediction period in each case forms a coherent interval in a signal block.
- the prediction period comprises the first n samples and / or the last m samples, where n and m natural numbers are smaller than the number of samples in the respective signal block.
- An input transducer or an output transducer of the hearing system includes any shape of an acousto-electric or an electro-acoustic transducer, for example a microphone or a loudspeaker.
- Direct transmission of the first audio signal to the signal processing unit is to be understood as meaning that the transmission of the first audio signal takes place immediately after its generation, that is to say in particular without another, via signal preprocessing, such as signal processing.
- a / D conversion and / or data compression beyond time delay takes place, as z. B. by a long-term physical storage, which is not based on the FIFO principle ("first-in-first-out"), would occur.
- the transmission takes place in particular locally within a hearing device, for example on the signal path predetermined by the signal lines.
- the transmission also takes place wirelessly, for example from a first hearing device of a binaural hearing system to a second hearing device of the binaural hearing system.
- Direct filtering of the first audio signal in the signal processing unit analogously means that the filtering process for the audio signal takes place immediately after its input in the signal processing unit, ie in particular without a further, beyond the direct signal transmission time delay as z. B. by a long-term storage, which is not based on the FIFO principle ("first-in-first-out"), would occur.
- an immediate generation of the output sound signal from the output signal means that immediately after the generation of the output signal is passed through the further processing, the output signal to the output transducer for output, ie in particular without a further, beyond the direct signal transmission outgoing time delay, z.
- the filter banks which are used to transform the audio signals generated by the input transducers into the frequency domain (analysis filter banks), and the filter banks for the inverse transformation of the frequency resolved, further processed audio signals into the time domain (Synthetic filter banks), the former usually have a larger proportion.
- the transmission of an audio signal from one hearing aid to another for the generation of a binaural output signal is associated with a certain delay.
- the latter is difficult to reduce given the restrictions on encoding for transmission.
- the effective length of the signal block can be reduced with an appropriate choice of the prediction period, without This affects the frequency resolution of the filter bank.
- the frequency resolution of the filter bank depends on the temporal information content of the signal blocks to be used for the filter process, ie of their length. Since the signal components are now estimated in a signal block for a period of time, the latency of the filter bank can be reduced by the duration corresponding to the associated prediction period.
- two temporally successive signal blocks overlap each other in part.
- the definition of the temporal sequence is preferably carried out via a reference sample for the respective signal block, e.g. the first sample.
- the consequence of the described overlap is that the relevant, successive signal blocks have several, preferably successive, samples in common.
- this improves the temporal resolution in the frequency domain, since this allows frequent frequency band information to be updated frequently, and on the other hand, the cost of estimating the signal components can be reduced because already estimated signal components are available for a subsequent block without a new estimation process stand.
- signal portions of the transformed signal block are output separately according to different frequency bands for further processing.
- the latency of the filter bank which is reduced by estimating the signal components of the prediction periods, is particularly advantageous given a constant high frequency resolution.
- the filter function in the prediction period preferably has an average lower transmission amplitude than outside the prediction period. This is to mean that the value of the transmission amplitude of the filter function averaged over the entire prediction period is lower than that over the remaining one Period of the signal block outside the prediction period averaged value of the transmission amplitude of the filter function.
- the transmission amplitude of the filter function is formed in each case by a logarithmically concave function, wherein the prediction period spans the maximum of the transmission amplitude of the filter function.
- a logarithmically concave function is defined as a function whose logarithm is concave in the domain of definition - which is given here by the individual samples of the respective signal block. Such a function may for example be given by an approximation of a Gaussian bell curve over a finite, discretized domain of definition.
- the advantage of the logarithmic concave behavior of the transmission amplitude is that it has a maximum of two inflection points in the domain of definition, and thus is not subject to any oscillations. This results in an advantageous filter behavior, since thus no relevant signal components with a minimum value of an oscillation of the filter function are filtered.
- a logarithmic concave function can be represented as a function reciprocal to a particular logarithmic convex function.
- a logarithmically convex function is in turn convex. This means that the reciprocal, logarithmically concave function due to the reciprocity property has a maximum of two inflection points.
- the maximum of the transmission amplitude lies in a convex region, so that beyond the inflection points the transmission amplitude runs concavely.
- the transmission amplitude usually already has sufficiently low values, so that the choice of the prediction time period in at least one of the two ranges can ensure that errors which can occur due to deviations in the estimation of the signal components from the real signal components, be largely suppressed due to the sufficiently lower transmission amplitude of the filter function, and thus not significantly enter into the transformed signal block.
- an empty signal is respectively estimated as signal components for the prediction period of at least one signal block.
- An empty signal is in this case the signal which has no amplitude for the period in question.
- the estimation of an empty signal is carried out in particular in the event that the signal components of the audio signal which are used for the estimation method of the signal components of the prediction period do not allow a qualitatively sufficiently high-quality estimate of the signal components as a result of insufficient correlations. This can occur, for example, if there is a high proportion of white noise in the audio signal, which reduces the correlation of successive samples and thus makes prediction more difficult.
- estimated signal components which are different from the empty signal, with respect to the quality of the estimate, are to be compared with the corresponding real signal components of the audio signal in order to be able to evaluate the quality of the prediction.
- excessive deviation - defined by a deviation measure such.
- B. an averaged over several samples difference and an associated upper bound for the deviation measure - is determined instead of the predicted signal components, an empty signal as estimated for the prediction signal component. It is also possible to check the signal components of the audio signal for correlations even before the prediction, and to set an empty signal as signal component for the prediction period if the correlation is too low.
- a second audio signal is generated from the sound signal by a second input transducer spatially separated from the first input transducer, the second audio signal is transmitted directly to the signal processing unit and filtered by a second filter bank, and wherein signal components of filtered second audio signal in the signal processing unit further processed and used to generate the output signal.
- the filtering of the second audio signal by means of the second filter bank in accordance with the above-described method for reducing the latency of a filter bank for filtering an audio signal is understood to mean that the transmission of the second audio signal without another, via a signal preprocessing such.
- a / D conversion and / or data compression and the direct signal transmission beyond time delay takes place, as z. B. by a long-term physical storage, which would not based on the FIFO principle ("first-in-first-out”) would occur.
- This named embodiment makes possible, in particular, a low-latency operation of a binaural hearing system, taking into account the special features occurring in such a hearing system as a result of the signal transmission from one hearing aid to another occurring for the generation of the binaural auditory sensation. Since, in the case of a binaural hearing system for compression, the real information content of signal portions of the audio signal received by the respective other hearing device for the generation of the binaural hearing sensation is often reduced for better transmission, for example by data compression, this is possible by estimating the signal components in the prediction period reduced errors in its meaning.
- a further advantage of using the method for low-latency operation of a binaural hearing system is that a certain latency of several ms is already introduced into the hearing system through the described transmission of the audio signals.
- the reduction of further possible latencies, e.g. in the present case through the filter banks, here helps to minimize the losses of the sound quality by comb filter effects as low as possible.
- the invention further provides a hearing aid, comprising at least one input transducer for generating an audio signal, an output transducer for generating an output sound signal, and a local signal processing unit having a first filter bank, which for carrying out the above-described method for reducing the latency of a filter bank for filtering an audio signal is set up.
- the invention also mentions a binaural hearing system with two hearing aids as described above, which is set up to carry out the method for low-latency operation of a hearing system with at least two input transducers.
- the advantages specified for the method and its developments can be transferred analogously to the binaural hearing system.
- Fig. 1 1 is a schematic block diagram of a binaural hearing system 1.
- the binaural hearing system 1 is in this case formed by a first hearing device 2 and a second hearing device 4.
- the first hearing device 2 has a first input transducer 8 configured as a microphone 6, which generates a first audio signal 10 from a sound signal 9.
- the second hearing device 4 has a second input transducer 14 configured as a microphone 12, which generates a second audio signal 16 from the sound signal 9.
- the first audio signal 10 and the second audio signal 16 are respectively prepared in the respective hearing device 2, 4 by a local signal preprocessing 18, 20, which in each case in particular comprises an A / D conversion, for the further signal processing processes.
- the local signal preprocessing 18, 20 comprises in particular only runtime processes, ie those processes which do not involve any further delay, in particular no longer-term storage and charging processes of the signal components, over the duration of the signal processing taking place.
- the first audio signal 10 is first transmitted immediately after the local signal pre-processing 18 in a binaural transmission process 22 from the first hearing aid 2 to the second hearing aid 4, where it is filtered in a signal processing unit 24 in a first filter bank 26 in a manner to be described.
- the binaural transmission process 22 takes place immediately after the local signal preprocessing 18, that is, in particular without further delay, in particular without longer-term storage and recharging the relevant signal components on a FIFO memory addition.
- the filtered first audio signal 28 will now be subjected to frequency bandwise signal processing algorithms 30, e.g. Noise suppression, directional microphone or dynamic compression applied.
- the second audio signal 16 is supplied immediately after the local signal pre-processing 20 of the signal processing unit 24, where it is first filtered in a second filter bank 32 in a manner to be described, wherein as a filtered second audio signal 34, the respective signal components are transmitted separately in individual frequency bands.
- the filtered second audio signal 34 resulting from the second filter bank 32 the respective signal components are output separately in individual frequency bands.
- frequency bandwise signal processing algorithms 28 such as noise reduction, directional microphone or dynamic compression are now applied. From the filtered first audio signal 26 and the filtered second audio signal 34, an output signal 36 is generated after the frequency band-wise signal processing 28, which locally reflects the binaural hearing at the location of the second hearing device 4.
- the output signal 36 is converted directly, ie in particular without further long-term storage and recharges of the signal components, from an output transducer 40 designed as a loudspeaker 38 into an output sound signal 42.
- Fig. 2 is against a time axis t, the first audio signal 10 after Fig. 1 which is split into individual partially overlapping signal blocks 50a-f.
- the individual signal blocks 50a-f are in this case formed from a large number of successive samples of the first audio signal 10, individual samples occurring as a result of the overlap of the successive signal blocks 50a-f in at least two signal blocks.
- the individual signal blocks 50a-f are now each transformed in a manner to be described, the frequency domain. Due to the short time interval of each of two consecutive signal blocks 50a-f, the spectral signal components of the first audio signal 10 can thus be updated in short intervals in the frequency domain.
- the individual signal blocks 50a-f are determined Signal portions, which is shown for the signal block 50c on the basis of a detail representation.
- the individual real signal components 52a, 52b are shown against a time axis t '.
- the real signal components 52a, 52b are each given by the amplitude of the corresponding sample.
- the transmission amplitude 54c of the filter function 56c is shown, which in the present case is approximately given by a Gaussian bell curve.
- the filter function 56c in this case represents a window function with which the edges of the signal block 50c for the transformation into the frequency domain are to be smoothed out. This is because without such a window function, the Fourier transform of the signal components of the signal block 50c is de facto a Fourier transform of the signal components of the first audio signal 10, which are multiplied by a rectangular function corresponding to the duration of the signal block.
- this multiplication in the time domain means a convolution of the frequency components of the first audio signal 10 with the Fourier transform of the rectangular function, which is given by a strongly oscillating sin (x) / x or Sinc function.
- the edges of the signal block 50c for the transformation into the frequency domain are "hidden” by means of a suitable filter function 56c. This is done by the transmission amplitude 54c of the filter function 56c converges to zero at the edges of the signal block 50c as free of oscillation as possible, ie in particular with as few turning points as possible.
- a function having such properties is given by a logarithmic concave function, such as e.g. the approximated Gaussian bell curve of the present case.
- the sub-interval 58c lies beyond the point of inflection 62c of the transmission amplitude 54c, ie in particular far away from the maximum 64c of the transmission amplitude 54c, so that in the sub-interval 58c, which defines the prediction time 60c, the transmission amplitude 54c has only low values.
- the signal components to be used for the transformation are now estimated there by means of a prediction algorithm, for example a linear prediction filter, instead of the real signal components 52b.
- the signal portions 66b estimated in the prediction period 60c and the signal portions 52a of the signal block 50c outside the prediction period 60c now form a predicted signal block 68c.
- This predicted signal block 68c is now multiplied by the filter function 56c and transformed into the frequency domain by means of a fast Fourier transformation, so that there the frequency-resolved information of the transformed signal block 50c is available for further processing by means of frequency band-dependent signal processing algorithms stands.
- the procedure described is used to estimate signal components for a prediction period which is to be chosen favorably on the basis of the respective filter function to be used, in order thus to reduce the latency for the transformation into the frequency domain. since then the last samples of a signal block need not yet exist, so that the transformation can be started several ms earlier due to the estimation.
- An important role in this case plays the course of the transmission amplitude 54c of the filter function 56c.
- a possible error which could result from the deviation of the signal portions 66b estimated for the prediction period 60c from the real signal portions 52b is suppressed by the fact that for the prediction period 60c the transmission amplitude 54c has only comparatively small values relative to its maximum 64c, and thus by the corresponding multiplication with the filter function 56c, the estimated signal components 66b make only a small contribution to the transformed signal block anyway.
- this contribution is important for the spectral resolution.
- tonal signal components can be estimated relatively well anyway by means of conventional prediction methods. Even with a white noise, which is unfavorable due to its static properties, due to the mentioned suppression of the errors due to possible deviations, the described method gives good results.
- the binaural hearing system 1 of Fig. 1 is the first audio signal 10 in the first filter bank 24 according to the basis of Fig. 2 described method filtered.
- the filtering of the second audio signal 16 in the second filter bank 32 can be done in the same way; however, a conventional filter method-that is to say without estimation of signal components for a respective prediction period of the individual signal blocks-can also be used for this purpose.
- the decision on this is made in particular as a function of the tolerable overall latency of the binaural hearing system 1 and the delay caused by the binaural transmission process.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Neurosurgery (AREA)
- Otolaryngology (AREA)
- Computer Networks & Wireless Communication (AREA)
- Stereophonic System (AREA)
- Compression, Expansion, Code Conversion, And Decoders (AREA)
- Circuit For Audible Band Transducer (AREA)
Description
Die Erfindung betrifft ein Verfahren zur Reduktion der Latenzzeit einer Filterbank zur Filterung eines Audiosignals, wobei aus dem Audiosignal eine Vielzahl von Signal-Blöcken in der Zeitdomäne gebildet wird, wobei für wenigstens eine Mehrzahl der Signal-Blöcke jeweils eine Filterfunktion vorgegeben wird, der Signal-Block mit der vorgegebenen Filterfunktion gefiltert in die Frequenz-Domäne transformiert wird, und hierdurch ein transformierter Signal-Block gebildet wird, und Signalanteile des transformierten Signal-Blocks zur Weiterverarbeitung ausgegeben werden. Die Erfindung betrifft weiter ein Verfahren zum latenzarmen Betrieb eines Hörsystems, wobei aus einem Schallsignal durch einen ersten Eingangswandler ein erstes Audiosignal erzeugt wird, wobei das erste Audiosignal in einer Signalverarbeitungseinheit mittels einer ersten Filterbank gefiltert wird, wobei Signalanteile des gefilterten ersten Audiosignals in der Signalverarbeitungseinheit weiterverarbeitet und zur Erzeugung eines Ausgabesignals verwendet werden, und wobei aus dem Ausgabesignal durch einen Ausgangswandler ein Ausgabe-Schallsignal erzeugt wird.The invention relates to a method for reducing the latency of a filter bank for filtering an audio signal, wherein a plurality of signal blocks in the time domain is formed from the audio signal, wherein in each case a filter function is specified for at least a plurality of the signal blocks, the signal Block is filtered with the given filter function transformed into the frequency domain, and thereby a transformed signal block is formed, and signal components of the transformed signal block are output for further processing. The invention further relates to a method for low-latency operation of a hearing system, wherein a first audio signal is generated from a sound signal by a first input transducer, wherein the first audio signal is filtered in a signal processing unit by means of a first filter bank, wherein signal components of the filtered first audio signal in the signal processing unit further processed and for generating an output signal, and wherein an output sound signal is generated from the output signal by an output transducer.
In einem Hörgerät wird ein von einem Mikrofon erzeugte Audiosignal nach einer Digitalisierung meist von der Zeitdomäne in die Frequenzdomäne transformiert, d. h., das Audiosignal liegt nach der Digitalisierung zunächst in Form zeitaufgelöster Samples vor, welche, ggf. zu einzelnen Signal-Blöcken (sog. "Frames") gruppiert, durch eine Fourier-Transformation wie z.B. FFT in einzelne spektrale Signalanteile des erzeugten Audiosignals zerlegt werden. Dies hat den Vorteil, dass frequenzselektiv Algorithmen wie Störgeräuschreduktion, Richtmikrofonie oder Dynamikkompression angewandt werden können. Die erwähnte Transformation hat jedoch den Nachteil, dass ein nach entsprechender, frequenzselektiver Bearbeitung in die Zeitdomäne zurückgewandeltes Audiosignal eine Verzögerung gegenüber dem Eingangssignal aufweist, die typischerweise in der Größenordnung von mehreren ms liegt. Diese Verzögerung, auch Latenz genannt, ist umso größer, je höher die Auflösung in der Frequenz-Domäne gewählt wird.In a hearing aid, an audio signal generated by a microphone is usually transformed from the time domain into the frequency domain after digitization, ie after the digitization the audio signal initially exists in the form of time-resolved samples which, if necessary, become individual signal blocks (so-called "signal blocks"). Frames "), are decomposed by a Fourier transform such as FFT into individual spectral signal components of the generated audio signal. This has the advantage that frequency-selective algorithms such as noise reduction, directional microphone or dynamic compression can be applied. However, the mentioned transformation has the disadvantage that an audio signal converted back into the time domain after a corresponding frequency-selective processing has a delay with respect to the input signal, which is typically of the order of magnitude of several ms. This delay, also known as latency, is greater the higher the resolution in the frequency domain is chosen.
Viele Schwerhörige leiden vorrangig unter einem Verlust des Hörvermögens bei hohen Frequenzen, beispielsweise einer merklich abgeschwächten Wahrnehmung ab 5-10 kHz, während sie für niedrige Frequenzen kaum eine Abweichung im Vergleich zu einer normal hörenden Person zeigen. In diesen Fällen werden hauptsächlich hohe Frequenzen erheblich verstärkt.Many hard of hearing suffer primarily from a loss of hearing at high frequencies, such as a noticeably weakened perception from 5-10 kHz, while for low frequencies they show little deviation compared to a normal hearing person. In these cases, mainly high frequencies are significantly increased.
Überdies wird hierbei wird auch häufig eine offene Anpassung des Hörgeräts gewählt, in welcher das von einem Lautsprecher des Hörgeräts Ausgabe-Schallsignal über einen Schallschlauch mit Schirmchen oder einen über einen Hörer mit Schirmchen im Gehörgang zum Trommelfell geleitet wird. Am Trommelfell selbst kommt somit eine Mischung aus einem frequenzselektiv gedämpften Direktschall der Umgebung sowie dem vom Hörgerät erzeugten Ausgabe-Schallsignal an. Je nach Hörverlust und Anpassungsart, welche wiederum frequenzabhängig die Dämpfung des Direktschalls von der Umgebung zum Gehör beeinflusst, findet man deshalb unterschiedliche Mischverhältnisse in Abhängigkeit der Frequenz vor.Moreover, an open adaptation of the hearing device is often chosen, in which the sound output from a speaker of the hearing aid output signal via a sound tube with Schirmchen or a listener with Schirmchen in the ear canal is passed to the eardrum. At the eardrum itself thus comes a mixture of a frequency selective attenuated direct sound of the environment as well as the output sound signal generated by the hearing aid. Depending on the hearing loss and the type of adaptation, which in turn influences the attenuation of the direct sound from the environment to the hearing in a frequency-dependent manner, different mixing ratios are therefore found as a function of the frequency.
Bei der Überlagerung korrelierter Signale mit Zeitversatz, wie sie im eben beschriebenen Fall am Trommelfell durch den Direktschall der Umgebung und das Ausgangs-Schallsignal des Hörgeräts vorliegen, treten oftmals Kammfiltereffekte auf. Diese erzeugen charakteristische Amplitudenminima ("Notches") mit gleichem Abstand über der Frequenz, bei welcher eine fast völlige Auslöschung des Signalanteils entsprechender Frequenz stattfindet. Je größer der zeitliche Abstand zwischen beiden überlagerten Signalen, desto geringer ist in der Frequenz-Domäne Abstand dieser Amplitudenminima. Dadurch wird das aus der Überlagerung resultierende Signal verzerrt, es tritt ein röhriger Klang auf. Gerade im Fall der binauralen Audiosignalverarbeitung, wie sie in binauralen Hörsystemen Anwendung findet, ist die Latenz besonders groß und daher die Anfälligkeit für Kammfiltereffekte besonders groß.In the superimposition of correlated signals with time offset, as in the case just described on the eardrum by the direct sound of the environment and the output sound signal of the hearing aid, often occur comb filter effects. These generate characteristic amplitude notches ("notches") at the same distance above the frequency at which an almost complete extinction of the signal component of the corresponding frequency takes place. The greater the time interval between the two superimposed signals, the lower is the frequency domain distance of these amplitude minima. This distorts the signal resulting from the overlay, causing a roaring sound. Especially in the case of binaural audio signal processing, as it is used in binaural hearing systems, the latency is particularly large and therefore the susceptibility to comb filter effects particularly large.
Um diese Kammfiltereffekte möglichst zu vermeiden, ist es also sinnvoll, die gesamte Latenz im binauralen Hörsystem zu reduzieren. Die beschriebenen Probleme mit Kammfiltereffekten sind jedoch nicht an ein binaurales Hörsystem gebunden, sondern können auch in einem monauralen Hörsystem mit nur einem Hörgerät auftreten, in welchem ein Direktschall der Umgebung und ein Ausgangs-Schallsignal eines Hörgeräts mit zeitlichem Versatz überlagert an das Trommelfell des Benutzers gelangen.So to avoid these comb filter effects as possible, so it makes sense to reduce the overall latency in the binaural hearing system. However, the described problems with comb filter effects are not tied to a binaural hearing system, but can also occur in a monaural hearing system with only one hearing aid, in which a direct sound of the environment and an output sound signal of a hearing device overlaid with temporal displacement reach the eardrum of the user ,
Der zeitliche Versatz ist hierbei vorrangig bedingt durch die interne Latenz des Hörsystems zur Signalverarbeitung und hierbei insbesondere in der Filterung.The temporal offset is primarily due to the internal latency of the hearing system for signal processing and in particular in the filtering.
In der
In der
Die
Der Erfindung liegt daher die Aufgabe zugrunde, ein Verfahren für eine möglichst latenzarme spektrale Filterung eines Audiosignals bei möglichst hoher spektraler Auflösung anzugeben. Der Erfindung liegt weiter die Aufgabe zugrunde, ein Verfahren zum möglichst latenzarmen Betrieb eines Hörsystems anzugeben.The invention is therefore based on the object of specifying a method for as low-latency spectral filtering of an audio signal with the highest possible spectral resolution. The invention is further based on the object of specifying a method for low-latency operation of a hearing system.
Die erstgenannte Aufgabe wird erfindungsgemäß gelöst durch ein Verfahren zur Reduktion der Latenzzeit einer Filterbank zur Filterung eines Audiosignals, wobei aus dem Audiosignal eine Vielzahl von Signal-Blöcken in der Zeitdomäne gebildet wird. Hierbei ist vorgesehen, dass für wenigstens eine Mehrzahl der Signal-Blöcke jeweils eine Filterfunktion vorgegeben wird, wenigstens ein Teilintervall des Signal-Blocks als ein Prädiktionszeitraum vorgegeben wird, Signalanteile des Signal-Blocks im wenigstens einen Teilintervall für den Prädiktionszeitraum geschätzt werden, und aus den für den Prädiktionszeitraum geschätzten Signalanteilen und den Signalanteilen des Signal-Blocks außerhalb des Prädiktionszeitraums ein prädizierter Signal-Block erzeugt wird. Weiter ist vorgesehen, dass der prädizierte Signal-Block mit der vorgegebenen Filterfunktion gefiltert in die Frequenz-Domäne transformiert wird, und hierdurch ein transformierter Signal-Block gebildet wird, und Signalanteile des transformierten Signal-Blocks zur Weiterverarbeitung ausgegeben werden.The first object is achieved according to the invention by a method for reducing the latency of a filter bank for filtering an audio signal, wherein a plurality of signal blocks in the time domain is formed from the audio signal. In this case, it is provided that for at least a plurality of the signal blocks in each case a filter function is specified, at least a subinterval of the signal block is given as a prediction period, signal components of the signal block are estimated in the at least one subinterval for the prediction period, and from the for the prediction period of estimated signal components and the signal components of the signal block outside the prediction period, a predicted signal block is generated. It is further provided that the predicted signal block is filtered with the predetermined filter function transformed into the frequency domain, and thereby a transformed signal block is formed, and signal components of the transformed signal block are output for further processing.
Die zweitgenannte Aufgabe wird erfindungsgemäß gelöst durch ein Verfahren zum latenzarmen Betrieb eines Hörsystems, wobei aus einem Schallsignal durch einen ersten Eingangswandler ein erstes Audiosignal erzeugt wird, wobei das erste Audiosignal unmittelbar zu einer Signalverarbeitungseinheit übertragen wird, und in der Signalverarbeitungseinheit unmittelbar mittels einer ersten Filterbank gemäß dem vorbeschriebenen Verfahren zur Reduktion der Latenzzeit einer Filterbank zur Filterung eines Audiosignals gefiltert wird, wobei Signalanteile des gefilterten ersten Audiosignals in der Signalverarbeitungseinheit weiterverarbeitet und zur Erzeugung eines Ausgabesignals verwendet werden, und wobei aus dem Ausgabesignal unmittelbar durch einen Ausgangswandler ein Ausgabe-Schallsignal erzeugt wird. Vorteilhafte und teils für sich gesehen erfinderische Ausgestaltungen sind in den Unteransprüchen und in der nachfolgenden Beschreibung dargelegt.The second object is achieved by a method for low-latency operation of a hearing system, wherein from a sound signal through a first input transducer, a first audio signal is generated, wherein the first audio signal is transmitted directly to a signal processing unit, and in the signal processing unit directly by means of a first filter bank according to the above-described method for reducing the latency of a filter bank for filtering an audio signal is filtered, wherein signal portions of the filtered first audio signal in the signal processing unit further processed and used to generate an output signal, and wherein from the output signal directly by an output transducer, an output sound signal is generated. Advantageous and partly inventive in themselves embodiments are set forth in the subclaims and in the description below.
Bevorzugt wird aus dem Audiosignal ein Signal-Block ("Frame") in der Zeitdomäne gebildet, indem das Audiosignal durch Zeit- und Amplitudendiskretisierung in eine Vielzahl von jeweils aufeinander folgenden Zeitpunkten zugeordneten Amplitudenkennwerten ("Samples") umgewandelt wird, und jeweils eine Vielzahl von aufeinander folgenden Samples zu einem Signal-Block zusammengefasst wird. Die Weiterverarbeitung der Signalanteile des transformierten Signal-Blocks umfasst insbesondere eine frequenzband-abhängige Verstärkung, eine frequenzband-abhängige Richtcharakteristik, eine frequenzband-abhängige Rauschunterdrückung sowie eine Rücktransformation frequenzband-abhängig behandelter Signalanteile in die Zeit-Domäne.Preferably, a signal block ("frame") in the time domain is formed from the audio signal by the audio signal is converted by time and amplitude discretization in a plurality of each successive time points associated amplitude ("samples"), and a plurality of consecutive samples is combined into a signal block. The further processing of the signal components of the transformed signal block comprises in particular a frequency band-dependent amplification, a frequency band-dependent directional characteristic, a frequency band-dependent noise suppression and a backward transformation of frequency band-dependent treated signal components in the time domain.
Das Schätzen der Signalanteile für den Prädiktionszeitraum eines jeweiligen Signal-Blocks erfolgt bevorzugt über einen Prädiktionsalgorithmus, wie z.B. durch ein lineares Prädiktionsfilter. Insbesondere ist auch eine adaptive Anpassung von zur Schätzung verwendeter, zeitkorrelierter Koeffizienten derart möglich, dass ein Schätzkoeffizient, welcher als Koordinate im Signal-Block jeweils einem Sample mit einer bestimmten Zeitverzögerung zuzuordnen ist, in Abhängigkeit des Fehlers zwischen einem geschätzten Sample und einem real aus dem Audiosignal gewonnenen Sample korrigiert wird, wobei die Korrektor in periodischen Abständen erneuert wird. Insbesondere wird ein für einen Signal-Block geschätzter Signalanteil auch für einen später folgenden Signal-Block verwendet, falls der dem Signalanteil entsprechende Zeitraum auch dann noch in den Prädiktionszeitraum des später folgenden Signal-Blocks fällt. Bevorzugt umfasst der Prädiktionszeitraum das jeweils erste und/oder das jeweils letzte Sample eines Signal-Blocks. Insbesondere bildet jeweils in einem Signal-Block der außerhalb des Prädiktionszeitraums liegende Zeitraum ein zusammenhängendes Intervall. Insbesondere umfasst der Prädiktionszeitraum die ersten n Samples und/oder die letzten m Samples, wobei n und m natürliche Zahlen kleiner der Anzahl an Samples im jeweiligen Signal-Block sind.The estimation of the signal components for the prediction period of a respective signal block preferably takes place via a prediction algorithm, such as by a linear prediction filter. In particular, an adaptive adaptation of time-correlated coefficients used for the estimation is possible such that an estimation coefficient, which is assigned as a coordinate in the signal block in each case to a sample with a specific time delay, depending on the error between an estimated sample and a real from the Audio signal obtained sample is corrected, whereby the corrector is renewed at periodic intervals. In particular, a signal component estimated for a signal block becomes is also used for a signal block which follows later, if the period corresponding to the signal component still falls within the prediction period of the signal block following later. The prediction period preferably comprises the respectively first and / or the last sample of a signal block. In particular, the period lying outside the prediction period in each case forms a coherent interval in a signal block. In particular, the prediction period comprises the first n samples and / or the last m samples, where n and m natural numbers are smaller than the number of samples in the respective signal block.
Unter einem Eingangswandler bzw. einem Ausgangswandler des Hörsystems ist jede Form eines akusto-elektrischen bzw. eines elektro-akustischen Wandlers umfasst, beispielsweise ein Mikrofon bzw. ein Lautsprecher. Unter einer unmittelbaren Übertragung des ersten Audiosignals zur Signalverarbeitungseinheit ist zu verstehen, dass die Übertragung des ersten Audiosignals unmittelbar nach dessen Erzeugung stattfindet, also insbesondere ohne eine weitere, über eine Signal-Vorverarbeitung wie z.B. A/D-Wandlung und/oder Datenkompression hinaus gehende Zeitverzögerung stattfindet, wie sie z. B. durch eine langfristige physische Speicherung, welche nicht auf dem FIFO-Prinzip basiert ("first-in-first-out"), eintreten würde. Die Übertragung erfolgt dabei insbesondere dabei lokal innerhalb eines Hörgerätes, beispielsweise auf dem durch die Signalleitungen vorgegebenen Signalweg. Insbesondere erfolgt die Übertragung aber auch drahtlos, beispielsweise von einem ersten Hörgerät eines binauralen Hörsystems zu einem zweiten Hörgerät des binauralen Hörsystems.An input transducer or an output transducer of the hearing system includes any shape of an acousto-electric or an electro-acoustic transducer, for example a microphone or a loudspeaker. Direct transmission of the first audio signal to the signal processing unit is to be understood as meaning that the transmission of the first audio signal takes place immediately after its generation, that is to say in particular without another, via signal preprocessing, such as signal processing. A / D conversion and / or data compression beyond time delay takes place, as z. B. by a long-term physical storage, which is not based on the FIFO principle ("first-in-first-out"), would occur. In this case, the transmission takes place in particular locally within a hearing device, for example on the signal path predetermined by the signal lines. In particular, however, the transmission also takes place wirelessly, for example from a first hearing device of a binaural hearing system to a second hearing device of the binaural hearing system.
Unter einer unmittelbaren Filterung des ersten Audiosignals in der Signalverarbeitungseinheit ist hierzu analog zu verstehen, dass der Filterprozess für das Audiosignal unmittelbar nach dessen Eingang in der Signalverarbeitungseinheit stattfindet, also insbesondere ohne eine weitere, über die direkte Signalübertragung hinaus gehende Zeitverzögerung, wie sie z. B. durch eine langfristige Speicherung, welche nicht auf dem FIFO-Prinzip basiert ("first-in-first-out"), eintreten würde. Ebenso ist unter einer unmittelbaren Erzeugung des Ausgabe-Schallsignals aus dem Ausgabesignal zu verstehen, dass unmittelbar nach der Erzeugung des Ausgabesignals durch die Weiterverarbeitung das Ausgabesignal zum Ausgangswandler zur Ausgabe weitergegeben wird, also insbesondere ohne eine weitere, über die direkte Signalübertragung hinaus gehende Zeitverzögerung, z. B. durch eine langfristige Speicherung.Direct filtering of the first audio signal in the signal processing unit analogously means that the filtering process for the audio signal takes place immediately after its input in the signal processing unit, ie in particular without a further, beyond the direct signal transmission time delay as z. B. by a long-term storage, which is not based on the FIFO principle ("first-in-first-out"), would occur. Likewise, an immediate generation of the output sound signal from the output signal means that immediately after the generation of the output signal is passed through the further processing, the output signal to the output transducer for output, ie in particular without a further, beyond the direct signal transmission outgoing time delay, z. By long-term storage.
In Hörsystemen fällt ein wichtiger Anteil der Latenz auf die Filterbänke, welche zur Transformation der von den Eingangswandlern erzeugten Audiosignale in die Frequenz-Domäne eingesetzt werden (Analyse-Filterbänke), sowie den Filterbänken für die Rücktransformation der frequenzaufgelösten, weiterverarbeiteten Audiosignale in die Zeit-Domäne (Synthese-Filterbänke), wobei erstere meistens einen größeren Anteil aufweisen. Weiter ist bei einem binauralen Hörsystem auch die Übertragung eines Audiosignals von einem Hörgerät zum anderen für die Erzeugung eines binauralen Ausgabesignals mit einer gewissen Verzögerung verbunden. Letztere ist jedoch angesichts der Restriktionen bei der Kodierung zur Übertragung nur schwer zu verringern. Somit ist es auch im Falle eines binauralen Hörsystems vorteilhaft, für einen möglichst latenzarmen Betrieb des Hörsystems die Latenzzeit für die frequenzbandweise Filterung des Audiosignals, also streng genommen des Analyse-Filters für die Transformation in die Frequenz-Domäne, zu reduzieren.In hearing systems, an important portion of the latency falls on the filter banks, which are used to transform the audio signals generated by the input transducers into the frequency domain (analysis filter banks), and the filter banks for the inverse transformation of the frequency resolved, further processed audio signals into the time domain (Synthetic filter banks), the former usually have a larger proportion. Furthermore, in a binaural hearing system, the transmission of an audio signal from one hearing aid to another for the generation of a binaural output signal is associated with a certain delay. However, the latter is difficult to reduce given the restrictions on encoding for transmission. Thus, even in the case of a binaural hearing system, it is advantageous to reduce the latency for the frequency band-wise filtering of the audio signal, that is strictly speaking the analysis filter for the transformation into the frequency domain, for as low-latency operation of the hearing system as possible.
Um die Latenzzeit des Analyse-Filters zu reduzieren, wäre es nun zunächst möglich, die einzelnen Signal-Blöcke, welche jeweils für einen Filterprozess herangezogen werden, kurzer zu wählen, d.h., weniger Samples in einem Signal-Block zu verarbeiten, da für die Verarbeitung eines Signal-Blocks bevorzugt immer erst alle benötigten Samples des Signal-Blocks vorliegen sollten. Da jedoch die Verringerung der Samples in einem Signal-Block eine Verringerung der zu insgesamt im Signal-Block zur Verfügung stehenden Information über die Signalanteile bedeutet, führt diese ohne die Durchführung von korrigierenden Maßnahmen auch zu einer verringerten Frequenzauflösung im transformierten Signal-Block. Dies ist jedoch unerwünscht, da viele Algorithmen zur Signalverarbeitung, welche in Hörsystemen Verwendung finden, für einen im Endergebnis zufrieden stellenden Klangcharakter eine besonders frequenzselektive Anwendung erfordern.In order to reduce the latency of the analysis filter, it would now be possible to choose the individual signal blocks, which are each used for a filtering process, shorter, ie to process fewer samples in a signal block, since for the processing a signal block should preferably always be present only all the required samples of the signal block. However, since the reduction of the samples in a signal block means a reduction of the information about the signal components available overall in the signal block, this also leads to a reduced frequency resolution in the transformed signal block without the implementation of corrective measures. However, this is undesirable since many signal processing algorithms used in hearing aids require a particularly frequency-selective application for a satisfactory sound character in the end result.
Dadurch, dass nun zur Filterung die Signalanteile für den Prädiktionszeitraum eines Signal-Blocks geschätzt werden, anstatt die entsprechenden, real aus dem Audiosignal erzeugten Signalanteile zu verwenden, kann bei einer geeigneten Wahl des Prädiktionszeitraumes die effektive Länge des Signal-Blocks verringert werden, ohne dass hierdurch die Frequenzauflösung der Filterbank beeinträchtigt wird. Die Frequenzauflösung der Filterbank hängt ab vom zeitlichen Informationsgehalt der für den Filterprozess zu verwenden Signal-Blöcke, also von deren Länge. Dadurch, dass nun in einem Signal-Block für einen Zeitraum die Signalanteile geschätzt werden, kann die Latenz der Filterbank um die dem zugehörigen Prädiktionszeitraum entsprechende Dauer verringert werden.By now estimating the signal components for the prediction period of a signal block instead of using the corresponding signal components actually generated from the audio signal, the effective length of the signal block can be reduced with an appropriate choice of the prediction period, without This affects the frequency resolution of the filter bank. The frequency resolution of the filter bank depends on the temporal information content of the signal blocks to be used for the filter process, ie of their length. Since the signal components are now estimated in a signal block for a period of time, the latency of the filter bank can be reduced by the duration corresponding to the associated prediction period.
Vorzugsweise überlappen sich dabei je zwei zeitlich aufeinander folgende Signal-Blöcke teilweise. Die Definition der zeitlichen Abfolge erfolgt hierbei bevorzugt über ein Referenz-Sample für den jeweiligen Signal-Block, z.B. das erste Sample. Die Folge des beschriebenen Überlapps ist, dass die betreffenden, aufeinander folgenden Signal-Blöcke mehrere, bevorzugt aufeinander folgende Samples gemeinsam haben. Dies verbessert einerseits die zeitliche Auflösung in der Frequenz-Domäne, da hierdurch ein häufiges Aktualisieren der frequenzbandweisen Information ermöglicht wird, andererseits kann hierdurch auch der Aufwand beim Schätzen der Signalanteile verringert werden, da bereits geschätzte Signalanteile für einen nachfolgenden Block ohne einen erneuten Schätzvorgang zur Verfügung stehen.Preferably two temporally successive signal blocks overlap each other in part. The definition of the temporal sequence is preferably carried out via a reference sample for the respective signal block, e.g. the first sample. The consequence of the described overlap is that the relevant, successive signal blocks have several, preferably successive, samples in common. On the one hand, this improves the temporal resolution in the frequency domain, since this allows frequent frequency band information to be updated frequently, and on the other hand, the cost of estimating the signal components can be reduced because already estimated signal components are available for a subsequent block without a new estimation process stand.
Zweckmäßigerweise werden jeweils Signalanteile des transformierten Signal-Blocks nach verschiedenen Frequenzbändern getrennt zur Weiterverarbeitung ausgegeben. Für eine derartige Weitergabe ist die durch das Schätzen der Signalanteile der Prädiktionzeiträume reduzierte Latenz der Filterbank bei gleichbleibender hoher Frequenzauflösung besonders vorteilhaft.Expediently, signal portions of the transformed signal block are output separately according to different frequency bands for further processing. For such a transfer, the latency of the filter bank, which is reduced by estimating the signal components of the prediction periods, is particularly advantageous given a constant high frequency resolution.
Bevorzugt weist jeweils die Filterfunktion im Prädiktionszeitraum eine im Mittel geringere Transmissionsamplitude auf als außerhalb des Prädiktionszeitraumes. Dies soll bedeuten, dass der über den ganzen Prädiktionszeitraum gemittelte Wert der Transmissionsamplitude der Filterfunktion geringer ist als der über den restlichen Zeitraum des Signal-Blocks außerhalb des Prädiktionszeitraums gemittelte Wert der Transmissionsamplitude der Filterfunktion. In diesem Fall ist nämlich davon auszugehen, dass bei einer entsprechenden Filterung in die Frequenz-Domäne mittels der Filterfunktion Fehler, welche für den Prädiktionszeitraum durch Abweichungen der Schätzung der Signalanteile von den realen Signalanteilen auftreten können, infolge der im Mittel geringeren Transmissionsamplitude der Filterfunktion weitgehend unterdrückt werden, und somit nicht nennenswert in den transformierten Signal-Block eingehen.In each case, the filter function in the prediction period preferably has an average lower transmission amplitude than outside the prediction period. This is to mean that the value of the transmission amplitude of the filter function averaged over the entire prediction period is lower than that over the remaining one Period of the signal block outside the prediction period averaged value of the transmission amplitude of the filter function. In this case, it can be assumed that with a corresponding filtering in the frequency domain by means of the filter function errors that can occur for the prediction by deviating the estimate of the signal components of the real signal components, largely suppressed due to the average lower transmission amplitude of the filter function and thus do not enter significantly into the transformed signal block.
In einer vorteilhaften Ausgestaltung wird die Transmissionsamplitude der Filterfunktion jeweils durch eine logarithmisch konkave Funktion gebildet, wobei der Prädiktionszeitraum das Maximum der Transmissionsamplitude der Filterfunktion ausspart. Eine logarithmisch konkave Funktion ist definiert als eine Funktion, deren Logarithmus im Definitionsbereich - welcher hier durch die einzelnen Samples des jeweiligen Signal-Blocks gegeben ist - konkav ist. Eine derartige Funktion kann beispielsweise gegeben sein durch eine Approximation einer Gaußschen Glockenkurve über einem endlichen, diskretisierten Definitionsbereich. Der Vorteil des logarithmisch konkaven Verhaltens der Transmissionsamplitude ist, dass diese maximal zwei Wendepunkte im Definitionsbereich aufweist, und somit keinerlei Oszillationen unterworfen ist. Dies hat ein vorteilhaftes Filterverhalten zur Folge, da somit keine an sich relevanten Signalanteile mit einem Minimumswert einer Oszillation der Filterfunktion gefiltert werden.In an advantageous embodiment, the transmission amplitude of the filter function is formed in each case by a logarithmically concave function, wherein the prediction period spans the maximum of the transmission amplitude of the filter function. A logarithmically concave function is defined as a function whose logarithm is concave in the domain of definition - which is given here by the individual samples of the respective signal block. Such a function may for example be given by an approximation of a Gaussian bell curve over a finite, discretized domain of definition. The advantage of the logarithmic concave behavior of the transmission amplitude is that it has a maximum of two inflection points in the domain of definition, and thus is not subject to any oscillations. This results in an advantageous filter behavior, since thus no relevant signal components with a minimum value of an oscillation of the filter function are filtered.
Als besonders zweckmäßig erweist es sich, wenn jeweils der Prädiktionszeitraum nur konvexe Bereiche der Transmissionsamplitude der Filterfunktion beinhaltet. Eine logarithmisch konkave Funktion lässt sich darstellen als eine zu einer bestimmten logarithmisch konvexen Funktion reziproke Funktion. Eine logarithmisch konvexe Funktion ist ihrerseits wiederum konvex. Dies bedeutet, dass die hierzu reziproke, logarithmisch konkave Funktion infolge der Reziprozitätseigenschaft maximal zwei Wendepunkte aufweist.It proves to be particularly expedient if in each case the prediction period contains only convex regions of the transmission amplitude of the filter function. A logarithmic concave function can be represented as a function reciprocal to a particular logarithmic convex function. A logarithmically convex function is in turn convex. This means that the reciprocal, logarithmically concave function due to the reciprocity property has a maximum of two inflection points.
Bei einer geeigneten Wahl der Filterfunktion, beispielsweise einer Approximation einer Gaußschen Glockenkurve, liegt das Maximum der Transmissionsamplitude in einem konvexen Bereich, so dass jenseits der Wendepunkte die Transmissionsamplitude konkav ausläuft. In diesen beiden Bereichen weist die Transmissionsamplitude üblicherweise bereits hinreichend geringe Werte auf, so dass mit der Wahl des Prädiktionszeitraums in wenigstens einem der beiden Bereiche sicher gestellt werden kann, dass Fehler, welche aufgrund der Abweichungen der Schätzung der Signalanteile von den realen Signalanteilen auftreten können, infolge der hinreichend geringeren Transmissionsamplitude der Filterfunktion weitgehend unterdrückt werden, und somit nicht nennenswert in den transformierten Signal-Block eingehen.With a suitable choice of the filter function, for example an approximation of a Gaussian bell curve, the maximum of the transmission amplitude lies in a convex region, so that beyond the inflection points the transmission amplitude runs concavely. In these two ranges, the transmission amplitude usually already has sufficiently low values, so that the choice of the prediction time period in at least one of the two ranges can ensure that errors which can occur due to deviations in the estimation of the signal components from the real signal components, be largely suppressed due to the sufficiently lower transmission amplitude of the filter function, and thus not significantly enter into the transformed signal block.
Als weiter vorteilhaft erweist es sich, wenn für den Prädiktionszeitraum wenigstens eines Signal-Blocks als Signalanteile jeweils ein leeres Signal geschätzt wird. Ein leeres Signal ist hierbei dasjenige Signal, welches für den betreffenden Zeitraum keinerlei Amplitude aufweist. Das Schätzen eines leeren Signals erfolgt insbesondere für den Fall, dass die Signalanteile des Audiosignals, welche für das Schätzverfahren der Signalanteile des Prädiktionszeitraumes verwendet werden, infolge mangelhafter Korrelationen keine qualitativ hinreichend hochwertige Schätzung der Signalanteile zulassen. Dies kann beispielsweise auftreten, wenn im Audiosignal ein hoher Anteil an weißem Rauschen vorliegt, was die Korrelation aufeinander folgender Samples verringert und damit eine Prädiktion erschwert.It proves to be further advantageous if an empty signal is respectively estimated as signal components for the prediction period of at least one signal block. An empty signal is in this case the signal which has no amplitude for the period in question. The estimation of an empty signal is carried out in particular in the event that the signal components of the audio signal which are used for the estimation method of the signal components of the prediction period do not allow a qualitatively sufficiently high-quality estimate of the signal components as a result of insufficient correlations. This can occur, for example, if there is a high proportion of white noise in the audio signal, which reduces the correlation of successive samples and thus makes prediction more difficult.
Insbesondere sind mittels einer Prädiktion geschätzte, vom leeren Signal verschiedene Signalanteile auf die Qualität der Schätzung hin mit den entsprechenden realen Signalanteilen des Audiosignals zu vergleichen, um die Qualität der Prädiktion bewerten zu können. Im Fall einer zu hohen Abweichung - definiert über ein Abweichungsmaß wie z. B. einen über mehrere Samples gemittelten Differenzbetrag und eine zugehörige Oberschranke für das Abweichungsmaß - wird statt der prädizierten Signalanteile ein leeres Signal als für den Prädiktionszeitraum geschätzter Signalanteil festgelegt. Ebenso ist es möglich, die Signalanteile des Audiosignals noch vor der Prädiktion auf Korrelationen hin zu überprüfen, und bei einer zu geringen Korrelation direkt ein leeres Signal als Signalanteil für den Prädiktionszeitraum festzulegen.In particular, by means of a prediction, estimated signal components, which are different from the empty signal, with respect to the quality of the estimate, are to be compared with the corresponding real signal components of the audio signal in order to be able to evaluate the quality of the prediction. In the case of excessive deviation - defined by a deviation measure such. B. an averaged over several samples difference and an associated upper bound for the deviation measure - is determined instead of the predicted signal components, an empty signal as estimated for the prediction signal component. It is also possible to check the signal components of the audio signal for correlations even before the prediction, and to set an empty signal as signal component for the prediction period if the correlation is too low.
In einer weiter vorteilhaften Ausgestaltung des Verfahrens zum latenzarmen Betrieb eines Hörsystems wird aus dem Schallsignal durch einen vom ersten Eingangswandler räumlich getrennten zweiten Eingangswandler ein zweites Audiosignal erzeugt, wobei das zweite Audiosignal unmittelbar zur Signalverarbeitungseinheit übertragen und mittels einer zweiten Filterbank gefiltert wird, und wobei Signalanteile des gefilterten zweiten Audiosignals in der Signalverarbeitungseinheit weiterverarbeitet und zur Erzeugung des Ausgabesignals verwendet werden.In a further advantageous embodiment of the method for low-latency operation of a hearing system, a second audio signal is generated from the sound signal by a second input transducer spatially separated from the first input transducer, the second audio signal is transmitted directly to the signal processing unit and filtered by a second filter bank, and wherein signal components of filtered second audio signal in the signal processing unit further processed and used to generate the output signal.
Insbesondere erfolgt die Filterung des zweiten Audiosignals mittels der zweiten Filterbank gemäß dem vorbeschriebenen Verfahren zur Reduktion der Latenzzeit einer Filterbank zur Filterung eines Audiosignals. Unter einer unmittelbaren Übertragung des zweiten Audiosignals zur Signalverarbeitungseinheit ist zu verstehen, dass die Übertragung des zweiten Audiosignals ohne eine weitere, über eine Signal-Vorverarbeitung wie z.B. A/D-Wandlung und/oder Datenkompression sowie die direkte Signalübertragung hinaus gehende Zeitverzögerung stattfindet, wie sie z. B. durch eine langfristige physikalische Speicherung, welche nicht auf dem FIFO-Prinzip basiert ("first-in-first-out"), eintreten würde.In particular, the filtering of the second audio signal by means of the second filter bank in accordance with the above-described method for reducing the latency of a filter bank for filtering an audio signal. Immediate transmission of the second audio signal to the signal processing unit is understood to mean that the transmission of the second audio signal without another, via a signal preprocessing such. A / D conversion and / or data compression and the direct signal transmission beyond time delay takes place, as z. B. by a long-term physical storage, which would not based on the FIFO principle ("first-in-first-out") would occur.
Diese genannte Ausgestaltung ermöglicht durch das Verfahren insbesondere einen latenzarmen Betrieb eines binauralen Hörsystems unter Berücksichtigung der in einem solchen Hörsystem infolge der für die Erzeugung des binauralen Hörempfindens stattfindenden Signalübertragung von einem Hörgerät zum anderen auftretenden Besonderheiten. Da oftmals bei einem binauralen Hörsystem zur Kompression der reale Informationsgehalt von Signalanteilen des Audiosignals, welches vom jeweils anderen Hörgerät für die Erzeugung des binauralen Hörempfindens empfangen wird, zur bessren Übertragung reduziert wird, beispielsweise durch Datenkompression, ist der durch die Schätzung der Signalanteile im Prädiktionszeitraum mögliche induzierte Fehler in seiner Bedeutung reduziert. Bei diesem Audiosignal findet durch die Übertragung bereits ein Informationsverlust statt, so dass die Abweichungen durch die Schätzung für den Prädiktionszeitraum keine zusätzliche kumulative, sondern nur eine als alternativ zu betrachtende Fehlerquelle darstellen. Kurz gesagt macht es wenig aus, ob ein Fehler statistisch durch die Datenkompression oder die Schätzung erfolgt.This named embodiment makes possible, in particular, a low-latency operation of a binaural hearing system, taking into account the special features occurring in such a hearing system as a result of the signal transmission from one hearing aid to another occurring for the generation of the binaural auditory sensation. Since, in the case of a binaural hearing system for compression, the real information content of signal portions of the audio signal received by the respective other hearing device for the generation of the binaural hearing sensation is often reduced for better transmission, for example by data compression, this is possible by estimating the signal components in the prediction period reduced errors in its meaning. In the case of this audio signal, there is already a loss of information due to the transmission, so that the deviations by the estimation for the prediction period do not represent an additional cumulative source of error but only an alternative source of error to be considered as an alternative. In short, it makes little difference whether an error is made statistically by the data compression or the estimation.
Ein weiterer Vorteil der Anwendung des Verfahrens zum latenzarmen Betrieb eines binauralen Hörsystems ist, dass durch die beschriebene Übertragung der Audiosignale bereits eine gewisse Latenz von mehreren ms ins Hörsystem eingeführt wird. Die Reduktion weitere möglicher Latenzen, wie z.B. im vorliegenden Fall durch die Filterbänke, hilft hier, die Verluste der Klangqualität durch Kammfiltereffekte möglichst gering zu halten.A further advantage of using the method for low-latency operation of a binaural hearing system is that a certain latency of several ms is already introduced into the hearing system through the described transmission of the audio signals. The reduction of further possible latencies, e.g. in the present case through the filter banks, here helps to minimize the losses of the sound quality by comb filter effects as low as possible.
Die Erfindung nennt weiter ein Hörgerät, umfassend wenigstens einen Eingangswandler zur Erzeugung eines Audiosignals, einen Ausgangswandler zur Erzeugung eines Ausgangs-Schallsignals, sowie eine lokalen Signalverarbeitungseinheit mit einer ersten Filterbank, welches zur Durchführung des vorbeschriebenen Verfahrens zur Reduktion der Latenzzeit einer Filterbank zur Filterung eines Audiosignals eingerichtet ist. Die für das Verfahren und seine Weiterbildungen angegebenen Vorteile können dabei sinngemäß auf das Hörgerät übertragen werden.The invention further provides a hearing aid, comprising at least one input transducer for generating an audio signal, an output transducer for generating an output sound signal, and a local signal processing unit having a first filter bank, which for carrying out the above-described method for reducing the latency of a filter bank for filtering an audio signal is set up. The advantages stated for the method and its developments can be transferred analogously to the hearing aid.
Die Erfindung nennt zudem ein binaurales Hörsystem mit zwei vorbeschriebenen Hörgeräten, welches zur Durchführung des Verfahrens zum latenzarmenBetrieb eines Hörsystems mit wenigstens zwei Eingangswandlern eingerichtet ist. Die für das Verfahren und seine Weiterbildungen angegebenen Vorteile können dabei sinngemäß auf das binaurale Hörsystem übertragen werden.The invention also mentions a binaural hearing system with two hearing aids as described above, which is set up to carry out the method for low-latency operation of a hearing system with at least two input transducers. The advantages specified for the method and its developments can be transferred analogously to the binaural hearing system.
Nachfolgend wird ein Ausführungsbeispiel der Erfindung anhand einer Zeichnung näher erläutert. Hierbei zeigen jeweils schematisch:
- Fig. 1
- in einem Blockdiagram ein binaurales Hörsystem mit zwei Hörgeräten, und
- Fig. 2
- in einer Zeitdarstellung ein von einem Hörgerät nach
Fig. 1 erzeugtes Audiosignal und in einer Ausschnittdarstellung ein Signal-Block des Audiosignals mit einer Filterfunktion und einem Prädiktionszeitraum.
- Fig. 1
- in a block diagram a binaural hearing system with two hearing aids, and
- Fig. 2
- in a time representation of a hearing aid
Fig. 1 generated audio signal and in a detail representation of a signal block of the audio signal with a filter function and a prediction period.
Einander entsprechende Teile und Größen sind in allen Figuren jeweils mit gleichen Bezugszeichen versehen.Corresponding parts and sizes are provided in all figures with the same reference numerals.
In
Das erste Audiosignal 10 wird unmittelbar nach der lokalen Signal-Vorverarbeitung 18 zunächst in einem binauralen Übertragungsprozess 22 vom ersten Hörgerät 2 zum zweiten Hörgerät 4 übertragen, wo es in einer Signalverarbeitungseinheit 24 in einer ersten Filterbank 26 in noch zu beschreibenden Weise gefiltert wird. Der binaurale Übertragungsprozess 22 erfolgt dabei unmittelbar nach der lokalen-Signal-Vorverarbeitung 18, also insbesondere ohne weitere Verzögerung, insbesondere ohne längerfristige Speicher- und erneute Ladevorgänge der betreffenden Signalanteile über einen FIFO-Speicher hinaus. Auf das gefilterte erste Audiosignal 28 werden nun frequenzbandweise Signalverarbeitungsalgorithmen 30 wie z.B. Rauschunterdrückung, Richtmikrofonie oder Dynamikkompression angewandt.The
Das zweite Audiosignal 16 wird unmittelbar nach der lokalen Signal-Vorverarbeitung 20 der Signalverarbeitungseinheit 24 zugeführt, wo es zunächst in einer zweiten Filterbank 32 in noch zu beschreibenden Weise gefiltert wird, wobei als ein gefiltertes zweites Audiosignal 34 die jeweiligen Signalanteile in einzelnen Frequenzbändern getrennt weitergegeben werden. Im aus der zweiten Filterbank 32 resultierenden gefilterten zweiten Audiosignal 34 sind die jeweiligen Signalanteile in einzelnen Frequenzbändern getrennt ausgegeben. Auch auf das gefilterte zweite Audiosignal 34 werden nun frequenzbandweise Signalverarbeitungsalgorithmen 28 wie z.B. Rauschunterdrückung, Richtmikrofonie oder Dynamikkompression angewandt. Aus dem gefilterten ersten Audiosignal 26 und dem gefilterten zweiten Audiosignal 34 wird nach der frequenzbandweisen Signalverarbeitung 28 ein Ausgangsignal 36 erzeugt, welches lokal das binaurale Hörempfinden am Ort des zweiten Hörgeräts 4 wiederspiegelt.The second audio signal 16 is supplied immediately after the local signal pre-processing 20 of the
Das Ausgangssignal 36 wird unmittelbar, also insbesondere ohne weitere längerfristige Speicher- und erneute Ladevorgänge der Signalanteile, von einem als Lautsprecher 38 ausgestalteten Ausgangswandler 40 in ein Ausgabe-Schallsignal 42 umgewandelt.The
In
Für den Signal-Block 50c sind gegen eine Zeitachse t' die einzelnen realen Signalanteile 52a, 52b gezeigt. Die realen Signalanteile 52a, 52b sind dabei jeweils gegeben durch die Amplitude des entsprechenden Samples. Weiter ist für den Signal-Block 50c die Transmissionsamplitude 54c der Filterfunktion 56c gezeigt, welche im vorliegenden Fall näherungsweise gegeben ist durch eine Gaußsche Glockenkurve.For the
Die Filterfunktion 56c stellt hierbei eine Fensterfunktion dar, mit der die Ränder des Signal-Blocks 50c für die Transformation in die Frequenz-Domäne geglättet "ausgeblendet" werden sollen. Dies erfolgt, da ohne eine derartige Fensterfunktion die Fourier-Transformation der Signalanteile des Signal-Blocks 50c de facto eine Fourier-Transformation der Signalanteile des ersten Audiosignals 10 ist, welche mit einer Rechteckfunktion entsprechend der Dauer des Signal-Blocks multipliziert werden. Infolge des Faltungstheorems bedeutet diese Multiplikation in der Zeit-Domäne eine Faltung der Frequenzanteile des ersten Audiosignals 10 mit der Fourier-Transformierten der Rechteckfunktion, welche gegeben ist durch eine stark oszillierende sin(x)/x- bzw. Sinc-Funktion. Um derartige Oszillationen zu vermeiden, werden die Ränder des Signal-Blocks 50c für die Transformation in die Frequenz-Domäne mittels einer geeigneten Filterfunktion 56c "ausgeblendet". Dies geschieht, indem die Transmissionsamplitude 54c der Filterfunktion 56c an den Rändern des Signal-Blocks 50c möglichst oszillationsfrei, also insbesondere mit möglichst wenigen Wendepunkten, gegen Null konvergiert. Eine Funktion mit derartigen Eigenschaften ist insbesondere gegeben durch eine logarithmisch konkave Funktion wie z.B. die approximierte Gaußsche Glockenkurve des vorliegenden Falls.The
Der beschriebene Verlauf der Transmissionsamplitude 54c der Filterfunktion 56c kann nun dazu ausgenutzt werden, die Latenz der ersten Filterbank 24 zu verringern, ohne dabei an Auflösungsvermögen in der Frequenz-Domäne einzubüßen. Hierfür wird ein Teilintervall 58c am zeitlichen Ende des Signal-Blocks 50c als ein Prädiktionszeitraum 60c definiert. Das Teilintervall 58c liegt jenseits des Wendepunktes 62c der Transmissionsamplitude 54c, also insbesondere weit abseits des Maximums 64c der Transmissionsamplitude 54c, so dass im Teilintervall 58c, welches den Prädiktionszeitraum 60c definiert, die Transmissionsamplitude 54c nur noch geringe Werte aufweist. Für den Prädiktionszeitraum 60c werden nun mittels eines Prädiktionsalgorithmus, z.B. eines linearen Prädiktionsfilters, statt der realen Signalanteile 52b dort die für die Transformation zu verwendenden Signalanteile geschätzt. Die im Prädiktionszeitraum 60c geschätzten Signalanteile 66b und die Signalanteile 52a des Signal-Blocks 50c außerhalb des Prädiktionszeitraums 60c bilden nun einen prädizierten Signal-Block 68c.The described profile of the
Dieser prädizierte Signal-Block 68c wird nun mit der Filterfunktion 56c multipliziert, und mittels einer schnellen Fourier-Transformation in die Frequenz-Domäne transformiert, so dass dort die frequenzaufgelöste Information des transformierten Signal-Blocks 50c für eine Weiterverarbeitung mittels Frequenzband-abhängiger Signalverarbeitungsalgorithmen zur Verfügung steht. Auch für die anderen Signal-Blöcke 50a, 50b, 50d-f erfolgt das beschriebene Vorgehen, Signalanteile für einen anhand der jeweils zu verwendenden Filterfunktion günstig zu wählenden Prädiktionszeitraum zu schätzen, um so die Latenz für die Transformation in die Frequenz-Domäne zu verringern, da dann die jeweils letzten Samples eines Signal-Blocks noch gar nicht vorzuliegen brauchen, so dass mit der Transformation infolge der Schätzung mehrere ms früher begonnen werden kann.This predicted
Eine wichtige Rolle spielt hierbei der Verlauf der Transmissionsamplitude 54c der Filterfunktion 56c. Ein möglicher Fehler, welcher durch die Abweichung der für den Prädiktionszeitraum 60c geschätzten Signalanteile 66b von den realen Signalanteilen 52b ergeben könnte, wird dadurch unterdrückt, dass für den Prädiktionszeitraum 60c die Transmissionsamplitude 54c bezogen auf ihr Maximum 64c nur noch vergleichsweise geringe Werte aufweist, und somit durch die entsprechende Multiplikation mit der Filterfunktion 56c die geschätzten Signalanteile 66b ohnehin nur einen geringen Beitrag zum transformierten Signal-Block leisten. Dieser Beitrag ist jedoch für die spektrale Auflösung wichtig. Insbesondere tonale Signalanteile lassen sich ohnehin mittels üblicher Prädiktionsmethoden relativ gut schätzen. Selbst bei einem weißen Rauschen, welches infolge seiner statischen Eigenschaften ungünstig zu schätzen ist, liefert infolge der genannten Unterdrückung der Fehler durch eventuelle Abweichungen das beschriebene Verfahren gute Resultate.An important role in this case plays the course of the
Im binauralen Hörsystem 1 der
Obwohl die Erfindung im Detail durch das bevorzugte Ausführungsbeispiel näher illustriert und beschrieben wurde, ist die Erfindung nicht durch dieses Ausführungsbeispiel eingeschränkt. Andere Variationen können vom Fachmann hieraus abgeleitet werden, ohne den Schutzumfang der Erfindung zu verlassen.Although the invention has been illustrated and described in detail by the preferred embodiment, the invention is not limited by this embodiment. Other variations can be deduced therefrom by those skilled in the art without departing from the scope of the invention.
- 11
- Binaurales HörsystemBinaural hearing system
- 22
- erstes Hörgerätfirst hearing aid
- 44
- zweites Hörgerätsecond hearing aid
- 66
- Mikrofonmicrophone
- 88th
- erster Eingangswandlerfirst input converter
- 99
- Schallsignalsound signal
- 1010
- erstes Audiosignalfirst audio signal
- 1212
- Mikrofonmicrophone
- 1414
- zweiter Eingangswandlersecond input converter
- 1616
- zweites Audiosignalsecond audio signal
- 1818
- lokale Signal-Vorverarbeitunglocal signal preprocessing
- 2020
- lokale Signal-Vorverarbeitunglocal signal preprocessing
- 2222
- binauraler Übertragungsprozessbinaural transmission process
- 2424
- SignalverarbeitungseinheitSignal processing unit
- 2626
- erste Filterbankfirst filter bank
- 2828
- gefiltertes erstes Audiosignalfiltered first audio signal
- 3030
- frequenzbandweise Signalverarbeitungfrequency bandwise signal processing
- 3232
- zweite Filterbanksecond filter bank
- 3434
- gefiltertes zweites Audiosignalfiltered second audio signal
- 3636
- Ausgangssignaloutput
- 3838
- Lautsprecherspeaker
- 4040
- Ausgangswandleroutput transducer
- 4242
- Ausgabe-SchallsignalOutput sound signal
- 50a-f50a-f
- Signal-BlockSignal block
- 52a, b52a, b
- reale Signalanteilereal signal components
- 54c54c
- Transmissionsamplitudetransmission amplitude
- 56c56c
- Filterfunktionfilter function
- 58c58c
- Teilintervallsubinterval
- 60c60c
- PrädiktionszeitraumPrädiktionszeitraum
- 62c62c
- Wendepunktturning point
- 64c64c
- Maximummaximum
- 66b66b
- geschätzte Signalanteileestimated signal components
- 68c68c
- prädizierte Signal-Blockpredicted signal block
- t, t't, t '
- Zeitachsetimeline
Claims (11)
- Method for reducing the latency period of a filter bank (26, 32) for filtering an audio signal (10, 16), wherein a large number of signal blocks (50a-f) in the time domain are formed from the audio signal (10, 16), wherein for at least a plurality of the signal blocks (50a-f) in each instance- a filter function (56c) is predetermined,- at least one partial interval (58c) of the signal block (50a-f) is predetermined as a prediction period (60c),- signal components (66b) of the signal block (50a-f) in the at least one partial interval (58c) are estimated for the prediction period (60c), and a predicted signal block (68c) is generated from the signal components (66b) estimated for the prediction period (60c) and from the signal components (52a) of the signal block (50a-f) outside the prediction period (60c), and- the predicted signal block (68c), filtered with the predetermined filter function (56c), is transformed into the frequency domain, and by this means a transformed signal block is formed, and- signal components of the transformed signal block are output for further processing.
- Method according to Claim 1,
wherein each two temporally consecutive signal blocks (50a-f) partially overlap. - Method according to Claim 1 or Claim 2,
wherein in each instance signal components of the transformed signal block according to various frequency bands are output separately for further processing (30) . - Method according to one of the preceding claims,
wherein in each instance the filter function (56c) exhibits a smaller - on average - transmission amplitude (54c) within the prediction period (60c) than outside the prediction period (60c). - Method according to Claim 4,
wherein the transmission amplitude (54c) of the filter function (56c) is constituted in each instance by a logarithmically concave function, and
wherein the prediction period (60c) omits the maximum (64c) of the transmission amplitude (54c) of the filter function (56c). - Method according to Claim 5,
wherein in each instance the prediction period (60c) includes only convex regions of the transmission amplitude (54c) of the filter function (56c). - Method according to one of the preceding claims,
wherein a blank signal is estimated in each instance as signal components (66b) for the prediction period (60c) of at least one signal block (50a-f). - Method for low-latency operation of a hearing system (1),
wherein a first audio signal (10) is generated from an acoustic signal (9) by a first input transducer (8), wherein the first audio signal (10) is transmitted immediately to a signal-processing unit (24) and filtered immediately in the signal-processing unit (24) by means of a first filter bank (26) in accordance with a method according to one of the preceding claims, wherein signal components of the filtered first audio signal (28) are subjected to further processing (30) in the signal-processing unit (24) and used for generating an output signal (36), and
wherein an output acoustic signal (42) is generated immediately from the output signal (36) by an output transducer (40). - Method according to Claim 8,
wherein a second audio signal (16) is generated from the acoustic signal (9) by a second input transducer (14) spatially separated from the first input transducer (8),
wherein the second audio signal (16) is transmitted immediately to the signal-processing unit (24) and filtered by means of a second filter bank (32), and wherein signal components of the filtered second audio signal (16) are subjected to further processing in the signal-processing unit (24) and used for generating the output signal (36). - Hearing aid (2, 4), comprising at least one input transducer (8, 14) for generating an audio signal (10, 16), an output transducer (40) for generating an output acoustic signal (42), and also a signal-processing unit (24) with a first filter bank (26), which hearing aid has been set up to implement the method according to Claim 8 or Claim 9.
- Binaural hearing system (1) with two hearing aids (2, 4) according to Claim 10, which has been set up to implement the method according to Claim 9.
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
DE102016200637.1A DE102016200637B3 (en) | 2016-01-19 | 2016-01-19 | Method for reducing the latency of a filter bank for filtering an audio signal and method for low-latency operation of a hearing system |
Publications (2)
Publication Number | Publication Date |
---|---|
EP3197181A1 EP3197181A1 (en) | 2017-07-26 |
EP3197181B1 true EP3197181B1 (en) | 2018-09-26 |
Family
ID=57714385
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP16204529.8A Active EP3197181B1 (en) | 2016-01-19 | 2016-12-15 | Method for reducing latency of a filter bank for filtering an audio signal and method for low latency operation of a hearing system |
Country Status (5)
Country | Link |
---|---|
US (1) | US10142741B2 (en) |
EP (1) | EP3197181B1 (en) |
CN (1) | CN106982409B (en) |
DE (1) | DE102016200637B3 (en) |
DK (1) | DK3197181T3 (en) |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
DE102018206689A1 (en) * | 2018-04-30 | 2019-10-31 | Sivantos Pte. Ltd. | Method for noise reduction in an audio signal |
DE102018207780B3 (en) | 2018-05-17 | 2019-08-22 | Sivantos Pte. Ltd. | Method for operating a hearing aid |
CN112218221B (en) * | 2020-10-21 | 2022-06-03 | 歌尔智能科技有限公司 | Hearing aid adapter and control method |
US11330376B1 (en) | 2020-10-21 | 2022-05-10 | Sonova Ag | Hearing device with multiple delay paths |
DE102021205251B4 (en) | 2021-05-21 | 2024-08-08 | Sivantos Pte. Ltd. | Method and device for frequency-selective processing of an audio signal with low latency |
DE102023200581A1 (en) | 2023-01-25 | 2024-07-25 | Sivantos Pte. Ltd. | Method for operating a hearing instrument |
Family Cites Families (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP3636361B2 (en) * | 1992-07-07 | 2005-04-06 | レイク・テクノロジイ・リミテッド | Digital filter with high accuracy and high efficiency |
SE517525C2 (en) * | 1999-09-07 | 2002-06-18 | Ericsson Telefon Ab L M | Method and apparatus for constructing digital filters |
US7277554B2 (en) * | 2001-08-08 | 2007-10-02 | Gn Resound North America Corporation | Dynamic range compression using digital frequency warping |
US8494193B2 (en) * | 2006-03-14 | 2013-07-23 | Starkey Laboratories, Inc. | Environment detection and adaptation in hearing assistance devices |
DK2027750T3 (en) * | 2006-06-13 | 2012-05-07 | Phonak Ag | Method and system for detecting acoustic shock and using this method in hearing aids |
TWI458258B (en) * | 2009-02-18 | 2014-10-21 | Dolby Int Ab | Low delay modulated filter bank and method for the design of the low delay modulated filter bank |
CN102256200A (en) * | 2010-05-19 | 2011-11-23 | 上海聪维声学技术有限公司 | WOLA (Weighted-Overlap Add) filter bank based signal processing method for all-digital hearing aid |
EP2521377A1 (en) * | 2011-05-06 | 2012-11-07 | Jacoti BVBA | Personal communication device with hearing support and method for providing the same |
CA2925734C (en) * | 2013-10-18 | 2018-07-10 | Guillaume Fuchs | Coding of spectral coefficients of a spectrum of an audio signal |
EP2897382B1 (en) * | 2014-01-16 | 2020-06-17 | Oticon A/s | Binaural source enhancement |
DE102014204557A1 (en) * | 2014-03-12 | 2015-09-17 | Siemens Medical Instruments Pte. Ltd. | Transmission of a wind-reduced signal with reduced latency |
EP3245798B1 (en) * | 2015-01-14 | 2018-07-11 | Widex A/S | Method of operating a hearing aid system and a hearing aid system |
-
2016
- 2016-01-19 DE DE102016200637.1A patent/DE102016200637B3/en not_active Expired - Fee Related
- 2016-12-15 EP EP16204529.8A patent/EP3197181B1/en active Active
- 2016-12-15 DK DK16204529.8T patent/DK3197181T3/en active
-
2017
- 2017-01-16 CN CN201710029676.7A patent/CN106982409B/en active Active
- 2017-01-19 US US15/409,701 patent/US10142741B2/en active Active
Non-Patent Citations (1)
Title |
---|
None * |
Also Published As
Publication number | Publication date |
---|---|
US10142741B2 (en) | 2018-11-27 |
CN106982409A (en) | 2017-07-25 |
DE102016200637B3 (en) | 2017-04-27 |
DK3197181T3 (en) | 2019-01-21 |
US20170208397A1 (en) | 2017-07-20 |
CN106982409B (en) | 2019-11-26 |
EP3197181A1 (en) | 2017-07-26 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3197181B1 (en) | Method for reducing latency of a filter bank for filtering an audio signal and method for low latency operation of a hearing system | |
DE69319494T2 (en) | Encoding device for audio signals and method therefor | |
EP1853089B1 (en) | Method for elimination of feedback and for spectral expansion in hearing aids. | |
EP1005695B1 (en) | Method and device for detecting a transient in a discrete-time audiosignal, and device and method for coding an audiosignal | |
DE10017646A1 (en) | Noise suppression in the time domain | |
DE2207141C3 (en) | Circuit arrangement for suppressing unwanted speech signals by means of a predictive filter | |
EP1386307B1 (en) | Method and device for determining a quality measure for an audio signal | |
EP3565270B1 (en) | Method for noise suppression in an audio signal | |
EP3065417B1 (en) | Method for suppressing interference noise in an acoustic system | |
EP3454572A1 (en) | Method for detection of a defect in a listening instrument | |
EP1189419B1 (en) | Method and device for eliminating the loudspeaker interference on microphone signals | |
EP3926982A2 (en) | Method for direction-dependent noise suppression for a hearing system comprising a hearing device | |
EP2981099B1 (en) | Method and device for suppressing feedback | |
DE102015204253B4 (en) | Method for frequency-dependent noise suppression of an input signal and hearing aid | |
EP3355592A1 (en) | Method for operating a binaural hearing aid system | |
DE602004006912T2 (en) | A method for processing an acoustic signal and a hearing aid | |
EP1351550B1 (en) | Method for adapting a signal amplification in a hearing aid and a hearing aid | |
EP3403260B1 (en) | Method and apparatus for conditioning an audio signal subjected to lossy compression | |
EP3340656A1 (en) | Method for operating a hearing aid | |
EP3048813B1 (en) | Method and device for suppressing noise based on inter-subband correlation | |
EP2190218B1 (en) | Filter bank system with specific stop-band attenuation for a hearing device | |
DE102005015647A1 (en) | compander | |
DE102018207780B3 (en) | Method for operating a hearing aid | |
DE102016212393A1 (en) | Computationally effective data rate mismatching for telephone clocks | |
EP4235664A1 (en) | Method for suppressing an acoustic reverberation in an audio signal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
AX | Request for extension of the european patent |
Extension state: BA ME |
|
17P | Request for examination filed |
Effective date: 20171127 |
|
RBV | Designated contracting states (corrected) |
Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
RAP1 | Party data changed (applicant data changed or rights of an application transferred) |
Owner name: SIVANTOS PTE. LTD. |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: H04R 25/00 20060101AFI20180209BHEP |
|
INTG | Intention to grant announced |
Effective date: 20180308 |
|
GRAJ | Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted |
Free format text: ORIGINAL CODE: EPIDOSDIGR1 |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
INTG | Intention to grant announced |
Effective date: 20180706 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
REG | Reference to a national code |
Ref country code: GB Ref legal event code: FG4D Free format text: NOT ENGLISH |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: REF Ref document number: 1047470 Country of ref document: AT Kind code of ref document: T Effective date: 20181015 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: FG4D Free format text: LANGUAGE OF EP DOCUMENT: GERMAN |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R096 Ref document number: 502016002092 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: NV Representative=s name: E. BLUM AND CO. AG PATENT- UND MARKENANWAELTE , CH |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 Effective date: 20190115 |
|
REG | Reference to a national code |
Ref country code: NL Ref legal event code: MP Effective date: 20180926 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: NO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181226 Ref country code: GR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181227 Ref country code: RS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: FI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: BG Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20181226 Ref country code: LT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
REG | Reference to a national code |
Ref country code: LT Ref legal event code: MG4D |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: LV Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: HR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: PL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: EE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: NL Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: ES Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: IS Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190126 Ref country code: CZ Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: RO Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: IT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: SK Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: PT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20190126 Ref country code: SM Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R097 Ref document number: 502016002092 Country of ref document: DE |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MC Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: LU Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181215 |
|
26N | No opposition filed |
Effective date: 20190627 |
|
REG | Reference to a national code |
Ref country code: IE Ref legal event code: MM4A |
|
REG | Reference to a national code |
Ref country code: BE Ref legal event code: MM Effective date: 20181231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: IE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181215 Ref country code: SI Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: BE Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20181231 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: MT Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: TR Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: CY Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: HU Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO Effective date: 20161215 Ref country code: SE Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT Effective date: 20180926 Ref country code: MK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20180926 |
|
REG | Reference to a national code |
Ref country code: AT Ref legal event code: MM01 Ref document number: 1047470 Country of ref document: AT Kind code of ref document: T Effective date: 20211215 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: AT Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20211215 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: GB Payment date: 20231220 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: FR Payment date: 20231219 Year of fee payment: 8 Ref country code: DK Payment date: 20231219 Year of fee payment: 8 Ref country code: DE Payment date: 20231214 Year of fee payment: 8 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20240110 Year of fee payment: 8 |