EP1441562B1 - Method for frequency transposition and use of the method in a hearing device and a communication device - Google Patents
Method for frequency transposition and use of the method in a hearing device and a communication device Download PDFInfo
- Publication number
- EP1441562B1 EP1441562B1 EP20040005270 EP04005270A EP1441562B1 EP 1441562 B1 EP1441562 B1 EP 1441562B1 EP 20040005270 EP20040005270 EP 20040005270 EP 04005270 A EP04005270 A EP 04005270A EP 1441562 B1 EP1441562 B1 EP 1441562B1
- Authority
- EP
- European Patent Office
- Prior art keywords
- frequency
- signal
- function
- transposition
- bin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Lifetime
Links
- 230000017105 transposition Effects 0.000 title claims description 77
- 238000000034 method Methods 0.000 title claims description 42
- 230000013707 sensory perception of sound Effects 0.000 title claims description 36
- 238000004891 communication Methods 0.000 title claims description 18
- 238000007906 compression Methods 0.000 claims description 94
- 230000006835 compression Effects 0.000 claims description 87
- 230000006870 function Effects 0.000 claims description 42
- 238000012545 processing Methods 0.000 claims description 30
- 238000001228 spectrum Methods 0.000 claims description 26
- 230000009466 transformation Effects 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 12
- 230000008447 perception Effects 0.000 claims description 12
- 238000013507 mapping Methods 0.000 claims description 10
- 230000001131 transforming effect Effects 0.000 claims description 4
- 230000002194 synthesizing effect Effects 0.000 claims description 3
- 230000000638 stimulation Effects 0.000 claims description 2
- 230000005236 sound signal Effects 0.000 description 13
- 230000005540 biological transmission Effects 0.000 description 11
- 208000032041 Hearing impaired Diseases 0.000 description 6
- 230000006837 decompression Effects 0.000 description 6
- 238000005070 sampling Methods 0.000 description 5
- 230000003595 spectral effect Effects 0.000 description 5
- 208000016354 hearing loss disease Diseases 0.000 description 4
- 230000003044 adaptive effect Effects 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 239000011295 pitch Substances 0.000 description 3
- 206010011878 Deafness Diseases 0.000 description 2
- 206010011891 Deafness neurosensory Diseases 0.000 description 2
- 208000009966 Sensorineural Hearing Loss Diseases 0.000 description 2
- 230000003321 amplification Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 230000015572 biosynthetic process Effects 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 210000003027 ear inner Anatomy 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 231100000888 hearing loss Toxicity 0.000 description 2
- 230000010370 hearing loss Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 238000003199 nucleic acid amplification method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 231100000879 sensorineural hearing loss Toxicity 0.000 description 2
- 208000023573 sensorineural hearing loss disease Diseases 0.000 description 2
- 238000003786 synthesis reaction Methods 0.000 description 2
- 238000005303 weighing Methods 0.000 description 2
- 240000006409 Acacia auriculiformis Species 0.000 description 1
- 238000007792 addition Methods 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000001627 detrimental effect Effects 0.000 description 1
- 210000000613 ear canal Anatomy 0.000 description 1
- 210000000959 ear middle Anatomy 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000005284 excitation Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 210000000867 larynx Anatomy 0.000 description 1
- 230000000116 mitigating effect Effects 0.000 description 1
- 238000002156 mixing Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000008054 signal transmission Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000000844 transformation Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/35—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using translation techniques
- H04R25/353—Frequency, e.g. frequency shift or compression
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0316—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude
- G10L21/0364—Speech enhancement, e.g. noise reduction or echo cancellation by changing the amplitude for improving intelligibility
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/06—Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids
- G10L2021/065—Aids for the handicapped in understanding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R2225/00—Details of deaf aids covered by H04R25/00, not provided for in any of its subgroups
- H04R2225/43—Signal processing in hearing aids to enhance the speech intelligibility
Definitions
- the present invention relates to a method for frequency transposition in a hearing device or in a communication device, respectively, according to the pre-characterizing part of claim 1, to a use of the method for a binaural hearing device, to a device according to the pre-characterizing part of claim 15 as well as to a use of the device in a communication device or in a hearing device.
- frequency shifting The many techniques for frequency transposition reported previously can be subdivided into three broad types: frequency shifting, frequency compression, and reducing the playback speed of recorded audio signals while discarding portions of the signal in order to preserve the original duration.
- frequency-transposing hearing instruments available commercially are those manufactured by AVR Ltd., a company based in Israel and Minnesota, USA (see http://www.avrsono.com).
- An instrument produced previously by AVR known as the TranSonic, has been superseded recently by the ImpaCt and Logicom-20 devices. All of these frequency-transposition instruments are based on the selective reduction of the playback speed of recorded audio signals. This is achieved by first sampling the input sound signal at a particular rate, and then storing it in a memory. When the recorded signal is subsequently read out of the memory, the sampling rate is reduced when frequency-lowering is required. Because the sampling rate can be changed, it is possible to apply frequency lowering selectively.
- the reported algorithm multiplies each frequency bin by a constant factor (less than 1) to produce the desired output signal in the frequency domain.
- Data loss resulting from this compression of the spectrum is minimized by linear interpolation across frequencies.
- the output signal is then converted back into the time domain by means of an inverse FFT (IFFT).
- IFFT inverse FFT
- One disadvantage of this technique is that it is very inefficient computationally due to the large size of the FFT, and would consume too much electrical energy if implemented in a hearing device.
- propagation delay of signals processed by this algorithm would be unacceptably long for hearing device users, potentially resulting in some interference with their lip-reading ability.
- the compression capabilities i.e. the range of the compression ratio
- linear, compression scheme are limited due to the applied proportional, i.e. linear, compression scheme.
- a feature extraction and signal resynthesis procedure and system based on a vocoder have been described by Thomson CSF, Paris in EP-1 006 511 .
- Information about pitch, voicing, energy, and spectral shape is extracted from the input signal.
- These features are modified (e.g. by compressing the formant frequencies in the frequency domain) and then used for synthesis of the output signal by means of a vocoder (i.e. a relatively efficient electronic or computational device or technique for synthesizing speech signals).
- a vocoder i.e. a relatively efficient electronic or computational device or technique for synthesizing speech signals.
- a very similar approach has also been described by Strong and Palmer in US-4 051 331 .
- Their signal synthesis is also based on modified speech features. However, it synthesizes voiced components using tones, and unvoiced components using narrow-band noises. Thus, these techniques are spectrum-destroying rather than spectrum-preserving.
- a phase vocoder system for frequency transposition is described in a paper by H. J. McDermott and M. R. Dean ("Speech perception with steeply sloping hearing loss", British Journal of Audiology, vol. 34, pp. 353-361, December 2000 ).
- a non-real-time implementation is disclosed using a computer program. Digitally recorded speech signals were low pass filtered, down sampled and windowed, and then processed by a FFT. The phase values from successive FFTs were used to estimate a more precise frequency for each FFT bin, which was used to tune an oscillator corresponding to each FFT bin. Frequency lowering was achieved by multiplying the frequency estimates for each FFT-bin by a constant factor.
- a problem with each of the above feature-extraction and resynthesis processing schemes is that it is technically extremely difficult to obtain reliable estimates of speech features (such as fundamental frequency and voicing) in a wearable, real-time hearing instrument, especially in unfavorable listening conditions such as when noise or reverberation is present.
- EP-0 054 450 describes the transposition and amplification of two or three different bands of the frequency spectrum into lower-frequency bands within the audible range.
- the number of "image" bands equals the number of original bands.
- the frequency compression ratio can be different across bands, but is constant within each band.
- the image bands are arranged contiguously, and transposed to frequencies above 500 Hz. In order to free this part of the spectrum for the image bands, the amplification for frequencies between 500 and 1000 Hz decreases gradually with increasing frequency. Frequencies below 500 Hz in the original signal are amplified with a constant gain.
- the input signal is subjected to adaptive noise canceling before filtering into at least two pass-bands takes place. Frequency compression is then carried out in at least one frequency band.
- a method and device for transposing the frequency of an input signal based on the use of an analytical circuit is described in EP-0 542 711 , where in one embodiment an element with a non-linear transfer characteristic is provided in order to shift individual frequency ranges to different extents, hence enabling non-linear compression.
- a frequency transposition to the spectrum of the acoustic signal to obtain a transposed spectrum, whereby the frequency transposition is being defined by a nonlinear frequency transposition function (i.e. the compression ratio is a function of the input frequency)
- a nonlinear frequency transposition function i.e. the compression ratio is a function of the input frequency
- harmonic relationships are not distorted in the lower frequency range, and at the same time, higher frequencies can be moved into a lower frequency range, namely to an audible frequency range of the hearing impaired person.
- the transposition scheme can be applied to the complete signal spectrum without the need for switching between non-transposition and transposition processing for different parts of the signal. Therefore, no artifacts due to switching are encountered when applying the present invention.
- frequency transposition is a potential means for providing profoundly hearing impaired patients with signals in their residual range.
- the process of frequency transposition is illustrated in Fig. 1 , wherein the magnitude spectrum
- a frequency band FB is transposed by a frequency transposition function to obtain a transposed magnitude spectrum
- linear or proportional frequency transposition (as it is shown in Figs. 3 and 4 by the dashed line), or linear frequency transposition applied to only parts of the spectrum of a acoustic signal, are the only meaningful schemes since other processing methods of the state of the art distort the signal in such a manner that potential subjects reject the processing.
- the application of linear frequency transposition is however limited in that in order to preserve a reasonable intelligibility of the speech signal, the frequency span of the compressed signal should not be less that 60 to 70% of the original bandwidth.
- Nonlinear transposition schemes were not considered so far because the distortion of the harmonic relationships in lower frequencies has a detrimental effect on vowel recognition and is therefore totally unacceptable.
- Fig. 2 shows a simplified block diagram of a digital hearing device according to the present invention comprising a microphone 1, an analog-to-digital converter unit 2, a transformation unit 3, a signal processing unit 4, an inverse transformation unit 5, a digital-to-analog converter unit 5 and a loudspeaker 7, also called receiver.
- the invention is not only suitable for implementation in a digital hearing device but can also readily be implemented in an analog hearing device. In the latter case, the analog-to-digital converter unit 2 and the digital-to-analog converter unit 6 are not necessary.
- a so-called vocoder is used in which the output signal is synthesized by a bank of sine wave generators.
- a vocoder is used in which the output signal is synthesized by a bank of sine wave generators.
- an implementation of the invention is not only limited to conventional hearing devices, such as BTE-(behind the ear), CIC-(completely in the canal) or ITE-(in the ear) hearing devices.
- An implementation in implantable devices is also possible.
- a transducer is used instead of the loudspeaker 7 which transducer is either operationally connected to the signal processing unit 4, or to the inverse transformation unit 5, or to the digital-to-analog converter unit 6, and which transducer is made for directly transmitting acoustical information to the middle or inner ear of the patient.
- a direct stimulation of receptor in the inner ear is conceivable by using the output signal of the signal processing unit 4.
- the sampled acoustic signal s(n) is transformed into the frequency domain by an appropriate frequency transformation function in order to obtain the discrete spectrum S(m).
- a Fast Fourier Transformation is applied in the transformation unit 3.
- any other suitable transformation can be used, such as for example the Paley, Hadamard, Haar or the slant transformation.
- any other suitable transformation such as for example the Paley, Hadamard, Haar or the slant transformation.
- a frequency transposition is being applied to the spectrum S(m) in order to obtain a transposed spectrum S'(m), whereby the frequency transposition is defined by a nonlinear frequency transposition function.
- the frequency transposition function must be such that lower frequencies are transposed weakly and essentially linearly, while higher frequencies are transposed more strongly, either in a linear or nonlinear manner.
- harmonic relationships are not distorted in the lower frequency range, and, at the same time, higher frequencies can be moved to such low frequencies that they can fall into the audible range of profoundly hearing impaired person. Therefore and in one embodiment of the present invention, a piecewise linear frequency transposition function is applied, wherein at least the part of the frequency transposition function which is sensitive to distortion of harmonic relationship constitutes a linear section.
- Frequency compression parameters are a compression ratio above the cut-off frequency, and a cut-off frequency of 1.5 to 2.5 kHz, preferably of 2 kHz. Parameter adjustment is done based on sound quality and speech intelligibility requirements.
- the nonlinear frequency transposition function has a perception-based scale, such as the Bark, ERB or SPINC scale.
- Bark reference is made to E. Zwicker and H. Fastl in "Psychoacoustics - Facts and Models” (2nd edition, Springer, 1999 )
- ERB reference is made to B. C. J. Moore and B. R. Glasberg in "Suggested formulae for calculating auditory-filter bandwidths and excitation patterns" (J. Acoust. Soc. Am., Vol. 74, no. 3, pp.
- a nonlinear frequency transposition function such as for example Bark, ERB or SPINC
- a piecewise approximation This can be accomplished, for example, by first, second or higher order approximation.
- Figs. 3 and 4 show different frequency transposition functions and transposition ratios, wherein the horizontal axis represents the input frequency f and the vertical axis represents the corresponding output frequency f'.
- the graphs drawn by a dotted line represent different frequency transposition functions according to the present invention.
- the graphs drawn by solid and dashed lines are for comparison and show corresponding state of the art frequency transposition functions.
- the frequency transposition function is stored in a look-up table which is provided in the signal processing unit 4.
- the look-up table can be easily accessed by the signal processing unit 4.
- Fig. 5 shows a weighting matrix for 1:1 frequency compression (i.e. no frequency compression or frequency transposition, respectively). Its interpretation is as follows: input frequencies falling, for example, into bin 2, i.e. between 125 and 375 Hz, are represented within the output frequency bin 2 with frequencies between 125 and 375 Hz.
- the input bin is mapped with a weight of one to the output bin which has centre frequency closest to the calculated transposed frequency. For the above-mentioned example, this would be output bin 2 with centre frequency 250 Hz (167 is closer to 250 than 0).
- FIG. 6 Such a weighing matrix, where always the closest output bin is chosen, is shown in Fig. 6 in which input bin 1 is mapped to output bin 1, input bin 2 is mapped to output bin 1, input bin 3 is mapped to output bin 2, input bin 4 is mapped to output bin 2, input bin 5 is mapped to output bin 3, etc. It is clear that this method is very simple, but it leads to distortions in the output sound. The desired mapping from input to output frequencies cannot be achieved with sufficient resolution.
- the input frequency is mapped onto two neighboring output bins with a total weight of 1, where each bin is weighed according to the distance of its centre frequency to the desired output frequency.
- input bin 3 with centre frequency 500 Hz is mapped to an output frequency of 167 Hz which lies between output bins 1 and 2.
- FIG. 7 Such a weighing matrix is shown in Fig. 7 .
- Input bin 1 is mapped onto output bin 1 only with weight 1.
- Input bin 2 is mapped onto output bin 1 with weight 0.9 and output bin 2 with weight 0.1 (i.e. 90% of the signal in input bin 1 is represented in output bin 1 and the remaining 10% in output bin 2).
- Input bin 3 is mapped onto output bin 2 with weight 0.6 and output bin 3 with weight 0.4 (i.e. 60% of the signal in input bin 3 is synthesized with the centre frequency of output bin 2, and the remaining 40% in output bin 3), etc.
- Fig. 8 shows a further weighting matrix analogous to the one presented in Fig. 7 but for the case of piecewise linear compression (i.e. a practical nonlinear compression scheme) with no compression below the cut-off frequency of 1.5 kHz and linear compression above the cut-off frequency.
- piecewise linear compression i.e. a practical nonlinear compression scheme
- Small communication devices are, for example, of the type "hearing device” as they are marketed by the company Phonak AG.
- These hearing devices typically consist of a portable module containing a microphone in connection with an FM-(frequency modulation) transmitter that can be placed on a desk or lectern, and an FM receiver which is directly connected to the hearing device itself, usually via a so-called “audio shoe” as adapter.
- an FM-(frequency modulation) transmitter that can be placed on a desk or lectern
- an FM receiver which is directly connected to the hearing device itself, usually via a so-called “audio shoe” as adapter.
- Current FM systems have an audio bandwidth of 5 to 7 kHz.
- frequency compression is used to include information from higher audio frequencies within the same transmission bandwidth. For example, the information of all frequencies up to 10 kHz can be compressed into the available bandwidth by the transmission system.
- a further application of the present invention is directed to binaural hearing device systems since one is confronted with similar transmission problems. Besides the limited bandwidth further technical difficulties must be overcome, as for example the size and power consumption while aiming at a high transmission rate.
- US-2 810 787 describes a voiced/unvoiced band switching system. It takes advantage of the fact that the significant energy of voiced sounds occupies the lower portion of the frequency spectrum while the significant energy of unvoiced sounds almost exclusively lies in the high portion of the audible frequency spectrum. Therefore, a voiced-unvoiced detector determines if the instantaneous speech input comprises a voiced or unvoiced sound and based on this decision the available transmission band is allocated to the most relevant portion of the audio spectrum for the particular input sound.
- a major drawback of this band-switching scheme is that a frequency shift synchronizing signal must be transmitted to the receiver to enable it to correctly restore the original speech signal.
- the present invention uses a simple method of frequency compression or frequency transposition, respectively, for audio signals using frequency domain compression.
- the resulting time domain audio signal can be transmitted over a narrower band width than the original signal, whilst still preserving audio quality.
- the frequency compression adjustment can be described qualitatively as aiming to achieve maximum speech transmission for the available bandwidth, whereby this bandwidth is given by the bandwidth of the used communication system.
- the available bandwidth is given by the bandwidth provided for information transmission by the communication device. Parameter adjustment is done based on sound quality and speech intelligibility requirements. With careful selection of the appropriate parameters and consideration of the application, de-compression at the receiving end may not be necessary.
- a frequency compression device can be built using a digital signal processor and included inside a mobile or a fixed line telephone handset.
- the frequency compression device receives an analog audio signal, digitizes and processes it as it has already been described along with Fig. 2 . If the compression device is to be included in an existing telephone, the signal may be converted back to analog and fed into the normal processing path in the telephone.
- the frequency compressed signal which is available in digital form, may be the most suitable for a digital telephone. Many telephones may already contain enough spare signal processing capabilities in the associated signal processing unit to implement the efficient algorithm.
- the output signal of the microphone of the telephone is connected to a signal processing unit in which an appropriate window is applied to the sampled audio signal (sampling rate of 16kHz, for example) before a Fast Fourier Transformation with 32 points, for example, is applied.
- the resulting frequency spectrum is compressed by combining several high frequency bins into low frequency bins thus compressing more high frequency information into the 300 to 3300 Hz range than previously.
- the frequency compression is performed in the same manner as has been explained in connection with Figs. 5 to 8 .
- the time domain signal is obtained by performing an inverse Fast Fourier Transformation (IFFT) on the compressed frequency domain signal.
- IFFT inverse Fast Fourier Transformation
- the time domain signal is generated by a bank of sine wave oscillators or phase vocoders.
- the amplitude and frequency control signals for each oscillator are derived from magnitude and phase change values of corresponding FFT bins. Depending on the requirement of the particular telephone, this signal may be converted back to analog, or simply passed on in digital form to the next stage in the telephone.
- the receiving telephone would not need any modifications or knowledge that frequency compression has been used by the sending or calling telephone.
- the listener would simply hear a frequency compressed signal.
- This particular implementation of the present invention allows the use of a frequency compression in any individual telephone, either by hardware/software modifications of an existing telephone, or to be built in to any new telephone. The users outgoing voice quality would be improved and any existing telephone could be used at the receiving end.
- the receiving telephone could have a decompression device (yet to be explained) which returns the compressed signal to near original state.
- this implementation requires both the receiving and transmitting telephones to be equipped with frequency compression devices, and also some modifications to the call setup protocol to signal that a compressed signal is being transmitted.
- the present invention is described in the context of the application to FM transmitters used in hearing devices and describes the de-compression process.
- the FM transmitter module performs frequency compression as described above, and the compressed signal with an audio bandwidth of 5 kHz is transmitted over the FM link.
- the hearing device which receives the compressed signal could use it directly, or perform de-compression to restore the signal to its original bandwidth.
- frequency compression be implemented with a bin combination that results in the best quality compressed audio signal. This could be implemented with a bin combination matrix similar to the one shown in Fig. 7 , with a cut-off frequency at 2 kHz.
- the bin combination matrix used to compress the signal needs to have a corresponding decompression matrix that provides good reconstruction of the original signal.
- the acoustic quality of the compressed signal which is transmitted is not important.
- an audio band 0 to 5 kHz corresponds to an equivalent of 10 FFT bins available for signal transmission (separated at 500Hz if we assume a typical sampling rate and FFT size).
- the input signal to be compressed may have a frequency range of 0 to 8 kHz corresponding to 16 FFT bins.
- the 16 bins must then be mapped onto 10 bins (or possibly less if a lower audio bandwidth must be obtained).
- the resulting time domain signal which need not have any acoustic resemblance to the original signal, is subsequently transmitted. Finally, the signal is reconstructed at the receiving end.
- the rules for bin combination for compression and decompression are outlined below by referring to a specific example:
- Figs. 9 and 10 show, in a graphical representation, a similar mapping of frequency bins for compression and decompression (i.e. expansion) as has already been described along with the weighting matrices of Figs. 5 to 8 .
Description
- The present invention relates to a method for frequency transposition in a hearing device or in a communication device, respectively, according to the pre-characterizing part of
claim 1, to a use of the method for a binaural hearing device, to a device according to the pre-characterizing part ofclaim 15 as well as to a use of the device in a communication device or in a hearing device. - Numerous frequency-transposition schemes for the presentation of audio signals via hearing devices for people with a hearing impairment have been developed and evaluated over many years. In each case, the principal aim of the transposition is to improve the audibility and discriminability of signals in a particular frequency range by modifying those signals and presenting them at other frequencies. Usually, high frequencies are transposed to lower frequencies where hearing device users typically have better hearing ability. However, various problems have limited the successful application of such techniques in the past. These problems include technological limitations, distortions introduced into the sound signals by the processing schemes employed, and the absence of methods for identifying suitable candidates and for fitting frequency-transposing hearing aids to them using appropriate objective rules.
- The many techniques for frequency transposition reported previously can be subdivided into three broad types: frequency shifting, frequency compression, and reducing the playback speed of recorded audio signals while discarding portions of the signal in order to preserve the original duration.
- Among frequency compression schemes, many linear and non-linear techniques including FFT/IFFT processing, vocoding, and high-frequency envelope transposition followed by mixing with unmodified low-frequency components have been investigated. Since harmonic patterns and formant relations are known to be important in the accurate perception of speech, it is also helpful to distinguish spectrum-preserving techniques from spectrum-destroying techniques. Each of these techniques is summarized briefly below.
- At present, the only frequency-transposing hearing instruments available commercially are those manufactured by AVR Ltd., a company based in Israel and Minnesota, USA (see http://www.avrsono.com). An instrument produced previously by AVR, known as the TranSonic, has been superseded recently by the ImpaCt and Logicom-20 devices. All of these frequency-transposition instruments are based on the selective reduction of the playback speed of recorded audio signals. This is achieved by first sampling the input sound signal at a particular rate, and then storing it in a memory. When the recorded signal is subsequently read out of the memory, the sampling rate is reduced when frequency-lowering is required. Because the sampling rate can be changed, it is possible to apply frequency lowering selectively. For example, different amounts of frequency-lowering can be applied to voiced and unvoiced speech components. The presence of each type of component in the input signal is determined by estimating the spectral shape: the signal is assumed to be unvoiced when a spectral peak is detected at frequencies above 2.5 kHz, voiced otherwise. In order to maintain the original duration of the signals, parts of the sampled data in the memory are discarded when necessary.
US assigned to AVR describes not only the compression of input frequencies (i.e. frequencies are transposed into lower ranges) but also frequency expansion (i.e. transposition into higher frequency ranges). Other similar methods of frequency transposition by means of reducing the playback speed of recorded audio signals have also been reported previously (e.g.Patent 5 014 319FR-2 364 520 DE-17 62 185 ). As mentioned, a major problem with any of these schemes is that portions of the input signal must be discarded when the playback speed is reduced (to compress frequencies) in order to maintain the original signal duration, which is essential in a real-time assistive listening system such as a hearing device. This could result in audible distortions in the output signal and in some important sound information being inaudible to the hearing device user. - Linear frequency compression by means of Fourier Transform processing has been investigated by Turner and Hurtig at the University of Iowa, USA (Turner, C. W. and R. R. Hurtig: "Proportional Frequency Compression of Speech for Listeners with Sensorineural Hearing Loss", Journal of the Acoustical Society of America, vol. 106(2), pp. 877-886, 1999), and has led to an international patent application having the
publication number WO 99/14 986 - A feature extraction and signal resynthesis procedure and system based on a vocoder have been described by Thomson CSF, Paris in
EP-1 006 511 . Information about pitch, voicing, energy, and spectral shape is extracted from the input signal. These features are modified (e.g. by compressing the formant frequencies in the frequency domain) and then used for synthesis of the output signal by means of a vocoder (i.e. a relatively efficient electronic or computational device or technique for synthesizing speech signals). A very similar approach has also been described by Strong and Palmer inUS-4 051 331 . Their signal synthesis is also based on modified speech features. However, it synthesizes voiced components using tones, and unvoiced components using narrow-band noises. Thus, these techniques are spectrum-destroying rather than spectrum-preserving. - A phase vocoder system for frequency transposition is described in a paper by H. J. McDermott and M. R. Dean ("Speech perception with steeply sloping hearing loss", British Journal of Audiology, vol. 34, pp. 353-361, December 2000). A non-real-time implementation is disclosed using a computer program. Digitally recorded speech signals were low pass filtered, down sampled and windowed, and then processed by a FFT. The phase values from successive FFTs were used to estimate a more precise frequency for each FFT bin, which was used to tune an oscillator corresponding to each FFT bin. Frequency lowering was achieved by multiplying the frequency estimates for each FFT-bin by a constant factor.
- Another system that can separately compress the frequency range of voiced and unvoiced speech components as well as the fundamental frequency has been described by S. Sakamoto, K. Goto, et. al. ("Frequency Compression Hearing Aid for Severe-To-Profound Hearing Impairments", Auris Nasus Larynx, vol. 27, pp. 327-334, 2000). This system allows independent adjustment of the frequency compression ratio for unvoiced and voiced speech, fundamental frequency, the spectral envelope, and the instrument's frequency response by the selection of different filters. The compression ratio for either voiced or unvoiced speech is adjustable from 10% to 90% in steps of 10%. The fundamental frequency can either be left unmodified, or compressed with a compression ratio either the same as, or lower than, that employed for voiced speech. A problem with each of the above feature-extraction and resynthesis processing schemes is that it is technically extremely difficult to obtain reliable estimates of speech features (such as fundamental frequency and voicing) in a wearable, real-time hearing instrument, especially in unfavorable listening conditions such as when noise or reverberation is present.
-
EP-0 054 450 describes the transposition and amplification of two or three different bands of the frequency spectrum into lower-frequency bands within the audible range. In this scheme, the number of "image" bands equals the number of original bands. The frequency compression ratio can be different across bands, but is constant within each band. The image bands are arranged contiguously, and transposed to frequencies above 500 Hz. In order to free this part of the spectrum for the image bands, the amplification for frequencies between 500 and 1000 Hz decreases gradually with increasing frequency. Frequencies below 500 Hz in the original signal are amplified with a constant gain. - In
US to Adelman, the input signal is subjected to adaptive noise canceling before filtering into at least two pass-bands takes place. Frequency compression is then carried out in at least one frequency band.Patent 4 419 544 - Other techniques described previously include the modulation of tones or noise bands in the low-frequency range based on the energy present in higher frequencies (e.g.
FR-1 309 425 US-3 385 937 ), and various types of linear and non-linear transposition of high-frequency components which are then superimposed onto the low-frequency part of the spectrum (e.g.US-5 077 800 andUS-3 819 875 ). Another approach (WO 00/75 920 - A method and device for transposing the frequency of an input signal based on the use of an analytical circuit is described in
EP-0 542 711 , where in one embodiment an element with a non-linear transfer characteristic is provided in order to shift individual frequency ranges to different extents, hence enabling non-linear compression. - Problems with each of the above described methods for frequency transposition include technical complexity, distortion or loss of information about sounds in some circumstances, and unreliability of the processing in difficult listening conditions, e.g. in the presence of background noise.
- It is therefore an object of the present invention to enable frequency transposition to be carried out more efficiently.
- This object is achieved, for a method for frequency transposition in hearing devices or in communication devices, respectively, by the elements of the characterizing part of
claim 1. Further achievements of the method according to the present invention, a use of the method, a device as well as a use of the method in a hearing device or a communication device, respectively, are subject to further claims. - By applying a frequency transposition to the spectrum of the acoustic signal to obtain a transposed spectrum, whereby the frequency transposition is being defined by a nonlinear frequency transposition function (i.e. the compression ratio is a function of the input frequency), it is possible to transpose different frequencies by different amounts, i.e. to let lower frequencies pass without transposition or to apply only a small amount of transposition to them, while higher frequencies are transposed more strongly. As a result thereof, harmonic relationships are not distorted in the lower frequency range, and at the same time, higher frequencies can be moved into a lower frequency range, namely to an audible frequency range of the hearing impaired person. The transposition scheme can be applied to the complete signal spectrum without the need for switching between non-transposition and transposition processing for different parts of the signal. Therefore, no artifacts due to switching are encountered when applying the present invention.
- The present invention is further explained by referring to an exemplified embodiment shown in drawings. It is shown in
- Fig. 1
- a magnitude as a function of frequency of an acoustic signal as well as the magnitude as a function of frequency of that signal after transposition;
- Fig. 2
- a block diagram of a hearing device according to the present invention;
- Figs. 3 and 4
- frequency transposition schemes having no compression, linear compression and perception-based compression;
- Fig. 5
- a weighting matrix with no frequency compression or no frequency transposition, respectively;
- Figs. 6 and 7
- two weighting matrices for linear frequency compression or frequency transposition, respectively, according to the present invention;
- Fig. 8
- a weighting matrix for piecewise linear frequency compression or frequency transposition, respectively, according to the present invention;
- Fig. 9
- mapping of frequency bins for compression and decompression (i.e. expansion) according to the present invention; and
- Fig. 10
- a further embodiment for a mapping of frequency bins for compression and de-compression (i.e. expansion) according to the present invention.
- As has already been mentioned, frequency transposition is a potential means for providing profoundly hearing impaired patients with signals in their residual range. The process of frequency transposition is illustrated in
Fig. 1 , wherein the magnitude spectrum |S (f)| is shown of an acoustic signal in the upper graph ofFig. 1 . A frequency band FB is transposed by a frequency transposition function to obtain a transposed magnitude spectrum |S'(f)| and a transposed frequency band FB'. It is assessed that the hearing ability of the patient is more or less intact in the transposed frequency band FB' whereas in the frequency band FB it is not. Therefore, it is possible by the frequency transposition to image a part of the spectrum from an inaudible into an audible range of the patient. As a measure for the frequency transposition, a so-called compression ratio CR is defined as follows: - So far, linear or proportional frequency transposition (as it is shown in
Figs. 3 and 4 by the dashed line), or linear frequency transposition applied to only parts of the spectrum of a acoustic signal, are the only meaningful schemes since other processing methods of the state of the art distort the signal in such a manner that potential subjects reject the processing. The application of linear frequency transposition is however limited in that in order to preserve a reasonable intelligibility of the speech signal, the frequency span of the compressed signal should not be less that 60 to 70% of the original bandwidth. This conclusion has been found by C. W. Turner and R. R. Hurtig in the paper entitled "Proportional Frequency Compression of Speech for Listeners with Sensorineural Hearing Loss" (Journal of the Acoustical Society of America, 106(2), pp. 877-886, 1999). The compression ratios are thus limited to values in the range of up to 1.5. - With the above-described limitation, common consonant frequencies lying in the range of 3 to 8 kHz can only be compressed into approximately 2 to 5 kHz. For most hearing impaired patients, however, these frequencies are still poorly audible or not audible at all. The desired benefit of frequency transposition can thus not be achieved.
- Nonlinear transposition schemes were not considered so far because the distortion of the harmonic relationships in lower frequencies has a detrimental effect on vowel recognition and is therefore totally unacceptable.
- The possibility to overcome the above-mentioned problems has been documented by Sakamoto et. al. (see above): Voiced and unvoiced components of the signal have been distinguished, and the frequency transposition has only been applied to the unvoiced components. Although nonlinear transposition might be suitable in this case because the important low frequent harmonic relationships are not transposed and therefore unchanged, switching between different processing schemes creates audible artifacts as well, and is therefore also disadvantageous. In addition, as mentioned earlier, it is very difficult to achieve the required speech feature recognition with sufficient reliability and robustness.
-
Fig. 2 shows a simplified block diagram of a digital hearing device according to the present invention comprising amicrophone 1, an analog-to-digital converter unit 2, atransformation unit 3, asignal processing unit 4, aninverse transformation unit 5, a digital-to-analog converter unit 5 and aloudspeaker 7, also called receiver. Of course, the invention is not only suitable for implementation in a digital hearing device but can also readily be implemented in an analog hearing device. In the latter case, the analog-to-digital converter unit 2 and the digital-to-analog converter unit 6 are not necessary. - In a further embodiment of the present invention, instead of the inverse transformation unit 5 a so-called vocoder is used in which the output signal is synthesized by a bank of sine wave generators. For further information regarding the functioning of a vocoder, reference is made to H. J. McDermott and M. R. Dean ("Speech perception with steeply sloping hearing loss", British Journal of Audiology, vol. 34, pp. 353-361, December 2000).
- Furthermore, an implementation of the invention is not only limited to conventional hearing devices, such as BTE-(behind the ear), CIC-(completely in the canal) or ITE-(in the ear) hearing devices. An implementation in implantable devices is also possible. For implantable devices, a transducer is used instead of the
loudspeaker 7 which transducer is either operationally connected to thesignal processing unit 4, or to theinverse transformation unit 5, or to the digital-to-analog converter unit 6, and which transducer is made for directly transmitting acoustical information to the middle or inner ear of the patient. In any case, a direct stimulation of receptor in the inner ear is conceivable by using the output signal of thesignal processing unit 4. - In the
transformation unit 3, the sampled acoustic signal s(n) is transformed into the frequency domain by an appropriate frequency transformation function in order to obtain the discrete spectrum S(m). In a preferred embodiment of the present invention, a Fast Fourier Transformation is applied in thetransformation unit 3. Fur further information, reference is made to the publication of Alan V. Oppenheim and Ronald W. Schafer "Discrete-time Signal Processing" (Printice-Hall Inc., 1989, ) . - Instead of applying the Fourier Transformation in the
transformation unit 3, any other suitable transformation can be used, such as for example the Paley, Hadamard, Haar or the slant transformation. For further information regard these transformations, reference is made to Claude S. Lindquist in "Adaptive & Digital Signal Processing" (1989, Steward & Sons, Miami, Florida, Section 2.8). - In the
signal processing unit 4, a frequency transposition is being applied to the spectrum S(m) in order to obtain a transposed spectrum S'(m), whereby the frequency transposition is defined by a nonlinear frequency transposition function. - In general, the frequency transposition function must be such that lower frequencies are transposed weakly and essentially linearly, while higher frequencies are transposed more strongly, either in a linear or nonlinear manner. Hence, harmonic relationships are not distorted in the lower frequency range, and, at the same time, higher frequencies can be moved to such low frequencies that they can fall into the audible range of profoundly hearing impaired person. Therefore and in one embodiment of the present invention, a piecewise linear frequency transposition function is applied, wherein at least the part of the frequency transposition function which is sensitive to distortion of harmonic relationship constitutes a linear section.
- It is pointed out that frequency compression fitting, and therewith the resulting frequency transposition function, can be described qualitatively as aiming at achieving maximum speech transmission for the available bandwidth, whereby this bandwidth is determined from the audiogram and from speech tests. Frequency compression parameters are a compression ratio above the cut-off frequency, and a cut-off frequency of 1.5 to 2.5 kHz, preferably of 2 kHz. Parameter adjustment is done based on sound quality and speech intelligibility requirements.
- In a further embodiment of the present invention, the nonlinear frequency transposition function has a perception-based scale, such as the Bark, ERB or SPINC scale. Regarding Bark, reference is made to E. Zwicker and H. Fastl in "Psychoacoustics - Facts and Models" (2nd edition, Springer, 1999), regarding ERB, reference is made to B. C. J. Moore and B. R. Glasberg in "Suggested formulae for calculating auditory-filter bandwidths and excitation patterns" (J. Acoust. Soc. Am., Vol. 74, no. 3, pp. 750-753, 1983), and regarding SPINC, reference is made to Ernst Terhardt in "The SPINC function for scaling of frequency in auditory models" (Acustika, no. 77, 1992, p.40-42). With these frequency transposition functions, lower frequencies are transposed almost linearly, while higher frequencies are transposed more strongly. Hence, harmonic relationships are not distorted in the lower frequency range, and, at the same time, higher frequencies can be moved into such low frequencies that they can fall into the audible range of profoundly hearing impaired patients. The frequency transposition function can be applied to the complete signal spectrum, without the need for identifying any speech features and switching between non-transposition and transposition processing for different parts of the signal.
- In a further embodiment of the present invention, a nonlinear frequency transposition function, such as for example Bark, ERB or SPINC, can be implemented by a piecewise approximation. This can be accomplished, for example, by first, second or higher order approximation.
-
Figs. 3 and 4 show different frequency transposition functions and transposition ratios, wherein the horizontal axis represents the input frequency f and the vertical axis represents the corresponding output frequency f'. The graphs drawn by a dotted line represent different frequency transposition functions according to the present invention. The graphs drawn by solid and dashed lines are for comparison and show corresponding state of the art frequency transposition functions. - In
Fig. 3 , three different transposition schemes are represented in the same graph: - solid line: no compression, therefore no frequency transposition;
- dashed line: linear compression with compression ratio CR = 1.2;
- dotted line: perception-based compression with compression ratio CR = 1.2.
- In
Fig. 4 , again three different transposition schemes are represented in the same graph with the following characteristics: - solid line: no compression, therefore no frequency transposition (same as in
Fig. 3 ); - dashed line: linear compression with compression ratio CR = 1.5;
- dotted line: perception-based compression with compression ratio CR = 1.5.
- In a preferred embodiment of the present invention, the SPINC-(spectral pitch increment) compression scheme is implemented by transforming the input frequency f into the SPINC scale Φ, applying the desired compression ratio CR in the SPINC scale, and transforming back to the linear frequency scale. Therefore, the corresponding frequency transposition function can be defined as follows:
- It goes without saying that similar frequency compression can also be achieved in other perception-based frequency transpositions such as by using the Bark or the ERB scale.
- In a further embodiment, the frequency transposition function is stored in a look-up table which is provided in the
signal processing unit 4. The look-up table can be easily accessed by thesignal processing unit 4. - In the following, an embodiment for the implementation of frequency compression with respect to a FFT bin matrix is explained by referring to
Figs. 5 to 10 . - In FFT-based processing, each frequency bin has a certain bandwidth and centre frequency. For example, for a 32 point FFT on a signal sampled with 16kHz, the bandwidth of each frequency bin is 16'000 Hz / 32 / 2 (looking at positive frequencies only) = 250 Hz. The centre frequencies of the individual bins are then spaced 250 Hz apart. The relationships are shown in the following table:
bin 1 2 3 4 centre frequency [Hz] 0 250 500 750 bandwidth [Hz] 250 250 250 250 frequency range [Hz] - 125 .. 125 125 .. 375 375 .. 625 625 .. 875 -
Fig. 5 shows a weighting matrix for 1:1 frequency compression (i.e. no frequency compression or frequency transposition, respectively). Its interpretation is as follows: input frequencies falling, for example, intobin 2, i.e. between 125 and 375 Hz, are represented within theoutput frequency bin 2 with frequencies between 125 and 375 Hz. -
- The centre frequency of
input bin 4, for example, then falls exactly onto output bin 2 (1/3*750 Hz = 250 Hz), but forinput bin 3, for example, the centre frequency falls betweenoutput bins 1 and 2 (1/3*500 Hz = 167 Hz). - In a first embodiment, the input bin is mapped with a weight of one to the output bin which has centre frequency closest to the calculated transposed frequency. For the above-mentioned example, this would be
output bin 2 with centre frequency 250 Hz (167 is closer to 250 than 0). - Such a weighing matrix, where always the closest output bin is chosen, is shown in
Fig. 6 in whichinput bin 1 is mapped tooutput bin 1,input bin 2 is mapped tooutput bin 1,input bin 3 is mapped tooutput bin 2,input bin 4 is mapped tooutput bin 2,input bin 5 is mapped tooutput bin 3, etc. It is clear that this method is very simple, but it leads to distortions in the output sound. The desired mapping from input to output frequencies cannot be achieved with sufficient resolution. - Therefore, in a further embodiment of the present invention, the input frequency is mapped onto two neighboring output bins with a total weight of 1, where each bin is weighed according to the distance of its centre frequency to the desired output frequency. In the above-mentioned example,
input bin 3 with centre frequency 500 Hz is mapped to an output frequency of 167 Hz which lies betweenoutput bins output bins 1 and 2 (the desired 167 Hz lie between 0 and 250 Hz) and assign the weight 0.67 to bin 2 (167/250 = 0.67) and 1-0.67=0.33 tobin 1. - Such a weighing matrix is shown in
Fig. 7 .Input bin 1 is mapped ontooutput bin 1 only withweight 1.Input bin 2 is mapped ontooutput bin 1 with weight 0.9 andoutput bin 2 with weight 0.1 (i.e. 90% of the signal ininput bin 1 is represented inoutput bin 1 and the remaining 10% in output bin 2).Input bin 3 is mapped ontooutput bin 2 with weight 0.6 andoutput bin 3 with weight 0.4 (i.e. 60% of the signal ininput bin 3 is synthesized with the centre frequency ofoutput bin 2, and the remaining 40% in output bin 3), etc. - Finally,
Fig. 8 shows a further weighting matrix analogous to the one presented inFig. 7 but for the case of piecewise linear compression (i.e. a practical nonlinear compression scheme) with no compression below the cut-off frequency of 1.5 kHz and linear compression above the cut-off frequency. - Although the various aspects of the present invention have been described in connection with downward frequency shifting, the same applies for upward frequency shifting (expansion) and the various aspects can also be readily applied for any upward frequency shifting. An application where such an upward frequency shifting could be utilized is in the context of mitigating the occlusion effect, also referred to as closure effect, in order to undo the unpleasant dullness of the own voice as it occurs when closing the ear canal with an ITE-(In-The-Ear) hearing device or an ear mold.
- In addition, it is expressly pointed out that all aspects of the present invention described above can also be used in connection with communication systems having a limited bandwidth for information transmission. For such communication systems, the same aspect of the present invention can be applied to significantly improve transmission quality. This will be further explained in the following:
- For most communication systems, information is transmitted over a limited bandwidth. For example, the audio bandwidth of the telephone network is currently limited to 300 to 3300Hz. As a result, important parts of speech beyond 3300Hz are not transmitted very well, especially unvoiced speech sounds such as "S", "SH" and "F".
- Other examples are so-called two-way radio systems (e.g. Walkie-Talkies) that are frequently used by police forces, fire fighters, ambulance services, etc. Most of these systems are analog systems with a very limited audio bandwidth (e.g. 2.5 kHz). This makes intelligibility very difficult, especially considering the often adverse listening conditions in which these professionals operate.
- Musicians need to hear their own voice or the instrument they are playing. Normally this is either done by placing loudspeakers on stage that amplify the necessary signals for a given musician or by a wireless feedback system. In the latter case, the musician wears a body worn receiver that is connected to an earpiece that delivers the sound to the ear. State of the art analog technology available today would basically allow integration of such a monitoring device into very small communication devices. The objection against this is bandwidth of the transmitted audio signal and the loudspeaker which can be characterized by a 7 kHz bandwidth.
- Small communication devices are, for example, of the type "hearing device" as they are marketed by the company Phonak AG. These hearing devices typically consist of a portable module containing a microphone in connection with an FM-(frequency modulation) transmitter that can be placed on a desk or lectern, and an FM receiver which is directly connected to the hearing device itself, usually via a so-called "audio shoe" as adapter. In this way, a hearing device user can remotely listen from a microphone placed close to the source. Current FM systems have an audio bandwidth of 5 to 7 kHz. According to the present invention, frequency compression is used to include information from higher audio frequencies within the same transmission bandwidth. For example, the information of all frequencies up to 10 kHz can be compressed into the available bandwidth by the transmission system.
- A further application of the present invention is directed to binaural hearing device systems since one is confronted with similar transmission problems. Besides the limited bandwidth further technical difficulties must be overcome, as for example the size and power consumption while aiming at a high transmission rate.
- In all of these applications, better intelligibility and understanding is achieved by the present invention, namely by compressing more information into the available bandwidth as it is described above.
- A number of techniques for improving the quality and intelligibility of speech transmitted over narrowband channels have been reported in the literature.
US-2 810 787 describes a voiced/unvoiced band switching system. It takes advantage of the fact that the significant energy of voiced sounds occupies the lower portion of the frequency spectrum while the significant energy of unvoiced sounds almost exclusively lies in the high portion of the audible frequency spectrum. Therefore, a voiced-unvoiced detector determines if the instantaneous speech input comprises a voiced or unvoiced sound and based on this decision the available transmission band is allocated to the most relevant portion of the audio spectrum for the particular input sound. A major drawback of this band-switching scheme is that a frequency shift synchronizing signal must be transmitted to the receiver to enable it to correctly restore the original speech signal.DE-31 12 221 A1DE-38 07 408 C1 describe methods that do not require such a synchronization signal and employ means to compress the audio signal in the transmitter and expand it again in the receiver. Unfortunately, the rather complicated analog signal processing circuitry limits the possible compression scheme to linear compression with a fixed compression ratio of 1/N, where N is an integer typically with a value of 2 or 3. In the publication entitled "Frequency Compression of 7.6 kHz Speech into 3.3 kHz Bandwidth" by Patrick et al. (IEEE Transactions on Communications; Vol. 31, No. 5, May 1983, pp. 692-701) an adaptive frequency mapping system is proposed. Depending on the characteristics of the momentary speech input, one of four possible compression rules is applied to the signal. This method promises better quality than previous solutions but has the drawback of considerable complexity, especially on the part of the speech analysis block which determines which compression rule to apply. - The present invention uses a simple method of frequency compression or frequency transposition, respectively, for audio signals using frequency domain compression. The resulting time domain audio signal can be transmitted over a narrower band width than the original signal, whilst still preserving audio quality. The frequency compression adjustment can be described qualitatively as aiming to achieve maximum speech transmission for the available bandwidth, whereby this bandwidth is given by the bandwidth of the used communication system.
- In general, the available bandwidth is given by the bandwidth provided for information transmission by the communication device. Parameter adjustment is done based on sound quality and speech intelligibility requirements. With careful selection of the appropriate parameters and consideration of the application, de-compression at the receiving end may not be necessary.
- In the following, the present invention is described in the context of a telephone network application where decompression of the signal at the receiving end is possible but not necessary.
- A frequency compression device can be built using a digital signal processor and included inside a mobile or a fixed line telephone handset. The frequency compression device receives an analog audio signal, digitizes and processes it as it has already been described along with
Fig. 2 . If the compression device is to be included in an existing telephone, the signal may be converted back to analog and fed into the normal processing path in the telephone. Alternatively, the frequency compressed signal, which is available in digital form, may be the most suitable for a digital telephone. Many telephones may already contain enough spare signal processing capabilities in the associated signal processing unit to implement the efficient algorithm. - The output signal of the microphone of the telephone is connected to a signal processing unit in which an appropriate window is applied to the sampled audio signal (sampling rate of 16kHz, for example) before a Fast Fourier Transformation with 32 points, for example, is applied. The resulting frequency spectrum is compressed by combining several high frequency bins into low frequency bins thus compressing more high frequency information into the 300 to 3300 Hz range than previously. The frequency compression is performed in the same manner as has been explained in connection with
Figs. 5 to 8 . - In a further embodiment, the time domain signal is obtained by performing an inverse Fast Fourier Transformation (IFFT) on the compressed frequency domain signal. In yet another embodiment of the present invention, the time domain signal is generated by a bank of sine wave oscillators or phase vocoders. The amplitude and frequency control signals for each oscillator are derived from magnitude and phase change values of corresponding FFT bins. Depending on the requirement of the particular telephone, this signal may be converted back to analog, or simply passed on in digital form to the next stage in the telephone.
- In a further, more simplified implementation of the present invention, the receiving telephone would not need any modifications or knowledge that frequency compression has been used by the sending or calling telephone. At the receiving telephone, the listener would simply hear a frequency compressed signal. This particular implementation of the present invention allows the use of a frequency compression in any individual telephone, either by hardware/software modifications of an existing telephone, or to be built in to any new telephone. The users outgoing voice quality would be improved and any existing telephone could be used at the receiving end.
- In a further implementation of the present invention, the receiving telephone could have a decompression device (yet to be explained) which returns the compressed signal to near original state. However, this implementation requires both the receiving and transmitting telephones to be equipped with frequency compression devices, and also some modifications to the call setup protocol to signal that a compressed signal is being transmitted.
- In the following, the present invention is described in the context of the application to FM transmitters used in hearing devices and describes the de-compression process.
- The FM transmitter module according to the present invention performs frequency compression as described above, and the compressed signal with an audio bandwidth of 5 kHz is transmitted over the FM link. The hearing device which receives the compressed signal could use it directly, or perform de-compression to restore the signal to its original bandwidth.
- If the signal is not to be de-compressed at the receiving end, then it is recommended that frequency compression be implemented with a bin combination that results in the best quality compressed audio signal. This could be implemented with a bin combination matrix similar to the one shown in
Fig. 7 , with a cut-off frequency at 2 kHz. - However, if the signal is to be de-compressed at the receiving end, then the bin combination matrix used to compress the signal needs to have a corresponding decompression matrix that provides good reconstruction of the original signal. In this case, the acoustic quality of the compressed signal which is transmitted is not important.
- In a FM transmission system an
audio band 0 to 5 kHz corresponds to an equivalent of 10 FFT bins available for signal transmission (separated at 500Hz if we assume a typical sampling rate and FFT size). The input signal to be compressed may have a frequency range of 0 to 8 kHz corresponding to 16 FFT bins. The 16 bins must then be mapped onto 10 bins (or possibly less if a lower audio bandwidth must be obtained). The resulting time domain signal, which need not have any acoustic resemblance to the original signal, is subsequently transmitted. Finally, the signal is reconstructed at the receiving end. Thereby, the rules for bin combination for compression and decompression are outlined below by referring to a specific example: - 1) Combine pairs of bins together. Sixteen bins will combine to make eight and map them to bins with frequencies within 0 to 5 kHz (actually eight bins can be transmitted at 0 to 4 kHz). De-compression is performed by splitting the signal in each compressed bin equally between the two bins which contributed to it. Unequal contributions to one compressed bin will not be mirrored in the de-compressed signal.
- 2) Transmit lower frequencies without compression, and only compress high frequency signals. This is likely to preserve better sound quality in the low frequencies. For example, bins one to four are not compressed and bins five to sixteen are combined in groups of three bins. This makes a total of four non-compressed bins and four compressed bins.
- a) De-compression can be performed by splitting the signal of each compressed bin equally between the three contributing bins, as indicated in
Fig. 9 , - b) or by mapping the total signal of each compressed bin all to the centre bin in each set of three. The other two bins in each group would be zero, as indicated in
Fig. 10 .
- a) De-compression can be performed by splitting the signal of each compressed bin equally between the three contributing bins, as indicated in
- 3) A compression strategy which combines more bins at higher frequencies than at low frequencies.
Combination in groups of odd numbers may be advantageous because de-compression can be performed by mapping the total power of each compressed bin to one frequency bin at the centre of each group of combining bins. -
Figs. 9 and10 show, in a graphical representation, a similar mapping of frequency bins for compression and decompression (i.e. expansion) as has already been described along with the weighting matrices ofFigs. 5 to 8 . - While exemplary preferred embodiments of the present invention are described herein with particularity, those skilled in the art will appreciate various changes, additions, and applications other than those specifically mentioned, which are within the scope of the appended claims.
Claims (22)
- Method for frequency transposition in a hearing device or in a communication device, respectively, comprising the steps of- transforming an acoustical signal into an electrical signal (s) and- transforming the electrical signal (s) from time domain into frequency domain to obtain a spectrum (S),- applying a nonlinear frequency transposition to the spectrum (S) in order to obtain a transposed spectrum (S'),characterized in that- the nonlinear frequency transposition function is perception-based meaning that harmonic relationships are not or only weakly distorted below a cut-off frequency and that frequencies above the cut-off frequency are moved to such lower frequencies that are still audible for a user, where the cut-off frequency lies in a range of 1.5 to 2.5 kHz, preferably is equal to 2 kHz.
- Method according to claim 1, characterized in that the nonlinear frequency transposition function is a continuous function.
- Method according to claim 1, characterized in that the nonlinear frequency transposition function is a piecewise approximation of a continuous function.
- Method according to claim 1, characterized in that the nonlinear frequency transposition function is a piecewise linear approximation of a continuous function.
- Method according to claim 1 or 2, characterized in that the perception-based frequency transposition function is being defined by one of the following functions:- Bark function;- ERB function; or- SPINC function.
- Method according to one of the claims 1 to 5, characterized in applying the transposed spectrum (S') to an output transducer (7) being a receiver or an implantable stimulation device.
- Method according to one of the claims 1 to 6, characterized in obtaining the transposed frequency spectrum (S') by using a weighting matrix which is applied to frequency input bins (Bin_IN) in order to map frequency components onto frequency output bins (Bin_OUT).
- Method according to claim 7, characterized in mapping an input bin (Bin_IN) with weight one to an output bin (Bin_OUT) which has a centre frequency closest to an exact calculated transposed frequency.
- Method according to claim 7, characterized in mapping an exact calculated transposed frequency onto neighboring output bins (Bin_OUT).
- Method according to one of the claims 1 to 9, characterized in a first communication device which is at least temporally connected to a second communication device, whereas the transposed spectrum (S') or its corresponding transposed signal, respectively, is being transmitted.
- Method according to claim 10, characterized in detransposing the transposed spectrum (S') or its corresponding transposed signal, respectively, in the second communication device to restore the electric signal (s) or its corresponding acoustic signal, respectively.
- Use of the method according to one of the claims 1 to 9 for a link between two hearing device parts of a binaural hearing device.
- Frequency compression device comprising- at least one microphone (1),- a transformation unit (3) to transform a time domain input signal into a frequency domain output signal, and- a signal processing unit (4),wherein the transformation unit (3) is operationally connected to the at least one microphone (1) and to the signal processing unit (4), and wherein a nonlinear frequency transposition function is applied to the frequency domain output signal of the transformation unit (3) in the signal processing unit (4), characterized in that the non-linear frequency transposition being perception-based meaning that harmonic relationships are not or only weakly distorted below a cut-off frequency and that frequencies above the cut-off frequency are moved to such lower frequencies that are still audible for a user, where the cut-off frequency lies in the range of 1.5 to 2.5 kHz, preferably is equal to 2 kHz.
- Frequency compression device according to claim 13, characterized in that the nonlinear frequency transposition function is a continuous function.
- Frequency compression device according to claim 13, characterized in that the nonlinear frequency transposition function is a piecewise approximation of a continuous function.
- Frequency compression device according to claim 14, characterized in that the perception-based frequency transposition function is defined by one of the following functions:- Bark function; or- ERB function; or- SPINC function.
- Frequency compression device according to one of the claims 13 to 16, characterized in that a look-up table is provided in which the frequency transposition function is defined, the look-up table being either operationally connected to the signal processing unit (4) or being integrated into the signal processing unit (4).
- Frequency compression device according to one of the claims 13 to 17, characterized in that at least one output transducer (7) is operationally connected to the signal processing unit (4).
- Frequency compression device according to one of the claims 13 to 17, characterized in that an inverse transformation unit (5) or any other synthesizing means are operationally connected to the signal processing unit (4).
- Frequency compression device according to claim 19, characterized in that at least one output transducer (7) is operationally connected to the inverse transformation unit (5) or to the other synthesizing means.
- Use of the frequency compression device according to one of the claims 13 to 20 in a communication device.
- Use of the frequency compression device according to one of the claims 13 to 20 in a hearing device.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP20040005270 EP1441562B1 (en) | 2003-03-06 | 2004-03-05 | Method for frequency transposition and use of the method in a hearing device and a communication device |
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP03005047A EP1333700A3 (en) | 2003-03-06 | 2003-03-06 | Method for frequency transposition in a hearing device and such a hearing device |
EP03005047 | 2003-03-06 | ||
EP20040005270 EP1441562B1 (en) | 2003-03-06 | 2004-03-05 | Method for frequency transposition and use of the method in a hearing device and a communication device |
Publications (3)
Publication Number | Publication Date |
---|---|
EP1441562A2 EP1441562A2 (en) | 2004-07-28 |
EP1441562A3 EP1441562A3 (en) | 2007-11-21 |
EP1441562B1 true EP1441562B1 (en) | 2010-03-31 |
Family
ID=8185870
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03005047A Withdrawn EP1333700A3 (en) | 2003-03-06 | 2003-03-06 | Method for frequency transposition in a hearing device and such a hearing device |
EP20040005270 Expired - Lifetime EP1441562B1 (en) | 2003-03-06 | 2004-03-05 | Method for frequency transposition and use of the method in a hearing device and a communication device |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP03005047A Withdrawn EP1333700A3 (en) | 2003-03-06 | 2003-03-06 | Method for frequency transposition in a hearing device and such a hearing device |
Country Status (3)
Country | Link |
---|---|
EP (2) | EP1333700A3 (en) |
DE (1) | DE602004026233D1 (en) |
DK (1) | DK1441562T3 (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10129659B2 (en) | 2015-05-08 | 2018-11-13 | Doly International AB | Dialog enhancement complemented with frequency transposition |
Families Citing this family (22)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
AU2003904207A0 (en) * | 2003-08-11 | 2003-08-21 | Vast Audio Pty Ltd | Enhancement of sound externalization and separation for hearing-impaired listeners: a spatial hearing-aid |
AU2004201374B2 (en) * | 2004-04-01 | 2010-12-23 | Phonak Ag | Audio amplification apparatus |
AU2005201813B2 (en) * | 2005-04-29 | 2011-03-24 | Phonak Ag | Sound processing with frequency transposition |
WO2007000161A1 (en) | 2005-06-27 | 2007-01-04 | Widex A/S | Hearing aid with enhanced high frequency reproduction and method for processing an audio signal |
EP1941782B1 (en) * | 2005-10-18 | 2018-07-18 | Widex A/S | Equipment for programming a hearing aid and a hearing aid |
WO2007135198A2 (en) | 2007-07-31 | 2007-11-29 | Phonak Ag | Method for adjusting a hearing device with frequency transposition and corresponding arrangement |
DE102008049466A1 (en) * | 2008-09-29 | 2010-04-08 | Siemens Medical Instruments Pte. Ltd. | Method for operating a hearing aid and hearing aid |
DE102008064382A1 (en) * | 2008-12-22 | 2010-07-08 | Siemens Medical Instruments Pte. Ltd. | Hearing device i.e. behind-the-ear hearing device, has transposition device for transposing part of frequency range of output signal, and control device releasing transposing of transposition device in cent steps |
DE102009021310B4 (en) | 2009-05-14 | 2011-02-24 | Siemens Medical Instruments Pte. Ltd. | Binaural hearing apparatus and method for operating a binaural hearing apparatus with frequency distortion |
DE102009058415B4 (en) * | 2009-12-16 | 2012-12-06 | Siemens Medical Instruments Pte. Ltd. | Method for frequency transposition in a hearing aid device and hearing aid device |
EP2372707B1 (en) | 2010-03-15 | 2013-03-13 | Svox AG | Adaptive spectral transformation for acoustic speech signals |
DK2375782T3 (en) | 2010-04-09 | 2019-03-18 | Oticon As | Improvements in sound perception by using frequency transposing by moving the envelope |
DE102010041643B4 (en) | 2010-09-29 | 2014-01-30 | Siemens Medical Instruments Pte. Ltd. | Hearing aid for frequency compression |
DE102010041653B4 (en) | 2010-09-29 | 2015-04-02 | Siemens Medical Instruments Pte. Ltd. | Method and apparatus for frequency compression with selective frequency shift |
DE102011006148B4 (en) | 2010-11-04 | 2015-01-08 | Siemens Medical Instruments Pte. Ltd. | Communication system with telephone and hearing device and transmission method |
DE102011006472B4 (en) | 2011-03-31 | 2013-08-14 | Siemens Medical Instruments Pte. Ltd. | Method for improving speech intelligibility with a hearing aid device and hearing aid device |
DE102011006515A1 (en) | 2011-03-31 | 2012-10-04 | Siemens Medical Instruments Pte. Ltd. | Method for improving speech intelligibility with a hearing aid device and hearing aid device |
DE102011006511B4 (en) | 2011-03-31 | 2016-07-14 | Sivantos Pte. Ltd. | Hearing aid and method for operating a hearing aid |
CN103733259B (en) * | 2011-06-23 | 2016-10-19 | 索诺瓦公司 | For running method and the hearing device of hearing device |
US9179222B2 (en) | 2013-06-06 | 2015-11-03 | Cochlear Limited | Signal processing for hearing prostheses |
US11184715B1 (en) | 2020-10-05 | 2021-11-23 | Sonova Ag | Hearing devices and methods for implementing an adaptively adjusted cut-off frequency |
US11962980B2 (en) | 2021-01-28 | 2024-04-16 | Sonova Ag | Hearing evaluation systems and methods implementing a spectro-temporally modulated audio signal |
Family Cites Families (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
FR2494988B1 (en) * | 1980-11-28 | 1985-07-05 | Lafon Jean Claude | IMPROVEMENTS ON HEARING AID DEVICES |
US4982434A (en) * | 1989-05-30 | 1991-01-01 | Center For Innovative Technology | Supersonic bone conduction hearing aid and method |
AT398670B (en) * | 1991-11-13 | 1995-01-25 | Viennatone Gmbh | METHOD FOR SHIFTING THE FREQUENCY OF SIGNALS |
FR2706103B1 (en) * | 1993-06-03 | 1997-01-31 | Ericsson Ge Mobile Communicat | Radiotelephone apparatus. |
US5479522A (en) * | 1993-09-17 | 1995-12-26 | Audiologic, Inc. | Binaural hearing aid |
US6577739B1 (en) * | 1997-09-19 | 2003-06-10 | University Of Iowa Research Foundation | Apparatus and methods for proportional audio compression and frequency shifting |
SE9902057D0 (en) * | 1999-06-03 | 1999-06-03 | Ericsson Telefon Ab L M | A Method of Improving the Intelligence of a Sound Signal, and a Device for Reproducing a Sound Signal |
-
2003
- 2003-03-06 EP EP03005047A patent/EP1333700A3/en not_active Withdrawn
-
2004
- 2004-03-05 DE DE200460026233 patent/DE602004026233D1/en not_active Expired - Lifetime
- 2004-03-05 DK DK04005270T patent/DK1441562T3/en active
- 2004-03-05 EP EP20040005270 patent/EP1441562B1/en not_active Expired - Lifetime
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10129659B2 (en) | 2015-05-08 | 2018-11-13 | Doly International AB | Dialog enhancement complemented with frequency transposition |
Also Published As
Publication number | Publication date |
---|---|
EP1333700A2 (en) | 2003-08-06 |
EP1441562A2 (en) | 2004-07-28 |
DE602004026233D1 (en) | 2010-05-12 |
EP1333700A3 (en) | 2003-09-17 |
DK1441562T3 (en) | 2010-07-19 |
EP1441562A3 (en) | 2007-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US7248711B2 (en) | Method for frequency transposition and use of the method in a hearing device and a communication device | |
EP1441562B1 (en) | Method for frequency transposition and use of the method in a hearing device and a communication device | |
US20040175010A1 (en) | Method for frequency transposition in a hearing device and a hearing device | |
EP1661434B1 (en) | Sound enhancement for hearing-impaired listeners | |
US8098859B2 (en) | Methods, devices and systems using signal processing algorithms to improve speech intelligibility and listening comfort | |
US7483831B2 (en) | Methods and apparatus for maximizing speech intelligibility in quiet or noisy backgrounds | |
JP4705300B2 (en) | Hearing aid incorporating signal processing technology | |
US6212496B1 (en) | Customizing audio output to a user's hearing in a digital telephone | |
US6980665B2 (en) | Spectral enhancement using digital frequency warping | |
Kates et al. | Speech intelligibility enhancement | |
EP2503794A1 (en) | Audio processing device, system, use and method | |
US20230127309A1 (en) | Signal processing in a hearing device | |
CN111107478B (en) | Sound enhancement method and sound enhancement system | |
AU2002300314B2 (en) | Apparatus And Method For Frequency Transposition In Hearing Aids | |
US9232326B2 (en) | Method for determining a compression characteristic, method for determining a knee point and method for adjusting a hearing aid | |
Edwards | Signal processing techniques for a DSP hearing aid | |
Noordhoek et al. | Measuring the threshold for speech reception by adaptive variation of the signal bandwidth. II. Hearing-impaired listeners | |
US9124963B2 (en) | Hearing apparatus having an adaptive filter and method for filtering an audio signal | |
EP2337378A2 (en) | Method for frequency transposition in a hearing aid and hearing aid | |
Muñoz et al. | Frequency lowering processing for listeners with significant hearing loss | |
AU2011226820B2 (en) | Method for frequency compression with harmonic correction and device | |
Preves | Approaches to noise reduction in analog, digital, and hybrid hearing aids | |
US20220406328A1 (en) | Hearing device comprising an adaptive filter bank | |
EP4040806A2 (en) | A hearing device comprising a noise reduction system | |
Holube et al. | Hearing-aid technology |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
17P | Request for examination filed |
Effective date: 20050111 |
|
PUAL | Search report despatched |
Free format text: ORIGINAL CODE: 0009013 |
|
AK | Designated contracting states |
Kind code of ref document: A3 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LI LU MC NL PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL LT LV MK |
|
17Q | First examination report despatched |
Effective date: 20080102 |
|
AKX | Designation fees paid |
Designated state(s): CH DE DK LI |
|
GRAP | Despatch of communication of intention to grant a patent |
Free format text: ORIGINAL CODE: EPIDOSNIGR1 |
|
GRAS | Grant fee paid |
Free format text: ORIGINAL CODE: EPIDOSNIGR3 |
|
GRAA | (expected) grant |
Free format text: ORIGINAL CODE: 0009210 |
|
AK | Designated contracting states |
Kind code of ref document: B1 Designated state(s): CH DE DK LI |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: EP |
|
REF | Corresponds to: |
Ref document number: 602004026233 Country of ref document: DE Date of ref document: 20100512 Kind code of ref document: P |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: NV Representative=s name: TROESCH SCHEIDEGGER WERNER AG |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: T3 |
|
PLBE | No opposition filed within time limit |
Free format text: ORIGINAL CODE: 0009261 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT |
|
26N | No opposition filed |
Effective date: 20110104 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DK Payment date: 20140325 Year of fee payment: 11 |
|
REG | Reference to a national code |
Ref country code: DK Ref legal event code: EBP Effective date: 20150331 |
|
PG25 | Lapsed in a contracting state [announced via postgrant information from national office to epo] |
Ref country code: DK Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES Effective date: 20150331 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R084 Ref document number: 602004026233 Country of ref document: DE |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: DE Payment date: 20230329 Year of fee payment: 20 |
|
P01 | Opt-out of the competence of the unified patent court (upc) registered |
Effective date: 20230530 |
|
PGFP | Annual fee paid to national office [announced via postgrant information from national office to epo] |
Ref country code: CH Payment date: 20230402 Year of fee payment: 20 |
|
REG | Reference to a national code |
Ref country code: DE Ref legal event code: R071 Ref document number: 602004026233 Country of ref document: DE |
|
REG | Reference to a national code |
Ref country code: CH Ref legal event code: PL |