JP3987429B2 - Method and apparatus for determining acoustic environmental conditions, use of the method, and listening device - Google Patents

Description

  The present invention relates to a method for determining an acoustic environment situation, use of the method, an apparatus for determining an acoustic environment situation, and a hearing device.

  Modern hearing devices can be adjusted to suit changing acoustic environmental conditions using various listening programs. This provides the hearing instrument wearer with the optimal use of the hearing apparatus in all situations.

  The selection of the listening program can be performed by remote control or by switch operation on the listening device itself. However, switching between the various listening programs is cumbersome or difficult for many wearers and sometimes even impossible. It is not always easy for a skilled hearing instrument wearer to determine which program gives the best comfort and the best speech clarity at what point. Therefore, automatic recognition of surrounding acoustic environment conditions and automatic switching of the listening program in the listening device corresponding to this are expected.

  At present, several methods are known for automatic classification of acoustic environmental conditions. In any of these methods, some features are extracted from the input signal obtained by one or more microphones in the hearing device. Based on these features, by applying a specific algorithm, the pattern recognition device determines which acoustic environment situation the analyzed input signal belongs to. These known methods are on the one hand in the analysis of the various features applied to the definition of the acoustic environment situation (signal analysis) and on the other hand the pattern recognition technique (signal identification) used to classify these features. Are different from each other.

  The publication of international patent application WO01 / 20965 discloses a method and apparatus for determining the acoustic environment status. Here, one-stage processing of the acoustic input signal is performed in the feature extraction device, and the features extracted thereby are classified in a classification device connected downstream, and classification information is generated. With this known teaching, good results are obtained, especially when auditory-based features are applied. However, further improvements are expected in the field of application of hearing devices. This is because it is necessary to classify the acoustic environment state particularly accurately in the field of such a hearing device. At the same time, the classification of a large number of sounds, including for example music and noise, creates a further difficulty. In other words, since the sound has a very wide range due to its various generation processes, the characteristics of the sound classification vary accordingly. For example, the sound classification “noise” includes various noises such as background conversation, station noise, hair dryer sound, and the sound classification “music” includes, for example, pop music, classical music, solo, singing, etc. including.

  In particular, due to the great diversity of these sound classifications, it is very difficult to obtain a good recognition rate in a feature extraction device and subsequent classification devices using known processing methods. That is, the reliability of the recognition system can be improved by the selection of appropriate features, such as the use of auditory-based features first disclosed in international patent application WO 01/20965, while general sound classification Are so varied that it is very difficult to clearly and unambiguously distinguish these sound classifications from each other.

  Therefore, an object of the present invention is to provide a method for determining an acoustic environment state that is more reliable and accurate than the conventional one.

  This problem is solved by the method according to claim 1 in the claims. Preferred forms, use of the method, apparatus and hearing device according to the invention are described in the further claims.

  The classification process is performed by at least two processing stages, and the acoustic input signal is processed by a method of multiple processing stages, preferably each processing stage having a feature extraction phase and an identification phase for classifying the features. Coarse and fine classification of environmental conditions is obtained. By the method of the present invention, for example, a situation where the sound classification “pop music” is erroneously classified into the sound classification “conversation in noise” can be reliably avoided. The method according to the invention also makes it possible to subdivide the general sound classification, eg “noise”, into subclasses such as “traffic noise” or “conversational background noise”. For example, special situations such as “in-vehicle noise” appearing in the car can be recognized. In general, room features can be identified and taken into account for further processing of important signal parts. Furthermore, it has been found that the method of the present invention can localize the sound source and thereby recognize the possibility of recognizing the presence of a specific sound source in a sound source mixed with many sounds.

  In the following, the invention will be described in greater detail by way of example with reference to the accompanying drawings.

  FIG. 1 shows a conventional one-stage processing apparatus for determining acoustic environment conditions. This apparatus includes a feature extraction apparatus F, a classification apparatus C, and a post-processing apparatus P as necessary in the order in which they are connected.

  For example, an acoustic input signal IN captured by a microphone is input to a feature extraction device F, where feature extraction is performed.

  Regarding feature extraction in speech signals, J. et al. M.M. Kates' paper “Classification of background noise applied to hearing aids” (1995 Acoustical Journal 97 (1), pages 461-469) proposes to perform temporal level variation and spectrum analysis of sound. To do. European patent EP-B1-0732036 also proposes an analysis of amplitude histograms for the same purpose. In the latest, this feature extraction has also been studied and implemented by analyzing various modulation frequencies. In this regard, two papers such as Ostendorf are helpful. That is, “empirical classification of various acoustic signals and speech by modulation frequency analysis” (1997 DAGA 97, pages 608 and 609) and “classification of acoustic signals based on modulation spectrum analysis applied to digital listening apparatus” (1998). Year DAGA 98, pages 402 and 403). An analogy to this is published in Edwards et al.'S "Signal Processing Algorithm for a New Digital Hearing Device Based on Software" (1998, The Hairing 51, pages 44-52). Another sound feature is the volume level itself or the zero crossover rate, which is described in, for example, H.D. L. It is described in the Hirsch paper "Statistical Signal Characterization Method" (1992 Artech House). As described above, the features used in the conventional speech signal analysis are purely technology-based.

  Furthermore, the publication of the aforementioned international patent application WO 01/20965 disclosed for the first time that the use of auditory-based features in addition to the technical features described above is effective.

  In FIG. 1, the feature M extracted by the feature extraction device F is guided to the classification device C, where one of the known pattern recognition methods is basically applied to the sound classification. Particularly suitable pattern recognition systems are so-called distance determination methods (Abstandsschaetzer), Bayesian classification methods, fuzzy logic systems or neural networks. The first two of these methods are described in the book "Pattern Classification and Scene Analysis" by Richard O. Duda and Peter E. Hart (1973 John Wiley & Sons). Regarding neural networks, reference is made to Christopher M. Bishop's paper "Neural Networks for Pattern Recognition" (1995 Oxford University Press). The following publications are also helpful. That is, Ostendorf et al., “Classification of Acoustic Signals Based on Modulation Spectrum Analysis Applied to a Digital Hearing Device” (1998 Audiology, pp. 148-150); Feldbusch's “Neural Network Speech Recognition” (1998 Audiology, pages 30-36); European Patent Publication EP-A1-0814636; and US Pat. No. 5,604,812. Each of these pattern recognition methods models only the static characteristics of the sound in the target range. In addition to these pattern recognition methods, the above-mentioned international patent application WO01 / There is a method described in the publication of No. 20965.

  In the prior art of FIG. 1, class information KI is obtained by a processing procedure performed in the classification device C. In some cases, this class information KI is sent to the post-processing device P, where possible classification attribution is corrected. As a result, corrected class information KI 'is obtained.

FIG. 2 shows one basic configuration for the apparatus of the present invention. This device has two processing stages S1 and S2, and each stage S1 and S2 is provided with a feature extraction device F1 and F2 and a corresponding classification device C1 and C2, respectively. The first input signal IN is input to the feature extraction devices F1 and F2 in both processing stages S1 and S2. The feature extraction devices F1 and F2 are connected to the corresponding classification devices C1 and C2, respectively. What should be noted here is that the class information KI1 classified into the speech class based on the calculation by the classification device C1 in the first processing stage S1 acts on the classification device C2 in the second processing stage S2. is there. That is, this action actually selects, for example, one of several possible pattern recognition methods and applies the class information KI1 thereby to the sound classification by the classification device C2 in the second processing stage S2. It is implemented to reflect.

The basic configuration of the present invention shown in FIG. 2 will be further described below based on specific examples.

  The feature extraction device F1 extracts the tone (Tonalitaet), the spectrum centroid (CGAV), the variation of the spectrum centroid (CGFS), the spectrum width and the settling time, and then the HMM (Hidden Markov model) in the classification device C1. Classification is performed using a classification technique. Thereby, the input signal IN is classified into one of the following classifications, that is, “conversation (human voice)”, “conversation in noise”, “noise”, or “music” by the HMM classification method. As a result, class information KI1 is obtained and input to the classification device C2 of the second processing stage S2. On the other hand, in the feature extraction device F2 of the second processing stage S2, a second feature set is extracted. In the feature extraction device F2, in addition to the tone, spectral centroid, and fluctuation of the spectral centroid as a feature, a sound harmonic structure (pitchvar) deformation (hereinafter referred to as Pitchvar) is extracted as a further feature. Based on these characteristics, the result of the first processing stage S1 is checked against the rule-based classification technique in the classification device C2, and is corrected if necessary. This rule-based classification approach is based on the above four features and includes several simple new decisions that are oriented taking into account the following points:

When the numerical value of the feature is completely outside the allowable numerical value range of the class information KI1 determined by the HMM classification method in the first classification device C1, the characteristic “tone” is corrected for the correction in each class of speech. used. The tone is expected to be high in “music”, medium in “conversation”, slightly lower in “conversation in noise”, and low in “noise”. For example, when the input signal IN is classified into the class “conversation” by the classification device C1, the corresponding signal portion of the input signal IN greatly fluctuates with respect to the classification device C1. It is expected that it has been instructed. On the other hand, when the tone for the input signal IN is very low, it is considered that “conversation in noise” is not a “classic conversation” with a high probability, but “conversation in noise” would be the correct classification. Similar considerations can be made for the other three features: harmonic structure deformation (Pitchvar), spectral centroid (CGAV) and spectral centroid variation (CGFS). Thereby, a rule for rule-based classification that can be applied in the classification device C2 can be configured as follows:

Above Symbol form status of, has also been found with a surprise that almost the same characteristics are used in the second treatment stage S2 not only in the first processing stage S1. Furthermore, it has been shown that the use of the feature “tone” is most suitable for correcting errors that occur in the classification device C1. Thereby, it can be determined that the application of tone is the most important for the rule-based classification method.

Result of the test by the shape condition described above, the simple two-stage processing scheme has achieved at least improvement of 3% hit rate compared to one-step process method. In some cases, a 91% improvement in hit rate was possible.

FIG. 3 is a diagram showing an embodiment of the present invention in the same manner as in FIG. Here, an n-stage processing method is handled. The processing stages S1 to Sn have, as a result of the above consideration, the feature extraction devices F1 to Fn and the classification devices C1 to Cn that are connected to the downstream and generate corresponding class information KI1 to KIn, respectively. . Furthermore , post-processing devices P1 to Pn that generate modified class information KI1 ′ to KIn ′ are provided in each of the processing stages S1 to Sn or a plurality of stages.

  In contrast to the embodiment of FIG. 2, the embodiment shown in FIG. 3 is particularly suitable for so-called coarse-fine classification. In the coarse-fine classification, the result obtained in the processing stage i is refined in the next processing stage i + 1. That is, the coarse classification is performed in the processing stage arranged at the upper level, and then the detailed classification is performed in the processing stage arranged at the lower level by the specific feature extraction and classification method based on the coarse classification. This scheme can also be seen in a scheme where a hypothesis is generated at a higher processing stage and this hypothesis is reviewed, i.e., confirmed or rejected, at a lower processing stage. In this regard, it is clear that the hypotheses (coarse classification) generated in the higher processing stages can be provided by other information sources, in particular by manual means such as remote control or switching. . This is performed by a control amount (Stellgroesse) ST, as representatively shown in the first processing stage S1 of FIG. 3, and thereby, for example, the calculation performed in the classification device C1 can be overturned. Obviously, the control amount ST can also be given to the classification devices C2 to Cn in different processing stages or to the post-processing devices P1 to Pn in the processing stages S1 to Sn.

  In a classification system according to the present invention having a plurality of processing stages S1 to Sn, the following targets can be set for each of the processing stages S1 to Sn, although not essential: a rough classification, a fine classification, a sound source Localization, testing for the presence of a particular sound source, such as car interior noise, or extracting certain signal parts of the input signal, such as echoes due to room features. Therefore, the processing stages S1 to Sn are different in the sense that different features are extracted and different classification methods are applied.

  In another embodiment of the invention, the localization of the individual signals in the mixture of the various signal parts is performed in the first processing stage S1, and the coarse classification of this localized signal source is the second The rough classification fine classification obtained in the second processing stage S2 is performed in the third processing stage S3.

  Following the localization of the sound source implemented in the first processing stage S1, it is possible to implement directional filtering, for example by means of a multiple microphone technique.

  Obviously, the feature extraction devices F1 to Fn can be divided into several classification devices C1 to Cn. That is, the results of the feature extraction devices F1 to Fn can be used in some classification devices C1 to Cn. Furthermore, it is also conceivable to use the classification devices C1 to Cn in several processing stages S1 to Sn. After all, the class information KI1 to KIn obtained in the different processing stages S1 to Sn or the modified class information KI1 ′ to KIn ′ is weighted in various forms in order to obtain a final classification. Is possible.

  FIG. 4 shows another embodiment according to the invention having a plurality of processing stages S1 to Sn. Unlike the embodiment shown in FIG. 3, the class information KI1 to KIn is not only used in the processing stage immediately after each of the class information KI1 to KIn, but also used in all the subsequent processing stages as necessary. it can. Similarly, the results of the higher processing stages S1 to Sn can have an effect on the subsequent feature extraction devices F1 to Fn or the features to be extracted.

  In the embodiment of FIG. 4 as well, post-processing devices P1 to Pn are provided, whereby intermediate classification results can be obtained, and modified class information KI1 'to KIn' can be generated.

  FIG. 5 shows a multi-stage processing apparatus for determining an acoustic environment situation as a further embodiment of the present invention. Similar to the plurality of processing stages S1 to Sn in the embodiment of FIGS. 3 and 4, feature extraction apparatuses F1 to Fn and classification apparatuses C1 to Cn corresponding to the processing stages are provided. The class information KI1 to KIn obtained in the processing stages S1 to Sn is guided to the determination device FD, where the final classification is obtained by generating the class information KI. In the decision device FD, the generation of feedback signals to be applied to the feature extraction devices F1 to Fn and / or to the classification devices C1 to Cn is carried out as required, so that, for example, one or several of these processing devices Can be adjusted or all classification devices C1-Cn can be replaced.

  It should be noted that the feedback action of the processing apparatus and its connection in the embodiment of FIGS. 3 to 5 are not limited to the illustrated form, and some of these feedback actions and their connections can be omitted. Basically, these processing apparatuses can be arbitrarily combined to obtain each required configuration.

  Further, when the present invention is applied to a hearing device, it is also conceivable to distribute several processing stages between two hearing devices, i.e., a hearing device attached to the left and right ears, respectively. Information exchange between both ears in this form is performed through wired or wireless communication coupling.

  FIG. 6 is a diagram for explaining possible configurations and combinations related to the above-described processing apparatus according to a further embodiment of the present invention in which the one in FIG. 2 is simplified. While there is only one feature extraction apparatus F1, two processing stages S1 and S2 are provided. The first processing stage S1 includes a feature extraction device F1 and a classification device C1. The second processing stage S2 uses the same features as those used in the first processing stage S1. Therefore, in the second processing stage S2, no new feature calculation is required, and the result of the feature extraction device F1 in the first processing stage S1 can be used. Thereby, in the second processing stage S2, the classification method is actually changed depending on the class information KI1 in the first processing stage S1.

  FIG. 7 shows an embodiment in which the device according to the present invention is used for a listening device, in which a signal transmission device 200 is basically provided as a listening device. Reference numeral 100 denotes a multi-stage processing apparatus realized by the embodiment shown in any of FIGS. The input signal IN is input not only to the multistage processing device 100 but also to the signal transmission device 200. In the signal transmission device 200, the acoustic input signal IN is processed using the class information KI1 to KIn obtained by the multistage processing device 100 or the modified class information KI1 'to KIn'. In this case, preferably, as described in the aforementioned international patent application WO 01/20965, selection of an appropriate listening program is performed based on the determined acoustic environment situation.

  That is, in FIG. 7, reference numeral 300 indicates a manual input device, and the input device 300 controls the multistage processing device 100 or the signal transmission device 200 according to circumstances, for example, by wireless communication as illustrated. For the signal transmission device 200 as a listening device, reference is made to the embodiment in the international patent application WO 01/20965 which forms an integral part of the description of the invention.

As a classification method applicable to all the above embodiments according to the present invention, any of the following can be used:
-Hidden Markov model classification method;
-Fuzzy logic;
-Bayesian classification method;
-Rule-based classification methods;
-Neural network-Minimum distance judgment method

  The feature extraction devices F1 to Fn (FIGS. 2 to 7) can extract technology-based and / or auditory-based features. The technical and auditory base features are again described in detail with respect to the embodiments in the international patent application WO 01/20965.

  The method of the invention for determining the acoustic environment situation is preferably used for selection of a listening program in a listening device. However, it is also worth considering the use of the device of the invention for speech recognition or the application of the method of the invention.

The figure which shows the conventional one-stage processing apparatus which determines an acoustic environment. The figure which shows the fundamental structure of the apparatus by this invention which implements a two-stage process. The figure which shows embodiment of the apparatus by this invention which implements multistage processing. The figure which shows another embodiment of the apparatus by this invention which implements multistage processing. The figure which shows another embodiment of the apparatus by this invention which implements multistage processing. The figure which shows further embodiment of this invention which simplified the two-stage processing apparatus of FIG. The figure which shows embodiment which uses the apparatus of this invention shown in FIGS. 2-6 for a listening apparatus.

Claims (18)

In a method for determining an acoustic environment situation, the method comprises:
Acoustic input signal (IN) to at least two processing stages become as processing (S1, ···, Sn) in,
Said processing stages (S1, ···, Sn) in one at least of, setting the extraction phase for extracting features from the input signal (IN),
Each of the processing stages (S1, ..., Sn) is provided with an identification phase for classifying the extracted features,
In at least one of the processing stages (S1,..., Sn), class information (KI1,..., KIn; KI1 ′,. .., KIn '),
A processing mode in a certain processing stage (S1,..., Sn) is selected based on class information obtained in another processing stage (S1,..., Sn), and the processing stage (S1,. .., Sn) is provided with a post-processing phase following the extraction phase, and the modified class information (KI1 ′,..., KIn) of the class information (KI1,..., KIn) is provided in the post-processing phase. ..., wherein the Rukoto such so as to generate the KIN ').   The class information (KI1,..., KIn; KI1 ′,..., KIn ′) obtained in the identification phase of the stage (i) having the processing stage (S1,. 2. The method according to claim 1, wherein a processing mode in the next stage (i + 1) of the processing stage (S1,..., Sn) is determined.   Based on the class information (KI1,..., KIn; KI1 ′,..., KIn ′) obtained in the stage (i) of the processing stage (S1,..., Sn). In the extraction phase of the next stage (i + 1) of the stage (S1,..., Sn), a particular feature is selected and / or the next stage of the processing stage (S1,..., Sn) ( The method according to claim 2, wherein a specific classification method is selected in the identification phase of i + 1). In the identification phase, the following classification method:
Hidden Markov model classification method;
Fuzzy logic;
Bayesian classification method;
Rule-based classification method;
neural network;
Minimum distance determination method;
A method according to any one of claims 1 to 3 , characterized in that one of the following is used. In the extraction phase, technology-based and / or method according to any one of claims 1 to 4, characterized by comprising extracting the auditory-based features. Method characterized by at least one hearing device using the method according to any one of claims 1 to 5, adapted to prevailing acoustic environment conditions. Adjusting a listening program in at least one hearing device or a signal transmission function between at least one microphone and a speaker based on the current acoustic environment situation determined using the method of claim 6. how to. Method characterized in that the speech recognition by the method according to any one of claims 1 to 5. An apparatus for determining an acoustic environment condition in an input signal,
At least two processing stages (S1,..., Sn),
At least one of the processing stages (S1,..., Sn) has a feature extraction device (F1,..., Fn), and a classification device ( C1, ···, Cn) is provided, et al. are,
The input signal (IN) is input to the feature extraction device (F1,..., Fn), and its output is guided to at least two of the classification devices (C1,..., Cn), and At least one of the at least two classification devices (C1,..., Cn) is connected to the other of the at least two classification devices (C1,..., Cn), so that another processing stage (S1,. ..., class information (KI1 in Sn), · · ·, Ri greens so as to adjust the process in accordance with KIN), and said processing stage (S1, · · ·, obtained in both one least of Sn) Further, the class information (KI1,..., KIn) is input to the post-processing device (P1,..., Pn), thereby correcting the class information (KI1) of the class information (KI1,..., KIn). ', ..., KIn') generated And wherein the Rukoto such as is. 10. The device according to claim 9 , wherein each of the processing stages (S1,..., Sn) is provided with the feature extraction device (F1,..., Fn). The class information (KI1,..., KIn; KI1 ′,..., KIn ′) is input to another stage of the processing stage (S1,..., Sn). The apparatus according to claim 9 or 10 . The class information (KI1,..., KIn; KI1 ′,..., KIn ′) of the stage (i) with the processing stage (S1,..., Sn) is stored in the processing stage (S1,. The device according to any one of claims 9 to 11 , wherein the device is input to the next stage (i + 1) of Sn). The class information (KI1,..., KIn; KI1 ′,..., KIn ′) of the stage (i) with the processing stage (S1,..., Sn) is stored in the processing stage (S1,. .., Input to the feature extraction device (F1,..., Fn) of the next stage (i + 1) of Sn) and / or the next stage of the processing stage (S1,..., Sn) 13. The device according to claim 12 , wherein the device is inputted to the classification device (C1,..., Cn) of (i + 1). Said processing stages (S1, ···, Sn) the class information for all stages of the (KI1, ···, KIn) according to claim 9 or 10, characterized in that is formed by the input to the decision unit (FD) The device described. The determination device (FD) is connected to at least one of the feature extraction devices (F1,..., Fn) and / or at least one of the classification devices (C1,..., Cn). 15. The apparatus of claim 14, wherein: Generates class information (KI1,..., KIn; KI1 ′,..., KIn ′) with a signal transmission device (200) connected to at least one microphone and to a transducer device, in particular a speaker. Therefore, the apparatus according to any one of claims 9 to 13 is provided, and the class information (KI1,..., KIn; KI1 ′,..., KIn ′) is included in the signal transmission device (200). Listening device characterized by being input to the sound. An input device (300) is provided, the input device (300) being connected to the signal transmission device (200) and / or to the device according to any of claims 9 to 13. Item 16. The hearing device according to Item 16 . 18. A hearing device according to claim 17 , wherein there is wireless communication between the input device (300) and the signal transmission device (200).

Images