KR100917419B1 - Speaker recognition systems - Google Patents

Abstract

In a speech recognition (identification and / or verification) method and system, speech models for registered speakers consist of sets of feature vectors representing the smoothed frequency spectrum of each of a plurality of frames, clustering into feature vectors of a frame. The algorithm is applied to obtain a reduced data set representing the original speech sample, with adjacent frames overlapping by at least 80%. This type of speech model shows the transient independence by modeling the static components of the speech sample. An identifier scheme is used in which modeling and classification processes are selected to give a false rejection rate that is approximately equal to zero. Each registered speaker is associated with a cohort of some other registered speakers, and the test sample is always matched with one of the relevant cohorts or claimed identification information. Thus, the overall error rate of the system depends only on the false acceptance rate, which is determined by the cohort size. False error rates are further reduced using multiple parallel modeling and / or classification processes. The speech models are normalized prior to classification using a normal model derived from one of the registered speaker samples or from the test speech sample (most preferably from the claimed identification registration sample).

Speaker recognition, speech recognition, score, feature vector, overlap, clustering

Description

Speaker recognition system {SPEAKER RECOGNITION SYSTEMS}

The present invention relates to a system, method and apparatus for performing speaker recognition.

Speaker recognition includes related areas for speaker verification and speaker identification. The primary purpose of speaker recognition is to identify the speaker's claimed identity from the utterances (known as "verification") or to recognize the speaker from the pronunciation ("identification"). Known as "). This verification and identification uses the human voice as a biometric measure and assumes a unique relationship between pronunciation and the person who pronounces it. This unique relationship allows both verification and identification. Speaker recognition technology analyzes test pronunciation and compares it with a template or model known to the person being recognized and verified. The effectiveness of such a system depends on the quality of the algorithms used in the process.

Speaker recognition systems have as many applications as possible. According to a further aspect of the present invention, speaker recognition technology may be used to permanently mark an electronic document with a biometric print for everyone viewing and editing its contents. This creates an audit trail to identify all of the users and to identify the number of connections and modifications. Since the user mark is biometric, it is very difficult for the user to challenge the reliability of the mark.

Other biometric measurement means can provide a reference for possible recognition systems such as iris scanning, fingerprint and facial features. All of these measuring means require additional recording hardware, while speaker recognition can be used with all voice inputs, such as over a telephone line, or using a standard multimedia personal computer without modification. Such techniques can be used in combination with other security measures and other biometric means for enhanced security. From the user's point of view, the behavior of such a system is very simple.

For example, if an online document is requested, the person requesting a connection would be required to provide the voice sample. This may be a simple prompt from the client software ("speak this phrase") or something similar. The pronounced phrases are then transmitted to a speech recognition server or database server via any data network, such as the Internet, to be stored as a key associated with the document and used to activate the document at a particular time. Thus, a permanent record for a document can be created over time to provide an audit trail for that document. The speaker authentication server may maintain a series of models (models) for all currently registered people and a history record of previously registered people.

Speaker recognition systems rely on some unique features from human speech. Again, this depends on the way in which human voices are spoken using vocal tracts and nasal tracts. For practical purposes, saints and nasal sinus can be regarded as two connected tubes capable of resonating in a similar way to musical instruments. The resonance generated thereby depends on the diameter and length of the tubes. In human speech structure, to some extent, the length and diameters of the pipe sections are controlled by articulators (primarily tongue, jaw, lips and soft palate or velum). Such resonances on the spectrum are called formant frequencies, and normally there are about four formant frequencies on a typical speech spectrum.

Like a musical instrument, the constriction of airflow occurs so that sound only occurs when vibrations and turbulence occur. In the case of the human voice, major vibrations occur when contractions occur in the glottis (vocal cords). When this happens, voiced sounds, usually vowel-like sounds, are produced. If such contractions occur in the tongue or thereby in the mouth, turbulence is produced (a hissing type of sound), and the generated voice is "s", "sh", "th (T) "or the like. From an engineering point of view, this is similar to the source signal (the result of the contraction) applied to a filter with the general characteristics of the vocal tract (ie the same resonances), and the resulting output signal is a speech sound. Becomes The actual speech is generated by dynamically changing the positions of the articulation organs.

All existing speaker recognition systems perform similar computations. These systems operate by creating a model or model for the registered speaker. This model is created by two main steps applied to speech samples: spectral analysis and statistical analysis. Subsequent recognition of the input speech sample by modeling the input sample (test pronunciation) in the same manner as in the speaker registration process and by performing pattern / classification matching of the input model against the registered speaker's database. Processing is performed. Existing systems differ in the way they are taken when performing all or some of these steps. In conventional (industrial standard) systems, spectroscopic analysis includes Linear Predictive Coding (LPC) / Cepstral Analysis (LPCC) or FFT / subbanding. Then there is a statistical analysis technique called Hidden Markov Modeling (HMM), where the above classification step is used to match and "impostor cohort" against the claimed speaker model. Or a combination of matches against a "world model" (ie, a series of other speaker models).

To enable effective processing of speech samples, all speaker recognition systems use time slices called frames, where the pronunciation is divided into frames, and each frame is processed again. The frames may or may not be the same in size with each other, and may or may not overlap. An example of a typical time signal representation of speech utterance divided into frames is shown in FIG. 1 of the accompanying drawings. A general speaker recognition system is shown in the form of a block diagram in FIG. 2, where test pronunciation is input filter 10, spectral analysis (LPCC) stage 12, and statistical analysis stage. analysis (HMM) stage) 14, and then before generating information about the speaker's identity (identification) or the speaker's claimed identity (verification). Using the database 18 of speaker models (registered speaker data-sets), the thresholding and processing by the score normalization and speaker classifier 16 are illustrated.

Such systems have several disadvantages or limitations. First, existing spectral analysis techniques produce poor modeling by generating limited and incomplete feature sets. Secondly, HMM technology is a "black box" method that combines ease of use with superior performance, even at the expense of transparency. The relative importance of the features extracted by these techniques is not visible to the designer. Third, model-against-model comparisons cannot be effectively performed on the characteristics of HMM models. Therefore, important structural configurations contained in registered speakers cannot be effectively analyzed and used to improve system performance. Fourth, HMM technology uses temporal information to construct the model, and thus becomes vulnerable to mimics who mimic the voices of others by temporarily changing pitch or the like. Fifthly, the world model / impostor cohort used by the system to test the claimed speaker's pronunciation cannot be easily optimized.

The performance of the speaker recognition system depends on the following points. That is, if true speaker pronunciation is tested against a model for that speaker, it will depend on the fact that the imposter pronunciation produces a score lower than the score produced when tested against the same model. This allows the allow / reject threshold to be set. Successive tests by the true speaker will not produce the same score. Instead, the scores will form a statistical distribution. However, the mean of the true speaker distribution will be significantly lower than the mean of the imposter distributions tested for the same model. This is shown in Fig. 3, in which 25 scores were made for each of the eight speakers when speaker 1 was the true speaker. 3, it can be seen that the scores of some speakers can be a problem closer to the true speaker than others.

The present invention relates to an improved speaker recognition method and system that provides improved performance over existing systems. The invention provides, in various other aspects, improved spectral analysis, transparency in its statistical analysis, improved modeling, models that can be compared so that the data-set structure can be analyzed and used to improve system performance, Provides improvements, including but not limited to improved classification methods, the use of statistically independent / partially independent parallel processing to improve system performance, and the like.

In addition, the present invention provides a computer program for implementing the method and system of the present invention, data carriers and storage media encoded with such a program, a data processing apparatus and system applied for implementing the method and system, and the device And data processing systems and apparatuses including systems.

Various aspects and preferred features of the invention are defined in the appended claims.
Embodiments of the present invention will now be described by way of example only with reference to the drawings.

1 is a diagram illustrating a time signal representation of an example of dividing speech utterance into frames.

2 is a block diagram showing a conventional conventional speaker recognition system.

3 is a diagram illustrating a speaker recognition score distribution of the number of speakers tested.

4 is a block diagram showing a first embodiment of the present invention.

5 is a block diagram showing a second embodiment of the present invention.

6 is a block diagram showing a third embodiment of the present invention.

7 is a block diagram showing another embodiment of a speaker recognition system according to the present invention.

8A is a diagram illustrating a time signal representation of an example of speech pronunciation divided into frames.

FIG. 8B is a diagram illustrating the corresponding frequency spectrum and the smoothed frequency spectrum of one of the frames of FIG. 8A.

9 shows the difference between the frequency spectra of two mis-aligned frames.

10 is a diagram illustrating a distribution of accumulated frame scores drawn with respect to the frequency of occurrence of misaligned frames.

FIG. 11A is a diagram showing the same accumulated score distributions as in FIG. 3 for comparison with FIG. 11B, and showing the corresponding accumulated score distributions obtained using the speaker recognition system according to the present invention.

FIG. 12 is a diagram showing the results of model-to-model comparison when obtained using the system according to the present invention and compared with actual test scores.

FIG. 13 is a diagram showing the distribution of a speaker model used by the system according to the present invention in a two-dimensional representation of a multidimensional data space.

Figure 14 illustrates the use of an imposter cohort as used in a system according to the present invention.

15 is a block diagram illustrating a normalization process according to an aspect of the present invention.

16 is a block diagram illustrating an example of a wide area user authentication system in accordance with the present invention.

FIG. 17 is a block diagram illustrating corruption of a speech signal by various noise sources and channel characteristics in an input channel of a speaker recognition system.

18 and 19 illustrate the effect of noise and channel characteristics on test pronunciations and registration models in a speaker recognition system.

20 illustrates a channel normalization method according to an aspect of the present invention.

The present invention includes various aspects and features that can be combined in various ways to provide an improved speaker recognition (verification and / or identification) system. Certain aspects of the present invention relate to the manner in which speech samples are modeled during speaker registration and during subsequent recognition of input speech samples. Other aspects relate to the way in which the input speech models are classified to reach a decision about the speaker's identification. Another aspect relates to normalizing voice signals input to the speaker recognition system (channel normalization). Another aspect relates to the application of speaker recognition systems.

Referring now to the drawings, Figures 4-6 illustrate the basic structure used in a system incorporating various aspects of the present invention. The inputs to all of the embodiments of the invention described herein are digital signals containing speech samples that have been digitized in advance by any suitable means (not shown), and that all the filters and other modules referenced are also digital. Of course.

In FIG. 4, speech samples are input to the system through channel normalization module 200 and filter 24. Instead of or in addition to this “front-end” normalization, channel normalization may be performed later in the course of processing voice samples, which will be discussed later. The sample is divided into a series of frames at some other point before input to filter 24 or before feature extraction. In some embodiments, the noise signal 206 may be added to the filtered signal (or may be added before the filter 24), as described later. Sample data is input to a modeling (feature extraction) module 202, which includes a spectral analysis module 26 and a statistical analysis module 28 (at least in the case of speech samples processed for registration). The model (feature set) output from the modeling module 202 includes a series of coefficients that represent the smoothed frequency spectrum of the input speech sample. During speaker registration, the model is added to a registered speaker's database (not shown). During the recognition of the input speech sample, the model (feature set) is input to the classification module 110, which compares the model (feature set) with the models selected from the registered speaker's data. Based on this comparison, a decision is made at 204 to identify the speaker and verify the claimed identification of the speaker. The channel normalization of the input sample and the addition of the noise signal 206 constitutes various features of the invention, as will be described in more detail later on, which corresponds to the preferred features of all embodiments of the invention. . In some embodiments, channel normalization may be applied during classification or after spectral analysis 26 without being applied to the input speech sample prior to processing as shown in FIGS. New aspects of the modeling and classification processes according to other aspects of the present invention will be described in more detail later.

Other aspects of the invention include various forms of parallel processing in the processing of speech samples for registration and / or recognition.

Referring to FIG. 5, the basic operation of the system is the same as that of FIG. 4 except for the following. That is, the difference is that the output from the modeling module 202 is input to a plurality of parallel classification processing units 110a, 110b, ..., 110n, and the outputs from the multiple classification processing units are combined to reach the final conclusion. This will be described in more detail later. In FIG. 6, the basic operation of the system is the same as that of FIG. 4 except for the following. That is, the input sample is processed by a number of parallel modeling processing units 202a, 202b, ..., 202n (mainly providing slightly different feature extraction / modeling as described later), which processing ( The noise signal 206 is shown to be added to an input signal upstream of the filters 24a, 24b, ... 24n) via the multiple filters 24a, 24b, ..., 24n. It is possible, and also outputs from the plurality of modeling processing units are input to the classification module 110, which will be described in more detail later. Preferably, a number of parallel modeling procedures of this type apply to both registration sample data and test sample data.

Multiple parallel modeling processes may be combined with multiple parallel classification processes. For example, an input to each of the parallel classification processing units 110a-n of FIG. 5 may be used for multiple parallel modeling processes as shown in FIG. 6. May be output from.

Hereinafter, various aspects of the present invention will be described in more detail with reference to the modeling, classification, and normalization processes shown in FIGS. 4 to 6.

modelling

The spectral analysis modules 26, 26a-n can be applied to spectral analysis methods similar to those used in existing speaker recognition systems. Preferably, the spectral analysis applied by modules 26a-n takes the form of extracting, for each frame of sample data, a series of feature vectors (coefficients) representing the smoothed frequency spectrum of that frame. . Preferably, this includes LPC / ceptral (LPCC) modeling, which produces an increased feature set that models a more delicate configuration of the spectrum, the delta ceptral or m of coefficients selected based on the weighting scheme. Modifications such as persis / de-emphasis may be included. Instead, similar coefficients can be obtained by the use of a filter bank or other means, such as a Fast Fourier Transform (FFT).

A complete sample is represented by a matrix of coefficients of one row of each frame of that sample. In a preferred embodiment of the invention, these matrices will each have an order size of 1000 (frames) x 24 (coefficients). In existing systems, statistical analysis such as HMM would have been performed on this type of first single matrix representing a complete original signal.

As will be appreciated by those skilled in the art, the LP transform effectively produces a series of filter coefficients representing the smoothed frequency spectrum for each frame of test pronunciation. LP filter coefficients are related to Z-plane poles. The cepstral transform compresses the dynamic range of the smoothed spectrum and moves LP poles by moving them closer to the Z-plane origin (away from the real frequency axis at z = e ^jw ). It has the effect of deemphasis. The septral transformation uses log functions for this purpose. Of course, other similar or equivalent techniques may be used in the spectral analysis of speech samples to obtain a smoothed frequency spectrum and to de-empt the poles. This de-emphasis results in a series of coefficients that are less dynamic and more balanced when converted back to the time domain (the spectral coefficients are the time signal or impulse response of an LP filter with de-emphasized poles). Similar to). The logarithm function also converts multiplicative processes into additive processes.

The model derived from the speech sample can be regarded as a series of feature vectors based on the frequency content of the sample signal. When the feature vector based on the frequency content is extracted from the signal, the order of the vector is important. If the order is too small, some important information may not be modeled. To avoid this, the order of the feature extractor (eg the number of poles of the LP filter) can be chosen to be greater than the predicted order. However, this inherently causes problems. The poles that match the resonances in the signal provide good results, while the coefficients as another result of the feature vector model the spurious aspects of the signal. Thus, comparing this vector with other models or criteria, the calculated distance measure can naturally be influenced by the values of those coefficients modeling the pseudo elements of the signal. The distance measurement (score) that is returned is therefore inaccurate, and is likely to give a bad score for a frame that is actually a good match.

According to one aspect of the present invention, this problem is caused by the noise signal n (t) having known (known) characteristics in the speech signal s (t) before the signal is input to the modeling process (Figs. By adding 206 of FIG. 6, it can be prevented or mitigated beforehand (ie, input signal = s (t) + n (t)). This same noise signal can be used during speaker registration and subsequent use of the system. Adding the known noise signal has the effect of forcing "extra" coefficients (actually more than necessary) to model the known function and thus to give more trouble-free consistent results during model / test vector comparisons. do. This is particularly relevant for suppressing the effects of noise (channel noise and other noise) during the "silences" of speech sample data. This problem may be addressed as a result of the use of largely overlapping sample frames, which will be described later.

As mentioned above, to enable effective processing of speech samples, all speaker recognition systems use time fragments, called frames, to divide the pronunciation into a series of frames, each frame being processed again. The frames may be the same size or not the same size, or may overlap. The models generated by the speaker recognition system thus comprise a plurality of feature sets (vectors corresponding to sets of coefficients) representing a plurality of frames. When the models are compared in the existing speaker recognition system, it is necessary to align the corresponding frames of each model. Different pronunciations for a given phrase cannot have exactly the same length, even if they are spoken by the same person. This makes it difficult to align the frames correctly for comparison.

Existing systems use frames or smoothed frames as shown in FIGS. 8A (representing a time signal divided into frames) and FIG. 8B (representing a corresponding frequency spectrum and a smoothed frequency spectrum of one of the frames of FIG. 8A). It will be converted into spectrum. The system then performs additional transformations and analysis (such as Septral Transform, Vector Quantization, Hidden Markcove Modeling (HMM) and Dynamic Time Warping (DTW)) to achieve the desired results. Frame boundaries can be assigned in various ways, but are primarily measured from any starting point that is assumed to be the starting point of a useful voice signal, to compensate for this starting point, and also to natural variations in the length of similar sounds. To compensate for, techniques such as HMM and DTW may be used when comparing two or more pronunciations, such as when constructing models or comparing test pronunciations with models. Using any set coefficients used to produce the HMM / DTW compensation, the system's support after spectral analysis. Does not refer to the original time signal, so alignment precision is not limited to the size of the frame, and these techniques also allow some frames to be predicted that an array of specific frames will be there. The assumption is that it will be in a fixed region of pronunciation within the frame, thus introducing a temporal element into the system as the estimated arrangement of the current frame depends on the arrangement of the previous frame and the arrangement of subsequent frames depends on the arrangement of the current frame. In practice, this generally means that a particular frame, such as a frame that exists for 200 ms in the pronunciation, is only compared to other frames in the 200 ms region of the model or other pronunciations used to construct the model. This approach uses speech recognition methods (e.g. speech to text conversion). Site is derived, wherein to estimate a sequence of phoneme (phonetic sequence) from a series of frames is used. The Applicant has determined that this way for the following reason that inappropriate in the speech recognition.

A) As the most serious, the existing approach provides only a crude alignment of the frames. Any assignment of starting points generally does not yield an exact arrangement of the starting points of each of the two frames, so that even two frames that give a "best match", as shown in Figure 9, are significantly different from each other. That means you can have

B) Secondly, the existing scheme relies on a temporal sequence of the frames, and speaker verification is based on spectral characteristics inferred from temporally adjacent frames.

According to another aspect of the present invention, in order to overcome the problems arising from the frame arrangement between the models (as described in (A) above), and to improve the quality of the obtained model, convolution in the registration modeling process of the present invention. This involves using very large frame overlap similar to. This technique is applied during speaker registration to obtain a model (preferably based on repeated pronunciation of an enrollment phrase). By massively overlapping the frames, the resulting model effectively approaches the model of all possible arrangements with relatively little difference between adjacent frames, providing good modeling of the patterns. Preferably, the frame overlap is selected to be at least 80%, more preferably in the range of 80% to 90%, and may be up to 95% possible.

These frames are transformed into representative coefficients using the LPCC transform as described above, so that each pronunciation included in the reference model generated by the registration process is (typically, 24 coefficients × The size of the order of 1000 frames). Typically, there may be ten such matrices representing ten pronunciations. Clustering or averaging techniques, such as vector quantification (described further below), are then used to reduce the data to generate the speaker's reference model. This model does not depend on the temporal order of the frames, so it can handle the problems mentioned in (B) above.

In preferred embodiments of the present invention, the large-scale overlapping of the aforementioned frames and the vector quantification as described below are combined. This provides a significantly different operating mode than the existing HMM / DTW system. In such existing systems, all frames were considered equally valid and generally lead to a yes / no decision to derive the final "score" for thresholding by accumulating the scores derived by comparing and matching each frame. Is used. The validity of the scores thus obtained is limited by the accuracy of the frame arrangements.

According to this aspect of the invention, the reference (registration) models will represent a large number of possible frame arrangements. This does not match each frame of the test pronunciation with each frame of the reference models, but derive scores for each matching pair of frames, but compare all the frames of the test pronunciation and score for every frame of the reference model. To give a statistical distribution of the frequency of occurrence of the frame score values. "Good" frame matches yield low scores, and "bad" frame matches yield high scores (or may be reversed depending on the scoring scheme). The test pronunciation frame tested for a large number of reference models will eventually be a normal distribution as shown in FIG. 10. Most frame scores will be close to the mean and within some standard deviations. Because of the large overlap of frames in the reference models, the score distributions will include "best matches" between the reference models and the correctly arranged corresponding frames of the test pronunciation. Thus, if a test pronunciation from a particular speaker is tested against a reference model for that speaker, the distribution will contain higher incidence of very low scores. This ultimately results in consistent "true speaker" scores, since some parts of the pronunciation are easily identified as originating from the true speaker, while others are less likely to originate from the true speaker. The lower the value. Imposter frame scores will not produce low scores and will be classified as originating from the general public.

That is, according to this aspect of the invention, the reference models comprise sets of coefficients derived for a plurality of massively overlapped frames, wherein the test pronunciation compares all of the frames of the test pronunciation with all of the frames of the relevant reference models. And by analyzing the distribution of frame scores obtained from it.

Large-scale overlapping of frames applied to speech samples for registration may be applied to input pronunciations during subsequent speaker recognition, but is not necessary.

Using massive overlaps in the registration sample data is also beneficial in addressing problems resulting from noise present within periods of silence in the sample data. Such problems are particularly important for text-independent speaker recognition systems. The presence of silence may or may not cause problems for individual models or verification attempts, but will cause overall system performance to deteriorate. Therefore, the question is how to completely eliminate it or minimize its adverse effects. Using large frame overlaps in accordance with the present invention involves an essential solution. Consider the following equation that represents the average of the frame spectrum (this will be described in detail later).

From this equation we can see that the average of the static parts (ss) is ss (ω), and each frame has the spectrum ss _n (ω) x sd _n (ω) For the spectrum of the frames ss ₁ (ω) xsd ₁ (ω) + (ss ₂ (ω) xsd ₂ (ω)) = ss (ω) x (sd ₁ (ω) + sd ₂ (ω)) Consider, here, the steady part is multiplied by the new spectrum (sd ₁ (ω) + sd ₂ (ω)), but the new spectrum is supposed to be reduced by averaging and also essentially Because they are dynamic or variable, they must behave in exactly the same way as randomly extracted frames, which means that the frames can be added randomly together with minimal impact on performance. Can have the case of valid speech frames added to silence frames that are valid speech frames Because, such a determination is not entirely correct. In fact, this is leading to the consequential performance improvements in accordance with the no longer contains the undesirable silence in the modeling.

If a typical signal with some minor silence problems has randomly added timeframes, the silences will be removed but the signal will appear to have suffered major corruption. However, the present invention using massively overlapped frames still functions. Interestingly, this means that channel echoes have no effect and can be ignored. This also clearly indicates that the preferred modes of operation of the present invention will extract more static portions of the spectrum than conventional verifiers (described further below). Adding frames in this manner will have substantially the same effect as adding colored noise, thus preventing unwanted modeling as described above.

According to another aspect of the present invention, clustering or averaging techniques such as vector quantification applied by modules 28, 28a-n are used in a manner different from the statistical analysis techniques used in existing speaker recognition systems.

Preferably, the system of the present invention utilizes vector quantification (VQ) techniques in processing the registration sample data output from the spectrum analysis modules 26, 26a-n. This simplifies the technique compared to statistical analysis techniques such as HMM used in many conventional systems, resulting in transparent modeling that provides models in a form that allows model-to-model comparison in subsequent classification steps. In addition, vector quantification (VQ) as used in the present invention does not use transient information such that the system is resistant to imposters.

The vector quantification process effectively compresses the LPCC output data by identifying clusters of data points, determining average values for each cluster, and abandoning data that certainly does not belong to any cluster. This in turn results in a set of second matrices representing LPCC data of the first set of matrices but having a reduced coefficient (typically 64 x 24 compared to 100 x 24, for example).

Hereinafter, the effects of using LPCC spectrum analysis and clustering / averaging in the present invention will be described.

The basic model assumes that the spectral magnitude is useful and that its phase can be ignored. This is known to apply to human hearing, and if not applied to the verfier, the system will show undesirable phase related problems such as sensitivity to the microphone's distance from the speaker. In addition, the spectral information of a speech sample can be regarded as being composed of two parts: a static part (ss (ω)) and a dynamic part (sd (ω)), and the processes are assumed to be multiplicative processes. Do it. Also assume that the dynamic part is considerably larger than the static part:

s (ω) = ss (ω) x sd (ω)                 

The static part is fixed by definition, making it more useful as a biometric as it relates to the static properties of the vocal tract. This results in some fixed physical properties as opposed to sd (ω), which relates to the dynamics of speech.

Complete extraction of ss (ω) will give the biometric properties of the physical biometric properties, i.e., the physical biometric properties that cannot be arbitrarily changed and do not deteriorate over time. Instead, the exclusive use of sd (ω) gives the biometric properties of the behavioral biometrics, i.e., the behavioral biometrics that can be changed arbitrarily and deteriorate over time. Will be done. Mixing these two should show intermediate features, but since sd (ω) is much larger than ss (ω), the combination is more likely to show the features of sd (ω) (i.e., become behavioral). high.

As with all frequency representations of the signal, the assumption is that the time signal is between -∞ and + ∞, which is not physically possible. In practice, all spectral estimates of the signal are made using a window, which is present for a finite period of time. This window can be rectangular or shaped by a function (such as a Hamming window).

Using a rectangular window it is possible to simply take a section of the signal in the region of interest and assume that it is zero elsewhere. This technique is common in speech processing in which sections of a signal are called frames. 1 shows a time signal with indicated frames.

Frames can be formalized using alternate windows. Interestingly, the main effect of windowing is a kind of spectral averaging in which the characteristic of a particular frequency spreads to its surrounding frequency. This effect is caused by the main lobe, in addition to which side lobes generate periodic spectral oscillations in the spectrum. The system of the present invention then extracts all-pole linear prediction coefficients, which has the intended effect of spectral smoothing, and the extra smoothing caused by windowing. extra smoothing is not a major problem. However, if the window size is changed inadvertently, the effect of the periodic side lobe may be difficult. But this can be avoided by good housekeeping.

If it is possible to divide the time signal into frames, the spectral characteristics for the frames 1 to N may be expressed as follows.

s ₁ (ω) = ss ₁ (ω) x sd ₁ (ω); s ₂ (ω) = ss ₂ (ω) x sd ₂ (ω); ......................

s _n (ω) = ss _n (ω) x sd _n (ω); .......

s _N (ω) = ss _N (ω) x sd _N (ω)

But by the definition:

ss (ω) = ss ₁ (ω) = ss ₁ (ω) = ss ₂ (ω) = ss ₃ (ω) = .... = ss _N (ω)

The first result seems to be to extract ss (ω) using the following averaging process.

가 스펙트럼을 평활화함으로써 추출될 수 있게 될 것이다. 이것은 N이 매우 크게 되면(N -> ∞) 그 경우가 될 가능성이 가장 클 것이다. 시간 도메인 - 주파수 도메인 - 시간 도메인 변환들의 선형 특성(linear nature)이 주어진 경우에는, 그와 유사한 분석이 시간 도메인에서 기술될 수 있었을 것이다.If the frames have independent spectral characteristics (each by random processing), U (ω) will tend to be white noise. That is, in this case U (ω) will have a flat spectrum

Can be extracted by smoothing the spectrum. This is most likely the case if N becomes very large (N-> ∞). Given the linear nature of the time domain-frequency domain-time domain transforms, a similar analysis could be described in the time domain.

Under real world conditions, one cannot assume that N becomes large in the sense that the frames have independent spectral characteristics. It is important to remember that N needs to be large under two conditions:

1. During model creation

2. During the verification event

Failure to comply with the conditions during such processing will potentially cause a system failure (error), but if conditions 1 are not met, they will remain a potential source of error until they are updated, which is more serious, while condition 2 If it is, it will be a single instance event.

If U (ω) cannot be guaranteed to converge to white noise, what can be done to cope with that situation?

First, consider the following.

1. U (ω) will be a variable quantity.

2. Smoothed over the frequency spectrum would ideally be flat, ie smoothed version Usm (ω) = 1.

3. U (ω) is the truncated sum of speech frames in which the number of speech frames ideally tends to infinity.

Consider the following equation.

If we return to the following frame based equivalent, the summation part that tends to be a flat spectrum is not an ideal performance measure.

If we take logarithms of that frame like this:

In the above equation, you can see that the relationship between the static and dynamic parts is now an additive relationship. Since the relationship between the time domain and the frequency domain is linear, the conversion from frequency to time is as follows.

lss (ω) + lsd (ω)-> cs (τ) + cd (τ) = c (τ)

In signal processing, c (?) Is known as a spectral transform of s (t) as described above.

In general, septal analysis consists of time_domain-> frequency_domain-> log (spectrum)-> time_domain (time_domain-> frequency_domain-> log (spectrum)-> time_domain). Septral transformation is used in many forms in speech analysis.

As mentioned above, in the use of the present invention, the septal coefficients for the frames are generated to extract the static portion as follows.

Ideally, the length of the voice signal will be long enough so that the dynamic part is completely random and the mean will tend to be zero. This will leave the static part cs (t) as the biometric measure of the present invention. However, there are various problems to be solved as follows.

1. How to deal with imperfect nature of sum-to-zero

2. Channel change

3. Endpoint Pointing

4. additive noise

Referring to the incomplete characteristic from the sum to zero, the characteristics of the septural coefficients decay over time, taking the form of an impulse response to stable systems. This means that the dynamic range of each coefficient is different and they are generally in descending order.

 The difference between the mean coefficients of the test sample and the frame coefficient values of the true speaker model and the frame coefficient values of the imposter model is not significant, and the simple sum for all the pronunciation frames for generating a distance score ( It can be seen that simple summation would be difficult to threshold with conventional methods.

If you think about the two difficult problems of this method together without separating them, you will find the answer. To reemphasize the two difficult points:

The pronunciations will never be long enough for the average of the zero-converging dynamic part.

2. The differences between true speakers and imposters are small and difficult to hold.

Consider two speakers with models based on the following equation.

Thus, the models become m1 (τ) and m2 (τ), where m1 (τ) is

At this time, e1 (τ) is an error.

In vector form, the models are as follows.

Expressed in the same form, the test pronunciation from Speaker 1 is as follows.

Using a simple distance measure, the true speaker distance is

In addition, the imposter distance becomes as follows.                 

Assuming that the convergence of the dynamic parts of the models is good (i.e., the error vectors are small compared to the static vector), then d1 <d2. This simply indicates that the models construct and display the registered speaker (a condition that can be easily checked during registration using the data available at that point in time). Interestingly, if e1 and e2 are small compared to the test signal error Te1, the distances are independent of e1 and e2. The condition under which the test error becomes large when compared to the model error is during text-independent test conditions. This shows that if the dynamic components of the registration speech samples are minimized in the registration models, such models can provide a good basis for text-independent speaker recognition.

The above mentioned errors e1 and e2 are average model configuration errors; The actual errors are in frames and will be distributed around the mean. This distribution can be modeled in several ways, the simplest being using standard clustering techniques such as k-means to model the distribution. The use of k-minz clustering is known in other forms as vector quantification (VQ), and is a major part of the Self Organizing Map (SOM), also known as Kohonen Artificial Neural Netowork. to be.

Directly described above is a system in which test pronunciation is applied to two models and the closest chosen is the variation of identification. In the case described above, if either Speaker 1 or Speaker 2 is registered as a registered speaker and they are tested, they will always be tested as true, so False Rejection Rate (FRR) = 0 do. If an unknown speaker requires speaker 1 or speaker 2, the speaker is classified as one or the other, with a 1/2 chance of success, so False Acceptance Rate (FAR) = 50% Becomes Once the same number of true speaker tests and random imposter tests are performed, the overall error rate can be calculated as (FRR + RAR) / 2 = (0 + 0.5) / 2 = 25%.

It is clear that the number of models tested by the test pronunciation (the cohort) affects the FAR, and that number will decrease as the cohort increases. For cohort sizes where the recognition accuracy increases under these conditions, it has asymptotic at 100% because FRR = 0, but the accuracy is as follows, so in general terms it is asymptotic at 100-FRR .

It is to be noted that at this point the FRR and FAR are greatly decoupled. That is, the FRR is fixed by the quality of the generated model, and the FAR is fixed by the cohort size. Also, to halve the error rate, the cohort size is doubled, for example, the cohort is 50 for 99% accuracy, the cohort is 100 for 99.5% accuracy, and the cohort is 200 for 99.75% accuracy. As the cohort increases, the computational load increases, and in fact, every half the error rate doubles the computational load. If the cohort increases by a very large number, FRR and FAR will break down and FRR will begin to increase.

Instead of constantly increasing the cohort size, at least another method is needed to reduce the FAR. The method is, in accordance with an aspect of the present invention, using parallel processing (discussed elsewhere in the detailed description of the present invention), which exhibits slightly different imposter characteristics, and thus an identifier strategy. In part, it is statistically independent. The idea is to take a core identifier with a FAR set to a cohort size, with a FRR of zero or nearly zero. And this front end processing of the core identifier is slightly modified to reorder the distances from the true speaker model of the cohort member models. This is done while maintaining FRR ˜ 0 and can be achieved by changing the transformed coefficients, such as changing the spectral shaping filters 24a-24n (see FIG. 7) or using delta-ceps or the like.

When a registered speaker uses the system, the test signal applies to all processes in parallel, but each process has a FRR to 0 and the speaker will pass through it. If an unknown imposter uses the system, the imposter will pass each process with a probability of 1 / cohort size. However, the present invention introduces conditional probabilities by parallel processing. In other words, conditional probabilities were introduced in such a way that the imposter passed through modified process 2 and then passed through process 1, which is likely to be determined. Although the probability that the imposter will pass through all processes is not statistically independent ("stastically_independent_result = process_prob ^{no_pf_process"} ), the probability is reduced by adding the process. For the FAR value of a given process, it can be seen that the accuracy of the overall system increases with the total number of processes.

When multiple parallelism is used in this way, a technique for matching test samples and claimed identities with each other may require a successful match in each process or some success. A matching rate may be required.

When constructing the registration model according to the present invention, a combination of massive sample frame overlaps and vector quantification (or equivalent) provides very special advantages. Massive overlapping can be applied at the time of the utterance test, but also when constructing models. This technique typically uses 80 to 90 percent of massive frame overlap to produce as many alignments as possible. The frames generated by the alignment are transformed into representative coefficients using the LPCC transform to produce a coefficient matrix (matrix) representing the entire alignment. This can solve the conventional frame alignment problems. In general, a matrix has a size of "no_of _frames x LPCC_order" (no_of _frames by LPCC_order), for example 1000 x 24. This is repeated for all the pronunciations that make up the model, typically producing 10 1000 x 24 matrices with 10 iterations. Vector quantification is then used to reduce the data for creating a model for speaker. This has the effect of averaging the frames in order to reduce the significance of the dynamic elements of the sampled speech data. The model of the final result is not essentially a model that changes over time because it does not recognize the position of the frame in the test pronunciation. This leads to a temporal dependency problem.

The combination of VQ and massive frame overlapping creates a different mode of operation from conventional systems based on HMM / DTW. In HMM / DTW all frames are equally valid and are used to form a final score for performing a "yes / no" thresholding. The invention tests by comparing every row (frame) of test sample data with every row of registration model data for the requested speaker and associated imposter cohort. The best match for each row of test sample data can be obtained in one row of the registration model and generates a test score for each relevant registration model. The test sample is matched with a registration model that produces the highest score. At this time, if the matching information is determined to be the required identification information, the test speaker is accepted. If the match information is determined to be an imposter, the test speaker is rejected.

The system uses LPCC and VQ modeling (or similar / same spectral analysis and clustering techniques) with massive overlapping of the sample frames to generate a reference model for each registrant. Are stored in. When using the system, the input test pronunciation is applied to pseudospectral analysis to obtain an input test model that can be examined against a registered speaker data-set. This approach has the advantage that it can be applied to achieve a very low false rejection rate (FRR), such as zero. The importance of this is explained in more detail below.

Parallel modeling

As mentioned above, the performance of the speaker recognition system according to the present invention can be improved by using multiple parallel processing for model generation.

Referring to FIG. 7, a preferred embodiment of a speaker recognition system to which parallel modeling is applied according to an embodiment of the present invention includes an input channel 100 and a channel normalization process 200 for inputting a signal representing a voice sample into the system. ), A plurality of parallel signal processing channels 102a, 102b ... 102n, a classification module 110 and an output channel 112. The system further includes a registered speaker data set 114, ie, a voice module database obtained from a registered speaker using the system. Speech sample data is processed in parallel by each of the processing channels 102a-n, and the output of each processing channel is input to a classification module 110, which communicates with a database 114 of registered speaker data. . Decision information regarding the identity of the source of the test pronunciation is output through the output channel 112.

Each processing channel 102a-n is sequentially subjected to a spectral shaping filter 24a-n, (optional) an added noise input 206a-n, a spectral analysis module. 26a-n, and statistical analysis module 28a-n. The output from each statistical analysis module 28a-n is input to the classification module 110.

The spectral shaping filter 24a-n consists of a bank of filters that separate the pronunciation signal into a plurality of overlapping frequency bands, each frequency band being a continuous module 26a-n or 28a-n. In parallel. The number of processing channels and number of frequency bands depends on how many more channels are included to provide more detailed information in subsequent analysis of the input data. It is preferred to employ at least two channels, even more preferred to employ at least four channels. The filters 24a-n constitute a low pass, band pass, and high pass filter bank. The bandwidth of the base filter 24a is chosen such that the false rejection rate (FRR) according to subsequent analysis of the output of the first channel 102a is "zero" or as close as possible to "zero." The continuous filters 24b-n have a gradually increasing bandwidth to pass progressively more signals from the input channel 100. Thus, the false rejection rate (FRR) for the output of each channel 102a-n remains close to " 0 " while the other channel outputs have slightly different false accept (FA) characteristics. Analysis of the combined output of channels 102a-n indicates a near-zero FRR and a reduced overall false acceptance rate (required identification is only accepted if the output of all channels has been accepted). . The importance of this is explained in detail below.

The use of multiple frequency bands increases the size of the feature vectors of interest in continuous statistical analysis, thus improving the conventional single channel spectral analysis.

Various kinds of parallelism may be used in the modeling process to provide multiple feature sets for modeling different facets of the input speech sample and / or provide other similar models. Other types of filter banks may be used in addition to or in place of the low pass filter. Various types and variations of spectral and / or statistical analysis techniques can be used in parallel processing channels. Parallel statistical analysis adds different weights to sets of feature coefficients to obtain a set of models that are in a slightly deviated state.

The structure shown in FIG. 7 can be used to obtain a registration model for storage in the database 114 and also to process test speech samples for testing against the registration model. Each registration model may include a data set for each of the plurality of registration pronunciations. For each registration pronunciation, there may be a matrix of data representing the output of each parallel modeling process. Each of these matrices represents a clustered / averaged spectral feature vector. Test sample data can be controlled by the same parallel spectrum analysis process without clustering / averaging process. Thus, the test model data consists of a matrix representing spectral analysis data for each of the parallel modeling processes. When testing a test model against a registration model, a test matrix representing a particular modeling process is tested by comparing the registration matrix generated by the same modeling process.

Classification

The characteristic of the reference model obtained by the modeling technique described above is that the reference models themselves make a model-to-model comparison directly. Thus, the system may use an identification strategy, where each registration model is associated with an imposter cohort. That is, in the case of the reference model of each registered speaker (subject, subject), there is an imposter cohort composed of a predetermined number of reference models of other registered speakers. At this time, the other registered speakers are clearly distinguished from the subject registered speakers and have a known predictable relationship with the subject speaker's reference model. This predictable relationship allows the performance of the system to be improved. FIG. 11A shows a result obtained by a conventional speaker recognition system in a manner similar to that of FIG. 3, comparing scores of input pronunciations checked against reference data for eight speakers. FIG. The speaker 1 is the actual "true" speaker. However, the scores of some other speakers are so close to each other that the system significantly reduces the degree of confidence that the system can identify with the correct speaker. 11B shows equivalent results obtained using the system according to the present invention. The result of the speaker 1 is very clearly distinguished from the result of the remaining speakers 2 to 8.

The speaker modeling method applied to the embodiment of the present invention is inherently simpler than using conventional techniques such as HMM and other possible methods such as Gaussian mixture models. However, the Applicant is convinced that the use of conventional tight statistical methods results in poor performance when applied to the "real world" because they are inherently flawed in the conventional methods. Surprisingly, the present invention has the effect of providing a much simpler statistical method in a relatively real case. As described above, there is a problem that the time-dependent characteristics of the HMM are easy to imitate, and the present invention seeks to solve this problem. Furthermore, the models of the present invention can ideally analyze the structure of the registered speaker data set by performing model-to-model tests.

There are two advantages to performing a model-to-model comparison using the speaker model of the present invention. First, the speaker model of the present invention is the most relevant imposters present in registered speaker data-sets (i.e., the data sets are very close to each other and uniformly distributed around a particular model). It provides the ability to identify relevant impostors and provides an effective and predictable speaker normalization mechanism. VQ modeling involves the selection of the size of the model, ie the number of coefficients (center "centres"). Once the model size is chosen, the center positions can be moved around until the best fit for all registration data vectors. This means that the center is effectively assigned to a cluster of registration vectors, so each center in the model represents a cluster of information that is important for the speaker's identity.

Model-to-model testing is registered in some way with a registered speaker or with a claimed database of broad sense, or a database of areas (i.e., system data spaces) that are restricted to the requested identification. Be able to predict if it is required. FIG. 12 shows a result of testing a reference model for a plurality of speakers 2 to 8 with respect to the reference model for the speaker 1. The ellipsoid represents the model-to-model result, while the stars notation represents the actual score of speaker utterances tested for model 1. Model-to-model tests can be used to predict the actual performance of a particular speaker for a particular reference model. Model-to-model results tend to be at the lowest level in the actual score distribution, indicating how well a particular imposter works with Model 1. As such, the basic approach to using model-to-model testing to predict actual performance is well known. As further described below, in accordance with one embodiment of the present invention, the approach can be extended to protect a particular model from an imposter using groupings each selected and statistically variable.

The second advantage obtained from model-to-model testing is the ability to predict the performance of test utterances for a few, or if necessary, all registered speaker models. This advantage allows a virtually unlimited number of test patterns to be used to verify the identification, which cannot be achieved in conventional systems.

Model to model test results can also be used to assemble the particular imposter cohort used with each reference model. This achieves accurate score normalization and also ensures that each model is effectively protected from imposters using statistically variable groupings selected for each registered speaker. This concept is illustrated in FIG. 13. Since each reference model can be regarded as a point existing in the multidimensional data space, the distance between the models can be calculated. 13 is shown two-dimensionally for clarity of this concept. In FIG. 13, each asterisk represents a model and a two-dimensional distance represents a distance between models.

Because the speaker model's distribution is not uniform, a world-model based on normalization techniques will not work properly in the same way on all speaker models. A few speaker models are relatively very close or very similar to each other, meaning that the imposter has the potential to successfully mimic the voices of registered speakers. This problem can be solved by creating a specific cohort of imposters around the subject model for each speaker model. This simplifies the normalization process and creates a guard for the imposters. This is illustrated in FIG. 14, in a manner similar to FIG. 13, where the object, i.e. the subject model, is represented by a circle, the members of the imposter cohort are marked with an asterisk, and the score of the imposter required by the subject is "x". Is displayed. The imposter scores are so close to the subject model that they can cause cognitive problems. However, since the speaker data-set allows us to predict how similar the real subject speaker is to the imposter cohort model, the above information tests the imposter against the cohort member model and the real object model, so that the imposter "x from the real subject can be tested. May be used for the purpose of distinguishing between " That is, the utterance "x" of the imposter is similar to some of the cohort members beyond what is expected of the actual subject, and may also be distinguished from others more than expected. This implies an imposter event and is represented by an imposter pronunciation that is rejected as a match with the actual subject.

This provides the basis for a two-step recognition process, in which the imposter determined to be not the claimed speaker, if necessary, with very similar utterances as determined by the requested speaker. Rejection is preferentially using the more detailed process applied.

In any application of the speaker identification system, it is very important to minimize the possibility of false rejections, i.e., if the identification required by the user is falsely rejected as being fake. According to one embodiment of the present invention an "identifier strategy" provides very low false rejection, while providing predictable system performance and in connection with the use of thresholds in accepting or rejecting the required identification. Minimize the problems that occur.

According to the method, the database of registered speakers (i.e. speaker space) is divided into several, so that each speaker registered in the system is assigned to a cohort of fixed N registered speakers as described above. A speaker classification module of the system (e.g., module 110 of the system of FIG. 4) is operated so that an input test utterance is compared with all members of the cohort associated with the identification information requested by the speaker, and the test pronunciation is best matched. It is classified as that of its member of the cohort. That is, test pronunciation is always matched with one member included in the cohort, and may be matched with other members of the cohort. If the cohort member whose pronunciation is matched corresponds to the requested identification information, the requested identification information is regarded as "true". If the pronunciation matches another member of the cohort, the requested identification is considered false and is rejected.

The modeling and classification process is tunable so the false rejection rate is effectively " 0 " (FR = 0%) (as described above). That is, the probability that the speaker is incorrectly identified as a cohort member rather than the required identification information is substantially "zero." Since this is facilitated by a model-to-model comparison method, the matching decision is based not only on the test pronunciation that matches one most similar model, but also on the relationship with other members of the cohort. If the cohort has a fixed size "N", the maximum possible false acceptance rate (FA) is "FA = 100 / N%" and the total average error rate is "(FA + FR) / 2 = 50 / N%". do. If the cohort size "N" is 20, the error rate is 2.5% (ie 97.5% accuracy). If the cohort size is fixed, the system can be scaled to populations of any size while maintaining a fixed predictable error rate. That is, the accuracy of the system is determined based on the size of the cohort and is not affected by the size of the general population, so that the system can be applied to a very wide range of population. Increasing the cohort size within a range where the false rejection rate does not increase significantly can improve accuracy.

This strategy does not matter whether the threshold is used to determine the result, but these thresholds can still be used to reduce false acceptance rates. That is, if the test pronunciation matches the required identification using the method described above, the thresholds can be applied to determine whether the match of both is similar enough to be finally accepted.

As mentioned above, a predetermined algorithm is used when selecting an imposter cohort associated with a particular registration model. Therefore, the members of the imposter cohort have a specific relationship to the registration model they want to know. In general, the degree of optimization can be provided in the classification process. However, the randomly selected imposter cohort is determined to work as well as most practical purposes. Here, the most important point is that the cohort size must be predetermined to provide predictable performance values. The imposter cohort for a particular registration model can be selected at registration or when testing test pronunciation.

Parallel Classification

The performance of the speaker recognition system according to the present invention can be improved by using multiple parallel classification processes. In general, this process can be statistically independent or partially independent. This approach can provide multiple classification results, which can be combined with each other to yield the final result as shown in FIG. 5.

For example, using the identification method as described above, the same test pronunciation can be checked for different number of cohorts, different registration phrases, or a combination thereof. If multiple cohorts are used, each cohort will apply a false rejection rate of "0" (FA = 0%) and a false acceptance rate of "FA = 100 / N%" as before. The total false acceptance rate for n cohorts of equal magnitude is "FA = 100 * M / N ⁿ %", and the average error rate is "50 * M / NN ⁿ %", where "M" is a coefficient greater than 1 As the effect of the processes that are not completely statistically independent. That is, if two cohorts are used and each cohort size is 20, the average error rate will be "0.125 * M%" compared to 2.5% when using one cohort as described above. Thresholds may also be applied to improve accuracy as described above.

Other types of partially statistically independent processes may be used in the modeling process, the classification process, or both, as described above. In addition to the above-described embodiments, one pronunciation may be divided into a plurality of parts and processed separately.

Normalization

Another problem that arises in conventional speaker recognition systems is that system performance is affected by differences in speech sampling systems used for initial registration and subsequent recognition. This difference arises from different transducers (microphones), sound cards and the like. According to another aspect of the present invention, this problem can be solved or mitigated by normalizing voice sampling based on the normalization characteristics obtained / stored in each sampling system (or possibly each type of sampling system). The sampling system is used to input speech samples into the recognition system. Alternatively (preferably), the normalization characteristic may determine an on-the-fly "on the fly" state when a voice sample is input to the system. Since the normalization characteristic can be applied to all input speech samples, the reference model and test scores are independent of the characteristics of the particular sampling system. Alternatively or additionally, according to another aspect of the present invention, the normalization process may be applied when checking a test sample against registered sample data.

The normalization characteristic can be effectively derived from the transfer function of the sampling system. For example, the normalization characteristic can be efficiently derived by inputting a known reference signal to the sampling system and processing the reference signal sampled through the speech recognition system. The resulting signal generated from the recognition system can be stored and then used to normalize speech samples that are continuously input through the same sampling system or the same kind of sampling system.

Alternatively, as shown in FIG. 15, the speech signal S (f) modified by the transfer function C (f) of the input channel 300 is estimated to predict the transfer function C (f) of the channel 300. ) Input the modified voice signal " S (f) * C (f) " into the module 302 and the normalization module 304, and convert the inverse value of the estimated transfer function 1 / C (f) into the normalization module. By input, it can be normalized to an "on-the-fly" state, so that the output of the normalization module becomes very similar to the input signal S (f). Prediction module 302 generates a digital filter according to the spectral characteristics of channel 300, and the inverse value of this filter is used to normalize the signal. For example, by determining all-pole filters that represent the spectral quality of a sample frame, an inverse filter can be calculated. The filter coefficients are smoothed across the frame so that signals outside the spectrum of the channel "C (f)" can be removed as much as possible. This estimation of the channel spectrum is used to produce an inverse filter " 1 / C (f) ". This basic approach can be improved so that the pole positions of the filters obtained for the frame are smoothed, along with the desired cancellation of the poles. It is known that the poles are not related to channel characteristics.

According to the nature of the transfer function / normalization characteristic, the normalization process may be applied to speech samples before being processed by the speaker recognition system, or may be applied to spectral data, or a model generated by the system. .

The preferred channel normalization method according to an embodiment of the present invention is applied to test model data and related registration models when examining test samples versus registration models.

The overall effect of channel characteristics on the speech signal can be expressed as follows.

는 화자 특성의 예측값(estimate)을 나타내고, cc(ω)는 채널 특성 혹은 적절히 변화된 채널 특성을 나타내고, 음성 신호는 전술한 바와 같이 고정(static) 부분 및 동적(dynamic) 부분으로 구성된 것으로 간주되어 처리된다. 이상적으로는(ideally), 원하지 않는 채널 특성은 예측되고 제거될 수 있다. 실제로, 이러한 채널 특성 제거는 시간영역, 주파수영역, 이들의 조합으로 이루어질 수 있다. 이들 양자 모두는 동일한 효과를 이루는데, 즉 역필터 혹은 스펙트럼 분할의 일부 형태를 이용하여 채널특성 "cc(ω)"을 예측하고 제거하는 효과를 얻는다.

가 원치 않는 채널의 스펙트럼의 예측값이면, 이는 다음과 같은 수식으로 표현될 수 있다. here,

Denotes a predictor of speaker characteristics, cc (ω) denotes a channel characteristic or an appropriately changed channel characteristic, and the speech signal is considered to be composed of a static part and a dynamic part as described above, and is processed. do. Ideally, unwanted channel characteristics can be predicted and eliminated. In practice, such channel characteristic removal can be accomplished in time domain, frequency domain, or a combination thereof. Both of these achieve the same effect, ie the effect of predicting and removing the channel characteristic "cc (ω)" using some form of inverse filter or spectral splitting.

If is a predicted value of the spectrum of the unwanted channel, it can be expressed by the following equation.

"로 양호하다면, 음성 신호의 예측치는 원치않는 스펙트럼 형상이 소거되어 양호하다는 것을 의미한다. 이것은 보통 FFT에 기반한 알고리즘을 사용하여 구현될 수 있다. If the estimate of channel characteristics is "

If good, then the prediction of the speech signal means that the unwanted spectral shape is canceled and is good. This can usually be implemented using an algorithm based on FFT.

Another embodiment is to model the channel characteristics as a filter (model) to be appropriate for the all-pole (all-pole) shape, which is expressed as follows.

The above equation is the most basic form of ARMA and can be extracted directly from the time signal using possible linear prediction.

Pseudo-normalization is feasible based on septal expressions.

In the septral domain, the voice signal may be represented as follows.

In addition, a speech signal modified by unwanted channel characteristics may be represented as follows.

In this case, it can be seen that it involves an additive process rather than multiplication. However, it should be noted that both "cs" and "cc" are fixed, and only one "cc" needs to be removed without removing "cs".

It is important to consider the different conditions (registration model, database, induction cohort, test speaker, etc.) and the context in which the signal "cc" is to be removed.

16 illustrates various sources of speech sample attenuation in a speaker recognition system. The input speech signal s (t) is changed by the ambient background noise b (t), recording device bandwidth r (t), electrical noise and channel crosstalk t (t) and transmission channel bandwidth c (t), thus recognizing the system. The signal input to becomes the change signal v (t). The system is easy to analyze in the frequency domain and the signal from the verifier is:

 v (ω) = ((s (ω) + b (ω)) r (ω) + t (ω)) c (ω) (Equation A).

In the verifier, two conditions can be defined: one speaking and one not speaking. These two related equations are: v (ω) = ((s (ω) + b (ω)) r (ω) + t (ω)) c (ω) and v (ω) = ((0 + b (ω) r (ω) + t (ω)) c (ω).

First, as applied to the system according to the invention, consider the simplified problem, b (t) = t (t) = 0, v (ω) = s (ω) r (ω) c (ω) = s (ω) h (ω) (where h () is a combined channel spectral characteristic), h (ω) = r (ω) c (ω) and v (ω) = s (ω) h (ω) Assume that ss (ω) sd (ω) h (ω).

The cohort model is selected from the speaker database recorded using the same channel (b), and the true speaker model is recorded using different channel (a). The test speaker may be a true speaker or an impostor and may be recorded using the third channel (c). 17 shows this diagrammatically. FIG. 18 shows the same represented in another form using the Septral coefficients. As represented in FIGS. 17 and 18, the signal component value is an average corresponding to the sum of the sample frame data.

v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ) (Equation B) and v _m (τ) = cs _m (τ) + cd _m ( Consider the cohort model established from τ) + h _b (τ) (Equation C).

The problem with the verifier is that there are two different channels used in the identifier. The claimed distinction represented by the cohort channel (b), assuming that the difference between them is hd (τ) = h _a (τ)-h _b (τ) or h _a (τ) = h _b (τ) + hd (τ) The information model consists of v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ) = cs ₁ (τ) + cd ₁ (τ) + h _b (τ) + hd (τ) and v ₁ (τ) = (cs ₁ (τ) + hd (τ)) + cd ₁ (τ) + h _b (τ).

It is found that the mean of the static part of the claimed identification model is shifted by the difference between the channels and can cause errors if the true speaker is tested using channel b or if the situation is not corrected. Can be. In addition, the use of channel a may cause problems with false acceptance.

One way to solve this problem is to remove the mean from the alleged ID model, but a simple mean removal _first yields v ₁ (τ) = cd ₁ (τ) if the static part of the speaker model is also removed. Will be. However, when looking at v (ω) = ((s (ω) + b (ω)) r (ω) + t (ω)) c (ω) (Equation A) (system model with additional noise) Considering the case where the speaker pauses, s (ω) = 0, where v (ω) = (b (ω)) r (ω) + t (ω)) c (ω) and v (ω) ) = n (ω) c (ω). Where n (ω) is a noise signal.

In the Septral form, v (τ) = n (τ) + c (τ) = sn (τ) + dn (τ) + c (τ), where "sn" is static of noise as described above. Part, and "dn" represents the result of the sum of the dynamic parts.

The model mean constructed from this is sn (τ) + c (τ), where sn is any steady state noise, such as an interference tone, and c is a channel.

Considering Equation A (required identification model setting condition) again, v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ), which is a case where there is no noise. When steady state noise is added, v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ) + sn (τ).

By constructing the speaker pause model for this case, we can obtain sn (τ) + h _a (τ), and if we remove the mean using this, v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ )to be.

This provides a model that is not polarized by the channel. A similar process is applied to each model, with the channel bias removed by the silence model. The test speaker is treated as the silence model is used to remove channel effects.

As mentioned above, channel characteristic removal (reduction) using the silence model requires full detection of the appropriate channel noise and silence portion of the pronunciation. They cannot be guaranteed, so they need to be relaxed (eg, if the silence contains some voice, it includes some of the speaker's static voice of the claimed identification and unintentionally removes it). Beneficially, they can be treated as one simple variant of the process, with all cohort models represented by the same silence model.

That is, in the cohort (including the claimed identification model), we re-add the silence average of the claimed identification model to all models. This represents all models with the same mean sn (τ) + h _a (τ). This normalization also applies to test models, representing all models and test pronunciations with the same reference point. In fact, the reference channel and noise condition are chosen to represent all others with the same reference point.

This is illustrated graphically in FIG. 19 and shows the cohort models 1 to m and the claimed identification information model, together with the cohort models 1 to m input to the classifier 110. Using the "silence model" or "normalization model" 400 derived from the claimed identification registration data to normalize each of these prior to entering the classifier, the actual input of the classifier is used for normalized test pronunciation, normalized required identification. Model and normalized cohort model. Ideally, the normalization model 400 is based on the data of the silence period in the identification registration sample claimed as described above. However, this can be derived from the complete required identification registration sample. Indeed, the terms, normalization model, comprise a single row of Septral coefficients, each coefficient being the average value of one column of the Septral coefficients (or selected members of one column) from the claimed identification model. These mean values are used to replace the mean value of each of the input data sets. That is, as an example, when test pronunciation is taken, the average value of each column of the test pronunciation septral coefficients is subtracted from each individual member of the column, and the corresponding average value from the normalization model is added to each individual member of the column. Similar behavior applies to each of the claimed identification model and cohort model.

The normalization model can be derived from any of the claimed identification model, test pronunciation or cohort model. The model is preferably derived from the claimed identification model or test pronunciation. In particular, it is most desirable that the model be derived from the claimed identification model. The normalization model is derived from the "raw" registration sample septr coefficients or the final model after vector quantification. That is, it can be derived at registration and stored with the registration model. Alternatively, it can be calculated on demand as part of the verification process. In general, the normalization model is preferably calculated for each registered speaker at registration and stored as part of the registered speaker database.

These normalization techniques can be used with various types of speaker recognition systems, but can be advantageously combined with the speaker recognition system of the present invention.

The speaker recognition system according to the present invention provides improved real performance for several reasons. First, the modeling technique used significantly improves the separation between the true speaker and the imposter. This improved modeling makes the system less susceptible to real problems such as changes in the sound system (voice sampling system) and speaker characteristic changes (eg, due to winding). Secondly, modeling techniques are not transient in nature and are less sensitive to transient speech changes, thus providing long-term persistence of the speaker model. Third, the model can be used in variable bandwidth conditions by using filter preprocessing. For example, models built using high fidelity sampling systems such as multimedia PCs operate on inputs received through reduced bandwidth input channels such as telephone systems.

The preferred method according to the invention is inherently suitable for use in text independent speaker recognition systems as well as text dependent systems.

system

Accordingly, the present invention provides a basis for a flexible, reliable, simple speech recognition system that operates on a local or wide area basis and utilizes various communication / input channels. FIG. 16 is a wide area operating over a local network and the Internet for authenticating users of a database system server 400 connected to a local network 402, such as an Ethernet network, and connected to the Internet 406 via a router 404. An example of a system is shown. The speaker authentication system server 408 implementing the speaker recognition system according to the present invention is connected to a local network for the purpose of authenticating a user of the database 400. Users of the system can be directly connected to the local network 402. More generally, users at sites such as 410 and 412 are equipped with microphones to access the system through desktop or laptop computers 414 and 416 connected to other local networks that in turn are connected to the Internet 406. Other users, such as 418, 420, and 422, can access the system by dial-up modem connection through public switched telephone network 424 and Internet service provider 426.

avatar

The algorithm used by the speaker recognition system according to the present invention may be implemented as a computer program using any suitable programming language such as C or C ++, and the executable program may be an independent application on a hardware / operating system (OS) platform. It may be implemented in any required form (hardware / firmware implementation) including embedded code in a DSP chip, etc., or may be embedded in an operating system (eg, MS Windows DLLs). Similarly, the user interface (for both system registration and subsequent system access) can be implemented in various forms, including a web-based client server system and a web browser-based interface, where the voice sampling case is, for example, ActiveX / It can be implemented using Java components.

Apart from desktop and laptop computers, the system is applicable to other terminal devices including palmtop devices, WAP enabled mobile phones, and the like, via wired and / or wireless data / mobile communication networks.                 

Application

The speaker recognition system with constant flexibility and reliability provided by the present invention has various applications. According to another aspect of the present invention one particular example provides an audit trail of a user who accesses and / or modifies digital information such as documents or database records. Such a transaction may be recorded as providing date / time and user identification related information as is known in the art. However, conventional systems do not normally verify or authenticate user identification information.

Preferably, speaker recognition using the speaker recognition system according to the present invention may be used to verify the identification information of the user when necessary, for example when opening and / or editing and / or storing a digital document, a database or the like. Can be. The document or record itself may be presented as data relating to the speaker verification procedure, or such data may be recorded as a separate audit trail to provide a validated record of access and changes to protected documents, records, and the like. Unauthorized users identified by the system will not be able to access the system, and system monitoring will prevent them from performing system related actions.

The invention includes many improvements and modifications without departing from the scope of the invention as defined in the appended claims.

Claims (33)

A speaker recognition method using a cohort including a registration model for each of a plurality of speakers, the speaker recognition method in which the registration model represents at least one voice sample for each speaker, Capturing a test speech sample from a speaker claiming to be one of the registered speakers; Modeling the test sample to provide a test model; And Classifying the test model by always matching the test model with one of the registered models of the cohort such that a false acceptance rate of the test sample is determined by the size of the cohort, Wherein the modeling and / or classification step causes the false rejection rate of the test sample to be zero. The method of claim 1, Providing a cohort plurality of registration models determined for each speaker using a plurality of parallel modeling processes; Applying the plurality of parallel modeling processes to the test sample to provide corresponding plurality of test models; And Classifying each test model by matching each test model with one of the corresponding modeled registration models of the cohort. The method of claim 2, The parallel modeling processes include applying at least one of different frequency banding, different spectral modeling, and different clustering to the speech samples. The method according to any one of claims 1 to 3, Wherein the classification steps include applying a plurality of parallel classification processes to the test models or to each of the test models. The method of claim 4, wherein The parallel classification processes include testing the test model for different cohorts of registered speakers or for different pronunciations represented by the registration models. 6. The method of claim 5, further comprising adding a known noise or another known signal to the test speech sample to generate a modified test sample, wherein the modeling step is performed on the modified test sample, Speaker recognition method. A speaker recognition system using a cohort including a registration model for each of a plurality of speakers, wherein the registration model represents at least one voice sample for each speaker, the speaker recognition system comprising: Means for capturing a test speech sample from a speaker claiming to be one of the registered speakers; Means for modeling the test sample to provide a test model; And Means for classifying the test model by always matching the test model with one of the registered models of the cohort such that a false rejection rate of the test sample is determined by the size of the cohort, Wherein said modeling means and / or classification means cause a false rejection rate of said test sample to be zero. The method of claim 7, wherein Each speaker of the cohort has a plurality of registration models determined using a plurality of parallel modeling processes, applying the plurality of parallel modeling processes to the test sample to provide a corresponding plurality of test models, and generating each test model. Means for classifying each test model by matching with one of the corresponding modeled registration models of the cohort. The method of claim 8, And the parallel modeling processes operate to apply at least one of different frequency banding, different spectral modeling, and different clustering to the speech samples. The method of claim 7, wherein The classification means operative to apply a plurality of parallel classification processes to the test models or to each of the test models. The method of claim 10, And the parallel classification processes operate to test the test model for different cohorts of registered speakers or for different pronunciations represented by the registration models. The method of claim 11, And means for adding a known noise or another known signal to the test speech sample to produce a modified test sample, the modeling means operative to act on the modified test sample. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images