KR20040028790A - Speaker recognition systems - Google Patents

Abstract

In a speech recognition (identification and / or verification) method and system, speech models for registered speakers consist of sets of feature vectors representing the smoothed frequency spectrum of each of a plurality of frames, clustering into feature vectors of a frame. The algorithm is applied to obtain a reduced data set representing the original speech sample, with adjacent frames overlapping by at least 80%. This type of speech model shows the transient independence by modeling the static components of the speech sample. An identifier scheme is used in which modeling and classification processes are selected to give a false rejection rate that is approximately equal to zero. The registered speaker is associated with a cohort of any other registered speakers, and the test sample is always matched with one of the relevant cohorts or claimed identification information. Thus, the overall error rate of the system depends only on the false tolerance rate, which is determined by the cohort size. False error rates are further reduced using multiple parallel modeling and / or classification processes. The speech models are normalized prior to classification using a normal model derived from one of the registered speaker samples or from the test speech sample (most preferably from the claimed identification registration sample).

Description

Speaker recognition system {SPEAKER RECOGNITION SYSTEMS}

Speaker recognition includes the related fields of speaker verification and speaker identification. Its main purpose is to identify the claimed identity from the speaker's utterances (known as "verification") or to recognize the speaker from the pronunciation ("identification"). Known). This verification and identification uses a person's voice as a biometric measure and assumes a unique relationship between the pronunciation and the person who generates the pronunciation. This unique relationship allows both verification and identification. Speaker recognition technology analyzes test pronunciation and compares it with a template or model known to the person being recognized and verified. The effectiveness of such a system depends on the quality of the algorithms used in the processing.

Speaker recognition systems have as many applications as possible. According to a further aspect of the invention, speaker recognition technology may be used to permanently mark an electronic document with a biometric print for everyone viewing and editing its contents. This creates an audit trail to identify all of the users and to identify the number of connections and modifications. Since the user mark is biometric, it is very difficult for the user to challenge the reliability of the mark.

Other biometric means can provide a reference for possible recognition systems such as iris scanning, fingerprint and facial features. All of these measuring means require additional recording hardware, while speaker recognition can be used with all voice inputs, such as by telephone lines, or without change using a standard multimedia personal computer. Such techniques can be used in combination with other security measures and other biometric means for enhanced security. From the user's point of view, the behavior of such a system is very simple.

For example, if an online document is requested, the person requesting the connection will be required to provide the voice sample. This would be a simple prompt from the client software ("speak this phrase") or something similar. The pronounced phrases are then transmitted to a speech recognition server or database server via any data network, such as the Internet, to be stored as a key associated with the document and used to activate the document at a particular time. Thus, over time, a permanent record for the document can be created to provide an audit trail for the document. The speaker authentication server may maintain a series of models (models) for all currently registered people and a history record of previously registered people.

Speaker recognition systems rely on some unique features from human speech. Again this will depend on how human voices are generated using vocal tracts and nasal tracts. For practical purposes, saints and nasal sinus can be regarded as two connected tubes that resonate in a similar way to musical instruments. The resonance generated thereby depends on the diameter and length of the tubes. In human speech production, to some extent, the lengths and diameters of the pipe sections are modified by articulators (primarily tongue, jaw, lips and soft palate or velum). Such resonances on the spectrum are called formant frequencies, and normally there are about four formant frequencies on a typical speech spectrum.

Like a musical instrument, the constriction of airflow occurs and only the sound is generated when vibrations and turbulence occur. In the case of human voices, major vibrations occur when contractions occur in glottis (vocal cords). When this happens, voiced sounds, usually vowel-like sounds, are produced. If such contractions occur in the tongue or thereby in the mouth, turbulence is produced (a hissing type of sound), and the generated voice is "s", "sh", "th (S) "and the like are referred to as friction sounds. From an engineering point of view, this is similar to the source signal (the result of the contraction) applied to a filter with the general characteristics of the vocal tract (ie the same resonances), and the resulting output signal is a speech sound. Becomes The actual speech is generated by dynamically changing the positions of the articulation organs.

All existing speaker recognition systems perform similar computations. These systems operate by creating a model or model for the registered speaker. This model is created by two main steps applied to speech samples: spectral analysis and statistical analysis. Subsequent recognition of the input speech sample by modeling the input sample (test pronunciation) in the same manner as in the speaker registration process and by performing pattern / classification matching of the input model to the registered speaker's database. Processing is performed. Existing systems differ in the way they are taken when performing all or some of these steps. In conventional (industrial standard) systems, spectroscopic analysis includes Linear Predictive Coding (LPC) / Cepstral Analysis (LPCC) or FFT / subbanding. Then there is a statistical analysis technique called Hidden Markov Modeling (HMM), where the above classification step is used to match and "impostor cohort" against the claimed speaker model. Or a combination of matches against a "world model" (ie, a series of other speaker models).

To enable effective processing of speech samples, all speaker recognition systems use time slices called frames, where the pronunciation is divided into frames, and each frame is processed again. The frames may or may not be the same in size with each other, and may or may not overlap. An example of a typical time signal representation of speech utterance divided into frames is shown in FIG. 1 of the accompanying drawings. A general speaker recognition system is shown in the form of a block diagram in FIG. 2, where test pronunciation is input filter 10, spectral analysis (LPCC) stage 12, and statistical analysis stage. analysis (HMM) stage) 14, and then before generating information about the speaker's identity (identification) or the speaker's claimed identity (verification). Using the database 18 of speaker models (registered speaker data-sets) and thresholding, processing by the score normalization and speaker classifier 16 is illustrated. It is.

Such systems have several disadvantages or limitations. First, existing spectral analysis techniques create limited and incomplete feature sets, resulting in poor modeling. Secondly, HMM technology is a "black box" method that combines superior performance with relatively ease of use at the expense of transparency. The relative importance of the features extracted by these techniques is not visible to the designer. Third, the nature of HMM models does not allow model-against-model comparisons to be performed effectively. Thus, important structural details contained in registered speakers cannot be effectively analyzed and used to improve system performance. Fourth, HMM technology uses temporal information to construct the model, and thus becomes vulnerable to mimics who mimic the voices of others by temporarily changing pitch or the like. Fifth, the world model / imposter cohort used by the system to test the claimed speaker's pronunciation cannot be easily optimized.

The performance of the speaker recognition system depends on the following points. That is, if true speaker pronunciation is tested against a model for that speaker, it will depend on the fact that the imposter pronunciation produces a score lower than the score produced when tested against the same model. Accordingly, the allow / reject threshold is set. Successive tests by a true speaker will not produce the same score. Instead, the scores will form a statistical distribution. However, the mean of the true speaker distribution will be significantly lower than the mean of the imposter distributions tested for the same model. This is shown in Fig. 3, in which 25 scores were made for each of the eight speakers when speaker 1 was the true speaker. 3, it can be seen that the scores of some speakers can be a problem closer to the true speaker than others.

The present invention relates to a system, method and apparatus for performing speaker recognition.

1 illustrates a signal representation of an example of dividing speech utterance into frames;

2 is a block diagram showing a general conventional speaker recognition system;

3 is a plot showing speaker distribution score distributions for the number of speakers tested;

4 is a block diagram showing a first embodiment of the present invention;

5 is a block diagram showing a second embodiment of the present invention;

6 is a block diagram showing a third embodiment of the present invention;

7 is a block diagram showing another embodiment of a speaker recognition system according to the present invention;

8A shows a time signal representation of an example of speech pronunciation divided into frames;

8B shows the corresponding frequency spectrum and smoothed frequency spectrum of one frame therein;

9 shows the difference between frequency spectra of two mis-aligned frames;

Fig. 10 shows the distribution of accumulated frame scores plotted against their frequency of occurrence;

FIG. 11A is a diagram showing the same accumulated score distributions as in FIG. 3 for comparison with FIG. 11B, showing corresponding accumulated score distributions obtained using the speaker recognition system according to the present invention; FIG.

12 shows the results of a model-to-model comparison when compared with an actual test score obtained with the system according to the invention;

13 shows the distribution of a speaker model used by the system according to the invention in a two-dimensional representation of a multidimensional data space;

14 illustrates the use of an imposter cohort as used in a system according to the present invention;

15 is a block diagram illustrating a normalization process in accordance with an aspect of the present invention;

16 is a block diagram illustrating an example of a wide area user authentication system in accordance with the present invention;

FIG. 17 is a block diagram illustrating the disruption of a speech signal by various noise sources and channel characteristics in an input channel of a speaker human system;

18 and 19 illustrate the effect of noise and channel characteristics on test pronunciations and registration models in a speaker recognition system; And

20 illustrates a channel normalization method according to an aspect of the present invention.

The present invention relates to an improved speaker recognition method and system that provides improved performance over existing systems. The invention provides, in many other respects, improved spectral analysis, transparency in statistical analysis, improved modeling, models that can be compared so that the data-set structure can be analyzed and used to improve system performance, and improved classification. Methods, including, but not limited to, the use of statistically independent / partially independent parallelisms to improve system performance, and the like.

In addition, the present invention provides a computer program for implementing the method and system of the present invention, data carriers and storage media encoded with such a program, a data processing apparatus and system applied to implement the method and system, and the device And data processing systems and apparatuses including systems.

Various aspects and preferred features of the invention are defined in the appended claims.

The present invention includes various aspects and features that can be combined in various ways to provide an improved speaker recognition (verification and / or identification) system. Certain aspects of the present invention relate to the manner in which speech samples are modeled during speaker registration and during subsequent recognition of input speech samples. Other aspects relate to the way in which the input speech models are classified to reach a decision about the speaker's identification. Another aspect relates to normalizing the speech signals input to the speaker recognition system (channel normalization). Another aspect relates to the application of speaker recognition systems.

Referring now to the drawings, Figures 4 through 6 illustrate the basic structure used in a system incorporating various aspects of the present invention. The inputs to all of the embodiments of the invention described herein are digital signals containing speech samples that have been digitized in advance by any suitable means (not shown), and all the filters and other modules referenced are also digital. You can see that.

In FIG. 4, speech samples are input to the system through channel normalization module 200 and filter 24. Instead of or in addition to such "front-end" normalization, channel normalization may be performed later in the processing of speech samples, which will be discussed later. The sample is divided into a series of frames at some other point before input to filter 24 or before feature extraction. In some embodiments, the noise signal 206 may be added to the filtered signal (or may be added before the filter 24), as described later. Sample data is input to a modeling (feature extraction) module 202, which includes a spectral analysis module 26 and a statistical analysis module 28 (at least in the case of speech samples processed for registration). The model (feature set) output from the modeling module 202 includes a set of coefficients that represent the smoothed frequency spectrum of the input speech sample. During speaker registration, the model is added to a registered speaker's database (not shown). During the recognition of the input speech sample, a model (feature set) is input to the classification module 110, which compares the model (feature set) with the models selected from the registered speaker's data. Based on this comparison, a determination is made at 204 to identify the speaker and verify the claimed identification of the speaker. The channel normalization of the input sample and the addition of the noise signal 206 constitute several aspects of the present invention, as described in more detail later on, which corresponds to the preferred features of all embodiments of the present invention. . In some embodiments, channel normalization may be applied during classification or after spectral analysis 26 rather than being applied to input speech samples prior to processing as shown in FIGS. 4-6. New aspects of modeling and classification processes according to other aspects of the present invention will be described in more detail later.

Other aspects of the invention include various forms of parallel processing in the processing of speech samples for registration and / or recognition.

Referring to FIG. 5, the basic operation of the system is the same as that of FIG. 4 except for the following. That is, the difference is that the output from modeling module 202 is input to multiple, parallel classification processes 110a, 110b, ..., 110n, and the outputs from multiple classification processes are combined to reach the final conclusion. This will be described in more detail later. In FIG. 6, the basic operation of the system is the same as that of FIG. 4 except for the following. That is, the input sample is processed by multiple, parallel modeling processes 202a, 202b, ..., 202n (mainly providing slightly different feature extraction / modeling as described later), which process ( If the noise signal 206 appears to be added to an input signal upstream of the filters 24a, 24b, ... 24n, it is via the multiple filters 24a, 24b, ..., 24n. Possible, and also outputs from the multiple modeling processes are input to the classification module 110, which will be described in more detail later. Preferably, this type of multiple parallel modeling procedures apply to both registration sample data and test sample data.

Multiple parallel modeling processes may be combined with multiple parallel classification processes, for example the input to each of the parallel classification processes 110a-n of FIG. 5 is output from the multiple parallel modeling processes as shown in FIG. 6. Can be

Hereinafter, various aspects of the present invention will be described in more detail with reference to the modeling, classification, and normalization processes illustrated in FIGS. 4 to 6.

modelling

The spectral analysis module 26, 26a-n can apply similar spectral analysis methods to those used in existing speaker recognition systems. Preferably, the spectral analysis applied by modules 26a-n takes the form of extracting, for each frame of sample data, a series of feature vectors (coefficients) representing the smoothed frequency spectrum of that frame. do. Preferably, this includes LPC / ceptral (LPCC) modeling, which produces an increased set of features modeling the more delicate details of the spectrum, the delta ceptral of the coefficients selected based on the weighting scheme or May include variations such as emphasis / de-emphasis. Instead, similar coefficients can be obtained by other means, such as a Fast Fourier Transform (FFT) or by a filter bank.

A complete sample is represented by a matrix of coefficients of one row of each frame of that sample. In a preferred embodiment of the invention, these matrices will each have an order size of 1000 (frames) x 24 (coefficients). In existing systems, statistical analysis such as HMM would have been performed on this type of first single matrix representing a complete original signal.

As will be appreciated by those skilled in the art, the LP transform effectively produces a series of filter coefficients representing the smoothed frequency spectrum for each frame of test pronunciation. LP filter coefficients are related to Z-plane poles. The cepstral transform compresses the dynamic range of the smoothed spectrum and moves the LP poles by moving them closer to the Z-plane origin (away from the real frequency axis at z = e ^jw ). It has the effect of deemphasis. The septral transformation uses log functions for this purpose. It will be appreciated that other similar or equivalent techniques may be used in the spectral analysis of speech samples to obtain a smoothed frequency spectrum and to de-emphasize those poles. This de-emphasis results in a series of coefficients that are less dynamic and more balanced when converted back to the time domain. Similar to). The logarithm function also converts multiplicative processes into additive processes.

The model derived from the speech sample can be regarded as a series of feature vectors based on the frequency content of the sample signal. When the feature vector based on the frequency content is extracted from the signal, the order of the vector is important. If the order is too small, some important information may not be modeled. To avoid this, the order of the feature extractor (eg the number of poles of the LP filter) may be chosen to be larger than the predicted order. However, this inherently causes problems. The poles that match the resonances in the signal provide good results, while the coefficients as another result of the feature vector model the spurious aspects of the signal. Thus, comparing this vector with other models or criteria, the calculated distance measure can naturally be influenced by the values of those coefficients modeling the pseudo elements of the signal. The distance measurement (score) that is returned is therefore inaccurate, and is likely to give a bad score for a frame that is actually a good match.

According to one aspect of the present invention, this problem is caused by a noise signal n (t) having known (known) characteristics in the speech signal s (t) (FIGS. 4 to 4) before the signal is input to the modeling process. By adding 206 of FIG. 6, it can be prevented or mitigated beforehand (ie, input signal = s (t) + n (t)). This same noise signal can be used during speaker registration and subsequent use of the system. Adding the known noise signal has the effect of forcing "extra" coefficients (actually more than necessary) to model the known function and thus to give more trouble-free consistent results during model / test vector comparisons. do. This is particularly relevant for suppressing the effects of noise (channel noise and other noise) during the "silence" of speech sample data. This problem may be addressed as a result of the use of largely overlapping sample frames, which will be described later.

As mentioned above, to enable effective processing of speech samples, all speaker recognition systems use time fragments called frames, whereby the pronunciation is divided into a series of frames, each frame being processed again. The frames may be the same size or not the same size, or may overlap. The models generated by the speaker recognition system thus comprise a plurality of feature sets (vectors corresponding to sets of coefficients) representing a plurality of frames. When the models are compared in the existing speaker recognition system, it is necessary to align the corresponding frames of each model. Different pronunciations for a given phrase cannot have exactly the same length, even if they are spoken by the same person. This makes it difficult to align the frames correctly for comparison.

Existing systems have either framed or smoothed frames as shown in FIG. 8A (representing a time signal divided into frames) and FIG. 8B (representing a corresponding frequency spectrum and smoothed frequency spectrum of one of the frames of FIG. 8A). Convert to a spectral or smoothed spectral equivalent. The system then performs additional transformations and analysis (such as Septral Transform, Vector Quantization, Hidden Markcove Modeling (HMM), and Dynamic Time Warping (DTW)) to achieve the desired results. Frame boundaries can be assigned in various ways, but are primarily measured from any starting point that is assumed to be the starting point of a useful voice signal, to compensate for this starting point, and also to natural variations in the length of similar sounds. To compensate for, techniques such as HMM and DTW may be used when comparing two or more pronunciations, such as when constructing models or comparing test pronunciations with models. Using any coefficients used to produce the HMM / DTW compensation, at the point in the system after spectral analysis And thus do not refer to the original time signal, so the alignment precision is not limited to the size of the frame, and these techniques also allow for some frames to be predicted that the arrangement of a particular frame will be there. It assumes that it will be in a fixed region of pronounced pronunciation, which introduces a temporal element into the system as the estimated arrangement of the current frame depends on the arrangement of the previous frame and the arrangement of subsequent frames depends on the arrangement of the current frame. In practice, this typically means that a particular frame, such as a frame that exists for 200 ms in pronunciation, is only compared to other frames in the 200 ms region of the model or other pronunciations used to construct the model. This approach can be used for speech recognition methods (eg voice to text). Is derived from a ring), where it is used to estimate the phoneme sequence (phonetic sequence) from a series of frames. The applicants have determined that this method for the following reasons inadequate in speech recognition.

A) As the most serious, the existing approach provides only a crude alignment of the frames. Whatever allocation of starting points means, in general, it is not possible to obtain an exact arrangement of the starting points of each of the two frames, so that even two frames giving a "best match" are significantly different from each other, as shown in FIG. It means that it can have different spectral characteristics.

B) Secondly, the existing scheme relies on a temporal sequence of the frames, and speaker verification is based on spectral characteristics inferred from temporally adjacent frames.

According to another aspect of the invention, in order to overcome the problems arising from the frame arrangement between the models (as described in (A) above) and to improve the quality of the obtained model, convolution in the registration modeling process of the invention This involves using very large frame overlap similar to. This technique is applied during speaker registration to obtain a model (preferably based on repeated pronunciation of an enrollment phrase). By massively overlapping the frames, the resulting model effectively approaches the model of all possible arrangements with relatively little difference between adjacent frames, providing good modeling of the patterns. Preferably, the frame overlap is selected to be at least 80%, more preferably in the range of 80% to 90%, up to 95% possible.

These frames are transformed into representative coefficients using the LPCC transform as described above, so that each pronunciation included in the reference model generated by the registration process is (typically, 24 coefficients as described above). The size of the order of 1000 frames). Typically, there may be ten such matrices representing ten pronunciations. Clustering or averaging techniques, such as vector quantification (described further below), are then used to reduce the data to generate the speaker's reference model. This model does not depend on the temporal order of the frames, thus dealing with the problems mentioned in (B) above.

In preferred embodiments of the present invention, a large overlap of the above-described frames and vector quantification as described below are combined. This provides a significantly different operating mode than the existing HMM / DTW system. In such an existing system, all frames were considered equally valid and generally lead to a yes / no determination of the final "score" for thresholding by accumulating the scores derived by comparing and matching each frame. It was used to The validity of the scores thus obtained is limited by the accuracy of the frame arrangements.

According to this aspect of the invention, the reference (registration) models will represent a large number of possible frame arrangements. This results in the generation of frame score values such that all frames of the test pronunciation are compared and scored for all frames of the reference model, rather than matching each frame of the test pronunciation and deriving scores for each matching pair of frames. Giving a statistical distribution of frequencies. "Good" frame matches result in low scores, and "bad" frame matches result in high scores (or may be reversed depending on the scoring scheme). The test pronunciation frame tested for a large number of reference models will eventually be a normal distribution as shown in FIG. 10. Most frame scores will be close to the mean and within some standard deviations. Because of the large overlap of frames in the reference models, the score distributions will include "best matches" between the reference models and the correctly arranged corresponding frames of the test pronunciation. Thus, if the test pronunciation from a particular speaker is tested against a reference model for that speaker, the distribution will contain higher incidence of very low scores. This ultimately results in consistently low "true speaker" scores, since some parts of the pronunciation are easily identified as coming from the true speaker, while others are less clearly identified as coming from the true speaker. Will be the value. Imposter frame scores will not produce low scores and will be classified as coming from the general public.

That is, according to this aspect of the invention, the reference models comprise sets of coefficients derived for a plurality of largely overlapped frames, wherein the test pronunciation compares all of the frames of the test pronunciation with all of the frames of the relevant reference models. And by analyzing the distribution of frame scores obtained from it.

Large overlaps of frames applied to speech samples for registration may be applied to input pronunciations during subsequent speaker recognition, which is not required.

Using massive overlaps in registration sample data is also beneficial in addressing problems resulting from noise present within the periods of silence in the sample data. Such problems are particularly important for text-independent speaker recognition systems. The presence of silence may or may not cause problems for individual models or verification attempts, but will cause overall system performance to deteriorate. Therefore, the question is how to eliminate it completely or minimize its adverse effects. Using large frame overlaps in accordance with the present invention is an inherent solution. Consider the following equation that represents the average of the frame spectrum (this will be described in detail later).

From this equation we can see that the average of the static parts (ss) is ss (ω), and each frame has the spectrum ss _n (ω) x sd _n (ω) For the spectrum of the received frames ss ₁ (ω) xsd ₁ (ω) + (ss ₂ (ω) xsd ₂ (ω)) = ss (ω) x (sd ₁ (ω) + sd ₂ (ω)) Consider, here, the steady part is multiplied by the new spectrum (sd ₁ (ω) + sd ₂ (ω)), but the new spectrum is supposed to be reduced by averaging and also essentially Because they are dynamic or variable, they must behave in exactly the same way as randomly extracted frames, which means that the frames can be added randomly together with minimal impact on performance. May have the case of valid speech frames added to silence frames that are valid speech frames. As such, such a judgment is not entirely correct—in fact, this results in performance improvements as the model no longer contains unwanted silences.

If a typical signal with some minor silence problems has randomly added timeframes, the silences will be removed but the signal will appear to have undergone major corruption. However, the present invention using massively overlapped frames still functions. Interestingly, this means that channel echoes have no effect and can be ignored. This also clearly indicates that the preferred modes of operation of the present invention will extract more static portions of the spectrum than conventional verifiers (described further below). Adding frames in this manner will have substantially the same effect as adding colored noise, thus preventing unwanted modeling as described above.

According to another aspect of the present invention, clustering or averaging techniques such as vector quantification applied by modules 28, 28a-n are used in a manner different from the statistical analysis techniques used in existing speaker recognition systems.

Preferably, the system of the present invention uses the Vector Quantization (VQ) technique in processing the registration sample data output from the spectrum analysis modules 26, 26a-n. This simplifies the technique compared to statistical analysis techniques such as HMM used in many conventional systems, resulting in transparent modeling that provides models in a form that allows model-to-model comparison in subsequent classification steps. In addition, vector quantification (VQ) as used in the present invention does not use transient information such that the system is resistant to imposters.

The vector quantification process effectively compresses the LPCC output data by identifying clusters of data points, determining average values for each cluster, and abandoning data that certainly does not belong to any cluster. This in turn results in a set of second matrices representing LPCC data of the first set of matrices but having a reduced coefficient (typically 64 x 24 compared to 100 x 24, for example).

Hereinafter, the effects of using LPCC spectrum analysis and clustering / averaging in the present invention will be described.

The basic model assumes that spectral magnitude is useful and that its phase can be ignored. This is known to apply to human hearing, and if not applied to the verifier, the system will show undesirable phase related problems such as sensitivity to the microphone's distance from the speaker. In addition, the spectral information of a speech sample can be regarded as being composed of two parts, a static part ss (ω) and a dynamic part sd (ω), and the processes are called multiplicative processes. Make a home. Also assume that the dynamic part is considerably larger than the static part:

s (ω) = ss (ω) x sd (ω)

The static part is fixed by definition, making it more useful as a biometric as it relates to the static properties of the vocal tract. This results in some fixed physical properties as opposed to sd (ω), which relates to the dynamics of speech.

Complete extraction of ss (ω) will give a biometric that exhibits the characteristics of the physical biometric, i.e. a physical biometric that cannot be arbitrarily altered and does not deteriorate over time. Instead, exclusive use of sd (ω) would give a biometric that exhibits the characteristics of a behavioral biometric, i.e., a behavioral biometric that can be arbitrarily altered and deteriorates over time. will be. Mixing these two should show intermediate features, but since sd (ω) is much larger than ss (ω), the combination is more likely to show the features of sd (ω) (i.e., become behavioral). high.

As with all frequency representations of the signal, the assumption is that the time signal is between -∞ and + ∞, which is not physically possible. In practice, all spectral estimates of the signal are made using a window, which is present for a finite period of time. This window can be shaped or rectangular by a function (such as a Hamming window).

Using a rectangular window simply takes a section of the signal in the region of interest and assumes that it is zero elsewhere. This technique is that sections of the signal are common in speech processing called frames. 1 shows a time signal with indicated frames.

Frames can be formatted using alternating windows. Interestingly, the main effect of windowing is a kind of spectral averaging in which the characteristic of a particular frequency spreads to its surrounding frequency. This effect is caused by the main lobe, in addition to which side lobes generate periodic spectral oscillations in the spectrum. The system of the present invention then extracts all-pole linear prediction coefficients, which have the intended effect of spectral smoothing, and the extra smoothing caused by windowing. extra smoothing is not a major problem. However, if the window size is changed inadvertently, the effect of the periodic side lobe may be difficult. But this can be avoided by good housekeeping.

If it is possible to divide the time signal into frames, the spectral characteristics for the frames 1 to N may be expressed as follows.

s ₁ (ω) = ss ₁ (ω) x sd ₁ (ω); s ₂ (ω) = ss ₂ (ω) x sd ₂ (ω); ......................

s _n (ω) = ss _n (ω) x sd _n (ω); .......

s _N (ω) = ss _N (ω) x sd _N (ω)

But by the definition:

ss (ω) = ss ₁ (ω) = ss ₁ (ω) = ss ₂ (ω) = ss ₃ (ω) = .... = ss _N (ω)

The first result seems to be to extract ss (ω) using the following averaging process.

If the frames have independent spectral characteristics (each by random processing), U (ω) will tend to be white noise. That is, in this case U (ω) will have a flat spectrum Can be extracted by smoothing the spectrum. This is most likely the case if N becomes very large (N-> ∞). Given the linear nature of the time domain-frequency domain-time domain transforms, a similar analysis could be described in the time domain.

Under real world conditions, one cannot assume that N becomes large in the sense that the frames have independent spectral characteristics. It is important to remember that this requires N to become large under the following two conditions.

1. During model creation

2. During the verification event

Failure to comply with the conditions during such processing will potentially cause a system crash (error), but if condition 1 is not met, it will remain a potential source of error until it is updated, which is more serious, while condition 2 If it is, it will be a single instance event.

If U (ω) cannot be guaranteed to converge to white noise, what can be done to cope with that situation?

First, consider the following.

1. U (ω) will be a variable quantity.

2. Smoothed over the frequency spectrum would ideally be flat, ie smoothed version Usm (ω) = 1.

3. U (ω) is the truncated sum of speech frames whose number tends to ideally be infinite.

Consider the following equation.

If we return to the following frame based equivalent, the summation part that tends to be the flat spectrum is not an ideal performance measure.

If we take logarithms of that frame like this:

From the above equation, we can now see that the relationship between the static and dynamic parts is additive. Since the relationship between the time domain and the frequency domain is linear, the conversion from frequency to time is as follows.

lss (ω) + lsd (ω)-> cs (τ) + cd (τ) = c (τ)

In signal processing, c (?) Is known as a spectral transform of s (t) as described above.

In general, septal analysis consists of time_domain-> frequency_domain-> log (spectrum)-> time_domain (time_domain-> frequency_domain-> log (spectrum)-> time_domain). Septral transformation is used in many forms in speech analysis.

As mentioned above, in the use of the present invention, the septal coefficients for the frames are generated to extract the static portion as follows.

Ideally, the length of the voice signal will be long enough so that the dynamic part is completely random and the mean will tend to be zero. This will leave the static part cs (t) as the biometric measure of the present invention. However, there are various problems to be solved as follows.

1. How to deal with the imperfect nature of sum-to-zero

2. Channel change

3. Endpoint Pointing

4. additive noise

Referring to the incomplete nature of the sum to zero, the properties of the Septral coefficients decay over time, taking the form of an impulse response to stable systems. This means that the dynamic range of each coefficient is different and they are generally in descending order.

The difference between the frame coefficient values of the imposter model, the frame coefficient values of the true speaker model, and the mean coefficients of the test sample is not significant, and a simple sum for all of the pronunciation frames to produce a distance score. Summation may show that it will be difficult to threshold in the traditional way.

If you think about the two difficult problems of this method together without separating them, you will find the answer. To reemphasize the two difficult points:

The pronunciations will never be long enough for the average of the dynamic part of the convergence to zero.

2. The differences between true speakers and imposters are small and difficult to hold.

Consider two speakers with models based on the following equation.

Thus, the models become m1 (τ) and m2 (τ), where m1 (τ) is

At this time, e1 (τ) is an error.

In vector form, the models are as follows.

Expressed in the same form, the test pronunciation from Speaker 1 is as follows.

Using a simple distance measure, the true speaker distance is

In addition, the imposter distance becomes as follows.

Assuming that the convergence of the dynamic parts of the models is good (i.e., the error vectors are small compared to the static vector), then d1 <d2. This simply indicates that the models construct and display the registered speaker (a condition that can be easily checked during registration using the data available at that point in time). Interestingly, if e1 and e2 are small compared to the test signal error Te1, the distances are independent of e1 and e2. The condition under which the test error becomes large when compared to the model error is during the text-independent test conditions. This shows that if the dynamic components of the registration speech samples are minimized in the registration models, they can provide a good basis for text-independent speaker recognition of such models.

The above mentioned errors e1 and e2 are average model configuration errors; The actual errors are in frames and will be distributed around the mean. This distribution may be modeled in many ways, and it is simplest by using standard clustering techniques such as k-means to model the distribution. The use of k-minz clustering is known as other forms of vector quantification (VQ), and is a major part of the Self Organizing Map (SOM), also known as Kohonen Artificial Neural Netowork. to be.

Directly described above is a system in which test pronunciation is applied to two models and the closest one chosen is the variation of identification. In the case described above, if either Speaker 1 or Speaker 2 is registered as the registered speaker and they are tested, they will always be tested as real, thus False Rejection Rate (FRR) = 0. If an unknown speaker requires speaker 1 or speaker 2, the speaker is classified as one or the other, with a 1/2 chance of success, so False Acceptance Rate (FAR) = 50% Becomes Once the same number of true speaker tests and random imposter tests are performed, the overall error rate can be calculated as (FRR + RAR) / 2 = (0 + 0.5) / 2 = 25%.

It is clear that the number of models tested for it (the cohort) affects the FAR, and that number will decrease as the cohort increases. For cohort sizes where the recognition accuracy increases under these conditions, it is asymptotic at 100% because FRR = 0, but the accuracy is as follows, and as a more general term it is asymptotic at 100-FRR.

It is to be noted that at this point the FRR and FAR are greatly decoupled. That is, the FRR is fixed by the quality of the generated model and the FAR is fixed by the cohort size. Also, to halve the error rate, the cohort size is doubled, for example, the cohort is 50 for 99% accuracy, the cohort is 100 for 99.5% accuracy, and the cohort is 200 for 99.75% accuracy. As the cohort increases, the computational load increases, and in fact, every half the error rate doubles the computational load. If the cohort increases by a very large number, FRR and FAR will break down and FRR will begin to increase.

Instead of constantly increasing the cohort size, another method is needed to reduce the FAR. The method, according to one aspect of the present invention, employs parallel processing (discussed elsewhere in the detailed description of the present invention), which exhibits slightly different imposter characteristics, and thus partially with respect to an identifier strategy. Is statistically independent. The idea is to take a core identifier with a FAR set to a cohort size, with a FRR of zero or nearly zero. And the processing of the beginning and end of this core identifier is slightly modified to reorder the distances from the true speaker model of the cohort member models. This is done while maintaining FRR ~ 0, and can be achieved by changing the transformed coefficients, such as using delta-ceps or the like, or by changing the spectral shaping filters 24a-24n (see FIG. 7).

When a registered speaker uses the system, the test signal applies to all processes in parallel, but each process has a FRR to 0 and the speaker will pass through it. If an unknown imposter uses the system, the imposter will pass each process with a probability of 1 / cohort_size. However, the present invention introduces conditional probabilities by parallel processes. In other words, conditional probabilities were introduced in such a way that the imposter passed through modified process 2 and then passed through process 1, which is likely to be determined. Although the probability that the imposter will pass through all processes is not statistically independent ("stastically_independent_result = process_prob ^{no_pf_process"} ), the probability is reduced by adding the process. For the FAR value of a given process, it can be seen that the accuracy of the overall system increases with the total number of processes.

When multiple parallel processes are used in this way, a technique for matching a test sample and the claimed identity with each other requires a successful match in each process or a predetermined predetermined value. A successful matching rate may be required.

When constructing the registration model according to the present invention, a combination of massive sample frame overlaps and vector quantization (or equivalent) provides very special advantages. Massive overlapping can be applied at the time of the utterance test, but also when constructing models. This technique typically uses 80 to 90 percent of massive frame overlap to produce as many alignments as possible. The frames generated by the alignment are transformed into representative coefficients using the LPCC transform to produce a coefficient matrix (matrix) representing the entire alignment. This can solve the conventional frame alignment problems. In general, a matrix has a size of "no_of _frames x LPCC_order" (no_of _frames by LPCC_order), for example 1000 x 24. This is repeated for all the pronunciations that make up the model, and typically generates 10 1000 x 24 matrices with 10 iterations. Vector quantization is then used to reduce the data for creating a model for speaker. This has the effect of averaging frames in order to reduce the significance of the dynamic elements of the sampled speech data. The model of the final result is not essentially a model that changes over time because it does not recognize the position of the frame in the test pronunciation. This leads to a temporal dependency problem.

The combination of VQ and massive frame overlapping creates a different mode of operation from conventional systems based on HMM / DTW. In HMM / DTW all frames are equally valid and are used to form a final score for performing a “yes / no” thresholding. The invention tests by comparing every row (frame) of test sample data with every row of registration model data for the requested speaker and associated imposter cohort. The best match for each row of test sample data can be obtained from one row of the registration model and generates a test score for the relevant registration model. The test sample is matched with a registration model that produces the highest score. At this time, if the matching information is determined to be the required identification information, the test speaker is accepted. If the match information is determined to be an imposter, the test speaker is rejected.

The system uses LPCC and VQ modeling (or similar / same spectrum analysis and clustering techniques) with massive overlapping of the sample frames to generate a reference model for each registrant. It is stored in the database. When using the system, the input test pronunciation is applied to pseudospectral analysis to obtain an input test model that can be examined against a registered speaker data-set. This approach has the advantage that it can be applied to achieve a very low FRR, such as zero. The importance of this is explained in more detail below.

Parallel modeling

As mentioned above, the performance of the speaker recognition system according to the present invention can be improved by using a multi-parallel process for model generation.

Referring to FIG. 7, a preferred embodiment of a speaker recognition system employing a parallel modeling process according to an embodiment of the present invention includes an input channel 100 and a channel normalization process for inputting an input signal representing speech samples to the system. 200, a plurality of parallel signal processing channels 102a, 102b ... 102n, a classification module 110 and an output channel 112. The system further includes a speech module database obtained from a registered speaker data set 114, i.e., a registered speaker using the system. Speech sample data is processed in parallel by each of the processing channels 102a-n, and the output of each processing channel is input to a classification module 110, which communicates with a database 114 of registered speaker data. . Decision information regarding the identity of the source of the test pronunciation is output through the output channel 112.

Each processing channel 102a-n is sequentially subjected to a spectral shaping filter 24a-n, (optional) added noise input 206a-n, spectral analysis module 26a-n, And statistical analysis modules 28a-n. The output from each statistical analysis module 28a-n is input to the classification module 110.

The spectral shaping filter 24a-n consists of a bank of filters that separate the pronunciation signal into a plurality of overlapping frequency bands, each frequency band being a continuous module 26a-n or 28a-n. In parallel. The number of processing channels and number of frequency bands depends on how many more channels are included to provide more detailed information in subsequent analysis of the input data. It is preferred to employ at least two channels, even more preferred to employ at least four channels. The filters 24a-n constitute a low pass, band pass, and high pass filter bank. The bandwidth of the base filter 24a is selected such that the error rejection rate FRR according to successive analysis of the output of the first channel 102a is "0" or as close as possible to "0". The continuous filters 24b-n have a gradually increasing bandwidth to pass progressively more signals from the input channel 100. Thus, different channel outputs have slightly different error acceptance (FA) characteristics, but the error rejection rate (FRR) for the output of each channel 102a-n remains close to " 0 ". Analysis of the combined output of the channels 102a-n shows an FRR close to " 0 " and a reduced overall error acceptance rate (required identification is only accepted if the output of all channels has been accepted). . The importance of this is explained in detail below.

The use of multiple frequency bands increases the size of the feature vectors of interest in continuous statistical analysis, thus improving the conventional single channel spectral analysis.

Several kinds of parallelism may be used in the modeling process to provide multiple feature sets for modeling different facets of the input speech sample and / or other similar models. Other types of filter banks may be used in addition to or in place of the low pass filter. Various types and variations of spectral and / or statistical analysis techniques can be used in parallel processing channels. Parallel statistical analysis adds different weights to sets of feature coefficients to obtain a set of models that are in a slightly deviated state.

The structure shown in FIG. 7 can be used to obtain a registration model stored in the database 114 and also to process test speech samples for the registration model. Each registration model may include a data set for each registration pronunciation. For each registration pronunciation, there may be a matrix of data representing the output of each parallel modeling process. Each of these matrices represents a clustered / averaged spectral feature vector. Test sample data can be controlled by the same parallel spectrum analysis process without clustering / averaging process. Thus, the test model data consists of a matrix representing spectral analysis data for each of the parallel modeling processes. When testing a test model against a registration model, a test matrix representing a particular modeling process is tested by comparing the registration matrix generated by the same modeling process.

Classification

The characteristic of the reference model obtained by the modeling technique described above is that the reference models themselves make a model-to-model comparison directly. This allows the system to use an identification strategy, where each registration model is associated with an imposter cohort. That is, as a reference or reference model of each registered speaker (subject, subject), an imposter cohort composed of a predetermined number of reference models of other registered speakers is used. At this time, the other registered speakers are clearly distinguished from the subject registered speakers, and have a known and predictable relationship with the subject model of the subject speaker. This predictable relationship allows the performance of the system to be improved. FIG. 11 shows the effect obtained by the conventional speaker recognition system in a manner similar to that of FIG. 3, comparing scores of input pronunciations checked against reference data for eight speakers. The speaker 1 is the actual "true" speaker. However, the scores of some other speakers are so close to each other that the system significantly reduces the degree of confidence that the system can identify with the correct speaker. 11B shows equivalent results obtained using the system according to the present invention. The result of the speaker 1 is very clearly distinguished from the result of the remaining speakers 2 to 8.

The speaker modeling method applied to the embodiment of the present invention is inherently simpler than using conventional techniques such as HMM and other possible methods such as Gaussian mixture models. However, the Applicant is convinced that the use of conventional tight statistical methods results in poor performance when applied to the "real world" because they are inherently flawed in the conventional methods. Based on this, the present invention has the effect of providing a much simpler statistical method in a relatively real case. As described above, there is a problem that the time-dependent characteristics of the HMM are easy to imitate, and the present invention seeks to solve this problem. Furthermore, the models of the present invention can ideally analyze the structure of the registered speaker data set by performing model-to-model tests.

There are two advantages to performing model-to-model comparison using the speaker model of the present invention. First, the speaker model of the present invention is the most relevant imposter present in registered speaker data-sets (ie, the data sets are very similar (or close to each other) and are evenly distributed around a particular model). Provide the ability to identify the most relevant impostors and an effective and predictable speaker normalization mechanism. VQ modeling requires the process of selecting the size of the model, i.e. the process of selecting the number of coefficients (center "centres"). Once the model size is selected, the center positions are moved until they are best suited for all registration data vectors. This means that the center is effectively assigned to a cluster of registration vectors, so each center in the model represents a cluster of information that is important for the speaker's identity.

The model-to-model test is in some way done against a registered speaker or a database of broad sense where the claimed identification is defined or a database of areas (i.e., system data spaces) restricted to the required identification. Allows you to predict if it is registered and required FIG. 12 shows a result of testing a reference model for a plurality of speakers 2 to 8 with respect to the reference model for the speaker 1. The ellipsoid represents the result of the model-to-model when the star notation represents the actual score of the speaker pronunciations (or utterances) tested for Model 1. Model-to-model tests can be used to predict the actual performance of a particular speaker for a particular reference model. Model-to-model results tend to be at the lowest level in the actual score distribution, indicating how well a particular imposter fits model 1. As such, the basic approach of using model-to-model testing to predict actual performance is well known. As will be described in detail below, in accordance with one embodiment of the present invention, the approach can be extended to protect a particular model from an imposter using respective selected and statistically variable groupings.

A second advantage obtained from model-to-model testing is that it can predict the performance of test utterances for a few, or if necessary, all registered speaker models. This advantage allows a virtually unlimited number of test patterns to be used to verify the identification, which is not possible with conventional systems.

In addition, model-to-model test results can be used to assemble the particular imposter cohort used with each reference model. This achieves accurate score normalization and also ensures that each model is effectively protected from imposters using statistically variable groupings selected for each registered speaker. This concept is illustrated in FIG. 3. Since each reference model can be regarded as a point existing in the multidimensional data space, the distance between the models can be calculated. 13 is shown two-dimensionally for clarity of this concept. As shown in Fig. 13, each asterisk represents a model, and a two-dimensional distance represents a distance between models.

Because the speaker model's distribution is not uniform, a world-model based on normalization techniques will not work properly in the same way on all speaker models. A few speaker models are relatively very close or very similar to each other, meaning that the imposter has the potential to successfully mimic the voices of registered speakers. This problem can be solved by creating a specific cohort of imposters around the subject model for each speaker model. This simplifies the normalization process and creates a guard for the imposters. This is illustrated in FIG. 14, in a manner similar to FIG. 13, where the object, i.e. the subject model, is indicated by a circle, the members of the imposter cohort are indicated by an asterisk, and the score of the imposter required by the subject is "x". Is displayed. The imposter scores are so close to the subject model that they can cause cognitive problems. However, since the speaker data-set allows us to predict how similar the real subject speaker is to the imposter cohort model, the above information tests the imposter against the cohort member model and the real object model, so that the imposter "x May be used for the purpose of distinguishing between That is, the utterance "x" of the imposter is similar to some of the cohort members beyond what is expected of the actual subject (i.e., the subject), and may also be distinguished from others more than expected. This implies an imposter event and is represented by an imposter pronunciation that is rejected as a match with the actual subject.

This provides the basis for a two-step recognition process, in which the imposter determined to be not the claimed speaker, if necessary, with very similar utterances as determined by the requested speaker. Rejection is preferentially using the more detailed process applied.

In an application of the speaker identification system (i.e., application), it is very important to minimize the possibility of false rejections, i.e., if the identification information requested by the user is falsely rejected as being fake. An identifier strategy according to one embodiment of the present invention provides very low error rejection rates, provides predictable system performance, and is generated in connection with the use of thresholds when accepting or rejecting the required identification. The problem can be minimized.

According to the method, the database of registered speakers (i.e. speaker space) is divided into several, so that each speaker registered in the system is assigned to a cohort of fixed N registered speakers as described above. A speaker classification module of the system (e.g., module 110 of the system of FIG. 4) is operated such that an input test utterance is compared with all members of the cohort associated with the identification information requested by the speaker, and the test pronunciation is best matched. is classified as that member of the cohort with (match). That is, the test pronunciation is always matched with one member included in the cohort, and can also be joined with other members of the cohort. If the cohort member whose pronunciation is matched corresponds to the requested identification information, the requested identification information is regarded as "true". If the pronunciation matches another member of the cohort, the requested identification is considered false and is rejected.

The modeling and classification process is tuned so that the error rejection rate is effectively " 0 " (FR = 0%). That is, the probability that the speaker is incorrectly identified as a cohort member rather than the required identification information is substantially "0". Since this is facilitated by the model-to-model comparison method, the match determination is based not only on test pronunciation matching the one most similar model, but also on the relationship with other members of the cohort. If the cohort has a fixed size "N", the maximum possible error acceptance rate (FA) is "FA = 100 / N%" and the overall average error rate is "(FA + FR) / 2 = 50 / N%". do. If the cohort size "N" is 20, the error rate is 2.5% (ie 97.5% accuracy). If the cohort size "N" is fixed, the system can be extended to populations of any size while maintaining a fixed and predictable error rate. That is, the accuracy of the system is determined based on the size of the cohort and is not affected by the size of the general population, so that the system can be applied to a very wide range of population. Increasing the cohort size within a range where the error rejection rate does not increase significantly can improve the accuracy.

This strategy does not matter whether or not thresholds are used to determine results, but these thresholds can still be used to reduce the error acceptance rate. That is, if the test pronunciation matches the required identification using the method described above, the thresholds can be applied to the process of determining whether the match of both is close enough / similar enough to be finally accepted. .

As mentioned above, a predetermined algorithm is used when selecting an imposter cohort associated with a particular registration model. Therefore, the members of the imposter cohort have a specific relationship to the registration model they want to know. In general, the degree of optimization can be provided in the classification process. However, randomly selected imposter cohorts are judged to fit the same well as most practical purposes. Here, the most important point is that the cohort size must be predetermined to provide predictable performance values. The imposter cohort for a particular registration model can be selected at registration or when testing test pronunciation.

Parallel Classification

The performance of the speaker recognition system according to the present invention can be improved using multiple parallel classification processes. In general, this process can be statistically independent or partially independent. This approach can provide multiple classification results, which can be combined with each other to yield the final result as shown in FIG. 5.

For example, using the above-described identification method, the same test pronunciation can be checked for different number of cohorts, different registration phrases or the test pronunciation combinations. If multiple cohorts are used, each cohort will apply an error rejection rate of "0" (FA = 0%) and an error acceptance rate of "FA = 100 / N%" as before. The overall error acceptance rate for N cohorts of equal size is "FA = 100 * M / N ⁿ %", and the average error rate is "50 * M / NN ⁿ %", where "M" is a coefficient greater than 1 As the effect of the processes that are not completely statistically independent. That is, if two cohorts are used and each cohort size is 20, the average error rate will be "0.125 * M%" compared to 2.5% when using one cohort as described above. Thresholds may also be applied to improve accuracy as described previously.

Other types of partially statistically independent processes may be used in the modeling process, the classification process, or both. In addition to the described embodiments, one pronunciation may be divided into a plurality of parts and processed separately.

Normalization

Another problem that arises in conventional speaker recognition systems is that system performance is affected by differences in speech sampling systems used for initial registration and subsequent recognition. This difference arises from different transducer (s) (ie microphone (s)) and soundcard (s) and the like. According to another embodiment of the present invention, this problem can be solved or mitigated by normalizing speech sampling based on the normalization characteristic obtained / stored in each sampling system (or possibly each type of sampling system). The sampling system is used to input speech samples to the recognition system. Alternatively (preferably), the normalization characteristic may determine to be on-the-fly "on the fly" state when a speech sample is input to the system. Since the normalization characteristic can be applied to all input speech samples, the reference model and test scores are independent of the characteristics of the particular sampling system. Alternatively or additionally, according to another embodiment of the present invention, the normalization process may be applied when checking a test sample against registered sample data.

The normalization characteristic can be effectively derived from the transfer function of the sampling system. For example, the normalization characteristic can be efficiently derived by inputting a known reference signal to the sampling system and processing the reference signal sampled through the speech recognition system. The resulting signal generated from the recognition system can be stored and then used to normalize the speech sample (s) that are subsequently input through the same sampling system or the same kind of sampling system.

Alternatively, as shown in FIG. 15, the speech signal S (f) modified by the transfer function C (f) of the input channel 300 may be estimated to estimate the transfer function C (f) of the channel 300 ( Input the modified speech signal " S (f) * C (f) " into the estimating module 302 and the normalization module 304 and normalize the inverse value of the estimated transfer function 1 / C (f). By inputting to the signal, it is normalized to the "on-the-fly" state, so that the output of the normalization module becomes very similar to the input signal S (f). Prediction module 302 generates a digital filter according to the spectral characteristics of channel 300, and the inverse value of this filter is used to normalize the signal. For example, by determining all-pole filters that represent the spectral quality of a sample signal, an inverse filter can be calculated. The filter coefficients are smoothed across the frame so that signals outside the spectrum of the channel "C (f)" can be removed as much as possible. This estimation of the channel spectrum is used to produce an inverse filter " 1 / C (f) ". This basic approach can be improved so that the pole positions of the filters obtained for the frame are smoothed, along with the desired cancellation of the poles. It is known that the poles are not related to channel characteristics.

According to the nature of the transfer function / normalization characteristic, the normalization process may be applied to speech samples before being processed by the speaker recognition system, or may be applied to spectral data, or models generated by the system. .

The preferred channel normalization method according to an embodiment of the present invention is applied to test model data and related registration models when examining test sample-to-registration models.

The overall effect of the channel characteristics on the speech signal can be expressed as follows.

here, Denotes a predictor of speaker characteristics, cc (ω) denotes a channel characteristic or an appropriately changed channel characteristic, and the speech signal is regarded as being composed of a static part and a dynamic part as described above. Is processed. Ideally, unwanted channel characteristics can be predicted and eliminated. In practice, such channel characteristic removal can be done in the time domain, frequency domain, or a combination thereof. Both of these achieve the same effect, ie the effect of predicting and eliminating the channel characteristic "cc (ω)" using some form of inverse filter or spectral segmentation. If is a predicted value of the spectrum of the unwanted channel, it can be expressed by the following equation.

If the estimate of channel characteristics is " If good, then the predictive value of the speech signal means that the unwanted spectrum is canceled and good. This can usually be implemented using an algorithm based on FFT.

Another embodiment is to model the channel characteristics as a filter (model) to be appropriate for the all-pole (all-pole) shape, which is expressed as follows.

The above equation is the most basic form of ARMA and can be extracted directly from the time signal using possible linear prediction.

Similar normalization is feasible based on Cepstral expressions.

In the Cepstral domain, the speech signal can be expressed as follows.

In addition, a speech signal modified by unwanted channel characteristics may be represented as follows.

As shown in this case, it involves an additive process rather than a product. However, it should be noted that both "cs" and "cc" are fixed, and only one "cc" needs to be removed without removing "cs".

It is important to consider the different conditions (registration model, database, induction cohort, test speaker, etc.) and the context in which the signal cc is to be removed.

16 illustrates various sources of speech sample modulation in a speaker recognition system. The input speech signal s (t) is changed by the ambient background noise b (t), recording device bandwidth r (t), electrical noise and channel crosstalk t (t) and transmission channel bandwidth c (t), thus recognizing the system. The signal input to becomes the change signal v (t). The system is easy to analyze in the frequency domain and the signal from the verifier is:

v (ω) = ((s (ω) + b (ω)) r (ω) + t (ω)) c (ω) (Equation A).

In the verifier, two conditions can be defined: one speaking and one not speaking. These two related equations are: v (ω) = ((s (ω) + b (ω)) r (ω) + t (ω)) c (ω) and v (ω) = ((0 + b (ω) r (ω) + t (ω)) c (ω).

First, as applied to the system according to the present invention, considering the simplified problem, b (t) = t (t) = 0, v (ω) = s (ω) r (ω) c (ω) = s (ω) h (ω) (where h () is a combined channel spectral characteristic), h (ω) = r (ω) c (ω) and v (ω) = s (ω) h (ω) Assume that ss (ω) sd (ω) h (ω).

The cohort model is selected from the speaker database recorded using the same channel (b), and the true speaker model is recorded using different channel (a). The test speaker may be a true speaker or an impostor and may be recorded using the third channel (c). 17 shows this diagrammatically. 18 shows the same expressed in different forms using Cepstral coefficients. As represented in FIGS. 17 and 18, the signal component value is an average corresponding to the sum of the sample frame data.

v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ) (Equation B) and v _m (τ) = cs _m (τ) + cd _m ( Consider the cohort model established from τ) + h _b (τ) (Equation C).

The problem with the verifier is that there are two different channels used in the identifier. The claimed distinction represented by the cohort channel (b), assuming that the difference between them is hd (τ) = h _a (τ)-h _b (τ) or h _a (τ) = h _b (τ) + hd (τ) The information model consists of v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ) = cs ₁ (τ) + cd ₁ (τ) + h _b (τ) + hd (τ) and v ₁ (τ) = (cs ₁ (τ) + hd (τ)) + cd ₁ (τ) + h _b (τ).

It is found that the mean of the static part of the claimed identification model is shifted by the difference between the channels and can cause errors if the true speaker is tested using channel b or if the situation is not corrected. Can be. In addition, the use of channel a may cause problems related to false acceptances.

One way to solve this problem is to remove the mean from the alleged ID model, but a simple mean elimination yields v ₁ (τ) = cd ₁ (τ) if the static part of the speaker model is also removed. Will be. However, when we look at v (ω) = ((s (ω) + b (ω)) r (ω) + t (ω)) c (ω) (Equation A) (system model with added noise) Considering the case where the speaker pauses, s (ω) = 0, where v (ω) = (b (ω)) r (ω) + t (ω)) c (ω) and v (ω) ) = n (ω) c (ω). Where n (ω) is a noise signal.

In the Cepstral form, v (τ) = n (τ) + c (τ) = sn (τ) + dn (τ) + c (τ), where sn denotes the static part of the noise, dn represents the result of the sum of the dynamic parts.

The model mean constructed from this is sn (τ) + c (τ), where sn is any fixed state noise, such as an interference tone, and c is a channel.

Considering Equation A (required identification model setting condition) again, v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ), which is a case where there is no noise. When the fixed state noise is added, v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ) + h _a (τ) + sn (τ).

By constructing the speaker pause model for this case, we can obtain sn (τ) + h _a (τ), and if we remove the mean using this, v ₁ (τ) = cs ₁ (τ) + cd ₁ (τ ).

This provides a model that is unbiased by the channel. A similar process is applied to each model, with the channel bias removed by the silence model. The test speaker is treated as the silence model is used to remove channel effects.

As mentioned above, channel characteristic removal (reduction) using the silence model requires full detection of the appropriate channel noise and silence portion of the pronunciation. They cannot be guaranteed, so they need to be relaxed (eg, if the silence contains some voice, include the speaker static voice of the claimed identification and remove it). Beneficially, they can be treated as one simple variant of the process, with all cohort models represented by the same silence model.

That is, in the cohort (including the claimed identification model), the silence average of the identification model claimed in all models is re-added. This represents all models with the same mean, sn (τ) + h _a (τ). This normalization also applies to test models, representing all models and test pronunciations with the same reference point. In fact, the reference channel and noise condition are chosen to represent all others with the same reference point.

This is illustrated schematically in FIG. 19 and shows the Cepstral coefficients of the test pronunciation along with the cohort models 1 to m and the asserted identification model input to the classifier 110. A "silence model" or "normalization model" 400 derived from the claimed identification registration data is used to normalize each of these prior to entering the classifier so that the actual input of the classifier is normalized test pronunciation, normalized required identification. Model and normalized cohort model. Ideally, the normalization model 400 is based on the data of the silence period in the identification registration sample claimed as described above. However, this can be derived from the complete required identification registration sample. Indeed, the normalization model comprises a single row of Cepstral coefficients, each coefficient being the average value of one column (or selected member of one column) of the Cepstral coefficients from the claimed identification model. These mean values are used to replace the mean value of each of the input data sets. That is, as an example, when test pronunciation is taken, the average value of each column of the test pronunciation Cepstral coefficients is subtracted from each individual member of the column, and the corresponding average value from the normalization model is added to each individual member of the column. Similar behavior applies to each of the claimed identification model and cohort model.

The normalization model can be derived from any of the claimed identification model, test pronunciation or cohort model. The model is preferably derived from the claimed identification model or test pronunciation. In particular, it is most desirable that the model be derived from the claimed identification model. The normalization model is derived from the "raw" registration sample Cepstral coefficients or the final model after vector quantization. That is, it can be derived at registration and stored with the registration model. Alternatively, it can be calculated on demand as part of the verification process. In general, the normalization model is preferably calculated for each registered speaker at registration and stored as part of the registered speaker database.

These normalization techniques can be used with various types of speaker recognition systems, but can be advantageously combined with the speaker recognition system of the present invention.

The speaker recognition system according to the present invention provides improved performance for several reasons. First, the modeling technique used significantly improves the separation between the true speaker and the imposter. This improved modeling makes the system less susceptible to real problems such as changes in the sound system (voice sampling system) and speaker characteristic changes (eg, due to winding). Secondly, modeling techniques are not transient in nature and are less sensitive to transient speech changes, thus providing long-term persistence of the speaker model. Third, the model can be used in variable bandwidth conditions by using filter preprocessing. For example, models built using high fidelity systems such as multimedia PCs operate on inputs received through reduced bandwidth input channels such as telephony systems.

The preferred method according to the invention is suitable for use in text independent speaker recognition systems as well as text dependent systems.

system

Accordingly, the present invention provides a basis for a flexible, reliable, simple speech recognition system that operates on a narrow or wide area basis and utilizes various communication / input channels. 16 is a broadband network operating over a local network and the Internet for authenticating a user of a database system server 400 connected to a local network 402, such as an Ethernet network, and connected to the Internet 406 via a router 404. An example of a system is shown. The speaker authentication system server 408 implementing the speaker recognition system according to the present invention is connected to a local network for the purpose of authenticating a user of the database 400. Users of the system can be directly connected to the local network 402. More generally, users at sites such as 410 and 412 are equipped with microphones to access the system through desktop or laptop computers 414 and 416 connected to other local networks that in turn are connected to the Internet 406. Other users, such as 418, 420, and 422, can access the system by dial-up modem connection through public switched telephone network 424 and Internet service provider 426.

avatar

The algorithm used by the speaker recognition system according to the present invention may be implemented as a computer program using any suitable programming language such as C or C ++, and the executable program may be an independent application on a hardware / operating system (OS) platform. It may be implemented in any required form (hardware / firmware implementation) including embedded code in the DSP chip, or may be embedded in an operating system (eg, MS Windows DLLs). Similarly, the user interface (for both system registration and subsequent system access) can be implemented in various forms, including a web-based client server system and a web browser-based interface, where the voice sampling case is, for example, ActiveX / It can be implemented using Java components.

Apart from desktop and laptop computers, the system is applicable to other terminal devices including palmtop devices, WAP enabled mobile phones, and the like, via wired and / or wireless data / mobile communication networks.

Application

The speaker recognition system with constant flexibility and reliability provided by the present invention has various applications. One particular example in accordance with aspects of the present invention provides an audit trail of a user who accesses and / or modifies digital information such as documents or database records. Such a transaction may be recorded while providing date / time and user identification related information as is known in the art. However, conventional systems do not normally verify or authenticate user identification information.

Preferably, speaker recognition using the speaker recognition system according to the present invention may be used to verify the identification information of the user when necessary, for example when opening and / or editing and / or storing a digital document, a database or the like. Can be. The document or record itself may be presented as data relating to the speaker verification procedure, or such data may be recorded in a separate audit trail while providing a validated record of access and changes to protected documents, records, and the like. Unauthorized users identified by the system will not be able to access the system, and system monitoring will prevent them from performing system related actions.

The invention includes many improvements and modifications without departing from the scope of the invention as defined in the appended claims.

Claims (33)

A method for processing speech samples to obtain a model of speech samples for use in a speaker recognition system, Dividing the speech sample into a plurality of frames; Obtaining a set of feature vectors representing each frame's smoothed frequency spectrum; Applying a clustering algorithm to the feature vectors of the frames to obtain a reduced data set representing the original speech sample, Said adjacent frames overlap by at least 80%. The method of claim 1, wherein the adjacent frames overlap by 95% or less. The method of claim 1, wherein the adjacent frames overlap by an amount ranging from 80% to 90%. 4. The method of any one of claims 1 to 3, wherein the clustering algorithm comprises a k-means algorithm or a vector quantification algorithm. The method of claim 1, wherein the set of feature vectors representing the smoothed frequency spectrum of the frame is linear predictive coding / septal (LPCC) analysis or fast Fourier transform (LPCC). Said method using a Fast Fourier Transform (FFT) or a filter bank. The method according to any one of claims 1 to 5, wherein the identification information of the speaker and the model of the voice sample are stored in a database of registration models of the speaker registered in the speaker recognition system. The method characterized in that it further comprises. 7. The method of claim 6, wherein each registration model comprises a plurality of speech sample models representing a plurality of different pronunciations. 8. The method as claimed in claim 6 or 7, wherein each registration model comprises a plurality of speech sample models representing the same pronunciation modeled using a plurality of parallel modeling processes. 9. The method of any one of claims 6 to 8, further comprising associating a model of the speech sample with a cohort that includes a predetermined number of other speakers registered with the speaker recognition system. Said method. 10. The method of claim 6, further comprising processing a second speech sample to obtain a test model of the second speech sample for testing with a database of the registration model. The second voice sample processing step Dividing the second speech sample into a plurality of frames; Acquiring a set of feature vectors representing the smoothed frequency spectrum of the frame for each of the frames. 11. The method of claim 10, wherein the feature vector set representing the smoothed frequency spectrum of the frame of the second sample is linear predictive coding / septal (LPCC) analysis or Fast Fourier Transform (Fast Fourier Transform). Or a filter bank. 12. The method of claim 10 or 11, wherein the test model includes a plurality of speech sample models representing the same pronunciation modeled using a plurality of parallel modeling processes. 13. The method of any one of claims 10 to 12, further comprising: testing the test model against a registration model for the associated cohort and the claimed identification as defined in claim 9, wherein Said identity being tested. In the speaker recognition method, A plurality of speakers recognized by the speaker recognition system are registered by storing a registration model for each speaker in a database of enrolled speakers, which registers at least one voice sample from the speaker. To indicate; The registered speakers are each associated with a cohort of a predetermined number of other registered speakers; A test speech sample from the speaker claiming to be one of the registered speakers is modeled and tested against the registration models of the associated cohort and the claimant's registration model using a classification process; And the classification process always matches the test model with the asserted speaker or the associated cohort such that a false acceptance rate of the system is determined by the size of the cohort. 15. The method of claim 14, wherein the modeling processes used to model the test speech sample and the registered speaker speech samples and / or the classification processes used to test the test model against the registration models are substantially equal to zero. Providing a false rejection rate such that the overall error rate of the system is substantially determined only by the false tolerance rate. 16. The test model according to claim 14 or 15, wherein the test model is tested using multiple parallel classification processes, and the test model is only tested if a predetermined number of the parallel classification processes produce a match with the registration model. Speaker matching method according to the registration model to reduce the false acceptance rate of the system for a given cohort size. 17. The apparatus of claim 16, wherein the registration models and the test model are each obtained using multiple parallel processes, wherein the parallel classification processes are applied to the registration speech samples and the results of the parallel modeling processes applied to the test speech sample. Speaker recognition method, characterized in that it is compared with corresponding results of the parallel model processes applied. 18. The apparatus of any of claims 8, 12 and 17, wherein the parallel modeling processes include: different frequency banding applied to the speech samples; Different spectral modeling applied to the speech samples; And at least one of different clusterings applied to feature vectors representing the speech samples. 17. The method of claim 16, wherein the parallel classification processes include testing the test model against different cohorts of registered speakers. 17. The method of claim 16, wherein the parallel classification processes include testing the test model for different pronunciations represented by the registration models. A method as claimed in any one of claims 14 to 20, wherein the registration models and the test model are obtained using the method of any one of claims 1 to 13. In the method of normalizing the speech models in the speaker recognition system of the type, Voice samples are input to the system through different input channels having different channel characteristics, Using a classification process, a test model representing a test sample is tested against a set of registration models representing speech samples from speakers registered with the system; Deriving a normalization model from one of the registration speech samples or from the test speech sample, and before testing the normalized test model against a normalized registration model, the test model and the registration model to be tested. Normalizing them using the normalization model. 23. The method of claim 22, wherein the normalization model is derived from a registered speech sample for identification claimed for the test speech sample. 24. The method of claim 23, wherein the normalization model is derived from the registration model for identification claimed for the test speech sample. 25. The apparatus of any one of claims 22 to 24, wherein the speech sample is divided into a plurality of frames; A set of feature vectors representing a smoothed frequency spectrum of each frame is obtained; Wherein said normalization model is obtained by calculating average values of sets of feature vectors from at least some of the frames of speech samples used to derive said normalization model. 26. The method of claim 25, wherein the frames used to derive the normalization model are frames corresponding to silence periods in the speech sample used to derive the normalization model. 27. The method according to claim 25 or 26, wherein the test model and the registration models are normalized by replacing the average values of the registration models and the feature vectors of the test model with corresponding average values from the normalization model. Way. 28. The method of any one of claims 22 to 27, wherein the speech samples are processed using the method of any one of claims 1 to 13. 22. The method according to any one of claims 14 to 21, wherein the test model and registration models are normalized prior to classification using the method of any one of claims 22 to 28. 30. A system as claimed in any preceding claim, wherein speech samples are input to a speaker recognition system via an input channel having a transfer function to modify the speech sample data; Estimating the transfer function of the input channel and normalizing the modified speech sample data using the inverse of the estimated transfer function. 31. A speaker recognition system comprising data processing and storage means adapted to implement the method of any one of claims 1-30. A computer program comprising a symbol code for instructing a computer to carry out the method of any one of claims 1 to 30. 31. A data carrier encoded with a computer program comprising a symbol code for instructing a computer to carry out the method of any one of claims 1-30.