JP2005091758A - System and method for speaker recognition - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To correct an inaccurate matching result due to insufficiency of speech data in registered model generation. <P>SOLUTION: The matching processing result of a 1st matching part 7 becomes better when the constitution of consonants vowels that a person to be matched utters when 1st registered model data are generated is similar to the constitution of consonants and vowels that the person to be matched utters in matching, but tends to become worse when the constitution of the consonants and vowels is different. The matching processing result of a 2nd matching part 13, at the same time, is generally not good since 2nd registered model data generated on the basis of speech data of many unspecified persons are used, but stable irrelevantly to differences between the constitution of consonants and vowels that the person to be matched utters when 2nd registered model data are generated and the constitution of consonants and vowels in matching. Consequently, the matching processing results of the 1st and 2nd matching parts are put together to perform final decision processing and then they complement each other to improve the decision precision. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to speaker recognition technology.

  The speaker recognition technology is a speaker authentication technology for registering the voice of a specific speaker in advance and determining whether or not the voice input later is the voice of the registered speaker. One of speaker identification techniques for registering voice and identifying which of the plurality of voices is most similar to the voice input later is shown. In any case, the basic processing is to calculate the similarity between the voice registered earlier and the voice inputted later.

  FIG. 1 shows an example of the prior art. The voice of the speaker is input by a voice input unit 1100 such as a microphone. The voice input unit 1100 converts voice waves that are air vibrations into electrical signals. The voice analysis unit 1102 digitizes a voice electrical signal, performs an analysis process for each analysis period (also referred to as a frame period) of about 5 ms to 30 ms in an analysis window (also referred to as a frame) of about 15 ms to 30 ms. A sequence of LPC (Linear Predictive Coding) cepstrum coefficients (vectors) is generated. Analysis processing for outputting LPC cepstrum coefficients from speech waves is well known, and is described, for example, on pages 7 to 12 of Seiichi Nakayama, “Speech Recognition by Probability Model” published by the Institute of Electronics, Information and Communication Engineers.

When the current process is speaker verification, the switching unit 1104 outputs the analysis result of the voice analysis unit 1102 to the verification unit 1108. On the other hand, when the current process is speaker registration, the switching unit 1104 The analysis result is output to the model generation unit 1106. The model generation unit 1106 models a series of LPC cepstrum coefficients (vectors), which is an analysis result of the voice analysis unit 1102. An example of the model is a multidimensional normal distribution model, and the model generation unit 1106 calculates an average vector μ and a covariance matrix Σ of LPC cepstrum coefficients (vectors) and stores them in the registered model storage unit 1110. Then, the matching unit 1108 calculates a sequence of likelihood λ in which a sequence of LPC cepstrum coefficients (vectors) of speech related to the matching target appears in the normal distribution specified by the average vector μ and the covariance matrix Σ. For speaker identification, the matching result determination unit 1112 compares the attribute value (for example, speaker ID) of the registered model having the largest overall likelihood λ _all with a threshold value for speaker authentication. It is determined whether the degree λ _all is equal to or greater than the threshold value, and the success or failure of the authentication is output.

  Japanese Patent Laid-Open No. 2002-268673 (Patent Document 1) also discloses a conventional technique as shown in FIG. That is, the audio input unit 1100 converts an audio wave that is air vibration into an electrical signal. The voice analysis unit 1102 digitizes a voice electrical signal, performs analysis processing for each frame period of about 5 ms to 30 ms in a frame of about 15 ms to 30 ms, and generates a series of LPC cepstrum coefficients (vectors), for example. When the current process is speaker verification, the switching unit 1104 outputs the analysis result of the voice analysis unit 1102 to the verification unit 1108. On the other hand, when the current process is speaker registration, the switching unit 1104 The analysis result is output to the model generation unit 1106. The model generation unit 1106 models a series of LPC cepstrum coefficients (vectors), which is the analysis result of the voice analysis unit 1102, and stores it in the registered model storage unit 1110.

Then, collation section 1108 calculates a sequence of likelihood λ in which a sequence of LPC cepstrum coefficients (vectors) of speech related to the collation target appears in the normal distribution specified by mean vector μ and covariance matrix Σ. However, when the likelihood that is the result of the matching process is less than a predetermined threshold within a predetermined time (a time corresponding to about one syllable), a process of reducing (for example, removing) the influence of the matching process result is performed. A matching result correction unit 1209 is provided. For speaker identification, for example, the verification result determination unit 1211 uses the attribute value (for example, speaker ID) of the registered model having the largest overall likelihood λ _all after being corrected by the verification result correction unit 1209 as the speaker authentication. If so, it is determined whether or not the overall likelihood λ _all after being corrected by the matching result correction unit 1209 compared to the threshold is equal to or greater than the threshold, and the success or failure of the authentication is output.
JP 2002-268673 A

  When such speaker recognition technology is adopted, a speaker may utter many phonemes and a registration model may be created by the model generation unit 1106. However, a registration model is not necessarily created based on a sufficient number of phonemes. I don't mean. If a registered model is created without a phoneme being uttered, and a phoneme that was not uttered at the time of creating the registered model is pronounced by the speaker during the matching process, the matching result for that phoneme will be significantly worsened.

  Patent Document 1 has been proposed to deal with the above problem, but it is assumed that if the likelihood λ is less than a predetermined threshold within a predetermined time, a phoneme that is insufficient in the registered model has been uttered. Therefore, the correction by the collation result correction unit 1209 may not always be a correct correction.

  Therefore, an object of the present invention is to provide a novel technique for correcting an inaccurate collation result caused by lack of voice data when creating a registration model.

  The speaker recognition system according to the present invention includes a first registration model data storage unit that stores first registration model data generated from voice data of a person to be collated, and voice data of a large number of unspecified speakers. A second registered model data storage unit for storing second registered model data generated by adapting the determined unspecified speaker model data to the verification target person, and analyzing the voice data of the verification target person Analyzing means for generating voice analysis data, first matching processing means for performing matching processing using the voice analysis data and the first registered model data stored in the first registered model / data storage unit, and voice Second collation processing means for performing collation processing using the analysis data and the second registration model data stored in the second registration model / data storage unit, and the first collation processing means and the second collation processing means. Based on the processing results, and a judging means for performing final determination processing for the collation object person.

  The collation processing result by the first collation processing means is similar in the configuration of the consonant vowel uttered by the person to be collated when generating the first registered model data and the structure of the consonant vowel uttered by the person to be collated during the collation. This is better in some cases, but tends to be worse if the consonant vowel composition is significantly different. On the other hand, the result of the collation processing by the second collation processing means is generally not very good, but the difference between the consonant vowel configuration uttered by the person to be collated when generating the second registered model data and the consonant vowel configuration at the time of collation It will be stable regardless of. Accordingly, if the final determination process is performed by combining the results of the verification processing by the first and second verification processing means, the results are complemented with each other and the determination accuracy is improved. Note that the final determination processing is determination of success or failure in the case of speaker authentication, and determination of who is the person to be verified in the case of speaker identification.

  Note that the determination means described above is a product of the first likelihood that is the result of the collation processing of the first collation processing means and (1-α) (α is a predetermined real number between 0 and 1), and the second Based on the value obtained by adding the product of the second likelihood and α, which is the result of the collation processing by the collation processing means, the final determination process for the person to be collated may be performed. In this way, the accuracy of determination can be improved by blending the verification processing results by the first and second verification processing means.

  Further, the first registered model data and the second registered model data described above are data of a mixed normal distribution model (for example, GMM (Gaussian Mixture Model)), and collation processing and second collation processing by the first collation processing means. In some cases, the matching process by the means is a matching process corresponding to the mixed normal distribution model. In this way, collation can be performed even in a state where the content (also referred to as text) uttered by the person to be collated is not specified.

  Further, the first registered model data is mixed normal distribution model data, the second registered model data is subword unit (for example, syllable) model data (for example, HMM (Hidden Marcov Model)), and the first matching processing means The collation process according to the above is a collation process corresponding to the mixed normal distribution model, and the second collation processing means connects the model data in units of subwords stored in the second registered model data storage unit, and the collation model data May be included, and a means for performing collation processing using the collation model data and the voice analysis data may be included.

  The first collation processing unit and the second collation processing unit do not necessarily have to perform the same type of processing. When the second registered model data is used as model data in subword units as described above, model data for verification is generated by connecting the model data in subword units in the second verification processing unit as described above. Then, the verification process is performed.

  Note that the present invention further includes means for determining a phrase (also referred to as text) for requesting utterance from the person to be collated, and the collation model data generating means described above is configured to register the second registered model data according to the phrase. Model data for collation may be generated by connecting model data in units of subwords stored in the storage unit. If it is a method of designating a phrase to be uttered by a person to be collated at the time of collation, it is possible to counter the person who records the voice of a genuine speaker and impersonates the person. In the present application, model data for sub-word units can be connected according to the specified phrase to generate collation model data, so that it is possible to deal with the above-mentioned impersonators.

  Further, the present invention provides means for generating first registered model data from voice analysis data of a person to be collated generated by the analysis means at the time of model data registration, and means generated by the analysis means at the time of model data registration. Second registered model data generation for generating second registered model data by adapting unspecified speaker model data stored in the unspecified speaker model data storage unit using the voice analysis data of the person to be verified And a means. The second registered model data generation means described above performs a process of adapting the model data of the subword uttered by the person to be collated at the time of model data registration according to a predetermined method, and is adapted. Alternatively, the second registered model data may be generated by connecting the model data in units of subwords. In some cases, model data in units of subwords is connected at the time of collation, and in other cases, connection is made at the time of registration.

  The speaker recognition system according to the present invention can be realized by a combination of a program and a computer. In this case, the program is, for example, a flexible disk, a CD-ROM, a magneto-optical disk, a semiconductor memory, a hard disk, or the like. It is stored in a storage medium or a storage device. The program may be distributed as a digital signal via a network. Note that data being processed is temporarily stored in the memory of the computer.

  According to the present invention, it is possible to appropriately correct an inaccurate collation result caused by lack of voice data when creating a registration model.

  FIG. 3 shows a functional block diagram of the speaker recognition system according to the embodiment of the present invention. The speaker recognition system according to the present embodiment includes a voice input unit 1, a voice analysis unit 3, a switching unit 5, a first verification unit 7, a model generation unit 9, and a first registered model storage unit 11. The second collation unit 13, the model correction unit 15, the second registered model storage unit 17, the collation result determination unit 19, the prior model storage unit 21, and the utterance text determination unit 25 are included. In order to generate data to be stored in the prior model storage unit 21, a preprocessing unit 23 including a prior speech data storage unit 231, a second speech analysis unit 233, and a prior model generation unit 235 is required. Not required for processing or model registration. That is, the pre-processing unit 23 may not be included in the speaker recognition system.

  The output of the voice input unit 1 is input to the voice analysis unit 3. The output of the voice analysis unit 3 is input to the switching unit 5. The output of the switching unit 5 is input to the first verification unit 7 and the second verification unit 13 during the speaker verification process, and is input to the model generation unit 9 and the model correction unit 15 during the model registration process. The first registered model data generated by the model generating unit 9 is stored in the first registered model storage unit 11. The first verification unit 7 can refer to the first registered model storage unit 7, and its output is input to the verification result determination unit 19. On the other hand, the model correction unit 15 performs an adaptation process on the advance model stored in the advance model storage unit 21 based on the output from the speech analysis unit 3, and the second registration model storage unit 17 as the second registration model data. To store. The second verification unit 13 can refer to the second registered model storage unit 17, and its output is input to the verification result determination unit 19. The verification result determination unit 19 outputs a final verification result based on the outputs from the first verification unit 7 and the second verification unit 13. In this embodiment, both speaker identification and speaker authentication can be performed by the same process, and information indicating who is the speaker is the final verification result of speaker identification. (Speaker ID, etc.) is output, and if it is the final verification result of speaker authentication, information indicating whether the authentication has succeeded or failed is output.

  The utterance text determination unit 25 determines the word / phrase when the speaker needs to determine the word / phrase to be uttered, and the data of the determined word / phrase and the second collation unit 13 and an output device (not shown, for example) Output to a device or a voice conversion processing unit and a speaker). Note that the data may be output to the model correction unit 15.

  The pre-voice data storage unit 231 in the pre-processing unit 23 stores a large number of digitized voice data of unspecified speakers. Then, the second speech analysis unit 233 processes the speech data stored in the prior speech data storage unit 231 and outputs the processing result to the prior model generation unit 235. The output of the advance model generation unit 235 is stored in the advance model storage unit 21. This prior model storage unit 21 is included in the speaker recognition system.

  Hereinafter, the processing contents of the speaker recognition system and the preprocessing unit 23 shown in FIG.

1. Embodiment 1
In the present embodiment, mixed normal distribution model (GMM) data is stored in the first registered model storage unit 11 and the second registered model storage unit 17, and mixed in the first matching unit 7 and the second matching unit 13. A matching process based on a normal distribution model (GMM) is performed.

First, what kind of processing is performed in the pre-processing unit 23 will be described with reference to FIG. The voice data storage unit 231 of the pre-processing unit 23 stores voice data (for example, digital data) by many unspecified speakers. Note that the speech data of many unspecified speakers includes the speech data of all consonant vowels. Therefore, the second speech analysis unit 233 reads the pre-speech data stored in the pre-speech data storage unit 231 and performs speech analysis for each frame to generate speech analysis data (step S1). More specifically, analysis processing is performed for each analysis period (frame period) of about 5 ms to 30 ms with an analysis window (frame) of about 15 ms to 30 ms, and for example, a series of LPC cepstrum coefficients (vectors) is generated. As shown in FIG. 5, the analysis window is set to be shifted for the analysis period for the sound wave, a predetermined analysis process is performed for each analysis window, and a cepstrum coefficient C _ij corresponding to the analysis window is output. For example, an LPC cepstrum coefficient of about 10 to 20 (dimensions) is calculated by one analysis process. Here, i is a frame number, i = 1 to N, and N is the total number of frames. j is a dimension number of the LPC cepstrum coefficient, j = 1 to n, and n is the number of dimensions. The LPC cepstrum coefficient obtained by the i-th analysis process becomes a feature vector X _i when expressed as follows.
X _i = (C _i1 , C _i2 ,... C _in ) ^T (1)
Such a process is performed for all audio data stored in the pre-audio data storage unit 231. The processing result is stored in the storage device.

ここでｘ_tは（１）式と同様に表される照合時のｎ次元特徴ベクトルである。
（２）式のように、ＧＭＭはｎ次元Ｍ混合のガウス分布Ｎ（ｘ_t|μ_sm,Σ_sm）を重みｗ_smで線形結合した確率モデルとなる。このＮ（ｘ_t|μ_sm,Σ_sm）は、以下のように表される。

Next, the prior model generation unit 235 performs processing for generating a mixed normal distribution model (GMM) for speech data by a large number of unspecified speakers stored in the prior speech data storage unit 231, and the processing result Are stored in the prior model storage unit 21 as prior model data (step S3). The mixed normal distribution of the speaker λ _s model is expressed by the following equation. However, here, the speaker λ _s is all of many unspecified speakers.

Here, x _t is an n-dimensional feature vector at the time of collation expressed in the same manner as the equation (1).
As shown in the equation (2), the GMM is a probability model in which an n-dimensional M-mixed Gaussian distribution N (x _t | μ _sm , Σ _sm ) is linearly combined with a weight w _sm . This N (x _t | μ _sm , Σ _sm ) is expressed as follows.

Here, μ _sm is M average vectors calculated from the feature vector X _t when the speaker model λ _s is registered. The average vector μ _sm is generated from the feature vector X _t by vector quantization or maximum likelihood estimation. As for whether the feature vector X _t is associated with any of the mean vector mu _sm, it can be determined by finding the closest mean vector mu _sm for each feature vector X _t.

平均ベクトルμ_smと共分散行列Σ_smとについては、以下同様に算出される。 Σ _sm represents the covariance matrix of the speaker model λ _s . That is, it is as follows. Note that M covariance matrices Σ _sm are obtained from the feature vector X _t related to the average vector μ _sm .

The average vector μ _sm and the covariance matrix Σ _sm are calculated in the same manner.

Furthermore, the weight w _{sm of the} mixture distribution has the following relationship.

However, since each w _sm cannot be determined analytically, for example, w _sm is determined by a known EM algorithm or the like so that the following expression becomes maximum.

Thus, in order to calculate the expressions (2) and (3), M μ _sm , M Σ _sm and M weights (strictly M−1 from the expression (7)) are used. w _sm is required, and these data become the pre-model data.

  Next, the processing flow of the speaker recognition system in the present embodiment will be described with reference to FIG. Here, a processing flow in the case of speaker authentication will be described. First, a process selection input for designating whether collation or registration is performed and input of speaker identification information (for example, speaker ID) are received from the speaker (step S11).

Next, the voice of the speaker is input via the voice input unit 1 such as a microphone (step S13). The voice input unit 1 converts a voice wave that is air vibration into an electric signal. Next, the voice analysis unit 3 digitizes the voice electrical signal, performs voice analysis for each frame of about 5 ms to 30 ms, and analyzes voice analysis data (for example, a sequence of LPC cepstrum coefficients) in an analysis window of about 15 ms to 30 ms. C _ij ) is generated (step S15). That is, feature vectors x _i are generated for the number of frames. The generated data is stored in a storage device (not shown).

Then, the switching unit 5 determines whether or not the process selection input received in step S11 is collation (step S17). When the process selection input is registration rather than verification (step S17: No route), the model generation unit 9 generates first registration model data for the input voice of the speaker, and corresponds to the speaker ID. It registers in the 1st registration model storage part 11 (step S19). The process of the model generation unit 9 is almost the same as the process of the prior model generation unit 235. That is, M average vectors μ _sm of feature vectors x _i that are speech analysis data are calculated, and M covariance matrices Σ _sm are calculated according to the equation (6). Further, for example, the weight w _sm is calculated so as to maximize the expression (8). The data calculated in this way is registered in the first registration model storage unit 11.

Further, the model correction unit 15 corrects the prior model based on the voice analysis data of the speaker's input voice, generates second registration model data, and stores it in the second registration model storage unit 17 (step S21). Specifically, an average vector of feature vectors stored in the prior model storage unit 21 is μ ₀ (each of M average vectors μ), and feature vectors x _i (1 ≦ i ≦ 1) that are speech analysis data. N) and the constant β are used to calculate the average vector μ _a of the feature vectors in the second registered model using the following equation.

In equation (9), the weight of the average vector μ ₀ in the prior model is determined by a constant β. Since this constant β depends on the environment, an appropriate value is determined experimentally. The covariance matrix Σ and the weight w included in the prior model may be adapted to the speaker using the speech analysis data of the input speech, but in this embodiment, only the average vector μ ₀ is adapted to the speaker. Make it. Therefore, M average vectors μ _a calculated by the equation (9) as the second registration model, M covariance matrices Σ and M (or M−1) weights w included in the prior model, Is registered in the second registration model storage unit 17 in correspondence with the speaker ID. Then, the process ends.

On the other hand, when the process selection input of the speaker is collation (step S17: Yes route), the first collation unit 7 reads the first registration model data corresponding to the speaker ID from the first registration model storage unit 11, A matching process is further performed using the feature vector x _i (1 ≦ i ≦ N), which is speech analysis data (step S23). That is, for each feature vector, P (x _t | λ _s ) and log likelihood logP (x _t | λ _s ) are calculated according to equations (2) and (3). Furthermore, the log likelihood total L1 is calculated according to the equation (8). The calculation result is stored in the storage device.

Further, the second collation unit 13 reads out the second registration model data corresponding to the speaker ID from the second registration model storage unit 17, and further extracts the feature vector x _i (1 ≦ i ≦ N) that is the voice analysis data. The collation process is performed using them (step S25). Similar to step S23, P (x _t | λ _s ) and log likelihood logP (x _t | λ _s ) are calculated for each feature vector using equations (2) and (3). Furthermore, the log likelihood total L2 is calculated according to the equation (8). Since the second registration model data is different from the first registration model data, the calculation results in step S23 and step S25 are different. The calculation result is stored in the storage device.

And the collation result determination part 19 implements a determination process using the two collation process results of step S23 and step S25, and outputs a determination process result (step S27). Here, the total likelihood L is calculated by adding the likelihoods that are two collation processing results according to the following equation.
L = L1 × (1−α) + L2 × α (10)
However, 0 ≦ α ≦ 1. Further, the optimum value of α is experimentally obtained so that the determination accuracy is improved. It has been found that 0.9 to 0.95 gives good results, depending on other experimental conditions.

  Then, by determining whether or not the total likelihood L exceeds a predetermined threshold value, it is determined whether the current speaker authentication has succeeded or failed. In this case, information indicating the success or failure of the authentication is output as the determination processing result.

  When the first registered model data is generated, it is sufficient that the speaker utters many consonant vowels. However, since the speaker is actually burdened, many consonant vowels are often not uttered. Accordingly, the likelihood calculated by the first matching unit 7 is similar to the configuration of the consonant vowels uttered by the speaker at the time of matching with the configuration of the consonant vowels uttered by the speaker when generating the first registered model data. It tends to get worse if the composition of consonant vowels is significantly different. On the other hand, the likelihood calculated by the second collation unit 13 is generally not very good, but the configuration of the consonant vowels spoken by the speaker when generating the second registered model data and the configuration of the consonant vowels at the time of collation It will be stable regardless of the difference. Therefore, as described above, if the final determination process is performed by combining the two collation processing results, the results are complemented to improve the determination accuracy.

  In the present embodiment, an example is shown in which both the first matching unit 7 and the second matching unit 13 perform a matching process for a text independent method in which the content of speech uttered at the time of registration or matching is not limited.

As a precaution, a simplified processing flow for speaker identification will be described with reference to FIG. First, the voice of the speaker is input via the voice input unit 1 such as a microphone (step S31). The voice input unit 1 converts a voice wave that is air vibration into an electric signal. Next, the voice analysis unit 3 digitizes the voice electrical signal, performs voice analysis for each frame of about 5 ms to 30 ms, and analyzes voice analysis data (for example, a sequence of LPC cepstrum coefficients) in an analysis window of about 15 ms to 30 ms. C _ij ) is generated (step S33). That is, feature vectors x _i are generated for the number of frames. The generated data is stored in a storage device (not shown). Since the description in the case of registration is omitted here, the switching unit 5 outputs the feature vectors x _i for the number of frames to the first matching unit 7 and the second matching unit 13.

The first collation unit 7 sequentially reads the first registration model data of each speaker ID from the first registration model storage unit 11 and collates it with the feature vector x _i (1 ≦ i ≦ N) which is voice analysis data. Processing is performed (step S35). That is, for each speaker ID, P (x _t | λ _s ) and log likelihood logP (x _t | λ _s ) are calculated for each feature vector using equations (2) and (3). Further, the sum of log likelihoods L1 is calculated for each speaker ID according to the equation (8). The calculation result is stored in the storage device.

Further, the second collation unit 13 sequentially reads out the second registration model data of each speaker ID from the second registration model storage unit 17, and for the feature vector x _i (1 ≦ i ≦ N) that is the voice analysis data. The collation process is performed (step S37). Similarly to step S35, P (x _t | λ _s ) and log likelihood logP (x _t | λ _s ) are calculated for each feature vector using the equations (2) and (3) for each speaker ID. . Further, the sum L2 of logarithmic likelihoods is calculated for each speaker ID according to the equation (8). The calculation result is stored in the storage device.

  And the collation result determination part 19 calculates total likelihood for every speaker ID using the two collation process results of step S35 and step S37, and stores it in a memory | storage device (step S39). Here, the likelihoods L1 and L2 which are two collation processing results are added according to the equation (10), and the total likelihood L is calculated for each speaker ID.

話者ＩＤがｓとして出力される。なお、ここでは総合尤度Ｌが１／Ｎされているが、しなくともよい。 And the collation result determination part 19 outputs speaker ID etc. with this highest total likelihood L as a final determination result (step S41). This can be expressed as follows:

The speaker ID is output as s. Here, the overall likelihood L is 1 / N, but it is not necessary.

  In this way, speaker identification processing can be performed. Except for step S39, which is the final step, the number of verification processes is only the number of speaker IDs, and there is no essential difference from the case of speaker authentication processing. Therefore, the determination accuracy is improved because the total likelihood is calculated and determined as in equation (8).

2. Embodiment 2
Next, model data in units of subwords (for example, syllables or phonemes) instead of GMM is adopted as the second registered model data, and a model for verification is generated by connecting the models in units of subwords in the second verification unit 13. In addition, a description will be given of processing in the case where text independent verification processing is performed.

First, processing contents of the preprocessing unit 23 in the present embodiment will be described with reference to FIGS. 4, 8, and 9. The voice data storage unit 231 of the pre-processing unit 23 stores voice data (for example, digital data) by many unspecified speakers. Note that the speech data of many unspecified speakers includes the speech data of all consonant vowels. Therefore, the second speech analysis unit 233 reads the pre-speech data stored in the pre-speech data storage unit 231 and performs speech analysis for each frame to generate speech analysis data (step S1). More specifically, an analysis process is performed for each analysis period (frame period) of about 5 ms to 30 ms with an analysis window (frame) of about 15 ms to 30 ms, and for example, a sequence of LPC cepstrum coefficients (feature vectors) is generated. . Here, the feature vector X _i is managed for each syllable. Such a process is performed for all audio data stored in the pre-audio data storage unit 231. The processing result is stored in the storage device.

  Next, the prior model generation unit 235 generates a hidden Markov model (HMM: Hidden Marcov Model) for speech data by many unspecified speakers stored in the prior speech data storage unit 231 for each syllable. And the processing result is stored in the advance model storage unit 21 as advance model data (step S3).

An example of the structure of the HMM is shown in FIG. The HMM is composed of a plurality of states 801 to 805 (here, J states S _{0 to} S _J-1 ) and transitions between the states (arrows connecting the states). It is assumed that the state transitions once every time one feature vector X _i of the input speech is output. Here, the probability a _kl of transition from the state S _k to S _l is expressed as follows.
a _kl = P (s _l = S _l | s _l-1 = S _k ) (11)

Further, the probability b _kl that the feature vector x is output when transitioning from the state S _k to S _l is expressed as follows.
b _kl = P (x | s _l = S _l , s _l-1 = S _k ) (12)
Note that b _kl is expressed by equation (2).

すなわち、Ｓ₀乃至Ｓ_J-1までの状態遷移パターンＳ毎に、その状態遷移パターンに従って（１１）式及び（１２）式の積を全部掛けて得られる値のうち最も大きい値をＰ（Ｘ｜Ｗ）とするものである。状態Ｓ₀からＳ_J-1まで遷移する間に特徴ベクトルＸ₀乃至Ｘ_T-1が生成されるため、状態遷移パターンＳは、図９に示すように、左下のポイント９０１と右上のポイント９０２とを水平方向と斜め方向の線分のみを用いて接続することのできる１又は複数のパターンである。図９では点線で表されたパターン９０３と実線で表されたパターン９０４の２つのパターンのみ示されているが、実際には多くのパターンが存在している。 From such a model W, a series of feature vectors X = {X ₀ , X ₁ ,. . . X _i,. . . The probability that _XT-1 } is output is expressed by the following equation.

That is, for each state transition pattern S from S _{0 to} S _J−1 , the largest value among the values obtained by multiplying all products of Equations (11) and (12) according to the state transition pattern is P (X | W). Since the feature vectors X _{0 to} X _T-1 are generated during the transition from the state S ₀ to S _J−1 , the state transition pattern S includes the lower left point 901 and the upper right point 902 as shown in FIG. Can be connected using only horizontal and diagonal line segments. In FIG. 9, only two patterns, a pattern 903 represented by a dotted line and a pattern 904 represented by a solid line, are shown, but there are actually many patterns.

In step S3, for each syllable, the value of equation (13) is maximized so that _{akl in} equation (11) and equation (2) (refer to equations (2) to (2)). The parameter of weight w _{sm in} is determined by, for example, a well-known EM algorithm. In addition, for each syllable, M average vectors μ _sm and M covariance matrices Σ _sm of the feature vector X _i are also calculated. The a _kl , M (or M−1) weights w _sm , M average vectors μ _sm of feature vectors, and covariance matrix Σ _sm for each consonant vowel obtained in this way are used as prior model data. It is stored in the prior model storage unit 21.

  By preparing the model data in units of subwords such as syllables in this way, the model correction unit 15 can appropriately adapt to the speaker.

  Next, the processing flow of the speaker recognition system in the present embodiment will be described with reference to FIG. Here, a processing flow in the case of speaker authentication will be described. First, a process selection input for designating whether collation or registration is performed and input of speaker identification information (for example, speaker ID) are received from the speaker (step S51).

Next, the voice of the speaker is input via the voice input unit 1 such as a microphone (step S53). The voice input unit 1 converts a voice wave that is air vibration into an electric signal. Next, the voice analysis unit 3 digitizes the voice electrical signal, performs voice analysis for each frame of about 5 ms to 30 ms, and analyzes voice analysis data (for example, a sequence of LPC cepstrum coefficients) in an analysis window of about 15 ms to 30 ms. C _ij ) is generated (step S55). That is, feature vectors x _i are generated for the number of frames. The generated data is stored in a storage device (not shown).

Then, the switching unit 5 determines whether or not the process selection input received in step S51 is collation (step S57). If the process selection input is registration rather than collation (step S57: No route), the model generation unit 9 generates first registration model data for the input voice of the speaker, and corresponds to the speaker ID. Register in the first registration model storage unit 11 (step S59). The process of the model generation unit 9 is almost the same as the process of the prior model generation unit 235 in the first embodiment. That is, M average vectors μ _sm of feature vectors x _i that are speech analysis data are calculated, and M covariance matrices Σ _sm are calculated according to the equation (6). Further, for example, the weight w _sm is calculated so as to maximize the expression (8). The data calculated in this way is registered in the first registration model storage unit 11.

Further, the model correction unit 15 corrects the prior model based on the voice analysis data of the speaker's input voice, generates second registration model data, and stores it in the second registration model storage unit 17 (step S61). Specifically, in the syllable unit of the speech input this time, the equation (13) is calculated for all the pre-model data of the syllable unit stored in the pre-model storage unit 21, and the syllable with the highest probability is calculated. Identify. Then, the average vector of the feature vectors included in the prior model data of the identified syllable is μ ₀ (each of the M average vectors μ), and the feature vector x _i (1 ≦ i) that is the speech analysis data of the input speech ≦ N) and the constant β are used to calculate M average vectors μ _a of the feature vectors in the second registered model according to equation (9).

In equation (9), the weight of the average vector μ ₀ in the prior model is determined by a constant β. An appropriate value for this constant β is determined experimentally. The covariance matrix Σ and the weight w included in the prior model may be adapted to the speaker using the speech analysis data of the input speech, but in this embodiment, only the average vector μ ₀ is adapted to the speaker. Make it.

As described above, for each syllable of the input speech, M average vectors μ _a calculated by the equation (9) as the second registration model, and M covariance matrices Σ and M (or M) included in the prior model. -1) weight w is registered in the second registration model storage unit 17 in correspondence with the speaker ID. Further, for the consonant vowels that are not included in the input speech, the prior model data is directly registered in the second registration model storage unit 17 as the second registration model data corresponding to the speaker ID.

On the other hand, when the process selection input of the speaker is collation (step S57: Yes route), the first collation unit 7 reads the first registration model data corresponding to the speaker ID from the first registration model storage unit 11, A matching process is further performed using the feature vector x _i (1 ≦ i ≦ N), which is speech analysis data (step S63). That is, for each feature vector, P (x _t | λ _s ) and log likelihood logP (x _t | λ _s ) are calculated according to equations (2) and (3). Furthermore, the log likelihood total L1 is calculated according to the equation (8). The calculation result is stored in the storage device.

  Further, the second collation unit 13 reads out the second registration model data corresponding to the speaker ID from the second registration model storage unit 17 and configures a collation model (step S65). In the present embodiment, the second collating unit 13 also adopts the text independent method, and therefore, for example, a syllable model is connected as shown in FIG. That is, the state 211 after transition from the start is shared by all syllable models, and all the syllable models 212 to 215 are connected in parallel. The state 216 before transitioning to the end is also shared by all syllable models. Further, a state transition 217 for returning from the state 216 to the state 211 is set. In other words, for each syllable of the input speech, all syllable models are collated, and the output from the syllable model with the highest probability is adopted. This is repeated until the last syllable of the input speech.

  And the 2nd collation part 13 implements collation processing using a collation model as shown in FIG. 11 (step S67). More specifically, the second registered model data is obtained by using the feature vector that is the voice analysis data related to the first syllable of the input speech and the model data related to all syllables included in the second registered model data. Probabilities are calculated for all included syllables according to equation (13). And the probability about the syllable from which the maximum probability was calculated is hold | maintained in a memory | storage device, for example. The model data related to the syllable next to the input speech is similarly calculated according to the equation (13), and the probability for the syllable for which the maximum probability is calculated is held in a storage device, for example. In this way, the processing as described above is repeated until the last syllable of the input speech, and all the probabilities held in the storage device are finally multiplied, and the calculated value is set as the likelihood L2. However, there are cases where the logarithms of the probabilities held in the storage device are calculated and the sum of these is used as the likelihood L2. The calculation result is stored in the storage device.

  And the collation result determination part 19 implements a determination process using the two collation process results of step S63 and step S67, and outputs a determination process result (step S69). Here, the total likelihood L is calculated by adding the likelihoods that are two collation processing results according to the equation (10).

  Then, by determining whether or not the total likelihood L exceeds a predetermined threshold value, it is determined whether the current speaker authentication has succeeded or failed. In this case, information indicating the success or failure of the authentication is output as the determination processing result.

  The present embodiment differs from the first embodiment in the contents of the second registered model data and the processing contents of the second collation unit 13, but the final result is obtained by combining the two collation processing results as in the first embodiment. Since the determination process is performed, the determination accuracy is improved because they are mutually supplemented.

  The speaker identification process can be implemented by replacing step S37 in FIG. 7 with steps S65 and S67 in FIG. 10 for all second registered model data. Therefore, the speaker identification accuracy of the speaker identification process is also improved.

3. Embodiment 3
Next, instead of GMM, model data in units of subwords (eg, syllables) is adopted as the second registered model data, and a model for verification is generated by connecting the models in units of subwords according to the designated text in the second verification unit 13 In addition, a description will be given of processing in the case of performing text-dependent collation processing. Note that the text dependence is a method of limiting the text to be uttered by the speaker at the time of collation or registration.

  Since the processing of the pre-processing unit 23 is the same as that described in the second embodiment, the description thereof is omitted here.

Next, the processing flow of the speaker recognition system in the present embodiment will be described with reference to FIG. Here, a processing flow in the case of speaker authentication will be described. First, a process selection input for designating whether collation or registration is performed and input of speaker identification information (for example, speaker ID) are received from the speaker (step S71). If registration is selected instead of collation by the speaker (step S73: No route), the voice of the speaker is input via the voice input unit 1 such as a microphone (step S75). The voice input unit 1 converts a voice wave that is air vibration into an electric signal. Note that the switching unit 5 switches the output destination of the voice analysis data to the model generation unit 9 and the model correction unit 15 at this stage. Next, the voice analysis unit 3 digitizes the voice electrical signal, performs voice analysis for each frame of about 5 ms to 30 ms, and analyzes voice analysis data (for example, a sequence of LPC cepstrum coefficients) in an analysis window of about 15 ms to 30 ms. C _ij ) is generated (step S77). That is, feature vectors x _i are generated for the number of frames. The generated data is stored in a storage device (not shown).

And the model production | generation part 9 produces | generates the 1st registration model data with respect to a speaker's input audio | voice, and registers it in the 1st registration model storage part 11 corresponding to a speaker ID (step S79). The process of the model generation unit 9 is almost the same as the process of the prior model generation unit 235 in the first embodiment. That is, M average vectors μ _sm of feature vectors x _i that are speech analysis data are calculated, and M covariance matrices Σ _sm are calculated according to the equation (6). Further, for example, M (or M−1) weights w _sm are calculated so as to maximize the expression (8). The data calculated in this way is registered in the first registration model storage unit 11.

Further, the model correction unit 15 corrects the prior model based on the voice analysis data of the speaker's input voice, generates second registration model data, and stores it in the second registration model storage unit 17 (step S81). Specifically, in the syllable unit of the speech input this time, the equation (13) is calculated for all the pre-model data of the syllable unit stored in the pre-model storage unit 21, and the syllable with the highest probability is calculated. Identify. Then, an average vector of feature vectors included in the pre-model data of the identified syllable is μ ₀ (each of M average vectors), and a feature vector x _i (1 ≦ i ≦ 1) that is speech analysis data of the input speech. N) and the constant β are used to calculate M average vectors μ _a of the feature vectors in the second registered model according to equation (9). The covariance matrix Σ and the weight w included in the prior model may be adapted to the speaker using the speech analysis data of the input speech, but in this embodiment, only the average vector μ ₀ is adapted to the speaker. Make it.

In this way, for the syllable of the input speech, M average vectors μ _a calculated by the equation (9) as the second registration model, and M covariance matrices Σ and M (M−1) included in the prior model. Weight) w is registered in the second registration model storage unit 17 in correspondence with the speaker ID. Further, for the consonant vowels that are not included in the input speech, the prior model data is directly registered in the second registration model storage unit 17 as the second registration model data corresponding to the speaker ID.

On the other hand, when the process selection input of the speaker is collation (step S73: Yes route), the utterance text determination unit 25 determines the utterance text (phrase) for requesting the speaker to utter, and displays a display device or voice (not shown). The data is output via a conversion device and a speaker (step S83). Then, the voice of the speaker for the designated utterance text is input via the voice input unit 1 such as a microphone (step S85). The voice input unit 1 converts a voice wave that is air vibration into an electric signal. Next, the voice analysis unit 3 digitizes the voice electrical signal, performs voice analysis for each frame of about 5 ms to 30 ms, and analyzes voice analysis data (for example, a sequence of LPC cepstrum coefficients) in an analysis window of about 15 ms to 30 ms. C _ij ) is generated (step S87). That is, feature vectors x _i are generated for the number of frames. The generated data is stored in a storage device (not shown).

Then, the first collation unit 7 reads the first registration model data corresponding to the speaker ID from the first registration model storage unit 11, and further extracts the feature vector x _i (1 ≦ i ≦ N) that is the voice analysis data. The collation process is performed using them (step S89). That is, for each feature vector, P (x _t | λ _s ) and log likelihood logP (x _t | λ _s ) are calculated according to equations (2) and (3). Furthermore, the log likelihood total L1 is calculated according to the equation (8). The calculation result is stored in the storage device.

  Further, the second collation unit 13 reads out the second registration model data corresponding to the speaker ID from the second registration model storage unit 17, and configures a collation model corresponding to the utterance text (step S91). In the present embodiment, the second collating unit 13 employs a text-dependent system, and therefore, for example, syllable models are connected as shown in FIGS. 13 (a) and 13 (b). Here, since the text for utterance is “Asahi”, model data “a”, “sa”, and “hi” are read from the second registered model data as shown in FIG. The model can be connected by replacing the last state of each syllable with the first state of the next syllable except for the model of the last syllable as shown in FIG. That is, the models are connected so that a significant probability (likelihood) is calculated only when the speaker utters Asahi.

  And the 2nd collation part 13 implements collation processing using a collation model as shown in FIG. 13 (step S93). More specifically, the probability is calculated according to the equation (13) from the feature vector that is the voice analysis data related to the input voice and the syllable model data included in the utterance text. Let the calculated value be the likelihood L2. The calculation result is stored in the storage device.

  And the collation result determination part 19 implements a determination process using the two collation process results of step S89 and step S93, and outputs a determination process result (step S95). Here, the total likelihood L is calculated by adding the likelihoods that are two collation processing results according to the equation (10).

  Then, by determining whether or not the total likelihood L exceeds a predetermined threshold value, it is determined whether the current speaker authentication has succeeded or failed. In this case, information indicating the success or failure of the authentication is output as the determination processing result.

  The present embodiment differs from the first embodiment in the contents of the second registered model data and the processing contents of the second collation unit 13, but the final result is obtained by combining the two collation processing results as in the first embodiment. Since the determination process is performed, the determination accuracy is improved because they are mutually supplemented. In addition, since the text-dependent method is employed in connection with the second collating unit 13, it is possible to counter an impersonator who uses a voice recording of a genuine speaker, for example.

  The speaker identification process can be implemented by replacing step S37 in FIG. 7 with steps S91 and S93 in FIG. 12 for the second registered model data for the text for utterance. Therefore, the speaker identification accuracy of the speaker identification process is also improved.

4). Other Embodiments (1) Model Correction Unit 15
The above shows an example (maximum posterior probability estimation method MAP) in which the average vector μ of the prior model is adapted to the speaker with the weight β, but the maximum likelihood linear regression method (MLLR) may be used.

(2) Text Dependent Method In the third embodiment, an example is shown in which a speaker speaks freely during speaker registration and a text for utterance is specified during speaker verification. The utterance text may be specified at the time of the speaker verification, and the same utterance text may be specified at the time of speaker verification. In this case, the model correction unit 15 performs the processing up to the configuration of the verification model performed by the second verification unit 13 in the third embodiment, and stores it in the second registered model storage unit 17.

  Although the embodiments of the present invention have been described above, the present invention is not limited to these. For example, although it is the functional block diagram shown in FIG. 3, a program module is not necessarily comprised corresponding to this.

The functional block diagram of the 1st prior art is shown. The functional block diagram of the 2nd prior art is shown. The functional block diagram which concerns on embodiment of this invention is shown. It is a figure which shows the processing flow of a pre-processing part. It is a schematic diagram which shows the relationship between a LPC cepstrum coefficient and a sound wave. FIG. 6 is a diagram illustrating a processing flow of collation and registration processing according to the first embodiment. It is a figure which shows the processing flow of speaker identification. It is a schematic diagram which shows an example of HMM. It is a schematic diagram for demonstrating the state transition pattern in HMM. FIG. 10 is a diagram illustrating a processing flow of collation and registration processing according to the second embodiment. It is a figure which shows the model for collation (for Embodiment 2) comprised from a 2nd registration model. FIG. 10 is a diagram illustrating a processing flow of collation and registration processing according to the third embodiment. It is a figure which shows the model for collation (for Embodiment 3) comprised from a 2nd registration model.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Voice input part 3 Voice analysis part 5 Switching part 7 1st collation part 9 Model production | generation part 11 1st registration model storage part 13 2nd collation part 15 Model correction part 17 2nd registration model storage part 19 Collation result determination part 21 Advance Model storage unit 23 Pre-processing unit 25 Speech text determination unit 231 Pre-speech data storage unit 233 Second speech analysis unit 235 Pre-model generation unit

Claims (9)

A first registered model data storage unit for storing first registered model data generated from voice data of a person to be verified;
Second registered model data storage for storing second registered model data generated by adapting unspecified speaker model data generated from voice data of a plurality of unspecified speakers to the verification target person And
Analyzing means for analyzing voice data of the person to be collated and generating voice analysis data;
First verification processing means for performing verification processing using the voice analysis data and the first registered model data stored in the first registered model / data storage unit;
A second matching processing means for performing a matching process using the voice analysis data and the second registered model data stored in the second registered model / data storage unit;
A determination unit that performs a final determination process on the verification target person based on the verification processing results of the first verification processing unit and the second verification processing unit;
A speaker recognition system. The determination means is
The product of the first likelihood and (1-α) (α is a predetermined real number greater than or equal to 0 and less than or equal to 1), which is the result of the first collation processing means, and the result of the collation processing of the second collation processing means. The speaker recognition system according to claim 1, wherein a final determination process is performed on the person to be collated based on a value obtained by adding a certain second likelihood and the product of α. The first registration model data and the second registration model data are mixed normal distribution model data,
The speaker recognition system according to claim 1 or 2, wherein the matching process by the first matching processing unit and the matching process by the second matching processing unit are matching processes corresponding to the mixed normal distribution model. The first registered model data is data of a mixed normal distribution model;
The second registration model data is model data in subword units,
The matching process by the first matching processing means is a matching process corresponding to the mixed normal distribution model,
The second matching processing means
Collation model data generation means for connecting the subword unit model data stored in the second registered model data storage unit to generate collation model data;
Means for performing a matching process using the matching model data and the voice analysis data;
The speaker recognition system according to claim 1 or 2, comprising: Means for determining a phrase to be uttered by the person to be collated;
The collation model data generating means is
5. The speaker recognition system according to claim 4, wherein model data for collation is generated by connecting the model data in units of subwords stored in the second registered model data storage unit according to the phrase. Means for generating the first registration model data from the voice analysis data of the person to be collated generated by the analysis means at the time of model data registration;
Adapting the unspecified speaker model data stored in the unspecified speaker model data storage unit using the voice analysis data of the verification target person generated by the analysis means at the time of model data registration, Second registered model data generating means for generating second registered model data;
The speaker recognition system according to claim 1, further comprising: The second registration model data generation means includes
A process of adapting the model data of the subword uttered by the person to be collated at the time of model data registration according to a predetermined method,
The speaker registration system according to claim 6, wherein the second registration model data is generated by connecting model data in units of subwords adapted to each other. Analyzing voice data of the person to be matched to generate voice analysis data;
A first collation processing step for performing collation processing between the voice analysis data and the first registered model data generated from the voice data of the person to be collated and stored in the first registered model / data storage device;
Second registered model generated by adapting unspecified speaker model data generated from voice data of a plurality of unspecified speakers to the verification target person and stored in the second registered model data storage device A second collation processing step for performing collation processing between the data and the voice analysis data;
A step of performing a final determination process on the person to be collated based on a collation process result of the first collation process step and the second collation process step;
Speaker recognition program to make the computer execute. Analyzing voice data of the person to be matched to generate voice analysis data;
A first matching processing step for performing a matching process using the first registered model data generated from the voice data of the person to be matched and stored in the first registered model data storage unit and the voice analysis data;
Second registered model generated by adapting unspecified speaker model data generated from voice data of a plurality of unspecified speakers to the verification target person and stored in the second registered model data storage unit A second collation processing step for performing collation processing using data and the voice analysis data;
A step of performing a final determination process on the person to be collated based on a collation process result of the first collation process step and the second collation process step;
A speaker recognition method executed by a computer.

Images