JPH0854891A - Device and method for acoustic classification process and speaker classification process - Google Patents

Abstract

PURPOSE:To classify inputted voices for every speaker without using a dictionary, to execute various processes including retrieval and presentation to the classified inputted voices and to execute classification and other processes to general acoustic signals based on a prescribed reference. CONSTITUTION:An acoustic feature levels 103 or an acoustic signal 101 extracted by an acoustic feature level extracting means 102 is stored for every generating condition of the signal 101 by a sorting means 104. The levels 103 is classified by a classifying means 105 employing the classification for every generating condition of the signal 101 as reference. As a result, the levels 103 as well as the signal 101 are automatically classified in an independent condition with respect to the generating condition of the signal 101 without using a specific dictionary, etc.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a technique for classifying acoustic signals and a technique for performing various processes such as retrieval and presentation on the classified acoustic signals.

[0002]

2. Description of the Related Art Conventionally, a standard pattern of utterances of a plurality of speakers registered in a dictionary and an input voice pattern (input pattern)
There is known a technique for identifying or collating a speaker who has uttered an input voice by comparing and.

Here, the identification of the speaker means to judge which voice of a plurality of speakers registered in advance is the input voice, and generally, the similarity with the input pattern. The speaker corresponding to the largest standard pattern is the identification result.

Further, the speaker verification is performed by inputting, together with the input voice, speaker identification information indicating who is the speaker who uttered the input voice, and the input voice really corresponds to the speaker identification information. It is determined whether or not it is the voice of the speaker, and matching is performed depending on whether or not the similarity between the input pattern and the standard pattern of the speaker corresponding to the speaker identification information is equal to or more than a threshold value.

In such a conventional speaker identification / collation technique, standard patterns of utterances of a plurality of speakers to be identified or collated are registered in advance as a dictionary. On the other hand, there are two types of speaker identification / matching techniques: a utterance content-dependent type in which utterance content is predetermined and a utterance content independent type in which utterance content may be arbitrary.

In the speech content-dependent speaker identification / collation technology, it is relatively easy to associate the same phonemes with each other in the matching between the standard pattern and the input pattern.
Many speaker identification / verification systems have been put to practical use because they can achieve relatively high recognition performance.

On the other hand, in the speaker identification / verification technique independent of the utterance content, it is difficult to associate the same phonemes in the matching of the standard pattern and the input pattern, and the information indicating the individuality of the speaker is phonological. Since it tends to be buried in information and its recognition performance is limited, there are currently few systems in practical use. As a powerful one of the utterance content independent speaker identification / matching technology,
There is a technique that uses distortion due to vector quantization. In this technique, a short-time spectrum of an arbitrary utterance is clustered in advance for each registered speaker, and the resulting set of centroids of each cluster is registered in a dictionary as a codebook corresponding to the registered speaker. When the input speech is input, its short-time spectrum is vector-quantized by the codebook of each registered speaker, and the average quantization distortion over the entire input speech is calculated. This vector quantization process for the input voice is executed for the codebooks of all registered speakers. Then, in the speaker identification, the registered speaker having the smallest average quantization distortion is set as the identification result. Further, in the speaker verification, the verification is performed by comparing the average quantization distortion of the speaker codebook corresponding to the speaker identification information with a threshold value.

[0008]

However, in any of the above-mentioned conventional techniques, it is necessary to register the standard pattern of the utterance of the speaker as a dictionary. Since such a dictionary registration operation requires a lot of time and the speakers are limited,
It has a problem that it may cause inconvenience in practical use.

Further, since the utterance pattern of the speaker fluctuates with the passage of time since the dictionary is created, in order to prevent the system performance from degrading, it is necessary to take measures such as recreating the dictionary according to the passage of time. It also has the problem of becoming.

Further, in recent years, there has been a growing interest in multimedia in society, and there is an increasing demand for a system having high flexibility and convenience, such as classifying conversation contents according to speakers and freely searching or presenting them. ing. However, the above-described conventional techniques that merely identify or verify the speaker cannot sufficiently meet the demand for multimedia, and in particular, the limitation on the type of speaker and the utterance content is flexible. This is a fatal problem in building a high-quality system.

The present invention makes it possible to classify input voices for each speaker without using a dictionary, and to perform various processes such as retrieval and presentation on the classified input voices. It is an object of the present invention to enable classification and various processing for a predetermined standard even for various acoustic signals.

[0012]

FIG. 1 is a basic principle block diagram of the present invention. Acoustic feature quantity extraction means 102
Is an acoustic feature amount 10 indicating an acoustic feature of the acoustic signal 101.
Extract 3.

The selection means 104 is an acoustic feature quantity extraction means 1
The acoustic feature amount 103 extracted by 02 is selected for each generation condition of the acoustic signal 101 corresponding to the acoustic feature amount 103. For example, when the present invention is implemented as a speaker classification processing device, each generation condition of the acoustic signal 101 means each vowel, and, for example, the mechanical system operates in a constant cycle and is synchronized with it. When sound is generated, it means every predetermined timing in the cycle, and for example, when a certain sound is generated when the mechanical system is in a certain operating condition, each operating condition (for example, in the rotating system) It means that every rotation number).

The classification means 105 classifies the acoustic feature quantity 103 sorted by the sorting means 104 by evaluating the degree of similarity between the acoustic feature quantities 103 on the basis of the classification for each generation condition of the acoustic signal 101, The classification result and acoustic signal 10
The classification information 106 including at least the correspondence relationship with 1 is created.

In the basic configuration of the invention described above, when the generation condition of the acoustic signal 101 is given from the outside, for example, the acoustic signal 101 itself may be selected.

In addition to the basic configuration of the invention described above, a data processing means for performing data processing using the classification information 106 described above may be provided. Further, the present invention can also be realized by using the acoustic classification processing device as a speaker classification processing device.

Furthermore, the present invention can be configured as a method that achieves the same function as the above device configuration.

[0018]

According to the present invention, the acoustic feature quantity 103 or the acoustic signal 10
1 is selected for each generation condition of the acoustic signal 101, and then the acoustic feature amount 103 is classified based on the classification for each generation condition of the acoustic signal 101.

With such a configuration, the acoustic feature quantity 103 and thus the acoustic signal 101 can be automatically classified in a state independent of the generation condition of the acoustic signal 101 without using a specific dictionary or the like.

[0020]

Embodiments of the present invention will now be described in detail with reference to the drawings. <First Embodiment> The first embodiment is a case where the present invention is implemented as a speaker classification processing system. Description of Speaker Classification Device FIG. 2 is a block diagram of the speaker classification device in the first embodiment. This device mainly includes a voicing section extraction unit 203, a frame division unit 205, a linear prediction coefficient / PARCOR coefficient calculation unit 207, an LPC cepstrum calculation unit 210, a vowel extraction unit 212, a voicing clustering unit 215, and the like.

First, the conversation data 201 is digital voice data input from a microphone (not shown) through an A / D converter (not shown). The sampling frequency is, for example, 11 kHz, and the number of quantization bits is, for example, 16 bits.

The conversation data 201 is the conversation file 202.
In particular, it is stored in a storage medium (IC memory card, disk storage device, etc.) not shown. The voicing section extraction unit 203 extracts stable sections (hereinafter referred to as voicing sections) other than spectrally unstable sections such as silent sections, unvoiced sections, beginning sections, and ending sections from the conversation file 202, and FIG. As shown in (a), the utterance section instruction file 204 in which the number N of utterance sections and the start sample position S and the end sample position E of each utterance section on the conversation file 202 are stored is created. It is stored in a storage medium (not shown). As described above, only the utterance section is extracted, and the vowel extraction unit 212, which will be described later, executes the vowel extraction processing on the section, thereby significantly improving the accuracy of the vowel extraction that is the preprocessing for classifying the speaker. Can be improved.
This operation algorithm is based on steps 1 to 1 shown below.
This is realized as the process of step 7. Step 1: The voice waveform sample of the conversation file 202 is read, and the value of the sample counter i and the value of the vocal section counter n are both initialized to 0. Step 2: After S = i, the value of the sample counter i is sequentially incremented while the sample value p
The sample position E = i immediately before the sample position where the sign of (i) changes is detected. As a result, a section [p (S), p (E)] in which the sample values have the same sign is determined. Value E
Is within the range of the number of samples in conversation file 202,
The processes of steps 3 to 6 below are repeated. Step 3: In the interval [p (S), p (E)], the maximum absolute value of the signal level is 2.5 dB (decibel).
If it is above, this section is regarded as the vocalization section, and the processing of the following Step 4 or Step 5 is executed. Step 4: A vocal section has been detected (n> 0), and the vocal section Sn immediately before and the vocal section currently being evaluated [p
(S), p (E)],

[0023]

[Equation 1]

If the above condition holds, even if there is an unvoiced section between these two utterances, it is judged that this is an unvoiced section included in the utterance of the onset sound, and [p (S), p (E)] is It is integrated with the immediately preceding vocal section Sn. That is, E _Sn = E is set, i = E + 1 is set to search for the next vocalization section, and step 2
The loop of is advanced. This is shown in FIG. Step 5: If Step 4 does not hold, the vocal section [p (S), p (E)] currently being evaluated is an independent vocal section. That is, the value of the vocalization section counter n is n =
It is incremented as n + 1, i = E + 1 is set to search for the next vocal section, and the loop of step 2 is advanced. Step 6: Within the section [p (S), p (E)], the maximum absolute value of the signal level is 2.5 dB (decibel).
If it is less than, it is considered that this section is not a vocal section and is ignored. In this case, i = E + 1 is set to search for the next vocalization section, and the loop of step 2 is advanced. Step 7: The number of vocal sections detected as a result of repeating the loop of Step 2, the start sample position S _Sn of each vocal section
And the end sample position E _Sn (1 ≦ n ≦ N) are written in the vocalization section instruction file 204 (FIG. 3 (a)). Returning to FIG. 2, the frame division unit 205 divides each utterance section in the conversation file 202 indicated by the utterance section instruction file 204 into a plurality of frames each having a predetermined length, and outputs the utterance corresponding to each utterance section. An n-frame division file 206 (1 ≦ n ≦ N) is created and stored in a storage medium (not shown). Utterance n frame division file 20
6 shows, as shown in FIG. 3 (b), a vocalization section number n indicating a vocalization section, the number of frames Mn in the vocalization section, and each frame m (1≤m≤Mn) in the vocalization section. The starting sample position s of is stored. Frame length is 2
56 samples, for example, 19 between adjacent frames
The sample data for every two samples is overlapped.

Linear prediction coefficient / PARCOR coefficient calculation unit 2
07 calculates a linear prediction coefficient set and a PARCOR coefficient set from the speech waveform sample in the conversation file 202 corresponding to each frame position in one utterance section indicated by the utterance n frame division file 206, and outputs the utterance n linear prediction coefficient. The file 208 and the utterance nPARCOR coefficient file 209 (1 ≦ n ≦ N) are created and stored in a storage medium (not shown). Utterance n linear prediction coefficient file 20
8 and the utterance nPARCOR coefficient file 209, as shown in FIGS. 3 (c) and 3 (d), the utterance section number n indicating the utterance section and the linear prediction coefficient corresponding to each frame in the utterance section, respectively. The set {α ₁ , α ₂ , ..., α _p } or the PARCOR coefficient set {k ₁ , k ₂ , ..., K _p } is stored. Here, the suffix p is the order.
The calculation algorithm of the linear prediction coefficient and the PARCOR coefficient is implemented as Step 1 to Step 8 shown below. Step 1: While incrementing the value of the vocalization section counter n from 1 to N, the following steps 2 to 8 are repeated. The number N of vocalization sections is stored at the beginning of the vocalization section instruction file 204. Step 2: While incrementing the value of the frame counter m from 1 to Mn, the frame 1 to the frame Mn (Fig. 3 (b)) designated by the utterance n frame division file 206 corresponding to the nth utterance section currently being processed For each frame (refer to FIG. 3), the processes of the following steps 3 to 8 are repeated. The frame number Mn is stored in the second storage position of the utterance n frame division file 206. Step 3: From the conversation file 202, the n currently being processed
The frame length (for example, 256 samples) from the start sample position s (see FIG. 3B) of the m-th frame currently being processed stored in the utterance n frame division file 206 corresponding to the utterance section The voice waveform sample is read. This sample sequence is x (i) (1 ≦ i ≦ frame length−1). Step 4: The voice waveform sample x (i) (1 ≦ i ≦ frame length −1) is multiplied by the window function based on the following equation, and the result is x _W (i) (1 ≦ i ≦ frame length −1). ) Is said. By multiplying by the window function, it is possible to reduce the influence of truncation error at the frame end in the calculation of the linear prediction coefficient and PARCOR coefficient described later.

[0026]

[Equation 2]

Step 5: Voice waveform sample x _W (i)
Difference from the immediately preceding sample for (1 ≦ i ≦ frame length-1)

[0028]

(Equation 3)

Is calculated. Step 6: An autocorrelation function V is calculated for y (i) (1≤i≤frame length-1). Step 7: From the autocorrelation function V, the recursive release of Durbin is used to calculate the linear prediction coefficient set α _j and the PARCOR coefficient set k _j (1 ≦ j ≦ p, p is the order). Step 8: n currently being processed, calculated in step 7
The linear prediction coefficient set α _j and the PARCOR coefficient set k _j corresponding to the m-th frame currently being processed in the utterance section
(1 ≦ j ≦ p) is the utterance n linear prediction coefficient file 208
(Fig. 3 (c)) and utterance nPARCOR coefficient file 20
9 (FIG. 3 (d)). Next, in FIG. 2, the LPC cepstrum calculation unit 210 calculates an LPC cepstrum coefficient set from the linear prediction coefficient set for each frame in the utterance n linear prediction coefficient file 208,
Spoken nLPC cepstrum file 211 (1 ≤ n ≤
N) is created and stored in a storage medium (not shown). In the utterance nLPC cepstrum file 211, as shown in FIG. 3 (e), the utterance section number n indicating the utterance section and the LPC cepstrum coefficient set {c ₁ , c ₂ , · · ·, c _p} is stored. The LPC cepstrum coefficient calculation algorithm is implemented as Step 1 to Step 5 shown below. Step 1: While incrementing the value of the vocalization section counter n from 1 to N, the following steps 2 to 5 are repeated. The number N of vocalization sections is stored at the beginning of the vocalization section instruction file 204. Step 2: While the value of the frame counter m is incremented from 1 to Mn, the processing of the following steps 3 to 5 is repeated. The number of frames Mn is
It is stored in the second storage position of the utterance n frame division file 206. Step 3: m being currently processed from the utterance n linear prediction coefficient file 208 corresponding to the nth utterance section being currently processed
Linear prediction coefficient set α _j (1 ≦ j
≤ p) is read. Step 4: LP from linear prediction coefficient set α _j according to the following equation
The C cepstrum coefficient set cj (1≤j≤p) is calculated.

[0030]

[Equation 4]

Step 5: The LPC cepstrum coefficient set cj (1) calculated in step 4 and corresponding to the m-th frame currently being processed in the n-th vocal section currently being processed.
≦ j ≦ p) is the utterance nLPC cepstrum file 21
1 (FIG. 3 (e)). Subsequently, in FIG. 2, the vowel extraction unit 212 outputs the utterance nP.
PA for each frame in the ARCOR coefficient file 209
By matching the RCOR coefficient set with the dictionary PARCOR coefficient set corresponding to each vowel in the 5-vowel PARCOR coefficient dictionary 213 stored in the storage medium (not shown), each of the utterance n-frame division files 206 is matched. A label of either 5 vowels or invalid is added to the frame, a voicing n vowel label file 214 (1 ≦ n ≦ N) is created, and stored in a storage medium (not shown). The utterance n vowel label file 214 stores utterance section numbers n indicating utterance sections and vowel labels corresponding to each frame in the utterance section, as shown in FIG. 3 (f).
This operation algorithm is based on steps 1 to 1 shown below.
This is realized as the process of step 9. Step 1: From the 5 vowel PARCOR coefficient dictionary 213,
A dictionary PARCOR coefficient set of five vowels of / a /, / i /, / u /, / e /, / o / is read. There are a plurality of dictionary PARCOR coefficient sets for one vowel. Here, the vowel v (v
= / A /, / i /, / u /, / e /, / o /) z-th dictionary PARCOR
The coefficient set is kv _j ^z (1≤j≤p, 1≤z≤Zv). Step 2: While the value of the vocal section counter n is incremented from 1 to N, the processing of the following steps 3 to 9 is repeated. The number N of vocalization sections is stored at the beginning of the vocalization section instruction file 204. Step 3: While the value of the frame counter m is incremented from 1 to Mn, the processing of the following steps 4 to 9 is repeated. The number of frames Mn is
It is stored in the second storage position of the utterance n frame division file 206. Step 4: From the utterance nPARCOR coefficient file 209 corresponding to the nth utterance section currently being processed, the PARCOR coefficient set k _j corresponding to the mth frame currently being processed
(1 ≦ j ≦ p) is read. Step 5: For each vowel v (v = / a /, / i /, / u /, / e /, / o /), the processing of the following steps 6 and 7 is repeated. Step 6: n currently being processed read in step 4
The PARCOR coefficient set k _j for the m-th frame currently being processed in the th vocal section and the dictionary PARCOR coefficient set kv _j ^z corresponding to the vowel v read in step 1.
The distance dv between and is calculated by the following equation.

[0032]

(Equation 5)

That is, each dictionary PARCOR belonging to the vowel v
Dv is calculated as the minimum value of the Euclidean distance from the coefficient set. W _j (1 ≦ j ≦ p) is a weight for each order. Min _z {} is a dictionary PARC belonging to the vowel v
An operation for selecting the minimum value among the values obtained by changing the OR coefficient set z is shown. Step 7: When the value of the distance dv is larger than the threshold value set for the vowel v, dv = D (constant value) is replaced.
By this processing, the variation in distance for each vowel v is normalized. Step 8: For the vowel v having the smallest value of the distance dv, if the value of the distance dv is less than the constant value D, the vowel n vowel label file 214 corresponding to the n-th vocal section currently being processed is stored. The label corresponding to the vowel v is stored at the position corresponding to the m-th frame currently being processed. Step 9: On the other hand, for the vowel v for which the value of the distance dv is the minimum, if the value of the distance dv is equal to the constant value D, then
A label indicating "invalid" is stored in a position corresponding to the m-th frame being currently processed in the utterance n vowel label file 214 corresponding to the n-th vocal segment being currently processed. Finally, in FIG. 2, the utterance clustering unit 215
Perform clustering for each vowel label among each of the N vowel n vowel label files 214 (1 ≦ n ≦ N) belonging to the conversation file 202, and thus the utterance sections of the N sections belonging to the process target conversation file 202. Each is classified into each speaker.

As a result, a speaker file-based speaker classification database 216 is created. The database includes a conversation file 202 and a speaker-specific utterance location information file 21.
7, a speaker-based / vowel-based PARCOR coefficient statistical information file 218, and a speaker-based / vowel-based LPC cepstrum statistical information file 219 are included.

The conversation file 202 is a conversation file targeted for processing. Spoken point information file by speaker 21
7, each speaker who is classified and given an appropriate tag (hereinafter, referred to as a speaker identification tag) has a conversation file 202.
It indicates which one or more of the N vocalization sections in which the vocalization is being performed. More specifically, as shown in FIG. 4A, the speaker-specific utterance location information file 217 includes a conversation number x, a speaker identification tag, and one or more utterance sections for each tag. One or more sets of a start sample position S and an end sample position E corresponding to each are stored.

As shown in FIG. 4B, the speaker-by-vowel-by-vowel PARCOR coefficient statistical information file 218 represents the PARCOR representing each speaker and each vowel for each speaker identification tag and every five vowels. The coefficient set is stored.

As shown in FIG. 4C, the LPC cepstral statistical information file 219 for each speaker and each vowel has an LPC representing each speaker and each vowel for each speaker identification tag and every 5 vowels. The cepstrum coefficient set is stored.

The clustering algorithm is as follows. {Procedure 1: Determination of same-speaker determination membership function Fv (d)} In clustering, it is determined whether the same vowels appearing in two compared vocal sections are caused by the same speaker or different speakers. There is a need to. The membership function Fv (d) is used to make this determination. Here, v = / a /, / i /, / u /, / e /, / o /, and d is the weighted Euclidean distance (number) between two LPC cepstrum coefficient pairs to be compared that show the same vowel. Equation 5).

In order to determine the above-mentioned membership function Fv (d), as a preliminary process, 5 in the same speaker and between different speakers are used.
For each of the vowels, the weighted Euclidean distance (hereinafter, simply referred to as distance) between two sets of LPC cepstrum coefficient sets is calculated for all the combinations, and when this is expressed as an appearance probability, each vowel is schematically shown in FIG. Have a distribution.

Based on this distance distribution, each vowel v = / a /, / i
For /, / u /, / e /, / o /, the same-speaker determination membership function Fv (d) having the outline shown in FIG. 7 is determined. Here, the section [d1, d3] of the distance d shown in FIG.
Fv (d) in (see also), where Qs (d) is the appearance probability of the distance d between the same speakers and Qa (d) is the appearance probability of the distance d between different speakers as shown in FIG. It can be represented by a formula.

[0041]

(Equation 6)

{Procedure 2: Calculation of similarity evaluation matrix for each vowel} Using the same-speaker determination membership function Fv (d) determined in step 1, the vowel segment extraction unit 203 of FIG. The data corresponding to the evaluation matrix indicating the similarity of the speakers between the extracted utterance sections is the following steps 1 to
It is calculated by the process of step 7. Step 1: While the value of the vocal section counter n is incremented from 1 to N, the processing of the following steps 2 and 3 is repeated. The number N of vocalization sections is stored at the beginning of the vocalization section instruction file 204. Step 2: Vowel label v (v = / a /, / i /, / u /, / e /, / o /
), The process of step 3 below is repeated. Step 3: The frame number to which the vowel label v is added is determined from the utterance n vowel label file 214 (FIG. 3 (f)) corresponding to the nth utterance section Sn currently being processed,
Next, the LPC cepstrum coefficient sets corresponding to all the above-mentioned determined frames are read from the utterance nLPC cepstrum file 211 (FIG. 3 (e)) corresponding to the n-th utterance section Sn currently being processed. Next, these LPs
The average value of each degree of the C cepstrum coefficient set is calculated, and the average LPC cepstrum coefficient set obtained as a result is set as the LPC cepstrum coefficient set representing the vowel v in the vocal section Sn. * By repeating the processing of step 1 to step 3, as shown in FIG. 8, each vocal section Sn (1 ≦ n ≦
N) corresponding average LPC cepstrum coefficient set for each vowel a _Sn (corresponding to vowel / a /), i _Sn (corresponding to vowel / i /), u
_Sn (corresponding to vowel / u /), e _Sn (corresponding to vowel / e /), and o _Sn (corresponding to vowel / o /) are calculated. Note that the average LPC cepstrum coefficient set of all five vowels does not have to be complete in each vocal section Sn, and vowels that did not exist in the vocal section are indicated by x in FIG. Further, in FIG. 8, each average LPC cepstrum coefficient set a _Sn , i _Sn ,
The vector symbol “→” is added to u _Sn , e _Sn , and o _Sn . Step 4: Following the repetition of Steps 1 to 3 described above, a combination of all two vocalization sections Si, Sj (i
≠ j, 1 ≤ i, j ≤ N),
The process of step 7 is repeated. Step 5: For each of the common vowel v calculated in the two vocalization sections Si and Sj, the processing of the following Step 6 and Step 7 is repeated. Step 6: Between the two average LPC cepstrum coefficient pairs for the common vowel v in the two vocal intervals Si, Sj,
The distance d (v _Si , v _Sj ) is calculated. Step 7: The distance d (v _Si , v calculated in Step 6
_Sj ) and the similarity Pv _SiSj (v = / a /, / i /, for the common vowel v in the two vocal intervals Si and Sj by the following equation:
/ u /, / e /, / o /, i ≠ j, 1 ≦ i, j ≦ N) is calculated.

[0043]

(Equation 7)

As a result of repeating the processing of steps 4 to 7, the data corresponding to the evaluation matrix having the similarity between any two voicing sections as an element is obtained for each of the five vowels. {Procedure 3: Relaxation} From the evaluation matrix of the similarity for each vowel calculated in procedure 2, the final degree of similarity between utterance intervals is shown by relaxation.
The data corresponding to the evaluation matrix as shown in
It is created by the processing of the following steps 1 to 7. Step 1: First, the average similarity P _SiSj in the two utterance sections Si, Sj is calculated as an average value of the similarity Pv _SiSj regarding the common vowel v in the two utterance sections Si, Sj as shown in the following equation. To be done.

[0045]

[Equation 8]

Step 2: The value of the repetition counter q for repetition of relaxation is initialized to 1. Step 3: Combination of two vocalization sections Si, Sj (i ≠ j,
The process of step 4 below is repeated for all of 1 ≦ i, j ≦ N). Step 4: The element value of the evaluation matrix (FIG. 10) showing the similarity between the two vocalization sections Si and Sj at q = 1 is M (1).
_SiSj, and the element value is 2 as shown in the following equation.
Initially set to the value of the average similarity P _SiSj in one vocal section Si, Sj.

[0047]

[Equation 9]

Step 5: While the value of the repeat counter q is incremented from 2 to the number of terminations, the processes of the following steps 6 and 7 are repeated. Step 6: Combination of two vocalization sections Si, Sj (i ≠ j,
The process of step 7 below is repeated for all of 1 ≦ i, j ≦ N). Step 7: The element value of the evaluation matrix (FIG. 10) showing the degree of similarity between the two vocalization sections Si and Sj in the q-th iteration of Step 5 is M (q) _SiSj , and the q-1st step The element value in the repetition of 5 is set to M (q-1) _SiSj, and M (q) _SiSj is updated by the following equation.

[0049]

[Equation 10]

Here, the score point _SiSj is calculated by the following steps 7-1 to 7-19. Step 7-1: The dynamic threshold 0 is set to all the element values M (q-1) _SiSj (i calculated in the q- _1th iteration of Step 5).
≠ j, 1 ≤ i, j ≤ N). In addition, the same-speaker boundary threshold for the similarity of speakers is threshold 1,
The threshold value for the cross-speaker boundary is set to threshold 2. Here, the threshold 1 and the threshold 2 are predetermined values, and 0 <threshold 2 <threshold 1 <
It is 1. Step 7-2: Two currently selected utterance sections S calculated in the (q-1) th iteration of Step 5
The element value M (q-1) _SiSj between i and Sj is M (q-1) _SiSj > threshold 0
If the above conditions are satisfied, that is, if the two vocalization sections Si and Sj are likely to be the same speaker, the following steps 7-3 to 7-10 are performed.
Is executed. Step 7-3 (Rule 1): All utterance sections k (≠ j) satisfying the average similarity P _SiSk > threshold value 1 for one utterance section Si of the two currently selected utterance sections
Is the other one of the two currently selected vocalization intervals
When the average similarity P _SkSj > threshold value 1 is satisfied also for one utterance section Sj, that is, when all of the utterance sections Sk that _seem to be the same speaker as the utterance section Si also appear to be the same speaker as the utterance section Sj,
The following steps 7-4 to 7-6 are executed. Step 7-4: All utterance sections k (≠ j) satisfying the average similarity P _SiSk <threshold value 2 with respect to one utterance section Si of the two currently selected utterance sections are currently selected 2 Another one of the one vocalization section Sj
In the case where the average similarity P _SkSj <threshold value 2 is satisfied, that is, when all of the utterance section Si and the utterance section Sk that seems to be a speaker are both the utterance section Sj and a speaker that is like a speaker, the following equation is given. In _addition , the maximum score + 4Δ (Δ is an appropriate value) for the score point _SiSj calculated in the (q-1) th iteration of step 5
Is given and is used for the calculation of the equation (10) this time (q-th time).

[0051]

[Equation 11]

Step 7-5: When step 7-4 is not established, the average similarity P _SiSk <for one utterance section Si of the two currently selected utterance sections
More than half of the utterance sections k (≠ j) satisfying the threshold value 2 are the other 1 of the two utterance sections currently selected.
When the average similarity P _SkSj <threshold value 2 is satisfied for two utterance sections Sj, that is, the utterance section Si and the utterance section that seems to be a different speaker
If it seems different speaker more than half with vocal section Sj of the Sk, as shown in the following equation, the score point _SiSj half of the maximum score on the score + 2Δ is given, the number of this time (q-th-th) 10 Used in formula calculations.

[0053]

[Equation 12]

Step 7-6: When both Step 7-4 and Step 7-5 are not established, that is, it is determined that both the utterance section Si and the utterance section Sj of the utterance section Sk that seems to be a different speaker are likely to be different speakers. Less than half, vocalization sections Si and Sj
Is not the same speaker, the score point _SiSj is set to 0 as shown in the following equation, and is used in the calculation of equation (10) at this time (q-th time).

[0055]

[Equation 13]

Step 7-7 (Rule 2): Although Rule 1 of Step 7-3 described above is not established, the average similarity P _SiSk > for one utterance section Si of the two utterance sections currently selected. A vocalization section k (≠≠
In j), more than half of them satisfy the average similarity P _SkSj > threshold value 1 with respect to another one vocalization section Sj of the currently selected two vocalization sections, and all of them are vocalized. When the average similarity P _SkSj > threshold value 2 is satisfied for the section Sj, that is, more than half of the utterance sections Sk that are likely to be the same speaker as the utterance section Si are the same speakers as the utterance section Sj and are different speakers. If all that doesn't seem to be,
The following steps 7-8 to 7-10 are executed. Step 7-8: As in the case of Step 7-4, one utterance section of the two currently selected utterance sections
All the vocalization sections k (≠ j) satisfying the average similarity P _SiSk <threshold value 2 with respect to Si are also averaged with respect to the other one vocalization section Sj of the currently selected two vocalization sections Sj. _{When SkSj} <threshold value 2 is satisfied, that is, when all of the vocalization section Si and the vocalization section Sk that seems to be a different speaker are also likely to be different speakers from the vocalization section Sj, as shown in the following equation, The score point _SiSj calculated in the repetition of step 5 is given a score + 2Δ which is half the maximum score, and is used for the calculation of the formula (10) this time (q-th time).

[0057]

[Equation 14]

Step 7-9: When step 7-8 is not established, the average similarity P for one utterance section Si of the two currently selected utterance sections is the same as in step 7-5. _More than half of the utterance sections k (≠ j) satisfying _SiSk <threshold value 2 are also average similarity P _SkSj <threshold value with respect to another one utterance section Sj of the currently selected two utterance sections. When 2 is satisfied, that is, when more than half of the utterance section Si and the speaker-like skeleton Sk that are both speaker-like and the _utterer section Sj also seem to be the speaker, as shown in the following expression, the score point _SiSj has the highest score. A score + Δ of ¼ is given, and is used in the calculation of Formula 10 of this time (qth time).

[0059]

(Equation 15)

Step 7-10: If neither Step 7-8 nor Step 7-9 is true, that is, Step 7
Similar to the case of −6, less than half of the utterance sections Si and the utterance sections Sk that seem to be different speakers were determined to be utterance sections in both utterance sections Sj, and the utterance sections Si and Sj are the same speaker. If you can't go, score points as shown in the following equation.
_SiSj is set to 0 and used in this time (q-th time) calculation of Expression 10.

[0061]

[Equation 16]

Step 7-11: The element value M (q-1) _SiSj between the two currently selected vocalization sections Si and Sj calculated in the q- _1th iteration of Step 5 is M ( q-1) _SiSj <
When the threshold value 0 is satisfied, that is, when the two vocalization sections Si and Sj are likely to be different speakers, the following steps 7-12 to 7
-19 is executed. Step 7-12 (Rule 3): 2 currently selected
All voicing sections k (≠ j) satisfying the average similarity P _SiSk <threshold value 2 with respect to one voicing section Si of one voicing section
Is the other one of the two currently selected vocalization intervals
When the average similarity P _SkSj <threshold value 2 is satisfied for two utterance sections Sj, that is, the utterance section Si and the utterance section that seems to be a different speaker
If all of Sk and the vocalization section Sj seem to be different speakers, the following steps 7-13 to 7-15 are executed. Step 7-13: Average similarity P _SiSk > for one vocalization section Si of the two currently selected vocalization sections
All utterance sections k (≠ j) satisfying the threshold 1 are the other one utterance section of the two currently selected utterance sections.
When Sj also satisfies the average similarity P _SkSj > threshold 1,
That is, when all of the utterance sections Sk that are likely to be the same speaker as the utterance section Si are also likely to be the same speaker as the utterance section Sj, they are calculated in the q−1-th iteration of step 5 as shown in the following equation. The lowest point _-4Δ is given to the score point _SiSj , and is used for the calculation of the equation (10) this time (qth time).

[0063]

[Equation 17]

Step 7-14: When step 7-1 3 is not satisfied, the average similarity P for one utterance section Si of the two utterance sections currently selected.
_More than half of the utterance sections k (≠ j) satisfying _SiSk > threshold 1 are also average similarity P _SkSj > threshold with respect to another one utterance section Sj of the two currently selected utterance sections. In the case where 1 is satisfied, that is, when more than half of the utterance intervals Sk that are likely to be the same speaker as the utterance interval Si are also likely to be the same speaker as the utterance interval Sj, as shown in the following expression, the score point _SiSj has the lowest deduction. The half point of -2Δ is given, and this time (qth q
It is used for the calculation of the equation 10 of the second time).

[0065]

(Equation 18)

Step 7-15: When neither Step 7-13 nor Step 7-14 is satisfied, that is, it is determined that the utterance section Sj of the utterance section Sk that is the same speaker as the utterance section Si is also the same speaker. If less than half of the speech is generated and the vocalization sections Si and Sj cannot be said to be different speakers, the score point _SiSj is set to 0 without deduction as shown in the following equation, (qth time) is used in the calculation of Expression 10.

[0067]

[Formula 19]

Step 7-16 (Rule 4): The above-mentioned Rule 3 of Step 7-12 is not established, but the average similarity P _SiSk <for one utterance section Si of the two currently selected utterance sections The vocalization section k (≠≠
In j), more than half of them satisfy the average similarity P _SkSj <threshold value 2 with respect to another one vocalization section Sj of the currently selected two vocalization sections, and all of them are vocalized. When the average similarity P _SkSj <threshold value 1 is satisfied with respect to the section Sj, that is, among the utterance section Si and the utterance section Sk that seems to be a different speaker,
If more than half of them are different speakers in the utterance section Sj and are not all the same speaker, the following steps 7-17 to 7-19 are executed. Step 7-17: Similar to Step 7-13,
All the utterance sections k (≠ j) satisfying the average similarity P _SiSk > threshold value 1 with respect to one utterance section Si of the two currently selected utterance sections out of the two currently selected utterance sections When the average similarity P _SkSj > threshold value 1 is satisfied with respect to another one utterance section Sj, that is, when all utterance sections Sk that are likely to be the same speaker as the utterance section Si are also the same speaker as the utterance section Sj. Is the q−1 th
Score poi calculated in the iteration of step 5
nt _SiSj is given a deduction of −2Δ, which is half of the lowest deduction, and is used in the calculation of Equation 10 this time (q-th time).

[0069]

[Equation 20]

Step 7-18: When step 7-17 is not established, the average similarity P for one utterance section Si of the two currently selected utterance sections is the same as in the case of step 7-14. _More than half of the utterance sections k (≠ j) satisfying _SiSk > threshold 1 are also average similarity P _SkSj > threshold with respect to another one utterance section Sj of the two currently selected utterance sections. In the case where 1 is satisfied, that is, when more than half of the utterance intervals Sk that are likely to be the same speaker as the utterance interval Si are also likely to be the same speaker as the utterance interval Sj, as shown in the following expression, the score point _SiSj has the lowest deduction. 1/4 of
Deduction point −Δ is given and used for the calculation of the equation (10) this time (q-th time).

[0071]

[Equation 21]

Step 7-19: If neither Step 7-17 nor Step 7-18 is true, that is, as in Step 7-15, the utterance section of the utterance section Sk that seems to be the same speaker as the utterance section Si. If less than half of the Sj are judged to be the same speaker, and if the vocalization sections Si and Sj cannot be said to be different speakers, as shown in the following equation, the score is not deducted _SiSj is set to 0 and used in this time (q-th time) calculation of Expression 10.

[0073]

[Equation 22]

The above-mentioned steps 5 to 7 are repeatedly executed by the utterance clustering unit 215 of FIG. 2 to perform relaxation, and when the value of the repeat counter q reaches the end count, the final utterance section is reached. The data M (q) _SiSj (i ≠ j, _1≤ ) corresponding to each element value of the evaluation matrix as shown in FIG.
i, j ≦ N) is calculated. The contents of the evaluation matrix of the similarity for each vowel calculated in step 2 are properly reflected in this data. {Procedure 4: Clustering} Data calculated in Procedure 3
In M (q) _SiSj (i ≠ j, 1 ≦ i, j ≦ N), the same speaker identification tag is attached to a combination of two or more utterance sections having a value equal to or _larger than a predetermined threshold value. As the predetermined threshold value, for example, the finally obtained data value M (q-1) _SiSj (i ≠ j,
Dynamic threshold 0 calculated as an average value of 1 ≦ i, j ≦ N)
Can be used.

In this way, after the utterance section belonging to each speaker identification tag is determined, the start sample position of each utterance section is determined from the utterance section instruction file 204 (see FIGS. 2 and 3 (a)). The set of S and end sample position E is read. For each speaker, a conversation number x, a speaker identification tag, and one or more sets of a start sample position S and an end sample position E corresponding to one or more utterance sections for each tag are stored. Spoken point information file 217 (Fig. 4
(a)) is created and registered as the conversation file-based speaker classification database 216. As described above, in the speaker classification device embodied by the present invention, the utterances in each conversation sequentially input by the user can be automatically classified for each speaker, and in that case, a special dictionary is required. The major feature is that it is not affected by long-term fluctuations in speaker characteristics.

In particular, it is not necessary to limit the type of speaker, and the speaker having a conversation at each time is automatically classified and an appropriate speaker identification tag is added. Furthermore, in the above-described embodiment, a vowel label is given to each frame in the utterance section, and clustering is performed for each vowel between the utterance sections to classify speakers in a state independent of the utterance content, that is, the phoneme. be able to.

In addition, only the stable sections other than the spectrally unstable sections such as the silent section, the unvoiced section, the beginning section, and the ending section in the conversation file 202 are used as the vowel section in which the vowels are extracted. Therefore, the vowel extraction error can be significantly reduced. As described above, the present embodiment effectively utilizes the fact that not all voice sections are necessary for speaker classification, and only stable sections are required. Description of Speaker Classification Result Presenting Device FIG. 11 is a block diagram of a speaker classification result presenting device that uses the speaker file-based speaker classification database 216 created by the speaker classification device of FIG. 2 in the first embodiment. Is.

The representative partial reproduction unit 1102 for each speaker, in the speaker classification database 216 created by the speaker classification apparatus of FIG. 2, is the speaker-specific utterance location information file 217 shown in FIG. 4A. For each speaker identification tag registered in, the start sample position S and the end sample position E of a typical (for example, the head or the longest) utterance section corresponding to the speaker identification tag are read, and the voice of the section The waveform sample is read from the conversation file 202 registered in the conversation classification database 216 for each conversation file, reproduced, and heard by the user.

The user determines the name of the speaker who is uttering the voice currently being heard, and inputs the text data of the speaker name using the speaker name input unit 1103. This text data is paired with the speaker identification tag currently reproduced by the representative partial reproduction unit 1102 for each speaker, and the speaker-specific name file 1101 is stored in the conversation file-based speaker classification database 216.
Registered as.

As described above, by using the speaker classification result presentation device having such a simple structure, the user is registered in the conversation file-based speaker classification database 216 with an intuitive and simple operation. The speaker name can be registered in the conversation file 202. Description of Speaker Classification Result Retrieval Device FIG. 12 shows a speaker classification having various retrieval functions for the conversation file-based speaker classification database 216 created by the speaker classification device of FIG. 2 in the first embodiment. It is a block diagram of a result search device.

Conversation data search unit 1201 based on the number of speakers
Allows the user to input the number of speakers, and sequentially searches the number of speaker identification tags in the speaker-specific utterance location information file 217 to obtain a number of speaker identification tags corresponding to the input number of speakers. The conversation number x (see FIG. 4A) is acquired from the speaker-specific utterance location information file 217 in which is registered, the name of the conversation file 202 corresponding to the conversation number x, and the output of the voice of the conversation file 202. , Display voice waveforms, etc. In this case, for example, the user may be configured to be able to add arbitrary recording information (simple contents, date and time, etc.) to the conversation file 202 when recording a conversation (during speaker classification).

As a more specific application example, for example, during many-to-many communication on a telephone or network, how many people are currently participating in the session is discussed for about 10 minutes from that point. Providing the file 202 by examining it can be easily realized by applying the above-described technique.

Conversation data search unit 120 by specific speaker name
2 allows a user to input text data of a specific speaker name to extract a speaker identification tag corresponding to the speaker name from the speaker-specific name file 1101, and the speaker identification tag is registered. Speaking point information file by speaker 21
Search for 7. Then, the conversation number x (see FIG. 4A) is acquired from the speaker-specific utterance location information file 217 in which the speaker identification tag is registered, and the name of the conversation file 202 corresponding to the conversation number x and its The voice of the conversation file 202 is output and the voice waveform is displayed.

The specific speaker's utterance location searching unit 1203 causes the user to input the conversation file name and the text data of the specific speaker name, and thereby the speaker-specific name file 1101.
A speaker identification tag corresponding to the speaker name is extracted from the corresponding speaker name on the speaker-specific utterance location information file 217 corresponding to the conversation file 202 having the input conversation file name. Each registered vocalization section (Fig. 4
(See (a)) Start sample position S and end sample position E
Are sequentially read out, the voice waveform samples of the respective sections are sequentially read out from the conversation file 202 having the input conversation file name, reproduced, and heard by the user. It may be configured to be able to display the voice waveform of the corresponding portion and present the utterance portion. If the speaker-specific name file 1101 is not registered in the conversation-file-specific speaker classification database 216 by the above-described speaker classification result presentation device, the conversation data search unit 1202 by the specific speaker name causes a conversation with the user. After inputting the file name, the speaker identification tags stored in the speaker-specific utterance location information file 217 corresponding to the conversation file 202 having the input conversation file name are displayed in a list on the liquid crystal display or the like, and the user Let the user select a specific speaker identification tag. Then, the voice in the utterance section corresponding to the speaker identification tag selected by the user is uttered or presented in the same manner as in the above case.

As a more specific application example, for example, by learning the voice on the answering machine for several months,
By applying the above-mentioned technique, it is possible to easily realize, for example, whether only a specific person's requirement is heard or by learning the tag from the learning result to know who made the call. Alternatively, it is also possible to easily present a voice of only a specific actor when dubbing a movie or a drama in which no strict scenario exists.

Conversation data retrieval unit 1 by trial listening of speaker utterance
Reference numeral 204 denotes a speaker classification database 216 for each conversation file.
In FIG. 4, for each speaker identification tag registered in the speaker-specific utterance location information file 217 shown in FIG. 4A, the start sample position S of each utterance section corresponding to the speaker identification tag
And the end sample position E are sequentially read out, and the voice waveform samples of each section are sequentially read out from the conversation file 202 and played back, so that the user can listen to them.

The conversation data retrieval unit 1205 based on the user's uttered voice causes the user to utter (for example, "extract my conversation."), And the conversation data shown in FIG. In the same manner as in the case of the person classification device, extraction of each vocal section, frame division, calculation of a linear prediction coefficient group / PARCOR coefficient group / LPC cepstrum coefficient group, addition of a vowel label for each frame in the vocal section, etc. Perform processing. After that, the search unit 1205 calculates, for example, the calculated LPC for each conversation file, each speaker, and each vowel.
Speaker-specific and vowel-specific LPC cepstrum statistical information file 2 corresponding to the cepstrum coefficient group and each conversation file 202
The Euclidean distance with the LPC cepstrum coefficient set stored in 19 is calculated, and the speaker identification tag corresponding to the speaker having the smallest average Euclidean distance of five vowels is extracted. Conversation file 20 to which the speaker identification tag belongs
2 is also extracted at the same time. Then, the search unit 1205 not only presents the extracted conversation file name, but also extracts the voice of the utterance section corresponding to the conversation file 202 and the speaker identification tag in the case of the operation of the uttered portion search unit 1203 of the specific speaker. Speak or present in the same manner as.

As described above, the speaker classification result retrieving apparatus according to the present embodiment is powerful and flexible with respect to the conversation file-based speaker classification database 216 created by the speaker classification apparatus shown in FIG. It is possible to construct a system that can perform high-level search / presentation, etc. and can sufficiently respond to requests for multimedia. The search / presentation process shown here is based on the conventional simple speaker identification / presentation process.
It cannot be realized in the verification system.

Further, as described above, the speaker classification apparatus implemented by the present invention does not require a special dictionary, but it can be combined with a system using a specific dictionary. For example, in a voice recognition system, by combining the speaker classification device described above,
It is possible to create a voice dictionary adapted to a plurality of speakers.

Furthermore, as an original process using the speaker classification result by the speaker classification device implemented by the present invention,
The following processing can be considered. That is, if the speaker classification result is combined with basic sound pressure (power) and basic frequency (pitch) information, the speakers can be classified, so that the sound pressure and the basic frequency of each speaker change with time. You can look up. As a result, it is possible to know which utterance is emphasized more than other utterances in the conversation of a speaker. As described above, when the basic acoustic feature quantities are used in combination, it becomes possible to extract the features that generate the utility value only after the speaker classification is completed. For example, by using the average spectral change rate for each utterance, it is possible to know which utterance of one speaker was quicker than the other utterance. Since such information is information linked to emotion information, it is possible to realize a kind of emotion detection system. <Second Embodiment> The second embodiment is a case where the present invention is implemented as a rotary abnormality detection system.

FIG. 13 is a block diagram of a rotating sound classification device according to the second embodiment. First, the sound generated by the rotating system 1301 such as a motor or an engine is heard by the microphone 1302, then digitized by the A / D converter 1303, and alternately recorded in two buffers 1304 or 1305 that switch at regular intervals. To be done.

The acoustic data 1306 read from the buffer in which the recording operation is not performed among the buffers 1304 or 1305 is divided into a plurality of frames having a predetermined length by the frame division unit 1307. As a result, the acoustic frame division file 1308 is created and stored in a storage medium (not shown). The data format of the acoustic frame division file 1308 can be similar to that shown in FIG. 3B, for example, and the number of frames and the start sample position of each frame are stored.

The spectrum analysis unit 1309 extracts frequency component values of a plurality of frequency bands from the acoustic data 1306 corresponding to each frame position indicated by the acoustic frame division file 1308 by, for example, bandpass filtering, and extracts the frequency component values. The information is stored as an acoustic spectrum analysis file 1310 in a storage medium (not shown).

The above-described operation is performed on the acoustic data 1306 which is successively input via the buffer 1304 or 1305 using the rotary system 1301 as a sound source under various circumstances.

The rotation speed extraction unit 1311 sequentially inputs the acoustic spectrum analysis file 1310 for an appropriate time (for example, one operating time of the rotation system 1301), and for example, the rotation system is started from the frequency component having the maximum value for each frame. The rotation speed of 1301 is detected.

The rotational speed selection unit 1312 sequentially inputs the acoustic spectrum analysis file 1310 for an appropriate time (for example, one operating time of the rotation system 1301), and the rotational speed extraction unit 1311 extracts each frame. By adding a label indicating the number of revolutions, the number of revolutions label file 13
13 is created and stored in a storage medium (not shown).

The acoustic clustering unit 1314 collects each acoustic spectrum analysis result in the acoustic spectrum analysis file 1310 corresponding to the frame to which each rotational speed label is attached, for each rotational speed label stored in the rotational speed label file 1313. The rotation sound classification database 1315 is created by clustering. This rotating sound classification database 1315 has an acoustic file 131.
6, cluster-specific pronunciation part information file 1317, and cluster-specific / rotational-frequency-specific spectrum analysis statistical information file 1
318 etc. are registered. For the clustering method, the same method as in the speaker classification device of FIG. 2 can be applied by replacing the vowels with the number of rotations.

The sound file 1316 is a file obtained by integrating the sound data 1306 for the appropriate time. The pronunciation part information file 1317 for each cluster is
It indicates to which sounding section on the acoustic file 1316 each cluster generated by the acoustic clustering unit 1314 corresponds.

The spectrum analysis statistical information file 1318 for each cluster / rotation number includes the acoustic spectrum analysis result (each frequency component output value of the bandpass filtering group) representing each cluster and each rotation number for each cluster and each rotation number. Etc.) are stored.

By utilizing the rotating sound classification database 1315 created by the rotating sound classifying apparatus shown above, for example, rotating sounds can be classified and analyzed independent of the number of rotations, and abnormal sounds can be detected. It can be easily performed. <Third Embodiment> A third embodiment is similar to the second embodiment in that the present invention is implemented as a rotary abnormality detecting system.

FIG. 14 is a block diagram of a rotating sound classification device according to the third embodiment. In FIG. 14, the parts given the same numbers as in FIG. 13 have the same functions as in FIG. The configuration of FIG. 14 is different from the configuration of FIG.
The rotation number is not extracted by the rotation number extraction unit 1311 from the acoustic spectrum analysis file 1310, but the rotary encoder 1401 located close to the rotation system.
From A / D converter 1402, buffers 1403 and 1
This is a point performed using the rotation speed data 1405 extracted via 404.

The phase of the rotation speed data 1405 is synchronized with the phase of the acoustic data 1306, and the rotation speed data 140
5 is divided into frames by a frame division unit 1406 similar to the frame division unit 1307, and is retained as a rotation speed frame division file 1407. The rotation number for each timing stored in this file is read by the rotation number-based sorting unit 1312. <Other Embodiments> The three embodiments of the present invention have been specifically described above, but the present invention is not limited to the above-described configuration.

In general, the present invention is, for example, as shown in FIG. 1, for each vowel when the present invention is implemented as a speaker classification processing system. When the mechanical system is realized as a detection system and generates sound in synchronization with it in a certain cycle, for example, the present invention is realized as a mechanical sound abnormality detection system at every predetermined timing in the cycle. When a certain sound is generated when a certain operating condition is generated, the acoustic feature amount or the acoustic signal is selected for each operating condition, such as for each operating condition, and then the acoustic signal generation condition is selected. If the configuration is such that each classification is performed, it can be applied to a very wide technical field.

[0104]

According to the present invention, under conditions independent of the acoustic signal generation conditions, without using a specific dictionary or the like,
It is possible to automatically classify the acoustic feature amount and thus the acoustic signal.

Then, using the classification information obtained by this classification, various data processing such as retrieval / presentation of the classification result or other data processing using the classification result can be performed.

As a more specific effect of the present invention, when the present invention is implemented as a speaker classification processing device, the utterances in each conversation sequentially input by the user are automatically classified one by one for each speaker. In this case, no special dictionary is required, it is not affected by long-term changes in the characteristics of the speaker, and there are no restrictions on the type of speaker.

In this case, a vowel label is given to each frame in the utterance section, and classification such as clustering is performed for each vowel between the utterance sections, thereby classifying speakers in a state independent of utterance content, that is, phonology. It becomes possible to do.

Further, since only the spectrally stable section on the conversation file is used as the vocal section in which the vowel extraction is performed, it is possible to greatly reduce the vowel extraction error.

On the other hand, with the speaker classification processing apparatus according to the present invention, a powerful and flexible search / presentation, etc. can be performed on the classification result database, and a system capable of sufficiently responding to requests for multimedia can be provided. It is possible to build. The search / presentation processing disclosed in the present application cannot be realized at all in the conventional simple speaker identification / collation system.

Further, the speaker classification processing device according to the present invention is
It does not require a special dictionary, but can be combined with a system that uses a specific dictionary. For example, when applied to a voice recognition system, it is possible to create voice dictionaries adapted to a plurality of speakers.

In addition, the following processing can be considered as an original processing using the speaker classification result according to the present invention.
That is, if the speaker classification result is combined with basic sound pressure (power) and basic frequency (pitch) information, the speakers can be classified, and thus the sound pressure and the basic frequency of each speaker change with time. It becomes possible to investigate. As a result, it is possible to know which utterance is emphasized more than other utterances in the conversation of a speaker. As described above, when the basic acoustic feature quantities are used in combination, it is possible to extract the features that are not useful until the speaker classification is completed. For example, by using the average spectrum change rate for each utterance, it is possible to know which utterance of a speaker was quicker than the other utterance. Since such information is information linked to emotion information, it is possible to realize a kind of emotion detection system.

[Brief description of drawings]

FIG. 1 is a basic principle block diagram of the present invention.

FIG. 2 is a configuration diagram of a speaker classification device in the first embodiment.

FIG. 3 is a diagram showing a data format of each file in the first embodiment.

FIG. 4 is a diagram showing a data format of a conversation file-based speaker classification database in the first embodiment.

FIG. 5 is an explanatory diagram of a vocalization segment extraction unit.

FIG. 6 is a diagram showing an example of a distance distribution between identical vowels.

FIG. 7 is a diagram showing an example of a same-speaker determination membership function.

FIG. 8 is a diagram showing a relationship between a vocalization section and an average LPC cepstrum coefficient set for each vowel.

FIG. 9 is a diagram showing an example of a vowel similarity similarity evaluation matrix between utterance sections.

FIG. 10 is a diagram showing an example of an evaluation matrix of similarity between occurrence sections.

FIG. 11 is a configuration diagram of a speaker classification result presentation device according to the first embodiment.

FIG. 12 is a configuration diagram of a speaker classification result search device according to the first embodiment.

FIG. 13 is a configuration diagram of a rotating sound classification device according to a second embodiment.

FIG. 14 is a configuration diagram of a rotating sound classification device according to a third embodiment.

Claims (16)

[Claims] 1. An acoustic feature quantity extraction means for extracting an acoustic feature quantity indicating an acoustic feature of an acoustic signal, and an acoustic feature quantity extracted by the acoustic feature quantity extraction means, the acoustic signal corresponding to the acoustic feature quantity. Selecting means for selecting each generation condition of the sound characteristics, and the acoustic feature quantity selected by the selecting means is classified by evaluating the degree of similarity between the acoustic feature quantities based on the classification for each generation condition of the acoustic signal. And a classification unit that creates classification information including at least the correspondence between the classification result and the acoustic signal. 2. An acoustic feature amount indicating an acoustic feature of an acoustic signal is extracted, the extracted acoustic feature amount is selected for each generation condition of an acoustic signal corresponding to the acoustic feature amount, and the selected acoustic feature amount is selected. The acoustic feature quantity is classified by evaluating the degree of similarity between the acoustic feature quantities on the basis of the classification for each generation condition of the acoustic signal, and classification information including at least a correspondence relationship between the classification result and the acoustic signal. An acoustic classification processing method characterized by: 3. A selection unit that selects an acoustic signal for each generation condition thereof, an acoustic feature amount extraction unit that extracts an acoustic feature amount indicating an acoustic feature of the acoustic signal, and an acoustic signal selected by the selection unit. Corresponding to the acoustic feature amount extracted by the acoustic feature amount extraction means,
A classifying unit that evaluates and classifies the degree of similarity between the acoustic feature quantities on the basis of the classification for each generation condition of the acoustic signal, and creates classification information including at least a correspondence relationship between the classification result and the acoustic signal. An acoustic classification processing device comprising: 4. An acoustic signal is selected for each generation condition thereof, an acoustic feature amount indicating an acoustic feature of the acoustic signal is extracted, and the acoustic feature amount corresponding to the selected acoustic signal is
Based on the classification for each generation condition of the acoustic signal, the degree of similarity between the acoustic feature quantities is evaluated and classified, and classification information including at least a correspondence relationship between the classification result and the acoustic signal is created. An acoustic classification processing method characterized by. 5. An audio classification processing device according to claim 1 or 3, or a data processing means for performing data processing using classification information created by the audio classification processing method according to claim 2 or 4. A sound classification processing device characterized by. 6. Data processing is performed using the classification information created by the sound classification processing device according to claim 1 or 3 or the sound classification processing method according to claim 2 or 4. Classification processing method. 7. A voice feature amount extraction means for extracting a voice feature amount indicating a voice feature of a voice signal, and a vowel utterance in the voice signal based on the voice feature amount extracted by the voice feature amount extraction means. A vowel extraction unit that extracts a frame and the type of vowels uttered in the frame; and a vowel utterance frame extracted by the vowel extraction unit by the vowel feature extraction unit that extracts the voice feature amount for each vowel. Based on the classification, the degree of similarity of the voice feature amount is evaluated, the voice feature is classified into one or more speakers, and classification information including at least the correspondence between the classification result of the speaker and the voice signal is created. A speaker classification processing device comprising: 8. A voice feature amount indicating a voice feature of a voice signal is extracted, and a vowel utterance frame in the voice signal and a type of a vowel uttered in the frame are extracted based on the extracted voice feature amount. The voice feature amount in the extracted vowel utterance frame is classified into one or more speakers by evaluating the degree of similarity of the voice feature amount based on the classification for each vowel, and the speaker A speaker classification processing method comprising: creating classification information including at least a correspondence between the classification result and the voice signal. 9. A voicing section extracting means for extracting a spectrally stable section from a voice signal as a voicing section, and each voicing section extracted by the voicing section extracting means,
Frame dividing means for dividing each utterance section into frames having a predetermined time length; and for each utterance section extracted by the utterance section extracting means and for each frame divided by the frame dividing means A voice feature amount extraction unit that extracts a voice feature amount indicating a feature, and a vowel utterance frame in each utterance section and a vowel uttered in the frame based on the voice feature amount extracted by the voice feature amount extraction unit. A vowel extraction unit for extracting the type of the vowel, and the vowel extraction unit for extracting the voice feature amount extracted by the voice feature amount extraction unit in the vowel utterance frame in each utterance section extracted by the vowel extraction unit. In addition, by evaluating the degree of similarity of the voice feature amount and classifying between the utterance sections, at least one utterance section is provided. Classified in correspondence with the speaker, speaker clustering processing apparatus characterized by comprising: a classification means for creating classification information that includes at least a correspondence relationship between the speech signal and the classification results of 該話's a. 10. A spectrally stable section is extracted from a voice signal as a utterance section, each utterance section is divided into frames having a predetermined time length, and each utterance section is divided into frames. For each frame, a voice feature amount indicating a voice feature of the voice signal is extracted, and based on the extracted voice feature amount, a vowel utterance frame in each utterance section and a vowel sound uttered in the frame are extracted. A type is extracted, and the speech feature amount in the vowel utterance frame in each of the extracted utterance intervals is evaluated based on the classification of each vowel to evaluate the degree of similarity of the speech feature amount and By classifying each of the utterance sections in association with one or more speakers, and creating classification information including at least the correspondence between the classification result of the speakers and the audio signal. Speaker classification processing method according to claim. 11. A speaker classification processing device according to claim 7 or 9, or a data processing means for performing data processing using classification information created by the speaker classification processing method according to claim 8 or 10. A speaker classification processing device characterized by the above. 12. The speaker classification processing device according to claim 7 or 9, or the speaker classification processing method according to claim 8 or 10, the data processing is performed using the classification information created. Speaker classification processing method. 13. The data processing according to claim 11, wherein the data processing is processing for searching for a voice signal uttered by a speaker having an input number of speakers based on the classification information. The described speaker classification processing device or speaker classification processing method. 14. The speaker classification according to claim 11, wherein the data processing is processing for searching a voice signal including a utterance by a predetermined speaker based on the classification information. Processor or speaker classification processing method. 15. The process according to claim 11, wherein the data processing is a process for searching a vocalization portion of a predetermined speaker from a predetermined voice signal based on the classification information. Speaker classification processing device or speaker classification processing method. 16. The data process is a process of searching for a voice signal including a utterance by the same speaker as the speaker of the input voice or a voicing portion in the voice signal based on the classification information. The speaker classification processing device or the speaker classification processing method according to claim 11.