JP2005173007A - Voice analysis processing, voice processor using same, and medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To temporarily regard not only speaker discrimination-difficult voice data as unknown speaker voice data but also to reduce the unknown speaker voice data as much as possible for the speaker discrimination-difficult speaker discrimination-impossible voice data. <P>SOLUTION: Determination of a speaking section lacking an amount of information is reserved, an analysis is completed once, and speaker discrimination precision is enhanced by reprocessing after speaker characteristic information is filled up. Further, characteristics of detected speakers are compared with each other, errors mistaking one speaker as a plurality of speakers are reduced. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a voice analysis process and a voice processing apparatus using the same.

  Conventionally, Japanese Patent Application Laid-Open No. 8-286663 discloses an invention for searching and reproducing desired voice data of a speaker by analyzing the voice and using the result to sort the voice data by speaker. Japanese Patent Laid-Open No. 2000-148189 discloses an invention for recording a state in which a speaker of voice data cannot be identified for voice data that cannot be distinguished by a speaker in practice when performing voice analysis. ing.

  As described above, when speech data is analyzed and speaker discrimination is actually performed, in particular, consideration is not given to speech data that cannot be speaker-discriminated, which is difficult to perform speaker discrimination due to a shortage of information such as short speech data. Met. In the present invention, not only the voice data that cannot be distinguished by such speaker, but the voice data that is difficult to perform speaker discrimination is temporarily treated as voice data with unknown speaker. The problem is to reduce the voice data of unknown speakers as much as possible.

  In addition, there is a problem that discrimination is not performed well and a mistake that discriminates one speaker into a plurality of voices is uttered. It is also an important issue to reduce this mistake as much as possible.

  In order to solve the above-described problems, in the voice analysis processing of the present invention, when distinguishing a speaker who appears, a personal feature of the utterance is extracted each time a person's utterance is detected, and the amount of information is determined in advance. When the condition is not satisfied, the analysis result information is stored as an utterance section with unknown utterance by treating the utterer alone. For the utterance section where the speaker is unknown, the analysis target speech signal is processed to the end, and then the processing is performed again or recursively.

  If an individual feature is extracted and the amount of information satisfies a pre-determined condition, if there is no new speaker, the result of the analysis is that the individual feature of the utterance and the utterance interval are combined as the first speaker as an exception process. It is stored in the information, but if there is a speaker who has already been detected, the individual characteristics of the utterance in this utterance section are compared with the individual characteristics of the utterance of the person who has already been detected. When it is estimated that the person is a person, the individual characteristics of the utterance and the utterance section are combined and stored in the analysis result information as a new person.

  On the other hand, when it is estimated that the personal characteristics of the utterance in the utterance section are the same as the already detected person, the utterance section is added to the analysis result information.

  The individual feature of the utterance described above includes at least a pair of acoustic feature information for all vowels existing in the utterance section. This is because it is well known that individual features of utterances appear in vowels, and comparison processing is performed for vowels existing in the detected vowel interval for each vowel of all persons already detected. It is possible to determine whether the detected utterance section belongs to a new person or the detected utterance section by calculating the distance between the pair of characteristic features and comparing the value with a predetermined threshold. is there.

  Furthermore, a pair of acoustic feature information with respect to a vowel as a personal feature is specifically a pair of formant information or cepstrum information with respect to all vowels existing in the utterance section, or all pairs existing in the voice pitch and utterance section. It can be expressed by a pair of formant information for vowels, or a pair of formant information or cepstrum information for all vowels existing in the voice pitch and the utterance section.

  This is because there is a resonance frequency of the vocal tract called vowel formant, which is also used for personal authentication. This is because vowels are recognized, formant information at each stable point is obtained for each vowel, and this is useful for individual discrimination.

  As for cepstrum, as with formants, analysis results reflecting the resonance frequency of vocal tract vocalizations are obtained, which is useful for individual discrimination.

  The stable point is the vicinity where the vowel is determined and the energy is maximum near the center.

  These voice pitch, formant, and cepstrum are used in known voice recognition processing.

  The comparison process is performed by obtaining a known formant distance and cepstrum distance and comparing them with a predetermined threshold value.

  The voice pitch reflects the pitch of sounds like men, women, children, etc., and is useful for discriminating people with different vocal cord vibrations even at the same vocal tract resonance frequency, so the difference between formant distance and cepstrum distance and pitch frequency. It is also possible to obtain a distance based on a weighted sum with the absolute value of and to compare this with a predetermined threshold value.

  By the way, the predetermined condition of the information amount of the personal feature when handling a single speaker as an utterance section with unknown utterance is a threshold value below a certain vowel type present in the detected utterance section, or It is assumed that the combination of the minimum necessary vowel types is insufficient.

  For example, a significant formant distance or cepstrum distance cannot be calculated if the utterance interval is short and the number of vowels included in the vowel is small, or the vowels that exist are similar in nature and cannot be used for comparison. .

Therefore, if the vowel type included in the utterance section or the combination of vowel types cannot satisfy the conditions for calculating a significant formant distance or cepstrum distance, the utterance determination process is postponed as an utterance section with an unknown speaker. To do.
The reason for postponing is that the personal characteristics of the detected speaker are enhanced as the process proceeds.

  When it is determined that the detected utterance section is a person who has already been detected, and there is a vowel missing in the personal feature, if the vowel exists in the detected vowel section, the acoustic of the vowel This is realized by memorizing pairs of feature information in individual features.

  And, after performing the analysis process of all speech, by further comparing the individual characteristics of the already detected person and the individual characteristics of the single speaker as the utterance section of the unknown speaker, as the utterance section of the unknown speaker A process of remapping a single speaker to a person who has already been detected is performed, but this may be performed alone or recursively.

  In addition, regarding the problem that one speaker is misidentified by multiple speakers, the individual characteristics of detected persons are compared, and analysis information is integrated as one speaker if the distance is below a certain threshold. To do. Specifically, the utterance section information that has been composed with another utterer is merged with the utterance section information of one speaker.

  The re-processing of the utterance section with unknown utterer and the utterance section information of the utterance section of the one utterer may be preceded with respect to the order of the merging process.

  According to the present invention, voice data that is difficult to perform speaker discrimination is treated as voice data with unknown speaker temporarily for voice data that is difficult to perform speaker discrimination, and speaker discrimination is possible. The feature information for speaker discrimination is learned using the analysis result of speech data, and after the analysis of a group of speech signals, the feature information enriched by this learning is used to temporarily identify the speaker unknown. It becomes possible to reduce the voice data of unknown speaker by re-processing the voice data that was supposed to be voice data. Furthermore, by comparing speaker characteristics again after speaker discrimination, it is possible to reduce mistakes that misidentify one speaker into multiple.

  By using these processing results, it has become possible to obtain valuable information for realizing effective functions such as listening to the utterance section of only the designated speaker.

  Examples of the present invention will be described in detail below.

  In the embodiment, a description will be given of performing voice segment information generation for each speaker by discriminating the voices that appear without registering prior personal identification information in the voice analysis processing.

  The concept of the embodiment of the present invention, as shown in the conceptual diagram of FIG. 1, utterance is performed by determining the speaker based on the utterance section detection process, the utterance feature extraction process, the utterance feature comparison process, and the result. It consists of a process of adding information to the speaker information DB and the utterance section information DB for each speaker.

  FIG. 2 shows a flow from the utterance section detection processing in this embodiment to generation of utterance section information for each utterer and storing it in the attribute information and analysis result information storage means.

  First, in S201, an audio signal is read.

  In the flow of FIG. 2, the audio signal is read into the memory at one time, but naturally, an embodiment in which the memory buffer is sequentially read as necessary can be easily considered.

  A known technique is used for the utterance section detection processing in S202. The audio signal is divided into a plurality of utterance sections.

  In S203, every time a speech segment head is detected and a speech segment end point is detected, a feature useful for individual discrimination of speech is extracted from the speech signal corresponding to the speech segment in the speech feature extraction processing of S204.

  Regarding vocalization, it is known that there is a resonance frequency of the vocal tract called formant of vowels, and it is also used for personal authentication. Therefore, vowels are recognized, formant information at the stable point is obtained for each vowel, and this is stored for individual discrimination.

  For formant estimation, a linear prediction analysis (LPC) based on an all-pole type speech generation model is performed, and the poles of the vocal tract transfer coefficient are obtained, thereby using a known technique such as obtaining the resonance frequency. good.

  That is, the data schema shown in FIG. 5 is taken as an example of the utterance feature.

  A pair of formant information is stored for each of the utterance section and the five vowels of “Aiueo” existing in the utterance section, and further, the voice pitch frequency is stored. Of course, if five vowels are not prepared during the utterance, only the existing vowels are described.

  The utterance section field in FIG. 5 includes a pair of a start position and an end position as shown in FIG. 6, and time expression is possible as a position expression method.

  In addition, regarding the formant information pairs, generally, up to the third formant can be used to express the nature of the vowel. Therefore, as shown in FIG. 7, from the three frequencies of the first, second and third formant frequencies. Is stored for each vowel in FIG.

  As a voice signal to be analyzed, a stable section of the pitch frequency and a stable pitch frequency may be used as a stable point of the vowel.

  However, not all types of vowels are present in the utterance section, and accurate comparison processing cannot be performed when there are few types of vowels or when the combination of vowels that are similar in nature and effective for comparison cannot be obtained.

  Therefore, in S205, if it is determined that the type of vowel existing in the detected utterance section is less than a certain threshold value or insufficient for the combination of the minimum required vowel types, a single utterance as an utterance section in which the speaker is unknown is determined in S206. Is stored in the analysis result information.

  If it is determined that the vowel type existing in the utterance section detected in S205 satisfies a certain threshold or a combination of the minimum required vowel types, it has already been detected using the utterance feature extraction in S207. It is determined whether the person is different from the already detected person by comparing the personal characteristics of the person's utterance shown in FIG. 8, specifically, the sum of the distances between the formant pair information.

  Regarding the utterance feature comparison processing of S207, the distance between pairs of formant information is calculated for only the vowels existing in the detected utterance section for each of the vowels of all the persons already detected, and the minimum value thereof is obtained. In S208, if this minimum value is equal to or less than a predetermined threshold, it is determined that the voice is of a person corresponding to the minimum distance. If this minimum value is greater than the threshold, it is determined that the person is a new person.

  If it is estimated that the same person as the person already detected in S208, the utterance section is additionally recorded in the utterance section information record of FIG. 9 corresponding to the person's speaker ID in S209.

  In addition, when there is a vowel whose formant information is missing in the personal characteristics of a person that has already been detected, and the missing vowel is included in a newly detected utterance section, the formant information of that vowel Enrich personal characteristics by memorizing the characteristics.

  On the other hand, if it is estimated that the person is not already detected in S208, a new speaker ID is issued as a new person in S211 and a new record is generated in the speaker information DB shown in FIG. Are recorded, and a new record is generated in the utterance section information DB of FIG. 9 with the utterer ID, and the utterance section is stored.

  Of course, since the person is not detected at the initial stage, if the vowel type satisfies a certain threshold or a combination of the minimum required vowel types, the individual characteristics of the utterance and the utterance interval are simply set as a new person. Together, it is stored in the analysis result information.

  By the way, as shown in FIG. 8 and FIG. 9, the individual characteristic information of the utterance and the utterance section information are separated, but there is naturally a method of merging these into one record to make a record for one speaker ID.

  After performing the analysis processing of all speech signals in this way, in S212, again compared with the individual feature of the person already detected for the utterance section treated as a single utterer as the utterance section with unknown speaker The process of remapping to the already detected person is performed.

  The reason is that personal characteristics are enriched and more accurate determination can be made than when the first utterance of the utterance section that has already been detected is compared.

  By performing this process alone or recursively, the number of voice segments treated as a single speaker is reduced as the voice segment whose voice is unknown.

  Details of the processing flow of S212 will be described with reference to FIG.

  After analyzing all speech signals, it is determined in S301 whether or not there is an unprocessed utterance unknown speech section. If not, the process ends. If there is, the process temporarily stores it in S302 in FIG. The utterance features of the utterance section are read and compared with the utterance features of all detected utterers in S303.

  In S304, the minimum value of the distance obtained in the comparison process of S303 and the detected speaker corresponding to the minimum value are determined, the distance is compared with a predetermined threshold value, and if the distance is equal to or less than the threshold value, a Y determination is performed. In S305, the utterance section of the detected speaker is added to the utterance section information of the detected speaker, and the missing information is added to the utterance feature of the detected speaker in S306, and the unprocessed utterance is again processed in S301. It is determined whether or not there is an unspoken utterance section, and the process is repeated until there are no unprocessed utterance sections with unknown speaker. On the other hand, if the distance is larger than the threshold value in S304, an N determination is made, and in S307, it is temporarily stored as an utterance section whose speaker is unknown.

  Of course, if the process of FIG. 3 is performed recursively, the utterance section whose utterer is unknown may be determined to be the utterance of the already detected speaker in the previous process because of S306, and the newly detected utterer's utterance Naturally, a method of performing recursive processing until no longer appears is also conceivable.

  Further, after the processing of S212, it is possible to compare the personal characteristics between the last detected persons in S213 of FIG. 2 and eliminate the mistake of dividing one person by apologizing to a plurality of persons.

  Details of the processing flow of S213 will be described with reference to FIG.

  In S401, the utterance features of the detected utterers are read from the utterer information DB, and all these utterance features are compared with each other.

  In S403, the minimum distance among the mutual comparison distances obtained in S402 is searched for and two corresponding detected speakers are obtained.

  If the mutual comparison distance is equal to or smaller than a predetermined threshold value in S404, the information of the two detected speakers is merged in S405 to be one detected speaker.

  Specifically, the other utterance feature is merged with the utterance feature of the utterer information DB record of one of the detected utterers, and the utterance interval information of the utterance interval information person DB record of one of the detected utterers is Are merged, and then the other speaker information DB record and the speech section information person DB record are deleted.

  After the configuration and utterance characteristics of the detected speaker are updated in this manner, the above process is repeated again until the minimum mutual comparison distance exceeds a predetermined threshold value in S404.

  This corrects a mistake that a person is apologized to a plurality of persons.

  Now, as an example of the utterance feature comparison process of S207, an example of distance calculation using pairs of formant information and pitch information will be described.

  Incidentally, as a vowel to be analyzed for formant information and pitch frequency, a section close to a smoothed energy peak and having a stable pitch frequency is used.

  In general, it is said that the use of up to the third formant can represent the characteristics of the vowels. Therefore, in this embodiment, an evaluation function considering the third formant is introduced.

  The primary, secondary, and tertiary formants of each vowel in the utterance section of interest, and the primary, secondary, and secondary vowels of the detected speaker that are stored in the speaker information DB shown in FIG. By reading the third formant information and comparing these features, it is determined which speaker is uttered or whether it is new or unknown.

  According to this evaluation function, the closer the utterance individual characteristics are, the closer the value is to zero, and zero when they completely match.

但し、Vow[i][j]は検出した発声区間における母音iのｊ次フォルマント周波数の値でありfは発声区間におけるピッチ周波数であり、
fref Vowref[i][j]は既に検出された人物の母音iのｊ次フォルマント周波数の値でありfrefはその人物のピッチ周波数である。

Where Vow [i] [j] is the value of the j-th formant frequency of the vowel i in the detected utterance interval, and f is the pitch frequency in the utterance interval,
fref Vowref [i] [j] is the value of the j-th formant frequency of the vowel i of the person already detected, and fref is the pitch frequency of the person.

      Wi: A weight corresponding to the degree of personal feature expression of each vowel, and a value that is large enough to reflect the personal feature.

Wf0: Weight for pitch frequency
i: 0-4 Subscript to indicate vowel distinction
j: 0-2 Subscript representing j-th formant WTotal: Sum of vowel weights Wi in which formant information exists in common Also, | Vow [i] [j] −Vowref [i] [j] | Is calculated only when Vow and Vowref exist in common, and the formant comparison term is normalized with WTotal, which is the sum of the weights Wi of vowels with this formant information in common.

  Using the evaluation function (1.1), an evaluation value group is obtained for the utterance personal characteristics of all the persons detected so far and the utterance personal characteristics of the newly detected generation section, and the minimum value Emin is obtained to determine the utterance candidate. Further, Emin is compared with the threshold value ε0, and when Emin <ε0 is satisfied, the utterance candidate is determined as the utterance person, and the utterance section is added to the analysis result information.

  In addition, if there is a vowel that is missing in the utterance personal feature of the determined utterance candidate and the vowel that is missing is included in the current utterance, information on the vowel is additionally stored in the utterance personal feature. To enhance the individual characteristics of the utterance.

  On the other hand, if Emin <ε0 is not satisfied, it is determined that the person is a new utterance person, new utterance person information is generated in the analysis result information, and the utterance personal feature and the utterance section are stored.

  By performing the above processing every time a person's utterance is detected, it is possible to refer to the utterance section in units of all the characters and to store the individual characteristics of the utterance in the analysis result information.

(Other examples)
In this embodiment, the explanation has been given by taking Japanese as an example. However, in a foreign language, if there are many vowels of each language, or if there are many vowels, only vowels useful for human discrimination are selected. Of course, it is conceivable to perform the processing according to the present invention.

  Moreover, although the embodiment which memorize | stores the characteristic of an utterance area in utterance area information DB was described, of course, there is also a method of discarding this information and not storing it in the DB when the utterer can be identified by comparing the utterance characteristics.

  However, it is necessary to temporarily store an utterance section whose speaker is unknown, temporarily in a form similar to the schema of the utterance section information DB.

  Further, in this embodiment, an example of distance calculation using pairs of formant information and pitch information is given, but naturally, an embodiment in which all formant information including DB schema is replaced with cepstrum information is also possible.

  Furthermore, after performing the process of remapping to the already detected person by comparing with the personal characteristics of the already detected person for the utterance section that has been treated as a single speaker as the utterance section of the unknown speaker, Although an example has been described in which individual characteristics between detected persons are compared and a process of eliminating a mistake in which one person is apologized to a plurality of persons is eliminated, an example in which this order is reversed is also conceivable.

  Further, after comparing the detected individual characteristics between the persons and eliminating the mistake of dividing one person apologizing to a plurality of persons, it is treated as a single speaker as an utterance section with unknown speaker. After performing the process of remapping to the already detected person by comparing with the personal feature of the already detected person for the utterance section, the individual feature between the detected persons is again compared with each other Naturally, an example of mutual processing, such as a process of eliminating mistakes by apologizing a person to a plurality of persons, is considered.

It is a figure showing the conceptual diagram of the audio | voice analysis process of this invention. It is a figure showing the speech detection processing processing flow in this invention. It is a figure showing the re-processing flow of the utterance area in which a speaker is unknown in this invention. It is a figure showing the processing flow which corrects the mistake which misidentifies one speaker in this invention in multiple. It is a figure showing the example of the utterance area information schema in this invention. It is a figure showing the example of the schema of the utterance area field in this invention. It is a figure showing the example schema of the formant field of each vowel in this invention. It is a figure showing an example of the schema of speaker information DB in this invention. It is a figure showing an example of the schema of utterance section information DB classified by speaker in this invention.

Claims (18)

A means for detecting a person's utterance and its interval;
Means for extracting personal features of utterances;
In the process of storing the utterance interval information for each individual,
Each time a person's utterance is detected, the personal features of the utterance are extracted, and if the amount of information does not satisfy a predetermined condition, it is stored in the analysis result information as a single utterer as an utterance section with an unknown speaker If a personal feature is extracted and the amount of information satisfies a predetermined condition, it is estimated that it is different from the already detected person by comparing it with the personal feature of the voice of the person who has already detected it. If this is the case, the individual characteristics of the utterance and the utterance interval are stored in the analysis result information as a new person,
Alternatively, when it is estimated that the person is the same as the already detected person, analysis result information that can be referred to in units of all characters is stored by adding the utterance section to the analysis result information. .   In Claim 1, the personal feature of utterance includes at least a pair of acoustic feature information for all vowels existing in the utterance interval, and the comparison processing is in the utterance interval detected for each of the already detected vowels of all persons. Calculating the distance between pairs of acoustic feature information for the vowels present in the vowel and comparing the value with a predetermined threshold, the detected utterance interval belongs to a person who has already been detected or a new A voice processing apparatus characterized by determining whether a person is a person.   In claim 2, the pair of acoustic feature information for vowels as individual features is a pair of formant information or cepstrum information for all vowels existing in the utterance interval, or for all vowels existing in the voice pitch and utterance interval. A speech processing apparatus characterized in that it is a pair of formant information, or a pair of formant information or cepstrum information for all vowels existing in a speech pitch and a speech section.   In Claim 1, the predetermined condition of the amount of information of personal characteristics when handling a single speaker as an utterance segment of unknown utterance is a threshold value that includes the type of vowels present in the detected utterance segment A speech processing apparatus characterized by being deficient in a combination of the following or the minimum necessary vowel types.   The utterance detected when the detected utterance section is determined to be a person who has already been detected and there is a missing vowel in the personal feature according to claim 1, claim 2, claim 3, and claim 4 A speech processing apparatus characterized by enriching personal features by storing a pair of acoustic feature information of the vowels in a personal feature when the vowel exists in a section.   In claim 1, claim 2, claim 3, claim 4, claim 5 and claim 7, after analyzing all voices, the personal characteristics of the person already detected and the utterance of unknown speaker A voice characterized by performing remapping processing alone or recursively to a person who has already detected a single speaker as an utterance section with an unknown speaker by comparing individual characteristics with a single speaker as a section Processing equipment.   In claim 1, claim 2, claim 3, claim 4, claim 5, and claim 6, after analyzing all voices, the individual characteristics of the detected persons are compared and An audio processing apparatus that performs a process of correcting an error in which a person is divided into a plurality of persons. A means for detecting a person's utterance and its interval;
Means for extracting personal features of utterances;
In the process of storing the utterance interval information for each individual,
Each time a person's utterance is detected, the personal features of the utterance are extracted, and if the amount of information does not satisfy a predetermined condition, it is stored in the analysis result information as a single utterer as an utterance section with an unknown speaker If a personal feature is extracted and the amount of information satisfies a predetermined condition, it is estimated that the person is different from the already detected person by comparing it with the personal feature of the utterance of the person who has already detected it. If this is the case, the individual characteristics of the utterance and the utterance interval are stored in the analysis result information as a new person,
Alternatively, when it is estimated that the person is the same as the already detected person, the analysis result information that can be referred to in units of all characters is stored by adding the utterance section to the analysis result information. Processing method.   9. The personal feature of utterance includes at least a pair of acoustic feature information with respect to all vowels existing in the utterance section, and the comparison processing includes in the detected utterance section for each of the vowels of all persons already detected. Calculating the distance between pairs of acoustic feature information for the vowels present in the vowel and comparing the value with a predetermined threshold, the detected utterance interval belongs to a person who has already been detected or a new An audio signal analysis processing method characterized by determining whether a person is a person.   In claim 9, the pair of acoustic feature information for vowels as individual features is a pair of formant information or cepstrum information for all vowels existing in the utterance interval, or for all vowels existing in the voice pitch and utterance interval. A speech signal analysis processing method characterized by being a pair of formant information and / or a pair of formant information or cepstrum information for all vowels existing in a speech pitch and a speech section.   9. The predetermined condition for the amount of information of personal characteristics when handling a single speaker as an utterance segment with unknown utterance is a threshold value that includes the type of vowels present in the detected utterance segment A speech signal analysis processing method characterized by being insufficient for a combination of the following or minimum required vowel types.   The utterance detected when the detected utterance section is determined to be a person who has already been detected and there is a missing vowel in the personal feature according to claim 8, claim 9, claim 10, and claim 11 An audio signal analysis processing method characterized in that, when a vowel exists in a section, the personal feature is enhanced by storing a pair of acoustic feature information of the vowel as a personal feature.   In claim 8, claim 9, claim 10, claim 11, and claim 14, after analyzing all voices, the personal characteristics of the person already detected and the utterance of unknown speaker A voice characterized by performing remapping processing alone or recursively to a person who has already detected a single speaker as an utterance section with an unknown speaker by comparing individual characteristics with a single speaker as a section Signal analysis processing method.   In claim 8, claim 9, claim 10, claim 11, claim 12 and claim 13, a comparison is made between the individual characteristics of the detected persons, and an error is determined by dividing one person into a plurality of persons. An audio signal analysis processing method characterized by performing correction processing.   8. A computer program for instructing an operation of a computer as the voice processing apparatus according to claim 1.   15. A computer program for instructing an operation capable of realizing the audio signal analysis processing method according to claim 8 by a computer.   8. A computer-readable storage medium storing program code for operating a computer as the voice processing apparatus according to claim 1. 15. A computer-readable storage medium storing a program code capable of implementing the audio signal analysis processing method according to claim 8 by a computer.

Images