JP5028599B2 - Audio processing apparatus and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve problems with a conventional speech processing device that the speech processing (singing voice evaluation and the like) meeting the speaker characteristics of an evaluation target object which is a speaker cannot be performed, and as a result thereof, the speech processing of high accuracy is not possible. <P>SOLUTION: The speech processing meeting the speaker characteristics can be performed by the speech processing device including a speech accepting section which accepts a speech, a first sampling frequency storage section which stores a first sampling frequwncy, a vocal tube length normalizing parameter storage section which stores the vocal tube length normalizing parameter being the information relating to the conversion rate of the sampling frequency, a vocal tube length normalization processing section which obtains second speech data by subjecting the speech accepted by the speech acceptance section to sampling processing at the second sampling frequency calculated with the vocal tube length normalizing parameter and the first sampling frequency as parameters, and a speech processing section which processes the second speech data. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a speech processing apparatus that evaluates input speech and recognizes input speech.

  As a conventional technique, there is the following voice processing apparatus (see Patent Document 1). This speech processing device is a language learning device, and the language learning device displays the pronunciation of the role selected by the learner by comparing it with the reference data, scoring it according to the degree of coincidence, and displaying an appropriate next screen depending on the score. It is a device that improves learning efficiency by displaying automatically. The conventional speech processing apparatus is configured such that the input speech signal is analyzed by speech recognition technology, and then the learner's pronunciation spectrum and inflection appear in the learner pronunciation display box. In the conventional technique, the standard sound data is compared with the learner's pronunciation spectrum and intonation, and the score is displayed.

  Further, as a conventional technique, there is the following voice processing apparatus (see Patent Document 2). The voice processing device is a singing voice evaluation device, and the singing voice evaluation device extracts a frequency component of the singing voice and a fundamental frequency component and a harmonic frequency component from the extracted frequency component, respectively. Specific frequency component extraction means, and evaluation means for calculating an evaluation value indicating evaluation of the singing voice according to the ratio of the harmonic frequency component to the fundamental frequency component extracted by the specific frequency component extraction means. And this singing voice evaluation apparatus evaluates the quality of the voice quality appropriately based on the frequency component of the singing voice, and reflects this in the singing voice scoring result, thereby making the singing voice scoring more human sensitive. The aim is to get closer.

Further, as a conventional technique, there is the following voice processing apparatus (see Patent Document 3). This speech processing device is a speech recognition device, which is a speech recognition device that collates an input speech pattern with a standard pattern using the DP method, and uses a standard pattern with the smallest collation distance as a recognition result. The input pattern is divided into phonemes, the variance of the deviation between the duration of each phoneme and the standard duration is calculated, and this is added to the verification distance to correct the distance. Then, the dividing unit divides into phonemes using the collation result, the time length deviation calculating unit calculates the variance of the deviation from the standard duration, and the distance correcting unit corrects the collation distance. In addition, the speech recognition apparatus includes a phoneme selection unit that selects a target phoneme for calculating a time length shift and a word selection unit that selects a target word for distance correction, so that the word recognition performance can be improved. is there.
JP 2003-228279 A (first page, FIG. 1 etc.) JP-A-2005-107088 (first page, FIG. 1 etc.) JP-A-6-4096 (first page, FIG. 1 etc.)

  However, in the conventional techniques of Patent Document 1 and Patent Document 2, speech processing according to the speaker characteristics of the evaluation target person who is a speaker of speech (including singing voice) cannot be performed, and as a result, highly accurate speech Could not process. Specifically, in the conventional technology, for example, the spectral envelope expands or contracts to a high frequency range or a low frequency range due to a difference in the vocal tract length of the evaluation subject, but a conventional pronunciation rating device, singing voice evaluation device, etc. In the speech processing apparatus, the evaluation result varies depending on the expansion and contraction of the spectrum envelope. In other words, in the conventional technology, even with the same skillful pronunciation and singing, the evaluation results of pronunciation and singing differ depending on the vocal tract length of the person to be evaluated, and high-accuracy evaluation cannot be performed.

  In addition, since the voice processing device of Patent Document 1 is configured to display the score by comparing the standard sound data with the learner's pronunciation spectrum and intonation, the accuracy of the similarity between the two is low, and In order to display the score at high speed in real time, a CPU having a very high processing capability and a large amount of memory are required.

  Further, in the speech processing apparatus of Patent Document 1, if there is a silent section, it is considered that the similarity is evaluated low, and the accuracy of the evaluation is low. In addition, it was not possible to detect the occurrence of special events such as phoneme substitution, insertion, or omission.

  Further, for example, in a speech processing apparatus that performs speech recognition processing as shown in Patent Document 3, the spectral envelope expands or contracts due to the difference in the vocal tract length of the evaluation target person, but depending on the speaker characteristics of the evaluation target person Voice recognition processing was not performed, and high-accuracy voice recognition was not possible.

  The voice processing device according to the first aspect of the invention includes a voice receiving unit that receives voice, a first sampling frequency storage unit that stores a first sampling frequency, and vocal tract length normalization that is information relating to a conversion rate of the sampling frequency. A vocal tract length normalization parameter storage unit storing parameters; a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters; On the other hand, the voice processing apparatus includes a vocal tract length normalization processing unit that performs sampling processing to obtain second voice data, and a voice processing unit that processes the second voice data.

  With this configuration, voice processing according to the speaker characteristics of the evaluation target person can be performed.

  The speech processing apparatus according to the second aspect of the present invention stores one or more teacher data, which is data related to the speech to be compared with the first aspect, and is data for one or more phonemes. A teacher data storage unit, a teacher data formant frequency storage unit that stores a teacher data formant frequency that is a formant frequency of the teacher data, and a formant frequency of an evaluation target person who is a voice speaker received by the voice receiving unit And an evaluation subject formant frequency storage unit storing the evaluation subject formant frequency, wherein the vocal tract length normalization parameter is calculated by “evaluation subject formant frequency / teacher data formant frequency”. The vocal tract length normalization processing unit is "the first sampling frequency x vocal tract length normalization parameter" The arithmetic expression to calculate a second sampling frequency, in the second sampling frequency, the audio of the voice accepting unit accepts, performs sampling processing, a voice processing apparatus for obtaining a second audio data.

  With this configuration, it is possible to perform highly accurate speech processing according to the speaker characteristics of the evaluation target person.

  The voice processing device according to the third aspect of the invention further includes a sampling unit that samples the voice received by the voice receiving unit at the first sampling frequency and obtains first voice data with respect to the second invention. Further, the vocal tract length normalization processing unit calculates a second sampling frequency by an arithmetic expression of "the first sampling frequency x vocal tract length normalization parameter", and the second sampling frequency is used to calculate the second sampling frequency. The audio processing apparatus performs sampling processing on one audio data to obtain second audio data.

  With this configuration, it is possible to perform highly accurate speech processing according to the speaker characteristics of the evaluation target person.

  The speech processing apparatus according to the fourth aspect of the present invention stores one or more teacher data, which is data related to the speech to be compared with the first invention, and is data for one or more phonemes. A vocal tract length normalization parameter calculating unit that calculates the vocal tract length normalization parameter based on the teacher data and the voice received by the voice reception unit; The vocal tract length normalization parameter of the normalization parameter storage unit is a voice processing device that is a vocal tract length normalization parameter calculated by the vocal tract length normalization parameter calculation unit.

  With this configuration, it is possible to dynamically perform vocal tract length normalization parameters for each evaluation target person and perform high-accuracy voice processing according to speaker characteristics.

  Further, the speech processing apparatus according to the fifth aspect of the invention is the learning acoustic data according to the fourth aspect, wherein the vocal tract length normalization parameter calculation unit is a connected HMM in which phoneme HMMs are connected according to the specified utterance content. Learning acoustic data storage means for storing the second cepstrum vector series calculating means for calculating a second cepstrum vector series by short section analysis from the voice received by the voice receiving unit, and the second cepstrum vector Cepstrum transformation means for transforming a sequence using a matrix having cepstrum transformation parameters as elements and calculating a frequency-warped third cepstrum vector sequence, and an occupation that is a posterior probability of speech accepted according to the specified utterance content Occupancy calculation means for calculating frequency, the learning acoustic data and the third cepstrum vector system And a cepstrum conversion parameter calculation unit that calculates a cepstrum conversion parameter based on the occupancy degree, a process in the cepstrum conversion unit, a process in the occupancy calculation unit, and the cepstrum conversion parameter calculation based on a predetermined rule. A final cepstrum conversion parameter acquisition unit that obtains a cepstrum conversion parameter, and a voice for calculating the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit. A speech processing apparatus including a road length normalization parameter calculation unit.

  With this configuration, it is possible to dynamically perform vocal tract length normalization parameters for each evaluation target person and perform high-accuracy voice processing according to speaker characteristics.

  The speech processing apparatus according to the sixth aspect of the invention provides a phoneme average cepstrum vector sequence in which the vocal tract length normalization parameter calculation unit arranges the phoneme average cepstrum vectors according to the designated utterance content. Learning acoustic data storage means for storing learning acoustic data, second cepstrum vector series calculation means for calculating a second cepstrum vector series by short section analysis from the voice received by the voice receiving unit, The second cepstrum vector sequence is converted by using a matrix having cepstrum conversion parameters as elements, and the cepstrum conversion means for calculating the frequency-warped third cepstrum vector sequence, and each of the speech voices uttered by the user Optimal phoneme sequence to obtain phoneme sequence that is a sequence of information identifying phonemes corresponding to frames Obtaining means, cepstrum conversion parameter calculation means for calculating a cepstrum conversion parameter using the learning acoustic data, the third cepstrum vector series, and the phoneme series, and processing in the cepstrum conversion means based on a predetermined rule And the cepstrum obtained by the final cepstrum conversion parameter acquisition unit and the cepstrum obtained by the final cepstrum conversion parameter acquisition unit by repeating the process in the optimum phoneme sequence acquisition unit and the process in the cepstrum conversion parameter calculation unit. A speech processing apparatus comprising vocal tract length normalization parameter calculation means for calculating the vocal tract length normalization parameter based on a conversion parameter.

  With this configuration, it is possible to dynamically perform vocal tract length normalization parameters for each evaluation target person and perform high-accuracy voice processing according to speaker characteristics.

  Further, the speech processing apparatus according to the seventh aspect of the invention is the fifth and sixth aspects of the invention, wherein the vocal tract length normalization parameter calculating means linearly calculates from the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition means. The speech processing apparatus calculates a frequency expansion ratio and calculates the vocal tract length normalization parameter from the linear frequency expansion ratio.

  With this configuration, the vocal tract length normalization parameter can be dynamically set for each person to be evaluated, and high-accuracy voice processing according to speaker characteristics can be performed.

  Further, in the voice processing device of the eighth invention, in contrast to the fifth and sixth inventions, the vocal tract length normalization parameter calculation unit stores frequency range designation information which is information for designating a frequency range. The cepstrum conversion parameter calculation unit is a voice processing device that calculates a cepstrum conversion parameter using the frequency range specification information.

  With this configuration, the vocal tract length normalization parameter can be dynamically set for each person to be evaluated, and more accurate voice processing can be performed according to speaker characteristics.

  Further, according to the ninth aspect of the present invention, in any one of the first to eighth aspects, the sound processing unit includes a frame dividing means for dividing the second audio data into frames, and the classification Frame audio data acquisition means for obtaining one or more frame audio data, which is audio data for each frame, and evaluation of the audio received by the audio reception unit based on the teacher data and the one or more frame audio data A speech processing apparatus comprising a rating means and an output means for outputting a rating result in the rating means.

  With this configuration, it is possible to evaluate speech with higher accuracy according to speaker characteristics.

  The speech processing apparatus according to the tenth aspect of the present invention is the optimum state for the ninth aspect, wherein the rating means determines an optimum state for at least one frame sound data of the one or more frame sound data. A voice evaluation value is calculated using the determination means, the optimum state probability value acquisition means for acquiring the probability value in the optimum state determined by the optimum state determination means, and the probability value acquired by the optimum state probability value acquisition means as parameters. It is a voice processing device comprising rating value calculation means.

  With this configuration, it is possible to evaluate speech with high accuracy according to the speaker characteristics of the evaluation target person.

  The speech processing apparatus according to the eleventh aspect of the present invention is the speech processing apparatus according to the ninth aspect, wherein the rating means includes an optimum state determining means for determining an optimum state of the one or more frame sound data, and the optimum state determination. The pronunciation interval frame phoneme probability value acquisition means for acquiring, for each pronunciation interval, one or more probability values in the overall state of the phoneme having the optimum state of each frame determined by the means, and the pronunciation interval frame phoneme probability value acquisition means The speech processing apparatus includes a rating value calculation unit that calculates a rating value of speech using one or more probability values for each of the one or more pronunciation intervals as a parameter.

  With this configuration, it is possible to evaluate speech with higher accuracy according to the speaker characteristics of the evaluation target person.

  Further, in the speech processing apparatus according to the twelfth aspect of the invention, in contrast to the ninth aspect of the invention, the speech processing unit detects that special speech has been input based on the input speech data for each frame. The voice processing device further comprising special voice detecting means, wherein the rating means evaluates the voice received by the voice receiving unit based on the teacher data, the input voice data, and the detection result in the special voice detecting means. It is.

  With this configuration, it is possible to evaluate speech with high accuracy according to the speaker characteristics of the evaluation subject, detect special speech, and evaluate speech corresponding to the special speech.

  The sound processing apparatus according to the thirteenth aspect of the present invention is directed to the twelfth aspect of the invention, wherein the special sound detection means is a silence data storage means for storing silence data which is data based on HMM indicating silence. And a silent processing unit comprising a silent section detecting means for detecting a silent section based on the input voice data and the silent data.

  With such a configuration, it is possible to evaluate speech with high accuracy according to the speaker characteristics of the evaluation target person, detect a silent section, and evaluate speech corresponding to the silent section.

  The speech processing apparatus according to the fourteenth aspect of the invention relates to the twelfth aspect of the invention, wherein the special speech detecting means has a rating value of the second half of one phoneme and the first half of the next phoneme. A voice that constitutes a rating result indicating that at least a phoneme has been inserted when the special voice detecting means detects that the predetermined voice condition has been detected. It is a processing device.

  With this configuration, it is possible to evaluate speech with high accuracy according to the speaker characteristics of the evaluation target person, detect insertion of phonemes, and evaluate speech corresponding to insertion of such phonemes.

  Further, in the voice processing apparatus of the fifteenth aspect of the invention, in contrast to the twelfth aspect of the invention, the special voice detecting means detects that a rating value of one phoneme satisfies a predetermined condition, and the rating means Is a speech processing device that constitutes an evaluation result indicating that at least a phoneme has been replaced or missing when the special speech detection means detects that the predetermined condition is satisfied.

  With this configuration, it is possible to evaluate speech with high accuracy according to the speaker characteristics of the evaluation subject, detect phoneme replacement or omission, and evaluate speech corresponding to such phoneme substitution or omission.

  Further, in the sixteenth aspect of the present invention, in contrast to any one of the first to fifteenth aspects, the voice processing device is a karaoke evaluation device, and the voice reception unit is an evaluation subject. The voice processing unit is a voice processing device that evaluates the singing voice.

  With this configuration, it can be used as an excellent karaoke evaluation apparatus.

  Further, in the seventeenth aspect of the present invention, in the sixteenth aspect of the invention, the sound receiving unit receives the input of the singing voice of the evaluation subject after receiving the sound of a predetermined vowel, The sampling unit also samples the voice of the vowel at the first sampling frequency, and obtains an evaluation target formant frequency that is an evaluation target formant frequency based on the sampled vowel voice. The speech processing apparatus further includes a frequency acquisition unit, and the evaluation subject formant frequency of the evaluation subject formant frequency storage unit is the evaluation subject formant frequency acquired by the evaluation subject formant frequency acquisition unit.

  With this configuration, it is possible to evaluate speech with high accuracy according to the speaker characteristics of the evaluation target person.

  In the eighteenth aspect of the present invention, the voice processing device according to any of the first to eighth aspects, wherein the voice processing unit performs a voice recognition process based on the second voice data. It is.

  With this configuration, it is possible to perform highly accurate speech recognition according to the speaker characteristics of the evaluation target person.

  In addition, the nineteenth aspect of the present invention provides a first sampling frequency storage unit that stores a first sampling frequency, and a vocal tract length that stores a vocal tract length normalization parameter that is information related to a conversion rate of the sampling frequency. A normalization parameter storage unit, and a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters, perform a sampling process on the voice received by the voice reception unit, It is a digital signal processor including a vocal tract length normalization processing unit that obtains two voice data.

  With this configuration, it is possible to provide a DSP that can perform highly accurate voice processing according to the speaker characteristics of the evaluation target person.

  According to the speech processing device of the present invention, speech processing with high accuracy according to the speaker characteristics of the evaluation subject can be performed.

Hereinafter, embodiments of a speech processing apparatus and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.
(Embodiment 1)

  In the present embodiment, a description will be given of a speech processing apparatus that can evaluate the similarity between the comparison target speech and the input speech with high accuracy and high speed. This speech processing device is a pronunciation rating device that evaluates speech (including singing). In particular, since the speech processing apparatus calculates the posterior probability of the optimum state with respect to the frame of the input speech using dynamic programming, the posterior probability is called DAP (Dynamic A Positive Probability) and is based on DAP. The degree calculation method and pronunciation rating device are called DAPS.

  In addition, the speech processing apparatus according to the present embodiment can be used for language learning, imitation practice, karaoke evaluation, and the like. FIG. 1 is a block diagram of a speech processing apparatus according to this embodiment. The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. , An evaluation subject formant frequency storage unit 108, a vocal tract length normalization processing unit 109, and a speech processing unit 110.

  The audio processing unit 110 includes a frame classification unit 1101, a frame audio data acquisition unit 1102, a rating unit 1103, and an output unit 1104.

  The rating unit 1103 includes an optimum state determination unit 11031, an optimum state probability value acquisition unit 11032, and a rating value calculation unit 11033.

  Note that the voice processing apparatus accepts input from input means such as a keyboard 342 and a mouse 343. The voice processing apparatus accepts voice input from voice input means such as a microphone 345. Further, the sound processing apparatus outputs information to an output device such as the display 344.

  The input receiving unit 101 inputs an operation start instruction for instructing an operation start of the speech processing apparatus, an output mode change instruction for instructing an output mode change of the input speech evaluation result, an end instruction for ending the process, and the like. Accept. The input means for such an instruction may be anything such as a numeric keypad, keyboard, mouse or menu screen. The input receiving unit 101 can be realized by a device driver for input means such as a numeric keypad and a keyboard, control software for a menu screen, and the like.

  The teacher data storage unit 102 is data relating to a target voice to be compared as teacher data, and stores one or more data based on a hidden Markov model (HMM) for each phoneme. The teacher data is preferably HMM-based data obtained by connecting hidden Markov models (HMM) for each phoneme. In addition, the teacher data is preferably data based on an HMM in which HMMs corresponding to phonemes constituting input speech are linked in accordance with the input order. However, the teacher data does not necessarily need to be data based on the HMM obtained by connecting the HMMs for each phoneme. The teacher data may be a simple set of all phoneme HMMs. The teacher data does not necessarily need to be data based on the HMM. The teacher data may be data based on other models such as a single Gaussian distribution model, a probability model (GMM: Gaussian mixture model), and a statistical model. The data based on the HMM has, for example, a state identifier and transition probability information for each frame. The data based on the HMM may be, for example, a model learned (estimated) from two or more data uttered by a foreigner who speaks a plurality of learning target languages as a native language. The teacher data storage unit 102 is preferably a non-volatile recording medium such as a hard disk or a ROM, but can also be realized by a volatile recording medium such as a RAM.

  The voice reception unit 103 receives voice. The voice reception unit 103 can be realized by, for example, driver software for the microphone 345. In addition, it may be considered that the voice reception unit 103 is realized by the microphone 345 and its driver. The sound may be input from the microphone 345 or may be input by reading from a recording medium such as a magnetic tape or a CD-ROM.

  The teacher data formant frequency storage unit 104 stores a teacher data formant frequency that is a formant frequency of teacher data. The teacher data formant frequency may be the first formant frequency (F1), the second formant frequency (F2), the third formant frequency (F3), or the like. The teacher data formant frequency in the teacher data formant frequency storage unit 104 may be stored in advance, or may be dynamically acquired from the teacher data at the time of evaluation. Since a technique for obtaining a formant frequency from audio data is a known technique, a description thereof will be omitted. The teacher data formant frequency storage unit 104 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium.

  The first sampling frequency storage unit 105 stores a first sampling frequency that is a first sampling frequency. The first sampling frequency is a sampling frequency when the voice of the evaluation subject is first sampled. The first sampling frequency storage unit 105 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The sampling unit 106 samples the sound received by the sound receiving unit 103 at the first sampling frequency of the first sampling frequency storage unit 105 and acquires first sound data. Since the technique for sampling the received voice is a known technique, detailed description thereof is omitted. The sampling unit 106 can be usually realized by an MPU, a memory, or the like. The processing procedure of the sampling unit 106 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The evaluation target person formant frequency acquisition unit 107 acquires the evaluation target person formant frequency, which is the formant frequency of the evaluation target person, from the first sound data acquired by the sampling unit 106. The formant frequency to be evaluated may also be the first formant frequency (F1), the second formant frequency (F2), or the third formant frequency (F3). However, the evaluation subject formant frequency and the teacher data formant frequency are the same type of formant frequency. Since the technique for acquiring the formant frequency from the first audio data obtained by sampling is a known technique, detailed description thereof is omitted. The evaluation subject formant frequency acquisition unit 107 can be usually realized by an MPU, a memory, or the like. The processing procedure of the evaluation subject formant frequency acquisition unit 107 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The evaluation subject formant frequency storage unit 108 stores at least temporarily the evaluation subject formant frequency, which is the formant frequency of the evaluation subject who is the speaker of the voice received by the voice receiving unit 103. The evaluation subject formant frequency in the evaluation subject formant frequency storage unit 108 is usually the formant frequency acquired by the evaluation subject formant frequency acquisition unit 107, but the evaluation subject formant frequency may be stored in advance. When the evaluation subject formant frequency is stored in advance in the evaluation subject formant frequency storage unit 108, the evaluation subject formant frequency acquisition unit 107 is not required in the speech processing apparatus. The evaluation subject formant frequency storage unit 108 may be a non-volatile recording medium or a volatile recording medium.

  The vocal tract length normalization processing unit 109 performs sampling processing on the voice received by the voice receiving unit 103 at the second sampling frequency to obtain second voice data. The second sampling frequency is a sampling frequency calculated by “first sampling frequency / (teacher data formant frequency / evaluator formant frequency)”. The vocal tract length normalization processing unit 109 preferably resamples the first audio data obtained by sampling the audio received by the audio receiving unit 103 to obtain second audio data. The audio received by the audio receiving unit 103 may be sampled to obtain second audio data directly. When the second audio data is obtained directly, for example, it is necessary that the hardware that performs the sampling process can perform the sampling process at a variable sampling frequency. The vocal tract length normalization processing unit 109 normally performs an operation “teacher data formant frequency / evaluator formant frequency” to obtain a frequency axis conversion rate (“r”). Then, the vocal tract length normalization processing unit 109 performs an operation “Fs / r” based on the first sampling frequency (Fs) and “r” of the first sampling frequency storage unit 105 to obtain a new sampling frequency (Fs / r) is obtained. This new sampling frequency (Fs / r) is the second sampling frequency. Next, the vocal tract length normalization processing unit 109 performs resampling processing on the first sound data at the second sampling frequency (Fs / r) to obtain second sound data. The vocal tract length normalization processing unit 109 can usually be realized by an MPU, a memory, or the like. The processing procedure of the vocal tract length normalization processing unit 109 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). It can be said that the reciprocal “1 / r” of the frequency axis conversion rate “r” is a vocal tract length parameter described later. Further, since the vocal tract length parameter is a parameter for normalizing the vocal tract length, the frequency axis conversion rate “r” may be considered as the vocal tract length parameter.

  The audio processing unit 110 processes the second audio data. Here, the voice processing unit 110 is a rating process. However, the voice processing unit 110 may perform other voice processing such as voice recognition and voice output. The audio output is simply a process of outputting the resampled audio. In the present embodiment, the audio processing unit 110 will be described as performing a rating process. The audio processing unit 110 can usually be realized by an MPU, a memory, or the like. The processing procedure of the audio processing unit 110 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Frame classification means 1101 constituting the audio processing unit 110 divides the second audio data into frames. The frame partitioning means 1101 can usually be realized by an MPU, a memory, or the like. The processing procedure of the frame sorting unit 1101 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The frame sound data acquisition means 1102 constituting the sound processing unit 110 obtains one or more frame sound data which is sound data for each divided frame. The frame audio data acquisition unit 1102 can be usually realized by an MPU, a memory, or the like. The processing procedure of the frame audio data acquisition unit 1102 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The rating means 1103 constituting the voice processing unit 110 evaluates the voice received by the voice receiving unit 103 based on the teacher data in the teacher data storage unit 102 and one or more frame voice data. A specific example of the rating method will be described later. Needless to say, the concept of “rating the voice received by the voice receiving unit 103” includes rating the second voice data. The rating means 1103 can usually be realized by an MPU, a memory, or the like. The processing procedure of the rating means 1103 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

Optimal state determination means 11031 constituting the rating means 1103 determines an optimal state for at least one frame sound data of one or more frame sound data. The optimum state determination unit 11031 acquires, for example, an HMM corresponding to one or more phonemes constituting speech such as words and sentences to be compared (learning target) from the whole phoneme HMM, and the acquired one or more From the HMM, data concatenated in the order of phonemes (data related to the speech to be compared and data based on the HMM concatenating hidden Markov models for each phoneme) is constructed. Then, based on each feature vector o _t constituting constituting the relevant data, and the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t). The algorithm for determining the optimum state is, for example, the Viterbi algorithm. In addition, the teacher data is data relating to the above-described target speech to be compared, and may be considered as data based on HMM obtained by concatenating hidden Markov models for each phoneme. It may be considered as HMM data.

  The optimum state probability value acquisition unit 11032 constituting the rating unit 1103 acquires the probability value in the optimum state determined by the optimum state determination unit 11031.

  The rating value calculating means 11033 constituting the rating means 1103 calculates the voice rating value using the probability value acquired by the optimum state probability value acquiring means 11032 as a parameter. It does not matter how the rating value calculation means 11033 uses the probability value to calculate the rating value. The rating value calculation unit 11033 calculates a voice rating value using, for example, the sum of the probability value acquired by the optimum state probability value acquisition unit 11032 and the probability value in all states of the frame corresponding to the probability value as a parameter. Here, the rating value calculation means 11033 normally calculates a rating value for each frame.

  The optimum state determination unit 11031, the optimum state probability value acquisition unit 11032, and the rating value calculation unit 11033 can be usually realized by an MPU, a memory, or the like. The processing procedure of the optimum state determining unit 11031 and the like is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 1104 outputs the rating result from the rating unit 1103. Various output modes of the output means 1104 can be considered. Output is a concept that includes display on a display, printing on a printer, sound output, transmission to an external device, storage in a recording medium, and the like. The output unit 1104 visually displays the evaluation result, for example. The output unit 1104 visually displays, for example, the evaluation result of the frame unit, or / and the phoneme / word unit, or / and the entire utterance. The output unit 1104 may or may not include an output device such as the display 344 or a speaker. The output unit 1104 can be realized by driver software for an output device or driver software for an output device and an output device.

  Next, the operation of the speech processing apparatus will be described with reference to the flowcharts of FIGS.

  (Step S201) The input reception unit 101 determines whether an operation start instruction for instructing the operation start of the speech processing apparatus has been received. If the operation start instruction is accepted, the process goes to step S202, and if the operation start instruction is not accepted, the process jumps to step S217.

  (Step S202) The voice receiving unit 103 determines whether a voice is received. If a voice is accepted, the process goes to step S203, and if no voice is accepted, the process jumps to step S216.

  (Step S203) The sampling unit 106 reads the first sampling frequency stored in the first sampling frequency storage unit 105, samples the audio received by the audio receiving unit 103 at the first sampling frequency, and outputs the first audio. Get the data.

  (Step S204) The vocal tract length normalization processing unit 109 obtains second voice data from the voice received by the voice receiving unit 103. Details of the vocal tract length normalization process, which is a process for obtaining such second audio data, will be described in detail with reference to the flowchart of FIG. The vocal tract length normalization process is a pre-process for rating that absorbs individual differences.

  (Step S205) The frame sorting unit 1101 temporarily stores the second audio data obtained in Step S204 in a buffer (not shown).

  (Step S206) The frame segmentation means 1101 segments the second audio data temporarily stored in the buffer into frames. At this stage, frame audio data which is audio data for each divided frame is configured. The frame division processing performed by the frame classification unit 1101 is, for example, preprocessing when the frame audio data acquisition unit 1102 extracts frame audio data, and the input audio data is divided into all frames at once. Not exclusively.

  (Step S207) The frame audio data acquisition means 1102 substitutes 1 for the counter i.

  (Step S208) The frame audio data acquisition unit 1102 determines whether or not the i-th frame exists. If the i-th frame exists, the process goes to step S209. If the i-th frame does not exist, the process goes to step S211.

  (Step S209) The frame sound data acquisition unit 1102 acquires the i-th frame sound data. The acquisition of the frame sound data means, for example, performing sound analysis on the divided sound data and extracting feature vector data. Note that the frame audio data is, for example, data obtained by dividing the input audio data into frames. The frame audio data includes, for example, feature vector data extracted from the divided audio data by audio analysis. This feature vector data is, for example, MFCC obtained by discrete cosine transform of a filter bank output of 24 channels using a triangular filter, and the static parameter, delta parameter, and delta delta parameter are further normalized to 12 dimensions, respectively. Power and delta power and delta delta power (39th dimension).

  (Step S210) The frame audio data acquisition unit 1102 increments the counter i by 1. The process returns to step S208.

  (Step S211) Optimal state determination means 11031 determines the optimal state of all frames. The algorithm for determining the optimum state by the optimum state determining unit 11031 is, for example, the Viterbi algorithm. Since the Viterbi algorithm is a known algorithm, detailed description thereof is omitted.

  (Step S212) The optimum state probability value acquisition unit 11032 calculates the forward likelihood and the backward likelihood of all states of all frames. For example, the optimal state probability value acquisition unit 11032 calculates the forward likelihood and the backward likelihood by the forward / backward algorithm using all the HMMs.

  (Step S213) The optimal state probability value acquisition unit 11032 calculates all of the optimal state probability values (optimal state probability values) using the forward likelihood and the backward likelihood acquired in step S212.

  (Step S214) The rating value calculation means 11033 calculates the rating value of one or more frames of speech from the one or more optimal state probability values calculated in step S213. The function for calculating the rating value by the rating value calculating means 11033 is not specified. The rating value calculation unit 11033 calculates a speech rating value using, for example, the sum of the acquired optimum state probability value and the probability value in all states of the frame corresponding to the optimum state probability value as a parameter. Details will be described later.

  (Step S215) The output unit 1104 outputs the rating result (here, the voice rating value) in step S214 according to the set output mode. The process returns to step S202. The output mode is a mode in which the rating value is displayed on the screen as a numerical value, a mode in which the transition of the rating value is displayed on the screen as a graph, a mode in which the rating value is output by voice, and a warning when the rating value is lower than the predetermined value Any mode may be used, such as a mode for displaying information to be shown. The output mode here is a mode set in step S218.

  (Step S216) The voice reception unit 103 determines whether or not a timeout has occurred. That is, the voice receiving unit 103 determines whether or not a voice input has been received for a predetermined time or more. If timed out, the process returns to step S201, and if not timed out, the process returns to step S202.

  (Step S217) The input receiving unit 101 determines whether an output mode change instruction has been received. If an output mode change instruction is accepted, the process proceeds to step S218, and if no output mode change instruction is received, the process jumps to step S219. The output mode change instruction is information having the output mode described above.

  (Step S218) The output unit 1104 writes information indicating the output mode included in the output mode change instruction received in Step S217, and sets the output mode. The process returns to step S201.

  (Step S219) The input receiving unit 101 determines whether an end instruction has been received. If an end instruction is accepted, the process ends. If no end instruction is accepted, the process returns to step S201.

  In the flowchart of FIG. 2, the pronunciation evaluation device may not have an output mode setting function.

  Next, details of the vocal tract length normalization process in step S204 will be described using the flowchart of FIG.

  (Step S301) The evaluation target person formant frequency acquisition unit 107 acquires the evaluation target person formant frequency (Fi) from the first sound data obtained by the sampling processing of the sampling unit 106, and the evaluation target person formant frequency storage unit 108. Temporarily store in. The evaluation subject formant frequency is, for example, the second formant frequency (F2).

  (Step S302) The vocal tract length normalization processing unit 109 reads the first sampling frequency (Fs) of the first sampling frequency storage unit 105.

  (Step S303) The vocal tract length normalization processing unit 109 reads the teacher data formant frequency in the teacher data formant frequency storage unit 104.

  (Step S304) The vocal tract length normalization processing unit 109 calculates a frequency axis conversion rate from the evaluation subject formant frequency acquired in step S301 and the teacher data formant frequency read in step S303. Specifically, the vocal tract length normalization processing unit 109 performs an operation “teacher data formant frequency / evaluator formant frequency” to obtain a frequency axis conversion rate (r).

  (Step S305) The vocal tract length normalization processing unit 109 performs the calculation “Fs / r” based on the first sampling frequency (Fs) and the frequency axis conversion rate (r) read in step S302, and performs the second sampling. Obtain the frequency (Fs / r).

  (Step S306) The vocal tract length normalization processing unit 109 performs resampling processing at the second sampling frequency (Fs / r) on the first audio data obtained by sampling by the sampling unit 106, Get audio data. Since the resampling process is a known technique, detailed description thereof is omitted. Return to upper function.

  In the flowcharts of FIGS. 2 and 3, the voice to be subjected to vocal tract length normalization processing may be different from the voice to be evaluated. That is, for example, the voice receiving unit 103 receives the voice of the evaluation target person after receiving the voice of the predetermined one or more vowels (for example, “U”), and the evaluation target formant frequency acquisition unit 107 Based on the voice of the above vowel, the evaluation target person formant frequency is acquired, and the vocal tract length normalization processing unit 109 performs the vocal tract length normalization process using the evaluation target person formant frequency as a parameter. Then, the voice processing unit 110 may process the received voice after receiving the voice of a predetermined vowel and evaluate the voice.

  Hereinafter, a specific operation of the speech processing apparatus according to the present embodiment will be described. In this specific example, the case where the speech processing apparatus is used for language learning will be described.

First, in this speech processing apparatus, a phonetic HMM of native pronunciation is learned from a speech database of native pronunciation by means not shown. Here, the number of phoneme types is L, and the HMM for the l-th phoneme is λ _l . Since this learning process is a known technique, a detailed description thereof is omitted. An example of HMM specifications is shown in FIG. The specification of the HMM is the same in the description of specific examples in other embodiments. However, it goes without saying that the specifications of the HMM may be other specifications.

  Then, by means not shown, an HMM corresponding to one or more phonemes constituting speech such as words or sentences to be learned is acquired from the learned L types of phoneme HMMs, and the acquired one or more HMMs are converted into phonemes. The teacher data concatenated in this order is configured. Then, the teacher data is held in the teacher data storage unit 102. Here, for example, the target speech to be compared is the speech of the word “right”. Here, it is assumed that the person (teacher) who uttered the teacher data is an adult.

  Next, the learner (evaluator) inputs an operation start instruction that is an instruction to start language learning. Such an instruction is made, for example, by pressing a predetermined button with a mouse. Here, it is assumed that the learner is, for example, a child (5 to 11 years old).

  First, it is assumed that the learner pronounces the vowel “U”. In such a case, it is preferable that the speech processing apparatus prompts the user to speak “U” for learning. In order to prompt the user to say “U”, the voice processing device may output, for example, “Please say“ U ”.” Or “Speak“ U ”.” May be output. Moreover, the vowel “U” is suitable for acquiring the learner's evaluation target formant frequency. In addition, it is assumed that the sound processing apparatus holds “22.05 KHz” as the first sampling frequency.

  Then, the sampling unit 106 samples the voice “U” received by the voice receiving unit 103 to obtain first voice data “U”.

  Next, the evaluation subject formant frequency acquisition unit 107 acquires the second formant frequency from the first audio data obtained by the sampling unit 106 sampling the audio “U”. Then, this second formant frequency is assumed to be the evaluation subject formant frequency (Fi. Now, let this Fi be “1725 Hz”. Then, the evaluation subject formant frequency acquisition unit 107 sets Fi (1725 Hz) to And temporarily stored in the evaluation formant formant frequency storage unit 108.

  Next, the vocal tract length normalization processing unit 109 reads the teacher data formant frequency in the teacher data formant frequency storage unit 104. It is assumed that the teacher data formant frequency stored in the teacher data formant frequency storage unit 104 is the second formant frequency of an adult and is now “1184 Hz”. In addition, the teacher data formant frequency is, for example, when the teacher data is constructed, the teacher utters “U”, for example, and after sampling the audio “U”, the acquired second formant frequency is used. is there.

  FIG. 5 shows the measurement results of the first formant frequency (F1) and the second formant frequency (F2) of “U” for each age group and sex. FIG. 5 shows that the values of the first formant frequency (F1) and the second formant frequency (F2) are greatly different depending on the age and sex.

  Next, the vocal tract length normalization processing unit 109 performs an operation “teacher data formant frequency / evaluation target person formant frequency” from the evaluation subject formant frequency “1725 Hz” and the teacher data formant frequency “1184 Hz”, and the frequency axis The conversion rate (r) is obtained. Specifically, the vocal tract length normalization processing unit 109 obtains the frequency axis conversion rate “0.686” from “1184/1725”.

  Next, the vocal tract length normalization processing unit 109 performs an operation “Fs / r” based on the first sampling frequency (Fs) and “r” to obtain a second sampling frequency (Fs / r). Here, the obtained second sampling frequency is “32.1” by “22.05 / 0.686”. Then, the vocal tract length normalization processing unit 109 temporarily stores the second sampling frequency “32.1 KHz”.

  Next, the learner pronounces the voice “right” to be learned. Then, the voice receiving unit 103 receives an input of a voice pronounced by the learner. It is preferable that the speech processing apparatus prompts the learner to utter “right” by displaying or outputting a voice such as “Please pronounce“ right ”” to the learner.

  Next, the sampling unit 106 samples the received voice “right” at the sampling frequency “22.05 KHz”. Then, the sampling unit 106 obtains the first sound data of the sound “right”.

  Next, the vocal tract length normalization processing unit 109 resamples the first voice data of “right” at the second sampling frequency “32.1 KHz”. Then, the vocal tract length normalization processing unit 109 obtains second voice data.

  Next, the audio processing unit 110 processes the second audio data as follows.

  First, the frame classification unit 1101 classifies the second audio data into short-time frames. It is assumed that the frame interval is determined in advance.

Then, the frame audio data acquisition unit 1102 performs spectrum analysis on the audio data classified by the frame classification unit 1101 and calculates a feature vector series “O = o ₁ , o ₂ ,..., O _T ”. T is a sequence length. Here, the feature vector series is a set of feature vectors of each frame. The feature vector is, for example, an MFCC obtained by performing discrete cosine transform on a filter bank output of 24 channels using a triangular filter, and the static parameter, the delta parameter, and the delta delta parameter are further normalized to 12 dimensions, respectively. Power and delta power and delta delta power (39th dimension). In spectral analysis, it is preferable to perform cepstrum average removal. An example of speech analysis conditions is shown in the table of FIG. Note that the voice analysis conditions are the same in the description of specific examples in other embodiments. However, it goes without saying that the voice analysis conditions may be other conditions. The sampling frequency for voice analysis is the first sampling frequency “22.05 KHz”.

Then, the optimal state determination unit 11031, based on the feature vector o _t constituting the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t). The algorithm for determining the optimum state by the optimum state determining unit 11031 is, for example, the Viterbi algorithm. In such a case, the optimum state determination unit 11031 determines the optimum state using the HMM connected as described above. The optimum state determination unit 11031 obtains an optimum state sequence that is an optimum state of two or more frames.

Next, the optimum state probability value acquisition unit 11032 calculates the optimum state probability value (γ _t (q _t ^* )) in the optimum state (q _t ^* ) according to the following Equation 1. Note that γ _t (q _t ^* ) is a value obtained by substituting q _t ^* for j in the posterior probability function γ _t (j) of the state j. Then, the posterior probability function γ _t (j) of the state j is calculated using Equation 2. This probability value _{(γ t} (j)) is, t th feature vector o _t is the posterior probability that is generated from the state j, is calculated using dynamic programming. Note that j is a state identifier for identifying a state.

In Equation 2, _{q t} represents the state identifier for _{o t.} This probability value (γ _t (j)) corresponds to the occupation frequency appearing in the Baum-Welch algorithm in the maximum likelihood estimation of the HMM.

In Equation 2, “α _t (j)” and “β _t (j)” are calculated by the forward-backward algorithm using all the HMMs. “Α _t (j)” is a forward likelihood, and “β _t (j)” is a backward likelihood. Since the Baum-Welch algorithm and the forward-backward algorithm are known algorithms, detailed description thereof is omitted.

  In Equation 2, N represents the total number of states over all HMMs.

  Note that the rating unit 1103 may first obtain an optimum state, and then obtain a probability value of the optimum state (where the probability value is 0 or more and 1 or less). Then, the probability values of all the states may be obtained, then the optimum state for each feature vector of the feature vector series may be obtained, and the probability value corresponding to the optimum state may be obtained.

  Next, the rating value calculation means 11033 calculates a speech rating value using, for example, the sum of the acquired optimum state probability value and the probability value in all states of the frame corresponding to the optimum state probability value as a parameter. In such a case, if the utterance corresponding to the learner's t-th frame is close to the pronunciation indicated by the teacher data (for example, correct native pronunciation), the numerator value of Equation (2) in Equation 2 is set to all other values. As compared with all possible phoneme states, the probability value (rating value) in the optimum state increases. Conversely, if the interval is not close to the pronunciation indicated by the teacher data, the rating value becomes small. In the case where it is not close to any native pronunciation, the rating value is approximately equal to 1 / N. Since N is the number of all states in all phoneme HMMs, it is usually a large value, and this rating value is sufficiently small. Here, the rating value is defined by the ratio between the probability value in the optimum state and the probability value in all possible states. Therefore, even if there is some spectrum variation due to differences in the sound collection environment or the like, if the learner pronounces correctly, the variation is offset and a score with a high rating value is maintained. Therefore, the rating value calculation means 11033 calculates the voice rating value using the probability value acquired by the optimum state probability value acquisition means 11032 and the sum of the probability values in all states of the frame corresponding to the probability value as parameters. Is very suitable.

  Examples of output of the rating value (also referred to as “DAP score”) calculated by the rating value calculation means 11033 are shown in FIGS. 7 and 8, the horizontal axis represents the analysis frame number, and the vertical axis represents the score in%. A thick broken line represents a phoneme boundary, a thin dotted line represents a state boundary (both obtained by the Viterbi algorithm), and a phoneme name is shown at the top of the figure. FIG. 7 shows the DAP score for the pronunciation of English “right” by an American male. The horizontal axis and vertical axis of the graph indicating the rating value are the same in the graph described later.

  FIG. 8 shows a DAP score of pronunciation of English “right” by a Japanese male. American pronunciation is basically higher than Japanese pronunciation. In addition, in FIG. 7, it turns out that the score has fallen in some places in the boundary of a state.

  Then, the output unit 1104 outputs the rating result of the rating unit 1103. Specifically, for example, the output unit 1104 outputs the rating result in a manner as shown in FIG. That is, the output unit 1104 displays the rating value of each frame as a score (score graph) indicating the goodness of pronunciation in each frame. In addition, the output unit 1104 displays a word to be learned (word display), a phoneme element display (phoneme display), a teacher data waveform display (teacher waveform), and a pronunciation waveform input by the learner (user). Waveform) may be displayed. In FIG. 9, when the “Record” button is pressed, an operation start instruction is input, and when the “Stop” button is pressed, an end instruction is input. Further, since the technology for displaying phoneme elements and displaying waveforms is a known technology, its detailed description is omitted. Further, the speech processing apparatus stores in advance data for outputting a word to be learned (such as “word1” in FIG. 9), a phoneme (such as “p1” in FIG. 9), and a teacher waveform. , And.

  Moreover, in FIG. 9, you may display the evaluation result of a phoneme unit, a word unit, and the whole utterance other than a frame unit. In the above processing, the evaluation value for each frame is calculated. In order to obtain the evaluation result for each word and the whole utterance, the evaluation value for each word and the whole utterance is obtained using one or more evaluation values for each frame as parameters. It is necessary to calculate. Such a calculation formula is not limited. For example, it is conceivable that an average value of one or more rating values for each frame constituting a word is used as a rating value for each word.

  In FIG. 9, when the voice processing device accepts a click at the location of the waveform display (teacher waveform or user waveform), it displays a playback menu, and within the phoneme section, the phoneme, the word to which the section belongs, and the entire waveform are displayed. It is possible to reproduce only the entire waveform outside the word section (silent part).

  Further, the display of the output means 1104 may be in the form as shown in FIG. In FIG. 10, a score for each phoneme, a word score, and a total score are displayed in numbers.

  Needless to say, the display of the output means 1104 may be as shown in FIGS.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. In this case, according to the present embodiment, highly accurate evaluation can be performed without being affected by individual differences, particularly differences in vocal tract length.

  Further, according to the present embodiment, since the optimum state is obtained by using the concatenated HMM, which is a concatenated HMM, and the evaluation value is calculated, the evaluation value can be obtained at high speed. Therefore, as described in the above specific example, the rating value for each frame, for each phoneme, and for each word can be output in real time. Further, according to the present embodiment, the posterior probability based on the dynamic programming is calculated as the probability value, so that the rating value can be obtained at higher speed. Further, according to the present embodiment, since the probability value is calculated for each frame, as described above, it is possible to output not only the frame unit but / or the phoneme / word unit or / and the entire utterance evaluation result. The degree of freedom of the output mode is high.

  Moreover, according to this Embodiment, although the audio processing apparatus was mainly demonstrated using for language learning, it can be utilized for imitation practice, karaoke rating, singing rating, etc. That is, the speech processing apparatus can accurately evaluate and output the similarity to the data related to the target speech to be compared with high speed, and the application is not limited. That is, for example, the voice processing device may be a karaoke evaluation device, and the voice reception unit may receive an input of the evaluation subject's singing voice, and the voice processing unit may evaluate the singing voice. The same applies to other embodiments.

  Further, in the present embodiment, after receiving voice input or operating the stop button, the user is inquired whether to execute scoring processing, and only when an instruction to perform scoring processing is received, FIG. A phoneme score, a word score, and a total score as shown in FIG. This also applies to other embodiments.

  Further, in the present embodiment, the teacher data is mainly related to the speech to be compared and is based on the hidden Markov model (HMM) for each phoneme, but is not necessarily data based on the HMM. There is no need. The teacher data may be data based on other models such as a single Gaussian distribution model, a probability model (GMM: Gaussian mixture model), and a statistical model. This also applies to other embodiments.

  In the specific example of the present embodiment, the learner pronounces the vowel “U”, and the speech processing apparatus obtains the second sampling frequency from the speech. However, the learner may generate one or more vowels such as the vowel “Aieo”, and the sound processing apparatus may obtain the second sampling frequency from the sound of the vowels. That is, the sound produced by the learner to obtain the second sampling frequency is not limited to “U”.

  In the present embodiment, the speech processing apparatus calculates the vocal tract length normalization parameter “r” in the vocal tract length normalization processing unit 109. However, the vocal tract length normalization parameter “1 / r” may be calculated separately, and the vocal tract length normalization parameter “1 / r” may be stored in the vocal tract length normalization parameter storage unit. . In such a case, the speech processing apparatus stores a speech acceptance unit that accepts speech, a first sampling frequency storage unit that stores the first sampling frequency, and a vocal tract length normalization parameter that is information related to the conversion rate of the sampling frequency. A voice sampling length normalization parameter storage unit, a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters, with respect to the voice received by the voice reception unit, A voice processing device including a vocal tract length normalization processing unit that performs sampling processing to obtain second voice data, and a voice processing unit that processes the second voice data. In such a case, in the speech processing apparatus, the teacher data formant frequency storage unit 104, the evaluation subject formant frequency acquisition unit 107, and the evaluation subject formant frequency storage unit 108 are not essential. This also applies to other embodiments.

  In the present embodiment, the following processing performed by the audio processing device may be performed by a single DSP (digital signal processor). That is, the DSP includes a first sampling frequency storage unit that stores a first sampling frequency, a sampling unit that samples received audio at the first sampling frequency, and acquires first audio data, and the teacher. A teacher data formant frequency storage unit storing teacher data formant frequency which is a formant frequency of data, and an evaluation object storing an evaluation object formant frequency which is a formant frequency of an evaluation object person who is the voice speaker A sampling form is performed on the received voice at the second formant frequency storage unit and the second sampling frequency “the first sampling frequency / (teacher data formant frequency / evaluation person formant frequency)”, and the second voice data With a vocal tract length normalization processing unit Digital signal processor, a. This also applies to other embodiments.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that implements the speech processing apparatus according to the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Target voice formant frequency) ", the voice received in the voice receiving step is subjected to a sampling process to obtain a second voice data, a vocal tract length normalization processing step, and a voice process to process the second voice data A program for executing a step.

  In the above program, the audio processing step includes a frame dividing step for dividing the second audio data into frames, and a frame audio data acquiring step for obtaining one or more frame audio data which are audio data for each of the divided frames. And a rating step for rating the received voice based on the teacher data and the one or more frame voice data, and an output step for outputting a rating result in the rating step.

Further, in the above program, the rating step includes an optimum state determination step for determining an optimum state for at least one frame sound data among the one or more frame sound data, and an optimum state determined in the optimum state determination step. It is preferable to include an optimum state probability value obtaining step for obtaining a probability value, and a rating value calculating step for calculating a speech evaluation value using the probability value obtained in the optimum state probability value obtaining step as a parameter.
(Embodiment 2)

  The speech processing apparatus according to the present embodiment differs from the speech processing apparatus according to the first embodiment in the rating algorithm in the rating unit. In the present embodiment, the rating value is calculated by evaluating the probability values of all the states in the phoneme including the optimum state in the pronunciation interval. The posterior probability calculated by the speech processing apparatus in the present embodiment is referred to as tp-DAP with respect to the DAP in the first embodiment.

  FIG. 11 is a block diagram of the speech processing apparatus according to this embodiment. The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. The evaluation subject person formant frequency storage unit 108, the vocal tract length normalization processing unit 109, the voice processing unit 1110, and the utterance prompting unit 1109 are provided.

  The audio processing unit 1110 includes a frame classification unit 1101, a frame audio data acquisition unit 1102, a rating unit 11103, and an output unit 1104.

  The rating unit 11103 includes an optimum state determining unit 11031, a pronunciation interval frame phoneme probability value acquiring unit 111032, and a rating value calculating unit 111033.

  The pronunciation interval frame phoneme probability value acquisition unit 111032 acquires, for each pronunciation interval, one or more probability values in the entire phoneme state having the optimal state of each frame determined by the optimal state determination unit 11031.

  The rating value calculation means 111033 calculates a speech rating value using one or more probability values for each of one or more pronunciation intervals acquired by the pronunciation interval frame phoneme probability value acquisition means 111032 as parameters. For example, the rating value calculation unit 1111033 obtains, for each frame, a sum of one or more probability values in the entire phoneme state having the optimal state of each frame determined by the optimal state determination unit 11031, and the probability value for each frame. 1 or more is obtained based on the sum of the above, and the speech rating value is calculated using the one or more time average values as parameters.

  The pronunciation interval frame phoneme probability value acquisition unit 111032 and the rating value calculation unit 1111033 can be realized usually by an MPU, a memory, or the like. The processing procedure of the pronunciation interval frame phoneme probability value acquisition unit 111032 and the like is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  When the input reception unit 101 receives an operation start instruction, the utterance prompting unit 1109 performs a process of prompting the evaluation target person to utter in order to calculate the second sampling frequency, or for the pronunciation evaluation of the evaluation target person. Or urge the user to speak. The process of prompting the evaluation subject to speak is, for example, a process of displaying “Please pronounce ~” on the display or outputting a sound from the speaker “Please pronounce ~”. The utterance prompting unit 1109 may or may not include an output device such as the display 344 or a speaker. The utterance prompting unit 1109 can be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the voice processing apparatus will be described with reference to the flowcharts of FIGS. In the flowchart of FIG. 12 etc., only steps different from the flowcharts of FIG. 2 and FIG. 3 will be described.

  (Step S <b> 1201) The utterance prompting unit 1109 displays on the display, for example, utter the vowel “U” in order to prompt the utterance for calculating the second sampling frequency.

  (Step S1202) The voice receiving unit 103 determines whether a voice is received. If the voice is accepted, the process goes to step S1203. If the voice is not accepted, the process goes to step S213.

  (Step S1203) The sampling unit 106 reads the first sampling frequency stored in the first sampling frequency storage unit 105, samples the sound received in step S1202 at the first sampling frequency, and obtains the first sound data. obtain.

  (Step S1204) The vocal tract length normalization processing unit 109 obtains a second sampling frequency from the first speech data obtained in Step S1203. The second sampling frequency calculation process will be described with reference to the flowchart of FIG.

  (Step S1205) The utterance prompting unit 1109 displays on the display, for example, “speak“ right ”in order to urge the utterance for evaluation.

  (Step S1206) The voice receiving unit 103 determines whether a voice is received. If a voice is accepted, the process goes to step S1207, and if no voice is accepted, the process goes to step S213.

  (Step S1207) The sampling unit 106 reads the first sampling frequency stored in the first sampling frequency storage unit 105, samples the audio received in step S1206 at the first sampling frequency, and obtains the first audio data. obtain.

  (Step S1208) The vocal tract length normalization processing unit 109 resamples the first audio data obtained in step S1207 at the second sampling frequency obtained in step S1204, and obtains second audio data.

  (Step S1209) The voice processing unit 1110 performs a rating process on the second voice data obtained in step S1208. Details of the rating process will be described with reference to the flowchart of FIG. The process returns to step S1202.

  In the flowchart of FIG. 12, the sound for calculating the second sampling frequency and the sound for rating may be the same or included.

  The second sampling frequency calculation process in step S1204 will be described using the flowchart of FIG. In the flowchart of FIG. 13, the processing from step S301 to step S305 in the flowchart of FIG. 3 is performed.

  The rating process in step S1209 will be described with reference to the flowchart in FIG.

  (Step S1401) The pronunciation period frame phoneme probability value acquisition unit 111032 calculates the forward likelihood and the backward likelihood of all states of all frames. Then, probability values for all frames and all states are obtained. Specifically, the pronunciation interval frame phoneme probability value acquisition unit 111032 calculates, for example, the posterior probability that each feature vector is generated from the target state. This posterior probability corresponds to the occupation frequency appearing in the Baum-Welch algorithm in the maximum likelihood estimation of the HMM. The Baum-Welch algorithm is a known algorithm and will not be described.

  (Step S1402) The sounding section frame phoneme probability value acquisition unit 111032 calculates optimum state probability values of all frames.

  (Step S1403) The pronunciation period frame phoneme probability value acquisition unit 111032 substitutes 1 for j.

  (Step S1404) The pronunciation period frame phoneme probability value acquisition unit 111032 determines whether or not the jth pronunciation period, which is the next evaluation target pronunciation period, exists. If the jth sound generation interval exists, the process goes to step S1405, and if the jth sound generation interval does not exist, the process returns to the upper function.

  (Step S1405) The sounding section frame phoneme probability value acquiring unit 111032 substitutes 1 for the counter k.

  (Step S1406) The pronunciation period frame phoneme probability value acquisition unit 111032 determines whether or not the kth frame is present in the jth pronunciation period. If the kth frame exists, the process goes to step S1407, and if the kth frame does not exist, the process jumps to step S1410.

  (Step S1407) The pronunciation period frame phoneme probability value acquisition unit 111032 acquires all probability values of phonemes including the optimal state of the kth frame.

  (Step S1408) The rating value calculation means 111033 calculates the rating value of one frame of speech using one or more probability values acquired in step S1407 as parameters.

  (Step S1409) The sounding section frame phoneme probability value acquiring unit 111032 increments k by 1. The process returns to step S1406.

  (Step S1410) The rating value calculation means 111033 calculates the rating value of the j-th pronunciation section. For example, the rating value calculation unit 1111033 obtains, for each frame, a sum of one or more probability values in the entire phoneme state having the optimal state of each frame determined by the optimal state determination unit 11031, and the probability value for each frame. Based on the sum total, the time average value of the sum total of the probability values of the sounding sections is calculated as the speech evaluation value of the sounding section.

  (Step S1411) The output means 1104 outputs the rating value calculated in step S1410.

  (Step S1412) The sound generation section frame phoneme probability value acquiring unit 111032 increments j by 1. The process returns to step S1404.

  Hereinafter, a specific operation of the speech processing apparatus according to the present embodiment will be described. In the present embodiment, the rating value calculation algorithm is different from that of the first embodiment, and therefore the operation will be mainly described.

  First, a learner (evaluator) inputs an operation start instruction that is an instruction to start language learning. Then, the voice processing device receives the operation start instruction, and next, the utterance prompting unit 1109 outputs, for example, “Please say“ U ”” to the screen.

  Here, for example, the learner is a child as in the first embodiment. Further, it is assumed that the teacher who has spoken to create the teacher data is a native adult. This also applies to the description of specific examples of other embodiments.

  Then, the evaluation target person utters “U”, and the speech processing apparatus obtains the second sampling frequency “32.1 KHz” from the utterance. Such processing is the same as the processing described in the first embodiment.

  Next, the utterance prompting unit 1109 outputs, for example, “Please utter“ right ”” on the screen. Then, the learner pronounces the voice “right” to be learned. Then, the voice receiving unit 103 receives an input of a voice pronounced by the learner.

  Next, the sampling unit 106 samples the received voice “right” at the sampling frequency “22.05 KHz”. Then, the sampling unit 106 obtains the first audio data “right”.

  Next, the vocal tract length normalization processing unit 109 resamples the first voice data of “right” at the second sampling frequency “32.1 KHz”. Then, the vocal tract length normalization processing unit 109 obtains second voice data. Next, the audio processing unit 1110 processes the second audio data as follows.

  First, the frame segmentation unit 1101 segments the second audio data “right” into short frames.

Then, the frame audio data acquisition unit 1102 performs spectrum analysis on the audio data classified by the frame classification unit 1101 and calculates a feature vector series “O = o ₁ , o ₂ ,..., O _T ”.

  Next, the pronunciation interval frame phoneme probability value acquisition unit 111032 calculates a posteriori probability (probability value) of each state of each frame. The probability value can be calculated by the above-described Equations 1 and 2.

Then, the optimal state determination unit 11031, based on the feature vector o _t constituting the obtained feature vector series to determine the optimum conditions for each frame (optimum condition for the feature vector o _t). That is, the optimum state determination unit 11031 obtains an optimum state sequence. In addition, the processing order of the process which calculates the posterior probability (probability value) of each state of each frame and the process which determines an optimal state is not ask | required.

Next, the pronunciation interval frame phoneme probability value acquisition unit 111032 acquires, for each pronunciation interval, all probability values of phonemes including the optimum state of each frame included in the pronunciation interval. Then, the rating value calculation unit 111033 calculates the sum of all probability values of phonemes including the optimum state of each frame for each frame. Then, the rating value calculation means 111033 averages the sum of the probability values calculated for each frame for each sounding section, and calculates a rating value for each sounding section. Specifically, the rating value calculation unit 111033 calculates the rating value using Equation 3. In Equation 3, p-DAP (τ) is a rating value calculated to represent the optimal posterior probability (probability value) of all phonemes in each frame, and can be calculated by Equation 4. . Note that tp-DAP in Equation 3 is a rating value calculated by evaluating the probability values of all states in the phoneme including the optimal state in the pronunciation interval. Further, in Equation 3, Τ (q _t ^* ) is a pronunciation interval of the evaluation target including the HMM including the state q _t ^* . | Τ (q _t ^* ) | is the section length of Τ (q _t ^* ). Further, in Equation 4, P (q _t ^* ) is a set of all state identifiers possessed by the HMM including the state q _t ^* .

  An output example of the rating value (also referred to as “tp-DAP score”) calculated by the rating value calculation unit 111033 is shown in the table of FIG. FIG. 15 shows the evaluation results of American men and Japanese men. Phoneme and Word indicate the range of time average in tp-DAP. Here, the time average of p-DAP is adopted instead of DAP. In FIG. 15, the American male pronunciation rating value is higher than the Japanese male pronunciation value, and a good rating result is obtained.

  Then, the output means 1104 sequentially outputs the calculated rating values for each calculated sounding section (here, for each phoneme). An example of such output is shown in FIG.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. In this case, according to the present embodiment, highly accurate evaluation can be performed without being affected by individual differences, particularly differences in vocal tract length.

  Further, according to the present embodiment, since the optimum state is obtained by using the concatenated HMM, which is a concatenated HMM, and the evaluation value is calculated, the evaluation value can be obtained at high speed. Therefore, as described in the specific example above, it is possible to output a rating value for each pronunciation interval in real time. Further, according to the present embodiment, the posterior probability based on the dynamic programming is calculated as the probability value, so that the rating value can be obtained at higher speed.

  In addition, according to the present embodiment, the rating value can be calculated in units of pronunciation intervals, and compared with the state unit DAP as in the first embodiment, the degree of similarity originally desired to be measured (similarity of pronunciation intervals) ) With high accuracy and stability.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Target voice formant frequency) ", the voice received in the voice receiving step is subjected to a sampling process to obtain a second voice data, a vocal tract length normalization processing step, and a voice process to process the second voice data A program for executing a step.

  In the above program, the audio processing step includes a frame dividing step for dividing the second audio data into frames, and a frame audio data acquiring step for obtaining one or more frame audio data which are audio data for each of the divided frames. And a rating step for rating the received voice based on the teacher data and the one or more frame voice data, and an output step for outputting a rating result in the rating step.

In the above program, the rating step includes an optimum state determination step for determining an optimum state of the one or more frame sound data, and an overall phoneme state having an optimum state for each frame determined in the optimum state determination step. One or more probability values are acquired for each sounding section, and a sounding section frame phoneme probability value acquiring step and one or more probability values for one or more sounding sections acquired in the sounding section frame phoneme probability value acquiring step are used as parameters. It is preferable to include a rating value calculation step for calculating a rating value of speech.
(Embodiment 3)

  In the present embodiment, a description will be given of a speech processing apparatus that can accurately evaluate the similarity between a comparison target speech and an input speech. In particular, the speech processing apparatus is a speech processing apparatus that can detect a silent section and can evaluate similarity in consideration of the silent section.

  FIG. 17 is a block diagram of the speech processing apparatus according to this embodiment. The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. The evaluation subject person formant frequency storage unit 108, the vocal tract length normalization processing unit 109, the voice processing unit 1710, and the utterance prompting unit 1109 are provided.

  The audio processing unit 1710 includes a frame classification unit 1101, a frame audio data acquisition unit 1102, a special audio detection unit 17101, a rating unit 17103, and an output unit 1104.

  The special voice detection unit 17101 includes a silence data storage unit 171101 and a silence interval detection unit 171012.

  The rating unit 17103 includes an optimum state determining unit 11031, an optimum state probability value acquiring unit 11032, and a rating value calculating unit 171033.

  The special sound detection unit 17101 detects that a special sound is input based on the input sound data for each frame. Here, the special voice includes silence. Further, the special speech detection unit 17101 acquires, for example, the probability value of the optimal state of the frame in a certain phoneme section, and the sum of one or more probability values of a certain phoneme section is lower than a predetermined value (assumed If it can be determined that it is not a phoneme), it is detected that a special voice is input in the phoneme section. An example of a specific algorithm for such detection will be described later. The special sound detection unit 17101 can be realized usually by an MPU, a memory, or the like. The processing procedure of the special sound detection means 17101 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The silence data storage unit 171101 is data indicating silence and stores silence data that is data based on the HMM. The silent data storage unit 171011 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The silent section detecting means 171012 detects a silent section based on the frame sound data acquired by the frame sound data acquiring means 1102 and the silent data of the silent data storage means 171101. When the similarity between the frame audio data acquired by the frame audio data acquisition unit 1102 and the silence data is equal to or higher than a predetermined value, the silence interval detection unit 171012 determines that the frame audio data is data of the silence interval. Also good. Further, the silent section detecting means 171012 has a probability value acquired by the optimum state probability value acquiring means 11032 described below equal to or less than a predetermined value, and the frame audio data and silent data acquired by the frame audio data acquiring means 1102 When the degree of similarity is equal to or greater than a predetermined value, the frame audio data may be determined to be silent section data. The silent section detection means 171012 can be usually realized by an MPU, a memory, or the like. The processing procedure of the silent section detection means 171012 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The rating unit 17103 evaluates the voice received by the voice receiving unit 103 based on the teacher data, the input voice data, and the detection result of the special voice detecting unit 17101. “Based on the detection result in the special voice detection unit 17101” means, for example, that the silent section is ignored when the special voice detection unit 17101 detects silence. Further, “based on the detection result in the special voice detection unit 17101” means that, for example, when the special voice detection unit 17101 detects replacement or dropout, the evaluation value is set to a predetermined numerical value by detecting the replacement or dropout. Subtract and calculate the rating value. The rating means 17103 can be usually realized by an MPU, a memory, or the like. The processing procedure of the rating means 17103 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The rating value calculation unit 171033 calculates a voice rating value using the probability value acquired by the optimum state probability value acquisition unit 11032 as a parameter, excluding the silent section detected by the silent segment detection unit 171012. Note that it does not matter how the rating value calculation means 171033 uses the probability value to calculate the rating value. The rating value calculation unit 171033 calculates a speech rating value using, for example, the sum of the probability value acquired by the optimum state probability value acquisition unit 11032 and the probability value in all states of the frame corresponding to the probability value as a parameter. Here, the rating value calculation means 21023 normally calculates a rating value for each frame except for the silent section detected by the silent section detection means 171012. Note that the rating value calculation unit 171033 does not necessarily calculate the rating value except for the silent section. The rating value calculation means 171033 may calculate the rating value so as to reduce the influence of the silent section. The rating value calculation unit 171033 can be realized usually by an MPU, a memory, or the like. The processing procedure of the rating value calculation means 171033 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the speech processing apparatus will be described using the flowcharts of FIGS. Note that the flowchart of FIG. 18 is different from the flowchart of FIG. 12 only in the rating process of step S1801, and therefore the description of the flowchart of FIG. 18 is omitted. Details of the rating process in step S1801 will be described with reference to the flowchart of FIG.

  (Step S1901) The rating means 17103 calculates a DAP rating value. Since the algorithm for calculating the DAP rating value has been described in the first embodiment, a detailed description thereof will be omitted. The process of calculating the DAP rating value is performed by the processes of steps S211 to S214 in the flowchart of FIG.

  (Step S1902) The special sound detection unit 17101 determines whether or not the value calculated in step S1901 is lower than a predetermined value. If it is lower than the predetermined value, the process goes to step S1903, and if it is not lower than the predetermined value, the process jumps to step S1906.

  (Step S1903) The silent section detecting means 171012 acquires the probability values of the silent data and all the teacher data.

  (Step S1904) The silent section detection means 171012 determines whether or not the probability value of silent data is the highest among the probability values acquired in step S1903. If the silence data has the highest probability value (in this case, it is determined that it is a silent section), the procedure goes to step S1905. If the silence data has a highest probability value, the procedure goes to step S1906.

  (Step S1905) The silent section detecting means 171012 increments the counter i by 1. The process returns to step S208.

  (Step S1906) The output unit 1104 outputs the rating value calculated in step S1901.

  In the flowchart of FIG. 19, the output unit 1104 does not output the rating value of the section determined to be a silent section (ignoring the silent section), but indicates that the section where the special voice is detected is a silent section. You may output in the aspect which specifies clearly that a silence area exists. The rating value calculation means 171033 preferably calculates the score while ignoring the rating value in the silent section when calculating the score in the sounding section or higher units. The influence of the value may be set to 1/10, for example, and the score of the pronunciation interval or the entire pronunciation may be calculated. The rating unit 17103 may evaluate the voice received by the voice receiving unit 103 based on the teacher data, the input voice data, and the detection result of the special voice detecting unit 17101.

  In the flowchart of FIG. 19, the special voice detection unit 17101 detects the special voice based on the DAP score of the i-th frame voice data. For example, the special voice detection unit 17101 uses the tp-DAP score described in the second embodiment. Special voices may be detected based on this.

  Hereinafter, a specific operation of the speech processing apparatus according to the present embodiment will be described. In the present embodiment, since the rating value is calculated in consideration of the silent section, the rating value calculation algorithm is different from that of the first and second embodiments. Therefore, the different processing will be mainly described.

  First, a learner (evaluator) inputs an operation start instruction that is an instruction to start language learning. Then, the voice processing apparatus accepts the operation start instruction, and then outputs, for example, “Please say“ U ”” to the screen.

  Then, the evaluation target person utters “U”, and the speech processing apparatus obtains the second sampling frequency “32.1 KHz” from the utterance. Such processing is the same as the processing described in the first embodiment.

  Next, the utterance prompting unit 1109 outputs, for example, “Please utter“ right ”” on the screen. Then, the learner pronounces the voice “right” to be learned. Then, the voice receiving unit 103 receives an input of a voice pronounced by the learner.

  Next, the sampling unit 106 samples the received voice “right” at the sampling frequency “22.05 KHz”. Then, the sampling unit 106 obtains the first audio data “right”.

  Next, the vocal tract length normalization processing unit 109 performs a resampling process on the first voice data “right” at the second sampling frequency “32.1 KHz”. Then, the vocal tract length normalization processing unit 109 obtains second voice data. Next, the audio processing unit 1710 processes the second audio data as follows.

  First, the frame segmentation unit 1101 segments the second audio data “right” into short frames.

Then, the frame audio data acquisition unit 1102 performs spectrum analysis on the audio data classified by the frame classification unit 1101 and calculates a feature vector series “O = o ₁ , o ₂ ,..., O _T ”.

Then, the optimal state determination unit 11031, based on the feature vector o _t constituting the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t).

  Next, the optimum state probability value acquisition unit 11032 calculates the probability value in the optimum state according to the above-described formulas 1 and 2.

  Next, the rating value calculation unit 171033 acquires, for example, one or more probability values in the entire state of the phoneme having the optimal state determined by the optimal state determination unit 11031, and the sum of the one or more probability values is used as a parameter for the speech. The rating value of is calculated. That is, the rating value calculation unit 171033 calculates, for example, a DAP score for each frame.

  Then, the special voice detection unit 17101 determines whether or not a special voice has been input, using a rating value (DAP score) corresponding to the calculated frame. Specifically, for example, if the rating value calculated for the evaluation target frame is lower than a predetermined numerical value, the special voice detection unit 17101 determines that a special voice has been input. Note that the special voice detection unit 17101 does not need to determine that a special voice has been input immediately because the rating value corresponding to one frame is small. That is, the special voice detection unit 17101 determines that a section corresponding to the group of consecutive frames is a section where a special voice is input when a predetermined number or more of frames having a small evaluation value corresponding to the frame are consecutive. May be.

  FIG. 20 illustrates a case where the special sound detection unit 17101 detects special sound. In FIG. 20A, the vertical axis represents the DAP score, and the horizontal axis represents the frame. In FIG. 20A, (V) shows Viterbi alignment. In FIG. 20A, it is determined that the DAP score in the shaded frame group is lower than a predetermined value and is a special speech section.

  Next, if it is determined that a special voice has been input, the silent section detecting unit 171012 acquires the silent data from the silent data storage unit 171011, calculates the similarity between the frame group and the silent data, If the degree is equal to or greater than a predetermined value, it is determined that the audio data corresponding to the frame group is silent data. FIG. 20B shows that as a result of the comparison with the silence data, the value of the posterior probability (“DAP score”) indicating the similarity to the silence data is high. As a result, the silent section detector 171012 determines that the special speech section is a silent section. In FIG. 20A, the DAP score in the shaded frame group is lower than a predetermined value and is determined to be a special speech section, and as a result of comparison with silence data, the DAP score is If it is low, it is determined that it is not a silent section. In such a section, for example, the pronunciation is simply not good and a low rating value is output. Note that, as shown in FIG. 20 (a), the silence period usually includes the second half of the last phoneme of the first word ("word1") and the second word ("word2" following the first word). )) The first half of the first phoneme score is low.

  Then, the output unit 1104 ignores the rating value to be output so as not to consider the rating value of the silent data section.

  Then, the output unit 1104 outputs a rating value corresponding to each frame. In this case, for example, the rating value of the silent data section is not output.

  Examples of output modes of such rating values are, for example, FIG. 9 and FIG.

  Note that the output performed by the output unit 1104 may be an output that only indicates the presence of a silent section.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. In this case, according to the present embodiment, highly accurate evaluation can be performed without being affected by individual differences, particularly differences in vocal tract length. Furthermore, since the speech processing apparatus evaluates the similarity in consideration of the silent section, a very accurate evaluation result can be obtained.

  It is preferable to ignore the silent section data and calculate the evaluation result. However, in this embodiment, for example, the evaluation value is output in consideration of the data of the silent section by a method other than ignoring, such as reducing the influence of the evaluation of the silent section compared with other sections. Also good.

  Further, according to the specific example of the present embodiment, the rating value is calculated using the DAP score. However, the rating value may be calculated in consideration of the silent section, and the other algorithm (tp) described above is used. The rating value may be calculated by DAP or the like) or another algorithm not described in this specification. In other words, according to the present embodiment, the voice received by the voice receiving unit is evaluated based on the teacher data, the input voice data, and the detection result of the special voice detection unit, and in particular, the evaluation is performed in consideration of silent data. What is necessary is just to calculate a value.

  In addition, according to the present embodiment, first, a section having a low DAP score is detected, and then a silent section is detected. However, a silent section may be detected by comparing with silent data without detecting a section having a low DAP score.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Target voice formant frequency) ", the voice received in the voice receiving step is subjected to a sampling process to obtain a second voice data, a vocal tract length normalization processing step, and a voice process to process the second voice data A program for executing a step.

  In the above program, the audio processing step includes a frame dividing step for dividing the second audio data into frames, and a frame audio data acquiring step for obtaining one or more frame audio data which are audio data for each of the divided frames. And, based on the input voice data for each frame, a special voice detection step for detecting that a special voice has been input, and based on detection results in the teacher data, the input voice data, and the special voice detection step, It is preferable that a rating step for rating the received voice and an output step for outputting a rating result in the rating step are preferable.

In the above program, it is preferable that the special voice detecting step detects a silent section based on silent data which is data based on HMM indicating silent and the input voice data.
(Embodiment 4)

  In the present embodiment, a speech processing apparatus that detects special speech in input speech and can accurately evaluate the similarity between the speech to be compared and the input speech will be described. In particular, the speech processing apparatus is a speech processing apparatus that can detect insertion of phonemes.

  FIG. 21 is a block diagram of the speech processing apparatus according to this embodiment. The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. The evaluation subject person formant frequency storage unit 108, the vocal tract length normalization processing unit 109, the voice processing unit 2110, and the utterance prompting unit 1109 are provided.

  The audio processing unit 2110 includes a frame classification unit 1101, a frame audio data acquisition unit 1102, a special audio detection unit 21101, a rating unit 21103, and an output unit 21104. The rating unit 21103 includes an optimum state determination unit 11031 and an optimum state probability value acquisition unit 11032.

  The special voice detection unit 21101 detects that the rating values of the second half of one phoneme and the first half of the next phoneme after the phoneme satisfy a predetermined condition. The length of the second half and the first half is not limited. The special sound detection means 21101 can be usually realized by an MPU, a memory, or the like. The processing procedure of the special sound detection means 21101 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The rating means 21103 constitutes a rating result indicating that at least a phoneme has been inserted when the special voice detecting means 21101 detects that a predetermined condition is satisfied. Note that the rating means 21103 determines whether or not silence has been inserted in the section in which the special voice detection means 21101 has detected that the predetermined condition is satisfied by the algorithm described in the third embodiment, and silence has been inserted. If not, it may be detected that another phoneme has been inserted. The rating means 21103 calculates a probability value for another phoneme HMM when no silence is inserted, and obtains a rating result that a phoneme having a probability value higher than a predetermined value is inserted. Also good. It can be said that the detection of the silent section described in the third embodiment is the detection of the insertion of a silent phoneme. The rating means 21103 can be usually realized by an MPU, a memory, or the like. The processing procedure of the rating means 21103 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 21104 outputs the rating result in the rating unit 21103. The rating result here includes a rating result indicating that a phoneme has been inserted. The rating result may be only the rating value (score) calculated by subtracting a predetermined numerical value when a phoneme is inserted. In addition, the rating result may be both that the phoneme has been inserted and the rating value (score). In addition, when the insertion of the phoneme which is not assumed in the teacher data is detected, the rating value is usually low. Here, the output is a concept including display on a display, printing on a printer, sound output, transmission to an external device, accumulation in a recording medium, and the like. The output unit 21104 may or may not include an output device such as a display or a speaker. The output means 21104 can be implemented by output device driver software, or output device driver software and an output device.

  Next, the operation of the speech processing apparatus will be described using the flowcharts of FIGS. Note that the flowchart of FIG. 22 is different from the flowchart of FIG. 12 only in the rating process in step S2201, and thus the description of the flowchart of FIG. 22 is omitted. Details of the rating process in step S2201 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 23, the description of the same processing as that of the flowcharts of FIGS. 2 and 19 is omitted.

  (Step S2301) The special sound detection unit 21101 determines whether data is stored in a buffer that temporarily accumulates data corresponding to a frame. The stored data is frame audio data evaluated as a rating value lower than a predetermined value in step S1902, or data that can be acquired from the frame audio data. If the data is stored, the process goes to step S2307, and if the data is not stored, the process returns to the upper function.

  (Step S2302) The special sound detection unit 21101 determines whether data is stored in the buffer. If data is stored, go to step S2307, and if data is not stored, go to step S2303.

  (Step S2303) The output means 21104 outputs the rating value calculated in step S1901.

  (Step S2304) The special sound detection means 21101 increments the counter i by 1. The process returns to step S208.

  (Step S2305) The special sound detection means 21101 temporarily stores in the buffer frame audio data evaluated as a rating value lower than a predetermined value or data obtainable from the frame sound data.

  (Step S2306) The special sound detection means 21101 increments the counter i by 1. The process returns to step S208.

  (Step S2307) The special sound detection means 21101 assigns 1 to the counter j.

  (Step S2308) The special sound detection unit 21101 determines whether or not the j-th data exists in the buffer. If the jth data exists, the process goes to step S2309, and if the jth data does not exist, the process jumps to step S2315.

  (Step S2309) The special voice detecting unit 21101 acquires a phoneme in an optimal state corresponding to the j-th data.

  (Step S2310) The special speech detection unit 21101 calculates the probability value of all the teacher data for the j-th data, and acquires the phoneme having the maximum probability value.

  (Step S2311) The special voice detection unit 21101 determines whether the phoneme acquired in step S2309 and the phoneme acquired in step S2310 are different phonemes. If the phoneme is different, the process goes to step S2312, and if not, the process jumps to step S2314.

  (Step S2312) The rating means 21103 constitutes a rating result indicating that a phoneme has been inserted.

  (Step S2313) The special sound detection means 21101 increments the counter j by 1. The process returns to step S2308.

  (Step S2314) The output means 21104 outputs all rating values corresponding to all data in the buffer. Here, the total rating value is, for example, a DAP score for each frame. Go to step S2313.

  (Step S2315) The output unit 21104 determines whether or not the information of “insertion” is included in the evaluation result. If “insertion” information is included, the process proceeds to step S2316, and if “insertion” information is not included, the process proceeds to step S2317.

  (Step S2316) The output means 21104 outputs a rating result.

  (Step S2317) The output means 21104 clears the buffer. The process returns to step S208.

  In the flowchart of FIG. 23, if a frame with a low rating value exists across two phonemes, it is determined that a phoneme has been inserted. That is, it is determined that a phoneme has been inserted when the rating value of the second half of at least one phoneme (at least the final frame) and the first frame of the next phoneme after the phoneme are lower than a predetermined value. However, in the flowchart of FIG. 23, when the evaluation values of the latter half of the predetermined phoneme and the first half of the phoneme next to the phoneme are all lower than the predetermined value, there is no phoneme insertion. You may make it judge that it was.

  Hereinafter, a specific operation of the speech processing apparatus according to the present embodiment will be described. In the present embodiment, the processing for detecting insertion of phonemes is different from that in the third embodiment. Therefore, the different processing will be mainly described.

  First, a learner (evaluator) inputs an operation start instruction that is an instruction to start language learning. Then, the voice processing apparatus receives the operation start instruction, and then outputs, for example, “Please say“ A ”” to the screen.

  Then, the learner utters “A”, and the speech processing apparatus obtains the second sampling frequency “32.1 KHz” from the utterance. Such processing is the same as the processing described in the first embodiment.

  Next, the utterance prompting unit 1109 outputs, for example, “Please utter“ right ”” on the screen. Then, the learner pronounces the voice “right” to be learned. Then, the voice receiving unit 103 receives an input of a voice pronounced by the learner.

  Next, the sampling unit 106 samples the received voice “right” at the sampling frequency “22.05 KHz”. Then, the sampling unit 106 obtains the first audio data “right”.

  Next, the vocal tract length normalization processing unit 109 resamples the first voice data of “right” at the second sampling frequency “32.1 KHz”. Then, the vocal tract length normalization processing unit 109 obtains second voice data. Next, the audio processing unit 2110 processes the second audio data as follows.

  First, the frame segmentation unit 1101 segments the second audio data “right” into short frames.

Then, the frame audio data acquisition unit 1102 performs spectrum analysis on the audio data classified by the frame classification unit 1101 and calculates a feature vector series “O = o ₁ , o ₂ ,..., O _T ”.

Then, the optimal state determination unit 11031 of the evaluation unit 21103, based on the feature vector o _t constituting the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t).

  Next, the optimum state probability value acquisition unit 11032 calculates the probability value in the optimum state according to the above-described formulas 1 and 2.

  Next, the rating unit 21103 acquires, for example, one or more probability values in the overall state of the phoneme having the optimal state determined by the optimal state determining unit 11031, and evaluates the speech using the sum of the one or more probability values as a parameter. Calculate the value. That is, the rating unit 21103 calculates a DAP score for each frame, for example. Here, the calculated score may be the above-described tp-DAP score or the like.

  Then, the special voice detection unit 21101 determines whether or not a special voice has been input using the rating value corresponding to the calculated frame. That is, it is determined whether or not there is a section where the rating value (for example, DAP score) is lower than a predetermined value.

  Next, as shown in FIG. 24, the special voice detection unit 21101 determines whether or not a section where the rating value (for example, DAP score) is lower than a predetermined value straddles two phonemes. If the phoneme is straddled, it is determined that the phoneme is inserted in the section. An example of a detailed algorithm in such a case has been described with reference to FIG. In FIG. 24, the shaded area is a section in which an unexpected phoneme is inserted.

  Next, the rating means 21103 constitutes a rating result (for example, “an unexpected phoneme has been inserted”) indicating that a phoneme has been inserted. When an unexpected phoneme is inserted, it is preferable that the rating unit 21103 calculates the rating value by subtracting a predetermined numerical value, for example. Then, the output unit 21104 outputs a configured rating result (which may include a rating value). FIG. 25 is an output example of the evaluation result. Note that the output means 21104 preferably outputs the rating value as described above for normal input speech.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. In this case, according to the present embodiment, highly accurate evaluation can be performed without being affected by individual differences, particularly differences in vocal tract length. Furthermore, since the speech processing apparatus can detect special speech, particularly unexpected insertion of phonemes, a highly accurate evaluation result can be obtained.

  In the present embodiment, it is only necessary to detect the insertion of phonemes, and the rating value calculation algorithm is not limited. The algorithm for calculating the rating value may be the above-described algorithm (DAP, tp-DAP), or another algorithm not described in this specification.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Target voice formant frequency) ", the voice received in the voice receiving step is subjected to a sampling process to obtain a second voice data, a vocal tract length normalization processing step, and a voice process to process the second voice data A program for executing a step.

  In the above program, the audio processing step includes a frame dividing step for dividing the second audio data into frames, and a frame audio data acquiring step for obtaining one or more frame audio data which are audio data for each of the divided frames. And, based on the input voice data for each frame, a special voice detection step for detecting that a special voice has been input, and based on detection results in the teacher data, the input voice data, and the special voice detection step, It is preferable that a rating step for rating the received voice and an output step for outputting a rating result in the rating step are preferable.

In the above program, it is preferable that the special speech detection step detects that the rating values of the second half of one phoneme and the first half of the next phoneme satisfy a predetermined condition.
(Embodiment 5)

  In the present embodiment, a speech processing apparatus that detects special speech in input speech and can accurately evaluate the similarity between the speech to be compared and the input speech will be described. In particular, the speech processing apparatus is a speech processing apparatus that can detect phoneme replacement.

  FIG. 26 is a block diagram of the speech processing apparatus according to this embodiment. The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. The evaluation subject person formant frequency storage unit 108, the vocal tract length normalization processing unit 109, the voice processing unit 2610, and the utterance prompting unit 1109 are provided.

  The audio processing unit 2610 includes a frame classification unit 1101, a frame audio data acquisition unit 1102, a special audio detection unit 26101, a rating unit 26103, and an output unit 21104. The rating unit 26103 includes an optimum state determination unit 11031 and an optimum state probability value acquisition unit 11032.

  The audio processing unit 2610 processes the second audio data. The audio processing unit 2610 includes special audio detection means 26101 that detects that special audio is input based on input audio data for each frame. The audio processing unit 2610 can usually be realized by an MPU, a memory, or the like. The processing procedure of the audio processing unit 2610 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The special voice detection means 26101 detects that the rating value of one phoneme is lower than a predetermined value. The special speech detection means 26101 also detects that the rating value of one phoneme is lower than a predetermined value, and that the phoneme immediately before the phoneme and the rating value of the phoneme immediately after the phoneme are higher than a predetermined value. May be. Further, the special voice detecting means 26101 detects that the rating value of one phoneme is lower than a predetermined value and that the rating value calculated based on the HMM of an unexpected phoneme is higher than the predetermined value. Also good. In other words, the special voice detection means 26101 only needs to be able to detect phoneme replacement by a predetermined algorithm. Various algorithms are conceivable. The special sound detection means 26101 can be usually realized by an MPU, a memory, or the like. The processing procedure of the special sound detection means 26101 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The rating unit 26103 constitutes a rating result indicating that at least the phoneme has been replaced when the special voice detecting unit 26101 detects that the predetermined condition is satisfied. The rating means 26103 may calculate a rating value (score) calculated by subtracting a predetermined numerical value when a phoneme is replaced. The rating means 26103 can usually be realized by an MPU, a memory, or the like. The processing procedure of the rating means 26103 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the speech processing apparatus will be described using the flowcharts of FIGS. Note that the flowchart of FIG. 27 is different from the flowchart of FIG. 12 only in the rating process of step S2701, and thus the description of the flowchart of FIG. 27 is omitted. Details of the rating process in step S2701 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 28, the description of the same processes as those in the flowcharts of FIGS. 2, 19, and 23 is omitted.

  (Step S2801) The special sound detection means 26101 determines whether or not the frame sound data group corresponding to the data stored in the buffer corresponds to one phoneme. If it is one phoneme, go to step S2802, and if it is not one phoneme, go to step S2810.

  (Step S2802) The special sound detection means 26101 calculates the rating value of the phoneme immediately before the phoneme of the frame sound data group corresponding to the data stored in the buffer. Such a rating value is, for example, the DAP score described above. Note that the immediately preceding phoneme is the immediately preceding phoneme with respect to the currently rated phoneme. Phoneme breaks can be calculated by the Viterbi algorithm.

  (Step S2803) The special sound detection unit 26101 determines whether or not the rating value calculated in Step S2802 is equal to or greater than a predetermined value. If it is equal to or larger than the predetermined value, the process goes to step S2804, and if it is smaller than the predetermined value, the process goes to step S2810.

  (Step S2804) The special speech detection means 26101 calculates the rating value of the immediately following phoneme. Such a rating value is, for example, the DAP score described above. The phoneme immediately after is the phoneme immediately after the phoneme currently being evaluated.

  (Step S2805) The special sound detection means 26101 determines whether or not the rating value calculated in step S2804 is equal to or greater than a predetermined value. If it is equal to or greater than the predetermined value, the process proceeds to step S2806, and if it is smaller than the predetermined value, the process proceeds to step S2810.

  (Step S2806) The special speech detection means 26101 determines whether or not there is one phoneme HMM that can obtain a rating value equal to or higher than a predetermined value among the phoneme HMMs stored in advance (except for the expected phoneme HMM). Determine whether. If there is a phoneme HMM that obtains a rating value greater than or equal to a predetermined value, the process proceeds to step S2807, and if there is no phoneme HMM that yields a rating value greater than or equal to the predetermined value, the process proceeds to step S2810. The phoneme HMM stored in advance is usually a large number of phoneme HMMs for all phonemes. In this step, the probability value of the phoneme HMM stored in advance is calculated, the phoneme having the maximum probability value is obtained, it is determined whether or not the phoneme in the optimum state is different from the phoneme in the optimum state. It may be determined that the phoneme has been replaced.

  (Step S2807) The rating means 26103 constitutes a rating result indicating that the phoneme has been replaced.

  (Step S2808) The output unit 21104 outputs the rating result configured in step S2807.

  (Step S2809) The output means 21104 clears the buffer. The process returns to step S208.

  (Step S2810) The output means 21104 outputs all rating values corresponding to all data in the buffer. Go to step S2809.

  Hereinafter, a specific operation of the speech processing apparatus according to the present embodiment will be described. In the present embodiment, the processing for detecting phoneme replacement is different from that in the fourth embodiment. Therefore, the different processing will be mainly described.

  First, a learner (evaluator) inputs an operation start instruction that is an instruction to start language learning. Then, the voice processing apparatus accepts the operation start instruction, and then outputs, for example, “Please say“ U ”” to the screen.

  Then, the evaluation target person utters “U”, and the speech processing apparatus obtains the second sampling frequency “32.1 KHz” from the utterance. Such processing is the same as the processing described in the first embodiment.

  Next, the utterance prompting unit 1109 outputs, for example, “Please utter“ right ”” on the screen. Then, the learner pronounces the voice “right” to be learned. Then, the voice receiving unit 103 receives an input of a voice pronounced by the learner.

  Next, the sampling unit 106 samples the received voice “right” at the sampling frequency “22.05 KHz”. Then, the sampling unit 106 obtains the first audio data “right”.

  Next, the vocal tract length normalization processing unit 109 resamples the first voice data of “right” at the second sampling frequency “32.1 KHz”. Then, the vocal tract length normalization processing unit 109 obtains second voice data. Next, the audio processing unit 2610 processes the second audio data as follows.

  First, the frame segmentation unit 1101 segments the second audio data “right” into short frames.

Then, the frame audio data acquisition unit 1102 performs spectrum analysis on the audio data classified by the frame classification unit 1101 and calculates a feature vector series “O = o ₁ , o ₂ ,..., O _T ”.

Then, the optimal state determination unit 11031 of the evaluation unit 26103, based on the feature vector o _t constituting the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t).

  Next, the optimum state probability value acquisition unit 11032 calculates the probability value in the optimum state according to the above-described formulas 1 and 2.

  Next, the rating unit 26103 acquires, for example, one or more probability values in the overall state of the phoneme having the optimal state determined by the optimal state determining unit 11031, and evaluates the speech using the sum of the one or more probability values as a parameter. Calculate the value. That is, the rating unit 26103 calculates a DAP score for each frame, for example. Here, the calculated score may be the above-described tp-DAP score or the like.

  Then, the special voice detection unit 26101 determines whether or not a special voice has been input, using a rating value corresponding to the calculated frame. That is, it is determined whether or not there is a section where the rating value (for example, DAP score) is lower than a predetermined value.

  Next, as shown in FIG. 29, the special speech detection means 26101 determines whether or not a section where the rating value (for example, DAP score) is lower than a predetermined value is within one phoneme (here, phoneme 2). Judging. If the rating value is low in one phoneme, then the special speech detection means 26101 then calculates a rating value (for example, DAP score) for the immediately preceding phoneme (phoneme 1) and / or the immediately following phoneme (phoneme 3). If the rating value is higher than a predetermined value, it is determined that there is a possibility that phoneme replacement has occurred. Next, if there is one phoneme HMM in which a rating value equal to or higher than a predetermined value is present among the phoneme HMMs stored in advance (excluding the HMM of an expected phoneme), the special speech detection unit 26101 can generate a phoneme. It is determined that the replacement has occurred. In FIG. 29, the phoneme 2 is a section in which phoneme replacement has occurred. In FIG. 29, the vertical axis represents a rating value, and the rating value may be DAP, tp-DAP, or the like.

  Next, the rating means 26103 constitutes a rating result (for example, “phoneme replacement has occurred”) indicating that there has been a phoneme replacement. Then, the output means 21104 outputs the configured evaluation result. Note that the output means 21104 preferably outputs the rating value as described above for normal input speech.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. In this case, according to the present embodiment, highly accurate evaluation can be performed without being affected by individual differences, particularly differences in vocal tract length. Furthermore, since this speech processing apparatus can detect special speech, particularly phoneme substitution, it is possible to obtain a highly accurate evaluation result.

  In the present embodiment, it is only necessary to be able to detect phoneme replacement, and the rating value calculation algorithm is not limited. The algorithm for calculating the rating value may be the above-described algorithm (DAP, tp-DAP), or another algorithm not described in this specification.

  In the present embodiment, the phoneme replacement detection algorithm may be another algorithm. For example, in the detection of the replacement of phonemes, it may be essential to detect the replacement section to have a section longer than a predetermined length. In addition, various details of the replacement detection algorithm can be considered.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Target voice formant frequency) ", the voice received in the voice receiving step is subjected to a sampling process to obtain a second voice data, a vocal tract length normalization processing step, and a voice process to process the second voice data A program for executing a step.

  In the above program, the audio processing step includes a frame dividing step for dividing the second audio data into frames, and a frame audio data acquiring step for obtaining one or more frame audio data which are audio data for each of the divided frames. And, based on the input voice data for each frame, a special voice detection step for detecting that a special voice has been input, and based on detection results in the teacher data, the input voice data, and the special voice detection step, It is preferable that a rating step for rating the received voice and an output step for outputting a rating result in the rating step are preferable.

In the above program, the special speech detection step detects that the rating value of one phoneme satisfies a predetermined condition, and if the special speech detection step detects that the predetermined condition is satisfied, It is preferable to construct a rating result indicating that there has been a substitution.
(Embodiment 6)

  In the present embodiment, a speech processing apparatus that detects special speech in input speech and can accurately evaluate the similarity between the speech to be compared and the input speech will be described. In particular, the speech processing apparatus is a speech processing apparatus that can detect missing phonemes.

  FIG. 30 is a block diagram of the speech processing apparatus according to this embodiment. The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. The evaluation subject person formant frequency storage unit 108, the vocal tract length normalization processing unit 109, the voice processing unit 3010, and the utterance prompting unit 1109 are provided.

  The audio processing unit 3010 includes a frame classification unit 1101, a frame audio data acquisition unit 1102, a special audio detection unit 30101, a rating unit 30103, and an output unit 21104. The rating unit 30103 includes an optimum state determination unit 11031 and an optimum state probability value acquisition unit 11032.

  The special voice detection unit 30101 detects that the rating value of one phoneme is lower than a predetermined value, and that the phoneme immediately before the phoneme or the rating value of the phoneme immediately after the phoneme is higher than a predetermined value. Further, the special voice detection means 30101 has a rating value of one phoneme lower than a predetermined value, a rating value of a phoneme immediately before the phoneme or a phoneme immediately after the phoneme is higher than a predetermined value, and the phoneme of the phoneme It may be detected that the section length is shorter than a predetermined length. The special speech detection unit 30101 may detect that the probability value corresponding to the immediately preceding phoneme or the probability value corresponding to the immediately following phoneme is higher than the probability value of the one phoneme. In such a case, it is preferable that the special voice detection unit 30101 detects missing phonemes. Furthermore, by including that the segment length of the phoneme is shorter than a predetermined length in the missing condition, the accuracy of detecting missing phonemes is improved. The special sound detection means 30101 can be usually realized by an MPU, a memory, or the like. The processing procedure of the special sound detection means 30101 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The rating unit 30103 constitutes a rating result indicating that at least a phoneme is missing when the special voice detecting unit 30101 detects that a predetermined condition is satisfied. The rating means 30103 can usually be realized by an MPU, a memory, or the like. The processing procedure of the rating means 30103 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the speech processing apparatus will be described using the flowcharts of FIGS. Note that the flowchart of FIG. 31 is different from the flowchart of FIG. 12 only in the rating process in step S3101, and thus the description of the flowchart of FIG. 31 is omitted. Details of the rating process in step S3101 will be described with reference to the flowchart of FIG. In the flowchart of FIG. 32, the description of the same processes as those of the flowcharts of FIGS. 2, 19, 23, and 28 is omitted.

  (Step S3201) The special voice detection means 30101 schedules the probability value of the teacher data corresponding to the immediately preceding phoneme or the probability value of the teacher data corresponding to the immediately following phoneme with respect to the data stored in the buffer. It is determined whether or not the probability value of the teacher data corresponding to the current phoneme is higher. If it is higher, go to step S3202, otherwise go to step S2810. As a condition for going to step S3202, it may be added that the section length of the frame audio data group corresponding to the data stored in the buffer is equal to or less than a predetermined length.

  (Step S3202) The rating means 30103 constitutes a rating result indicating that a phoneme is missing. Go to step S2808.

  In the flowchart of FIG. 32, the section length of the phonemes to be evaluated (phonemes that will be missing) may be shorter than a predetermined length (for example, 3 frames), or there is no such condition. May be.

  Hereinafter, a specific operation of the speech processing apparatus according to the present embodiment will be described. In the present embodiment, the processing for detecting missing phonemes is different from that in the fifth embodiment. Therefore, the different processing will be mainly described.

  First, a learner (evaluator) inputs an operation start instruction that is an instruction to start language learning. Then, the voice processing apparatus accepts the operation start instruction, and then outputs, for example, “Please say“ U ”” to the screen.

  Then, the evaluation target person utters “U”, and the speech processing apparatus obtains the second sampling frequency “32.1 KHz” from the utterance. Such processing is the same as the processing described in the first embodiment.

  Next, the utterance prompting unit 1109 outputs, for example, “Please utter“ right ”” on the screen. Then, the learner pronounces the voice “right” to be learned. Then, the voice receiving unit 103 receives an input of a voice pronounced by the learner.

  Next, the sampling unit 106 samples the received voice “right” at the sampling frequency “22.05 KHz”. Then, the sampling unit 106 obtains the first audio data “right”.

  Next, the vocal tract length normalization processing unit 109 resamples the first voice data of “right” at the second sampling frequency “32.1 KHz”. Then, the vocal tract length normalization processing unit 109 obtains second voice data. Next, the audio processing unit 3010 processes the second audio data as follows.

  First, the frame segmentation unit 1101 segments the second audio data “right” into short frames.

Then, the frame audio data acquisition unit 1102 performs spectrum analysis on the audio data classified by the frame classification unit 1101 and calculates a feature vector series “O = o ₁ , o ₂ ,..., O _T ”.

Then, the optimal state determination unit 11031, based on the feature vector o _t constituting the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t).

  Next, the optimum state probability value acquisition unit 11032 calculates the probability value in the optimum state according to the above-described formulas 1 and 2.

  Next, the rating unit 30103 acquires, for example, one or more probability values in the overall state of the phoneme having the optimum state determined by the optimum state determining unit 11031, and evaluates the speech using the sum of the one or more probability values as a parameter. Calculate the value. That is, the rating unit 30103 calculates, for example, a DAP score for each frame. Here, the calculated score may be the above-described tp-DAP score or the like.

  Then, the special voice detection unit 30101 determines whether or not a special voice has been input using the rating value corresponding to the calculated frame. That is, it is determined whether or not there is a section where the rating value (for example, DAP score) is lower than a predetermined value.

  Next, as shown in FIG. 33, the special speech detection means 30101 determines whether or not a section where the rating value (for example, DAP score) is lower than a predetermined value is within one phoneme (here, phoneme 2). Judging. If the rating value is low in one phoneme, the special speech detection unit 30101 calculates a rating value (for example, DAP score) for the immediately preceding phoneme (phoneme 1) or the immediately following phoneme (phoneme 3), and the rating. If the value is higher than a predetermined value, it is determined that there is a possibility that a phoneme is missing. If the section length is, for example, 3 frames or less, it is determined that such phonemes are missing. FIG. 33 shows that the phoneme 2 is missing. In FIG. 33, the vertical axis represents the rating value, and the rating value may be DAP, tp-DAP, or the like. Further, the predetermined value of the section length is not “3 frames or less” but may be “5 frames or less” or “6 frames or less”.

  Next, the rating means 30103 configures a rating result (for example, “phoneme missing has occurred”) indicating that a phoneme is missing. Then, the output means 21104 outputs the configured evaluation result. Note that the output means 21104 preferably outputs the rating value as described above for normal input speech.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. In this case, according to the present embodiment, highly accurate evaluation can be performed without being affected by individual differences, particularly differences in vocal tract length. Furthermore, since this speech processing apparatus can detect special speech, particularly missing phonemes, an extremely accurate rating result can be obtained.

  In the present embodiment, it is only necessary to detect missing phonemes, and the algorithm for calculating the rating value is not limited. The algorithm for calculating the rating value may be the above-described algorithm (DAP, tp-DAP), or another algorithm not described in this specification.

  In this embodiment, another algorithm may be used as the phoneme loss detection algorithm. For example, in the detection of missing phonemes, a section having a length less than a predetermined length may be essential in detecting the missing section, or the section length may not be considered.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Target voice formant frequency) ", the voice received in the voice receiving step is subjected to a sampling process to obtain a second voice data, a vocal tract length normalization processing step, and a voice process to process the second voice data A program for executing a step.

  In the above program, the audio processing step includes a frame dividing step for dividing the second audio data into frames, and a frame audio data acquiring step for obtaining one or more frame audio data which are audio data for each of the divided frames. And, based on the input voice data for each frame, a special voice detection step for detecting that a special voice has been input, and based on detection results in the teacher data, the input voice data, and the special voice detection step, It is preferable that a rating step for rating the received voice and an output step for outputting a rating result in the rating step are preferable.

In the above program, the special speech detection step detects that the rating value of one phoneme satisfies a predetermined condition, and if the special speech detection step detects that the predetermined condition is satisfied, It is preferable to configure a rating result indicating that there is a lack.
(Embodiment 7)

  The voice processing of the voice processing apparatus in the present embodiment is voice recognition.

  FIG. 34 is a block diagram of the speech processing apparatus according to this embodiment.

  The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a teacher data formant frequency storage unit 104, a first sampling frequency storage unit 105, a sampling unit 106, and an evaluation target person formant frequency acquisition unit 107. The evaluation subject person formant frequency storage unit 108, the vocal tract length normalization processing unit 109, the voice processing unit 3410, and the utterance prompting unit 1109 are provided.

  The voice processing unit 3410 includes voice recognition means 34101 and output means 34102.

  The voice recognition unit 34101 of the voice processing unit 3410 performs voice recognition processing based on the second voice data. The algorithm for speech recognition is not limited. The voice recognition process may be a known algorithm. In the present embodiment, highly accurate speech recognition is possible by performing speech recognition based on the resampled second speech data. The audio processing unit 3410 can be usually realized by an MPU, a memory, or the like. The processing procedure of the audio processing unit 3410 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 34102 outputs a voice recognition result. Here, the output is a concept including display on a display, printing on a printer, sound output, transmission to an external device, accumulation in a recording medium, and the like. The output unit 34102 may or may not include an output device such as a display or a speaker. The output unit 34102 can be realized by output device driver software, or output device driver software and an output device.

  Next, the operation of the speech processing apparatus will be described using the flowchart of FIG. In the flowchart of FIG. 37, the description of the same processing as that of the flowcharts of FIGS. 2 and 12 is omitted.

  (Step S3501) The voice recognition unit 34101 performs voice recognition processing based on the second voice data obtained by resampling in step S1208. Note that the voice recognition unit 34101 obtains a recognition result by matching with the teacher data and recognizing that the sound is close to the teacher data.

  (Step S3502) The output means 34102 outputs the speech recognition result in step S3501. The process returns to step S1206.

  As described above, according to the present embodiment, speech recognition can be performed with high accuracy.

Note that the software that implements the speech processing apparatus according to the present embodiment is the following program. In other words, the program samples the received voice at the first sampling frequency and acquires the first voice data to the computer, and the second sampling frequency “first sampling frequency / (teacher data formant frequency / evaluation”. Subject formant frequency) ”, the voice received in the voice receiving step is subjected to a sampling process to obtain the second voice data, and the voice based on the second voice data It is the program for performing the audio | voice processing step which performs a recognition process.
(Embodiment 8)

  In the present embodiment, a description will be given of a speech processing apparatus that can evaluate the similarity between the comparison target speech and the input speech with high accuracy and high speed. The audio processing apparatus will be described as being a pronunciation rating apparatus that mainly evaluates audio (including singing). Further, in the present embodiment, a description will be given of a speech processing apparatus that can perform pronunciation evaluation according to speaker characteristics of the evaluation target person with higher accuracy than the speech processing apparatus described in the above embodiment. Specifically, in the present embodiment, a speech processing apparatus that is less susceptible to the speaker characteristics of the evaluation target will be described based on a simple vocal tract length normalization method based on the least square error criterion.

  In addition, the speech processing apparatus according to the present embodiment can be used for language learning, imitation practice, karaoke evaluation, and the like.

  FIG. 36 is a block diagram of the speech processing apparatus according to this embodiment.

  The speech processing apparatus includes an input reception unit 101, a teacher data storage unit 102, a speech reception unit 103, a first sampling frequency storage unit 105, a sampling unit 106, a vocal tract length normalization parameter calculation unit 3601, a vocal tract length normalization parameter. A storage unit 3602, a vocal tract length normalization processing unit 3609, and a speech processing unit 110 are provided.

  Note that the voice processing apparatus accepts input from input means such as a keyboard 342 and a mouse 343. The voice processing apparatus accepts voice input from voice input means such as a microphone 345. Further, the sound processing apparatus outputs information to an output device such as the display 344.

  The vocal tract length normalization parameter calculation unit 3601 includes a frequency range designation information storage unit 36011, a long-time cepstrum average vector storage unit 36012, a second cepstrum vector series calculation unit 36013, a cepstrum conversion unit 36014, a cepstrum conversion parameter calculation unit 36015, a final Cepstrum conversion parameter acquisition means 36016 and vocal tract length normalization parameter calculation means 36017 are provided.

  The vocal tract length normalization parameter calculation unit 3601 calculates a vocal tract length normalization parameter based on the teacher data in the teacher data storage unit 102 and the voice received by the voice reception unit 103. The vocal tract length normalization parameter is a sampling frequency conversion rate for absorbing speaker characteristics (difference in vocal tract length) of the evaluation target person. In the present embodiment, first, a cepstrum conversion (cepstrum warping) parameter is calculated. Since this cepstrum conversion parameter does not directly represent conversion of the vocal tract length, the vocal tract length normalization parameter calculation unit 3601 uses the approximate conversion formula described later to calculate the final vocal tract length normalization parameter (sampling frequency). Conversion rate). The vocal tract length normalization parameter calculation unit 3601 can usually be realized by an MPU, a memory, or the like. The processing procedure of the vocal tract length normalization parameter calculation unit is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The frequency range designation information storage unit 36011 stores frequency range designation information (W) that is information for designating a frequency range. The frequency range designation information (W) has a frequency within a range in which frequency warping by bilinear transformation such as a linear allpass function can be approximated by linear frequency expansion and contraction when calculating an optimal cepstrum transformation parameter (α) described later. This is information for limiting the range. Such a frequency range is preferably about 0 to 1/3 to 1/2 of Nyquist rate. However, since the bilinear transformation is performed not in the spectral domain but in the cepstrum domain, the frequency range designation information (W) is preferably, for example, matrix information described below. The frequency range designation information is calculated as follows, for example, by frequency range designation information calculation means (not shown). Sampling frequency is F _s (Hz), maximum frequency in the specified frequency range is F _max (Hz), N is a sufficiently large natural number (512, 1024, etc.), “N _m = N × F _max / F _s ” Then, the frequency range designation information calculation means calculates the (i, j) component of the frequency range designation matrix W for the cepstrum vector according to the following formula 5. Specifically, the frequency range designation information calculation means is configured to store a sampling frequency (F _s ), a maximum frequency (F _max ), and a predetermined sufficiently large natural number (N) stored in a computer recording medium (not shown). Is read. Then, the frequency range designation information calculation means reads information “N _m = N × F _max / F _s ” held by itself, substitutes the read F _s , F _max , and N into the arithmetic expression, N _m is calculated. Then, the frequency range designation information calculation means reads out the stored information of the following Expression 5, and increments i and j by 1 in order from 0, while performing loop processing (becomes a double loop processing), {W} _{i, j} is calculated. Then, the frequency range designation information calculation means stores all of the calculated {W} _{i, j} in the frequency range designation information storage means 36011 at least temporarily. In Equation 5, “k” is a frequency index, and the range of “k” is “k = 0, 1, 2,..., N / 2”. “N” is a discrete-time index, and the range of “n” is “n = ...− 2, −1, 0, 1, 2,.

  In addition, the data structure of frequency range designation | designated information is not ask | required. The frequency range designation information storage unit 36011 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

The long-time cepstrum average vector storage unit 36012 stores a long-time cepstrum average vector (μ). The long-term cepstrum average vector (μ) is the time average of the first cepstrum vector sequence calculated from the data constituting the teacher data by the short interval analysis. The first cepstrum vector series (x _t (t = 1, 2,..., T ₀ )) is usually obtained by performing a short-term analysis on a single phoneme (eg, / u /) constituting teacher data. Is done. Then, the long-term cepstrum average vector (μ) is calculated from the vector (x _t ) according to the following Equation 6.

Note that the cepstrum vector has M + 1 dimensions including the 0th order coefficient, and the vector (x _t ) and the vector (μ) are expressed by Expression 7 and Expression 8, respectively.

In Equations 7 and 8, (...) ^T represents transposition of a matrix or a vector.

The first cepstrum vector sequence (x _t ) is calculated by the first cepstrum vector sequence calculation means (not shown) from the data (single phoneme (for example, / u /)) constituting the teacher data by short section analysis. You may do it.

Further, a long-time cepstrum average vector acquisition unit (not shown) may calculate the time average of the first cepstrum vector series (x _t ) using Equation 6 to acquire the long-time cepstrum average vector (μ). The long-term cepstrum average vector storage means 36012 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

The second cepstrum vector sequence calculation means 36013 uses the second cepstrum vector sequence (C _t ) from the speech received by the speech accepting unit 103 (usually a single phoneme (eg, / u /)) by short interval analysis. ) Is calculated. The second cepstrum vector sequence calculation unit 36013 may calculate the second cepstrum vector sequence (C _t ) from the first audio data obtained by sampling by the sampling unit 106, and in this case, the audio reception unit 103 also calculates. It can be said that the second cepstrum vector series (C _t ) was calculated from the received speech by short interval analysis. Note that the process of analyzing a phoneme for a short interval and calculating a cepstrum vector series is a process according to a known technique, and thus the description thereof is omitted. The second cepstrum vector series calculation unit 36013 can usually be realized by an MPU, a memory, or the like. The processing procedure of the second cepstrum vector series calculation unit 36013 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). Note that the phoneme (for example, / u /) that is the voice processed by the second cepstrum vector series calculation unit 36013 and received by the voice receiving unit 103 is the same phoneme (for example, / u /). The second cepstrum vector series (C _t ) is expressed by Equation 9 as an M + 1 dimensional vector that also includes the zeroth order coefficient.

Further, the second cepstrum vector sequence calculation unit 36013 performs the following equation 10 on the calculated second cepstrum vector sequence (C _t ) for the processing of the cepstrum conversion parameter calculation unit 36015 described later. It is preferable to obtain a vector (C _t ⁻ ) (t = 1, 2,..., T).

The cepstrum transforming unit 36014 linearly transforms the second cepstrum vector series (C _t ) using a matrix (F (α)) having cepstrum transform parameters (α) as elements, and a frequency-warped third cepstrum. A vector series (O _t ) is calculated. Specifically, first, the cepstrum conversion unit 38014 sets an initial value (α ^to ) of the cepstrum conversion parameter (α). (Α ^˜ ) is preferably an approximate value of the optimal value of α, and is usually set to α = 0. For example, when it can be predicted that the optimal value is “α> 0”, a small positive value is used. α may be used. Note that the initial values (α ^to ) are stored in advance in a storage medium, a memory, or the like by the cepstrum conversion means 38014. Next, the cepstrum conversion means 38014 calculates a third warped cepstrum vector sequence (O _t ) according to the following Equation 11 using the given cepstrum conversion parameter (α) as a parameter. The cepstrum conversion means 38014 first calculates (O _t (α ^˜ )).

  The cepstrum conversion unit 36014 can be usually realized by an MPU, a memory, or the like. The processing procedure of the cepstrum conversion means stage 36014 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The cepstrum conversion parameter calculation unit 36015 calculates a cepstrum conversion parameter (α) based on the long-time cepstrum average vector (μ) and the third cepstrum vector series (O _t ). More preferably, the cepstrum conversion parameter calculation unit 36015 calculates the cepstrum conversion parameter based on the long-time cepstrum average vector and the third cepstrum vector series (O _t ) in the frequency range indicated by the frequency range designation information (W). To do.

Specifically, first, the cepstrum conversion parameter calculation unit 36015 calculates a vector (u _t (α)) by the following Expression 12. Then, the cepstrum conversion parameter calculation unit 36015 calculates the optimum value of α (α ^* ) by the following Expression 13. Note that the optimum value of α (α ^* ) is the optimum value in the current iteration step.

  The cepstrum conversion parameter calculation unit 36015 can be usually realized by an MPU, a memory, or the like. The processing procedure of the cepstrum conversion parameter calculation means 36015 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The final cepstrum conversion parameter acquisition unit 36016 repeats the processing in the cepstrum conversion unit 36014 and the processing in the cepstrum conversion parameter calculation unit 36015 based on a predetermined rule, and obtains a final optimum cepstrum conversion parameter. Here, the predetermined rule is, for example, that the process is repeatedly performed a predetermined number of repetitions. Also, the predetermined rule is, for example, that the change in the optimum value of α (α ^* ) is smaller than the predetermined value (the difference between the previous (α ^* ) and the current (α ^* ) is less than a threshold value, etc.) It is. Further, the predetermined rule may be another rule. The final cepstrum conversion parameter acquisition unit 36016 can be usually realized by an MPU, a memory, or the like. The processing procedure of the final cepstrum conversion parameter acquisition unit 36016 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The vocal tract length normalization parameter calculation unit 36017 calculates the vocal tract length normalization parameter (γ) based on the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit 36016. More specifically, the vocal tract length normalization parameter calculation unit 36017 first calculates the linear frequency expansion / contraction ratio (ρ) from the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit 36016, for example, according to the following Expression 14. calculate.

Next, the vocal tract length normalization parameter calculation unit 36017 calculates the vocal tract length normalization parameter (γ) from the linear frequency expansion / contraction ratio (ρ), for example, according to the following Expression 15.

  The vocal tract length normalization parameter calculation means 36017 can be usually realized by an MPU, a memory, or the like. The processing procedure of the vocal tract length normalization parameter calculation means 36017 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The vocal tract length normalization parameter storage unit 3602 stores a vocal tract length normalization parameter (γ), which is information regarding the conversion rate of the sampling frequency. The vocal tract length normalization parameter (γ) is a vocal tract length normalization parameter acquired by the vocal tract length normalization parameter calculation unit 3601. The vocal tract length normalization parameter storage unit 3602 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium. Here, the vocal tract length normalization parameter calculation unit 3601 also performs a process of accumulating the calculated vocal tract length normalization parameter (γ) in the vocal tract length normalization parameter storage unit 3602.

  The vocal tract length normalization processing unit 3609 performs sampling processing on the voice received by the voice reception unit 103 at the second sampling frequency calculated using the vocal tract length normalization parameter (γ) and the first sampling frequency as parameters. To obtain second audio data. “For voice received by the voice receiving unit 103” means that the first voice data sampled and acquired by the sampling unit 106 is subjected to resampling processing at the second sampling frequency to obtain second voice data. In addition, the sampling unit 106 obtains the second audio data by sampling the audio received directly by the audio receiving unit 103 at the second sampling frequency without acquiring the first audio data. Including. The vocal tract length normalization processing unit 3609 can usually be realized by an MPU, a memory, or the like. The processing procedure of the vocal tract length normalization processing unit 3609 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the voice processing apparatus will be described. First, the process of calculating the vocal tract length normalization parameter in the speech processing apparatus will be described with reference to the flowchart of FIG. Note that the vocal tract length normalization parameter calculation process does not necessarily have to be performed by the voice processing device.

  (Step S3701) The speech processing apparatus performs initialization processing. The initialization process is, for example, a process for prompting the user (evaluation subject) to say “/ u /” (for example, a process of displaying “Please pronounce“ U ”” on the display). ) And the frequency range designation information stored in the frequency range designation information storage unit 36011 and the long time cepstrum average vector stored in the long time cepstrum average vector storage unit 36012.

  (Step S3702) It is judged whether the audio | voice reception part 103 received the audio | voice from an evaluation object person. If a voice is accepted, the process goes to step S3703, and if no voice is accepted, the process returns to step S3702.

  (Step S3703) The sampling unit 106 samples the sound received in step S3702, obtains first sound data, and stores at least temporarily in the memory. Note that the process of sampling audio is a known technique.

(Step S3704) The second cepstrum vector series calculation means 36013 acquires the first voice data obtained in step S3703, and calculates the second cepstrum vector series (C _t ) from the first voice data by short section analysis. The second cepstrum vector series (C _t ) is at least temporarily stored in the memory.

(Step S3705) The second cepstrum vector sequence calculation unit 36013 obtains the second cepstral vector sequences calculated in step S 3704 _{(C t),} from the second cepstral vector series _{(C t),} the vector _{(C t} ^- ) (T = 1, 2,..., T) are acquired, and the vector (C _t ⁻ ) is at least temporarily stored in the memory.

(Step S3706) cepstrum conversion unit 36014 reads the initial value of the cepstrum transformation parameters stored in advance (alpha ^~), set the variable (alpha) and (alpha ^~).

(Step S3707) cepstrum conversion unit 36014 reads the second cepstral vector series _{(C t),} the second cepstral vector series _{(C t),} using a matrix for cepstrum conversion parameter a (alpha) and elements A third cepstrum vector sequence (O _t ) subjected to linear transformation and frequency warped is calculated, and at least temporarily stored in the memory.

(Step S3708) The cepstrum transformation parameter calculation unit 36015 reads out the third cepstrum vector sequence (O _t ) calculated in step S3707 and the vector (C _t ⁻ ) calculated in step S3705, and the third cepstrum vector sequence. From (O _t ) and the vector (C _t ⁻ ), a vector (u _t (α)) is calculated (see Expression 12). Needless to say, when the vector (u _t (α)) is calculated, the stored information of Equation 12 is read out.

(Step S3709) The cepstrum conversion parameter calculation means 36015 reads information of Equation 13 stored in advance, and in the equation of Equation 13, a vector (u _t (α)), a long-time cepstrum average vector (μ), a vector _{(C t} ^-), it gives information ^{(W T,} etc. including also) in the frequency range specification information (W), and calculates the formula 13, and calculates the optimum value of alpha the (alpha ^*). This optimal value (α ^* ) of α is the optimal value in this loop. Then, the cepstrum conversion parameter calculation means 36015 temporarily stores the optimal value (α ^* ) of α at least in the memory.

(Step S3710) The final cepstrum conversion parameter acquisition unit 36016 determines whether or not a predetermined rule (rule information is stored in advance) is met. If it matches the rule, the process goes to step S3711, and if it does not match the rule, the process goes to step S3713. Note that, as described above, the rule is, for example, the loop processing (the optimum value of α (α ^* ) as many times as a predetermined number of repetitions (information on this number is stored in advance in a memory or the like) ^. ) And the process of substituting α ^* into α is repeatedly performed.

(Step S3711) The vocal tract length normalization parameter calculation unit 36017 acquires the cepstrum conversion parameter (final α ^* ) obtained by the final cepstrum conversion parameter acquisition unit 36016, gives the parameter to the calculation formula of Formula 14, and linearly The frequency expansion / contraction ratio (ρ) is calculated. Note that the information of the arithmetic expression of Expression 14 is stored in advance by the vocal tract length normalization parameter calculation means 36017. The vocal tract length normalization parameter calculation means 36017 temporarily stores the calculated linear frequency expansion / contraction ratio (ρ) in at least a memory.

  (Step S3712) The vocal tract length normalization parameter calculation means 36017 reads the linear frequency expansion / contraction ratio (ρ), calculates the vocal tract length normalization parameter (γ) from the linear frequency expansion / contraction ratio (ρ), and at least the memory Temporarily store in. For example, the vocal tract length normalization parameter calculation unit 36017 reads information of Formula 15 stored in advance, substitutes the linear frequency expansion / contraction ratio (ρ), executes the calculation formula of Formula 15, and normalizes the vocal tract length. The parameter (γ) is obtained.

(Step S3713) The final cepstrum conversion parameter acquisition unit 36016 substitutes (α ^* ) calculated in step S3709 for α. Then, the process returns to step S3707.

  The operation of the speech processing apparatus (speech rating process) will be described with reference to the flowcharts of FIGS. In the flowchart of FIG. 38, only the content of the vocal tract length normalization process in step S204 is different from the operation of the speech processing apparatus described in FIG.

  Next, the contents of the vocal tract length normalization process will be described using the flowchart of FIG.

  (Step S3901) The vocal tract length normalization processing unit 3609 reads the first sampling frequency from the first sampling frequency storage unit 105.

  (Step S3902) The vocal tract length normalization processing unit 3609 reads the vocal tract length normalization parameter (γ) from the vocal tract length normalization parameter storage unit 3602.

(Step S3903) The vocal tract length normalization processing unit 3609 reads out information on an arithmetic expression for calculating a predetermined second sampling frequency (F ₂ ), and the vocal tract length normalization parameter ( γ) and the first sampling frequency (F ₁ ) are substituted as parameters, an arithmetic expression is executed, and the second sampling frequency (F ₂ ) is calculated. An arithmetic expression for calculating the predetermined second sampling frequency (F ₂ ) is, for example, (F ₂ = F ₁ × γ).

  (Step S3904) The vocal tract length normalization processing unit 3609 performs resampling processing on the first audio data at the second sampling frequency to obtain second audio data.

  Note that the processing is ended by powering off or interruption for aborting the processing in the flowchart in FIG.

Hereinafter, the concept of the vocal tract length normalization parameter calculation process in the speech processing apparatus according to the present embodiment will be described using the conceptual diagram of FIG. The vocal tract length normalization parameter is calculated by converting the average cepstrum vector μ of the phoneme (phoneme with teacher data) of the speaker (reference speaker) used for system design and the user utterance of the same phoneme. It is determined so that the square error with the cepstrum vector O _t is minimized. However, since the obtained parameter (α?) Is a cepstrum transformation (cepstrum warping) parameter and does not directly represent the transformation of the vocal tract length as it is, the final vocal tract is obtained using the approximate transformation formula (1 / ρ). The long normalization parameter (sampling frequency conversion rate) γ is calculated. In addition, the mathematical formula for calculating the vocal tract length normalization parameter is performed by the mathematical formula 5 to the mathematical formula 15 described above.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. Further, in this case, according to the present embodiment, it is possible to make a highly accurate evaluation that is not affected by individual differences, particularly vocal tract length differences.

  Note that, according to the present embodiment, the voice processing device performs the process of calculating the vocal tract length normalization parameter and the vocal tract length normalization process. However, a configuration in which different devices perform the process of calculating the vocal tract length normalization parameter and the process of normalizing the vocal tract length may be used. In such a case, the speech processing apparatus includes a speech reception unit that receives speech, a first sampling frequency storage unit that stores a first sampling frequency, and a vocal tract length normalization parameter that is information related to a conversion rate of the sampling frequency. A voice sampling length storage parameter storing unit, a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters, and the voice received by the voice receiving unit The voice processing apparatus includes a vocal tract length normalization processing unit that performs sampling processing to obtain second voice data, and a voice processing unit that processes the second voice data. Further, the apparatus for calculating the vocal tract length normalization parameter stores a long-time cepstrum average vector that is a time average of the first cepstrum vector sequence calculated by the short interval analysis from the data constituting the teacher data. Long-term cepstrum average vector storage means, second cepstrum vector series calculation means for calculating a second cepstrum vector series from received speech by short interval analysis, and the second cepstrum vector series, cepstrum conversion parameters as elements And a cepstrum conversion means for calculating a frequency-warped third cepstrum vector sequence and a cepstrum conversion parameter based on the long-time cepstrum average vector and the third cepstrum vector sequence. Cepstra to calculate A conversion parameter calculation means, a final cepstrum conversion parameter acquisition means for obtaining a cepstrum conversion parameter by repeating the process in the cepstrum conversion means and the process in the cepstrum conversion parameter calculation means based on a predetermined rule, and the final The apparatus includes a vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained by the cepstrum conversion parameter acquisition unit.

  Further, according to the present embodiment, the frequency range is limited when calculating the cepstrum conversion parameter, but such processing is not essential. The same applies to the ninth and tenth embodiments.

  Moreover, according to this Embodiment, the audio | voice processing part performed pronunciation evaluation based on DAP. However, pronunciation evaluation may be performed by other algorithms. Other algorithms include, for example, the tp-DAP described in the second embodiment, the similarity evaluation considering the silent section described in the third embodiment, and the phoneme insertion described in the fourth embodiment. For example, the similarity evaluation in consideration of the phonetic substitution described in the fifth embodiment, the similarity evaluation in consideration of the phoneme omission described in the sixth embodiment, and the like. This also applies to the ninth and tenth embodiments.

  Further, according to the present embodiment, the voice processing unit in the voice processing apparatus mainly performs pronunciation evaluation, but the voice processing unit may perform voice recognition processing based on the second voice data. This also applies to the ninth and tenth embodiments.

  In the present embodiment, the following processing performed by the audio processing device may be performed by a single DSP (digital signal processor). That is, the DSP stores a first sampling frequency storage unit that stores the first sampling frequency and a vocal tract length normalization parameter storage that stores a vocal tract length normalization parameter that is information related to the conversion rate of the sampling frequency. And a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters, sampling the voice received by the voice receiving unit, A digital signal processor including a vocal tract length normalization processing unit. This also applies to the ninth and tenth embodiments.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program has a voice receiving step for receiving voice, a stored vocal tract length normalization parameter, and a second sampling frequency calculated using the stored first sampling frequency as parameters. A program for performing a vocal tract length normalization processing step for obtaining a second voice data by performing a sampling process on the voice received in the reception step and a voice processing step for processing the second voice data. .

Further, in the above program, one or more teacher data, which is data related to a target voice to be compared and is data for each one or more phonemes, is stored, and is based on the teacher data and the voice received in the voice receiving step. And further executing a vocal tract length normalization parameter calculating step for calculating the vocal tract length normalization parameter, wherein the stored vocal tract length normalization parameter is the vocal tract length normalization parameter calculation step. It is preferable that the vocal tract length normalization parameter calculated in (1) is used.
(Embodiment 9)

  In the present embodiment, a description will be given of a speech processing apparatus that can evaluate the similarity between the comparison target speech and the input speech with high accuracy and high speed. The audio processing apparatus will be described as being a pronunciation rating apparatus that mainly evaluates audio (including singing). Further, in the present embodiment, a description will be given of a speech processing apparatus that can perform pronunciation evaluation according to speaker characteristics of the evaluation target person with higher accuracy than the speech processing apparatus described in the above embodiment. Specifically, in the present embodiment, a speech processing apparatus that is less susceptible to the speaker characteristics of the evaluation subject will be described based on a simple vocal tract length normalization method based on the appearance probability maximization criterion. Note that the speech processing apparatus according to the present embodiment differs from the speech processing apparatus according to the eighth embodiment in the calculation algorithm of the vocal tract length normalization parameter.

  In addition, the speech processing apparatus according to the present embodiment can be used for language learning, imitation practice, karaoke evaluation, and the like.

  FIG. 41 is a block diagram of the speech processing apparatus according to this embodiment.

  This speech processing device is different from the speech processing device in FIG. 36 in the vocal tract length normalization parameter calculation unit 4101.

  Note that the voice processing apparatus accepts input from input means such as a keyboard 342 and a mouse 343. The voice processing apparatus accepts voice input from voice input means such as a microphone 345. Further, the sound processing apparatus outputs information to an output device such as the display 344.

  The vocal tract length normalization parameter calculation unit 4101 includes a frequency range designation information storage unit 41011, a learning acoustic data storage unit 41012, a second cepstrum vector series calculation unit 36013, a cepstrum conversion unit 36014, an occupancy degree calculation unit 41013, and a cepstrum conversion parameter calculation. Means 41015, final cepstrum conversion parameter acquisition means 36016, and vocal tract length normalization parameter calculation means 36017 are provided. Here, the final cepstrum conversion parameter acquisition unit 36016 repeats not only the processing in the cepstrum conversion unit 36014 and the processing in the cepstrum conversion parameter calculation unit 41015 but also the processing in the occupation frequency calculation unit 41013 based on a predetermined rule. To get the final optimal cepstrum transformation parameters.

The frequency range designation information storage unit 41011 stores frequency range designation information (W) that is information for designating a frequency range. The frequency range designation information (W) has a frequency within a range in which frequency warping by bilinear transformation such as a linear allpass function can be approximated by linear frequency expansion and contraction when calculating an optimal cepstrum transformation parameter (α) described later. This is information for limiting the range. Such a frequency range is preferably about 0 to 1/3 to 1/2 of Nyquist rate. However, since the bilinear transformation is performed not in the spectral domain but in the cepstrum domain, the frequency range designation information (W) is preferably, for example, matrix information described below. The frequency range designation information (W) is calculated as follows, for example, by a frequency range designation information calculation unit (not shown). The sampling frequency is F _s (Hz), the maximum frequency in the specified frequency range is F _max (Hz), and “ω _max = 2πF _max / F _s ” is set, and the frequency range specification matrix W for the cepstrum vector (i , J) component is calculated by the frequency range designation information calculating means according to the following equation (16). Specifically, the frequency range designation information calculation means reads a sampling frequency (F _s ) and a maximum frequency (F _max ) stored in a computer recording medium (not shown). Then, the frequency range designation information calculation means reads information “ω _max = 2πF _max / F _s ” held by itself, and calculates ω _max . Then, the frequency range designation information calculating means reads out the stored information of the following Expression 16, and increments i and j by 1 in order from 0, while performing loop processing (becomes a double loop processing), {W} _{i, j} is calculated. Then, the frequency range designation information calculation means stores all of the calculated {W} _{i, j} in the frequency range designation information storage means 41011 at least temporarily.

  In addition, the data structure of frequency range designation | designated information is not ask | required. The frequency range designation information storage unit 41011 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

The learning acoustic data storage means 41012 stores learning acoustic data. The learning acoustic data is generated as follows. That is, a learning acoustic data generation unit (not shown) obtains a cepstrum vector sequence of the speaker's voice from the data constituting the teacher data (for example, data in the teacher data storage unit 102) by short-term analysis, and uses this to obtain 2 The above reference speaker phoneme HMM is acquired and stored at least temporarily in the memory. Then, the learning acoustic data generating means connects two or more reference speaker phoneme HMMs according to the specified utterance content for normalizing the vocal tract length (for example, / Aiueo /) to generate a connected HMM. Note that the cepstrum vector has M + 1 dimensions including the zeroth order coefficient. Further, if the mean vector and covariance matrix of the mth Gaussian distribution component in the jth state of the connected HMM are μ _{j, m} and Σ _{j, m} , μ _{j, m} is expressed by the following Expression 17. .

In addition, (...) ^T indicates transposition of a matrix or a vector.

  Then, the learning acoustic data generation means (not shown) accumulates the generated learning acoustic data in the learning acoustic data storage means 41012.

  The learning acoustic data is preferably a connected HMM, but may be data based on other models such as a single Gaussian distribution model, a probability model (GMM: Gaussian mixture model), and a statistical model. The learning acoustic data storage means 41012 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

The occupation frequency calculation means 41013 calculates the occupation frequency (γ _t (j, m)), writes it in the memory, and holds it at least temporarily. The occupation frequency (γ _t (j, m)) is a posterior probability that the t-th frame is generated from the m-th Gaussian component in the j-th state. The t-th frame is, for example, the t-th frame obtained from the received voice by the voice receiving unit 103 receiving the utterance of the specified utterance content (for example, / Aiueo /) for normalizing the vocal tract length. It is a frame. The occupancy degree calculation means 41013 uses the third cepstrum vector sequence (O _t ) calculated by the cepstrum conversion means 36014 and the learning acoustic data of the learning acoustic data storage means 41012 to occupy degrees (γ _t (j, m)). Is calculated. More preferably, the occupancy frequency calculation means 41013 uses the third cepstrum vector sequence (O _t ), the learning acoustic data, and the frequency range designation information (W) of the frequency range designation information storage means 41011 to use the occupancy frequency (γ _t (J, m)) is calculated. The occupation frequency calculation means 41013 calculates the occupation frequency (γ _t (j, m)) using a forward / backward algorithm. Since the forward / backward algorithm is a known technique, a detailed description thereof will be omitted.

More specifically, the occupation frequency calculation means 41013 calculates the occupation frequency (γ _t (j, m)) by the following processing. Here, let Λ be a concatenated HMM in which learning phoneme HMMs learned from speech of teacher data are concatenated according to the utterance content (for example, / aiueo /) designated for vocal tract length normalization parameter estimation. The occupation frequency (γ _t (j, m)) used for the cepstrum transformation parameter estimation process is the cepstrum vector sequence “Wo ₁ (α), Wo ₂ (α),..., Wo _T (α) ”and Λ are defined as the posterior probabilities that arise from the mth Gaussian component of the jth state of Λ. The frequency (γ _t (j, m)) is defined by Equation 18.

Further, the occupation frequency (γ _t (j, m)) in Expression 18 is calculated as Expression 19 using the forward likelihood A _t (j) and the backward likelihood B _t (j). That is, the occupation frequency calculation means 41013 holds the information of the equation represented by Equation 19, and calculates the occupation frequency (γ _t (j, m)) using the given cepstrum vector sequence and Λ.

Note that the occupation frequency calculation means 41013 converts At (j) and Bt (j) of Equation 19 into a forward-backward algorithm known from the cepstrum vector series “Wo ₁ (α)... W o _T (α)” and Λ. Calculate using.

V _{j, m} is a weighting factor for the mth Gaussian component in the jth state. Moreover, occupied frequency computing unit 41013, the output distribution _{b j,} m the (Wo _t (α)), is calculated using Equation 20. Occupied frequency computing unit 41013 is prestores information of formula 20, reads the information, performs operations to obtain power distribution _{b j,} m the (Wo _t (α)).

  In Equation 20, M represents a cepstrum order, and | A | represents a determinant of the matrix A.

  The occupancy frequency calculation means 41013 can usually be realized by an MPU, a memory, or the like. The processing procedure of the occupation frequency calculation means 41013 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The cepstrum conversion parameter calculation unit 41015 reads out the learning acoustic data in the learning acoustic data storage unit 41012 and reads out the third cepstrum vector sequence (O _t ) temporarily stored in the memory, Based on the third cepstrum vector series (O _t ), a cepstrum transformation parameter (α) is calculated. More preferably, the cepstrum conversion parameter calculation means 41015 is the learning acoustic data, the third cepstrum vector series (O _t ), and the occupancy frequency (γ _t (j, m) in the frequency range indicated by the frequency range designation information (W). ) To calculate a cepstrum conversion parameter. The third cepstrum vector series (O _t ) is data calculated by the cepstrum conversion unit 36014.

Specifically, first, the cepstrum conversion parameter calculation unit 41015 calculates a vector (u _t (α)) by the above-described Expression 12. Note that the cepstrum conversion parameter calculation means 41015 stores information indicating the above formula 12, reads the information of the formula 12, and stores the third cepstrum vector series (O _t ), cepstrum conversion parameter ( α) and vector (C _t ⁻ ) information are given, and the mathematical formula is calculated. Then, the cepstrum conversion parameter calculation means 41015 calculates the optimum value (α ^* ) of α by the following Equation 21. Note that the optimum value of α (α ^* ) is the optimum value in the processing in the current loop in the iteration step (loop processing). Further, the cepstrum conversion parameter calculation means 41015 stores information indicating the following formula 21 in advance.

  The cepstrum conversion parameter calculation means 41015 can usually be realized by an MPU, a memory, or the like. The processing procedure of the cepstrum conversion parameter calculation means 41015 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the voice processing apparatus will be described. First, the process of calculating the vocal tract length normalization parameter in the speech processing apparatus will be described with reference to the flowchart of FIG. In the flowchart of FIG. 42, only steps that are different from the flowchart of FIG. 37 will be described. Note that the vocal tract length normalization parameter calculation process does not necessarily have to be performed by the voice processing device. The initialization process in FIG. 42 includes, for example, a process for prompting the user (evaluation subject) to say “/ aiueo”, frequency range designation information in the frequency range designation information storage unit 41011, and This is a process of reading the learning sound data in the learning sound data storage means 41012.

(Step S4201) The occupation frequency calculation means 41013 uses the third cepstrum vector sequence (O _t ) obtained in Step S3707, the learning acoustic data of the learning acoustic data storage means 41012, and the frequency range designation information (W). The occupancy frequency (γ _t (j, m)) is calculated and temporarily stored in the memory.

(Step S4202) The cepstrum transformation parameter calculation means 41015 converts the learning acoustic data, vector (u _t (α)), and occupancy frequency (γ _t (j, m)) in the frequency range indicated by the frequency range designation information (W). Based on this, the optimum cepstrum transformation parameter (α ^* ) in this loop is calculated. The cepstrum conversion parameter calculation means 41015, for example, reads the stored information of Equation 21 and also learns the learning acoustic data, the vector (u _t (α)) and the occupation frequency (γ _t (j, m)) on the memory. Is substituted into Equation 21, and a cepstrum conversion parameter (α ^* ) is calculated.

  Next, the operation (voice evaluation process) of the voice processing apparatus will be described. The operation of this speech processing apparatus is the same as the operation in the flowcharts of FIGS.

Hereinafter, the concept of the vocal tract length normalization parameter calculation process in the speech processing apparatus according to the present embodiment will be described using the conceptual diagram of FIG. First, phoneme HMMs (phoneme HMMs possessed by teacher data) are concatenated from the system design speech database according to the specified utterance content (for example, / Aiueo /) to form a concatenated HMM (reference speaker HMM in FIG. 43). Then, the appearance probability with respect to Λ (Λ is a concatenated HMM concatenated according to the user speech) of the concatenated HMM and the cepstrum vector O _t converted from the user speech of the same phoneme sequence is maximized (see FIG. 43 parameter). However, since the obtained parameter (α?) Is a cepstrum transformation (cepstrum warping) parameter and does not directly represent the transformation of the vocal tract length as it is, the final vocal tract is obtained using the approximate transformation formula (1 / ρ). The long normalization parameter (sampling frequency conversion rate) γ is calculated. Note that the mathematical formulas for calculating the vocal tract length normalization parameters are calculated by the above mathematical formulas 16 to 21, 9 to 12, 12, 14 and 15.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. Further, in this case, according to the present embodiment, it is possible to make a highly accurate evaluation that is not affected by individual differences, particularly vocal tract length differences.

  Note that, according to the present embodiment, the voice processing device performs the process of calculating the vocal tract length normalization parameter and the vocal tract length normalization process. However, a configuration in which different devices perform the process of calculating the vocal tract length normalization parameter and the process of normalizing the vocal tract length may be used. In such a case, the speech processing apparatus includes a speech reception unit that receives speech, a first sampling frequency storage unit that stores a first sampling frequency, and a vocal tract length normalization parameter that is information related to a conversion rate of the sampling frequency. A voice sampling length storage parameter storing unit, a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters, and the voice received by the voice receiving unit The voice processing apparatus includes a vocal tract length normalization processing unit that performs sampling processing to obtain second voice data, and a voice processing unit that processes the second voice data. Further, the apparatus for calculating the vocal tract length normalization parameter obtains a cepstrum vector sequence of the speaker's speech from the data constituting the teacher data by short interval analysis, and uses it to obtain two or more reference speaker phoneme HMMs. Acquiring and storing learning acoustic data which is a connected HMM obtained by connecting the two or more reference speaker phoneme HMMs according to the specified utterance content for normalizing the vocal tract length (for example, / Aiueo /) Learning acoustic data storage means; second cepstrum vector series calculation means for calculating a second cepstrum vector series from the received speech by short interval analysis; and the second cepstrum vector series, with cepstrum conversion parameters as elements. A cepstrum conversion means for performing linear conversion using a matrix and calculating a frequency-warped third cepstrum vector sequence; An occupancy degree calculating means for calculating an occupancy degree that is a posterior probability of the received voice according to the utterance content, and calculating a cepstrum conversion parameter based on the learning acoustic data, the third cepstrum vector series, and the occupancy degree. Based on the cepstrum conversion parameter calculation means and a predetermined rule, the process in the cepstrum conversion means, the process in the occupancy calculation means, and the process in the cepstrum conversion parameter calculation means are repeated to obtain a cepstrum conversion parameter. An apparatus comprising: a final cepstrum conversion parameter acquisition unit; and a vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit. That.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program has a voice receiving step for receiving voice, a stored vocal tract length normalization parameter, and a second sampling frequency calculated using the stored first sampling frequency as parameters. A program for performing a vocal tract length normalization processing step for obtaining a second voice data by performing a sampling process on the voice received in the reception step and a voice processing step for processing the second voice data. .

  Further, in the above program, one or more teacher data, which is data related to a target voice to be compared and is data for each one or more phonemes, is stored, and is based on the teacher data and the voice received in the voice receiving step. And further executing a vocal tract length normalization parameter calculating step for calculating the vocal tract length normalization parameter, wherein the stored vocal tract length normalization parameter is the vocal tract length normalization parameter calculation step. It is preferable that the vocal tract length normalization parameter calculated in (1) is used.

  Further, in the above program, learning acoustic data, which is a concatenated HMM in which phoneme HMMs are concatenated according to designated utterance contents, is stored, and the vocal tract length normalization parameter calculation step is performed based on the speech received in the speech reception step. A second cepstrum vector sequence calculating step for calculating a second cepstrum vector sequence by short interval analysis, and linearly transforming the second cepstrum vector sequence using a matrix having cepstrum transformation parameters as elements, and frequency warping is performed. A cepstrum conversion step for calculating a third cepstrum vector sequence, an occupancy degree calculating step for calculating an occupancy degree which is a posterior probability of speech received according to the designated utterance content, the learning acoustic data and the third cepstrum Vector series and the above occupancy frequency Therefore, the cepstrum conversion parameter calculation step for calculating the cepstrum conversion parameter, the processing in the cepstrum conversion step, the processing in the occupancy calculation step, and the processing in the cepstrum conversion parameter calculation step are repeated based on a predetermined rule. A final cepstrum conversion parameter acquisition step for obtaining a cepstrum conversion parameter, and a vocal tract length normalization parameter for calculating the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained in the final cepstrum conversion parameter acquisition step. It is preferable to include a calculation step.

  In the above program, in the vocal tract length normalization parameter calculation step, a linear frequency expansion / contraction ratio is calculated from the cepstrum conversion parameter obtained in the final cepstrum conversion parameter acquisition step, and the vocal tract length normalization is calculated from the linear frequency expansion / contraction ratio. It is preferable to calculate the optimization parameter.

Further, in the above program, frequency range designation information that is information for designating a frequency range is stored, and in the cepstrum conversion parameter calculation step, learning acoustic data in the frequency range indicated by the frequency range designation information, It is preferable to calculate the cepstrum transformation parameter based on the cepstrum vector series and the occupation frequency.
(Embodiment 10)

  In the present embodiment, a description will be given of a speech processing apparatus that can evaluate the similarity between the comparison target speech and the input speech with high accuracy and high speed. The audio processing apparatus will be described as being a pronunciation rating apparatus that mainly evaluates audio (including singing). Further, in the present embodiment, a description will be given of a speech processing apparatus that can perform pronunciation evaluation according to speaker characteristics of the evaluation target person with higher accuracy than the speech processing apparatus described in the above embodiment. Specifically, in the present embodiment, a speech processing apparatus that is less susceptible to the speaker characteristics of the evaluation target, based on a simple vocal tract length normalization method based on a least square error criterion using a plurality of phonemes Will be described.

  More specifically, in the present embodiment, a reference speaker average cepstrum vector is calculated for each phoneme from the speech of a reference speaker (system design speaker). Then, an optimal warping parameter is obtained so that the square error between the frequency warped cepstrum vector sequence of the user speech and the above-described reference speaker average cepstrum vector sequence is minimized. At this time, it is preferable that the user voice is uttered according to the specified utterance content, and the reference speaker average cepstrum sequence is a sequence of reference speaker average cepstrum for each phoneme according to the same utterance content. It is. However, since the parameter calculated here is a cepstrum warping parameter and does not directly represent vocal tract length conversion, the final vocal tract length normalization parameter (sampling frequency conversion) is determined using a predetermined approximate conversion formula. Rate).

  Note that the speech processing apparatus in the present embodiment differs from the speech processing apparatuses in the eighth and ninth embodiments in the calculation algorithm of the vocal tract length normalization parameter.

  In addition, the speech processing apparatus according to the present embodiment can be used for language learning, imitation practice, karaoke evaluation, and the like.

  FIG. 44 is a block diagram of the audio processing apparatus according to the present embodiment.

  This speech processing device is different from the speech processing device in FIG. 41 in a vocal tract length normalization parameter calculation unit 4401.

  The vocal tract length normalization parameter calculation unit 4401 includes a frequency range designation information storage unit 41011, a learning acoustic data storage unit 44012, a second cepstrum vector sequence calculation unit 36013, a cepstrum conversion unit 36014, an optimum phoneme sequence acquisition unit 44013, and a cepstrum conversion parameter. Calculation means 44015, final cepstrum conversion parameter acquisition means 36016, and vocal tract length normalization parameter calculation means 36017 are provided. Here, the final cepstrum conversion parameter acquisition unit 36016 performs not only the process in the cepstrum conversion unit 36014 and the process in the cepstrum conversion parameter calculation unit 44015, but also the process in the optimum phoneme sequence acquisition unit 44003 based on a predetermined rule. Repeat to get the final optimal cepstrum transformation parameters.

The learning acoustic data storage unit 44012 stores learning acoustic data. The learning acoustic data here is a phoneme average cepstrum vector sequence in which phoneme average cepstrum vectors are arranged according to the specified utterance content. The learning acoustic data is generated as follows, for example. In other words, a learning acoustic data generation unit (not shown) applies to each phoneme l (l = 1, 2,..., L) from data constituting the teacher data (for example, data in the teacher data storage unit 102). Then, the speech cepstrum vector series of the reference speaker is obtained by short interval analysis, and the time average (μ _l ) is calculated. Note that the cepstrum vector has M + 1 dimensions including the zeroth order coefficient. The time average (μ ₁ ) is expressed by the following formula 22.

In addition, (...) ^T indicates transposition of a matrix or a vector.

  Then, the learning acoustic data generation means (not shown) accumulates the generated learning acoustic data in the learning acoustic data storage means 44012.

  Note that the learning acoustic data storage unit 44012 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

Optimal phoneme sequence acquisition means 44013 is a phoneme sequence (s ^* _t (t = 1, 2,..., T)) that is information of phoneme numbers corresponding to each frame t constituting speech uttered by the user. To get. The algorithm for obtaining the phoneme sequence (s ^* _t ) by the optimum phoneme sequence acquisition unit 44013 is not limited. The optimal phoneme sequence acquisition unit 44013 may accept an input of a phoneme sequence (s ^* _t ) from a user (keyboard or the like), for example. The optimal phoneme sequence acquisition unit 44013 may calculate a phoneme sequence (s ^* _t ) from the user cepstrum vector sequence (C _t ) by auto-segmentation. Since auto segmentation is a well-known technique, description thereof is omitted. Also, the optimal phoneme sequence acquisition unit 44013 acquires a phoneme sequence S _t (t = 1, 2,..., T) that minimizes J in Expression 23 below by dynamic programming (DP matching). and it may acquire the _{S t} as a phoneme sequence ^(s _{* t).} Further, the phoneme sequence may not be information on a sequence of phoneme numbers. The phoneme sequence may be a string of information for identifying a phoneme.

The cepstrum conversion parameter calculation unit 44015 reads out the learning acoustic data of the learning acoustic data storage unit 44012, reads out the third cepstrum vector sequence (O _t ) temporarily stored in the memory, and the optimum phoneme sequence acquisition unit 44013. There reads the obtained phoneme sequences (s * _^t), using the read training acoustic data and the third cepstrum vector sequences (O _t) and the phoneme sequence (s * _^t), to calculate the cepstrum transformation parameter (alpha) . The cepstrum conversion parameter calculation means 44015 is more preferably based on the frequency range designation information (W), the read learning acoustic data, the third cepstrum vector sequence (O _t ), and the phoneme sequence (s ^* _t ). Calculate the parameters.

Specifically, first, the cepstrum conversion parameter calculation unit 36015 calculates a vector (u _t (α)) by the following Expression 24. Then, the cepstrum conversion parameter calculation means 44015 calculates the optimum value (α ^* ) of α by the following formula 25. Note that the optimum value of α (α ^* ) is the optimum value in the current iteration step.

  Note that the cepstrum conversion parameter calculation unit 44015 normally stores information on the above formulas 24 and 25, reads out the information on the formulas, and performs calculation. The cepstrum conversion parameter calculation unit 44015 can be usually realized by an MPU, a memory, or the like. The processing procedure of the cepstrum conversion parameter calculation means 44015 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

Note that the vocal tract length normalization parameter calculation unit 36017 calculates the linear frequency expansion / contraction ratio (ρ) from the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit 36016 in the ninth embodiment according to Equation 14. However, the vocal tract length normalization parameter calculation unit 36017 may calculate the linear frequency expansion / contraction ratio (ρ) from the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit 36016 according to the following Equation 26.

  Next, the operation of the voice processing apparatus will be described. First, processing for calculating a vocal tract length normalization parameter in the speech processing apparatus will be described with reference to the flowchart of FIG. In the flowchart of FIG. 45, only steps that are different from the flowcharts of FIGS. 37 and 42 will be described. Note that the vocal tract length normalization parameter calculation process does not necessarily have to be performed by the voice processing device. The initialization process in FIG. 45 includes, for example, a process for prompting the user (evaluation subject) to say “/ aiueo /”, frequency range designation information in the frequency range designation information storage unit 41011, and This is a process of reading the learning sound data in the learning sound data storage means 44012.

(Step S4501) Optimal phoneme sequence acquisition means 44013 acquires a phoneme sequence (s ^* _t (t = 1, 2,..., T)) corresponding to each frame t constituting the speech voice uttered by the user. To do.

(Step S4502) The cepstrum transformation parameter calculation unit 44015 learns acoustic data in the frequency range indicated by the frequency range designation information (W), the vector (u _t (α)), and the phoneme sequence (s ^* _t ) acquired in step S4501. Based on the above, the optimum cepstrum transformation parameter (α ^* ) in this loop is calculated. The cepstrum conversion parameter calculation means 44015 reads, for example, the stored information of Equation 25, and also reads the learning acoustic data, the vector (u _t (α)) and the phoneme sequence (s ^* _t ) on the memory, Substituting into 25, the cepstrum transformation parameter (α ^* ) is calculated.

  Next, the operation (voice evaluation process) of the voice processing apparatus will be described. The operation of this speech processing apparatus is the same as the operation in the flowcharts of FIGS.

The concept of vocal tract length normalization parameter calculation processing in the speech processing apparatus according to the present embodiment will be described below using the conceptual diagram of FIG. First, a vector sequence in which the average cepstrum vector for each phoneme calculated from the reference speaker speech database is arranged according to the phoneme sequence of the specified utterance content (for example, / Aiueo /) and the user utterance speech of the same phoneme sequence are converted. The parameter (α ^* ) is determined so that the square error with the cepstrum vector is minimized. However, since the obtained parameter (α ^* ) is a cepstrum transformation (cepstrum warping) parameter and does not directly represent the transformation of the vocal tract length as it is, the final vocal tract is obtained using the approximate transformation formula (1 / ρ). The long normalization parameter (sampling frequency conversion rate) γ is calculated. It should be noted that the equations for calculating the vocal tract length normalization parameter are performed by the above-described Equations 22 to 26, Equations 9 to 11, Equations 14 to 16, and the like.

  As described above, according to the present embodiment, it is possible to calculate and output the similarity (rating value) indicating how the pronunciation input by the user is similar to the teacher data. Further, in this case, according to the present embodiment, it is possible to make a highly accurate evaluation that is not affected by individual differences, particularly vocal tract length differences.

  Note that, according to the present embodiment, the voice processing device performs the process of calculating the vocal tract length normalization parameter and the vocal tract length normalization process. However, a configuration in which different devices perform the process of calculating the vocal tract length normalization parameter and the process of normalizing the vocal tract length may be used. In such a case, the speech processing apparatus includes a speech reception unit that receives speech, a first sampling frequency storage unit that stores a first sampling frequency, and a vocal tract length normalization parameter that is information related to a conversion rate of the sampling frequency. A voice sampling length storage parameter storing unit, a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters, and the voice received by the voice receiving unit The voice processing apparatus includes a vocal tract length normalization processing unit that performs sampling processing to obtain second voice data, and a voice processing unit that processes the second voice data. The apparatus for calculating the vocal tract length normalization parameter obtains a cepstrum vector sequence of a speaker's voice from the data constituting the teacher data by short interval analysis, and uses it to calculate two or more reference speaker phoneme average cepstrum. The learning acoustic data, which is an average cepstrum vector sequence obtained by acquiring vectors and arranging the two or more reference speaker phoneme average cepstrum vectors according to the specified utterance content for normalization of vocal tract length (for example, / aiueo /), The stored learning acoustic data storage means, the second cepstrum vector series calculation means for calculating the second cepstrum vector series by the short interval analysis from the received speech, and the second cepstrum vector series, the cepstrum conversion parameter A third cepstra that is linearly transformed using a matrix with frequency warped Cepstrum conversion means for calculating a vector series, optimum phoneme sequence acquisition means for acquiring a phoneme series that is a string of information for identifying a phoneme corresponding to each frame t constituting speech speech uttered by a user, and the learning acoustic data And cepstrum conversion parameter calculation means for calculating a cepstrum conversion parameter based on the third cepstrum vector series and the phoneme series, processing in the cepstrum conversion means based on a predetermined rule, and the optimum phoneme sequence acquisition means And the cepstrum conversion parameter calculating unit to repeat the process in step 1 and the cepstrum conversion parameter calculating unit to obtain a cepstrum conversion parameter, and the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit. There are a device having a vocal tract length normalization parameter calculating means for calculating the vocal tract length normalization parameter.

  Furthermore, the software that implements the speech processing apparatus in the present embodiment is the following program. In other words, the program has a voice receiving step for receiving voice, a stored vocal tract length normalization parameter, and a second sampling frequency calculated using the stored first sampling frequency as parameters. A program for performing a vocal tract length normalization processing step for obtaining a second voice data by performing a sampling process on the voice received in the reception step and a voice processing step for processing the second voice data. .

  Further, in the above program, one or more teacher data, which is data related to a target voice to be compared and is data for each one or more phonemes, is stored, and is based on the teacher data and the voice received in the voice receiving step. And further executing a vocal tract length normalization parameter calculating step for calculating the vocal tract length normalization parameter, wherein the stored vocal tract length normalization parameter is the vocal tract length normalization parameter calculation step. It is preferable that the vocal tract length normalization parameter calculated in (1) is used.

Further, the vocal tract length normalization parameter calculating step in the program includes a second cepstrum vector sequence calculating step for calculating a second cepstrum vector sequence by a short interval analysis from the speech received in the speech receiving step, and the second cepstrum vector sequence calculating step. The cepstrum vector sequence is transformed using a matrix having cepstrum transformation parameters as elements, and a cepstrum transformation step for calculating a frequency-warped third cepstrum vector sequence, and each frame t constituting the speech voice uttered by the user An optimal phoneme sequence acquisition step for acquiring a phoneme sequence that is a sequence of information that identifies a phoneme corresponding to, a stored cepstrum vector sequence, and a cepstrum conversion parameter using the third cepstrum vector sequence and the phoneme sequence Cepstrum transformation to calculate Based on the parameter calculation step and a predetermined rule, the process in the cepstrum conversion step, the process in the optimal phoneme sequence acquisition step, and the process in the cepstrum conversion parameter calculation step are repeated to obtain a cepstrum conversion parameter. Cepstrum conversion parameter acquisition means;
It is preferable to include a vocal tract length normalization parameter calculation sub-step for calculating the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained in the final cepstrum conversion parameter acquisition step.

  Further, in the vocal tract length normalization parameter calculation substep of the program, a linear frequency expansion / contraction ratio is calculated from the cepstrum conversion parameter obtained in the final cepstrum conversion parameter acquisition step, and the vocal tract length normalization is calculated from the linear frequency expansion / contraction ratio. It is preferable to calculate the parameters.

  Further, in the above program, frequency range designation information that is information for designating a frequency range is stored, and in the cepstrum conversion parameter calculation step, learning acoustic data in the frequency range indicated by the frequency range designation information, It is preferable to calculate the cepstrum conversion parameter based on the cepstrum vector sequence and the phoneme sequence.

  In addition, the special voice detected in the above embodiment is silence, insertion, replacement, and omission. It goes without saying that the sound processing device may detect all such special sounds. In addition, the speech processing apparatus mainly detects the special speech using the rating value calculation algorithm described in the first embodiment and the second embodiment, but uses other rating value calculation algorithms. May be.

  The special voice is not limited to silence, insertion, replacement, and omission. For example, the special voice may be a garbage (miscellaneous phonemes such as noise). When garbage is mixed in the received voice, it is often desirable to exclude that section from the similarity calculation target. For example, in pronunciation evaluation, a learner's utterance usually has many breathing and unvoiced intervals, and it is preferable to remove the corresponding utterance intervals from the evaluation target. Note that silence is generally considered a type of garbage.

Therefore, a miscellaneous phoneme (garbage phoneme) that does not belong to any phoneme is set, and a garbage HMM is stored in advance. If the rating value (γ _t (j)) for the garbage HMM is larger than a predetermined value in the score lowering section, it is preferable to determine the section as the garbage section. In particular, in the pronunciation evaluation, when the garbage section extends over two words, it is extremely preferable to exclude the evaluation value from the evaluation value as the occurrence of breathing or the like.

  FIG. 47 shows the external appearance of a computer that executes the programs described in this specification to realize the above-described audio processing apparatuses according to various embodiments. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 47 is a general view of the computer system 340, and FIG. 48 is a block diagram of the computer system 340.

  47, a computer system 340 includes a computer 341 including an FD (Flexible Disk) drive and a CD-ROM (Compact Disk Read Only Memory) drive, a keyboard 342, a mouse 343, a monitor 344, and a microphone 345. .

  48, in addition to the FD drive 3411 and the CD-ROM drive 3412, the computer 341 includes a CPU (Central Processing Unit) 3413, a bus 3414 connected to the CPU 3413, the CD-ROM drive 3412, and the FD drive 3411, and a boot. A ROM (Read-Only Memory) 3415 for storing a program such as an up program, and a RAM (Random Access Memory) connected to the CPU 3413 for temporarily storing instructions of an application program and providing a temporary storage space 3416 and a hard disk 3417 for storing application programs, system programs, and data. Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 340 to execute the functions of the sound processing apparatus according to the above-described embodiment is stored in the CD-ROM 3501 or the FD 3502, inserted into the CD-ROM drive 3412 or the FD drive 3411, and further stored in the hard disk 3417. May be forwarded. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 341 to execute the functions of the voice processing apparatus according to the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Moreover, the computer which performs said program may be single, and plural may be sufficient as it. That is, centralized processing may be performed, or distributed processing may be performed.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the speech processing device according to the present invention has an effect of being able to perform speech processing with high accuracy according to the speaker characteristics of the evaluation target person, as a pronunciation rating device, a karaoke rating device, a speech recognition device, and the like. Useful.

Block diagram of a speech processing apparatus according to Embodiment 1 Flow chart for explaining the operation of the audio processing apparatus Flowchart explaining the same vocal tract length normalization process Diagram showing an example of the HMM specification The figure which shows the measurement result of F1 and F2 Figure showing the same voice analysis conditions The figure which shows the example which expressed the calculated evaluation value with the graph The figure which shows the example which expressed the calculated evaluation value with the graph Figure showing the same output example Figure showing the same output example Block diagram of a speech processing apparatus according to Embodiment 2 Flow chart for explaining the operation of the audio processing apparatus Flowchart for explaining the second sampling frequency calculation process Flow chart explaining the rating process The figure which shows the same evaluation result (tp-DAP score) Figure showing the same output example Block diagram of a speech processing apparatus according to Embodiment 3 Flow chart for explaining the operation of the audio processing apparatus Flow chart explaining the rating process The figure explaining the detection of the silence data Block diagram of a speech processing apparatus according to Embodiment 4 Flow chart for explaining the operation of the audio processing apparatus Flow chart explaining the rating process The figure explaining detection of insertion of the same phoneme Figure showing the same output example Block diagram of a speech processing apparatus according to Embodiment 5 Flow chart for explaining the operation of the audio processing apparatus Flow chart explaining the rating process The figure explaining the detection of substitution of the same phoneme Block diagram of a speech processing apparatus according to Embodiment 6 Flow chart for explaining the operation of the audio processing apparatus Flow chart explaining the rating process Diagram explaining detection of missing phonemes Block diagram of a speech processing apparatus according to Embodiment 7 Flow chart for explaining the operation of the audio processing apparatus Block diagram of a speech processing apparatus according to Embodiment 8 Flowchart explaining the same vocal tract length normalization parameter calculation processing Flow chart for explaining the operation of the audio processing apparatus Flowchart explaining the same vocal tract length normalization process The figure explaining the concept of the same vocal tract length normalization parameter calculation process Block diagram of a speech processing apparatus according to Embodiment 9 Flowchart explaining the same vocal tract length normalization parameter calculation processing The figure explaining the concept of the same vocal tract length normalization parameter calculation process Block diagram of the speech processing apparatus in the tenth embodiment Flowchart explaining the same vocal tract length normalization parameter calculation processing The figure explaining the concept of the same vocal tract length normalization parameter calculation process Overview of the computer system that composes the speech processing unit Block diagram of a computer constituting an audio processing apparatus

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Input reception part 102 Teacher data storage part 103 Voice reception part 104 Teacher data formant frequency storage part 105 1st sampling frequency storage part 106 Sampling part 107 Evaluation object person formant frequency acquisition part 108 Evaluation object person formant frequency storage part 109, 3609 Voice Road length normalization processing unit 110, 1110, 1710, 2110, 2610, 3010, 3410 Audio processing unit 1101, 34101 Frame segmentation unit 1102, 34102 Frame audio data acquisition unit 1103, 11103, 17103, 21103, 26103, 30103 Rating unit 1104 , 21104, 34102 Output means 1109 Voice prompting unit 3601, 4101, 4401 Voice tract length normalization parameter calculation unit 3602 Voice tract length normalization parameter storage 11031 Optimum state determination means 11032 Optimal state probability value acquisition means 11033, 21023, 1111033, 171033 Rating value calculation means 17101, 21101, 26101, 30101 Special speech detection means 34101 Speech recognition means 111032 Sounding section frame phoneme probability value acquisition means 171011 Silence data Storage means 171012 Silent section detection means 36011, 41011 Frequency range designation information storage means 36012 Long-term cepstrum average vector storage means 36013 Second cepstrum vector series calculation means 36014 Cepstrum conversion means 36015, 41015, 44015 Cepstrum conversion parameter calculation means 36016 Final cepstrum conversion Parameter acquisition means 36017 Vocal tract length normalization parameter calculation means 410 2,44012 training acoustic data storage unit 41013 occupied frequency computing unit 44013 best phoneme sequence obtaining means

Claims (11)

A voice reception unit for receiving voice;
A first sampling frequency storage unit storing a first sampling frequency;
A vocal tract length normalization parameter storage unit storing a vocal tract length normalization parameter that is information on the conversion rate of the sampling frequency;
A vocal tract that obtains second audio data by performing sampling processing on the audio received by the audio receiving unit at a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters. A long normalization processing unit;
An audio processing unit for processing the second audio data ;
A teacher data storage unit that stores one or more teacher data that is data relating to the speech to be compared and that is data for one or more phonemes;
A vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on the teacher data and the voice received by the voice reception unit;
The vocal tract length normalization parameter of the vocal tract length normalization parameter storage unit is a voice processing device that is a vocal tract length normalization parameter calculated by the vocal tract length normalization parameter calculation unit ,
The vocal tract length normalization parameter calculation unit
Learning acoustic data storage means for storing learning acoustic data which is a phoneme average cepstrum vector sequence in which phoneme average cepstrum vectors are arranged according to the specified utterance content;
A second cepstrum vector sequence calculating means for calculating a second cepstrum vector sequence by a short interval analysis from the voice received by the voice receiving unit;
Cepstrum conversion means for converting the second cepstrum vector sequence using a matrix having cepstrum conversion parameters as elements and calculating a frequency-warped third cepstrum vector sequence;
Optimal phoneme sequence acquisition means for acquiring a phoneme sequence that is a sequence of information for identifying a phoneme corresponding to each frame constituting the speech voice uttered by the user;
Cepstrum conversion parameter calculating means for calculating a cepstrum conversion parameter using the learning acoustic data and the third cepstrum vector sequence and the phoneme sequence;
Based on a predetermined rule, final cepstrum conversion parameter acquisition means for obtaining a cepstrum conversion parameter by repeating processing in the cepstrum conversion unit, processing in the optimal phoneme sequence acquisition unit, and processing in the cepstrum conversion parameter calculation unit When,
A speech processing apparatus comprising: a vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on a cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit. The vocal tract length normalization parameter calculation unit, instead of the configuration according to claim 1,
Learning acoustic data storage means for storing learning acoustic data which is a concatenated HMM in which phoneme HMMs are concatenated according to designated utterance content;
A second cepstrum vector sequence calculating means for calculating a second cepstrum vector sequence by a short interval analysis from the voice received by the voice receiving unit;
Cepstrum conversion means for converting the second cepstrum vector sequence using a matrix having cepstrum conversion parameters as elements and calculating a frequency-warped third cepstrum vector sequence;
An occupancy degree calculating means for calculating an occupancy degree which is a posterior probability of the voice received according to the designated utterance content;
Cepstrum conversion parameter calculation means for calculating a cepstrum conversion parameter using the learning acoustic data, the third cepstrum vector series, and the occupation frequency;
A final cepstrum conversion parameter obtaining unit that obtains a cepstrum conversion parameter by repeating the process in the cepstrum conversion unit, the process in the occupancy calculation unit, and the process in the cepstrum conversion parameter calculation unit based on a predetermined rule; ,
Said final cepstrum conversion parameter acquisition means based on the cepstrum transformation parameters obtained, the audio processing apparatus according to claim 1, further comprising a vocal tract length normalization parameter calculating means for calculating the vocal tract length normalization parameter. The vocal tract length normalization parameter calculating means includes:
The speech processing apparatus according to claim 1 or 2 , wherein a linear frequency expansion / contraction ratio is calculated from the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition means, and the vocal tract length normalization parameter is calculated from the linear frequency expansion / contraction ratio. . The vocal tract length normalization parameter calculation unit
Further comprising frequency range designation information storage means for storing frequency range designation information which is information for designating a frequency range;
The cepstrum conversion parameter calculation means includes
4. The speech processing apparatus according to claim 2 , wherein a cepstrum conversion parameter is calculated also using the frequency range designation information. The teacher data includes two or more state identifiers for identifying states and information on transition probabilities between states,
The voice processing unit
Frame dividing means for dividing the second audio data into frames;
Frame audio data acquisition means for obtaining one or more frame audio data which are audio data for each of the divided frames;
Rating means for rating the voice received by the voice receiving unit based on the teacher data and the one or more frame voice data;
Comprising output means for outputting a rating result in the rating means ,
The rating means is
Optimal state determination means for determining an optimal state in which the transition probability of at least one frame audio data of the one or more frame audio data is the highest ;
Optimal state probability value acquisition means for acquiring a probability value indicating the posterior probability of the optimal state determined by the optimal state determination means;
Claims having a rating value calculating means for calculating an evaluation value of the speech by using the probability value, wherein the optimal state probability value obtaining means has obtained, the ratio of the sum of the probability values in all the states of the frame corresponding to the probability value The speech processing apparatus according to any one of claims 1 to 4 . The rating means is
Optimal state determination means for determining an optimal state in which the transition probability of the one or more frame audio data is the highest ;
A pronunciation interval frame phoneme probability value acquisition unit that acquires, for each pronunciation interval, one or more probability values in the overall state of the phoneme having the optimal state of each frame determined by the optimal state determination unit;
A rating value calculating means for calculating a time average value of two or more probability values for each of one or more sounding sections acquired by the sounding section frame phoneme probability value acquiring means, and calculating a speech rating value using the time average value. The speech processing apparatus according to claim 5 , comprising: The voice processing device is a karaoke evaluation device,
The voice reception unit
Accepts singing voice of the person being evaluated,
The voice processing unit
The speech processing apparatus according to claim 1, wherein the singing voice is evaluated. A first sampling frequency storage unit storing a first sampling frequency;
A vocal tract length normalization parameter storage unit storing a vocal tract length normalization parameter that is information on the conversion rate of the sampling frequency;
A vocal tract that obtains second audio data by performing sampling processing on the audio received by the audio receiving unit at a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters. A long normalization processing unit ;
An audio processing unit for processing the second audio data;
A teacher data storage unit that stores one or more teacher data that is data relating to the speech to be compared and that is data for one or more phonemes;
A vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on the teacher data and the voice received by the voice reception unit;
The vocal tract length normalization parameter of the vocal tract length normalization parameter storage unit is a vocal tract length normalization parameter calculated by the vocal tract length normalization parameter calculation unit,
The vocal tract length normalization parameter calculation unit
Learning acoustic data storage means for storing learning acoustic data which is a phoneme average cepstrum vector sequence in which phoneme average cepstrum vectors are arranged according to the specified utterance content;
A second cepstrum vector sequence calculating means for calculating a second cepstrum vector sequence by a short interval analysis from the voice received by the voice receiving unit;
Cepstrum conversion means for converting the second cepstrum vector sequence using a matrix having cepstrum conversion parameters as elements and calculating a frequency-warped third cepstrum vector sequence;
Optimal phoneme sequence acquisition means for acquiring a phoneme sequence that is a sequence of information for identifying a phoneme corresponding to each frame constituting the speech voice uttered by the user;
Cepstrum conversion parameter calculating means for calculating a cepstrum conversion parameter using the learning acoustic data and the third cepstrum vector sequence and the phoneme sequence;
Based on a predetermined rule, final cepstrum conversion parameter acquisition means for obtaining a cepstrum conversion parameter by repeating processing in the cepstrum conversion unit, processing in the optimal phoneme sequence acquisition unit, and processing in the cepstrum conversion parameter calculation unit When,
A digital signal processor comprising: a vocal tract length normalization parameter calculating unit that calculates the vocal tract length normalization parameter based on a cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit . The vocal tract length normalization parameter calculation unit, instead of the configuration according to claim 8,
Learning acoustic data storage means for storing learning acoustic data which is a concatenated HMM in which phoneme HMMs are concatenated according to designated utterance content;
A second cepstrum vector sequence calculating means for calculating a second cepstrum vector sequence by a short interval analysis from the voice received by the voice receiving unit;
Cepstrum conversion means for converting the second cepstrum vector sequence using a matrix having cepstrum conversion parameters as elements and calculating a frequency-warped third cepstrum vector sequence;
An occupancy degree calculating means for calculating an occupancy degree which is a posterior probability of the voice received according to the designated utterance content;
Cepstrum conversion parameter calculation means for calculating a cepstrum conversion parameter using the learning acoustic data, the third cepstrum vector series, and the occupation frequency;
A final cepstrum conversion parameter obtaining unit that obtains a cepstrum conversion parameter by repeating the process in the cepstrum conversion unit, the process in the occupancy calculation unit, and the process in the cepstrum conversion parameter calculation unit based on a predetermined rule; ,
9. The digital signal processor according to claim 8, further comprising vocal tract length normalization parameter calculation means for calculating the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition means. The computer,
A voice reception unit for receiving voice;
A first sampling frequency storage unit storing a first sampling frequency;
A vocal tract length normalization parameter storage unit storing a vocal tract length normalization parameter that is information on the conversion rate of the sampling frequency;
A vocal tract that obtains second audio data by performing sampling processing on the audio received by the audio receiving unit at a second sampling frequency calculated using the vocal tract length normalization parameter and the first sampling frequency as parameters. A long normalization processing unit;
An audio processing unit for processing the second audio data;
A teacher data storage unit that stores one or more teacher data that is data relating to the speech to be compared and that is data for one or more phonemes;
A program for functioning as comprising the vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on the teacher data and the voice received by the voice reception unit,
The vocal tract length normalization parameter of the vocal tract length normalization parameter storage unit is a vocal tract length normalization parameter calculated by the vocal tract length normalization parameter calculation unit,
The vocal tract length normalization parameter calculation unit
Learning acoustic data storage means for storing learning acoustic data which is a phoneme average cepstrum vector sequence in which phoneme average cepstrum vectors are arranged according to the specified utterance content;
A second cepstrum vector sequence calculating means for calculating a second cepstrum vector sequence by a short interval analysis from the voice received by the voice receiving unit;
Cepstrum conversion means for converting the second cepstrum vector sequence using a matrix having cepstrum conversion parameters as elements and calculating a frequency-warped third cepstrum vector sequence;
Optimal phoneme sequence acquisition means for acquiring a phoneme sequence that is a sequence of information for identifying a phoneme corresponding to each frame constituting the speech voice uttered by the user;
Cepstrum conversion parameter calculating means for calculating a cepstrum conversion parameter using the learning acoustic data and the third cepstrum vector sequence and the phoneme sequence;
Based on a predetermined rule, final cepstrum conversion parameter acquisition means for obtaining a cepstrum conversion parameter by repeating processing in the cepstrum conversion unit, processing in the optimal phoneme sequence acquisition unit, and processing in the cepstrum conversion parameter calculation unit When,
A program for functioning as comprising a vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on a cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit . The vocal tract length normalization parameter calculation unit, instead of the configuration according to claim 10,
Learning acoustic data storage means for storing learning acoustic data which is a concatenated HMM in which phoneme HMMs are concatenated according to designated utterance content;
A second cepstrum vector sequence calculating means for calculating a second cepstrum vector sequence by a short interval analysis from the voice received by the voice receiving unit;
Cepstrum conversion means for converting the second cepstrum vector sequence using a matrix having cepstrum conversion parameters as elements and calculating a frequency-warped third cepstrum vector sequence;
An occupancy degree calculating means for calculating an occupancy degree which is a posterior probability of the voice received according to the designated utterance content;
Cepstrum conversion parameter calculation means for calculating a cepstrum conversion parameter using the learning acoustic data, the third cepstrum vector series, and the occupation frequency;
A final cepstrum conversion parameter obtaining unit that obtains a cepstrum conversion parameter by repeating the process in the cepstrum conversion unit, the process in the occupancy calculation unit, and the process in the cepstrum conversion parameter calculation unit based on a predetermined rule; ,
The vocal tract length normalization parameter calculation unit that calculates the vocal tract length normalization parameter based on the cepstrum conversion parameter obtained by the final cepstrum conversion parameter acquisition unit is provided to function as comprising the vocal tract length normalization parameter calculation unit. program.