JP4859125B2 - Pronunciation rating device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the following problem: evaluation accuracy by phoneme units is not sufficient. <P>SOLUTION: A pronunciation evaluating apparatus obtains a feature vector series from a teacher data and one or more frame voice data, compares the feature vector series with an acoustic model according to a phoneme series to be evaluated, obtains an optimal state series which is a set of an optimal state for each frame, identifies one or more optimal phoneme series in which the same phoneme continues among the optimal state series, obtains one or more optimal phoneme portion series which are groups of the one or more optimal phoneme series, obtains one or more feature vector portion series which are groups of one or more feature vectors corresponding to each optimal phoneme portion series. obtains a posterior probability at which the feature vector series is a phoneme to be evaluated, and calculates an evaluation value from the posterior probability. The evaluation by phoneme units is highly accurately performed by the pronunciation evaluating apparatus. <P>COPYRIGHT: (C)2008,JPO&amp;INPIT

Description

  The present invention relates to an apparatus for evaluating input speech, and more particularly to a pronunciation rating apparatus that can be used for language learning and the like.

Conventionally, one or more teacher data, which is data related to the speech to be compared and is based on a hidden Markov model (HMM) for each phoneme, is stored, and a speech reception unit that receives speech input, and the received speech Are divided into frames, a frame audio data acquisition unit that obtains one or more frame audio data that is audio data for each of the divided frames, the teacher data, and the one or more frame audio data There has been a pronunciation rating device including a rating unit that evaluates a voice received by the voice receiving unit, and an output unit that outputs a rating result of the rating unit (for example, see Patent Document 1). In the above-described pronunciation rating device, for example, pronunciation rating is performed using an algorithm called p-DAP. The p-DAP calculates a rating value so as to represent an optimal posterior probability (probability value) of phonemes among all phonemes in each frame.
JP 2006-227587 A (first page, FIG. 1 etc.)

  However, in the conventional pronunciation rating device, information of speech data other than the phoneme section to which the frame belongs is also mixed in the rating score. Therefore, it cannot be said that the evaluation accuracy of phoneme units is sufficient.

  The pronunciation rating device according to the first aspect of the present invention includes a teacher data storage unit storing at least one teacher data, which is an acoustic model for each of one or more phonemes, and an array of one or more phonemes to be evaluated. Is an optimal phoneme subsequence storage unit that stores one or more sets of optimal phoneme subsequences that are sets of one or more optimal phoneme sequences that are continuous, a voice reception unit that receives voice input, and the voice reception unit The voice is divided into frames, a frame voice data acquisition unit that obtains one or more frame voice data that is voice data for each divided frame, and a set of feature vectors for each frame from the one or more frame voice data. A feature vector sequence acquisition unit that acquires a feature vector sequence, and each optimal phoneme portion of the optimal phoneme subsequence storage unit from the feature vector sequence acquired by the feature vector sequence acquisition unit A feature vector partial sequence acquisition unit that acquires one or more feature vector partial sequences that are a set of one or more feature vectors corresponding to a sequence, and teacher data is read from the teacher data storage unit, and the teacher data is used to The feature vector partial sequence acquisition unit calculates a posterior probability that the feature vector partial sequence is a phoneme to be evaluated, and calculates a speech evaluation value from the posterior probability, and the evaluation value calculation unit calculates This is a pronunciation rating device having an output unit for outputting the rated value.

  With this configuration, the phoneme unit can be evaluated with high accuracy.

  In addition, the pronunciation rating device according to the second aspect of the present invention, in contrast to the first aspect, is a rating target phoneme sequence storage unit that stores a rating target phoneme sequence that is information on an arrangement of one or more phonemes to be rated; An acoustic model along the evaluation target phoneme sequence is read from the teacher data storage unit, and the acoustic model is a feature vector sequence acquired by the feature vector sequence acquisition unit, and a feature vector sequence corresponding to the acoustic model. The optimal state sequence acquisition unit that acquires an optimal state sequence that is a set of optimal states for each frame and one or more consecutive continuous phonemes in the optimal state sequence acquired by the optimal state sequence acquisition unit An optimal phoneme subsequence acquisition unit that identifies an optimal phoneme sequence and acquires one or more optimal phoneme subsequences that are a set of the one or more optimal phoneme sequences; Is a feature vector partial sequence that is a set of one or more feature vectors corresponding to each optimum phoneme partial sequence acquired by the optimal phoneme partial sequence acquisition unit from the feature vector sequence acquired by the feature vector sequence acquisition unit. This is the pronunciation rating device to be acquired.

  With this configuration, the phoneme unit can be evaluated with high accuracy. Moreover, it is not necessary to prepare one or more sets of optimal phoneme subsequences in advance.

  Also, the pronunciation rating device of the third aspect of the invention is based on the condition that the rating value calculation unit is provided with teacher data that is all acoustic models for either the first or second aspect of the invention. For each time of phoneme segment speech data, calculate the posterior probability that the phoneme segment speech data is in the state of the acoustic model corresponding to the correct phoneme to be rated, and calculate the phoneme rating value from the posterior probability It is a pronunciation rating device.

  With this configuration, the phoneme unit can be evaluated with high accuracy.

  Further, the pronunciation rating device according to the fourth aspect of the present invention is the first or second aspect of the invention, wherein the rating value calculation unit evaluates the phoneme segment speech data for each time of the phoneme segment speech data. Is a pronunciation rating device that calculates a posterior probability that is a phoneme that is a correct answer and calculates a rating value of the phoneme from the posterior probability.

  With this configuration, the phoneme unit can be evaluated with higher accuracy at high speed.

  Also, in the pronunciation rating device of the fifth invention, the rating value calculation unit, with respect to any one of the first and second inventions, has a posterior probability that the phoneme segment speech data is a phoneme that is a correct answer to be rated. It is a pronunciation rating device that calculates and calculates a phoneme rating value from the posterior probability.

  With this configuration, the phoneme unit can be evaluated with higher accuracy at a higher speed.

  In addition, the pronunciation rating device according to the sixth aspect of the invention is the pronunciation rating device according to the fifth aspect, wherein the rating value calculation unit acquires the co-occurrence probability value by a forward algorithm without using a backward algorithm. It is.

  With this configuration, the phoneme unit can be evaluated with higher accuracy at high speed.

  Further, the pronunciation rating device of the seventh aspect of the present invention is the first to sixth aspects of the invention, wherein the rating value calculator is a sentence or word based on the rating value for each frame or phoneme. This is a pronunciation rating device that calculates the rating value.

  With this configuration, the rating value of a sentence or word can be calculated.

  The pronunciation rating device of the eighth aspect of the invention further comprises a phoneme time information acquisition unit that acquires phoneme time information, which is information related to the time of each phoneme, with respect to the seventh aspect of the invention, and the rating value calculation unit Is a pronunciation rating device that calculates a rating value of a sentence or a word from a weighted average of phoneme rating values weighted by phoneme time length.

  With this configuration, it is possible to calculate sentence and word rating values with high accuracy.

  According to the pronunciation rating device according to the present invention, the phoneme unit can be evaluated with high accuracy.

Hereinafter, embodiments of a pronunciation rating device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.
(Embodiment 1)

  In the present embodiment, a pronunciation evaluation apparatus that performs pronunciation evaluation using an algorithm that calculates a rating point for each phoneme section that can be obtained from the optimum state series will be described. This algorithm calculates a dynamic posterior probability for each phoneme section and adds a qualifier to distinguish it from the algorithm described later, PDAPS-PE (Phoneme Ergodic Phoneme unit Dynamic A posteriori Probability based pronunciation Scoring / Phonetic ergodic phonetic unit pronunciation evaluation based on dynamic posterior probability).

  The PDAPS-PE algorithm is an algorithm in which speech data used for calculating a rating value is limited to a phoneme section and information other than the phoneme section is excluded.

  FIG. 1 is a block diagram of a pronunciation rating device in the present embodiment. The pronunciation evaluation apparatus includes an input reception unit 101, a teacher data storage unit 102, a rating target phoneme sequence storage unit 103, a speech reception unit 104, a frame audio data acquisition unit 105, a feature vector sequence acquisition unit 106, and an optimum state sequence acquisition unit 107. , An optimal phoneme partial sequence acquisition unit 108, a feature vector partial sequence acquisition unit 109, a rating value calculation unit 110, and an output unit 111.

  The input receiving unit 101 receives inputs such as an operation start instruction for instructing an operation start of the pronunciation rating device and an end instruction for ending the process. The input means for such an instruction may be anything such as a numeric keypad, keyboard, mouse or menu screen. The input receiving unit 101 can be realized by a device driver for input means such as a numeric keypad and a keyboard, control software for a menu screen, and the like.

  The teacher data storage unit 102 stores one or more teacher data that are acoustic models for one or more phonemes (hereinafter also referred to as acoustic model parameters as appropriate). The teacher data is preferably data based on a hidden Markov model (HMM). Furthermore, in the pronunciation rating device (when used in the PDAPS-PE algorithm) in the present embodiment, the teacher data is an HMM, and from the terminal state of one phoneme HMM, the one phoneme or all other phonemes. It is preferable that the acoustic model is connected to the starting end state. In other words, in the pronunciation evaluation apparatus after the second embodiment (when used in the PDAPS, PAPPS, and PAPPS-FN algorithms), the teacher data is transmitted from the terminal state of one phoneme HMM to one phoneme or all other phonemes. It is not an acoustic model connected to the starting state of phonemes.

  The teacher data does not necessarily need to be data based on the HMM in which the HMMs for each phoneme are connected. The teacher data may be a simple set of all phoneme HMMs. The teacher data does not necessarily need to be data based on the HMM. The teacher data may be data based on other models such as a single Gaussian distribution model, a probability model (GMM: Gaussian mixture model), and a statistical model. Note that the acoustic model is, for example, a set of an ID for identifying a sound and a feature vector that characterizes the sound. The teacher data storage unit 102 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The rating target phoneme sequence storage unit 103 stores a rating target phoneme sequence, which is information on the arrangement of one or more phonemes to be rated. The rating-target phoneme series is information indicating a phoneme string such as a word or sentence (correct word or sentence) that the user must originally pronounce. The phoneme information is, for example, a phoneme ID or a character code indicating the phoneme. The information on the arrangement of one or more phonemes is, for example, a phoneme character code string such as [a, o, i] or a phoneme ID string such as [1, 5, 2]. Further, the rating target phoneme sequence in the rating target phoneme sequence storage unit 103 may be information received by the input receiving unit 101 or may be stored in advance. The evaluation target phoneme sequence storage unit 103 is preferably a non-volatile recording medium, but can also be realized by a volatile recording medium.

  The voice receiving unit 104 receives an input of voice to be rated. Here, “acceptance” is usually acceptance from a microphone, but it may be processing for reading audio stored in a recording medium, reception processing from an external device, or the like. The voice reception unit 104 can be realized by, for example, microphone driver software. In addition, it may be considered that the voice reception unit 104 is realized by a microphone and its driver. The sound may be input from a microphone or may be input by reading from a recording medium such as a magnetic tape or a CD-ROM.

  The frame audio data acquisition unit 105 divides the audio received by the audio reception unit 104 into frames, obtains one or more frame audio data that are audio data for each of the divided frames, and obtains the one or more frame audio data. Place on memory. The frame audio data acquisition unit 105 can be realized by a known technique. The frame audio data acquisition unit 105 can be usually realized by an MPU, a memory, or the like. The processing procedure of the frame audio data acquisition unit 105 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The feature vector sequence acquisition unit 106 uses a feature vector sequence “O = o ₁ , o ₂ ,..., Which is a set of feature vectors for each frame, from one or more frame audio data acquired by the frame audio data acquisition unit 105. to get the o _T ". The feature vector series obtaining unit 106 obtains a feature vector (o _t ) by performing spectrum analysis on the frame audio data. The feature vector is, for example, an MFCC obtained by performing discrete cosine transform on a filter bank output of 24 channels using a triangular filter, and the static parameter, the delta parameter, and the delta delta parameter are further normalized to 12 dimensions, respectively. Power and delta power and delta delta power (39th dimension). In spectral analysis, it is preferable to perform cepstrum average removal. However, it goes without saying that the voice analysis conditions may be other conditions. Note that the feature vector series acquisition unit 106 can be realized by a known technique. The feature vector series acquisition unit 106 can usually be realized by an MPU, a memory, or the like. The processing procedure of the feature vector sequence acquisition unit 106 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The optimum state sequence acquisition unit 107 reads the rating target phoneme sequence from the rating target phoneme sequence storage unit 103, reads the acoustic model along the read rating target phoneme sequence from the teacher data storage unit 102, and arranges it on the memory. . Then, the optimal state sequence acquisition section 107, an acoustic model reading the feature vector series feature vector series acquisition unit 106 has acquired _{(O = o 1, o 2} , ·· o t ··, o T) be compares the sequence of feature vectors corresponding to acoustic models, the optimal state sequence is a set of optimal conditions for each frame (optimum condition for the feature vector ^{_{o t) (q * 1,}} q * 2, ·· q * t ·・, Q ^* _T ) is acquired. Optimal state sequence acquisition section 107, based on each feature vector o _t constituting the obtained feature vector series, to determine the optimal conditions for a given frame (optimum condition for the feature vector o _t). The algorithm by which the optimum state sequence acquisition unit 107 determines the optimum state is, for example, the Viterbi algorithm. In such a case, the optimum state sequence acquisition unit 107 usually determines the optimum state using the acoustic model (here, HMM) connected along the phoneme sequence to be evaluated as described above. The optimum state sequence acquisition unit 107 obtains an optimum state sequence that is the optimum state of two or more frames. The optimum state sequence acquisition unit 107 can be usually realized by an MPU, a memory, or the like. The processing procedure of the optimum state sequence acquisition unit 107 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The optimal phoneme subsequence acquisition unit 108 selects the same phoneme among the optimal state sequences (q ^* ₁ , q ^* ₂ ,... Q ^* _t. , Q ^* _T ) acquired by the optimal state sequence acquisition unit 107. One or more continuous optimal phoneme sequences are identified, and one or more optimal phoneme subsequences, which are a set of the one or more optimal phoneme sequences, are acquired. That is, when an acoustic model belongs optimum state ^(q _{* t)} and ^(p _{* t),} the optimum phoneme partial sequence acquisition section 108, the optimal state sequence _{^{(q * 1, q * 2}} , ·· q * t ·· , Q ^* _T ), the acoustic model corresponding to each optimum state is read from the teacher data storage unit 102 and the optimum phoneme sequence (p ^* ₁ , p ^* ₂ ,... P ^* _t. , P ^* _T ) is read. get. Then, the optimum phoneme subsequence acquisition unit 108 detects a portion in which the same phonemes are continuous in the optimum phoneme sequence (p ^* ₁ , p ^* ₂ ,... P ^* _t. , P ^* _T ), and the phoneme sequence. ({P ^* ₁ , p ^* ₂ ,...} ⁽¹⁾ ... {..., P ^* _t .., p ^* _{T (n)} } ^(n). · ^to obtain ^{_{p * T} (s)}}} . Note that the phoneme ^(p _{* t)} is, for example, a character code indicating the ID and phonemic identifying phonemes. Further, {· · ·, ^{p *} _t ··, p ^* _{T (n)} } ⁽ⁿ⁾ is one of the optimum phoneme subsequences, and the optimum phoneme subsequence acquisition unit 108 can be usually realized by an MPU, a memory, etc. Optimal phoneme subsequences The processing procedure of the acquisition unit 108 is usually realized by software, and the software is recorded in a recording medium such as a ROM. , It may be realized by hardware (dedicated circuit).

The feature vector subsequence acquisition unit 109 is a set of one or more feature vectors corresponding to each optimum phoneme subsequence acquired by the optimal phoneme subsequence acquisition unit 108 from the feature vector sequence acquired by the feature vector sequence acquisition unit 106. Feature vector subsequence ({o ^* ₁ , o ^* ₂ ,...} ⁽¹⁾ ... {..., o ^* _t. , O ^* _{T (n)} } ^(n). .., O ^* _T } ^(s) } is acquired, wherein the feature vector subsequence is the phoneme segment speech data to be evaluated, and the representative phoneme of the optimal phoneme subsequence corresponding to it is the target of the evaluation The feature vector subsequence acquisition unit 109 can be usually realized by an MPU, a memory, etc. The processing procedure of the feature vector subsequence acquisition unit 109 is usually realized by software, and the software It is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The rating value calculation unit 110 calculates a phoneme rating value from phoneme interval data. The rating value calculation unit 110 does not use all input speech data when calculating the phoneme rating value. In other words, the rating value calculation unit 110, under the condition that teacher data that is all acoustic models is given, for each time of the phoneme segment speech data, the phoneme segment speech data becomes a phoneme that is the correct answer to be evaluated. The posterior probability that is the state of the acoustic model corresponding to is calculated, and the phoneme rating value is calculated from the posterior probability. It should be noted that the state to be rated is the one that the acoustic model corresponding to the correct phoneme has. Specifically, the rating value calculation unit 110 calculates a voice rating value (PDPS-PE (t)) by the following formulas 1 and 2, for example. That is, the rating value calculation unit 110 obtains the optimal phoneme partial sequence acquired by the optimal phoneme partial sequence acquisition unit 108 and the feature vector partial sequence “O ⁽ⁿ⁾ acquired by the feature vector partial sequence acquisition unit 109 corresponding to the partial sequence. ”And all the acoustic models read out from the teacher data storage unit 102, for each optimum phoneme“ P _t ^* ”for each time of the optimum phoneme subsequence, at the time (frame)“ t ”of the feature vector subsequence. The posterior probability with the state “j” is obtained in all states included in the optimal phoneme “P _t ^* ”, and the sum of the posterior probabilities is obtained, so that the speech rating value (PNAPS-PE (t)) Is calculated. More specifically, the rating value calculation unit 110 includes the optimal phoneme partial sequence acquired by the optimal phoneme partial sequence acquisition unit 108 and the feature vector partial sequence “O” acquired by the feature vector partial sequence acquisition unit 109 corresponding to the partial sequence. ^(N) ”and all the acoustic models read out from the teacher data storage unit 102, and for each optimal phoneme“ P _t ^* ”for each time of the optimal phoneme subsequence, The feature vector series “O ⁽ⁿ⁾ ” is observed under the condition that the acoustic model parameter “λ ^all ” is given, and the state of the vector series at time (frame) “t” is “j”. It acquires a certain probability, the probability value for each the acquired state, when all the acoustic model parameters "lambda ^all" and the feature vector series "O ^(n)" is given Obtained in all states the state at that time of vector sequence (frame) "t" is included in the optimal a posteriori probability of the "j" phoneme "P _t ^*", by obtaining the sum of the posterior probabilities, The voice rating value (PDAPS-PE (t)) is calculated.

In Equation 1, “λ ^all ” is a parameter of all acoustic models (teacher data), and “N” is the total number of states of the entire teacher data. Formula 2 indicates that the probability value is calculated by a forward-backward algorithm that is a known algorithm. In addition, in Equations 1, 2, etc., “Pr (q _t = j, O ⁽ⁿ⁾ | λ ^all )” is a feature vector portion under the condition that ^all acoustic model parameters “λ ^all ” are given. The probability is that the sequence “O ⁽ⁿ⁾ ” is observed and the state of the vector sequence at time (frame) “t” is “j”.

That is, Equation 2 uses the forward-backward algorithm to observe the feature vector partial series “O ⁽ⁿ⁾ ” under the condition that ^all acoustic model parameters “λ ^all ” are given, and the vector series Is a mathematical formula for calculating the probability that the state at time (frame) “t” is “j”.

Hereinafter, a method for calculating the probability value in Expression 2 will be described. Conventional phoneme-based acoustic models (HMMs) used for pronunciation evaluation and speech recognition are independent for each phoneme and are defined to be executed by only one acoustic model. The forward-backward algorithm cannot be executed using ^all acoustic model parameters “λ ^all ” as shown in Equation 2. Therefore, the acoustic model for each independent phoneme is modified so that it behaves as one large acoustic model.

  Specifically, for example, one left-to-right phoneme HMM has a start end state and a terminal end state, and is independent in a form in which a state transition including a self transition is connected therebetween. The state transition is virtually connected from the terminal state of the one phoneme HMM to the start state of all the phoneme HMMs. In this way, a virtual state transition from the end to the start is created and connected in all phoneme HMMs, so that a plurality of phoneme HMMs are mounted as one large acoustic model. Therefore, it is preferable that the teacher data is a hidden Markov model (HMM) and is an acoustic model connected from the terminal state of one phoneme HMM to the start state of the one phoneme or all other phonemes. .

  By using the acoustic model modified as described above, the forward / backward algorithm of Expression 2 is executed. The modified acoustic model and the forward-facing backward algorithm executed by using the acoustic model are referred to as a phoneme ergodic acoustic model and a phoneme ergodic forward-backward algorithm.

  That is, the rating value calculation unit 110 calculates the rating value of the speech using the phoneme probability value based on the acquired one or more posterior probabilities as a parameter. For example, the rating value calculation unit 110 calculates an average value or median value of the probability values of the phonemes for each time in the time interval, and uses the calculated value as the rating value. If the time interval is set to a segment such as one sentence, one word, or one phoneme, the rating value calculation unit 110 can calculate a rating value for each sentence, each word, or each phoneme. The rating value calculation unit 110 can usually be realized by an MPU, a memory, or the like. The processing procedure of the rating value calculation unit 110 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 111 outputs the rating value calculated by the rating value calculation unit 110. The output mode of the rating value does not matter. The rating value may be output as a numerical value, or may be output as a line graph or a bar graph. Here, the output is a concept including display on a display, printing on a printer, sound output, transmission to an external device, accumulation in a recording medium, and the like. The output unit 111 may be considered as including or not including an output device such as a display or a speaker. The output unit 111 can be realized by output device driver software, or output device driver software and an output device.

  Next, the operation of the pronunciation rating device will be described with reference to the flowcharts of FIGS.

  (Step S201) The input receiving unit 101 determines whether or not an operation start instruction for instructing an operation start of the pronunciation rating device has been received. If an operation start instruction is accepted, the process goes to step S202, and if an operation start instruction is not accepted, the process jumps to step S214.

  (Step S202) The voice receiving unit 104 determines whether or not a voice input has been received. If a voice input is accepted, the process goes to step S203, and if a voice input is not accepted, the process jumps to step S213.

  (Step S203) The frame audio data acquisition unit 105 temporarily stores the audio data received in step S202 in a buffer (not shown).

  (Step S204) The frame audio data acquisition unit 105 divides the audio data temporarily stored in the buffer into frames, acquires frame audio data that is audio data for each of the divided frames, and obtains one or more frame audio data. Place on memory.

(Step S205) The feature vector sequence acquisition unit 106 performs speech analysis on each of the one or more frame audio data acquired by the frame audio data acquisition unit 105, extracts one or more feature vectors, and extracts a feature vector sequence (O = o ₁ , get _o 2, ·· _o t ··, the _{o T).} This feature vector is, for example, a MFCC obtained by discrete cosine transform of a filter bank output of 24 channels using a triangular filter, and the static parameter, the delta parameter, and the delta delta parameter are further normalized to 12 dimensions, respectively. Power and delta power and delta delta power (39th dimension).

  (Step S <b> 206) The optimum state sequence acquisition unit 107 reads the rating target phoneme sequence from the rating target phoneme sequence storage unit 103.

  (Step S207) The optimum state sequence acquisition unit 107 reads out the acoustic model along the evaluation target phoneme sequence read out in step S206 from the teacher data storage unit 102 and arranges it on the memory.

(Step S208) optimal state sequence acquisition section 107, an acoustic model read in step S207, the obtained feature vector series in step _{S205 (O = o 1, o} 2, ·· o t ··, o T) and comparison, the optimal state sequence is a set of (optimal conditions for the feature vector _{o t)} optimal conditions for each frame ^{_{(q * 1, q * 2}} , ·· q * t ··, q * T) , and the Viterbi algorithm get.

(Step S209) The optimal phoneme subsequence acquisition unit 108 selects the same phoneme in the optimal state sequence (q ^* ₁ , q ^* ₂ ,... Q ^* _t. , Q ^* _T ) acquired in step S208. One or more continuous optimal phoneme sequences are identified, one or more optimal phoneme subsequences that are a set of the one or more optimal phoneme sequences are acquired, and one or more optimal phoneme subsequences are arranged in the memory. Any information may be used as the delimiter for each optimal phoneme subsequence. That is, each optimum phoneme subsequence may be stored in a different buffer, or delimiter information (for example, “,”) may be inserted between the optimum phoneme subsequences.

(Step S210) The feature vector partial sequence acquisition unit 109 is a feature vector partial sequence that is a set of one or more feature vectors corresponding to each optimum phoneme partial sequence acquired in step S209 from the feature vector sequence acquired in step S205 ( {O ^* ₁ , o ^* ₂ , ...} ⁽¹⁾ ... {..., o ^* _t ..., o ^* _{T (n)} } ⁽ⁿ⁾ ... {..., o ^* _T } ^(s) } is acquired in one or more sets, and feature vector partial sequence acquisition section 109 takes the correspondence between the optimal phoneme partial sequence and the feature vector partial sequence based on the frame ID and time.

  (Step S211) The rating value calculation unit 110 calculates a rating value. Details of an example algorithm for calculating the rating value will be described with reference to the flowchart of FIG.

  (Step S212) The output unit 111 outputs the rating value calculated by the rating value calculation unit 110.

  (Step S213) The voice reception unit 104 determines whether or not a timeout has occurred. That is, the voice receiving unit 104 determines whether or not a voice input has been received for a predetermined time or more. If timed out, the process returns to step S201, and if not timed out, the process returns to step S202.

  (Step S214) The input receiving unit 101 determines whether an end instruction has been received. If an end instruction is accepted, the process ends. If no end instruction is accepted, the process returns to step S201.

  In the flowchart of FIG. 2, the process is terminated by powering off or a process termination interrupt.

  Next, details of an example algorithm for calculating the rating value in step S211 will be described using the flowchart of FIG.

  (Step S301) The rating value calculation unit 110 initializes a buffer into which the rating value is substituted.

  (Step S302) The rating value calculation unit 110 substitutes 1 for the counter i. The counter i is a counter indicating an address (array suffix) of a buffer (for example, an array) into which a rating value is substituted.

  (Step S303) The rating value calculation unit 110 reads out all acoustic models from the teacher data storage unit 102 and arranges them on the memory.

  (Step S304) The rating value calculation unit 110 assigns 1 to the counter n.

  (Step S305) The rating value calculation unit 110 determines whether or not the nth optimum phoneme subsequence exists. If the nth optimum phoneme subsequence exists, the process goes to step S306, and if the nth optimum phoneme subsequence does not exist, the process goes to step S318.

  (Step S306) The rating value calculation unit 110 acquires the nth optimal phoneme subsequence and arranges it on the memory.

  (Step S307) The rating value calculation unit 110 acquires the n-th feature vector partial series and places it on the memory.

  (Step S308) The rating value calculation unit 110 calculates the probabilities for each time of the feature vector partial series and for each state of all acoustic models using a forward-backward algorithm using all acoustic model parameters, and stores them in the buffer. .

  (Step S309) The rating value calculation unit 110 substitutes 1 for the counter t.

(Step S310) The rating value calculation unit 110 determines whether or not “t <= T ⁽ⁿ⁾ ” is satisfied, that is, whether or not the t-th frame to be evaluated exists. If the t-th frame exists, the process goes to step S311. If the t-th frame does not exist, the process goes to step S317.

  (Step S311) The rating value calculation unit 110 calculates the sum of the state probabilities of the t-th frame and places it on the memory.

(Step S312) The rating value calculation unit 110 acquires the state (j) included in the optimal phoneme “P _t ^* ”.

  (Step S313) The rating value calculation unit 110 calculates the posterior probability of the state (j). More specifically, the rating value calculation unit 110 calculates the posterior probability of the state (j) by “the probability of the state (j) / the sum of the state probabilities”.

  (Step S314) The rating value calculation unit 110 adds the posterior probability calculated in step S313 to the i-th rating value, and substitutes it into the i-th rating value buffer.

(Step S315) The rating value calculation unit 110 determines whether or not the next state exists in the optimal phoneme “P _t ^* ”. If the next state exists, the process goes to step S312, and if the next state does not exist, the process goes to step S316.

  (Step S316) The rating value calculation unit 110 increments the counter t and the counter i by 1, and returns to step S310.

  (Step S317) The rating value calculation unit 110 increments the counter n by 1, and returns to step S305.

  (Step S318) The rating value calculation unit 110 calculates the rating values of the phoneme section, the word section, and the sentence section from the rating value for each frame, and arranges the calculated values on the memory. As a method for calculating the rating values of the phoneme section, the word section, and the sentence section, there are a method of calculating an average value and a median value in each section of the rating values for each frame. Return to upper process.

  As described above, according to the present embodiment, it is possible to perform pronunciation evaluation with high accuracy. Specifically, according to the present embodiment, it is possible to accurately evaluate phonemes.

  In addition, the result of the evaluation experiment of the pronunciation rating device in the present embodiment will be described later together with the result of the evaluation experiment of another pronunciation rating device.

Further, according to the specific example of the present embodiment, the rating value calculation unit 110 calculates the posterior probability value according to Equations 1 and 2. However, the algorithm by which the rating value calculation unit 110 calculates the rating value is not always based on Equations 1 and 2. For example, in the calculation of the posterior probability of Formula 1, all the possible events as the state of the acoustic model may be only those that represent phonemes of vowels. In such a case, it is clear that the acoustic model parameter “λ ^all ” used for obtaining the probability of Equation 2 is also only the one representing the vowel phoneme, and only the vowel representing the vowel is allowed to be evaluated. is there. That is, the pronunciation rating device in the present embodiment is based on the posterior probability that the phoneme segment speech data is in the state to be evaluated for each time of the phoneme segment speech data under the condition that all acoustic model parameters are given. The phoneme rating value can be calculated.

  In the present embodiment, the speech data to be evaluated may be a set of phoneme interval speech data which is speech data previously divided into phoneme intervals. In such a case, in the pronunciation rating device, the optimal state sequence acquisition unit and the optimal phoneme partial sequence acquisition unit are not necessary. In such a case, the pronunciation rating device includes an optimal phoneme partial sequence storage unit that stores one or more optimal phoneme partial sequences that are a set of one or more optimal phoneme sequences in which the same phoneme is continuous.

  In the present embodiment, it is preferable that the pronunciation rating device calculates a rating value of a sentence or a word based on a rating value for each frame. Specifically, it is preferable that the pronunciation rating device calculates a rating value from an average value or a median value of a plurality of rating values for each frame included in a sentence or a word.

  Furthermore, the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded and distributed on a recording medium such as a CD-ROM. This also applies to other embodiments in this specification. Note that the software that realizes the pronunciation rating device in the present embodiment is the following program. In other words, this program divides a computer into a voice reception unit that receives voice input and a voice received by the voice reception unit into frames, and one or more frame voice data that is voice data for each of the divided frames. A frame audio data acquisition unit, a feature vector sequence acquisition unit that acquires a feature vector sequence that is a set of feature vectors for each frame from the one or more frame audio data, and a feature vector acquired by the feature vector sequence acquisition unit A feature vector partial sequence acquisition unit for acquiring one or more feature vector partial sequences, which are a set of one or more feature vectors corresponding to each stored optimal phoneme subsequence, and stored teacher data from the sequence; The feature vector partial sequence acquired by the feature vector partial sequence acquisition unit using the teacher data A program for calculating a posteriori probability that is a phoneme to be rated, calculating a speech rating value from the posteriori probability, and an output unit for outputting the rating value calculated by the rating value calculator .

  Further, in the above program, the acoustic model along the evaluation target phoneme sequence stored in the computer is read, and the acoustic model and the feature vector sequence acquired by the feature vector sequence acquisition unit correspond to the acoustic model. An optimal state sequence acquisition unit that compares a feature vector sequence and acquires an optimal state sequence that is a set of optimal states for each frame, and among the optimal state sequences acquired by the optimal state sequence acquisition unit, the same phoneme The feature vector partial sequence further functions as an optimal phoneme partial sequence acquisition unit that identifies one or more continuous optimal phoneme sequences and acquires one or more optimal phoneme partial sequences that are a set of the one or more optimal phoneme sequences. The acquisition unit obtains each optimum phoneme acquired by the optimal phoneme subsequence acquisition unit from the feature vector sequence acquired by the feature vector sequence acquisition unit. Program for operating such a set feature vectors subsequences is one or more feature vectors corresponding to the partial sequence to obtain one or more sets that, is is preferred.

  Further, in the above program, the phoneme segment speech data is evaluated for each time of the phoneme segment speech data under the condition that the evaluation value calculation unit is provided with teacher data that is all acoustic models. It is preferable to be a program for calculating a posterior probability that is a state of an acoustic model corresponding to a correct phoneme and functioning to calculate a rating value of the phoneme from the posterior probability.

Further, the teacher data is a hidden Markov model (HMM), and is an acoustic model connected from the terminal state of one phoneme HMM to the starting state of the one phoneme or all other phonemes. Is preferred.
(Embodiment 2)

  In the present embodiment, a pronunciation rating apparatus that performs pronunciation rating using an algorithm obtained by improving the PDAPS-PE algorithm described in the first embodiment in terms of phoneme evaluation will be described. This algorithm is an improvement of the PDAPS-PE algorithm, and is a modifier of its name, which removes the phoneme ergodic elements that were characteristic of the algorithm and more purely evaluates the target phoneme interval. Therefore, it is called PDAPS (phoneme unit dynamic A posteriori probability based pronunciation scoring).

In the PDAPS-PE algorithm described in the first embodiment, although the correct phonemes included in the speech data to be rated are clearly limited, they are the phonemes to be evaluated in the numerator of Equation 1. A feature vector string “O ⁽ⁿ⁾ ” representing the probability of a state is observed, and “λ ^all ” is used to calculate the probability that the state at time “t” is “j”, so that the phonemes to be evaluated Phoneme probability values other than were mixed. At this time, the probabilities of phonemes other than the phoneme to be obtained are also mixed in the values representing the probabilities of the states in the denominator of Equation 1. Here, the PDAPS algorithm is obtained by removing such mixed elements.

  FIG. 4 is a block diagram of the pronunciation rating device in the present embodiment. The pronunciation evaluation apparatus includes an input reception unit 101, a teacher data storage unit 102, a rating target phoneme sequence storage unit 103, a speech reception unit 104, a frame audio data acquisition unit 105, a feature vector sequence acquisition unit 106, and an optimum state sequence acquisition unit 107. , An optimal phoneme partial sequence acquisition unit 108, a feature vector partial sequence acquisition unit 109, a rating value calculation unit 410, and an output unit 111.

  This pronunciation evaluation apparatus differs from the pronunciation evaluation apparatus described in the first embodiment only in the evaluation value calculation unit.

The rating value calculation unit 410 calculates a phoneme rating value from phoneme interval data. The rating value calculation unit 410 calculates, for each time of phoneme segment speech data (speech data separated into phoneme segments), a posterior probability that the phoneme segment speech data is a correct phoneme to be evaluated, and the posterior probability The phoneme rating value is calculated from More specifically, the rating value calculation unit 410 uses, for example, the following Formula 3 and Formula 4, the optimal phoneme partial sequence acquired by the optimal phoneme partial sequence acquisition unit 108, and the feature vector corresponding to the partial sequence: The feature vector partial sequence “O ⁽ⁿ⁾ ” acquired by the partial sequence acquisition unit 109 and all the acoustic models are read from the teacher data storage unit 102, and each optimal phoneme “P _t ^* ” for each time of the optimal phoneme subsequence is read. In all states of all acoustic models, the feature vector series “O ⁽ⁿ⁾ ” is observed under the condition that the acoustic model parameter “λ ^{P (j)} ” to which the state “j” belongs is given, And the probability that the state at the time (frame) “t” of the vector series is “j” is acquired, and the sound for all phonemes (acoustic models) is obtained from the probability value for each acquired state. Under the condition that the model parameter "lambda ^{P (j)"} is given, and acquires the probabilities are observed feature vector sequence "O ^(n)", from the probability values for each the acquired phonemic feature vector series When “O ⁽ⁿ⁾ ” is given, the posterior probability that the vector series is the optimal phoneme “P _t ^* ” is acquired, and the speech rating value (PDPS (t)) is calculated.

In Equation 3, “λ ^{(Pt *)} ” is a parameter of the acoustic model of the phoneme “P _t ^* ”, and “λ ^{P (j)} ” is a parameter of the acoustic model including the state “j”. If the state “j” is a state of the acoustic model of the phoneme “P _t ^* ”, “λ ^{P (j)} ” and “λ ^{(Pt *)} ” represent parameters of the same acoustic model. “M” is the number of all acoustic models, and “N” is the total number of states of the entire acoustic model. In Formula 3, “Pr (m)” represents the prior probability of the phoneme “m”, and means the prior probability “Pr (λ (m))” of the acoustic model parameter “λ (m)” (“Pr ( The same applies to _Pt ^* )). In Formula 3, the phoneme prior probability “Pr (m)” is assumed to be constant for all phonemes, and is omitted in the middle of the formula. Equation 4 shows that the feature vector sequence “O ⁽ⁿ⁾ ” is observed under the condition that the acoustic model parameter “λ ^{P (j)} ” to which the state “j” belongs is given, and the time ( It shows that the probability that the state in the frame “t” is “j” is calculated by the forward-backward algorithm.

At this time, the numerator in each term of Equation 3 represents the probability that the feature vector series “O ⁽ⁿ⁾ ” is observed under the condition that the acoustic model parameter of the phoneme to be evaluated is given. Probabilities other than are removed. Further, in the denominator, the probability of representing each state is excluded from the mixture of probabilities other than the phoneme to which the state belongs. The rating value calculated by Equation 3 is that the input phoneme segment rating target speech data is speech data that is close (similar) to the phoneme model to be rated and is far from (not similar to) the other phonemes. If it is speech data, the value of the numerator and denominator approaches and increases (close to 1), and the value of the numerator and denominator moves away (numerator <denominator) and decreases when close to the phoneme model other than the evaluation target (Close to 0). Further, if it is far from any phoneme model, each probability value of the numerator and denominator is small and the evaluation value is small. In this way, the PDAPS algorithm can obtain high evaluation accuracy by evaluating the phoneme section of the evaluation target speech data more purely as the phoneme to be evaluated.

  The rating value calculation unit 410 can be usually realized by an MPU, a memory, or the like. The processing procedure of the rating value calculation unit 410 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the pronunciation rating device will be described. In the operation of this pronunciation rating device, the rating value calculation process is different from that of the pronunciation rating device of the first embodiment. Therefore, the rating value calculation process of the pronunciation rating device will be described with reference to the flowchart of FIG. In the flowchart of FIG. 5, only steps different from the flowchart of FIG. 3 will be described.

  (Step S501) The rating value calculation unit 410 calculates the probability for each state of all the acoustic models for each time of the feature vector partial series using the acoustic model parameter to which the state belongs, and stores it in the buffer. To do. Note that the rating value calculation unit 410 acquires the acoustic model parameter to which the state belongs from the acoustic model acquired in step S303 before calculating the probability.

  (Step S502) The rating value calculation unit 410 substitutes 1 for the counter m. Note that m is a phoneme counter.

  (Step S503) The rating value calculation unit 410 determines whether or not an mth phoneme acoustic model exists. If the mth phoneme acoustic model exists, the process proceeds to step S504. If the mth phoneme acoustic model does not exist, the process proceeds to step S506.

  (Step S504) The rating value calculation unit 410 calculates the probability of the mth phoneme from the sum of the probabilities at time t of all states included in the mth phoneme acoustic model.

  (Step S505) The rating value calculation unit 410 increments the counter m by 1, and returns to step S503.

(Step S506) The rating value calculation unit 410 calculates the posterior probability of the optimal phoneme “P _t ^* ”.

  (Step S507) The rating value calculation unit 410 stores the calculated posterior probability in the i-th rating value buffer.

  (Step S508) The rating value calculation unit 410 increments the counter t and the counter i by 1, and returns to step S310.

  As described above, according to the present embodiment, it is possible to perform pronunciation evaluation with high accuracy. Specifically, according to the present embodiment, it is possible to accurately evaluate phonemes.

  In addition, the result of the evaluation experiment of the pronunciation rating device in the present embodiment will be described later together with the result of the evaluation experiment of another pronunciation rating device.

  Further, according to the specific example of the present embodiment, the rating value calculation unit 410 calculates the above-described posterior probability value using Equations 3 and 4. However, the algorithm by which the rating value calculation unit 410 calculates the rating value is not always based on Formulas 3 and 4. For example, in the calculation of the posterior probability of Equation 3, all possible events as phonemes may be only those representing vowel phonemes. In such a case, it is clear that only the phonemes representing the vowels are allowed to be evaluated. That is, the rating value calculation unit 410 calculates the posterior probability that the phoneme segment speech data is the correct phoneme to be evaluated for each time of the phoneme segment speech data, and calculates the phoneme rating value from the posterior probability. Good.

  In the present embodiment, it is preferable that the pronunciation rating device calculates a rating value of a sentence or a word based on a rating value for each frame. Specifically, it is preferable that the pronunciation rating device calculates a rating value from an average value or a median value of a plurality of rating values for each frame included in a sentence or a word.

  Note that the software that realizes the pronunciation rating device in the present embodiment is the following program. In other words, this program divides a computer into a voice reception unit that receives voice input and a voice received by the voice reception unit into frames, and one or more frame voice data that is voice data for each of the divided frames. A frame audio data acquisition unit, a feature vector sequence acquisition unit that acquires a feature vector sequence that is a set of feature vectors for each frame from the one or more frame audio data, and a feature vector acquired by the feature vector sequence acquisition unit A feature vector partial sequence acquisition unit for acquiring one or more feature vector partial sequences, which are a set of one or more feature vectors corresponding to each stored optimal phoneme subsequence, and stored teacher data from the sequence; The feature vector partial sequence acquired by the feature vector partial sequence acquisition unit using the teacher data A program for calculating a posteriori probability that is a phoneme to be rated, calculating a speech rating value from the posteriori probability, and an output unit for outputting the rating value calculated by the rating value calculator .

  Further, in the above program, the acoustic model along the evaluation target phoneme sequence stored in the computer is read, and the acoustic model and the feature vector sequence acquired by the feature vector sequence acquisition unit correspond to the acoustic model. An optimal state sequence acquisition unit that compares a feature vector sequence and acquires an optimal state sequence that is a set of optimal states for each frame, and among the optimal state sequences acquired by the optimal state sequence acquisition unit, the same phoneme The feature vector partial sequence further functions as an optimal phoneme partial sequence acquisition unit that identifies one or more continuous optimal phoneme sequences and acquires one or more optimal phoneme partial sequences that are sets of the one or more optimal phoneme sequences. The acquisition unit is configured to acquire each optimal phoneme unit acquired by the optimal phoneme subsequence acquisition unit from the feature vector sequence acquired by the feature vector sequence acquisition unit. It is preferably a program, to function so as to obtain a feature vector subsequences is a set of one or more feature vectors corresponding to the sequence one or more sets.

Further, in the above program, the rating value calculation unit calculates a posterior probability that the phoneme segment speech data is a correct phoneme to be evaluated for each time of the phoneme segment speech data, and determines a phoneme rating value from the posterior probability. It is preferable that the program is a program for causing the function to be calculated.
(Embodiment 3)

  In the present embodiment, a pronunciation rating apparatus that performs pronunciation rating using an algorithm obtained by further improving the PDAPS algorithm described in the second embodiment will be described. The PDAPS algorithm is an algorithm that calculates a rating point for each phoneme segment obtained from the entire input speech data to be evaluated and its optimum state sequence, and removes phoneme ergodic elements to evaluate the phoneme segment purely. . The rating value calculation formula of the PDAPS algorithm includes a duplicate calculation from the meaning of processing efficiency that the rating value is the same at any time (frame) in the phoneme section of the rating target, and it is improved by removing the duplication This algorithm is the algorithm described in this embodiment.

  This algorithm has been improved by removing duplicate processing from the PDAPS algorithm, and since dynamic elements are no longer used directly in the calculation of rating values in the process of improvement, PAPPS (Phonemic A Posteriori Probability based pronunciation Scoring) This is called pronunciation evaluation based on posterior probabilities.

Hereinafter, the difference between the PDAPS algorithm described in the second embodiment and the PAPPS algorithm in the present embodiment will be described in more detail. The PDAPS algorithm is a value obtained by multiplying a forward probability as a calculated value of the forward algorithm and a backward probability as a calculated value of the backward algorithm as a result of the forward and backward algorithm shown in Equation 4 as a basis of the evaluation value calculation process. The occurrence probability of the feature vector sequence is obtained by adding together the co-occurrence probability that the state at time “t” is “j” with all the states of the given model parameter, Since the time and state variables are eliminated and the occurrence probability is the same value at any time, the numerator and denominator of Equation 3 have the feature vector series “O ^{(n) of the} evaluation target phoneme section. The value is the same at any time within “”, and the rating value is not different due to the difference in time. Further, when the probability at the final time “T ⁽ⁿ⁾ ” of the feature vector series is obtained using the forward and backward algorithm, the backward algorithm is not required and can be obtained only by the forward algorithm. PAPPS is an algorithm that simplifies the PDAPS algorithm by using these features and calculates exactly the same rating value at high speed. However, although the PDASP algorithm calculates the rating value for each time (frame) although it is the same value in the phoneme section to be evaluated, the PAPPS algorithm obtains the rating value for each phoneme section to be evaluated.

  FIG. 6 is a block diagram of the pronunciation rating device in the present embodiment. The pronunciation evaluation apparatus includes an input reception unit 101, a teacher data storage unit 102, a rating target phoneme sequence storage unit 103, a speech reception unit 104, a frame audio data acquisition unit 105, a feature vector sequence acquisition unit 106, and an optimum state sequence acquisition unit 107. An optimal phoneme partial sequence acquisition unit 108, a feature vector partial sequence acquisition unit 109, a rating value calculation unit 610, an output unit 111, and an optimal phoneme representative sequence acquisition unit 601.

The optimal phoneme representative sequence acquisition unit 601 acquires one representative phoneme representing a phoneme for each of the one or more sets of optimal phoneme subsequences acquired by the optimal phoneme subsequence acquisition unit 108, and acquires the acquired one or more phonemes An optimal phoneme representative sequence (P = {P ₁ , P ₂ ,..., P _n ,..., P _s }) having a representative phoneme is acquired, and the optimal phoneme representative sequence is arranged in a memory. The optimal phoneme representative sequence acquisition unit 601 includes, for example, three optimal phoneme subsequences {{a, a,..., A} {o, o,..., O} {i, i,. }}, An optimal phoneme representative sequence having three representative phonemes {a, o, i} is acquired for each optimal phoneme subsequence of {}. This process is a known character string process when there is a phoneme ID string or a character code string indicating a phoneme. The optimal phoneme representative sequence acquisition unit 601 can be usually realized by an MPU, a memory, or the like. The processing procedure of the optimum phoneme representative sequence acquisition unit 601 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

The rating value calculation unit 610 calculates a phoneme rating value from phoneme interval data. The rating value calculation unit 610 calculates a posterior probability that the phoneme segment speech data is a phoneme that is a correct answer to be evaluated, and calculates a phoneme rating value from the posterior probability. For each representative phoneme “P _n ” acquired by the optimal phoneme representative sequence acquisition unit, for example, the rating value calculation unit 610 acquires the feature vector partial sequence acquisition unit corresponding to the phoneme by the following formulas 5 and 6. The feature vector subsequence “O ⁽ⁿ⁾ ” and all acoustic models are read from the teacher data storage unit, and the acoustic model parameter “λ ^{P (j)} ” to which the state “j” belongs in all states of all acoustic models. The probability that the feature vector sequence “O ⁽ⁿ⁾ ” is observed and the state of the vector sequence at the final time (frame) “T ⁽ⁿ⁾ ” is “j” is obtained and, observation of the probability values for each the acquired state, under the condition that all of the phonemes acoustic model parameters "lambda ^{P (j)"} is given, the feature vector series "O ⁽ⁿ⁾ ' Get the probability that, from the probability values for each the obtained phonemes, when the feature vector sequence "O ^(n)" given that the vector sequence is to obtain the posterior probability is the representative phoneme "P _n" The voice rating value (PAPPS (P _n )) is calculated. It is preferable that the rating value calculation unit 610 obtains the probability value for each state by the forward algorithm without using the backward algorithm.

In Equation 5, “P _n ” is the “n” -th phoneme of the optimal phoneme representative sequence. In Equation 5, “λ ^(Pn) ” is an acoustic model parameter of the phoneme “P _n ”, and “λ ^{P (j)} ” is a parameter of the acoustic model including the state “j”. If the state “j” is the state of the acoustic model of the phoneme “P _n ”, “λ ^{P (j)} ” and “λ ^(Pn) ” represent parameters of the same acoustic model. “M” is the number of all acoustic models, and “N” is the total number of states of the entire acoustic model. In Formula 5, “Pr (m)” represents the prior probability of the phoneme “m” and means the prior probability “Pr (λ (m))” of the acoustic model parameter “λ (m)” (“Pr”). The same applies to “(P _n )”). In Formula 5, the phoneme prior probability “Pr (m)” is assumed to be constant for all phonemes, and is omitted in the middle of the formula.

In addition, Equation 6 shows that the feature vector series “O ⁽ⁿ⁾ ” is observed under the condition that the acoustic model parameter “λ ^(Pn) ” to which the state “j” belongs is given, and the final of the vector series It indicates that the probability that the state at time (frame) “T ⁽ⁿ⁾ ” is “j” is calculated by the forward algorithm. The PAPPS rating value calculation formula shown in Formula 5 matches the PDAPS rating value calculation formula shown in Formula 3 at the final time “T ⁽ⁿ⁾ ” of the feature vector series.

  Furthermore, the rating value calculation unit 610 temporarily stores the PAPPS rating value calculated for each phoneme section in the memory according to Equation 5, and uses the rating value for each phoneme section as a parameter, and the rating value for each sentence or word. May be calculated. In order to calculate a rating value for each sentence or word, the rating value calculation unit 610 can be obtained by a calculation method using an average value or a median value from PAPPS rating values of phonemes constituting the sentence or word to be rated. Is preferred. In such a case, the rating value calculation unit 610 weights the value of each phoneme using the length (number of frames) of the optimal phoneme subsequence of each phoneme acquired by the optimal phoneme subsequence acquisition unit 108 as a parameter, May be calculated, or the average value or median value of the rating values of each phoneme may be simply used as the rating value of a sentence or an entire word.

  The rating value calculation unit 610 can be usually realized by an MPU, a memory, or the like. The processing procedure of the rating value calculation unit 610 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the pronunciation rating device will be described. In the operation of the pronunciation rating device, the rating value calculation process and the pre-processing thereof are different from those of the pronunciation rating device of the second embodiment. Therefore, the rating value calculation process of the pronunciation rating device will be described with reference to the flowchart of FIG. In the flowchart of FIG. 7, only steps different from those in the flowcharts of FIGS. 3 and 5 will be described.

  (Step S701) The rating value calculation unit 610 determines whether the nth representative phoneme and the optimum phoneme subsequence are present. If the nth representative phoneme etc. exists, it will go to step S702, and if there is no nth representative phoneme etc., it will go to step S709.

  (Step S702) The rating value calculation unit 610 acquires the nth representative phoneme of the optimum phoneme representative sequence and places it on the memory.

  (Step S703) The rating value calculation unit 610 acquires the nth feature vector partial series and places it on the memory.

(Step S704) The rating value calculation unit 610 uses the acoustic model parameter to which the state belongs to determine the probability for each state of all acoustic models at the final time “T ⁽ⁿ⁾ ” of the feature vector partial series. Is calculated and stored in the buffer. Note that the rating value calculation unit 610 acquires the acoustic model parameter to which the state belongs from the acoustic model acquired in step S303 before calculating the probability.

(Step S705) The rating value calculation unit 610 calculates the probability of the mth phoneme from the sum of the probabilities at the time “T ⁽ⁿ⁾ ” of all states included in the mth phoneme acoustic model.

(Step S706) The rating value calculation unit 610 calculates the posterior probability of the representative phoneme “P _n ”.

  (Step S707) The rating value calculation unit 610 stores the calculated posterior probability in the buffer of the nth rating value.

  (Step S708) The rating value calculation unit 610 increments the counter n by 1, and returns to step S701.

  (Step S709) The rating value calculation unit 610 calculates the rating value of the word section and the sentence section from the rating value for each phoneme, and arranges the calculated value on the memory. As a method of calculating the rating value of the word section and the sentence section, there is a method of calculating an average value or a median value in each section of the rating value for each phoneme. Return to upper process.

  As described above, according to this embodiment, it is possible to perform pronunciation evaluation with high accuracy. Specifically, according to the present embodiment, the phoneme unit can be evaluated with high accuracy. Further, according to the present embodiment, it is possible to evaluate phonemes at high speed with high accuracy.

  Note that according to the present embodiment, the rating value calculation unit 610 calculates the rating value using Equations 5 and 6. However, the algorithm by which the rating value calculation unit 610 calculates the rating value is not necessarily based on the mathematical formulas 5 and 6. For example, Formula 6 was calculated by a forward algorithm. The rating value calculation unit 610 may calculate a backward algorithm instead of Equation 6. Further, for example, in the calculation of the posterior probability of Formula 5, all possible events as phonemes may be only those representing vowel phonemes. In such a case, it is clear that only the phonemes representing the vowels are allowed to be evaluated. The rating value calculation unit 610 may calculate a posterior probability that the phoneme segment speech data is a phoneme that is a correct answer to be evaluated, and calculate a phoneme rating value from the posterior probability.

  In the present embodiment, it is preferable that the pronunciation rating device calculates a rating value of a sentence or a word based on a rating value for each phoneme. Specifically, it is preferable that the pronunciation rating device calculates a rating value from an average value or a median value of a plurality of rating values for each phoneme included in a sentence or a word.

  Note that the software that realizes the pronunciation rating device in the present embodiment is the following program. In other words, this program divides a computer into a voice reception unit that receives voice input and a voice received by the voice reception unit into frames, and one or more frame voice data that is voice data for each of the divided frames. A frame audio data acquisition unit, a feature vector sequence acquisition unit that acquires a feature vector sequence that is a set of feature vectors for each frame from the one or more frame audio data, and a feature vector acquired by the feature vector sequence acquisition unit A feature vector partial sequence acquisition unit for acquiring one or more feature vector partial sequences, which are a set of one or more feature vectors corresponding to each stored optimal phoneme subsequence, and stored teacher data from the sequence; The feature vector partial sequence acquired by the feature vector partial sequence acquisition unit using the teacher data A program for calculating a posteriori probability that is a phoneme to be rated, calculating a speech rating value from the posteriori probability, and an output unit for outputting the rating value calculated by the rating value calculator .

  Further, in the above program, the acoustic model along the evaluation target phoneme sequence stored in the computer is read, and the acoustic model and the feature vector sequence acquired by the feature vector sequence acquisition unit correspond to the acoustic model. An optimal state sequence acquisition unit that compares a feature vector sequence and acquires an optimal state sequence that is a set of optimal states for each frame, and among the optimal state sequences acquired by the optimal state sequence acquisition unit, the same phoneme The feature vector partial sequence further functions as an optimal phoneme partial sequence acquisition unit that identifies one or more continuous optimal phoneme sequences and acquires one or more optimal phoneme partial sequences that are sets of the one or more optimal phoneme sequences. The acquisition unit is configured to acquire each optimal phoneme unit acquired by the optimal phoneme subsequence acquisition unit from the feature vector sequence acquired by the feature vector sequence acquisition unit. It is preferably a program, to function so as to obtain a feature vector subsequences is a set of one or more feature vectors corresponding to the sequence one or more sets.

  Further, in the above program, the rating value calculation unit calculates a posterior probability that the phoneme segment speech data is a phoneme that is a correct answer to be evaluated, and functions to calculate a phoneme rating value from the posterior probability. A program is preferred.

In the above program, the rating value calculation unit is preferably a program for causing the posterior probability to be obtained by a forward algorithm without using a backward algorithm.
(Embodiment 4)

  In the present embodiment, a pronunciation evaluation apparatus that performs pronunciation evaluation of sentences and words by an algorithm that normalizes the PAPPS rating value obtained for each phoneme by the time length (number of frames) of the evaluation target phoneme will be described. This algorithm is referred to as PAPPS-FN (pronunciation based on posterior probabilities of phoneme normalized by frame normalization) by adding a modifier to PAPPS in the third embodiment.

  Obtaining phoneme rating values in the PAPPS-FN algorithm is exactly the same as in the PAPPS algorithm of the third embodiment.

  FIG. 8 is a block diagram of the pronunciation rating device in the present embodiment. The pronunciation evaluation apparatus includes an input reception unit 101, a teacher data storage unit 102, a rating target phoneme sequence storage unit 103, a speech reception unit 104, a frame audio data acquisition unit 105, a feature vector sequence acquisition unit 106, and an optimum state sequence acquisition unit 107. , An optimal phoneme subsequence acquisition unit 108, a feature vector partial sequence acquisition unit 109, a rating value calculation unit 810, an output unit 111, an optimal phoneme representative sequence acquisition unit 601, and a phoneme time information acquisition unit 801.

  The phoneme time information acquisition unit 801 acquires phoneme time information that is information regarding the time of each optimum phoneme subsequence. The phoneme time information is, for example, the number of frames of a phoneme constituting a word or a sentence, or time (usually calculated from the number of frames). More specifically, the phoneme time information is the number of elements constituting each partial sequence of the optimal phoneme partial sequence or the feature vector partial sequence. The phoneme time information acquisition unit 801 can be usually realized by an MPU, a memory, or the like. The processing procedure of the phoneme time information acquisition unit 801 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

For each representative phoneme “P _n ” acquired by the optimal phoneme representative sequence acquisition unit, for example, the rating vector calculation unit 810 uses the above-described Equations 5 and 6 to calculate the feature vector partial sequence acquisition unit corresponding to the phoneme. The acquired feature vector partial series “O ⁽ⁿ⁾ ” and all acoustic models are read from the teacher data storage unit, and the acoustic model parameter “λ ^{P (j)} to which the state“ j ”belongs in all states of all acoustic models. ⁾ “”, The probability that the feature vector sequence “O ⁽ⁿ⁾ ” is observed and the state of the vector sequence at the final time (frame) “T ⁽ⁿ⁾ ” is “j”. acquires, from the probability values for each the acquired state, under the condition that all of the phonemes acoustic model parameters "lambda ^{P (j)"} is given, the feature vector series "O ^(n)" is observed Gets the probabilities, from the probability values for each the obtained phonemes, when the feature vector sequence "O ^(n)" given that vector sequence obtains a posteriori probability, which is a representative phoneme "P _n", Using the one or more acquired posterior probabilities and the phoneme time information corresponding to each phoneme as parameters, for example, the speech rating value (PAPPS-FN (P)) is calculated by Equation 7 below.

The rating value calculation unit 810 obtains a phoneme rating value according to the above-described formulas 5 and 6. Next, the rating value calculation unit 810 calculates a rating value for each sentence or word using the following Equation 7. Formula 7 means “a weighted average of phoneme evaluation values weighted by phoneme time length”. In Equation 7, “p” is a phoneme series constituting a sentence or a word. “P _n ” is a phoneme. In Equation 7, the rating value is calculated using all phonemes of the rating target phoneme series, assuming that all speech data to be rated is one sentence or one word. When calculating the rating values of multiple sentences and multiple words from speech data, obtain the rating values of multiple sentences and multiple words by applying a calculation formula to each phoneme sequence that composes the sentence or word. be able to.

In Equation 7, “s” is the number of optimal phoneme subsequences (number of phonemes). “T ⁽ⁿ⁾ ” is the sequence length of the “n” th partial sequence.

  The rating value calculation unit 810 can be usually realized by an MPU, a memory, or the like. The processing procedure of the rating value calculation unit 810 is usually realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  Next, the operation of the pronunciation rating device will be described with reference to the flowchart of FIG. The first step (step S901) in the flowchart in FIG. 9 follows step S709 in the flowchart in FIG. That is, the flowchart of FIG. 9 includes all the steps of FIG.

  (Step S901) The rating value calculation unit 810 substitutes 1 for a counter n.

  (Step S902) The rating value calculation unit 810 determines whether the nth representative phoneme and the optimum phoneme subsequence are present. If the nth representative phoneme etc. exists, it will go to step S903, and if there is no nth representative phoneme etc., it will go to step S908.

  (Step S903) The rating value calculation unit 810 acquires the n-th optimum phoneme partial sequence length and arranges it on the memory. Note that the partial sequence length of the feature vector may be used instead of the optimal phoneme partial sequence length.

  (Step S904) The rating value calculation unit 810 acquires the nth phoneme rating value.

  (Step S905) The rating value calculation unit 810 multiplies the nth phoneme rating value acquired in step S904 by the nth optimal phoneme partial sequence length acquired in step S903, and adds the value to the numerator value.

  (Step S906) The rating value calculation unit 810 adds the n-th optimal phoneme partial sequence length to the denominator value.

  (Step S907) The rating value calculation unit 810 increments the counter n by 1, and returns to step S902.

  (Step S908) The rating value calculation unit 810 calculates a rating value based on “numerator value / denominator value”. Return to upper process.

  As described above, according to the present embodiment, it is possible to obtain a highly accurate evaluation result suitable for a unit to be rated such as a word or a sentence that the user wants to rate.

  According to the present embodiment, rating value calculation unit 810 calculates the rating value by Equation 7. However, the algorithm by which the rating value calculation unit 810 calculates the rating value is not always based on Equation 7. The rating value calculation unit 810 may calculate a voice rating value using the acquired rating value of one or more phonemes and phoneme time information corresponding to each phoneme as parameters.

  Furthermore, the software that realizes the pronunciation rating device in the present embodiment is the following program. In other words, this program divides a computer into a voice reception unit that receives voice input and a voice received by the voice reception unit into frames, and one or more frame voice data that is voice data for each of the divided frames. A frame audio data acquisition unit, a feature vector sequence acquisition unit that acquires a feature vector sequence that is a set of feature vectors for each frame from the one or more frame audio data, and a feature vector acquired by the feature vector sequence acquisition unit A feature vector partial sequence acquisition unit for acquiring one or more feature vector partial sequences, which are a set of one or more feature vectors corresponding to each stored optimal phoneme subsequence, and stored teacher data from the sequence; The feature vector partial sequence acquired by the feature vector partial sequence acquisition unit using the teacher data A program for calculating a posteriori probability that is a phoneme to be rated, calculating a speech rating value from the posteriori probability, and an output unit for outputting the rating value calculated by the rating value calculator .

  In the above program, the computer reads out the acoustic model along the stored phoneme series to be evaluated, and the acoustic model and the feature vector series acquired by the feature vector series acquisition unit, corresponding to the acoustic model And an optimal state sequence acquisition unit that acquires an optimal state sequence that is a set of optimal states for each frame, and the same phoneme in the optimal state sequence acquired by the optimal state sequence acquisition unit Are further functioned as an optimal phoneme subsequence acquisition unit that identifies one or more optimal phoneme sequences that are consecutive, and acquires one or more optimal phoneme subsequences that are a set of the one or more optimal phoneme sequences, The sequence acquisition unit is configured to obtain each optimum phoneme acquired by the optimal phoneme subsequence acquisition unit from the feature vector sequence acquired by the feature vector sequence acquisition unit. Program for operating such a set feature vectors subsequences is one or more feature vectors corresponding to the partial sequence to obtain one or more sets that, is is preferred.

  Further, in the above program, the computer is a program for causing the rating value calculation unit to function to calculate a rating value of a sentence or a word based on the rating value for each frame or phoneme. Is preferred.

  Further, in the above program, the computer is further caused to function as a phoneme time information acquisition unit that acquires phoneme time information that is information related to the time of each phoneme, and the rating value calculation unit evaluates the phoneme with the phoneme time length as a weight. It is preferable that the program is a program for calculating a weighted average of values and functioning to calculate a rating value of a sentence or a word from the weighted average.

  The results of experiments conducted on the pronunciation rating device according to the above embodiment will be described below. The following experiment was conducted on a pronunciation rating device that implements three algorithms: PDAPS-PE, PDAPS, and PAPPS.

(Experiment 1)
First, there are six Japanese women, five Japanese men, and two English native men who pronounce English sentences and words. The voice reception unit 104 receives the voices, and the pronunciation rating device that implements each algorithm Pronunciation evaluation was performed. The algorithms are tp-DAP (using the average value of all time intervals), tp-DAP-PM (average of phoneme time intervals by phoneme number), PDAPS-PE (all time intervals) This is an algorithm using average values), PDAPS (using average values for all time intervals), and PAPPS (average phoneme evaluation values). Then, the pronunciation rating device calculates and outputs the average value of the sentence rating value and the word rating value obtained by each algorithm. FIG. 10 is a set of tables showing the comparison results of the algorithms. Note that tp-DAP is a known algorithm described in Patent Document 1. The tp-DAP-PM is an average of the median values in the phoneme time interval of the rating values output for each frame by the tp-DAP algorithm by the number of phonemes. “PM” means “Phoneme Median”. Note that the PAPPS-FN algorithm was not included in the comparison control in FIG. 10 and the like because the rating value to be output was exactly the same as the rating value of sentences / words using the PDAPS algorithm.

  FIG. 10A shows an average value for all speakers. FIG.10 (b) shows the average value in a Native speaker. FIG.10 (c) shows the average value in a Japanese speaker. FIG. 11A shows an average value in the task 1. FIG. 11B shows an average value in the task 2. 10 to 13, the value indicated by “%” indicates the ratio of the rating value calculated by the compared algorithm.

  From FIG. 10 and FIG. 11, in all cases, the results of t−p−DAP−PM <t−p−DAP <PDAPS−PE <PAPPS <PDAPS were obtained. The task represents the type of sentence or word that the speaker is pronounced. That is, task 1 and task 2 are different sentences and words.

  Among the results shown in FIGS. 10 and 11, the magnitude relationship regarding the average value among the Native speakers is particularly important. Since it is originally a teacher, it is desirable for the pronunciation rating device to output a perfect score for the pronunciation of Native speakers. However, in reality, if the model speaker that is the teacher of the pronunciation rating device is different from the speaker to be rated, it is difficult to always output a perfect score due to differences in speaker characteristics. Furthermore, it is difficult to always output a perfect score due to the difference between the recording environment of the model speaker's pronunciation (location or microphone) and the recording environment of the pronunciation to be rated. However, when evaluating the same speech to be evaluated using the same model speaker, the difference is only due to the algorithm. In this case, an algorithm that can output a higher score for the pronunciation of the Native speaker is more desirable. It can be said. For these reasons, it can be said that the PDAPS-PE, PDAPS, PAPPS, and PAPPS-FN algorithms invented this time are better pronunciation rating algorithms than the DAP and p-DAP algorithms described in Patent Document 1. .

(Experiment 2)
In Experiment 1, it was clarified that the PDAPS-PE, PDAPS, PAPPS, and PAPPS-FN algorithms output favorable values from the comparison results of the rating values for Native speakers. However, in the experiment, the number of Native speakers is two, so it cannot be said that the amount of data was sufficient. As a subsequent experiment, we used a speech database of English native speakers who pronounced English sentences and words, and compared the average values of sentences and word ratings obtained with the tp-DAP, PDAPS-PE, PDAPS, and PAPPS algorithms. Went. The voice database is a database in which 221 people, both men and women, input a total of 193427 voices, and the voice receiving unit 104 receives the voice data and stores it in the voice database (storage medium). Then, the pronunciation rating device reads voice data from the voice database and performs pronunciation rating. The experimental results are shown in FIG.

  According to FIG. 12, it is possible to obtain a result of tp-DAP <PDAPS-PE <PAPPS <PDAPS with an average rating value of all pronunciations. It was confirmed that the evaluation value for Native speakers was better than that of the p-DAP algorithm.

  In Experiment 2, the processing time required to evaluate all pronunciation data was measured, and a result of t-p-DAP> PDAPS-PE> PDAPS> PAPPS could be obtained.

(Experiment 3)
What has been confirmed in Experiment 1 and Experiment 2 is whether a higher score can be output for correct pronunciation. However, that alone does not completely satisfy the properties of a good pronunciation rating algorithm. This is because there is a possibility that a high score will be output even if the pronunciation is wrong, for example, even if the pronunciation has a content that is completely incomprehensible as a language.

  Therefore, in Experiment 3, a large number of Japanese speakers pronounced English sentences and words that were not completely correct, and the pronunciation rating device accepted the speech and performed pronunciation rating. FIG. 13A shows a comparison of the average values of sentences and word rating values obtained by t-p-DAP, PDAPS-PE, PDAPS, and PAPPS algorithms.

  FIG. 13B shows a correlation coefficient between the human rating score and the score calculated by the pronunciation rating algorithm. By examining the correlation coefficient, it is possible to confirm that an algorithm that measures the proximity to human senses and outputs a better rating value as the correlation is higher. The human rating score is a pronunciation rating value manually scored.

  As a result of Experiment 3, the new algorithm (PDAPS-PE, PDAPS, PAPPS) scored higher in the same way as Experiments 1 and 2 in terms of the average value. Looking at the correlation coefficient with the human rating score (FIG. 13B), the new algorithm shows a higher correlation than the p-DAP algorithm used so far, and is a better pronunciation rating algorithm. I was able to confirm. Furthermore, in the relationship between the new algorithms for the correlation coefficient, we were able to see a relationship different from the average value relationship. In terms of average value, the PDAPS algorithm output the highest value, but the PAPPS algorithm obtained the highest value in the correlation coefficient. This indicates that the PAPPS algorithm calculates the rating value more accurately even for wrong (completely incorrect) pronunciation, and is an algorithm that can calculate a rating value that is close to the human sense. It can be said.

  From the above experimental results, it was shown that the pronunciation rating algorithms PDAPS-PE, PDAPS, PAPPS, and PAPPS-FN described in this specification are superior to the known p-DAP algorithm.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  FIG. 14 shows the external appearance of a computer that executes the program described in this specification to realize the pronunciation rating device of the various embodiments described above. The above-described embodiments can be realized by computer hardware and a computer program executed thereon. FIG. 14 is an overview diagram of the computer system 340, and FIG. 15 is a block diagram of the computer system 340.

  In FIG. 14, a computer system 340 includes a computer 341 including an FD (Flexible Disk) drive and a CD-ROM (Compact Disk Read Only Memory) drive, a keyboard 342, a mouse 343, a monitor 344, and a microphone 345. .

  In FIG. 15, in addition to the FD drive 3411 and the CD-ROM drive 3412, a computer 341 includes a CPU (Central Processing Unit) 3413, a bus 3414 connected to the CD-ROM drive 3412 and the FD drive 3411, and a boot-up program. ROM (Read-Only Memory) 3415 for storing programs such as a RAM, and a RAM (Random Access Memory) 3416 connected to the CPU 3413 for temporarily storing application program instructions and providing a temporary storage space , An application program, a system program, and a hard disk 3417 for storing data. Although not shown here, the computer 341 may further include a network card that provides connection to the LAN.

  A program that causes the computer system 340 to execute the functions of the pronunciation rating device of the above-described embodiment is stored in the CD-ROM 3501 or the FD 3502, inserted into the CD-ROM drive 3412 or the FD drive 3411, and further stored in the hard disk 3417. May be forwarded. Alternatively, the program may be transmitted to the computer 341 via a network (not shown) and stored in the hard disk 3417. The program is loaded into the RAM 3416 at the time of execution. The program may be loaded directly from the CD-ROM 3501, the FD 3502, or the network.

  The program does not necessarily include an operating system (OS), a third-party program, or the like that causes the computer 341 to execute the function of the pronunciation rating device of the above-described embodiment. The program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 340 operates is well known and will not be described in detail.

  Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that two or more communication means (such as an information transmission unit) existing in one apparatus may be physically realized by one medium.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the pronunciation rating device according to the present invention has an effect that the pronunciation rating can be performed with high accuracy, and is useful as a language learning support device or the like.

Block diagram of the pronunciation rating device in the first embodiment Flow chart explaining the operation of the pronunciation rating device Flow chart explaining the rating value calculation process Block diagram of the pronunciation rating device in the second embodiment A flowchart for explaining the operation of the rating value calculation process Block diagram of the pronunciation rating device in the third embodiment A flowchart for explaining the operation of the rating value calculation process Block diagram of the pronunciation rating device in the fourth embodiment Flow chart explaining the operation of the pronunciation rating device Figure showing a table of the experimental results Figure showing a table of the experimental results Figure showing a table of the experimental results Figure showing a table of the experimental results External view of a computer that realizes the same pronunciation rating device Block diagram of a computer system that realizes the same pronunciation rating device

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Input reception part 102 Teacher data storage part 103 Evaluation object phoneme series storage part 104 Voice reception part 105 Frame audio | voice data acquisition part 106 Feature vector series acquisition part 107 Optimal state series acquisition part 108 Optimal phoneme partial series acquisition part 109 Feature vector partial series Acquisition unit 110, 410, 610, 810 Rating value calculation unit 111 Output unit 601 Phoneme representative sequence acquisition unit 801 Phoneme time information acquisition unit

Claims (9)

A teacher data storage unit that stores one or more teacher data that is an acoustic model of one or more phonemes;
An optimal phoneme subsequence storage unit that stores one or more optimal phoneme subsequences that are sets of one or more optimal phoneme sequences that are sequences of two or more phonemes to be rated, and the same phonemes are continuous;
A voice reception unit for receiving voice input;
The voice received by the voice receiving unit is divided into frames, and a frame voice data acquisition unit that obtains one or more frame voice data that is voice data for each divided frame;
A feature vector sequence acquisition unit that acquires a feature vector sequence that is a set of feature vectors for each frame from the one or more frame audio data;
A feature that acquires one or more sets of feature vector partial sequences that are sets of one or more feature vectors corresponding to each optimum phoneme partial sequence in the optimum phoneme partial sequence storage unit from the feature vector sequence acquired by the feature vector sequence acquisition unit A vector subsequence acquisition unit;
The teacher data is read from the teacher data storage unit, and using the teacher data, a posterior probability that the feature vector partial sequence acquired by the feature vector partial sequence acquisition unit is a phoneme to be evaluated is calculated. A rating value calculation unit for calculating the rating value of
A pronunciation rating device including an output unit that outputs a rating value calculated by the rating value calculation unit. A rating-target phoneme sequence storage unit that stores a rating-target phoneme sequence that is information on a sequence of two or more phonemes to be rated;
An acoustic model along the evaluation target phoneme sequence is read from the teacher data storage unit, and the acoustic model is a feature vector sequence acquired by the feature vector sequence acquisition unit, and a feature vector sequence corresponding to the acoustic model. An optimal state sequence acquisition unit that compares and acquires an optimal state sequence that is a set of optimal states for each frame;
Among the optimum state sequences acquired by the optimum state sequence acquisition unit, one or more optimum phoneme sequences in which the same phoneme continues are identified, and one set of optimum phoneme subsequences that is a set of the one or more optimum phoneme sequences Further comprising an optimal phoneme subsequence acquisition unit for acquiring the above,
The feature vector subsequence acquisition unit
One or more feature vector partial sequences that are sets of one or more feature vectors corresponding to each optimum phoneme partial sequence acquired by the optimum phoneme partial sequence acquisition unit are acquired from the feature vector sequence acquired by the feature vector sequence acquisition unit. The pronunciation rating device according to claim 1. The rating value calculation unit
Under the condition that all acoustic models are provided as teacher data, the phoneme segment speech data is in the state of the acoustic model corresponding to the correct phoneme to be evaluated for each time of the phoneme segment speech data. 3. The pronunciation rating apparatus according to claim 1, wherein a certain posterior probability is calculated, and a phoneme rating value is calculated from the posterior probability. The rating value calculation unit
3. The phoneme section speech data is calculated for each time of the phoneme section speech data, a posterior probability that the phoneme section speech data is a correct phoneme to be evaluated, and a phoneme rating value is calculated from the posterior probability. Pronunciation rating device. The rating value calculation unit
The phonetic rating device according to claim 1 or 2, wherein the phoneme interval speech data calculates a posterior probability that is a correct phoneme to be evaluated, and calculates a phoneme rating value from the posterior probability. The rating value calculation unit
6. The pronunciation rating device according to claim 1, wherein a rating value of a sentence or a word is calculated based on the rating value for each frame or phoneme. Further comprising a phoneme time information acquisition unit for acquiring phoneme time information that is information about the time of each phoneme;
The rating value calculation unit
7. The pronunciation rating apparatus according to claim 6, wherein a weighted average of phoneme rating values with a phoneme time length as a weight is calculated, and a rating value of a sentence or a word is calculated from the weighted average. On the recording medium,
One or more teacher data that is an acoustic model for one or more phonemes;
One or more optimal phoneme subsequences that are sets of one or more optimal phoneme sequences in which two or more phonemes to be rated are arranged and the same phoneme continues are stored.
Computer
A voice reception unit for receiving voice input;
The voice received by the voice receiving unit is divided into frames, and a frame voice data acquisition unit that obtains one or more frame voice data that is voice data for each divided frame;
A feature vector sequence acquisition unit that acquires a feature vector sequence that is a set of feature vectors for each frame from the one or more frame audio data;
A feature that acquires one or more sets of feature vector partial sequences that are sets of one or more feature vectors corresponding to each optimum phoneme partial sequence stored in the recording medium from the feature vector sequence acquired by the feature vector sequence acquisition unit. A vector subsequence acquisition unit;
Reading the teacher data stored in the recording medium, and using the teacher data, calculate a posterior probability that the feature vector partial sequence acquired by the feature vector partial sequence acquisition unit is a phoneme to be evaluated, and the posterior probability A rating value calculation unit for calculating a rating value of voice from
A program for functioning as an output unit that outputs a rating value calculated by the rating value calculation unit. In the storage medium,
It further stores a phoneme series to be graded, which is information on the arrangement of two or more phonemes to be graded,
Computer
An acoustic model along the evaluation target phoneme sequence stored in the storage medium is read, and the acoustic model is a feature vector sequence acquired by the feature vector sequence acquisition unit, and a feature vector sequence corresponding to the acoustic model And obtaining an optimum state sequence that is a set of optimum states for each frame,
Among the optimum state sequences acquired by the optimum state sequence acquisition unit, one or more optimum phoneme sequences in which the same phoneme continues are identified, and one set of optimum phoneme subsequences that is a set of the one or more optimum phoneme sequences Further function as an optimal phoneme subsequence acquisition unit to acquire the above,
The feature vector subsequence acquisition unit
One or more feature vector partial sequences that are sets of one or more feature vectors corresponding to each optimum phoneme partial sequence acquired by the optimum phoneme partial sequence acquisition unit are acquired from the feature vector sequence acquired by the feature vector sequence acquisition unit. 9. The program according to claim 8, wherein the program is made to function as follows.

Images