JP5042485B2 - Voice feature amount calculation device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make voice intonation of a learner closer to the voice intonation of a model, in language learning. <P>SOLUTION: A language learning device regenerates model voice when an example sentence is pronounced, when one of the displayed example sentences is selected. Then, a first pitch curve for showing time change of a pitch of the model voice is generated and a route of the first pitch curve is calculated. When regeneration of the model voice is finished, the language learning device requests the learner to pronounce the example sentence. The language learning device 1 stores input voice when the voice of the learner is input. The language learning device 1 generates a second pitch curve for showing time change of the stored voice of the learner and calculates a route of the second pitch curve. Thereafter, the language learning device compares the route of the first pitch curve with the route of the second pitch curve, and according to difference of the route, a difference point between the pitch of the model voice and the pitch of the learner voice is output to the learner. <P>COPYRIGHT: (C)2007,JPO&amp;INPIT

Description

  The present invention relates to a technique for comparing and evaluating a model inflection and a learner's pronunciation inflection.

In language learning, when practicing pronunciation, a learning method is widely used in which a model voice recorded on a recording medium such as a CD (Compact Disk) is reproduced and pronounced by imitating the model voice. . The purpose of this is to acquire correct pronunciation by imitating model voices. In this kind of learning, the learner understands his / her pronunciation, recognizes the difference from the model voice and improves the pronunciation, but whether or not the same pronunciation as the model voice can be made. It is difficult for learners to listen to their own pronunciation and grasp it objectively.
Thus, for example, as disclosed in Patent Document 1, a technique has been devised that enables a learner to objectively grasp his / her voice. The language learning device disclosed in Patent Literature 1 extracts speech information such as intonation from a speech that serves as an example of pronunciation and a learner's speech. Then, the similarity between the model voice and the learner's voice is obtained, the learner's pronunciation is scored based on the similarity, and the score such as inflection is displayed on the display. According to the technique disclosed in Patent Document 1, the learner's pronunciation is objectively evaluated and the evaluation result is displayed, so that the learner knows whether or not his / her confidence pronunciation is close to a model. Can do.
JP 2000-347560 A

  Patent Document 1 discloses a method for judging the similarity of intonation by superimposing a curve showing inflection of a model voice and a curve showing inflection of a learner's voice. It is possible to know whether the pronunciation is close to a model. However, it is only possible to know whether or not they are similar only by the degree of similarity, and it is not possible to know how to approximate the pronunciation of the model. For this reason, until a pronunciation that matches the pronunciation of the model can be achieved, it is necessary to carry out a persistent learning that repeats improvement and evaluation of pronunciation by trial and error.

  The present invention has been made under the above-described background, and an object of the present invention is to provide a technique for allowing the learner's voice inflection to approach that of the model voice in language learning.

Further, the present invention provides a storage means for storing an example sentence and model voice data indicating a model voice when the example sentence is spoken, and reads the model voice data from the storage means, and the voice represented by the read model voice data Generating a first pitch curve indicating a temporal change in the pitch of the first, a first calculating means for calculating a path of the generated first pitch curve, a voice input means for inputting a learner's uttered voice, and the voice input Recording speech inputted to the means, correcting means for correcting the utterance time of the recorded utterance voice to the same time as the utterance time of the model voice, and time of pitch of the utterance voice corrected by the correction means A second calculation unit that generates a second pitch curve indicating a change, calculates a path of the generated second pitch curve, a path calculated by the first calculation unit, and a path calculated by the second calculation unit Of determining the difference in intonation of the speech and the model voice in accordance with the difference, as difference from the pitch of the pitch and the model voice of the speech input to the speech input means, said the determined intonation Provided is an audio feature amount calculating apparatus having output means for outputting a difference determination result .

  According to the present invention, in language learning, a learner can approximate the inflection of speech to the inflection of a model speech.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
[Configuration of the embodiment]
FIG. 1 is a block diagram illustrating a hardware configuration of a language learning device 1 according to an embodiment of the invention. As shown in FIG. 1, each unit of the language learning device 1 is connected to a bus 101, and signals and data are exchanged between the units via the bus 101.

  The microphone 109 is connected to the sound processing unit 108, converts input sound into an analog electrical signal (hereinafter referred to as a sound signal), and outputs the analog signal to the sound processing unit 108. The speaker 110 is connected to the sound processing unit 108 and outputs a sound corresponding to the signal output from the sound processing unit 108. The audio processing unit 108 converts the audio signal input from the microphone 109 into digital data (hereinafter referred to as “learner data”) and outputs it, or converts the digital data representing the audio into an analog audio signal, and the speaker. The function to output to 110 is provided.

  The display unit 106 includes, for example, a display device such as a liquid crystal display, and displays a character string, various messages, a menu screen for operating the language learning device 1, and the like under the control of the CPU 102. The input unit 107 includes an input device such as a keyboard and a mouse (both not shown), and outputs a signal corresponding to the operation content to the CPU 102 in response to a key press or a mouse operation.

  The storage unit 105 includes an HDD (Hard Disk Drive) device that permanently stores data, and stores various data. Specifically, the storage unit 105 stores learner data output from the voice processing unit 108. In addition, the storage unit 105 includes example sentence text data representing example sentences used for language learning, and digital data (hereinafter referred to as model voice data) representing voices (hereinafter referred to as model voices) when the native speaker reads the example sentences. ) Is remembered. The storage unit 105 stores the example sentence table TB1 in the format illustrated in FIG. 2, and the table corresponds to the example sentence text data, the file name of the model voice data, and the identifier for uniquely identifying each example sentence text data. It is attached and stored.

  A CPU (Central Processing Unit) 102 executes a program stored in a ROM (Read Only Memory) 103 using a RAM (Random Access Memory) 104 as a work area. When the CPU 102 executes the program, each part is controlled by the CPU 102, the model voice is compared with the input learner's voice (hereinafter referred to as the learner voice), and the evaluation result of the inflection of the learner voice is output. Function is realized.

  FIG. 3 is a functional block diagram showing a configuration of functions realized by executing a program. The time axis correction unit 10 corrects the voice represented by the learner data so that the pronunciation time of the voice represented by the model voice data is the same as the pronunciation time of the voice represented by the learner data stored in the storage unit 105. To do. The pitch extraction unit 20 divides the sound indicated by the input data into a plurality of frames at predetermined time intervals on the reproduction time axis, and extracts the pitch of the sound of each frame for each divided frame. The pitch interpolating unit 30 performs linear interpolation or cubic processing between frames adjacent to these frames, such as unvoiced sections and unvoiced consonant pronunciation sections, for which the pitch extraction unit 20 could not extract the pitch. Interpolation such as spline interpolation is performed to determine the pitch of these frames. The pitch curve generation unit 40 generates a pitch curve by connecting the pitch obtained for each frame by the pitch extraction unit 20 and the pitch interpolated by the pitch interpolation unit 30. The journey calculation unit 50 calculates the journey of the pitch curve generated by the pitch curve generation unit 40.

[Operation of the embodiment]
Next, the operation of this embodiment will be described. First, when the learner performs an operation to instruct display of a list of example sentences, the CPU 102 reads out example sentence text data stored in the example sentence table TB1 (FIG. 4: step SA1), and displays a list of example sentences represented by the read data. The information is displayed on the display unit 106 (step SA2). Thereafter, when the learner operates the input unit 107 and performs an operation of selecting one of the displayed example sentences (step SA3; YES), the CPU 102 displays the screen displayed on the display unit 106, the input unit, and the like. Based on the signal sent from 107, the selected example sentence is specified (step SA4). When CPU 102 identifies the selected example sentence, CPU 102 reads out the file name of the model voice data stored in association with the selected example sentence in example sentence table TB1 (step SA5). For example, in the table shown in FIG. 2, when an example sentence with the identifier “001” is selected, the file name “a001” is read.

  Next, the CPU 102 reads out the model voice data specified by the read file name from the storage unit 105 and extracts the pitch of the voice indicated by the read data. Specifically, first, the CPU 102 divides the voice indicated by the model voice data at a predetermined time interval (for example, 5 msec) on the playback time axis as shown in FIG. 5 (step SA6) (hereinafter, referred to as “step SA6”). Each divided section is called a frame). Next, the CPU 102 extracts the audio pitch of each frame for each divided frame (step SA7). Note that the time interval of one frame may be other time intervals such as 10 msec instead of 5 msec. When the CPU 102 extracts the pitch for each divided frame, the CPU 102 generates a pitch curve connecting the pitches determined for each frame (hereinafter, this pitch curve is referred to as a first pitch curve), and the generated first pitch curve. Is stored in the storage unit 105 (step SA8). Note that, in a frame where the pitch cannot be extracted, such as an unvoiced section or an unvoiced consonant pronunciation section, interpolation such as linear interpolation or cubic spline interpolation is performed to generate a first pitch curve.

  When the generation of the first pitch curve is completed, the CPU 102 reads out the model voice data specified by the read file name from the storage unit 105, and outputs the read out model voice data to the voice processing unit 108 (step SA9). When the model voice data is input to the voice processing unit 108, the model voice data which is digital data is converted into an analog signal and output to the speaker 110, and the model voice is reproduced from the speaker 110.

  When the reproduction of the model voice is finished, the CPU 102 controls the display unit 106 to display, for example, a message for prompting pronunciation of the example sentence such as “Press the key to pronounce and then press the key again when the pronunciation is finished” ( Step SA10). The learner listens to the model voice output from the speaker 110, and then operates the input unit 107 according to the message to read the example sentence by imitating the model voice. When the learner pronounces, the learner's voice is converted into a voice signal by the microphone 109, and the converted signal is output to the voice processing unit 108. When the audio signal output from the microphone 109 is input, the audio processing unit 108 converts the audio signal into learner data that is digital data. This learner data is output from the voice processing unit 108 and stored in the storage unit 105.

  Next, the CPU 102 monitors a signal sent from the input unit 107 and determines whether or not the learner has finished pronunciation. When the learner finishes the pronunciation and operates the input unit 107 (step SA11; YES), the CPU 102 generates the pronunciation time of the voice represented by the model voice data and the pronunciation of the voice represented by the learner data stored in the storage unit 105. The voice represented by the learner data is corrected so that the time is the same (step SA12).

  Next, the CPU 102 divides the voice indicated by the learner data into a plurality of frames on the reproduction time axis in the same manner as the process of step SA6 (step SA13), and the voice of each frame is divided for each divided frame. The pitch is extracted (step SA14). When CPU 102 extracts the pitch for each frame, CPU 102 generates a pitch curve (hereinafter referred to as a second pitch curve) connecting the pitches determined for each frame in the same manner as in step SA8, and generates the generated second pitch curve. Is stored in the storage unit 105 (step SA15). In this case as well, in an unvoiced section, an unvoiced consonant pronunciation section, etc., interpolation such as linear interpolation or cubic spline interpolation is performed to generate a second pitch curve.

  When the generation of the second pitch curve is completed, the CPU 102 calculates the path of the first pitch curve, and then calculates the path of the second pitch curve (step SA16). The distance of the pitch curve can be obtained by calculating the integral of the absolute value of the first derivative of the pitch curve f (t) as shown in FIG. As shown in FIG. 8, since a voice with a large inflection has a large pitch change, a pitch curve has a long path, while a voice with a small inflection has a small pitch change. The journey is shortened. That is, it can be said that the distance of the pitch curve represents the magnitude of the inflection.

  Next, the CPU 102 compares the path of the first pitch curve with the path of the second pitch curve (step SA17). Since the pitch curve path represents the magnitude of the inflection change, by comparing the pitch curve paths, it is known which of the inflection changes is greater between the model voice inflection and the learner voice inflection. be able to. As shown in FIG. 8, when the path of the first pitch curve is longer than the path of the second pitch curve, the change amount of the inflection of the learner speech is smaller than the change amount of the inflection of the model speech. The message “It is a voice with little change” is displayed on the display unit 106, and information useful for improving the pronunciation is output to the learner (step SA18). Also, if the path of the second pitch curve is longer than the path of the first pitch curve, the amount of change in the inflection of the learner's voice is greater than the amount of change in the inflection of the model voice. Is displayed on the display unit 106, and information useful for improving pronunciation is output to the learner (step SA18). In addition, when the journey of the first pitch curve and the journey of the second pitch curve are the same, the change amount of the inflection of the learner's voice is the same as the change amount of the inflection of the model voice. Is displayed on the display unit 106 (step SA18).

  When the CPU 102 finishes outputting the message to the learner, the CPU 102 controls the display unit 106 to display a menu screen for confirming whether to practice pronunciation of the selected example sentence again or practice pronunciation of another example sentence ( Step SA19). When the learner operates the input unit 107 and performs an operation for practicing pronunciation of another example sentence (step SA20; YES), the CPU 102 returns the process flow to step SA1, and performs the processes after step SA1. Run again. When the learner operates the input unit 107 to perform the pronunciation practice of the selected example sentence again (step SA21; YES), the CPU 102 returns the process flow to step SA6, and after step SA6. The above process is executed again.

  As described above, according to the present embodiment, how to improve the pronunciation is output to the learner, so that the pronunciation is not repeated by repeating trial and error to improve and evaluate the pronunciation. Can be brought closer to the model voice.

[Second Embodiment]
[Configuration of the embodiment]
Next, a second embodiment of the present invention will be described. The hardware configuration of the language learning device 1A according to the present embodiment is the same as that of the language learning device 1 according to the first embodiment. For this reason, the description of the hardware configuration is omitted. In the present embodiment, the function realized by the CPU 102 executing the program is different from the first embodiment.

  FIG. 6 is a functional block diagram showing a configuration of functions realized by the CPU 102 executing the program. In FIG. 6, the same functional blocks as those in the first embodiment are denoted by the same reference numerals as those in FIG. 3, and the description thereof is omitted. The Hz → Cent conversion unit 60 converts the pitch expressed in Hz into Cent. The filter unit 70 functions as a low-pass filter and removes a minute change in pitch.

[Operation of the embodiment]
Next, the operation of this embodiment will be described. FIG. 7 is a flowchart illustrating the processing flow of the CPU 102 in the present embodiment. In FIG. 7, the same processes as those of the first embodiment are denoted by the same reference numerals as those of the first embodiment.

  When the learner selects an example sentence, the CPU 102 reads out model voice data corresponding to the selected example sentence from the storage unit 105 (steps SA1 to SA5). Then, the voice indicated by the model voice data is divided at predetermined time intervals on the reproduction time axis (step SA6). Next, the CPU 102 extracts the audio pitch of each frame for each divided frame (step SA7). When the CPU 102 extracts the pitch for each divided frame, it converts the unit of the extracted pitch from Hz to Cent (step SB1). When the conversion from Hz to Cent is completed, the CPU 102 removes minute changes in pitch such as fluctuations in pronunciation for each frame (step SB2). Then, the CPU 102 generates a pitch curve (first pitch curve) connecting the pitches determined for each frame, and stores the curve data indicating the generated first pitch curve in the storage unit 105 (step SA8). Note that, in a frame where the pitch cannot be extracted, such as an unvoiced section or an unvoiced consonant pronunciation section, interpolation such as linear interpolation or cubic spline interpolation is performed to generate a first pitch curve.

  Thereafter, the CPU 102 displays a message for prompting pronunciation of the example sentence (step SA10). When the learner reads the example sentence by imitating the model voice, the learner voice is converted into learner data. The CPU 102 monitors the signal sent from the input unit 107, and when the learner finishes pronunciation and operates the input unit 107 (step SA <b> 11; YES), the sound generation time of the voice represented by the model voice data and the storage unit 105. The voice represented by the learner data is corrected so that the pronunciation time of the voice represented by the stored learner data is the same (step SA12).

Next, the CPU 102 divides the voice indicated by the learner data into a plurality of frames on the reproduction time axis in the same manner as the process of step SA6 (step SA13). Then, the CPU 102 extracts the audio pitch of each frame for each divided frame (step SA14). When the CPU 102 extracts a pitch for each divided frame, the CPU 102 converts the extracted pitch unit from Hz to Cent (step SB3). When the conversion from Hz to Cent is completed, the CPU 102 removes minute changes in pitch such as fluctuations in pronunciation for each frame (step SB4). Then, the CPU 102 generates a pitch curve (second pitch curve) connecting the pitches obtained for each frame, and stores the curve data indicating the generated second pitch curve in the storage unit 105 (step SA15). Note that, in a frame where the pitch cannot be extracted, such as an unvoiced section or an unvoiced consonant pronunciation section, interpolation such as linear interpolation or cubic spline interpolation is performed to generate a second pitch curve.
Since the flow of processing after step SA15 is the same as that of the first embodiment, description thereof is omitted.

  As described above, this embodiment also outputs to the learner how to improve the pronunciation specifically, so that the pronunciation can be reproduced without repeating the improvement and evaluation of the pronunciation through trial and error. It is possible to approach the model voice. In addition, since the pitch is compared in units of Cent, an evaluation closer to human hearing is possible.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the present invention may be implemented by modifying the above-described embodiment as follows.

In the above-described embodiment, the process of step SA12, that is, the process of matching the pronunciation time of the model voice and the pronunciation time of the learner voice may not be performed. In the process of step SA12, the pronunciation time of each word in the model voice may be the same as the pronunciation time of each word in the learner voice. In the process of step SA12, the pronunciation time of each phoneme in the model voice may be the same as the pronunciation time of each phoneme in the learner voice.
Further, when comparing the distances of the first pitch curve and the second pitch curve, the time interval to be compared may be a whole example sentence section, a part of example sentence specified in advance, or a part of section specified by the user. It may be.

  In the embodiment described above, the path of the first pitch curve is compared with the path of the second pitch curve, and according to the comparison result, for example, “Let's pronounce with a little inflection” May be displayed. In the above-described embodiment, for example, a word in speech may be recognized, and an inflection evaluation result may be displayed for each word.

  In the above-described embodiment, a pitch curve may be generated only for the learner's voice to determine the pitch curve path, and the calculated path may be output.

It is the figure which showed the hardware constitutions of the language learning apparatus which concerns on embodiment of this invention. It is the figure which illustrated the format of example sentence table TB1. It is the figure which illustrated the structure of the functional block implement | achieved when CPU102 concerning 1st Embodiment runs a program. It is the flowchart which showed the flow of the process which CPU102 concerning 1st Embodiment performs. It is a figure for demonstrating the process of step SA6. It is the figure which illustrated the structure of the functional block implement | achieved when CPU102 concerning 2nd Embodiment runs a program. It is the flowchart which showed the flow of the process which CPU102 concerning 2nd Embodiment performs. It is the figure which illustrated the pitch curve of a learner's voice, and the pitch curve of a model voice.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Language learning apparatus, 101 ... Bus, 102 ... CPU, 103 ... ROM, 104 ... RAM, 105 ... Memory | storage part, 106 ... Display part, 107 ... Input unit, 108... Voice processing unit, 109... Microphone, 110.

Claims (1)

Storage means for storing an example sentence and model voice data indicating a model voice when the example sentence is uttered;
First calculation means for reading out the model voice data from the storage means, generating a first pitch curve indicating a temporal change in the pitch of the voice represented by the read model voice data, and calculating a path of the generated first pitch curve. When,
A voice input means for inputting the learner's speech;
Correcting means for recording the utterance voice input to the voice input means, and correcting the utterance time of the recorded utterance voice to the same time as the utterance time of the exemplary voice;
Second calculation means for generating a second pitch curve indicating a temporal change in the pitch of the speech voice corrected by the correction means, and calculating a path of the generated second pitch curve;
The difference between the model voice and the spoken voice is determined according to the difference between the journey calculated by the first calculation means and the journey calculated by the second calculation means, and input to the voice input means A speech feature quantity calculation apparatus comprising: output means for outputting a determination result of the determined inflection as a difference between the pitch of the uttered speech and the pitch of the exemplary speech.

Images