JP2017161632A - Pronunciation learning support system and pronunciation learning support method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a pronunciation learning support system having relatively simple structure while taking liaison learning or accent learning into consideration.SOLUTION: A pronunciation learning support system includes: a voice acquisition section 105 which acquires a voice of a word, a phrase, or a sentence by a learner; a feature extraction section which obtains feature quantity time series data of the acquired voice; a database 111 which stores a plurality of pattern feature quantity time series data previously prepared for the voice of the word, the phrase or the sentence; a selection section 109 which collates the feature quantity time series data of the acquired voice with the plurality of pattern feature quantity time series data, selects the pattern feature quantity time series data most similar to the feature quantity time series data of the acquired voice; and an output section for learner 117 which displays data corresponding to the selected pattern feature quantity time series data.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a pronunciation learning support system and a pronunciation learning support method used when a learner learns a language such as a foreign language.

  A pronunciation learning support system used when learning a language, which recognizes the pronunciation of a speaker from the voice data of the speaker who is the learner and the image of the mouth (video data), and correct pronunciation and wrong speaker A system has been developed for comparing and displaying images in the mouth of each of the pronunciations and feeding back to the learner (for example, Non-Patent Document 1). However, the above system is a large-scale system including an image analysis system. In addition, the above-mentioned system does not particularly take into consideration continuous tone learning or accent learning like a French liaison, which is difficult for foreign language learners.

  Therefore, there is a need for a pronunciation learning support system with a relatively simple configuration that takes into account learning of continuous sounds and accent learning, and a pronunciation learning support method using the same.

Olov Engwall et al., Designing the user interface of the computer-based speech training system ARTUR based on early user tests, Behavior & Information Technology, Vol. 25, No. 4, July-August 2006, 353-365

  SUMMARY OF THE INVENTION An object of the present invention is to provide a pronunciation learning support system having a relatively simple configuration in consideration of continuous tone learning and accent learning, and a pronunciation learning support method using the same.

  The pronunciation learning support method according to the first aspect of the present invention uses a pronunciation learning support system, and the pronunciation learning support system acquires a voice of a word, phrase or sentence by a learner, The pronunciation learning support system obtains the acquired feature time series data of the speech, and the pronunciation learning support system stores the acquired feature time series data of the speech and the pronunciation learning support system database Comparing the plurality of type feature quantity time-series data of the speech of the word, phrase or sentence, and selecting the type feature quantity time-series data most similar to the acquired voice feature quantity time-series data And the pronunciation learning support system displaying image data corresponding to the selected type feature quantity time-series data.

  In the pronunciation learning support method according to this aspect, the pronunciation learning support system is configured such that the word, phrase, or sentence stored in the database of the pronunciation learning support system is stored in the database of the pronunciation learning support system. A plurality of type feature quantity time-series data of sentence speech, and a type feature quantity time-series data that is most similar to the acquired voice feature quantity time-series data is selected. Compared with the case where no type feature quantity time-series data is used, the system configuration can be simplified.

  In addition, since the feature amount time series data of the learner's voice is associated with any type feature amount time series data, data corresponding to the type feature amount time series data created in advance is output to the learner. Can do.

  In the pronunciation learning support method according to the first embodiment of the first aspect of the present invention, the data corresponding to the selected type feature quantity time-series data includes a vocal tract image corresponding to speech.

  According to the present embodiment, since the vocal tract image corresponding to the learner's voice can be displayed to the learner, the learner's pronunciation learning effect is improved.

  In the pronunciation learning support method according to the second embodiment of the first aspect of the present invention, the data corresponding to the selected type feature amount time-series data includes moving image data.

  According to this embodiment, since moving image data is used, the learner's pronunciation learning effect is further improved.

  In the pronunciation learning support method according to the third embodiment of the first aspect of the present invention, the data corresponding to the selected type feature quantity time-series data includes stereoscopic image data.

  According to the present embodiment, since the stereoscopic image data is used, the learner's pronunciation learning effect is further improved.

  The pronunciation learning support method according to the fourth embodiment of the first aspect of the present invention further includes the step of obtaining the accent of the acquired speech, and the data corresponding to the selected type feature amount time-series data is: The acquired voice accent data is included.

  According to this embodiment, since the accent data of the learner's voice is used, the learner's accent learning effect is improved.

  In the pronunciation learning support method according to the fifth embodiment of the first aspect of the present invention, in the displaying step, in addition to the data corresponding to the selected type feature quantity time-series data, Output data corresponding to series data.

  According to this embodiment, in addition to the data corresponding to the selected type feature quantity time-series data, data corresponding to the standard voice feature quantity time-series data is output, so the learner compares the data of both And learning effect can be improved.

  In the pronunciation learning support method according to the sixth embodiment of the first aspect of the present invention, at least part of the data corresponding to the selected type feature quantity time series data, or the feature quantity time series data of the standard speech At least part of the data corresponding to is associated with phonemes.

  According to this embodiment, since data associated with phonemes is output, the learner's understanding deepens and the learning effect is improved.

  The pronunciation learning support device according to the second aspect of the present invention includes a voice acquisition unit that acquires a voice of a word, a phrase, or a sentence by a learner, a feature extraction unit that calculates feature amount time-series data of the acquired voice, A database that stores a plurality of type feature amount time series data prepared in advance for speech of words, phrases, or sentences, and a comparison between the acquired feature amount time series data of speech and the plurality of type feature amount time series data A selection unit that selects type feature amount time-series data that is most similar to the acquired feature amount time-series data, and a learner output that displays data corresponding to the selected type feature amount time-series data Part.

  The pronunciation learning support system according to this aspect includes a plurality of voice feature amount time-series data acquired from a word, phrase, or sentence, and a plurality of voices of the word, phrase, or sentence stored in the database of the pronunciation learning support system. The feature type time-series data is collated and selected so as to select the type-feature feature time-series data that is most similar to the acquired voice feature-value time-series data. Compared to the case where data is not used, the system configuration can be simplified.

  In addition, since the feature amount time series data of the learner's voice is associated with any type feature amount time series data, data corresponding to the type feature amount time series data created in advance is output to the learner. Can do.

  The pronunciation learning support system according to the first embodiment of the second aspect of the present invention further includes an accent analysis unit that obtains the accent of the acquired speech, and the output unit for learner includes the analyzed accent. The selected type feature amount time series data is output in association with the selected type feature amount time series data.

  According to the present embodiment, since the learner's accent is output in association with the selected type feature amount time-series data, the learner's understanding deepens and the learning effect is improved.

It is a figure which shows the structure of the pronunciation learning assistance system of one Embodiment of this invention. It is a flowchart for demonstrating operation | movement of the pronunciation learning assistance system. It is a figure which shows an example of a structure of the data memorize | stored in a database. It is a figure which shows the character of the standard data of the word "April", the phonetic symbol, the accent display, and the image of a vocal tract. It is a flowchart for demonstrating the usage method of the pronunciation learning assistance system of this invention.

  The pronunciation learning support system of the present invention is for assisting the learner in pronunciation of words, phrases or sentences when the learner learns a language such as a foreign language. The pronunciation learning support system obtains voices of words, phrases or sentences pronounced by the learner, analyzes them, and provides the learners with appropriate information for correcting pronunciation.

  FIG. 1 is a diagram showing a configuration of a pronunciation learning support system 100 according to an embodiment of the present invention. The pronunciation learning support system 100 includes a learner input unit 101 that receives an input such as a command from a learner, a control unit 103 that controls the operation of each unit, and a voice that acquires the voice of a learner's word, phrase, or sentence. An acquisition unit 105, a feature extraction unit 107 that extracts the feature amount of the acquired speech, a database 111 that stores data including a plurality of type feature amounts prepared in advance for speech of words, phrases, or sentences; The voice feature and the plurality of type feature values stored in the database 111 are collated, and the selection unit 109 that selects the type feature value most similar to the acquired voice feature value, and analyzes the acquired voice accent An accent analysis unit 113 that performs display, a display content creation unit 115 that creates display content to the learner, and a learner output unit 11 that performs display and audio output to the learner. And, equipped with a.

  FIG. 2 is a flowchart for explaining the operation of the pronunciation learning support system 100.

  In step S1010 of FIG. 2, the control unit 103 of the pronunciation learning support system 100 determines a learning target word, phrase, or sentence. The pronunciation learning support system 100 may be configured such that a learner selects a word, phrase, or sentence to be learned from a predetermined list and designates it from the learner input unit 101.

  In step S1020 of FIG. 2, the voice acquisition unit 105 of the pronunciation learning support system 100 acquires voice data of a learner's word, phrase, or sentence. The pronunciation learning support system 100 may be configured such that the learner output unit 117 outputs a voice or an image according to an instruction from the control unit 103 and prompts the learner to utter a specified word, phrase or sentence. Good. Subsequently, the control unit 103 instructs the voice acquisition unit 105 to acquire voice data of the learner's word, phrase, or sentence. For example, the voice acquisition unit 105 determines a voice section of a learner's word, phrase, or sentence using a power threshold, and acquires voice data of the word, phrase, or sentence.

  In step S1030 of FIG. 2, the feature extraction unit 107 of the pronunciation learning support system 100 receives the audio data acquired from the audio acquisition unit 105 according to the instruction of the control unit 103, and takes a time frame of 10 to 30 milliseconds as an example. Create time-series data of the spectral feature value of each pronunciation. A cepstrum can be used as the feature amount. The cepstrum is defined as the FFT of the logarithm of the power spectrum obtained by performing FFT processing on audio data. The cepstrum is described in, for example, JP-A-8-87292 and JP-A-9-305195. In general, the spectrum of a human voice is a product of a vocal cord vibration spectrum and a vocal tract spectrum. The pronunciation of a word, phrase or sentence is related to the vocal tract spectrum, and the loudness or height of the voice is related to the vocal cord vibration spectrum. By using the cepstrum, the characteristics of the vocal tract spectrum related to the pronunciation of words, phrases or sentences can be grasped. The vocal tract will be explained later. The feature extraction unit 107 creates a time series of cepstrum for each time frame as feature amount time series data of speech data of words, phrases or sentences.

  In step S1040 of FIG. 2, the selection unit 109 of the pronunciation learning support system 100 receives the feature amount time series data of the pronunciation by the learner from the feature extraction unit 107 according to the instruction of the control unit 103, and the feature of the pronunciation by the learner. The feature time series data and the feature amount time series data stored in the database 111 are collated, and the feature amount most similar to the feature amount time series data of the pronunciation by the learner from the stored feature amount time series data Select time series data. A method for collation will be described in detail later.

  FIG. 3 is a diagram illustrating an example of a configuration of data stored in the database 111. The database 109 stores standard data and a plurality of type data for each word, phrase or sentence to be learned. FIG. 3 shows only the word A and the word B. The number of type data of word A and word B is n. The standard data is, for example, data on standard voices of words. The type data is data about wrong speech of the same word. The standard data and the type data include data for each phoneme that is a unit of speech of language. The data for each phoneme includes a phoneme phonetic symbol, a time series of phoneme feature quantities, and an image of the vocal tract when the phoneme is uttered.

  Here, the vocal tract will be described. In general, the vocal tract consists of the larynx, pharynx, oral cavity, and nasal cavity. It is the shape and position of the tongue and lips of the oral cavity that have a particularly great influence on the pronunciation of words and the like. In this specification, the vocal tract refers to, in particular, the shape and position of the tongue and lips of the oral cavity, although not limited thereto. In general, since the relationship between the phoneme and the shape and position of the tongue and lips has been elucidated, an image of the vocal tract regarding the shape and position of the tongue and lips can be created for each phoneme. If necessary, a vocal tract image regarding the shape and position of the tongue and lips for each phoneme may be created based on statistical analysis of MRI data. Alternatively, an image of the vocal tract may be created using Electromagnetic Articulography (EMA) measurement.

  A time series of phoneme feature quantities arranged in the order of phonemes is feature quantity time series data collated with feature quantity time series data by a learner. The standard data and type data may include voice data, accent display data, and the like in addition to the data for each phoneme.

FIG. 4 is a diagram exemplarily showing standard data characters, phonetic symbols, accent display, and vocal tract images of the word “April”. The vocal tract image is merely conceptual. An example of a phonetic symbol of the word “April” is as follows.
eipril
Since the phonetic symbol corresponds to a phoneme, the number of phonemes in this case is six. Since the vocal tract image data corresponds to phonemes, there are six image data corresponding to six phonemes.

The plurality of type data of the word “April” corresponds to pronunciations represented by the following phonetic symbols, for example.
april, aplil, aprir, eiplil, eiprir
Pronunciations of many learners are collected and pronunciations that are prone to errors are stored as type data. In this example, for each type of data, image data of 5 or 6 vocal tracts corresponding to each phoneme is stored.

  Here, a method of collating the learner's feature amount time-series data with the stored feature amount time-series data will be described. The following problems exist when comparing feature quantity time-series data by a learner with stored feature quantity time-series data. The length of the learner's voice data is not the same as the length of the type voice data. In addition, there are a portion that is uttered fast and a portion uttered late in the audio data. Therefore, in order to collate such two pieces of time-series data, it is necessary to collate the time-series data while expanding / decreasing the time interval according to the time. Specifically, two time-series frame lengths are represented by I and J, and each frame is represented by x and y. Two time-series frames from (1, 1) to (I, J) are represented by Among the combinations of routes to be connected, the combination that minimizes the sum of the collation distances is obtained. Such collation is performed by DP matching using dynamic programming. About DP matching, it describes in Unexamined-Japanese-Patent No. 9-305195 etc., for example.

  In step S1050 of FIG. 2, the accent analysis unit 113 of the pronunciation learning support system 100 receives voice data of a learner's word, phrase, or sentence from the voice acquisition unit 105 according to an instruction from the control unit 103, and obtains an accent. . As described above, since the feature amount time-series data frame by the learner is associated with the stored feature amount time-series data frame, any one of the stored accent positions of the learner's pronunciation is stored. Can be associated with the phoneme data.

  In step S1060 of FIG. 2, the output content creation unit 115 of the pronunciation learning support system 100 follows the learner's accent of the word, phrase, or sentence, and the feature amount time series of pronunciation by the learner according to the instruction of the control unit 103 Display output contents such as phonetic symbols and vocal tract image data corresponding to the stored feature amount time series data most similar to the data are created. Further, the learner output unit 117 of the pronunciation learning support system 100 displays the created display output content to the learner. The vocal tract image data may be displayed in correspondence with the phonetic symbols. For example, the color of the phonetic phonetic symbol corresponding to the vocal tract image data to be displayed may be changed. Also, the vocal tract image data may be displayed as a moving image. For words composed of a series of phonemes, moving image data may be created by interpolating image data for each phoneme by an appropriate method.

  Further, in contrast to the content corresponding to the stored feature amount time-series data most similar to the feature amount time-series data by the learner, it corresponds to the accent of the standard data and the feature amount time-series data of the stored standard data. The contents may be displayed. Or you may mark and display the phoneme different from the phoneme of a standard pronunciation among the phonemes of a learner's pronunciation.

  Image data such as vocal tract image data may be stereoscopic data. It is known that the learning effect is improved by using stereoscopic data.

In addition, comments relating to pronunciation correction for each type of data are stored in the database 111, and the output content creation unit 115 selects a comment for each type of data and displays it as a character display or sound from the learner output unit 117. You may make it output. For example, if the type data is
eiprir
For the final phoneme, a comment such as “Please touch the tip of the tongue under the gums for the pronunciation of“ L ”” is displayed or output from the learner output unit 117. You may do it.

ここで、１）及び２）において、英語の発音記号を使用して音素を表している。このような現象による発音は、単語のみの発音の学習では、学習することができない。本発明によれば、単語だけではなく語句または文を学習の対象とすることができるので、このような現象による発音を学習することもできる。一般に、本発明は、フランス語のリエゾンを含む、種々の言語の連音の学習に有効である。 Further advantages of the present invention will be described. For example, in English, r at the end of a word is not pronounced, but there is a phenomenon in which r is pronounced when a word starting with a vowel continues. This phenomenon corresponds to the French liaison. In the following 1), there is no pronunciation of r, but in 2) there is an is that begins with a vowel, so there is a pronunciation of r.

Here, in 1) and 2), phonemes are expressed using English phonetic symbols. Pronunciation by such a phenomenon cannot be learned by learning pronunciation of words only. According to the present invention, not only words but also phrases or sentences can be targeted for learning, so pronunciation by such a phenomenon can also be learned. In general, the present invention is effective for learning a continuous tone of various languages including a French liaison.

  Moreover, since the position of the learner's accent can be associated with the phoneme as described above, an accented phonetic symbol can be displayed for the learner's pronunciation.

  FIG. 5 is a flowchart for explaining how to use the learner in the pronunciation learning support system 100 of the present invention.

  In step S2010 of FIG. 5, the learner activates the pronunciation learning support system 100 and designates a word, a phrase, or a sentence to be learned by the learner input unit 101. The pronunciation learning support system 100 prepares to learn a specified word, phrase, or sentence, and when the preparation is completed, the learner output unit 117 instructs the learner to pronounce.

  In step S2020 in FIG. 5, the learner pronounces a word, phrase, or sentence according to the instruction from the pronunciation learning support system 100. The pronunciation learning support system 100 performs the processing shown in the flowchart of FIG. 2 and outputs the result from the learner output unit 117 as display of image data corresponding to the learner's pronunciation.

  In step S2030 of FIG. 5, the learner observes image data corresponding to the pronunciation displayed on the learner output unit 117.

  In step S2040 of FIG. 5, the learner learns using the image data. In response to an instruction via the learner input unit 101 of the learner, image data corresponding to the pronunciation of the learner is repeatedly displayed on the learner output unit 117, and the data corresponding to the standard voice is compared with the data. It may be displayed.

  In step S2050 of FIG. 5, the learner determines whether or not to continue learning. When continuing, it returns to step S2010. If not continued, the process is terminated.

  According to the pronunciation learning support method of the present invention, the learner can visually recognize the characteristics of his / her pronunciation clearly by the phonetic symbol corresponding to the phoneme and the image of the vocal tract. By storing the learner's prone pronunciations as type data in the database 111, it is possible to cope with various learners' pronunciations. Further, since the position of the accent of the learner's pronunciation can be displayed in association with any stored phoneme data, the learner can clearly recognize his own accent. Further, since the phoneme and accent data of standard pronunciation can be displayed in contrast with the data related to the pronunciation of the learner, the learner's understanding deepens and the learning effect is improved. In particular, since image data including a vocal tract image is displayed as stereoscopic image data, the learner's understanding is further deepened and the learning effect is further improved.

Claims (9)

A pronunciation learning support method using a pronunciation learning support system,
The pronunciation learning support system acquiring a voice of a word, phrase or sentence by a learner;
The pronunciation learning support system obtaining the feature amount time-series data of the acquired speech;
The pronunciation learning support system includes the acquired speech feature amount time-series data, and a plurality of type feature amount time-series data of the speech of the word, phrase or sentence stored in the database of the pronunciation learning support system; And selecting type feature amount time series data most similar to the acquired feature amount time series data of speech;
A pronunciation learning support method comprising: outputting the data corresponding to the selected type feature amount time-series data.   The pronunciation learning support method according to claim 1, wherein the data corresponding to the selected type feature amount time-series data includes a vocal tract image corresponding to speech.   The pronunciation learning support method according to claim 1, wherein the data corresponding to the selected type feature amount time-series data includes moving image data.   The pronunciation learning support method according to claim 1, wherein the data corresponding to the selected type feature amount time-series data includes stereoscopic image data.   5. The method according to claim 1, further comprising a step of obtaining an accent of the acquired voice, and data corresponding to the selected type feature amount time-series data includes data of the acquired voice accent. The pronunciation learning support method described.   6. In the displaying step, in addition to data corresponding to the selected type feature quantity time-series data, data corresponding to feature quantity time-series data of standard speech is output. Pronunciation learning support method.   7. At least a part of data corresponding to the selected type feature quantity time-series data or at least a part of data corresponding to the feature quantity time-series data of the standard speech is associated with a phoneme. The pronunciation learning support method according to any of the above. An audio acquisition unit that acquires audio of words, phrases or sentences by learners;
A feature extraction unit for obtaining feature amount time-series data of the acquired speech;
A database for storing a plurality of type feature amount time series data prepared in advance for speech of the word, phrase or sentence;
The acquired feature amount time series data of speech and the plurality of type feature amount time series data are collated, and the type feature amount time series data most similar to the acquired feature amount time series data of the speech is selected. A selection section;
A pronunciation learning support system including: a learner output unit that displays image data corresponding to the selected type feature amount time-series data.   Further comprising an accent analysis unit for obtaining the accent of the acquired speech, wherein the learner output unit is configured to output the analyzed accent in association with the selected type feature amount time-series data. Item 9. The pronunciation learning support system according to Item 8.

Images