KR20060073502A - Language learning system and voice data providing method for language learning - Google Patents

Abstract

The present invention makes it possible to learn using exemplary voices similar to learners. For each of the plurality of speakers, a database for storing the feature amounts extracted from the speaker's voice and one or a plurality of pieces of audio data for language learning spoken by the speaker are stored. The learner's voice is acquired, and the feature amount of the learner's voice is extracted from the acquired voice. The extracted feature amount of the learner is compared with the feature amounts of a plurality of speakers recorded in the database, and based on the comparison, voice data of one speaker is selected from the database. The audio for language learning is reproduced and output according to the selected audio data.

Exemplary voice, voice data, speaker feature, learner feature, user voice

Description

LANGUAGE LEARNING SYSTEM AND VOICE DATA PROVIDING METHOD FOR LANGUAGE LEARNING}

1 is a block diagram showing a functional configuration of a language learning system 1 according to a first embodiment of the present invention.

2 illustrates the contents of a database DB1.

3 is a block diagram showing a hardware configuration of the language learning system 1.

4 is a flowchart showing the operation of the language learning system 1.

5 is a flowchart showing an update operation of the database DB1 in the language learning system 1;

6 illustrates spectral envelope of exemplary voice (top) and user voice (bottom).

<Explanation of symbols for the main parts of the drawings>

1: language learning system

2: language learning system

11: memory

12: input unit

13: feature extraction unit

14: voice data extraction unit

15: playback unit

16: memory

17: comparison unit

18: DB update unit

21: fire speed conversion unit

101: CPU

102: RAM

104: HDD

105: display

106: microphone

107: voice processing unit

108: speaker

109: keyboard

110: the bus

111: I / F

[Patent Document 1] Japanese Patent Application Laid-Open No. 2002-244547

[Patent Document 2] Japanese Patent Application Laid-Open No. 2004-133409

The present invention relates to a language learning system that supports language learning.

In the language learning of a foreign language or a native language, in particular, the pronunciation or utterance reading, a learning method of reproducing exemplary voices recorded on a recording medium such as a CD (Compact Disk) and imitating the exemplary voices to pronounce or speak is widely used. It is used. This aims to learn correct pronunciation by mimicking exemplary voices. Here, in order to advance learning more effectively, it is necessary to evaluate the difference between the exemplary voice and one's own voice. However, the exemplary voice recorded on the CD is most often the voice of any particular announcer or native speaker. That is, for many learners, since these exemplary voices are emitted by voices having completely different characteristics from their own voices, it is difficult to evaluate how accurate their pronunciation is compared to the exemplary voices. There was a problem.

As a technique which solves such a problem, the technique of patent document 1, 2 is mentioned, for example. The technique described in Patent Document 1 converts exemplary voices into voices similar to user voices by reflecting parameters such as user intonation, speech rate, and sound quality in exemplary voices. The technique described in Patent Literature 2 enables the learner to select any of a plurality of exemplary voices.

However, according to the technique described in Patent Literature 1, the correction of the intonation is possible, but there is a problem that the correction is difficult even though the pronunciation is clearly different, such as "r and l" or "s and th" in English. . In addition, there is a problem that the processing becomes complicated because correction is performed on the audio waveform. In addition, in the technique described in Patent Document 2, since it is a method of selecting the exemplary voice, the learner needs to select the exemplary voice on his own and there is a problem that it is complicated.

SUMMARY OF THE INVENTION The present invention has been made in view of the above circumstances, and an object thereof is to provide a language learning system and method capable of learning using a model voice similar to a learner by simpler processing.

MEANS TO SOLVE THE PROBLEM In order to solve the above-mentioned subject, this invention provides the database which stores the feature quantity extracted from the speaker's voice for each of a plurality of speakers, the database which memorize | stores one or more voice data of the speaker, and stores them, respectively, A feature amount extracting means for extracting a feature amount of the learner's voice from the sound acquiring means for acquiring the speech, a feature acquired by the speech acquiring means, and the feature amount of the learner extracted by the feature amount extracting means, A voice data selection means for comparing the voice data of a plurality of speakers recorded in the database with a feature and selecting one speaker's voice data from the database based on the comparison, and one selected by the voice data selection means. A language learning system having reproducing means for outputting speech in accordance with speech data of the present invention.

In a preferred embodiment, the voice data selecting means includes, for each speaker, an approximate index indicating the difference between the feature amounts of the plurality of speakers recorded in the database and the feature amount of the learner extracted by the feature quantity extracting means. An approximation calculation means for calculating, and based on the approximation index calculated by the approximation calculation means, one piece of audio data associated with a feature amount of one speaker satisfying a predetermined condition is stored in the database. It is characterized by extracting selectively from the. In this case, the predetermined condition may be a condition that a voice data of one speaker corresponding to the approximate index indicating the highest approximation is selected.

In another preferred aspect, the language learning system further has speech rate converting means for converting speech rates of speech data selected by the speech data selecting means, wherein the reproducing means is adapted to speech data converted by speech rate converting means. You may output audio accordingly.

In another preferred aspect, the language learning system compares the memory means for storing exemplary voices with the learner's voices acquired by the exemplary voices and the voice acquiring means to generate information indicating the approximate degree of both. When the approximation degree represented by the comparing means and the information generated by the comparing means satisfies a predetermined condition, the learner's speech acquired by the speech obtaining means is extracted by the feature quantity extracting means. You may further be provided with the database update means added to the said database corresponding to a feature amount.

According to the present invention, a voice made by a speaker having a sound characteristic similar to that of a learner can be reproduced as a voice of an example sentence in learning. Therefore, the learner can recognize the pronunciation to be simulated (targeted) more accurately, thereby improving the learning efficiency.

<Example>

EMBODIMENT OF THE INVENTION Hereinafter, the Example of this invention is described with reference to drawings.

<1. Configuration>

1 is a block diagram showing a functional configuration of a language learning system 1 according to a first embodiment of the present invention. The memory | storage part 11 memorize | stores the database DB1 which recorded the feature quantity extracted from the speaker's voice | voice, and the audio | voice data of the voice | voice by the speaker. The input unit 12 obtains the voice of the learner (user) and outputs it as user voice data. The feature extraction unit 13 extracts a feature amount from the learner's voice. The voice data extraction (selection) unit 14 compares the feature amount extracted by the feature extraction unit 13 with the feature amount recorded in the database DB1, and satisfies a predetermined speaker's feature. The amount is extracted, and the audio data corresponding to the extracted feature amount of one speaker is extracted (selected) from the database DB1. The reproduction unit 15 reproduces the audio data extracted (selected) by the audio data extraction (selection) unit 14 and audibly sounds through a speaker or earphone.

The details of the database DB1 will be described later, but the language learning system 1 also has the following components for updating the database DB1. The memory | storage part 16 memorize | stores the exemplary audio | voice database DB2 which corresponded and recorded the exemplary audio | voice data used as a sample of language learning and the text data of the exemplary audio | voice. The comparison unit 17 compares the user voice data acquired by the input unit 12 with the exemplary voice data stored in the storage unit 16. As a result of the comparison, if the user voice satisfies a predetermined condition, the DB update unit 18 adds the user voice data to the database DB1.

2 is a diagram illustrating the contents of the database DB1. In the database DB1, the speaker ID ("ID001" in Fig. 2), which is an identifier for identifying the speaker, and the feature amount extracted from the speaker's voice data are recorded. The database DB1 further records an example sentence ID, which is an identifier for identifying an example sentence, audio data of the example sentence, and a pronunciation level (described later) of the example sentence. The database DB1 has a plurality of data sets consisting of example sentence IDs, voice data, and pronunciation levels, and each data set is recorded in association with the speaker ID provided to the speaker of the voice data. That is, the database DB1 has voice data of a plurality of example sentences by a plurality of speakers, and these data are corresponded and recorded for each speaker according to the speaker ID and the feature amount.

3 is a block diagram showing the hardware configuration of the language learning system 1. CPU (Central Processing Unit) 101 is a program stored in ROM (Read Only Memory) 103 or HDD (Hard Disk Drive) 104 with RAM (Random Access Memory) 102 as a work area. Read it and run it. The HDD 104 is a storage device that stores various application systems and data. The HDD 104 also stores the database DB1 and the exemplary voice database DB2. The display 105 is a display device that displays characters and images under the control of the CPU 101, such as a cathode ray tube (CRT) or a liquid crystal display (LCD). The microphone 106 is a sound collecting device for acquiring a user's voice, and outputs a voice signal corresponding to the voice issued by the user. The voice processing unit 107 converts the analog voice signal output by the microphone 106 into digital voice data, or converts the voice data stored in the HDD 104 into a voice signal and outputs the voice signal to the speaker 108. Has the function. In addition, the user can input the instruction to the language learning system 1 by operating the keyboard 109. Each component described above is connected to each other via the bus 110. In addition, the language learning system 1 can communicate with other devices via the I / F (interface) 111.

<2. Action>

Subsequently, the operation of the language learning system 1 according to the present embodiment will be described. Here, the operation of reproducing the audio of the example sentence will first be described, and then the operation of updating the contents of the database DB1 will be described. In the language learning system 1, the CPU 101 executes the language learning program stored in the HDD 104 to have the function shown in FIG. The learner (user) inputs a user ID which is an identifier for identifying the user by operating the keyboard 109 at the time of starting the language learning program. The CPU 101 stores the input user ID in the RAM 102 as the user ID of the learner currently using the system.

<2-1. Play voice>

4 is a flowchart showing the operation of the language learning system 1. When the language learning program is executed, the CPU 101 of the language learning system 1 searches the exemplary voice database DB2 and creates a list of available sentences. Based on this list, the CPU 101 displays a message on the display 105 prompting the selection of the example sentence. The user selects one example sentence from the example sentences in the list according to the message displayed on the display 105. The CPU 101 reproduces the audio of the selected example sentence (step S101). Specifically, the CPU 101 reads the exemplary voice data of the example sentence from the exemplary voice database DB2 and outputs the read exemplary voice data to the voice processing unit 107. The voice processing unit 107 digitally / analogs the input exemplary voice data and outputs the analog voice signal to the speaker 108 as an analog voice signal. In this way, the exemplary voice is reproduced from the speaker 108.

The user hears the exemplary voice reproduced from the speaker 108 and imitates the exemplary voice toward the microphone 106 to utter an example sentence. That is, user voice is input (step S102). Specifically, it is as follows. When reproduction of exemplary voice is finished, CPU101 says, "Next is your turn. Please pronounce the example sentence ”on the display 105 to prompt the user to generate the example sentence. The CPU 101 also displays a message on the display 105 instructing an operation for inputting the user's voice, such as "press the space key and then pronounce it, and press the space key again when the pronunciation is finished." Display. The user operates the keyboard 109 in accordance with the message displayed on the display 105 to input user voice. That is, after pressing the space key of the keyboard 109, an example sentence is made toward the microphone 106. When utterance ends, the user presses the space key again.

The user's voice is converted into an electrical signal by the microphone 106. The microphone 106 outputs a user voice signal. The user voice signal is converted into digital voice data by the voice processing unit 107 and recorded in the HDD 104 as user voice data. After the reproduction of the exemplary voice is completed, the CPU 101 starts recording of the user voice data by pressing the space key, and ends recording of the user voice data by triggering the pressing of the space key again. That is, the user's voice from the first time the user presses the space key until the user presses the space key again is recorded on the HDD 104.

Subsequently, the CPU 101 performs a feature variable extraction process on the obtained user voice data (step S103). Specifically, it is as follows. The CPU 101 divides the voice data every predetermined time (frame). The CPU 101 obtains the number of amplitude spectra obtained by Fourier transforming the waveform indicated by the exemplary speech data decomposed into frames and the waveform indicated by the user speech signal, and inversely transforms it to obtain spectral envelope for each frame. The CPU 101 extracts the formant frequencies of the first formant and the second formant from the spectral envelope thus obtained. In general, vowels are characterized by the distribution of first and second formants. The CPU 101 matches the formant frequency distribution obtained for each frame with the formant frequency distribution of a predetermined vowel (for example, "a") from the head of the audio data. If it is determined by matching that the frame corresponds to the vowel "a", the CPU 101 determines a predetermined formant (for example, first, second, and first) among the formants in the frame. Calculate the formant frequency of three formants of three). The CPU 101 stores the calculated formant frequency in the RAM 102 as the feature amount P of the user's voice.

Subsequently, the CPU 101 extracts (selects) the audio data associated with the feature amount similar to the feature amount P of the user's voice from the database DB1 (step S104). Specifically, the extracted feature variable P is compared with the feature variable recorded in the database DB1, and the closest approximation to the feature variable P is specified. In the comparison, for example, the difference between the values of the first to third formant frequencies is calculated between the feature amount P and the database DB1, and the sum of the absolute values of the differences between the three formant frequencies is added to each other. It is computed as an approximate index indicating the approximate degree of. The CPU 101 specifies, from the database DB1, the smallest calculated approximation index, that is, the feature amount that is closest to the feature amount P. FIG. The CPU 101 further extracts the voice data associated with the specified feature amount and stores the extracted voice data in the RAM 102.

Subsequently, the CPU 101 reproduces the audio data (step S105). Specifically, it is as follows. The CPU 101 outputs voice data to the voice processing unit 107. The voice processing unit 107 digitally / analogs the input voice data and outputs it to the speaker 108 as a voice signal. In this way, the extracted audio data is reproduced from the speaker 108 as audio. Since the voice data is extracted by matching the feature amounts, the reproduced voice is a voice similar in feature to the learner's voice. Therefore, the learner may have a sound characteristic that is very similar to his own, even if it is difficult to imitate simply by listening to a voice made by a speaker (announcer, native speaker, etc.) having a completely different sound characteristic. If the voice is made by a speaker having a voice, the pronunciation to be simulated can be more accurately understood, and the learning efficiency can be improved.

<2-2. Database Update>

Subsequently, the update operation of the database DB1 will be described.

5 is a flowchart showing an update operation of the database DB1 in the language learning system 1. First, reproduction of exemplary voices and input of user voices are performed by the processes of steps S101 to S102 described above. Subsequently, the CPU 101 performs a comparison process between the exemplary voice and the user voice (step S201). Specifically, it is as follows. The CPU 101 divides the waveform indicated by the exemplary voice data every predetermined time (frame). The CPU 101 also divides the waveform indicated by the user voice data for each frame. The CPU 101 obtains the amplitude spectrum obtained by Fourier transforming the waveform indicated by the exemplary speech data decomposed into frames and the waveform represented by the user speech signal, and inversely transforms the Fourier transform to obtain spectral envelope for each frame.

6 is a diagram illustrating the spectral envelope of exemplary voice (above) and user voice (bottom). The spectral envelope shown in FIG. 6 is composed of three frames of frames I to III. The CPU 101 compares the obtained spectral envelope for each frame and performs a process of digitizing the approximate degree of both. The numerical value of the approximation (calculation of the approximate index) is performed as follows, for example. The CPU 101 may calculate, as an approximation index, the sum of the distances between the two points when the characteristic formant frequency and the spectral density are shown in the spectral density-frequency diagram. Alternatively, the integral of the difference in spectral density at a specific frequency with respect to the entire voice data may be calculated as an approximate index. In addition, since the exemplary voice and the user voice usually have different lengths (times), it is preferable to perform a process of matching the lengths of the two prior to the above-described processing.

This will be described with reference to FIG. 5 again. The CPU 101 determines whether or not to update the database DB1 based on the calculated approximation index (step S202). Specifically, it is as follows. The HDD 104 stores in advance a condition for additionally registering the acquired audio data in the database DB1. The CPU 101 determines whether or not the approximation index calculated in step S201 satisfies this registration condition. If the registration condition is satisfied (step S202: Yes), the CPU 101 proceeds to step S203, which will be described later. If the registration condition is not satisfied (step S202: NO), the CPU 101 ends the process.

If the registration condition is satisfied, the CPU 101 performs a database update process (step S203). Specifically, it is as follows. The CPU 101 assigns to the voice data that satisfies the registration condition, a user ID for specifying a learner (user) who is the speaker of the voice data. The CPU 101 retrieves the same user ID as the user ID from the exemplary voice database DB2, and further registers the voice data in the exemplary voice database DB2 in association with the user ID. If the user ID extracted from the update request is not registered in the exemplary voice database DB2, the CPU 101 further registers this user ID and registers the voice data in correspondence with this user ID. In this way, the voice data of the learner is additionally registered in the database DB1 and updated.

The database update operation described above may be performed in parallel with the above-described audio reproduction operation or may be performed after completion of the audio reproduction operation. In this way, the voice data of the learners is sequentially added to the database DB1, so that the voice data of many speakers are accumulated in the database DB1. Therefore, as the language learning system 1 is used, the voice data of many speakers is registered in the database DB1, and at the same time, the probability that a voice similar to the characteristics of oneself is reproduced even when a new learner uses the language learning system 1 is increased. Goes.

<3. Variation>

The present invention is not limited to the above-described embodiment, but various modifications can be made.

<3-1. Modification Example 1

In the above-described embodiment, after storing the speech data extracted in step S104 in the RAM 102, the CPU 101 may perform the speech rate conversion process on the speech data. Specifically, it is as follows. The RAM 102 stores in advance a variable a that specifies the rate of speech rate before and after the rate of speech conversion process. The CPU 101 performs a process of multiplying the time (the time required for reproduction from the head to the end of the voice data) of the voice with respect to the extracted voice data. In the case of a> 1, the speech length is extended by the speech rate converting process. In other words, the fire speed slows down. On the contrary, in the case of a <1, the length of speech is reduced by the speech rate converting process. In other words, the speed of fire becomes faster. In this embodiment, a value larger than 1 is set as the initial value of the variable a. Therefore, the exemplary voice is reproduced, and after the user voice is continuously input, the example sentence reproduced with the voice similar to the user voice is reproduced more slowly than the exemplary voice. Thus, the learner can more clearly recognize the pronunciation to be simulated (the pronunciation to be targeted).

<3-2. Modification Example 2>

In the above-described embodiment, in step S104, the voice data associated with the feature amount extracted most closely from the feature amount extracted from the learner's (user's) voice is extracted, but the condition for extracting the voice data is the feature amount of the learner's voice and the most. It is not limited to the approximation. For example, in the database DB1, the speech level (the index representing the approximate degree to the typical speech; the higher the speech level approximates the better speech) is recorded in correspondence with the speech data of the example sentence. The level may be included in the condition of voice data selection. As specific conditions, for example, a condition in which the feature amount is most approximated among those whose ignition level is at least a certain level may be extracted. Alternatively, the condition of extracting the highest ignition level among the approximations of the feature amounts is greater than or equal to an arbitrary value. The ignition level may be performed similarly to the calculation of the approximate index in step S201.

<3-3. Variation Example 3>

In addition, the structure of a system is not limited to what was demonstrated in embodiment mentioned above. The language learning system 1 is connected to the server device via a network, and the server device may be responsible for some of the functions of the language learning system described above.

In addition, in the above-described embodiment, although the CPU 101 executes the language learning program, the function as the language learning system is realized in software. However, hardware is implemented using an electronic circuit or the like corresponding to the functional component shown in FIG. It is also possible to realize the system.

<3-4. Modification 4>

In the above-described embodiment, the aspect using the formant frequencies of the first to third formants as the feature amount of the speaker's voice has been described, but the feature amount of the voice is not limited to the formant frequency. The feature quantity calculated based on other speech analysis methods, such as a spectrogram, may be sufficient.

According to the present invention, it is possible to provide a language learning system and method capable of learning using exemplary voices similar to learners with simpler processing.

Claims (6)

For each of the plurality of speakers, a database for storing a feature amount extracted from the speaker's voice, one or more voice data of the speaker, and storing the correspondences; Voice acquisition means for acquiring the learner's voice; Feature amount extracting means for extracting a feature amount of the learner's speech from the speech acquired by the speech obtaining means; The feature quantity of the learner extracted by the feature quantity extracting means is compared with the feature quantities of a plurality of speakers recorded in the database, and based on the comparison, voice data of one speaker is selected from the database. Voice data selection means Reproducing means for outputting voice in accordance with one voice data selected by said voice data selecting means Language learning system having a. The method of claim 1, The voice data selecting means calculates an approximate index for each speaker, which indicates a difference between the feature amounts of the plurality of speakers recorded in the database and the feature amount of the learner extracted by the feature extracting means. Selecting from the database one piece of audio data that includes a calculation means and is associated with a feature amount of one speaker that satisfies a predetermined condition based on the approximation index calculated by the approximation calculation means. A language learning system characterized by the above. The method of claim 2, And said predetermined condition is a condition that a voice data of one speaker corresponding to an approximation index indicating the highest approximation is selected. The method of claim 1, Further comprising a speech rate converting means for converting speech rates of speech data selected by said speech data selecting means, And said reproducing means outputs speech in accordance with speech data converted by speech rate by said speech rate converting means. The method according to any one of claims 1 to 4, Memory means to remember exemplary voices, Comparison means for comparing the exemplary voice with the learner's voice acquired by the voice acquiring means, and generating information indicative of both approximations; When the approximation degree represented by the information generated by the comparing means satisfies a predetermined condition, the learner's voice acquired by the speech obtaining means corresponds to the feature quantity extracted by the feature extracting means. Means for updating the database to add it to the database Language learning system having more. For each of the plurality of speakers, the learner uses the database to store the feature data extracted from the speaker's voice and the speaker's voice data for one or more language lessons spoken by the speaker. As a method of providing The process of acquiring the voice spoken by the learner, Extracting the feature amount of the learner's speech from the acquired speech; Comparing the extracted feature amount of the learner with feature amounts of a plurality of speakers recorded in the database, and selecting voice data of one speaker from the database based on the comparison; Outputting a voice according to the selected single voice data Voice data providing method comprising the.

Images