JP2006178334A - Language learning system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a language learning system by which a learner learns language by using a model speech. <P>SOLUTION: The language learning system includes: a database in which the featured values extracted from the speeches of a speaker and the speech data of the speaker are made correspondent to each other and are recorded; a speech acquisition means which acquires the learner's speeches; a featured value extraction means which extracts the featured values of the speeches of the learner from the speeches acquired by the speech acquisition means; an approximation degree calculation means which calculates the approximation degree index indicating the difference between the featured values recorded in the database, and the featured values extracted by the featured value extraction means; a speech data extraction means which extracts the speech data made correspondent to the featured values at which the approximation degree index calculated by the approximation degree calculation means from the database satisfies first conditions: and a reproduction means which outputs the speeches in compliance with the speech data extracted by the speech data extraction means. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a language learning system that supports language learning.

  In language learning of a foreign language or native language, especially in self-study of pronunciation or utterance, the model voice recorded on a recording medium such as a CD (Compact Disk) is played, and the model voice is imitated to pronounce or speak. The learning method is widely used. The purpose of this is to acquire correct pronunciation by imitating model voices. Here, in order to advance learning more effectively, it is necessary to evaluate the difference between the model voice and one's own voice. However, the model voice recorded on the CD is mostly the voice of a specific announcer or native speaker. That is, for many learners, these model voices are uttered by voices that have completely different characteristics from their own voices, so how accurate their pronunciation is compared to the model voices. There was a problem that evaluation was difficult.

As a technique for solving such a problem, there are techniques described in Patent Documents 1 and 2, for example. The technique described in Patent Literature 1 reflects parameters such as user inflection, speech speed, and voice quality in the model voice, and converts the model voice into a voice similar to the user voice. The technique described in Patent Document 2 enables a learner to select an arbitrary one from a plurality of model sounds.
JP 2002-244547 A JP 2004-133409 A

  However, although it is possible to calibrate intonation according to the technique described in Patent Document 1, it is difficult to calibrate a material whose pronunciation is clearly different, such as “r and l” and “s and th” in English. There was a problem. Furthermore, since the speech waveform is corrected, there is a problem that the processing becomes complicated. Further, the technique described in Patent Document 2 is a method of selecting a model voice, and thus requires a learner to select the model voice by himself, which is cumbersome.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a language learning apparatus that can learn using model speech similar to a learner with simpler processing.

  In order to solve the above-described problem, the present invention provides a database that records, for a plurality of speakers, data in which feature amounts extracted from the speech of the speaker are associated with one or more speech data of the speaker, Voice acquisition means for acquiring the learner's voice; feature quantity extraction means for extracting the feature quantity of the learner's voice from the voice acquired by the voice acquisition means; and a plurality of speakers recorded in the database. Approximation degree calculating means for calculating an approximation index indicating a difference between the feature quantity and the feature quantity extracted by the feature quantity extracting means for each speaker, and the approximation calculated by the approximation degree calculating means from the database Voice data extracting means for extracting one voice data associated with a feature quantity whose degree index satisfies the first condition, and outputting voice according to the one voice data extracted by the voice data extracting means That provides a language learning system and a reproducing means.

In a preferred embodiment, in the language learning system, the first condition may be a condition that the one having the highest degree of approximation is extracted.
In another preferred embodiment, the language learning system further includes speech speed converting means for converting the speech speed of the voice data extracted by the extracting means, and the reproducing means converts the speech speed by the speech speed converting means. Audio may be output according to the audio data.
In yet another preferred embodiment, the language learning system compares a storage unit that stores a model voice, the model voice, and a learner's voice acquired by the voice acquisition unit, and quantifies the degree of approximation between the two. And when the approximation obtained by the comparison unit satisfies the second condition, the learner's voice acquired by the acquisition unit is extracted from the feature amount extracted by the feature amount extraction unit. You may further have a database update means to match and add to the said database.

  According to the present invention, a voice uttered by a speaker having a voice characteristic similar to that of a learner is reproduced as a voice of an example sentence in learning. Therefore, the learner can more accurately recognize the pronunciation to be imitated (targeted), thereby improving the learning efficiency.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<1. Configuration>
FIG. 1 is a block diagram showing a functional configuration of a language learning system 1 according to the first embodiment of the present invention. The storage unit 11 stores a database DB1 in which a feature amount extracted from a speaker's voice and voice data of the speaker's voice are recorded in association with each other. The input unit 12 acquires the learner (user) voice and outputs it as user voice data. The feature extraction unit 13 extracts feature amounts from the learner's voice. The voice data extraction unit 14 compares the feature amount extracted by the feature extraction unit 13 with the feature amount recorded in the database DB1, and extracts the one that satisfies a predetermined condition. The voice data extraction unit 14 further extracts voice data associated with the extracted feature amount. The reproduction unit 15 reproduces the audio data extracted by the audio data extraction unit 14.

  Although the detailed contents of the database DB1 will be described later, the language learning system 1 further includes the following components to update the database DB1. The storage unit 16 stores an exemplary speech database DB2 in which exemplary speech data serving as a model for language learning and text data of the exemplary speech are recorded in association with each other. The comparison unit 17 compares the user voice data acquired by the input unit 12 with the model voice data stored in the storage unit 16. As a result of the comparison, when the user voice satisfies a predetermined condition, the DB update unit 18 adds the user voice data to the database DB1.

  FIG. 2 is a diagram illustrating the contents of the database DB1. In the database DB1, a speaker ID (“ID001” in FIG. 2), which is an identifier for identifying a speaker, and a feature amount extracted from the voice data of the speaker are recorded. The database DB1 further stores an example sentence ID that is an identifier for identifying an example sentence, voice data of the example sentence, and a pronunciation level (described later) of the example sentence in association with each other. The database DB1 has a plurality of data sets composed of example sentence IDs, voice data, and sound generation levels, and each data set is recorded in association with a speaker ID given to the speaker of the voice data. That is, the database DB1 has audio data of a plurality of example sentences by a plurality of speakers, and these data are recorded in association with each speaker by the speaker ID and the feature amount.

  FIG. 3 is a block diagram illustrating a hardware configuration of the language learning system 1. A CPU (Central Processing Unit) 101 reads and executes a program stored in a ROM (Read Only Memory) 103 or an HDD (Hard Disk Drive) 104 using a RAM (Random Access Memory) 102 as a work area. The HDD 104 is a storage device that stores various application programs and data. The HDD 104 also stores a database DB1 and an exemplary voice database DB2. The display 105 is a display device that displays characters and images under the control of the CPU 101, such as a CRT (Cathode Ray Tube) or an LCD (Liquid Crystal Display). The microphone 106 is a sound collection device for acquiring the user's voice, and outputs a voice signal corresponding to the voice uttered by the user. The sound processing unit 107 has a function of converting an analog sound signal output from the microphone 106 into digital sound data, and a function of converting sound data stored in the HDD 104 into a sound signal and outputting the sound signal to the speaker 108. Further, the user can input instructions to the language learning system 1 by operating the keyboard 109. Each component described above is connected to each other via a bus 110. The language learning system 1 can communicate with other devices via an I / F (interface) 111.

<2. Operation>
Next, the operation of the language learning system 1 according to this embodiment will be described. Here, after first describing the operation of reproducing the sound of the example sentence, the operation of updating the contents of the database DB1 will be described. In the language learning system 1, the CPU 101 has a function shown in FIG. 1 by executing a language learning program stored in the HDD 104. A learner (user) inputs a user ID, which is an identifier for identifying himself / herself, by operating the keyboard 109 at the start of a language learning program. The CPU 101 stores the input user ID in the RAM 102 as the user ID of the learner who is currently using the system.

<2-1. Audio playback>
FIG. 4 is a flowchart showing the operation of the language learning system 1. When the language learning program is executed, the CPU 101 of the language learning system 1 searches the model speech database DB2 and creates a list of usable example sentences. Based on this list, the CPU 101 displays a message prompting the user to select an example sentence on the display 105. The user selects one example sentence from the example sentences in the list according to the message displayed on the display 105. The CPU 101 reproduces the voice of the selected example sentence (step S101). Specifically, the CPU 101 reads out the model voice data of the example sentence from the model voice database DB2, and outputs the read out model voice data to the voice processing unit 107. The voice processing unit 107 performs digital / analog conversion on the input model voice data and outputs the analog voice signal to the speaker 108. Thus, the model voice is reproduced from the speaker 108.

  The user listens to the model voice reproduced from the speaker 108, and utters an example sentence simulating the model voice toward the microphone 106. That is, a user voice is input (step S102). Specifically, it is as follows. When the reproduction of the model voice is finished, the CPU 101 displays a message on the display 105 urging the user to generate an example sentence such as “Next is your turn. Please pronounce the example sentence”. Further, the CPU 101 displays a message on the display 105 instructing an operation for inputting the user voice, such as “Sound after pressing the space key and press the space key again when the sound is finished”. The user operates the keyboard 109 according to the message displayed on the display 105 and inputs the user voice. That is, after the space key on the keyboard 109 is pressed, the example sentence is uttered toward the microphone 106. When the utterance is finished, the user presses the space key again.

  The user's voice is converted into an electric signal by the microphone 106. The microphone 106 outputs a user voice signal. The user voice signal is converted into digital voice data by the voice processing unit 107 and recorded in the HDD 104 as user voice data. After the reproduction of the model voice is completed, the CPU 101 starts the recording of the user voice data with the press of the space key as a trigger, and ends the recording of the user voice data with the press of the space key again as a trigger. That is, the user voice from when the user first presses the space key until the user presses the space key again is recorded in the HDD 104.

  Subsequently, the CPU 101 performs feature amount extraction processing on the obtained user voice data (step S103). Specifically, it is as follows. The CPU 101 divides the audio data every predetermined time (frame). The CPU 101 obtains the logarithm of the amplitude spectrum obtained by Fourier transforming the waveform indicated by the exemplary voice data decomposed into frames and the waveform indicated by the user voice signal, and inverse Fourier transforms it to obtain a spectrum envelope for each frame. . The CPU 101 extracts the formant frequencies of the first formant and the second formant from the spectrum envelope thus obtained. In general, a vowel is characterized by a distribution of first and second formants. The CPU 101 matches the formant frequency distribution obtained for each frame from the head of the audio data with the formant frequency distribution of a predetermined vowel (eg, “a”). When it is determined by matching that the frame corresponds to the vowel “a”, the CPU 101 determines the formant (for example, first, second, and third) determined in advance among the formants in the frame. The formant frequency of formant) is calculated. The CPU 101 stores the calculated formant frequency in the RAM 102 as the feature amount P of the user's voice.

  Subsequently, the CPU 101 extracts voice data associated with a feature amount similar to the feature amount P of the user's voice from the database DB1 (step S104). Specifically, the extracted feature quantity P and the feature quantity recorded in the database DB1 are compared, and the closest approximation to the feature quantity P is specified. In the comparison, for example, a difference between the values of the first to third formant frequencies is calculated between the feature amount P and the database DB1, and an amount obtained by adding the absolute values of the differences between the three formant frequencies is also approximated between the two. Calculated as an approximation index indicating degrees. The CPU 101 specifies from the database DB1 a feature amount that has the smallest calculated approximation index, that is, a feature amount that is closest to the feature amount P. The CPU 101 further extracts audio data associated with the specified feature amount, and stores the extracted audio data in the RAM 102.

Subsequently, the CPU 101 reproduces audio data (step S105). Specifically, it is as follows. The CPU 101 outputs audio data to the audio processing unit 107. The audio processing unit 107 performs digital / analog conversion on the input audio data and outputs the audio data to the speaker 108 as an audio signal. Thus, the extracted audio data is reproduced as audio from the speaker 108. Here, since the voice data is extracted by feature amount matching, the reproduced voice is a voice having features similar to those of the learner. Therefore, even if the learner has an example sentence that was difficult to imitate only by listening to the voice produced by a speaker (announcer, native speaker, etc.) having a voice characteristic completely different from that of himself / herself, If the voice is produced by a speaker with similar voice characteristics, the pronunciation that should be imitated can be understood more correctly and the learning efficiency can be improved.

<2-2. Database update>
Subsequently, an update operation of the database DB1 will be described.
FIG. 5 is a flowchart showing the update operation of the database DB1 in the language learning system 1. First, the reproduction of the model voice and the input of the user voice are performed by the processes in steps S101 to S102 described above. Subsequently, the CPU 101 performs a comparison process between the model voice and the user voice (step S201). Specifically, it is as follows. The CPU 101 divides the waveform indicated by the exemplary audio data for each predetermined time (frame). The CPU 101 also divides the waveform indicated by the user voice data for each frame. The CPU 101 obtains the logarithm of the amplitude spectrum obtained by performing Fourier transform on the waveform indicated by the exemplary voice data decomposed into frames and the waveform indicated by the user voice signal, and inverse Fourier transforms it to obtain a spectrum envelope for each frame. .

  FIG. 6 is a diagram illustrating a spectrum envelope of an exemplary voice (upper) and a user voice (lower). The spectrum envelope shown in FIG. 6 is composed of three frames, frame I to frame III. The CPU 101 performs a process of comparing the obtained spectrum envelopes for each frame and digitizing the degree of approximation between them. The numerical value of the approximation (calculation of the approximation index) is performed as follows, for example. The CPU 101 may calculate, as an approximation index, the sum of the distance between two points when the characteristic formant frequency and spectral density are represented in the spectral density-frequency diagram for the entire audio data. Alternatively, an approximation index may be calculated by integrating the spectral density difference at a specific frequency with respect to the entire audio data. Note that the typical voice and the user voice usually have different lengths (time), and therefore, it is preferable to perform a process of aligning the lengths of the both before the above-described process.

  A description will be given with reference to FIG. 5 again. The CPU 101 determines whether or not to update the database DB1 based on the calculated approximation index (step S202). Specifically, it is as follows. The HDD 104 stores in advance conditions for additionally registering the acquired audio data in the database DB1. The CPU 101 determines whether the approximation index calculated in step S201 satisfies this registration condition. When the registration conditions are satisfied (step S202: YES), the CPU 101 advances the process to step S203 described later. If the registration condition is not satisfied (step S202: NO), the CPU 101 ends the process.

  When the registration conditions are satisfied, the CPU 101 performs database update processing (step S203). Specifically, it is as follows. The CPU 101 assigns a user ID that identifies a learner (user) who is a speaker of the voice data to the voice data that satisfies the registration conditions. The CPU 101 searches the model voice database DB2 for the same user ID as the user ID, and additionally registers the voice data in the model voice database DB2 in association with the user ID. When the user ID extracted from the update request is not registered in the model voice database DB2, the CPU 101 additionally registers this user ID and registers voice data in association with this user ID. In this way, the learner's voice data is additionally registered and updated in the database DB1.

  The database update operation described above may be performed in parallel with the above-described audio reproduction operation or may be performed after the audio reproduction operation is completed. Thus, the learner's voice data is sequentially added to the database DB1, so that the voice data of many speakers is accumulated in the database DB1. Therefore, as the language learning system 1 is used, voice data of many speakers are registered in the database DB1, and at the same time, when a new learner uses the language learning system 1, a voice having characteristics similar to that of himself / herself is recorded. The probability of being played increases.

<3. Modification>
The present invention is not limited to the above-described embodiment, and various modifications can be made.
<3-1. Modification 1>
In the above-described embodiment, after the voice data extracted in step S104 is stored in the RAM 102, the CPU 101 may perform a speech speed conversion process on the voice data. Specifically, it is as follows. The RAM 102 stores in advance a variable a that designates a speech speed ratio before and after the speech speed conversion process. The CPU 101 performs processing for multiplying the extracted audio data by a times the audio time (the time required for reproduction from the beginning to the end of the audio data). When a> 1, the speech length is increased by the speech speed conversion process. That is, the speaking speed becomes slow. Conversely, when a <1, the speech length is reduced by the speech speed conversion process. That is, the speaking speed is increased. In the present embodiment, a value greater than 1 is set as the initial value of the variable a. Therefore, after the model voice is reproduced and subsequently the user voice is input, the example sentence reproduced with the voice similar to the user voice is reproduced more slowly than the model voice. Therefore, the learner can more clearly recognize the pronunciation to be imitated (pronunciation to be targeted).

<3-2. Modification 2>
In the above-described embodiment, the voice data associated with the feature quantity most similar to the feature quantity extracted from the voice of the learner (user) is extracted in step S104. It is not limited to the one closest to the voice feature amount. For example, in the database DB1, the speech level of the speech (an index indicating the degree of approximation with the model speech is associated with the speech data of the example sentence. Those having a high speech level are more similar to the model speech). This utterance level may be incorporated into the audio data selection condition. As a specific condition, for example, a condition in which a feature amount closest to the utterance level is extracted from a certain level or higher may be used. Alternatively, it may be a condition that the highest speech level is extracted from features whose degree of approximation of the feature amount is greater than a certain value. The speech level may be performed in the same manner as the calculation of the approximation index in step S201, for example.

<3-3. Modification 3>
The system configuration is not limited to that described in the above embodiment. The language learning system 1 may be connected to a server device via a network, and the server device may have a part of the functions of the language learning system described above.
Furthermore, in the above-described embodiment, the function as the language learning system is realized by software by the CPU 101 executing the language learning program. However, an electronic circuit or the like corresponding to the functional components shown in FIG. 1 is used. The system may be realized in hardware.

<3-4. Modification 4>
In the above-described embodiment, the aspect in which the formant frequencies of the first to third formants are used as the feature amount of the speaker's voice has been described. However, the feature amount of the voice is not limited to the formant frequency. It may be a feature amount calculated based on another speech analysis method such as a spectrogram.

It is a block diagram which shows the function structure of the language learning system 1 which concerns on 1st Embodiment of this invention. It is a figure which illustrates the contents of database DB1. 2 is a block diagram showing a hardware configuration of a language learning system 1. FIG. 3 is a flowchart showing the operation of the language learning system 1. It is a flowchart which shows the update operation | movement of database DB1 in the language learning system 1. FIG. It is a figure which illustrates the spectrum envelope of model voice (upper) and user voice (lower).

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Language learning system, 2 ... Language learning system, 11 ... Memory | storage part, 12 ... Input part, 13 ... Feature extraction part, 14 ... Audio | voice data extraction part, 15 ... Playback | regeneration part, 16 ... Memory | storage part, 17 ... Comparison part, 18 ... DB update unit, 21 ... speech speed conversion unit, 101 ... CPU, 102 ... RAM, 104 ... HDD, 105 ... display, 106 ... microphone, 107 ... voice processing unit, 108 ... speaker, 109 ... keyboard, 110 ... bus 111 ... I / F

Claims (4)

A database that records, for a plurality of speakers, data that associates feature amounts extracted from the speaker's voice with one or more voice data of the speaker;
Voice acquisition means for acquiring learner's voice;
Feature quantity extraction means for extracting feature quantities of the learner's voice from the voice acquired by the voice acquisition means;
A degree-of-approximation calculation means for calculating an approximation index indicating the difference between the feature quantity of the plurality of speakers recorded in the database and the feature quantity extracted by the feature quantity extraction means;
Voice data extraction means for extracting from the database one voice data in which the approximation index calculated by the approximation calculation means is associated with a feature amount satisfying a first condition;
A language learning system comprising: reproducing means for outputting a sound in accordance with one sound data extracted by the sound data extracting means.   2. The language learning system according to claim 1, wherein the first condition is a condition that the one having the highest degree of approximation is extracted. A speech speed converting means for converting the speech speed of the voice data extracted by the extracting means;
The language learning system according to claim 1, wherein the reproduction unit outputs a voice according to the voice data subjected to the speech speed conversion by the speech speed conversion unit. Storage means for storing the model voice;
A comparison means for comparing the model voice and the learner's voice acquired by the voice acquisition means, and quantifying the degree of approximation between both;
When the degree of approximation obtained by the comparison unit satisfies a second condition, the database of the learner's voice acquired by the acquisition unit is associated with the feature amount extracted by the feature amount extraction unit The language learning system according to claim 1, further comprising: database updating means to be added to.

Images