JP2007139868A - Language learning device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a language learning device by which a learner can easily comprehend difference between learner's pronunciation and model pronunciation. <P>SOLUTION: A band in which a word is displayed inside shows the model pronunciation, and a band which is filled shows the learner's pronunciation. An arrangement position of a vertical direction of the band which is filled is determined according to a pitch stored to be made correspond to each word, and the arrangement position of the vertical position of the band which is filled is determined according to the pitch obtained from the learner's pronunciation. When the pitch of the model pronunciation is higher than that of the learner's pronunciation, for example, like the band in which a word "critical" is shown inside, the band showing the model pronunciation is displayed on the upper side of the band showing the learner's pronunciation. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for supporting language learning.

  In language learning of a foreign language or native language, especially in self-study of pronunciation or utterance, the model voice recorded on a recording medium such as a CD (Compact Disk) is played, and the model voice is imitated to pronounce or speak. The learning method is widely used. This is intended to acquire the correct pronunciation by imitating the model voice, but in order to advance learning more effectively, you need to know your voice specifically, You need to know the difference from your voice and bring your pronunciation closer to the correct pronunciation.

As a technique for enabling a learner to specifically grasp his / her voice, there are techniques disclosed in Patent Documents 1 and 2, for example. Patent Document 1 discloses a technique for displaying a learner's speech waveform, power, pitch, and the like. Patent Document 2 discloses a technique for displaying a voice waveform of an exemplary voice and a voice waveform of a learner's voice side by side.
Japanese Patent Laid-Open No. 1-154189 JP 2002-23613 A

  According to the technique disclosed in Patent Document 1, since the voice waveform, power, pitch, and the like of the voice are visualized, the learner can grasp his / her voice. Further, according to the technique disclosed in Patent Document 2, the learner can compare the speech waveforms and see the difference between the model speech and his / her speech. However, with the technique disclosed in Patent Document 1, it is difficult for ordinary learners who do not have specialized knowledge about speech waveforms to grasp improvements in pronunciation based on differences in waveforms and the like. There is a problem that it is difficult to grasp the difference between them specifically and to understand the part where pronunciation is bad. In the technique disclosed in Patent Document 2, the waveform of the model speech and the waveform of the learner's speech are displayed side by side, but an ordinary learner who does not have specialized knowledge about the speech waveform has a difference in waveform. There is a problem that it is difficult for the learner himself to understand how to improve pronunciation.

  The present invention has been made under the above-described background, and an object of the present invention is to provide a technique that enables a learner to easily grasp the difference between a model voice and a learner's voice.

  In order to solve the above-mentioned problems, the present invention provides a storage means for storing an example sentence of conversation and a pitch of the uttered voice of each word in the example sentence, a voice input means for inputting a voice, and the voice input means. Utterance voice extraction means for extracting utterance voice corresponding to each word in the example sentence, pitch extraction means for extracting the pitch of each voice extracted by the utterance voice extraction means, and the storage The example sentences stored in the means are displayed in horizontal composition, and a plurality of band-like figures representing the utterances of each word extracted by the utterance voice extraction means are displayed in horizontal composition, and the words in the example sentences Display means for displaying the word-like speech voice in association with the display, and the vertical display position of each word in the example sentence, the pitch of the speech voice of each word stored in the storage means To high and low And a display position determining means for determining the vertical display positions of the plurality of strip-shaped figures according to the pitch of each word extracted by the pitch extracting means. To do.

  Further, the present invention provides a storage means for storing an example sentence of conversation and an utterance time of an utterance voice of each word in the example sentence, a voice input means for inputting voice, and a voice input to the voice input means. Utterance voice extraction means for extracting utterance voice corresponding to each word in the example sentence, utterance time extraction means for extracting utterance time for each utterance voice extracted by the utterance voice extraction means, and the storage means The stored example sentences are displayed in horizontal composition, and a plurality of band-like figures representing the utterances of each word extracted by the utterance voice extraction means are displayed in horizontal composition, and the words in the example sentences, Display means for displaying in association with a band-like figure representing the utterance voice of a word, and the display length of each word in the example sentence to the length of the utterance time of the utterance voice of each word stored in the storage means As well as to decide Wherein the plurality of the display length of the strip shape, to provide a language learning apparatus and a display length determining means for determining in accordance with the length of speech time of each word extracted by said utterance time extracting means.

  Further, the present invention provides a storage means for storing example sentences and pitches of speech sounds of each phoneme or syllable in the example sentences, a voice input means for inputting a voice, and a voice input to the voice input means. Utterance voice extraction means for extracting utterance voice corresponding to each phoneme or syllable in the example sentence, pitch extraction means for extracting the pitch of each phoneme or syllable extracted by the utterance voice extraction means, and the storage means The stored example sentences are displayed in horizontal composition, and a plurality of band-like figures representing the speech sounds of each phoneme or syllable extracted by the utterance voice extraction means are displayed in horizontal composition, and the phonemes in the example sentences or Display means for displaying the syllable and the band-like figure representing the phoneme or the utterance voice of the syllable in association with each other, and the display position in the vertical direction of each phoneme or syllable in the example sentence, Each of the phonemes or syllables stored in the memory means is determined according to the pitch of the utterance voice, and the vertical display positions of the plurality of band-like figures are determined for each phoneme or syllable extracted by the pitch extracting means. There is provided a language learning device having display position determining means for determining the pitch according to the height of the pitch.

  According to the present invention, the learner can easily grasp the difference between the model voice and the learner's voice.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[Configuration of the embodiment]
FIG. 1 is a block diagram illustrating a hardware configuration of a language learning device 1 according to an embodiment of the invention. As shown in FIG. 1, each unit of the language learning device 1 is connected to a bus 101, and signals and data are exchanged between the units via the bus 101.

  The microphone 109 is connected to the sound processing unit 108, converts input sound into an electrical signal (hereinafter referred to as a sound signal), and outputs the electric signal to the sound processing unit 108. The speaker 110 is connected to the sound processing unit 108 and outputs a sound corresponding to the signal output from the sound processing unit 108. The audio processing unit 108 converts the audio signal input from the microphone 109 into digital data (hereinafter referred to as “learner data”) and outputs it, or converts the digital data representing the audio into an analog audio signal, and the speaker. The function to output to 110 is provided.

  The display unit 106 includes, for example, a display device such as a liquid crystal display, and displays a character string, various messages, a menu screen for operating the language learning device 1, and the like under the control of the CPU 102. The input unit 107 includes an input device such as a keyboard and a mouse (both not shown), and outputs a signal corresponding to the operation content to the CPU 102 in response to a key press or a mouse operation.

  The storage unit 105 includes, for example, an HDD (Hard Disk Drive) device, and stores various data. Specifically, the storage unit 105 stores learner data output from the voice processing unit 108. The storage unit 105 also stores example sentence text data representing example sentences used for language learning, and digital data (hereinafter referred to as model data) representing voices (hereinafter referred to as model sounds) when the native speaker reads out the example sentences. Is remembered. The storage unit 105 stores an example sentence table TB1 in the format illustrated in FIG. 2. In this table, example sentence text data, a file name of model data representing a voice when the native speaker reads the example sentence, and each example sentence An identifier for uniquely identifying text data is stored in association with each other. The storage unit 105 also stores word text data indicating the words included in the example sentence, the pitch of the pronunciation sound of each word in the model voice, and the utterance start time of each word in the model voice. . The storage unit 105 stores a word table TB2 in the format illustrated in FIG. 3. In this table, an identifier indicating an example sentence, text data of each word included in the example sentence, and pronunciation sound of each word are stored. The pitch and the utterance start time are stored in association with each other. The storage unit 105 stores the learner data output from the voice processing unit 108.

  A CPU (Central Processing Unit) 102 executes a program stored in a ROM (Read Only Memory) 103 using a RAM (Random Access Memory) 104 as a work area. When the CPU 102 executes the program, each unit is controlled by the CPU 102 to realize a function of displaying the difference between the model voice and the learner's voice.

[Operation of the embodiment]
Next, the operation of this embodiment will be described.
First, when the learner performs an operation to instruct display of a list of example sentences, the CPU 102 reads out example sentence text data stored in the example sentence table TB1 (FIG. 4: step SA1), and displays a list of example sentences represented by the read data. The information is displayed on the display unit 106 (step SA2). Thereafter, when the learner operates the input unit 107 and performs an operation of selecting one of the displayed example sentences (step SA3; YES), the CPU 102 displays the screen displayed on the display unit 106, the input unit, and the like. Based on the signal sent from 107, the selected example sentence is specified (step SA4). When CPU 102 identifies the selected example sentence, CPU 102 reads out the file name of the model data stored in association with the selected example sentence in example sentence table TB1 (step SA5). For example, in the table shown in FIG. 2, when an example sentence with the identifier “001” is selected, the file name “a001” is read.

  Next, the CPU 102 reads out the model data specified by the read file name from the storage unit 105, and outputs the read out model data to the sound processing unit 108 (step SA6). When the model data is input to the sound processing unit 108, the model data that is digital data is converted into an analog signal and output to the speaker 110, and the model sound is reproduced from the speaker 110.

  When the reproduction of the model voice is finished, the CPU 102 controls the display unit 106 to display, for example, a message for prompting pronunciation of the example sentence such as “Press the key to pronounce and then press the key again when the pronunciation is finished” ( Step SA7). The learner listens to the model voice output from the speaker 110, and then operates the input unit 107 according to the message to read the example sentence by imitating the model voice. When the learner pronounces, the learner's voice (hereinafter referred to as learner voice) is converted into a voice signal by the microphone 109, and the converted signal is output to the voice processing unit 108. When the audio signal output from the microphone 109 is input, the audio processing unit 108 converts the audio signal into learner data that is digital data. This learner data is output from the voice processing unit 108 and stored in the storage unit 105.

  When the learner finishes pronunciation and operates the input unit 107 (step SA8; YES), the CPU 102 adjusts the length of the voice indicated by the learner data, the length of the model voice indicated by the model voice data, and the learner. The learner data is processed so that the length of the learner's voice indicated by the data is the same (step SA9). FIG. 5 is a diagram exemplifying the waveform of the model voice and the waveform of the learner voice input to the microphone 107. In FIG. 5, the waveform of the model voice and the waveform of the learner voice are shown when the same example sentence is uttered, but the lengths of the voice waveforms are different because the utterance speeds are different. The CPU 102 analyzes the model voice data and the learner data, and determines the length of the model voice and the length of the learner voice (Δt in FIG. 5). As shown in FIG. 5, when the length of the learner voice is longer by Δt than the length of the model voice, a process of reducing the length of the learner voice by Δt is performed.

  Next, the CPU 102 divides the waveform of the model voice and the waveform of the learner voice into a plurality of frames by dividing them at predetermined time intervals as shown in FIG. Then, the speech waveform of each frame of the model speech and the speech waveform of each frame of the learner speech are associated using a DP (Dynamic Programming) matching method (step SA10). For example, in the waveform illustrated in FIG. 6, the model voice frame A1 is associated with the learner voice frame B1, and the model voice frame A3 is associated with the learner voice frame B4.

  When the association between the model voice and the learner voice ends, the CPU 102 divides each voice waveform for each word pronunciation (step SA11). Specifically, first, for the model voice, the utterance start time is read from the word table TB1. Here, since the example sentence selected by the learner is “The critical region for ecology though is the east end of Panama.”, First, the utterance start time “0.0 sec” of “The” is read from the word table TB2. . As shown in FIG. 6, the CPU 102 adds information indicating the word break (hereinafter referred to as word break information C) to the frame at the position of “0.0 sec” in the speech waveform (frame A1). Next, the CPU 102 reads the utterance time “0.3 sec” of “critical” from the word table TB2, and adds the word break information C to the frame (frame A3) corresponding to the position 0.3 sec after the start of pronunciation.

  After adding the word break information C to the last word “Panama” for the model voice, the CPU 102 then adds the word break information for the learner voice. First, the CPU 102 extracts a frame to which word break information is added in the model voice. Then, a frame corresponding to the extracted frame is specified in the learner's voice, and word break information C is added to the specified frame. For example, when the frame A1 to which the word break information C is added is extracted, since the frame A1 is associated with the frame B1 by the process of step SA10 described above, the CPU 102 determines the frame B1 corresponding to the frame A1. And the word break information is added to the frame B1. Further, when the frame A3 to which the word break information is added is extracted, the frame A3 is associated with the frame B4 by the process of step SA10 described above, so the CPU 102 determines the frame B4 associated with the frame A3. Identify and add word break information C to this frame B4.

  When the CPU 102 adds the word break information to the frame and divides the speech waveform for each pronunciation of the word, first, for the exemplary speech, the utterance time of each word is calculated (step SA12). For example, in the case of the model voice “The”, the voice waveforms from frame A1 to frame A2 are extracted as voice waveforms representing the pronunciation of “The”. Then, the extracted speech waveform is analyzed, and the sound generation time is calculated.

  Next, the CPU 102 calculates the voice pitch and utterance time of each word for the learner voice (step SA13). For example, in the case of “The” of the learner's voice, voice waveforms from frame B1 to frame B3 are extracted as a voice waveform representing the pronunciation of “The”. Then, the extracted speech waveform is analyzed, and the pitch and sounding time of the speech are calculated.

When the calculation of the voice pitch and the utterance time of each word is completed, the CPU 102 displays the pitch and the utterance time for each word as illustrated in FIG. 7 according to the obtained pitch and utterance time (step SA14). ).
In the pitch display part A1 in FIG. 7, the band in which the word is displayed inside represents the pronunciation of the model voice, and the band in which the inside is filled represents the pronunciation of the learner. The vertical position of the band in which the word is displayed is determined according to the pitch stored in association with each word in the word table TB2, and the vertical position of the band in which the inside is filled The position is determined according to the pitch of the learner's voice obtained in step S13. Each band is arranged on the screen according to the pitch of the pronunciation based on a predetermined display position on the screen. When the pitch is high, the band is displayed upward, and when the pitch is low, the band is displayed downward. Is done. For example, the arrangement position of the band in which “The” is displayed is determined in accordance with the pitch stored in association with “The” in the word table TB2, and the learner pronounces “The”. The band to be represented is determined according to the pitch calculated in step S13. Here, when the pitch of the model voice matches the pitch of the learner's voice, a band indicating the pronunciation of the learner is not displayed. For example, when the pitch of the pronunciation of the learner “The” and the pitch of the pronunciation of the model voice “The” are the same, as illustrated in FIG. 7, a band whose interior is filled is not displayed.

  When the pitch of the model voice is higher than the pitch of the learner's voice, for example, as illustrated in FIG. 7, a band in which the word “critical” is displayed, that is, a band indicating the pronunciation of the model voice. Is displayed above the band indicating the pronunciation of the learner's “critical”. When the pitch of the model voice is lower than the pitch of the learner's voice, for example, as illustrated in FIG. 7, a band in which the word “region” is displayed, that is, the pronunciation of the model voice is shown. A band is displayed below the band indicating the pronunciation of the learner's “region”.

Further, in the utterance time display part A2 of FIG. 7, the band in which the word is displayed inside represents the utterance time of the model voice, and the colored band inside represents the utterance time of the learner.
The length of the band representing the length of the utterance time of the model voice is determined according to the utterance time calculated in Step SA12, and the length of the band colored inside is calculated in Step SA13. It is determined according to the utterance time. For example, the length of the band in which “The” is displayed is determined according to the utterance time calculated in step SA12, and the length of the band indicating the pronunciation of the learner “The” is determined in step SA13. It is determined according to the utterance time calculated in (1). Here, when the utterance time of the model voice matches the utterance time of the learner, the band representing the utterance time of the model voice and the band representing the utterance time of the learner have the same length. For example, if the utterance time of the learner “region” is the same as the utterance time of the “region” of the model voice, a band representing the utterance time of the “region” of the model voice as illustrated in FIG. The bands representing the utterance time of the learner's “region” have the same length.

  When the utterance time of the model voice is longer than the utterance time of the learner, for example, as illustrated in FIG. 7, the band in which the word “critical” is displayed inside, that is, the utterance time of the model voice is set. The band shown is displayed longer than the band showing the utterance time of the learner's “critical”. Also, when the utterance time of the model voice is shorter than the utterance time of the learner, for example, as illustrated in FIG. 7, the band in which the word “though” is displayed inside, that is, the utterance time of the model voice is set. The band indicated is displayed shorter than the band indicating the utterance time of the learner's “though”.

  As described above, according to the present embodiment, the pitch of the model voice and the pitch of the learner's voice are displayed together for each word, so that the difference from the model voice can be easily known. The difference between the model voice and the learner's voice can be specifically grasped. In addition, since the voice time of the model voice and the voice time of the learner's voice are displayed together, it is possible to easily know the points different from the model voice, and the voice of the model voice and the learner's voice. The difference from the voice can be grasped specifically.

[Modification]
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. For example, the present invention may be implemented by modifying the above-described embodiment as follows.

  In the embodiment described above, both the pitch and the utterance time are displayed on one screen, but only the pitch or only the pronunciation speed may be displayed, and either the pitch or the pronunciation speed may be displayed. It may be displayed or selected by a learner's operation. Further, in the above-described embodiment, as illustrated in FIG. 8, an icon for notifying that the pronunciation is different from the model voice may be displayed in a portion of the pronunciation different from the model voice.

  In the embodiment described above, the pitch and utterance time are displayed for each word, but the model voice is stored for each phoneme or syllable, and the pitch and utterance for each phoneme are compared with the model voice and the learner's voice. You may make it display time.

  In the above-described embodiment, when there is a silent section in the model voice or the learner voice, the silent section may be represented by a blank band as shown in FIG.

  In the above-described embodiment, as illustrated in FIG. 10A, a band representing the speech time may be displayed in close contact with the adjacent band. Further, in this aspect, when an operation of clicking a band representing the utterance time of a word is performed, the display position of each band may be changed based on the display position of the clicked band. . For example, when the utterance time of each word is displayed as shown in FIG. 10 (a), if an operation of clicking on a band representing the utterance time of the word “region” is performed, the language learning device will As shown in Fig. 4, the left edge position of the "region" band of the model voice is aligned with the left edge position of the "region" band of the learner voice, and the display positions of the other bands are set to "region". You may make it display according to a display position.

  Further, in the above-described embodiment, when the utterance time of the learner's word is significantly longer than the utterance time of the word of the model voice corresponding to the learner's word, the word band before the word is replaced with the word band having the longer utterance time You may make it display away from.

It is the figure which illustrated the hardware constitutions of the language learning apparatus 1 which concerns on embodiment of this invention. It is the figure which illustrated the format of example sentence table TB1. It is the figure which illustrated the format of word table TB2. It is the flowchart which illustrated the flow of the process which CPU102 performs. It is the figure which illustrated the waveform of the model voice, and the waveform of the learner voice. It is a figure when the waveform of an exemplary voice and the waveform of a learner voice are divided into a plurality of frames. 4 is a diagram illustrating a screen displayed on a display unit 106. FIG. In the modification of this invention, it is the figure which illustrated the screen displayed on the display part. In the modification of this invention, it is the figure which illustrated the screen displayed on the display part. In the modification of this invention, it is the figure which illustrated the screen displayed on the display part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Language learning apparatus, 101 ... Bus, 102 ... CPU, 103 ... ROM, 104 ... RAM, 105 ... Memory | storage part, 106 ... Display part, 107 ... Input unit, 108... Voice processing unit, 109... Microphone, 110.

Claims (3)

Storage means for storing an example sentence of the conversation and a pitch of the utterance voice of each word in the example sentence;
Voice input means for inputting voice;
Utterance voice extraction means for extracting utterance voice corresponding to each word in the example sentence from the voice input to the voice input means;
Pitch extraction means for extracting the pitch of each voice extracted by the utterance voice extraction means;
The example sentences stored in the storage means are displayed in horizontal composition, and a plurality of band-like figures representing the utterance voices of the words extracted by the utterance voice extraction means are displayed in horizontal composition, Display means for displaying a word in association with a band-like figure representing the speech of the word;
The vertical display position of each word in the example sentence is determined according to the pitch of the utterance voice of each word stored in the storage means, and the vertical display position of the plurality of band-like figures is determined. A language learning apparatus comprising: display position determining means that determines the pitch according to the pitch of each word extracted by the pitch extracting means. Storage means for storing an example sentence of a conversation and an utterance time of an utterance voice of each word in the example sentence;
Voice input means for inputting voice;
Utterance voice extraction means for extracting utterance voice corresponding to each word in the example sentence from the voice input to the voice input means;
Utterance time extraction means for extracting the utterance time for each utterance voice extracted by the utterance voice extraction means;
The example sentences stored in the storage means are displayed in horizontal composition, and a plurality of band-like figures representing the utterance voices of the words extracted by the utterance voice extraction means are displayed in horizontal composition, Display means for displaying a word in association with a band-like figure representing the speech of the word;
The display length of each word in the example sentence is determined according to the length of the utterance time of the utterance voice of each word stored in the storage means, and the display length of the plurality of band-like figures is determined by the utterance A language learning apparatus, comprising: a display length determining unit that determines according to the length of the utterance time of each word extracted by the time extracting unit. Storage means for storing an example sentence and the pitch of the utterance voice of each phoneme or syllable in the example sentence;
Voice input means for inputting voice;
Utterance voice extraction means for extracting utterance voice corresponding to each phoneme or syllable in the example sentence from the voice input to the voice input means;
Pitch extraction means for extracting the pitch of each phoneme or syllable extracted by the speech voice extraction means;
The example sentences stored in the storage means are displayed in horizontal composition, and a plurality of band-like figures representing the speech sounds of each phoneme or syllable extracted by the utterance voice extraction means are displayed in horizontal composition. Display means for displaying the phoneme or syllable in the middle and a band-like figure representing the utterance voice of the phoneme or syllable;
The display position in the vertical direction of each phoneme or syllable in the example sentence is determined according to the pitch of the utterance voice of each phoneme or syllable stored in the storage means, and the vertical direction of the plurality of band-like figures Display position determining means for determining the display position according to the phoneme extracted by the pitch extracting means or the pitch of each syllable.

Images