JP4797597B2 - Language learning device - Google Patents

Description

  The present invention relates to a technique for showing a learner the difference between a sound as a model and a sound emitted by a learner.

Conventionally, various systems that support language learning have been proposed. For example, Patent Literature 1 discloses a pronunciation learning method for evaluating the quality of pronunciation by comparing an audio signal indicating the pronunciation content of a learner with an audio signal indicating the pronunciation content of a native speaker. According to this method, the learner can obtain an objective evaluation value for his / her pronunciation ability.
JP 2002-40926 A

  However, in the technique described in Patent Document 1, only the evaluation value of pronunciation is shown to the learner, so that the learner can obtain an objective evaluation value for his / her pronunciation ability. I can't know how the pronunciation of the example is different from my pronunciation. For this reason, until a pronunciation that matches the pronunciation of the model can be achieved, it is necessary to carry out a persistent learning that repeats improvement and evaluation of pronunciation by trial and error.

  The present invention has been made under the above-described background, and an object of the present invention is to provide a technique that enables a learner to understand the difference between a learner's voice and a model voice in language learning.

In order to solve the above-described problems, the present invention provides voice input means for outputting input voice as speech voice data, and model voice data indicating the voices of a plurality of words, and the speech duration of each voice of the words First pitch curve generating means for generating a first pitch curve indicating a temporal change in the pitch of the voice from the exemplary voice data including the first word break information indicating the first pitch curve, and a first representing a temporal change in the pitch of the utterance voice data. A second pitch curve generating means for generating a two pitch curve; a frame specifying means for analyzing the model voice data and the uttered voice data in units of a predetermined frame and specifying corresponding frames; and the first pitch curve. First pitch average value specifying means for specifying the average value of the second pitch value as the first average value, and second pitch average value specifying means for specifying the average value of the second pitch curve as the second average value And displaying the first pitch curve and the second pitch curve so as to indicate the pitch height in the vertical direction and the temporal change in the horizontal direction, for each frame specified by the frame specifying means, The first pitch curve and the second pitch curve are displayed so that the pitches of the corresponding frames of the model voice data and the speech voice data are the same in the left-right direction position, and the first pitch curve and the second pitch curve are displayed. The pitch curve is displayed in a positional relationship in which the display position in the vertical direction of the first average value in the first pitch curve matches the display position in the vertical direction of the second average value in the second pitch curve. Corresponding to each word in the speech data based on the display means, the first word break information, and the frame specified by the frame specifying means Specifying means for specifying voice utterance time as second word break information; and for the first pitch curve, specifying one representative value for each section based on the first word break information, and for the second pitch curve, Word pitch specifying means for specifying one representative value for each section based on the second word break information, and a figure corresponding to each word in the exemplary speech data for each word specified by the word pitch specifying means A graphic display for displaying a figure corresponding to each word in the utterance voice data at a position corresponding to the pitch height of each word specified by the word pitch specifying means while displaying the position at a position corresponding to the pitch height. Difference display means for detecting a word having a pitch difference value of a means, the first pitch curve and the second pitch curve being a predetermined value or more and displaying information for identifying the word A selection means for selecting any one of the information displayed by the difference display means according to the operation of the operation means; A storage means for storing the model voice data and the utterance voice data, and an output means for reading out the model voice data or the utterance voice data corresponding to the word indicated by the information selected by the selection means from the storage means and outputting it. In the case of outputting the model voice data, an output unit is provided that outputs the model voice data by performing a time stretch process so as to have the same utterance time as the utterance time of the utterance voice data. A language learning device is provided.

In a preferred aspect of the present invention, there is provided a detecting means for detecting a different portion between the first pitch curve and the second pitch curve, and the display means identifies and displays the different portion detected by the detecting means. And

  According to the present invention, in language learning, a learner can grasp a difference between a learner's voice and a model voice.

<A: Configuration>
FIG. 1 is a block diagram illustrating a hardware configuration of a language learning device 1 according to an embodiment of the invention. In the figure, reference numeral 11 denotes a control unit including an arithmetic device such as a CPU (Central Processing Unit) and various memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). Reference numeral 12 denotes a storage unit composed of a large-capacity storage device such as a hard disk. The arithmetic unit of the control unit 11 controls each unit of the language learning device 1 via the bus 13 by reading and executing a computer program stored in a memory such as a ROM or the storage unit 12.

  Reference numeral 14 denotes a display unit composed of, for example, a liquid crystal display, and displays a character string, various messages, a menu screen for operating the language learning device 1 and the like under the control of the control unit 11. An input unit 15 includes an input device such as a keyboard and a mouse, and outputs a signal corresponding to the operation content to the control unit 11 in response to a key press or a mouse operation. Reference numeral 16 denotes a microphone that collects sound, and reference numeral 17 denotes an audio processing unit. The microphone 16 is connected to the sound processing unit 17, and the sound processing unit 17 converts the sound input from the microphone 16 (hereinafter referred to as learner sound) into digital data (hereinafter referred to as learner data). To the control unit 11. Reference numeral 18 denotes a speaker connected to the sound processing unit 17 and outputs a sound corresponding to a signal output from the sound processing unit 17.

  The storage unit 12 of the language learning device 1 stores a table TBL1. FIG. 2 is a diagram illustrating an example of the structure of the table TBL1. In this table, items of “example sentence text data”, “exemplary speech data”, and “word break information” are stored in association with each other as shown in the figure. Among these items, the “example sentence text data” item stores text data representing example sentences used for language learning. In the “model voice data” item, WAVE-format digital data (hereinafter referred to as model voice data) representing voice (hereinafter referred to as model voice) when the native speaker reads out the example sentence is stored. In the item “word break information”, information indicating the utterance time of each word in the model voice (hereinafter referred to as first word break information) is stored.

  Next, a screen displayed on the display unit 14 by the control unit 11 of the language learning device 1 will be described. FIG. 3 is a diagram illustrating an example of a screen displayed on the display unit 14 of the language learning device 1. As shown in the figure, this screen is divided into a pitch display portion 100 in the upper part of the screen, a rhythm display part 200 in the middle part of the screen, and an operation button display part 300 in the lower part of the screen.

The pitch display part 100 is a part that displays the temporal change of the pitch (pitch) of the model voice and the learner voice in association with each other. The pitch display portion 100 is further divided into a radio button display portion 110 and a pitch graph display portion 120.
In the radio button display portion 110, a radio button 111 for switching the contents displayed in the pitch graph display portion 120 is displayed. The radio button 111 can select either “graph display” or “word display”. When the learner performs an operation such as moving the mouse pointer to the area where the radio button 111 is displayed and left-clicking, the control unit 11 switches the content displayed on the pitch graph display portion 120.
When “graph display” is selected in the radio button 111, the pitch graph display portion 120 has a graph (pitch curve) showing temporal changes in the pitches of the model voice and the learner voice as shown in FIG. Is displayed. On the other hand, when “word display” is selected in the radio button 111, the pitch graph display portion 120 displays a time change of the pitch of the model voice and the learner voice for each word as shown in FIG. A strip-shaped figure is displayed.

Next, the rhythm display portion 200 is a portion that displays the rhythm (utterance time) of the model speech and the learner speech in association with each other. The rhythm display portion 200 is further divided into a radio button display portion 210 and a rhythm graph display portion 220.
In the radio button display part 210, a radio button 211 for switching the graphic displayed in the rhythm graph display part 220 is displayed. The radio button 211 can select any one of “word comparison”, “same size comparison”, and “full text comparison”.
When “word comparison” is selected in the radio button 211, the rhythm graph display portion 220 displays the utterance time of each word in the utterance time of the entire example sentence of the model voice and the learner voice as shown in FIG. A band-like figure for comparing the ratio is displayed.
When “same comparison” is selected, the rhythm graph display portion 220 displays a graphic for comparing the absolute lengths of pronunciation times of the words. Specifically, as shown in FIG. 5, a band-like figure indicating the utterance time of each word in the model voice and the learner voice is displayed.
Further, when “full text comparison” is selected, a band-like figure indicating the utterance time of the entire example sentence of the model voice and the learner voice as shown in FIG. 4 is displayed.

  As described above, the display contents of the pitch graph display part 120 and the rhythm graph display part 220 can be appropriately changed by the radio buttons 111 and 211, and these display contents are displayed by combining arbitrary ones. It is possible to make it.

Next, in the operation button display part 300 in FIG. 3, various buttons for causing the learner to perform various operations such as a recording operation and a reproduction operation are displayed. In the operation button display portion 300 shown in FIG. 3, reference numeral 301 denotes a button for inputting a voice input start instruction or end instruction. Reference numeral 302 denotes a button for inputting an instruction to reproduce a model voice. Reference numeral 303 denotes a radio button for selecting a playback speed in playback of the model voice, and one of “normal playback”, “slow playback”, and “according to recording” can be selected. When “synchronize with recording” is selected, the model voice that has been subjected to the time stretch processing so as to have the same utterance time as the utterance time of the recorded example sentence of the learner voice is reproduced. When “slow playback” is selected, the model voice subjected to the time stretch process at a predetermined ratio is played back.
A button 304 is used to input a playback instruction for the recorded learner voice. Reference numeral 305 denotes a button for inputting a learner's voice scoring instruction, and reference numeral 306 denotes an evaluation result display portion that displays an evaluation result for the learner's pronunciation voice. When detecting that the button 305 has been clicked, the control unit 11 compares the model voice data with the learner data, calculates a score using a predetermined algorithm based on the degree of coincidence, and evaluates the calculated score. It is displayed on the display portion 306.
Reference numeral 307 denotes a button for inputting a scoring detail display instruction. When detecting that this button has been clicked, the control unit 11 displays details of the scoring result. For example, the scoring results of a plurality of items such as “pitch” and “rhythm” may be displayed for each item, or the scoring results for each word included in the example sentence may be displayed for each word. .

Reference numeral 308 denotes a button for instructing voice output of a word in which the difference is remarkable in the difference between the model voice and the learner voice. When it is detected that the button has been clicked, the control unit 11 of the language learning device 1 reproduces each word having the most significant difference in pitch comparison between the model voice and the learner voice. Furthermore, the difference is most remarkable in the rhythm comparison, and each word is also reproduced.
The above is the description of the screen displayed on the display unit 14.

<B: Operation>
Next, the operation of this embodiment will be described with reference to FIG.
<B-1: Pitch graph display processing>
FIG. 6 is a flowchart showing a flow of processing performed by the control unit 11 of the language learning device 1. First, the learner operates the input unit 15 of the language learning device 1 to input an instruction to display a list of example sentences. When the control unit 11 of the language learning device 1 detects that an instruction to display a list of example sentences is input, the example sentence text data stored in the table TBL1 is read (step SA1), and the list of example sentences represented by the read data is displayed. Is displayed on the display unit 14 (step SA2). Thereafter, when the learner operates the input unit 15 and performs an operation of selecting one of the displayed example sentences (step SA3; YES), the control unit 11 includes a screen displayed on the display unit 14, Based on the signal sent from the input unit 15, the selected example sentence is specified (step SA4). When the selected example sentence is specified, the control unit 11 reads out model voice data stored in association with the selected example sentence in the table TBL1 (step SA5).

  Next, the control unit 11 extracts the pitch of the voice indicated by the read exemplary voice data, and generates a pitch curve indicating the temporal change of the pitch (step SA6). Specifically, the pitch curve is generated as follows. First, the control unit 11 divides the voice indicated by the read exemplary voice data at a predetermined time interval (for example, 100 msec) on the reproduction time axis (hereinafter, each divided section is referred to as a frame). Next, the control unit 11 extracts the pitch of the voice of each section for each divided section. When a pitch is extracted for each divided frame, a pitch curve connecting the pitches obtained for each frame is generated (hereinafter, this pitch curve is referred to as a first pitch curve), and a curve indicating the generated first pitch curve is generated. Data is stored in the storage unit 12.

  Subsequently, the control unit 11 controls the display unit 14 to display, for example, a message prompting the pronunciation of the example sentence, such as “Press the key to generate a sound and then press the key again when the sound is finished” (step SA8). .

  The learner operates the input unit 15 according to the message displayed on the display unit 14 and reads the example sentence. When the learner pronounces, the learner's voice is converted into a voice signal by the microphone 16, and the converted signal is output to the voice processing unit 17. The voice processing unit 17 converts the voice signal output from the microphone 16 into digital data to obtain learner data. The learner data is output from the voice processing unit 17 and stored in the storage unit 12.

  Next, the control unit 11 monitors a signal sent from the input unit 15 and determines whether or not the learner has finished pronunciation. When it is detected that the learner has finished sounding and operated the input unit 15 (step SA9; YES), the control unit 11 directly compares the waveforms of both the model voice data and the learner data, for example, The temporal correspondence between the model voice data and the learner voice data is performed for each frame by DTW (Dynamic Time Warping) or the like, and each of the learner data in the learner data is based on the result of this correspondence and the first word break information. Information indicating the utterance time of the word (hereinafter referred to as second word break information) is specified (step SA10). For example, as shown in FIG. 7, two waveforms of a model voice and a learner voice are associated with each other.

  Subsequently, the control unit 11 of the language learning device 1 performs a process similar to that in step SA6 on the learner data, extracts the pitch of the learner data, and then a pitch curve (hereinafter referred to as a second pitch curve). And the data indicating the generated second pitch curve is stored in the storage unit 12 (step SA11).

  Next, the control part 11 calculates the average value of the pitch of the model audio | voice data memorize | stored in the memory | storage part 12 (step SA12). Specifically, for example, an average value of pitch values for each frame divided in step SA6 is calculated. Further, the control unit 11 performs the same process on the learner data stored in the storage unit 12 and calculates the average value of the pitch of the learner data (step SA13).

  Next, the control unit 11 controls the display unit 14 to display the selected example sentence, the first pitch curve, and the second pitch curve on the pitch graph display part 120 as illustrated in FIG. SA14). At this time, the control unit 11 expands or contracts the second pitch curve for each frame in the time axis direction so that the utterance time of each frame in the learner data matches the utterance time of the corresponding frame in the first pitch curve. Expand and display. Further, the control unit 11 has a positional relationship in which the display position of the average pitch value in the first pitch curve matches the display position of the average pitch value in the second pitch curve, and the first pitch curve and the second pitch curve. Is displayed.

  In the pitch graph display portion 120 shown in FIG. 3, a curve indicated by a chain line represents a first pitch curve, and a curve indicated by a solid line represents a second pitch curve. As shown in the figure, the vertical display position of the first pitch curve and the second pitch curve is adjusted so that the average value of both pitches coincides with the zero point of the vertical axis (for example, the position of the horizontal axis). . Thereby, the learner can visually grasp the difference in the temporal change (intonation) of the pitch between the model voice and his / her pronunciation.

  Moreover, the control part 11 extracts the location from which pitch differs in a model audio | voice and a learner audio | voice, and displays the icon which shows the extracted location. This extraction process is performed, for example, by calculating a difference value between pitch values at corresponding positions between the first pitch curve and the second pitch curve and specifying a position where the calculated difference value is equal to or greater than a predetermined value. Then, as shown in FIG. 3, the control unit 11 displays the icons I1 and I2 that notify that the pitch of the pronunciation is different from that of the model voice in a portion where the pitch difference between the two is equal to or larger than a predetermined value, and the second The part of the pitch curve is displayed by changing the color or changing the thickness. In FIG. 3, the corresponding part in the second pitch curve is indicated by a thick solid line.

At this time, the learner can operate the input unit 15 to perform an operation for outputting the difference portion as a sound. This is performed, for example, by an operation such as moving the mouse pointer to the area where the displayed icons I1 and I2 are displayed and left-clicking. When the control unit 11 of the language learning device 1 detects that an operation for outputting a voice is performed, for the word including the selected difference, the voice data of the word part in the model voice and the learner voice The voice data of the word part is sequentially read from the storage unit 12 and supplied to the voice processing unit 17. Thereby, both voices are sequentially output via the speaker 18.
For example, in FIG. 3, when the icon I1 is left-clicked, the control unit 11 sequentially transmits the voice “One” in the model voice and the voice “One” in the learner's voice through the speaker 18. Output. Note that the unit for outputting the sound may not be for each word. For example, the sound of the section including the words before and after the problematic word may be output. In this way, it is possible to understand the difference in pronunciation in a series of word flows.

By the way, when the pitch curve of the learner voice and the pitch curve of the model voice are displayed, when the difference between the utterance time of the learner and the utterance time of the model voice is large, as shown in FIG. When the pitch curves are displayed side-by-side or up and down, it is possible for the learner to visually grasp which part of the pitch curve corresponds and which part of his pronunciation and model voice is different. It was difficult.
Furthermore, even if these pitch curves are displayed superimposed on the same scale, the average pitch of the model voice and the learner voice is generally different. It is difficult to grasp the part. In particular, when the gender of the model voice and the learner are different, the average pitch of the model voice and the average pitch of the learner voice are very different, and how it is specifically corrected even if two pitch curves are superimposed. It was difficult for learners to figure out what to do.

  In contrast, in the present embodiment, as shown in FIG. 3, the first pitch curve and the second pitch curve are displayed in a positional relationship in which the display positions of the average values of both pitches coincide with each other. It is easy to visually grasp the difference of intonation between the pronunciation of a person and the pronunciation of a model voice.

  Further, the learner data is displayed by expanding / contracting or expanding the second pitch curve for each frame so that the utterance time of each frame in the learner data matches the utterance time of the corresponding frame of the first pitch curve. Even if there is a large difference in duration between the utterance time and the speech time of the model voice, you can visually understand which part of the graph corresponds to which part ( It is possible to easily know which word (pitch) (intonation) should be corrected.

  Also, by displaying icons with different pitches from the model voice, the learner can visually grasp the difference between the model and his pronunciation, and which part of the pronunciation should be corrected. It can be easily grasped. Furthermore, the words at the different points are output as voices for each of the model voice and the learner voice, so that the learner can easily grasp the word to be corrected and the details of the correction.

<B-2: Rhythm word ratio display processing>
Returning to the description of FIG. The control unit 11 of the language learning device 1 displays a strip-like figure indicating a rhythm on the rhythm graph display portion 220 as shown in FIG. 3 together with the display of the pitch graph (step SA15). Specifically, a band-shaped figure showing the ratio of the utterance time of each word in the utterance time of the entire example sentence of the model voice, and a band-shaped figure showing the ratio of the utterance time of each word in the utterance time of the entire example sentence of the learner voice Are displayed on the rhythm graph display portion 220.
In the rhythm graph display unit 220 shown in FIG. 3, the band in which the word is displayed represents the ratio of the pronunciation time of each word in the utterance time of the entire example sentence of the model voice. It represents the ratio of the pronunciation time of each word in the utterance time of the entire example sentence.
The length of the band in which the word is displayed is determined according to the utterance time (first word break information) for each word stored in the table TBL1, and the length of the band in which the inside is filled is This is determined according to the utterance time (second word break information) obtained in step SA10. Specifically, the length of each band corresponds to the ratio of the utterance time of the word. When the ratio of the utterance time is large, the band is displayed long, and when the ratio of the utterance time is small, the band is displayed short.
Further, the control unit 11 extracts a portion where the rhythm (word utterance time) differs between the model voice and the learner voice, and displays an icon I3 in the vicinity of the extracted portion as shown in FIG.

  Thus, by displaying the ratio of the utterance time of each word for each of the model voice and the learner voice, it is easy to visually understand the difference between the rhythm (word utterance time) of the learner voice and the model voice. Thus, it becomes easy for the learner to visually know which word utterance time (pronunciation speed) should be corrected.

<B-3: Pitch word graph display process>
Next, the display process of the pitch word graph in the pitch graph display portion 120 will be described below.
When the learner operates the input unit 15 and selects “word display” of the radio button 111 in a state where the screen shown in FIG. 3 is displayed on the display unit 14, the control unit of the language learning device 1 11 detects the operated content and switches the display of the pitch graph display portion 120 to the pitch word graph display. Specifically, first, the control unit 11 specifies, for each word in the model voice data, the highest pitch value in the word. Next, for each word in the learner data, the highest pitch value in that word is specified. And the control part 11 displays the zone | band which shows the utterance time of each word as shown in FIG.

  In the pitch graph display portion 120 of FIG. 4, the band in which the word is displayed represents the pronunciation of the model voice, and the shaded band represents the pronunciation of the learner. The arrangement position in the vertical direction of the band in which the word is displayed is determined according to the maximum pitch value for each word. Specifically, each band is arranged on the screen according to the pitch of the pronunciation with reference to a predetermined display position on the screen. When the pitch is high, the band is displayed upward, and when the pitch is low, the band is displayed. Is displayed in the downward direction. Further, in this case, each band is displayed in a positional relationship in which the display position of the average value of the pitch of the model voice data matches the display position of the average value of the pitch of the learner data.

  In this way, for each word, by displaying a band-like figure at a position corresponding to the pitch of the word, the learner can visually identify which word pitch is shifted from the pitch of the model pronunciation. It is possible to present it automatically.

<B-4: Rhythm equal magnification comparison display processing>
Next, the rhythm equal magnification comparison display process will be described. When the learner operates the input unit 15 and selects “same size comparison” of the radio button 211 in a state where the screen shown in FIG. 3 is displayed on the display unit 14, the control of the language learning device 1 is performed. The unit 11 switches the display of the rhythm graph display part 220 to the rhythm equal magnification comparison display. Specifically, the control unit 11 displays a band-like figure indicating the utterance time of each word in the model voice data and the learner data, as shown in FIG. In the rhythm graph display portion 220 shown in FIG. 5, the band in which the word is displayed represents the utterance time of each word in the model voice, and the band shown in shaded represents the utterance time of each word in the learner's utterance voice. Represents.

  At this time, the learner operates the input unit 15 while checking the screen displayed on the display unit 14, and shows the utterance time of the word in the learner's voice displayed on the rhythm graph display unit 220. Can be selected. This is performed, for example, by an operation such as moving the mouse pointer to the area where the band is displayed and left-clicking. The control unit 11 of the language learning device 1 changes the display position of the selected band by performing the processing shown in steps SA20 and SA21 in FIG. This display position changing process will be described below with reference to FIG.

  First, when the control unit 11 detects that an operation for selecting a band has been performed (step SA20; YES), it is the same word as the word represented by the selected band and is included in the model voice. The drawing position at the left end of the band representing the utterance time (hereinafter referred to as the left end coordinates) is acquired. For example, when the band B1 is selected in the example shown in FIG. 5, the left end coordinate of the band B2 is acquired. Then, the control unit 11 changes the left end coordinate of the selected band to the acquired left end coordinate and redraws the band (step SA21). The example shown in FIG. 8 shows a screen displayed when a band corresponding to the word “ten” is selected.

  In the screen shown in FIG. 5, it can be seen at a glance which utterance time is longer, the utterance time of the word “ten” by the model voice and the utterance time of the word “ten” by the pronunciation of the learner. It was difficult to grasp. On the other hand, in the example shown in FIG. 8, since the drawing position at the left end of the band corresponding to the word “ten” is the same, it is easy to grasp which word has a longer utterance time at a glance. It becomes possible. Thereby, the learner can visually grasp whether the pronunciation of the word should be lengthened or shortened.

<B-5: Rhythm full text comparison display processing>
Next, the rhythm full text comparison display process will be described. When the operation of selecting “full text comparison” of the radio button 211 is performed on the screen illustrated in FIG. 3, the control unit 11 of the language learning device 1 indicates the utterance time of the entire example sentence of the exemplary speech as illustrated in FIG. 4. A band-shaped figure and a band-shaped figure indicating the utterance time of the entire example sentence of the learner's voice are displayed on the rhythm graph display portion 220. As a result, the learner can visually grasp the difference in pronunciation time of the entire sentence between the model voice and the own voice.

  As described above, in this embodiment, in language learning, the intonation (temporal change in pitch) and rhythm in the learner's pronunciation are visualized and displayed in a form that is easy to compare with the intonation and rhythm of the model. This makes it possible for the learner to recognize what should be done to bring his pronunciation closer to the model pronunciation.

<C: Modification>
As mentioned above, although embodiment of this invention was described, this invention is not limited to embodiment mentioned above, It can implement with another various form. An example is shown below.
(1) In the above-described embodiment, both pitch and rhythm are displayed on one screen. However, only the pitch or only the rhythm may be displayed. It may be displayed or selected by a learner's operation.

(2) In the above-described embodiment, language data indicating model speech when the example sentence is pronounced, word break information indicating the utterance time of each word in the model voice, and text data indicating the example sentence Although it was made to memorize | store in the memory | storage part of a learning apparatus, such a data may not be memorize | stored in a memory | storage part beforehand, but a learner may operate the input part 15 and may input it.

(3) In the embodiment described above, in the pitch display or rhythm display, a band-like figure corresponding to each word (or the entire example sentence) is displayed on the display unit. The figure corresponding to the word (or the entire example sentence) is not limited to a band-like figure, and for example, a plurality of circular figures may be connected and displayed as shown in FIG. Alternatively, a figure indicating the start and end of each word (or the entire example sentence) (for example, a figure imitating a person's face (a face character)) is displayed at the corresponding location, A connected graphic may be displayed to represent each word (or the entire example sentence). In short, any figure may be used as long as it corresponds to each word (or the entire example sentence).

(4) In the above-described embodiment, the model audio data stored in the storage unit 12 is WAVE format data. However, the data format is not limited to this, and any data indicating audio can be used. Data in such a format may be used.
In the above-described embodiment, the model voice data and the learner data are digital data, but analog data may be used.

(5) In the embodiment described above, the language learning device 1 performs the pitch curve generation process, the pitch curve display process, and the like according to the present invention. On the other hand, two or more devices connected by a communication network share the functions according to the above-described embodiment, and a system including the plurality of devices realizes the language learning device 1 of the same embodiment. Good.
For example, as a system in which a computer device including a microphone, a speaker, a display device, an input device, and the like and a server device that stores exemplary voice data and performs comparison processing between the exemplary voice data and the voice data are connected via a communication network. It may be configured. In this case, the computer device converts the sound input from the microphone into sound data and transmits it to the server device, compares the sound data received by the server device with the model sound data, and generates a pitch curve, etc. Then, it may be transmitted to the language learning device 1.

(6) Programs executed by the control unit 11 of the language learning device 1 in the above-described embodiment are a magnetic tape, a magnetic disk, a floppy (registered trademark) disk, an optical recording medium, a magneto-optical recording medium, and a CD (Compact Disk). -It can be provided in a state stored in a recording medium such as a ROM, a DVD (Digital Versatile Disk), or a RAM.

It is a block diagram which shows an example of the hardware constitutions of the language learning apparatus which is 1st Embodiment of this invention. It is a figure which shows an example of the data structure of the table memorize | stored in the memory | storage part of the embodiment. It is a figure which shows an example of the screen displayed on the display part of the embodiment. It is a figure which shows an example of the screen displayed on the display part of the language learning apparatus which is 2nd Embodiment of this invention. It is a figure which shows an example of the screen displayed on the display part of the embodiment. It is a flowchart which shows the flow of the process which the control part of the embodiment performs. It is a figure which shows an example of the waveform of an exemplary audio | voice, and the waveform of a learner audio | voice. It is a figure which shows an example of the screen displayed on the display part of the embodiment. It is a figure which shows an example of a pitch graph. It is a figure which shows an example of the screen which concerns on the modification of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Language learning apparatus, 11 ... Control part, 12 ... Memory | storage part, 13 ... Bus, 14 ... Display part, 15 ... Input part, 16 ... Microphone, 17 ... Audio | voice processing part, 18 ... Speaker.

Claims (2)

Voice input means for outputting input voice as speech voice data;
A first pitch curve indicating temporal change in pitch of speech from exemplary speech data including speech of each word and including first word delimiter information indicating speech time of each speech. First pitch curve generating means for generating;
Second pitch curve generation means for generating a second pitch curve representing a temporal change in pitch of the speech data;
Analyzing the exemplary voice data and the spoken voice data in predetermined frame units, and specifying a frame corresponding to both;
First pitch average value specifying means for specifying an average value of the first pitch curve as a first average value;
Second pitch average value specifying means for specifying an average value of the second pitch curve as a second average value;
The first pitch curve and the second pitch curve are displayed so that the pitch level is indicated in the vertical direction and the temporal change is indicated in the horizontal direction, and for each frame specified by the frame specifying means, the model voice is displayed. The first pitch curve and the second pitch curve are displayed so that the pitches of the corresponding frames of the data and the speech voice data are the same in the left-right direction, and the first pitch curve and the second pitch curve are displayed. Is displayed in a positional relationship in which the vertical display position of the first average value on the first pitch curve matches the vertical display position of the second average value on the second pitch curve. When,
Specifying means for specifying speech utterance time corresponding to each word in the utterance voice data as second word break information based on the first word break information and the frame specified by the frame specifying means;
For the first pitch curve, one representative value is specified for each section based on the first word break information, and for the second pitch curve, one representative value is specified for each section based on the second word break information Word pitch identification means to
A graphic corresponding to each word in the exemplary speech data is displayed at a position corresponding to the pitch of each word specified by the word pitch specifying means, and a graphic corresponding to each word in the speech audio data Graphic display means for displaying at a position corresponding to the pitch of each word specified by the word pitch specifying means ,
A difference display means for detecting a word having a difference value of a pitch between the first pitch curve and the second pitch curve being a predetermined value or more and displaying information for specifying the word;
An operation means operated by an operator;
A selection unit that selects one of a plurality of pieces of information displayed by the difference display unit according to an operation of the operation unit;
Storage means for storing the model voice data and the utterance voice data;
Output means for reading out the model voice data or the speech voice data corresponding to the word indicated by the information selected by the selection means from the storage means and outputting the model voice data, A language learning apparatus comprising: output means for performing time-stretch processing on the exemplary voice data so as to have the same utterance time as the utterance time of the utterance voice data . Detecting means for detecting a difference between the first pitch curve and the second pitch curve;
The language learning apparatus according to claim 1, wherein the display unit identifies and displays a different portion detected by the detection unit.

Images