JP5580019B2 - Language learning support system and language learning support method - Google Patents

Description

  The present invention relates to a language learning support system and a language learning support method for acquiring a foreign language, and more particularly to a language learning support system and a language learning support method for improving the conversational ability of a foreign language.

  In today's globalized society, learning foreign languages is becoming increasingly important. In particular, the demand for acquiring conversation skills necessary for communication with foreigners is quite high. Many language schools provide services, and various educational materials are available on the market that allow learners to learn alone. Yes. However, there are quite a few cases where foreign languages cannot be spoken sufficiently despite years of studying foreign languages. Such problems have been pointed out in the past, and various methods have been proposed as learning methods effective for improving the foreign language, particularly foreign language conversation skills.

In general, it is said that the following four abilities must be comprehensively improved to improve foreign language conversation skills.
(1) Ability to listen to the sound itself (2) Ability to connect the heard sound and meaning (or concept) (3) Ability to speak the sound itself Ability to connect meaning (or concept)

  The utterance ability here is not just a word-based “pronunciation” ability, but “speaking = speech” which is felt natural for the native speaker of the language in the actual scene of everyday conversation. Linguistically called prosody (prosody), intonation (intonation), rhythm (rhythm), stress (strength), pitch (high and low), tone (tone), tempo, voice size, vocalization speed, etc. , Refers to various phonetic features. In English education at elementary schools, voice (pronunciation) education that emphasizes this prosody aspect has also been proposed.

  “Shadowing”, “reproduction”, etc. are effective training methods for improving the above-mentioned abilities, based on past research and actual educational results.

  For example, “shadowing” disclosed in Patent Documents 1 and 2 is a method of uttering a sound heard by a trainee while listening to pronunciation by a native speaker or the like and covering it. This is a training method for those who aim to become professional simultaneous interpreters. To improve the above four abilities, especially the abilities (1) and (3) above, by shadowing. It is said to be effective. However, shadowing cannot be performed without a certain level of syntax knowledge and meaning, so it is not suitable for beginners and beginners, and there are many cases where appropriate effects cannot be obtained.

  In addition, shadowing means that you must listen to the voice of your native speaker (hereinafter referred to as the “original text”) while listening to your utterance (the sound you hear through your bones). It imposes a cognitively burdensome task. As a result, the experiments by the inventors of the present application indicate that, in particular, at the initial stage of learning a completely unknown language after adulthood, it is possible that the original text may be listened to and the quality of the recitation utterance itself may be reduced. I know.

  On the other hand, “reproduction” is a training method in which a learner speaks by imitating it immediately after listening to a certain length of original text. In general reproduction, the original text is divided into sentences (sentences having a meaning of a group) and the sound is temporarily stopped. Therefore, the length of the original text varies, and in some cases, it may become considerably long.

  However, according to experiments conducted by the inventors of the present application, the longer the original text is, the more likely it is that the reproducibility of the original text (accurate quality of repeated utterances) by the learner tends to be lost and the quality of the original text decreases. It has been found. This is considered to be related to the theory of human short-term memory for verbal speech, which is disclosed in Non-Patent Document 1 and the like. For these reasons, the conventional re-production is not necessarily an efficient learning method.

JP 2003-241629 A JP 2003-307997 A

Miller GA, “The Magical Number Seven, Plus or Minus Two: Some Limits, Some Limits on our capacity for processing information), Psychological Review, Volume 63, p. 81-97 “[Chunk] Speaking ability is determined by“ small English ”” [online], Eigoriki.net, [searched September 23, 2009], Internet <http://eigoriki.net/2008/04/post-9 .html>

As mentioned above, although various methods have been proposed to improve foreign language conversation skills, they do not necessarily take into account the characteristics of human memory, but as an efficient learning method. It has not been established.
The present invention has been made in view of these points, and an object of the present invention is to provide a language learning support system and a language learning support method capable of performing efficient training especially when learning a foreign language conversation. It is to be.

The first invention made to solve the above-mentioned problem is that the learner listens to the reproduced voice output from the voice output means based on the original voice of the language to be learned, and utters according to the reproduced voice. A language learning support system for training
a) sentence dividing means for acquiring a sentence of the original speech of the language to be learned and dividing it into chunks to obtain an original voice chunk;
b) a native speech chunk acquisition means for acquiring a native speech chunk having a meaning corresponding to each of the original speech chunks in the learner's native language;
c) echo adding means for performing echo addition processing such that each of the original voice chunk and the native voice chunk is repeated a predetermined number of times with a delay time at which a repeated sound can be heard independently during reproduction;
d) Elapsed time so that the original speech chunk and the native speech chunk of the learning target language, each echo-added, are alternately output in time in units of chunks and at a predetermined time interval. Data creating means for creating first audio data in which the output timing associated with is controlled;
And the first sound data created by the data creating means is outputted from the sound output means.

Further, the second invention made to solve the above problem is that the learner listens to the reproduced sound output from the sound output means based on the original sound of the language to be learned, and utters the sound according to the reproduced sound. A language learning support method for training conversation,
a) a sentence division step of obtaining a sentence of the original speech of the language to be learned and dividing it into chunks to obtain an original voice chunk;
b) a native speech chunk acquisition step of acquiring a native speech chunk having a meaning corresponding to each of the original speech chunks in the learner's native language;
c) an echo adding step for adding an echo that repeats a predetermined number of times with a delay time that allows each of the repeated sounds to be heard independently during playback, respectively, for the original speech chunk and the native speech chunk;
d) The time passes so that the original speech chunk and the native speech chunk of the learning target language that are echoed are alternately output in time in units of chunks and at a predetermined time interval. A data creation step for creating first audio data with controlled output timing;
The first audio data created by the data creation step is output from the audio output means.

  As is well known to those skilled in the art, a chunk is a unit of utterance output composed of one or more words having a shorter meaning than a sentence in conversation or the like (see, for example, Non-Patent Document 2). In the language learning support system according to the first invention of the present application, the sentence dividing means executes silence detection or the like using speech waveform analysis for a given learning target language, for example, one sentence of the original English speech. , Divide the sentence into multiple chunks. In many cases, such chunk division is possible automatically, but depending on the sentence or the characteristics of the speaker's way of speaking, proper chunk division may not be possible. In such a case, the worker may evaluate the result of the chunk division that is automatically executed, and may manually correct the result if it is not appropriate.

  On the other hand, the native language voice chunk acquisition means, for example, takes in the native language voice uttered by a translator or the like for each original voice chunk and stores it for each chunk. Since the pronunciation of the native language speech itself is not a learning target, the native language speech may be synthesized speech by a computer or the like.

  The echo adding means executes an echo adding process on each of a plurality of original speech chunks constituting a sentence and a plurality of native speech chunks corresponding thereto. Here, the number of echo repetitions is not particularly limited, but the time delay is set to such a time delay that each sound can be heard independently. Moreover, it is preferable that the time length of the voice chunk after the echo addition process is within about 3 to 5 seconds. This is a time range in consideration of the length capable of short-term memory of accurate speech. According to past research such as Non-Patent Document 1, it is said that the limit of general short-term memory is about 15-20 seconds, but this is when the meaning and concept of the memory object is understood. If you don't understand or lack that understanding, the time you can remember is considerably reduced. According to the experiments conducted by the inventors of the present application, it is only within a few seconds that it is possible to keep in memory, particularly in the early stage of learning a completely unknown language after adulthood. Therefore, in such learning, it is preferable to set the time length to about 3 to 5 seconds.

  The data creation means rearranges the original speech chunk subjected to the echo addition processing as described above and the native speech chunk semantically corresponding to the chunk so that the chunks are alternately output in time and one chunk. The first audio data is created by inserting a silent period so that a predetermined time interval is left between the end time of the sound output of the next and the start time of the sound output of the next chunk, and this is stored in, for example, a storage device To do. The learner reads out the audio data stored in the storage device by the audio output means and reproduces / listens. In the reproduced sound, for example, the meaning of one chunk is repeated a predetermined number of times in Japanese as a native language, and then one chunk is repeated a predetermined number of times in English as a learning target language.

  Therefore, even if the learner does not fully understand the meaning of the chunk, the learner can listen to the meaning and the English voice alternately at a good tempo. In addition, the playback time length of each chunk is within the time that humans can easily memorize, and the audio output gradually decreases during the playback time, so the learner uses the stored English voice as the playback voice. It can be easily spoken to cover. As a result, it is possible to promote the understanding of meanings while training English speech.

  In one aspect of the language learning support system according to the first aspect of the present invention, the data creation means may be configured to store the original speech chunk and the native speech chunk in different speech channels. The different audio channels here are usually the left (L) channel and the right (R) channel of stereo audio. According to this, when the audio output means can output the left and right channels independently, for example, it is possible to listen only to the language to be learned, or conversely to listen only to the native language speech.

In the language learning support system according to the first invention and the language learning support method according to the second invention, in addition to the reproduction output of the first audio data, the reproduction output of other forms of audio data is performed in order to enhance the learning effect. You may be able to do it.
For example, preferably, the data creation means is configured so that the original voice chunk that has not been subjected to the echo addition process is intermittently output with a time interval that allows the learner to speak at least between each chunk. The second audio data in which the output timing with the passage of time is controlled may be created, and the first audio data and the second audio data may be selectively reproduced and output from the audio output means.

  In this configuration, the data creation means is intermittent with a time interval that allows the learner to utter at least for each chunk with respect to the original speech chunk that has been divided as described above and has not been subjected to echo addition processing. The second audio data in which the output timing with the passage of time is controlled so that the audio is output is generated and stored in, for example, a storage device. The learner reads out the second audio data stored in the storage device by the audio output means and reproduces / listens. Usually, the length of the voice of one chunk that has not been subjected to echo addition processing is about 1 second, so that the learner repeats it while listening to the original voice chunk that is played back intermittently. You can repeat the voice.

  In this case, the data creation means controls the output timing so that the original voice chunk is output in accordance with the tempo of a reference sound (for example, a metronome sound) emitted at regular time intervals, and the reference sound is output to the sound. It is recommended to synthesize. By determining the output timing so that the original sound chunk is output in time with the reference sound and the reference sound is generated even during the time interval in which the learner can speak, the learner can You can repeatedly utter each chunk while taking the rhythm. In particular, since it is easy to grasp the timing of utterance by the reference sound, the utterance is promoted, and the effect of training to speak English speech can be further enhanced. Furthermore, by generating a reference sound that promotes utterance, a forcing force in a psychologically good sense acts, and a positiveness unintended by the learner is born, and the learning effect can be enhanced. In general, there is also a secondary effect that the learner is less tired and easy to train for a longer time than when the learner utters at the timing without using the reference sound.

  Further, as is generally known, the learning effect can be further enhanced by providing visual information in parallel with the voice. Therefore, in the language learning support system according to the first aspect of the invention, image data storage means for storing image data corresponding to each original voice chunk and output when the voice data is reproduced and output from the voice output means. It is preferable to further comprise image reproduction means for reproducing the corresponding image data in synchronization with the sound. Here, the image data may be either a still image or a moving image.

  The language learning support system according to the first invention can provide a learning environment to the learner in various manners, but the learner can learn for an arbitrary length of time at an arbitrary time while at home. preferable. Therefore, as a preferred embodiment of the language learning support system according to the first invention, the system includes a host computer and a terminal connected to each other via a network, and the voice output means is provided in the terminal, The voice data created by the data creation means may be stored in a storage device of the host computer, and the voice data may be sent from the host computer to the terminal in response to a request from the terminal to the host computer. .

  In this case, the Internet can be used as the network, and a general personal computer can be used as the terminal. Audio data (or audio data with images) is sent from the host computer to the terminal in a data format that can be played back by application software for playing audio files (audio files with images) that are standardly installed in personal computers. In this case, it is not necessary to install special software on the terminal side.

  As described above, according to the language learning support system and the language learning support method according to the present invention, the speech of the language to be learned and the speech of the native language having the corresponding meaning are repeatedly and alternately output in units of chunks. Therefore, even beginners, beginners, etc., who lacked knowledge of the syntax and meaning of the language being studied, imitated the sound they heard while linking the sound and meaning Speaking training can be performed. In particular, the length of the speech unit that the learner listens to is often within the time range that allows short-term memory, so it is difficult for the original speech to be missed and the repeat utterance must be performed accurately. Can do. This makes it possible to learn foreign language conversation with high effectiveness.

1 is a system configuration diagram of an embodiment of a language learning support system according to the present invention. The figure which shows an example of the reproduction | regeneration raw material file used as the raw material which produces | generates the moving image file stored in the moving image file storage part of a server. The figure which shows an example of the basic sound in the language learning assistance system of a present Example. The figure which shows the audio | voice output at the time of the audio | voice file for 1st practice modes based on the example shown in FIG. The figure which shows the audio | voice output at the time of reproduction | regeneration of the audio | voice file for 2nd practice modes based on the example shown in FIG. The figure which shows the audio | voice output at the time of reproduction | regeneration of the audio | voice file for 3rd practice modes based on the example shown in FIG. The figure which shows the audio | voice output at the time of the audio | voice file for review modes based on the example shown in FIG. The figure which shows the state at the time of the audio | voice file for 2nd practice modes with a still image based on the example shown in FIG. The flowchart which shows the procedure of the chunk division | segmentation process in the language learning assistance system of a present Example. The flowchart which shows the procedure of the audio | voice processing process for 1st practice modes in the language learning assistance system of a present Example. The flowchart which shows the procedure of the audio | voice processing process for 2nd practice modes in the language learning assistance system of a present Example. The flowchart which shows the procedure of the audio | voice processing process for 3rd practice modes in the language learning assistance system of a present Example. The flowchart which shows the procedure of the speech processing for review modes in the language learning support system of a present Example.

  An embodiment of a language learning support system according to the present invention will be described with reference to the accompanying drawings. In the language learning support system exemplified here, a learner whose native language is Japanese learns an English (learning target language) conversation, but the learner and the learning target language are not limited to this. It is clear from the explanation.

  FIG. 1 is an overall configuration diagram of an English conversation learning support system according to the present embodiment. In the English conversation learning support system of the present embodiment, the server 30 on the service provider 3 side and the terminal 10 at hand of the learner 11 are connected via a network 2 such as the Internet.

  The terminal 10 is a general personal computer (PC) in which an operation unit 14 such as a keyboard, a display unit 13 such as an LCD monitor, a speaker 15, and the like are attached to the main body 12. This PC is equipped with general Internet browsing software, audio file playback software, and the like, and the learner 11 performs learning as described later by using functions that can be realized by such application software. It is possible.

The server 30 includes a moving image file storage unit 32 in which a moving image file in which moving image data with sound to be transmitted to the learner 11 is stored, and a reproduction file related database in which related information of the moving image file is collected and stored. In response to a request from (DB) 33 and the terminal 10 via the network 2, an appropriate moving image file is read from the moving image file storage unit 32 using related information stored in the reproduction file related database 33. A data transmission processing unit 31 to be transmitted to the terminal 10. The data transmission processing unit 31 is a functional block realized by dedicated software installed in the server 30. The moving image file stored in the moving image file storage unit 32 is an FLV format file that has been frequently used for moving image distribution in recent years. Usually, the AVI format or the AVI format created on the material creation computer 40 different from the server 30 is used. When a moving image file such as the WMV format is uploaded to the server 30, it is converted to the FLV format and stored in the moving image file storage unit 32. The related information stored in the playback file related database 33 includes information indicating the storage location of the moving image file on the moving image file storage unit 32, a so-called file such as a file name, comment information about the file, a file size, and a playback time. Contains attribute information.
In addition, the material creation computer 40 includes a voice input unit 45, a data storage unit, an image information creation unit, a chunk division unit 42, a voice data processing unit as functional blocks realized by attached hardware and installed software. A unit 41, a moving image file generation unit 46, and the like.

FIG. 2 is a diagram illustrating an example of a reproduction material file that is a material for generating a moving image file stored in the moving image file storage unit of the server. The reproduction material file is roughly classified into an audio file in which audio data is stored in a predetermined format such as WAVE format, and an image file in which still image data in a predetermined format such as JPEG format is stored. There are five types of audio files: basic audio files, first practice mode audio files, second practice mode audio files, third practice mode audio files, and review mode audio files .

Next, in the English conversation learning support system of the present embodiment, an audio output form of the English conversation learning material that the learner 11 can listen to through the speaker 15 (or headphones or the like not shown) of the terminal 10 will be described in detail with an example. .
FIG. 3 is a diagram showing an example of the five types of basic voices to be processed, FIG. 4 is a schematic diagram showing the playback output of the first practice mode voice, and FIG. 5 is the playback of the second practice mode voice. FIG. 6 is a schematic diagram showing the reproduction output of the third practice mode audio, and FIG. 7 is a schematic diagram showing the reproduction output of the review mode audio.

  Here, as an example, as shown in FIG. 3, it is assumed that a sentence in English as a learning target language is “Where is the check in counter for this flight?”. The English sentence of the basic voice shown in FIG. 3 is typically voice spoken by a native speaker at a standard speaking speed. This single sentence voice is divided into a plurality of chunks. As described above, a chunk is a unit of utterance output consisting of one or more meaningful words. The above example sentence is divided into three chunks: “Where is”, “the check in counter”, and “for this flight?”. On the other hand, a Japanese voice corresponding to this English sentence, “Where is the check-in counter for my plane,” is prepared separately. There are also three Japanese voice chunks, "Check-in counter" and "This flight". In addition to such audio data, metronome sound data of a quarter time (andante) is also prepared.

  From these five types of basic audio, four types of audio data described below are created. In FIGS. 4 to 8, the Japanese voice chunks “where”, “check-in counter”, and “this flight” are written as J1, J2, and J3, respectively, “Where is”, “the check” The English voice chunks “in counter” and “for this flight?” are described as E1, E2, and E3, respectively. Also, write the original English voice “Where is the check in counter for this flight?” As E, which is not chunked, and the original Japanese voice “Where is the check-in counter for my flight?” It is described.

  As shown in FIG. 4, in the first practice mode audio, Japanese audio is recorded on the left channel (Lch) of the stereo audio, and English audio is recorded on the right channel (Rch). For both English and Japanese voices, repeated voices in multiple chunks (in this example, five times) can be heard as echo sounds, that is, the voice output level is lowered every time a voice chunk is played. Echo is added to. Then, prior to the echoed English voice chunk, the echoed Japanese voice chunk indicating the meaning of the English voice chunk is reproduced and output so that the echoed Japanese voice chunk is reproduced and output. English voice chunks are played back and output alternately in time.

  Here, the delay time and the number of repetitions are adjusted so that the repetition time length of one chunk of English speech, that is, the playback time t1 of the English speech chunk with echo added is within about 5 seconds. This is determined in consideration of the length of time during which short-term memory is possible when the learner 11 does not understand the meaning or concept of the language. Also, a predetermined time t2 (0.5 to 0.5) is reached from the end of the reproduction output of the echoed voice chunk until the start of the reproduction output of the next echo chunk (in the other language). A silence period of about 1 second) is provided.

  When the learner 11 listens to the normal playback voice of the voice for the first practice mode, the meaning is uttered in a simple Japanese voice chunk prior to the English voice chunk. Promoted. In addition, the playback time of the echoed English voice chunk is limited to a time range that can be easily memorized even by a beginner, so it is not missed, for example, until the next voice chunk starts playing. The learner 11 can utter the English voice chunk within time t2. That is, even beginners can do the same training as shadowing without difficulty. In addition, since chunks have a single meaning, they can perform training that is substantially equivalent to re-production. In normal learning, Lch voice (Japanese voice) and Rch voice (English voice) are listened together. On the terminal 10 capable of stereo playback, the learner 11 selects only Japanese voice or only English voice. You can also listen to it.

  As shown in FIG. 5, in the second practice mode sound, a quarter-beat metronome sound is recorded on the stereo sound Lch, and an English sound is recorded on the Rch. In this case, there is no Japanese voice. A metronome sound is emitted every second, for example, a sound of “chin” is generated every 4 seconds corresponding to the first beat, and a sound of “click, click, click” is generated after the second beat. Can be. Of course, electronic sounds may be used.

  In English voice recorded in Rch, first, in the first round, English voice chunks are output in order of E1, E2, E3 in synchronization with the first beat of the Lch metronome sound, and on the third beat of the metronome sound. Synchronously, the first voice English chunk whose voice output level is reduced is output. In the second round, the output of the English voice chunk synchronized with the first beat of the metronome sound is the same as in the first round, but the third beat is silent.

  As basic training, first, the learner 11 listens to the playback output of the English voice chunk of the first beat, and at the timing when the English voice chunk whose level is lowered is outputted on the third beat, the learner 11 Speak in the way of Ing. At this time, the voice may be uttered so as to cover the attenuated English voice. In the next second round, the learner 11 listens to the playback output of the English voice chunk of the first beat, and speaks in the manner of shadowing in accordance with the metronome sound of the third beat. In this way, the learner 11 only needs to utter the English voice chunk that was listened to just before the periodic metronome sound, so the learner 11 can easily take the timing of the utterance. In this way, the learner 11 can be encouraged to actively speak, so the effects of shadowing and re-production are high.

  As shown in FIG. 6, in the third practice mode sound, a synthesized sound of a quarter-beat metronome sound and a Japanese sound is recorded in the stereo sound Lch, and an English sound is recorded in the Rch. The metronome sound is the same as the sound for the second practice mode.

  In the Japanese voice synthesized with the metronome sound, each Japanese voice chunk is output in the order of J1, J2, J3 in synchronization with the first beat of the metronome sound, and this is repeated three times. On the other hand, in English speech of Rch, each English speech chunk is output in the order of E1, E2, and E3 in synchronization with the second beat of the metronome sound. Is the attenuated audio output level, and in the third round, it becomes a completely muted silence state. Therefore, in the first and second rounds, Japanese voice chunks and English voice chunks are alternately output in time with the metronome sound tempo (however, in the second round, English voice is output at a low output level). During the tour, the learner 11 can speak during the silence period following the Japanese voice chunk in time with the tempo of the metronome sound.

  As a result, the learner 11 can utter English speech chunks rhythmically while understanding (confirming) the meaning of Japanese in the order of playback of the English speech chunks, that is, in the order according to the English grammar. it can.

  As shown in FIG. 7, in the review mode audio, Japanese audio is recorded on the stereo audio Lch and English audio is recorded on the Rch. In this case, the English speech E and the Japanese speech J that are not divided are used for both the English speech and the Japanese speech, not in chunk units. That is, Japanese speech J and English speech E are alternately reproduced and output in units of one sentence. Japanese speech J has the same speech speed and the same output level, whereas English speech E is The second time is the normal speech speed and the standard output level, and the first time the speech speed is increased to about 1.2 to 1.5 times (indicated by → E ← in FIG. 7). The time domain corresponding to the third time is silent, the fourth time is normal speech speed, and the output level is a low output level attenuated.

  By listening to the English voice with a higher speaking speed first, it is heard slowly when listening to the English voice with the standard speaking speed for the second time. Thereby, the learner 11 can listen to the English sentence accurately at the first time and the second time, and can utter it at the third time and the fourth time in the manner of re-production.

  In the English conversation learning support system according to the present embodiment, learning (training) in which the above three types of practice mode voices and one type of review mode voice are appropriately combined is possible. Typically, an English sentence conversation training is executed in the order of first practice mode voice → second practice mode voice → third practice mode voice → review mode voice. Of course, one of the first to third practice mode voices and the review mode voice may be repeated depending on the level of the learner and the degree of understanding.

Furthermore, in the English conversation learning support system of the present embodiment, in addition to the sound output from the speaker 15 or the like, a visual learning effect is added by drawing an image synchronized with the sound on the screen of the display unit 13. . Specifically, image data corresponding to each English audio chunk is prepared, and the corresponding image is synchronized with the English audio chunk and the Japanese audio chunk semantically corresponding to the playback and output. They are displayed sequentially. FIG. 8 shows an example of display output when the second practice mode sound is reproduced and output for the example sentence shown in FIG. By displaying these images in synchronism with the sound, the learner 11 can visually grasp the situation and situation in which the English sound chunk is used, which is much more than when only the sound is used. High learning effect is obtained.
In order to avoid the learner's utterance relying on character information, English (learning target language) characters are not basically displayed, but only for image information such as abstract emotional expressions. When an appropriate situation cannot be expressed, a minimum number of Japanese (native language) characters may be displayed, for example, as a comic balloon.

  In the English conversation learning support system according to the present embodiment, the moving image file storage unit 32 stores the audio data stored in the audio files such as the audio files for the practice modes and the audio files for the review mode and the corresponding image files. A moving image file generated by combining the image data thus stored is stored.

  For example, when the learner 11 requests the service provider 3 to provide a learning material providing service in advance, authentication information for identifying the learner 11 is held in the server 30. Under such circumstances, when the learner 11 operates the terminal 10 at hand and issues a request for English conversation learning to the server 30, the server 30 is the person who has given the request the permitted learner 11. Authenticate whether or not. When the authentication is completed, the data transmission processing unit 31 in the server 30 reads the related information of the file necessary for performing the learning requested from the terminal 10 from the reproduction file related database 33, and the corresponding moving image file is determined by the information. The moving image file is read from the moving image file storage unit 32 and transmitted to the terminal 10 via the network 2. The terminal 10 decodes each of the audio data and the image data in the file sent on the predetermined application software, and performs streaming reproduction. Thereby, the audio | voice output and display output as above-mentioned are performed from the terminal 10, and the learner 11 can implement English conversation learning using this.

  Of course, when the learner 11 is permitted to store the transmission file, the terminal 10 temporarily stores (that is, downloads) the file transmitted from the server 30 in the internal storage device, and then offline. You may be able to play it.

  Next, in the English conversation learning support system of the present embodiment, various voice file creation processing executed by the material creation computer 40 will be described with reference to FIGS. 9 is a flowchart showing the procedure of the chunk division process, FIG. 10 is a flowchart showing the procedure of the first practice mode voice processing process, FIG. 11 is a flowchart showing the procedure of the second practice mode voice processing process, and FIG. FIG. 13 is a flowchart showing a procedure of the voice processing for the three practice modes, and FIG. 13 is a flowchart showing a procedure of the voice processing for the review mode.

  The English text (English speech) that is the source of the teaching material is taken, for example, by the voice input unit 45 that is spoken by the native speaker as the teaching material, and is stored in the data storage unit 44 as it is or after being appropriately compressed. Can do. Of course, it is also possible to use generally recorded conversations as teaching materials instead of teaching materials. However, the background noise is preferably as small as possible in view of performing processing as described later. Separately, it is necessary to prepare a Japanese sentence corresponding to the English sentence prepared as a teaching material. This is because the native speaker reads the Japanese sentence translated from the English sentence appropriately, and this is read out by the voice input unit. It can be obtained by storing in the data storage unit 44 via 45.

For the English speech prepared as described above, first, the chunk division unit 42 starts chunk division processing according to FIG.
That is, the chunk division unit 42 reads the above English speech (original speech) from the data storage unit 44 into the internal working memory (step S1). Next, a speech waveform analysis is performed on the original speech to identify a silent region and a speech region (step S2). For example, when a silent state (a state where the output level is equal to or lower than a predetermined threshold) continues for a predetermined time or more, it is determined that the silent state is from the start point to the end point, and the rest is a voice region. Judge. A voice area sandwiched between two silence areas can be regarded as one chunk. It should be noted that various parameters for such determination (for example, a threshold value of time length for silent region determination) are set in advance by the operator, and default values are used for parameters that are not set. .

  In general, when a native English speaker speaks, a silent period is generated because a breath is taken at a chunk delimiter having a grouped meaning or the content of the next story is considered. Therefore, in many cases, one sentence can be divided into chunks based on the silent region and speech region identification results as described above. However, depending on the way of speaking, there may be a silent area in the middle of one chunk and it may be mistakenly divided into multiple chunks, or conversely, multiple chunks may be judged as one chunk. is there. Therefore, after the processing of step S2 is completed, for example, the operator evaluates the identification result of step S2 (step S3), and determines that the proper identification is not made (that is, chunk division is not appropriate). Correct the chunk division by automatic determination by shifting the identification position of the silent area and the voice area by manual operation or inserting a forced division instruction as appropriate (step S4), and return to step S3 Evaluate the modified chunk split again. If it is determined in step S3 that the chunk division is appropriate, the chunk-divided voice is stored in the data storage unit 44 as one of the basic voices shown in FIG. 3 by a predetermined operation of the operator (step S5). ).

  Japanese speech may be divided into chunks by the same process as English speech, but in general, Japanese speech chunks are not translated into Japanese in order to convey the meaning of English speech chunks in a concise and accurate manner. A simple division is not appropriate. For this reason, it is preferable that Japanese speech chunks are captured via the speech input unit 45 as spoken in units of chunks by a native speaker, regardless of the processing described above.

  When the first practice mode voice file creation process is started by an operator's instruction, the voice data processing unit 41 performs chunked English speech (English voice chunk) and chunk-divided Japanese for an arbitrary sentence. The voice (Japanese voice chunk) is read from the data storage unit 44 into the internal working memory (step S11). In the example shown in Fig. 3, there are three English voice chunks "Where is", "the check in counter", and "for this flight?", "Where is", "Check-in counter", " Are read with three Japanese voice chunks.

  Next, the echo addition process is executed a predetermined number of times for each chunk (step S12). Parameters such as the echo time delay and the number of repetitions at this time may be set in advance by the operator, and default values may be used for parameters that are not set. As a result, for example, voice data for each chunk to which five echoes are added as shown in FIG. 4 is obtained. Next, each chunk subjected to echo addition processing is distributed to each channel as shown in FIG. 4, and Japanese voice chunks and English voice chunks are alternately output at predetermined time intervals. The data arrangement is adjusted (step S13).

  However, when the process is automatically executed in steps S12 and S13, it may be unnatural when a human hears the processed sound or may not be suitable for intended learning. . Therefore, after the processing of step S13 is completed, for example, the operator evaluates the reproduced sound after the processing (step S14), and when it is determined that appropriate processing has not been performed, the operator corrects parameters such as the echo delay time, for example. (Step S15), the process returns to Step S12 to execute the process again. Alternatively, processing under a plurality of parameters may be executed in advance to create different audio files, and the operator may listen to them and select the most appropriate audio. If it is determined in step S14 that the reproduced sound is appropriate, the created sound file is stored in the data storage unit 44 as a completed sound file for the first practice mode by a predetermined operation of the operator (step S16). .

  When the second practice mode audio file creation process is started in accordance with the operator's instruction, the audio data processing unit 41 converts the chunked English voice (English voice chunk) and metronome sound for any one sentence into data. The data is read from the storage unit 44 into the internal working memory (steps S21 and S22).

  Next, based on the metronome sound of a quarter beat, the playback of each English voice chunk described above is started just at the first beat, and the English voice chunk of the first beat is copied and the audio output level is set to a predetermined value. The data arrangement on the time axis is adjusted so that the level is attenuated and the reproduction is just started at the third beat (steps S23 and S24). In the second round, the third English voice chunk is deleted. Then, as shown in FIG. 5, the English voice chunk whose playback output timing is adjusted as described above is stored in the Rch of the stereo voice, and the metronome sound is stored in the Lch to form a voice file (step S25).

  After the processing of step S25 is completed, for example, the operator evaluates the reproduced sound after the processing (step S26), and when the operator determines that appropriate processing has not been performed, the operator sets parameters such as the amount of attenuation of the audio output level. It corrects (step S27), returns to step S23, and executes processing again. Alternatively, processing under a plurality of parameters may be executed in advance to create different audio files, and the operator may listen to them and select the most appropriate audio. If it is determined in step S26 that the reproduced sound is appropriate, the created sound file is stored in the data storage unit 44 as a completed sound file for the second practice mode by a predetermined operation of the operator (step S28). .

  When the third practice mode voice file creation process is started by the operator's instruction, the voice data processing unit 41 sends an English voice chunk and a Japanese voice chunk from the data storage unit 44 to an internal working memory for an arbitrary sentence. (Step S31). Further, the metronome sound is read from the data storage unit 44 into the internal working memory (step S32).

  Next, with the metronome sound of a quarter time as a reference, the above-mentioned Japanese voice chunks are arranged in order so that playback starts just at the first beat, and this is repeated three times (three rounds). Each Japanese voice chunk is copied to and the data arrangement on the time axis is adjusted (step S33). Then, the processed Japanese voice chunk and metronome sound are synthesized on the time axis (step S34). Next, as shown in FIG. 6, the English voice chunks of one sentence are arranged in order so that playback starts at the third beat of the metronome sound, and each English voice is repeated three times (three rounds). The chunk is copied and the data arrangement on the time axis is adjusted (step S35). Further, the output level of the second voice chunk of the English voice chunk processed in this way is attenuated by a predetermined level, and the third voice chunk is deleted (step S36). Then, the English sound is stored in the stereo sound Rch and the Japanese sound is stored in the Lch to form a sound file (step S37).

  After the processing of step S37 is completed, for example, the operator evaluates the reproduced sound after the processing (step S38), and when it is determined that the appropriate processing is not performed, the operator sets parameters such as the attenuation amount of the audio output level, for example. It corrects (step S39), returns to step S33, and executes the process again. Alternatively, processing under a plurality of parameters may be executed in advance to create different audio files, and the operator may listen to them and select the most appropriate audio. If it is determined in step S38 that the reproduced sound is appropriate, the created sound file is stored in the data storage unit 44 as a completed sound file for the third practice mode by a predetermined operation of the operator (step S40). .

  When the review mode audio file creation process is started by an operator's instruction, the audio data processing unit 41 sends an arbitrary one-sentence English speech and non-chunk English speech, that is, the original speech from the data storage unit 44. Read into the internal working memory (step S41). Then, an English voice E1 whose speech speed is increased by 1.2 to 1.5 times from the original English voice E, an English voice E2 whose voice output level is attenuated by a predetermined level, and a silenced English voice E3. Create (step S42). Next, as shown in FIG. 7, Japanese speech is stored in Lch, English speech as described above is stored in Rch, and Japanese speech J and English speech E1, E, E2, E3 are alternately output. The data arrangement is adjusted as described above (step S43).

  After the process of step S43 is completed, for example, the operator evaluates the reproduced sound after the process (step S44), and when it is determined that an appropriate process has not been performed, the operator corrects parameters such as the speech speed (step S45), returning to step S42, the process is executed again. Alternatively, processing under a plurality of parameters may be executed in advance to create different audio files, and the operator may listen to them and select the most appropriate audio. If it is determined in step S44 that the reproduced sound is appropriate, the created audio file is stored in the data storage unit 44 as a completed audio file for the review mode by a predetermined operation of the operator (step S46).

  The first to third practice mode audio files and the review mode file are prepared in the data storage unit 44 by the processes described above. In addition, the operator creates an appropriate image corresponding to each English voice chunk by the image information creation unit 43 and also stores it in the data storage unit 44. The moving image file generation unit 46 uses the file stored in the data storage unit 44 as described above, and synthesizes the image data and the audio data so that the audio output and the image output are synchronized for each audio file corresponding to one sentence. Then, for example, a moving image file in a file format such as AVI or WMV is automatically generated. Then, when a predetermined file name is assigned to the generated moving image file and uploaded to the server 30, the moving image file is converted into an FLV format moving image file and stored in the moving image file storage unit 32. At the same time, related information of the moving image file (for example, the file storage location and file attribute information as described above) is collected and stored in the reproduction file related database 33. Thereby, when the data transmission processing unit 31 receives an instruction to reproduce a certain sentence from the learner 11, the data transmission processing unit 31 refers to the related information stored in the reproduction file related database 33 and specifies the corresponding moving image file. At the same time, the storage location can be found immediately, and the data constituting the corresponding video file can be read from the video file storage unit 32 and sent to the terminal 10.

  It should be noted that the above embodiment is merely an example of the present invention, and it is obvious that modifications, additions, and modifications as appropriate within the scope of the present invention are included in the scope of the claims of the present application.

DESCRIPTION OF SYMBOLS 10 ... Terminal 11 ... Learner 12 ... PC main body 13 ... Display part 14 ... Operation part 15 ... Speaker 2 ... Network 3 ... Service provider 30 ... Server 31 ... Data transmission processing part 32 ... Movie file storage part 33 ... Reproduction file related Database 40 ... Material creation computer 41 ... Audio data processing unit 42 ... Chunk division unit 43 ... Image information creation unit 44 ... Data storage unit 45 ... Audio input unit 46 ... Movie file generation unit

Claims (10)

A language learning support system in which a learner listens to a reproduced voice output from a voice output unit based on an original voice of a language to be learned, and trains a conversation of the language by speaking according to the voice,
a) Acquiring a sentence of the original speech of the language to be learned , performing speech waveform analysis on the original speech, and identifying the silence area and the speech area, thereby dividing the sentence of the original speech into chunks. Sentence dividing means for obtaining an original voice chunk,
b) a native speech chunk acquisition means for acquiring a native speech chunk having a meaning corresponding to each of the original speech chunks in the learner's native language;
c) echo adding means for performing echo addition processing such that each of the original voice chunk and the native voice chunk is repeated a predetermined number of times with a delay time at which a repeated sound can be heard independently during reproduction;
d) Elapsed time so that the original speech chunk and the native speech chunk of the learning target language, each echo-added, are alternately output in time in units of chunks and at a predetermined time interval. Data creating means for creating first audio data in which the output timing associated with is controlled;
A language learning support system characterized in that the first voice data created by the data creation means is output from the voice output means. The language learning support system according to claim 1,
The language learning support system, wherein the data creating means stores the original voice chunk and the native voice chunk in different voice channels. The language learning support system according to claim 1,
The data creation means accompanies the passage of time so that an original voice chunk that has not been subjected to echo addition processing is intermittently output with a time interval that allows the learner to utter at least every chunk. A language learning support system characterized in that second audio data whose output timing is controlled is created, and the first audio data and the second audio data can be selectively reproduced and output from the audio output means. The language learning support system according to claim 3,
The data creation means controls the output timing so that the original voice chunk is output in audio in accordance with the tempo of the reference sound emitted at a fixed time interval, and the second sound data obtained by synthesizing the reference sound with the audio output Language learning support system characterized by creating The language learning support system according to any one of claims 1 to 4,
Image data storage means for storing image data corresponding to each original audio chunk;
The language learning support system further comprising: an image reproduction unit that reproduces image data corresponding to the output audio when the audio data is reproduced and output from the audio output unit. The language learning support system according to any one of claims 1 to 5,
A system including a host computer and a terminal connected to each other via a network, wherein the voice output means is provided in the terminal, and the voice data created by the data creation means is stored in a storage device included in the host computer. A language learning support system which is stored and voice data is transmitted from the host computer to the terminal in response to a request from the terminal to the host computer. A language learning support method in which a learner listens to a reproduced voice output from a voice output unit based on an original voice of a learning target language, and trains a conversation of the language by uttering according to the voice,
a) sentence dividing means obtains a sentence of the original speech of the language to be learned, performs speech waveform analysis on the original speech, and discriminates between the silent area and the speech area, thereby A sentence division step for obtaining an original voice chunk by dividing into chunks;
b) a native speech chunk acquisition means for acquiring a native speech chunk having a meaning corresponding to each of the original speech chunks in the learner's native language;
c) echo adding means, respectively to said original audio chunk and the native voice chunk, and the echo adding step of adding an echo as the sound repeatedly at the time of reproduction is repeated a predetermined number of times a delay time that can be listened to independently ,
d) The data creation means outputs the voices of the original speech chunks and the native speech chunks of the learning target language to which the echoes are added alternately in time in units of chunks and at a predetermined time interval. And a data creation step for creating first audio data in which the output timing with the passage of time is controlled,
A language learning support method, characterized in that the first voice data created in the data creation step is output from the voice output means. The language learning support method according to claim 7,
In the data creation step, with the passage of time, an original voice chunk that has not undergone echo addition processing is intermittently output with a time interval that allows the learner to speak at least between each chunk. A language learning support method, characterized in that second audio data whose output timing is controlled is created, and the first audio data and the second audio data can be selectively reproduced and output from the audio output means. The language learning support method according to claim 8,
In the data creation step, the output timing is controlled so that the original voice chunk is output as a sound in accordance with the tempo of the reference sound generated at regular time intervals, and the second sound data obtained by synthesizing the reference sound with the sound output. Language learning support method characterized by creating The language learning support method according to any one of claims 7 to 9,
Image data corresponding to each original audio chunk is prepared, and when the audio data is reproduced and output from the audio output means, the corresponding image data is reproduced in synchronization with the output audio. Language learning support method.

Images