JP2019045575A - Language learning device, language learning method, and language learning program - Google Patents

Abstract

To provide a language learning device capable of improving the learning effect of English even for Japanese.SOLUTION: This device executes a listening step in which the image including the English text and the English voice are output in a plurality of scenes, and thereafter executes a record step consisting of a plurality of scenes. At this time, the listening step outputs the images and texts which constitute one scene, and repeats the output of the images and the voice in each of the plurality of viewing scenes each of which is constituted by different English sentences different from each other in the same grammar. In the recording step, the images and texts of one recording scene are output, and the voice of the user who pronounces the English corresponding to the image and the voice is recorded and played back. At this time, the recording step repeats the output of the image and the voice, the collection of the voice of the user, and the recording and the playback of each scene composed of different English texts in the same grammar.SELECTED DRAWING: Figure 1

Description

  The present invention belongs to the technical fields of a language learning device, a language learning method, and a language learning program, and more specifically, a language learning device and a language learning used for Japanese to learn other languages other than Japanese. The method belongs to the technical field of a program for the language learning device.

  In recent years, the need for practical English education for Japanese people has been increasing, and in response to this, the need for practical English education for lower age groups such as primary school children has been pointed out. In addition to English language education by groups at schools etc., English language education has often been performed personally using personal computers and tablet terminals operated by the user. In the following description, a personal computer or a tablet terminal is simply referred to as “PC (Personal Computer) or the like”. Such personal English education can be expected to have effects that can not be obtained by group English education, such as being able to adjust the degree of difficulty according to the degree of progress and progress of the individual who receives the education.

  And as a prior art document about English education using the above-mentioned PC etc., the following patent documents 1 are mentioned, for example. In the technology described in Patent Document 1, in the system for language education, a step of presenting a first description of an event corresponding to a first viewpoint and a step of presenting a second description of an event corresponding to a second viewpoint And the first description of the event and the second description of the event are configured to be a common language.

JP, 2011-197691, A

  However, when using the technology described in Patent Document 1 for English language education for Japanese, there are problems such as being too advanced or easy to get bored, and effective English language education is expected There is a problem that it can not do.

  Therefore, the present invention has been made in view of the above problems, and an example of the problem is to improve the learning effect (practicing effect) of a language (for example, English) to be learned even if it is Japanese. It is an object of the present invention to provide a language learning device and a language learning method that can be

  In order to solve the above-mentioned problems, the invention according to claim 1 is a viewing method which outputs an image including a sentence spelled in a language to be learned and a voice uttering the sentence as a plurality of viewing scenes. Language learning apparatus comprising: viewing step execution means such as a listening step processing unit that executes steps; and recording step execution means such as a record step processing unit that executes a recording step consisting of a plurality of recording scenes after the viewing step. The viewing step execution means is a viewing output means such as a display and a speaker for outputting a viewing image and a viewing voice as the image and the audio constituting the one viewing scene, the viewing image and the viewing; The output of voice is repeated for each of the plurality of viewing scenes respectively configured by the different sentences of the same grammar. Viewing control means such as a listening step processing unit for controlling the viewing output means, and the recording step execution means, the recording image and the recording voice which are the image and the sound constituting the one recording scene Recording output means such as a display and a speaker to output, recording means such as a record step processing unit for collecting and recording voice of a learner who pronounced the recorded image and the sentence corresponding to the recorded voice, and the recording The same as the output of the recorded image and the recorded voice, the collection and recording of the voice of the learner, and the output of the voice of the learner. The recording output means, the recording means, and the recording means are repeated for each of a plurality of the recording scenes respectively composed of the sentences different from each other in the grammar. Comprising a recording control means such as a record step processing unit that controls the learner voice output means.

  In order to solve the above-mentioned problems, the invention according to claim 20 is a viewing method which outputs an image including a sentence spelled in a language to be learned and a voice uttering the sentence as a plurality of viewing scenes. Viewing step execution means such as a listening step processing unit including viewing step execution means for executing steps, including viewing output means such as a display and a speaker, and viewing control means such as a listening step processing portion; Recording step executing means for executing a recording step consisting of a plurality of recording scenes, the recording output means such as a speaker, the recording means such as a record step processor, and the learner voice output means such as a speaker; Recording control means, and recording step execution means such as a recording step processing unit; A language learning method, wherein the viewing step includes a viewing output step of outputting, by the viewing output means, the viewing image and the viewing audio which are the image and the audio constituting the one viewing scene, and the viewing image And a viewing control step of controlling the viewing output means by the viewing control means so that the output of the viewing sound is repeated for each of a plurality of the viewing scenes respectively composed of the different sentences of the same grammar. In the recording step, the recording step corresponds to the recording output step of outputting the recording image and the recording voice which are the image and the voice constituting the one recording scene and the recording output means, and the recording image and the recording voice. A recording step of collecting and recording the voice of the learner who pronounced the sentence by the recording means; and the voice of the learner who has recorded the voice The learner voice output step output by the learner voice output means, the output of the recorded image and the recorded voice, the collection and recording of the learner's voice, and the output of the learner's voice Recording control step of controlling the recording output means, the recording means, and the learner's voice output means by the recording control means so as to repeat for each of a plurality of the recording scenes respectively composed of different sentences. Including.

  In order to solve the above-mentioned problems, the invention according to claim 21 comprises a computer included in a language learning device, an image including a sentence spelled in a language to be learned, and a voice uttering the sentence A language learning program that causes a viewing step execution unit that executes a viewing step to output as a plurality of viewing scenes, and a recording step execution unit that executes a recording step consisting of a plurality of recording scenes after the viewing step. A viewing output means for outputting the viewing image and the viewing audio that are the image and the audio that constitute one viewing scene, the computer functioning as the viewing step execution means; The output of the viewing audio is a plurality of the viewing scenes each composed of the different sentences of the same grammar. Function as the view control means for controlling the view output means to repeat the above, and the computer functioning as the recording step execution means, the recorded image and the sound which are the image and the sound constituting the one recording scene Recording output means for outputting a voice, recording means for collecting and recording the voice of a learner who pronounces the recorded image and the sentence corresponding to the recorded voice, learner voice for outputting the voice of the recorded learner A plurality of output means, and outputs of the recorded image and the recorded voice, collection and recording of the learner's voice, and output of the learner's voice by the sentences having the same grammar but different from one another Recording control means for controlling the recording output means, the recording means, and the learner voice output means so as to repeat for each of the recording scenes of To function.

  According to the invention described in any one of claims 1, 20 or 21, an image including a sentence spelled in a language to be learned and a plurality of sounds uttering the sentence are generated. A viewing step for outputting as a viewing scene is executed, and then a recording step consisting of a plurality of recording scenes is executed. At this time, as a viewing step, outputting a viewing image and a viewing sound that constitute one viewing scene is repeated as a plurality of viewing scenes each composed of different sentences of the same grammar. On the other hand, in the recording step, the recording image and the recording voice constituting one recording scene are output, and the voice of the learner who pronounces the recording image and the sentence corresponding to the recording voice is collected and recorded, Outputting the voice is repeated for each of a plurality of recording scenes each composed of mutually different sentences of the same grammar. Therefore, the learning effect (practice effect) of the language to be learned can be improved by executing the viewing step and the recording step as so-called pattern practice configured by different sentences of the same grammar.

  In order to solve the above problems, the invention according to claim 2 is the language learning device according to claim 1, wherein the operation of the operation unit or the like operated when sound collection and recording are started by the recording means. And a recording unit configured to perform the sound collection and the recording when the operation using the operation unit is performed.

  According to the second aspect of the invention, in addition to the function of the invention of the first aspect, the operation using the operation means operated at the time of starting sound collection and recording by the recording means is performed. If necessary, perform voice collection and recording of the learner's voice. Therefore, since pronunciation and recording can be performed at an arbitrary timing of the learner, the language can be learned with a margin.

  In order to solve the above-mentioned subject, the invention according to claim 3 is a language learning device according to claim 1 or claim 2, wherein timing at which repetition of a plurality of viewing scenes by the viewing control means is finished The information processing apparatus further comprises commentary information output means such as a display for outputting grammar commentary information which is a commentary of the grammar corresponding to the viewing step.

  According to the third aspect of the invention, in addition to the operation of the invention according to the first or second aspect, the grammar commentary information is output at the timing when the repetition for the plurality of viewing scenes is completed, so that the viewing step is performed. You can easily understand the content of the grammar that was the subject of learning.

  In order to solve the above problems, the invention according to claim 4 relates to the language learning device according to any one of claims 1 to 3, wherein a plurality of the recording scenes by the recording control means are stored. The information processing apparatus further includes labor information output means such as a display for outputting labor information to labor the learner at the timing when the repetition ends.

  According to the fourth aspect of the invention, in addition to the operation of the invention according to any one of the first to third aspects, labor information is output at the timing when repetition of a plurality of recording scenes is completed. Therefore, the learner's willingness to continue learning can be maintained or enhanced.

  In order to solve the above-mentioned subject, the invention according to claim 5 recognizes the contents of the voice of the recorded learner in the language learning device according to any one of claims 1 to 4 A match such as a display that compares recognition means such as a record step processing unit, the recognition result by the recognition means, and the contents of each of the recorded image and the recorded voice, and outputs matching degree information indicating the matching degree of both And a degree information output unit.

  According to the invention of claim 5, in addition to the operation of the invention according to any one of claims 1 to 4, the voice of a learner who pronounces a recorded image and a sentence corresponding to the recorded voice The content is recognized, the recognition result is compared with the content of each of the recorded image and the recorded voice, and the matching degree information indicating the degree of matching between the two is output, so the content of the voice uttered by the learner matches It is possible to advance learning of the language while quickly recognizing whether or not it is present.

  In order to solve the above-mentioned subject, the invention according to claim 6 is the language learning device according to any one of claims 1 to 5, wherein between the viewing step and the recording step, An iterative step executing means for executing an iterative step consisting of a plurality of repetitive scenes, wherein the image forming one of the repetitive scenes and the repetitive image and the repetitive speech as the sound are repetitive output means such as a display and a speaker A pronunciation promoting means such as a display for outputting pronunciation promotion information for prompting the pronunciation of words constituting the sentence corresponding to each of the outputted repetitive images and repetitive speech, output of the repetitive images and the repetitive speech, and the pronunciation The repetition such that the output of the promotion information is repeated for each of the plurality of repetitive scenes respectively constituted by the different sentences of the same grammar. Further comprising a repetition control means such as a repeat step processing unit that controls the force means and the sound facilitating means, an iteration step execution means such repeat step processor comprising a.

  According to the invention set forth in claim 6, in addition to the operation of the invention set forth in any one of claims 1 to 5, an iteration consisting of a plurality of repetitive scenes is provided between the viewing step and the recording step. Execute the step At this time, as an iterative step, outputting repetitive images constituting one repetitive scene and repetitive speech, and outputting provocation information prompting the user to pronounce words constituting the repetitive image and a sentence corresponding to the repetitive speech, It is repeated for each of a plurality of repetitive scenes each composed of different sentences of the same grammar. Therefore, as the so-called pattern practice composed of different sentences of the same grammar, the repetition step is performed in addition to the viewing step and the recording step, thereby further improving the learning effect (practicing effect) of the language to be learned Can.

  In order to solve the above problems, the invention according to claim 7 is the language learning device according to claim 6, wherein the repetition is performed at the timing when the repetition of a plurality of repetitive scenes by the repetition control means is finished. It further comprises iterative commentary information output means such as a display for outputting grammar commentary information that is a commentary of the grammar corresponding to the step.

  According to the invention described in claim 7, in addition to the operation of the invention described in claim 6, as grammatical commentary information is outputted at the timing when the repetition for a plurality of repetitive scenes ends, learning as an iterative step You can easily understand the content of the targeted grammar.

  In order to solve the above problems, the invention according to claim 8 relates to the language learning device according to claim 6 or 7, wherein the number of repetitions of the viewing scene and the number of repetitions of the recording scene and the repetition scene The repetition number of is configured to be the same.

  According to the invention as set forth in claim 8, in addition to the operation of the invention as set forth in claim 6 or claim 7, since the number of repetitions of viewing scenes, the number of repetitions of recording scenes and the number of repetitions of repetition scenes are the same. , Can further enhance the learning effect as pattern practice.

  In order to solve the above-mentioned subject, invention of Claim 9 is comprised in the language learning apparatus of Claim 8 so that each said repetition number is four times.

  According to the invention as set forth in claim 9, in addition to the action of the invention as set forth in claim 8, since each repetition number is four times, it is possible to prevent boredom and enhance the learning effect as pattern practice. it can.

  In order to solve the above-mentioned subject, the invention according to claim 10 consists of a plurality of translation scenes in the language learning device according to any one of claims 1 to 9, after the recording step. A translation step executing means for executing a translation step, such as a display such as a display which constructs one of the translation scenes and displays a Japanese image including Japanese sentences spelled in Japanese corresponding to the image and the voice; Translation and recording means such as a word image display means, and a translation step processing unit for collecting and recording the voice of the learner in the language corresponding to the Japanese sentence contained in the displayed Japanese image as a translated voice A first translation output unit such as a speaker for outputting the recorded translated speech, an image including a sentence of the language corresponding to the Japanese sentence, and a sentence of the language Second translation output means such as a speaker for outputting the voiced sound, display of the Japanese image, collection and recording of the translation voice, output of the translation voice, the image by the second translation output means, and The Japanese image display means, the translation recording means, the first translation output means, and the first translation output means, so that the output of voice is repeated for each of a plurality of the translation scenes respectively composed of different Japanese sentences of the same grammar. And translation control means such as a translation step processing unit for controlling the second translation output means, and translation step execution means such as a translation step processing unit.

  According to the invention of claim 10, in addition to the operation of the invention according to any one of claims 1 to 9, after the recording step, a translation step consisting of a plurality of translation scenes is performed. At this time, as a translation step, a Japanese image constituting one translation scene is displayed, a translated voice corresponding to the Japanese sentence included in the displayed Japanese image is recorded, and the recorded translated voice is output. A plurality of translation scenes each composed of different Japanese sentences of the same grammar that an image including a sentence of a language to be learned equivalent to a Japanese sentence and a voice uttering the sentence of the language are outputted It is repeated every time. Therefore, by further executing the translation step as so-called pattern practice using different Japanese sentences of the same grammar, it is possible to further improve the learning effect (practice effect) of the language to be learned.

  In order to solve the above-mentioned problems, the invention according to claim 11 relates to the language learning device according to claim 10, wherein an operation unit or the like operated at the start of sound collection and recording by the translation and recording means The system further comprises translation operation means, and the translation recording means is configured to collect and record the translated speech when an operation using the translation operation means is performed.

  According to the invention as set forth in claim 11, in addition to the action of the invention as set forth in claim 10, the operation is carried out using the translation operation means operated at the start of sound collection and recording by the translation recording means. If it is, collect and record translated speech. Therefore, since the translated speech can be pronounced and recorded at any timing of the learner, the language can be learned with a margin.

  In order to solve the above-mentioned subject, in the invention according to claim 12, in the language learning device according to claim 10 or 11, the position of the reading point in the Japanese sentence corresponds to the Japanese sentence It is configured to be a position corresponding to a word break in a sentence of the language.

  According to the invention as set forth in claim 12, in addition to the action of the invention as set forth in claim 10 or claim 11, a sentence of a language to be studied in which the position of the reading point in the Japanese sentence corresponds to the Japanese sentence As a position corresponding to the word break in, it is possible to collect and record translated speech more accurately.

  In order to solve the above-mentioned subject, the invention according to claim 13 is the language learning device according to any one of claims 10 to 12, wherein the Japanese contained in the displayed Japanese image Translation speech output operation means such as an operation unit operated when outputting a voice corresponding to a word sentence and uttering a translated sentence which is a sentence spelled in the language, and operation using the translation speech output operation means And a translated voice output unit such as a speaker for outputting a voice generated by uttering the translated sentence.

  According to the invention as set forth in claim 13, according to the action of the invention as set forth in any one of claims 10 to 12, a display is made when the operation using the translation voice output operation means is performed. Outputs a voice that utters a translated sentence corresponding to the Japanese sentence contained in the Japanese image in the middle. Therefore, when it is difficult for the learner to translate Japanese sentences, the language learning can be advanced without stagnation by outputting the speech in which the translated sentences are pronounced by the operation of the learner using the translated speech output operation means. Can.

  In order to solve the above problems, the invention according to claim 14 is a language learning device according to any one of claims 10 to 13, wherein a transformer for recognizing the contents of the recorded translated speech is provided. The translation recognition means such as a translation step processing unit, the recognition result by the translation recognition means, and the content of the sentence of the language corresponding to the Japanese sentence are compared, and the coincidence degree information indicating the coincidence degree of both is output And a translation matching degree information output unit such as a display.

  According to the invention as set forth in claim 14, in addition to the function of the invention as set forth in any one of claims 10 to 13, the contents of the translated speech which is the speech of the learner equivalent to the Japanese sentence Since the recognition result is compared with the content of the sentence of the language to be learned corresponding to the Japanese sentence, and the matching degree information indicating the matching degree between the two is output, the translated speech uttered by the learner It is possible to advance learning of the language while quickly recognizing whether or not the content of

  In order to solve the above-mentioned problems, the invention according to claim 15 relates to the language learning device according to any one of claims 10 to 14 in which claims 1 and 6 are cited. The number of the Japanese sentences corresponding to the translation scene corresponds to the number of the images corresponding to the one viewing scene and the number of the sentences corresponding to the sound, and the images and the sounds constituting the one recording scene The number of sentences is less than any of the number of the sentences and the number of the sentences corresponding to the image and the sound constituting one repetitive scene.

  According to the invention as set forth in claim 15, in addition to the operation of the invention as set forth in any one of claims 10 to 14 in which claim 1 and claim 6 are cited, it corresponds to one translation scene. The number of Japanese sentences corresponds to the number of sentences corresponding to an image etc. constituting one viewing scene, the number of sentences corresponding to an image etc. constituting one recording scene, an image etc. constituting one repetitive scene Since the number of sentences to be translated is smaller than any of the number of sentences to be translated, it is possible to suppress the number of Japanese sentences in a highly difficult translation scene and prevent a decline in learning motivation.

  In order to solve the above problems, the invention according to claim 16 relates to the language learning device according to claim 10, wherein claim 1 and claim 6 are cited, wherein the viewing output means, the recording output means, At least any one of the repetition output means or the second translation output means displays the display mode in the image at the output timing as the voice of the word constituting the sentence corresponding to the image and voice being output. It is configured to output an image and a sound while making different the display mode in the image of the other word output as the voice after the output timing.

  According to the operation of the invention set forth in claim 16, in addition to the operation of the invention set forth in claim 10 relating to claim 1 and claim 6, words constituting the text corresponding to the image and sound being output Since the image and sound are outputted while making the display mode in the image at the output timing as voice different from the display mode in the image of another word output as voice after the output timing, It can be easily recognized whether the word is output.

  In order to solve the above problems, the invention according to claim 17 is configured such that, in the language learning device according to claim 16, the display mode is a display color in the image of the word.

  According to the operation of the invention set forth in claim 17, in addition to the operation of the invention set forth in claim 16, within the image at the output timing as sound of the word constituting the sentence corresponding to the image and sound being outputted Since the image and the sound are outputted while the display color of is made different from the display color form of the other words outputted as the speech after the output timing, it is easier to determine which word is outputted as the speech Can be recognized.

  In order to solve the above-mentioned problems, the invention according to claim 18 relates to the language learning device according to claim 10, wherein claim 1 and claim 6 are cited. And at least characters constituting the scene of the recording scene and the scene of the translation scene are identical.

  According to the operation of the invention set forth in claim 18, in addition to the operation of the invention set forth in claim 10 relating to claim 1 and claim 6, mutually corresponding viewing scenes, repetitive scenes, recorded scenes and translations Since at least the characters constituting the scene of the scene are the same, the learning effect can be further enhanced by advancing learning by the same character as the viewing step → repetition step → recording step → translation step.

  In order to solve the above problems, the invention according to claim 19 relates to the language learning device according to claim 10, wherein claim 1 and claim 6 are cited, wherein the viewing step, the repetitive step, the recording step And end information recording means such as a processing unit that records the end information indicating the end when the end of the translation step is output in association with the end learner and enables output.

  According to the operation of the invention set forth in claim 19, in addition to the operation of the invention set forth in claim 10 relating to claim 1 and claim 6, each of the viewing step, the repetition step, the recording step and the translation step is completed. In this case, the end information indicating the end is recorded in an outputable manner in association with the end learner, so that it is possible to further increase the desire to learn after the learner.

  According to the present invention, a viewing step of outputting an image including a sentence spelled out in a language to be learned and a voice uttering the sentence as a plurality of viewing scenes is executed, and then a plurality of recording scenes are performed. Execute the recording step consisting of At this time, as a viewing step, outputting a viewing image and a viewing sound that constitute one viewing scene is repeated as a plurality of viewing scenes each composed of different sentences of the same grammar. On the other hand, in the recording step, the recording image and the recording voice constituting one recording scene are output, and the voice of the learner who pronounces the recording image and the sentence corresponding to the recording voice is collected and recorded, Outputting the voice is repeated for each of a plurality of recording scenes each composed of mutually different sentences of the same grammar.

  Therefore, the learning effect (practice effect) of the language to be learned can be improved by executing the viewing step and the recording step as so-called pattern practice configured by different sentences of the same grammar.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows schematic structure of the language learning system which concerns on 1st Embodiment. It is a figure which illustrates the contents of the database memorized by the server apparatus contained in the language learning system concerning a 1st embodiment. It is a flowchart which shows the whole of the language learning process which concerns on 1st Embodiment. It is a flowchart which shows the content of the listening step in the language learning process which concerns on 1st Embodiment. It is figure (I) which shows the example of an image displayed in the listening step in the language learning process which concerns on 1st Embodiment. It is a figure (II) which shows the example of an image displayed in the listening step in the language learning process which concerns on 1st Embodiment, and (a) is a figure which shows the example of an image displayed in the 1st listening scene in the said listening step. (B) shows an example of an image displayed in the second listening scene in the listening step, (c) shows an example of an image displayed in the third listening scene in the listening step, d) shows an example of an image displayed in the fourth listening scene in the listening step, and (e) shows an example of an image displayed as an explanation in the listening step. It is a flowchart which shows the content of the repeat step in the language learning process which concerns on 1st Embodiment. It is a figure which shows the example of an image displayed in the repeat step in the language learning process which concerns on 1st Embodiment, (a) is a figure which shows the example of an image displayed in the 1st repeat scene in the said repeat step. Is an example of an image displayed at the time of sound generation in the first repeat scene, (c) is an example of an image displayed in the second repeat scene in the repeat step, and (d) is the illustration It is a figure which shows the example of an image displayed at the time of the pronunciation | sound_production in a 2nd repeat scene, (e) is a figure which shows the example of an image displayed as commentary in the said repeat step. It is a flowchart which shows the content of the record step in the language learning process which concerns on 1st Embodiment. It is a figure which shows the example of an image displayed in the record step in the language learning process which concerns on 1st Embodiment, (a) is a figure which shows the example of an image displayed in the 1st record scene in the said record step. Is an example of an image displayed at the time of recording in the first record scene, (c) is an example of an image displayed in the second record scene at the record step, and (d) is the illustration It is a figure which shows the example of an image displayed at the time of sound recording in a 2nd record scene. It is a flowchart which shows the content of the translation step in the language learning process which concerns on 1st Embodiment. It is a figure (I) which shows the example of an image displayed in the translation step in the language learning process which concerns on 1st Embodiment, and (a) is displayed at the time of the problem presentation of the 1st translation scene in the said translation step. It is a figure which shows the example of an image, (b) is a figure which shows the example of an image displayed at the time of the problem presentation of the 2nd translation scene in the said translation step, (c) is the 3rd transformer in the said translation step. It is a figure which shows the example of an image displayed at the time of the problem presentation of the relationship scene, (d) is a figure which shows the example of an image displayed at the time of the problem presentation of the 4th translation scene in the said translation step. It is a figure (II) which shows the example of an image displayed in the translation step in the language learning process which concerns on 1st Embodiment, and (a) is displayed at the time of the answer presentation of the 1st translation scene in the said translation step. It is a figure which shows the example of an image, (b) is a figure which shows the example of an image displayed at the time of the answer presentation of the 2nd translation scene in the said translation step, (c) is the 3rd transformer in the said translation step. It is a figure which shows the example of an image displayed at the time of the answer presentation of a repetition scene, (d) is a figure which shows the example of an image displayed at the time of the answer presentation of the 4th translation scene in the said translation step. It is a figure which shows the example of the item acquired in the language learning process which concerns on 1st Embodiment, (a) is a figure which illustrates the map before item acquisition, (b) is a figure which shows the example of one item. (C) is a figure which illustrates the map after item acquisition. It is a flowchart which shows the content of the record step in the language learning process which concerns on 2nd Embodiment. It is a flowchart which shows the content of the translation step in the language learning process which concerns on 2nd Embodiment.

  Next, the best mode for carrying out the present invention will be described based on the drawings. In each of the embodiments described below, the present invention is applied to a language learning system used when a user who is Japanese learns English using a terminal device owned by the user. It is. Here, a PC etc. which the said user owns are mentioned as an example of the said terminal device.

(I) First Embodiment At the beginning, the first embodiment according to the present invention will be described with reference to FIGS.

(I) Outline of Configuration of Language Learning System According to First Embodiment First, the outline of the configuration of the language learning system according to the first embodiment and the outline of language learning processing according to the first embodiment will be described with reference to FIG. Each will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of the language learning system according to the first embodiment, and FIG. 2 is a view exemplifying the contents of a database stored in a server device included in the language learning system. .

  As shown in FIG. 1, the language learning system S according to the first embodiment includes the terminal device T1, the terminal device T2, the terminal device T3,..., And the terminal device Tn (n is a natural number) each according to the first embodiment. The server apparatus SV according to the first embodiment, the terminal apparatus T1, the terminal apparatus T2, the terminal apparatus T3, ..., and the existing network NW such as the Internet connecting the terminal apparatus Tn and the server apparatus SV ing. At this time, since the terminal device T1, the terminal device T2, the terminal device T3,..., And the terminal device Tn basically have the same configuration and functions, in the description of the first embodiment below, these are collectively referred to as “ The configuration and functions of the terminal device T will be described.

  Next, the server apparatus SV includes a communication unit 20, a processing unit 21 including a CPU, a random access memory (RAM) and a read only memory (ROM), and a hard disc drive (HDD) or a solid state drive (SSD). And a database 22.

  On the other hand, the terminal device T is provided on the display surface of the processing unit 1, the communication unit 2, the keyboard, the mouse or the display 4 which comprises a CPU or the like and mainly executes the language learning process according to the first embodiment. An operation unit 3 comprising a touch panel or the like, a display 4 comprising a liquid crystal display or the like, a recording unit 5 comprising an HDD or SSD, a memory 6 comprising RAM and ROM, a speaker 7 and a microphone 8 . Further, the processing unit 1 is configured of a listening step processing unit 10, a repeat step processing unit 11, a record step processing unit 12, and a translation step processing unit 13. At this time, each of the listening step processing unit 10, the repeat step processing unit 11, the record step processing unit 12 and the translation step processing unit 13 may be configured by hardware logic circuits such as a CPU constituting the processing unit 1. The program may be executed as software by reading and executing a program corresponding to a flowchart showing language learning processing according to the first embodiment described later by the processing unit 1.

  The processing unit 1 is an example of the "end information recording means" according to the present invention, and the listening step processing unit 10 is an example of the "viewing step execution means" according to the present invention and an example of the "viewing control means". The record step processing unit 12 is an example of the "recording step execution means" according to the present invention, an example of the "recording means", an example of the "recording control means", an example of the "recognition means" and "coincidence information output means". In one example, the repeat step processing unit 11 is an example of the "repeated step execution means" according to the present invention and the translation step processing unit 13 is an example of the "translation step execution means" according to the present invention. The examples correspond to an example of “translation recording means”, an example of “translation control means”, and an example of “translation recognition means”. Furthermore, the display 4 and the speaker 7 correspond to an example of the "viewing output means" and an example of the "repetitive output means" according to the present invention, and the speaker 7 an example of the "recording output means" according to the present invention This corresponds to an example of “output means”, an example of “first translation output means”, an example of “second translation output means”, and an example of “translation voice output means”. Furthermore, the operation unit 3 corresponds to an example of the "operation means", an example of the "translation operation means" and an example of the "translation voice output operation means" according to the present invention, and the display 4 corresponds to the "commentary information according to the present invention One example of “output means”, one example of “work information output means”, one example of “pronunciation promotion means”, one example of “repeated commentary information output means”, one example of “Japanese image display means” and “translational coincidence information output means Each corresponds to an example of

  In the above configuration, the communication unit 20 of the server device SV controls the exchange of various data between each terminal device T and the server device SV via the network NW under the control of the processing unit 21. As a result, the processing unit 21 uses the data, which will be described later, stored in the database 22 and exchanges data with the terminal devices T via the network NW and the communication unit 20 in the first embodiment. Execute language learning process related to

  On the other hand, under the control of the processing unit 1, the communication unit 2 of each terminal device T controls exchange of various data with the server device SV via the network NW. On the other hand, the operation unit 3 includes operation buttons such as a skip button for skip operation described later, a retry button for retry operation, a recording button for recording operation, an answer confirmation operation button for answer confirmation operation, a mouse, etc. An operation signal corresponding to the operation of the operation button or the like is generated and output to the processing unit 3. At this time, each of the operation buttons may be an operation button that can be physically pressed, such as a keyboard, or the position of the touch panel corresponding to the operation icon displayed on the display 4 corresponding to each operation button An operation corresponding to the operation button may be realized by touching.

  Next, the recording unit 5 and the memory 6 record, under the control of the processing unit 1, data or the like necessary for the language learning process according to the first embodiment executed by the processing unit 1 in a volatile or non-volatile manner. These data and the like are read by the processing unit 1 as necessary, and are provided to the language learning processing according to the first embodiment. In addition, the display 4 displays an image or the like described later, which is necessary in the language learning process according to the first embodiment, under the control of the processing unit 1. Furthermore, under the control of the processing unit 1, the speaker 7 outputs (sounds off) a sound or a sound described later necessary in the language learning process according to the first embodiment. Finally, the microphone 8 collects the voice emitted by the user of the terminal device T, and outputs it to the processing unit 1 as a voice signal.

  Next, the listening step processing unit 10 of the processing unit 1 uses the image display by the display 4 and the output of the voice and the like by the speaker 7 based on the operation signal described above to perform later-described listening in language learning processing according to the first embodiment. Mainly execute the process related to the step. Further, the repeat step processing unit 11 of the processing unit 1 mainly performs the process related to the repeat step described later in the language learning process according to the first embodiment using the image display and the output of the voice and the like based on the operation signal. Run. In addition, the record step processing unit 12 of the processing unit 1 mainly performs the process related to the later-described record step in the language learning process according to the first embodiment using the image display and the output of the voice based on the operation signal. Run. Finally, based on the operation signal, the translation step processing unit 13 of the processing unit 1 relates to a translation step described later in the language learning processing according to the first embodiment using the image display and the output of the voice and the like. The processing is mainly performed.

  Here, an overview of the listening step, the repeat step, the record step, and the translation step that constitute the language learning process according to the first embodiment will be described. Here, the listening step is an example of the "viewing step" according to the present invention, the repeat step is an example of the "repeating step" according to the present invention, and the record step is an example of the "recording step" according to the present invention. The translation step corresponds to an example of the “translation step” according to the present invention.

  First, the listening step is composed of four listening scenes. This listening scene corresponds to an example of the "viewing scene" according to the present invention. In one listening scene, two sentences of an image including a character having a preset action and an English sentence expressing the action of the character in English are displayed on the display 4. At this time, a grammatical relationship to be learned as the listening scene is set in advance between the two English sentences. The predetermined grammatical relationship is, for example, so-called "temporal agreement" or the like. Further, in one listening scene, an image including a character and each corresponding English sentence are displayed, and at the same time, an audio obtained by reading the English sentence in English is outputted through the speaker 7. In each English sentence displayed at this time, a word (for example, a verb or the like) to be alerted to the user as a corresponding grammatical relation is displayed in a display mode different from other words so as to emphasize it. Examples of different display modes in this case include a display mode in which only the word to be emphasized is underlined, and a display mode in which a font or a display color different from other words is displayed.

  In addition to this, in one listening scene, the display mode of the word displayed as the voice in English is the other word in which the corresponding voice is not output at the output timing of the voice obtained by reading the word aloud (ie, reading aloud The displayed voice is displayed differently from the display mode of the other words in which the voice is output, before or after the word in which the voice is output. In this case, examples of the different display modes include display modes in which the display color, font, size at the time of display, etc. are made different. The above-mentioned display control processing for displaying the display mode of the word in English in different manner from the display mode of the other word at the output timing of the speech in which the word in the English aloud is the language learning process according to the first embodiment It is also used in the other above-mentioned repeat step in above, the above-mentioned record step and the above-mentioned translation step. Therefore, such display control is hereinafter referred to as "English karaoke-like display".

  Then, in the four listening scenes constituting one listening step, two sentences each of English sentences expressing the action of the character in English are displayed, but the same above-mentioned grammatical relationship is set in advance for the four listening scenes. There is. In one listening step, four listening scenes which express the action of the character in English and which includes two sentences each having English sentences in the same grammatical relationship between the listening scenes are used once for different English sentences representing the actions of different characters. Repeated. Then, after the four listening scenes are finished, an explanation about the above-mentioned grammatical relation which is the same among the four listening scenes is displayed on the display 4.

  Next, the repeat step will be described. The repeat step is composed of four repeat scenes. This repeat scene corresponds to an example of the "repeated scene" according to the present invention. In one repeat scene, an image including a character performing the same operation as the above-mentioned listening step and an English sentence expressing the operation of the character in English are displayed on the display 4 in two sentences. At this time, the same grammatical relationship as the above-mentioned listening step is set in advance between the two English sentences.

  More specifically, in one repeat scene, first, an image including a character and a corresponding first English sentence are displayed, and at the same time, a voice in which the first English sentence is read in English is used as the speaker 7. Output through The control of the display mode of the word to be alerted in the first English sentence displayed at this time and the display of the English karaoke like manner are the same as the listening step. Next, in the one repeat scene, after the display of the first English sentence and the output of the corresponding aloud reading voice, a voice instructing (propelling) a user's pronunciation in English imitating the reading aloud voice is output . Thereafter, in the one repeat scene, only the first English-language karaoke-like display of the first English language except for the audio of the aloud corresponding to the first English language is executed. Thereafter, in the one repeat scene, the corresponding image and the second English English karaoke-like display of the second English corresponding to the above image, the output of the second English English speech, and the second Only the output of the pronunciation instruction voice corresponding to the English sentence and the English karaoke like indication of the second English sentence except the speech of the reading aloud corresponding to the second English sentence are repeated as in the first English sentence.

  The same grammatical relationship as that of the above-described listening scene is set in advance for the two English sentences displayed on four repeat scenes constituting one repeat step. In one repeat step, the four repeat scenes are repeated once for different English sentences representing the same different character movement as the listening step. Then, after the four repeat scenes are finished, an explanation of the same grammatical relationship among the four repeat scenes (that is, the same grammatical relationship as the grammatical relationship in the listening scene) is displayed on the display 4 Be done.

  Next, the above record step will be described. The record step is composed of four record scenes. This record scene corresponds to an example of the "recording scene" according to the present invention. In one record scene, an image including a character that performs the same operation as the listening step and the repeating step and an English sentence expressing the operation of the character in English are displayed on the display 4 in two sentences. At this time, the same grammatical relationship as the listening step and the repeating step is set in advance between the two English sentences.

  More specifically, in one record scene, at the same time as the image including the character and the corresponding first English sentence are displayed, the sound of the first English sentence read in English is used as the speaker 7. Output through The control of the display mode of the word to be alerted in the first English sentence displayed at this time and the display of the English karaoke like manner are the same as the listening step and the repeating step. Next, in the one record scene, after the display of the first English sentence and the output of the corresponding read aloud voice, the user's pronunciation in English imitating the read aloud voice is recorded, and further the recorded user's Audio is played back. After that, in the one record scene, only the output of the sound of the reading aloud corresponding to the first English sentence and the English karaoke-like display of the first English sentence are executed. Thereafter, in the one record scene, the second image corresponding to the above-mentioned image and the corresponding image and the second English-language karaoke-like display of the second English sentence, recording and reproduction of the voice reading voice of the second English user And, the output of the English reading speech corresponding to the second English sentence and the English karaoke like indication of the second English sentence are repeated in the same manner as the first English sentence.

  The same grammatical relationship as the above-mentioned listening scene and the above-mentioned repeat step is set in advance for the two English sentences displayed by four record scenes constituting one record step. In one record step, the four record scenes are repeated once for different English sentences representing the same different character movement as the listening step and the repeating step. Then, after the four record scenes are finished, labor information such as “Good Job !!” indicating that the user who finished the four record scenes is worked is displayed on the display 4.

  Finally, the translation step will be described. The translation step consists of four translation scenes. This translation scene corresponds to an example of the "translation scene" according to the present invention. In one translation scene, an image including a character that performs the same operation as the above-mentioned listening step to the above-mentioned recording step and a day text representing the operation of the character in Japanese are displayed on the display 4 in a single sentence. At this time, the grammar in the Japanese text is in accordance with the grammatical relationship in each of the listening step to the recording step.

  More specifically, in one translation scene, first, an image including a character and a corresponding day text are displayed. Thereafter, in the one translation scene, after the display of the Japanese text, the voice of the user who translated the Japanese into English and is pronounced is recorded, and the recorded voice is reproduced. After that, in the one translation scene, the output of the sound obtained by reading the English text (that is, the English text as the answer) translated from the Japanese text into English and the English karaoke-like display of the English text are executed.

  With regard to the text and the like of the answer and the corresponding text displayed on the four translation scenes constituting one translation step, the same grammar as the above-mentioned listening scene to the above-mentioned record step in the four translation scenes. The relationship is preset. In one translation step, the four translation scenes are repeated four times for different Japanese sentences representing the same different character actions as the listening step and the recording step.

  As described above, in the language learning process according to the first embodiment, the listening step includes four listening scenes, the repeat step includes four repeat scenes, the record step includes four record scenes, and the translation step. Are each made up of four translation scenes. And in each step, English learning according to the movement of the same character is performed along the same grammatical relation. According to such language learning processing according to the first embodiment, it is possible to effectively perform English learning in accordance with the so-called pattern practice concept.

  In addition, it is also possible to record collectively the data of the image and sound displayed or output in each scene mentioned above, for example in the recording part 5 of each terminal device T beforehand. However, when the terminal device T is, for example, a tablet terminal having a small recording capacity, it is centrally recorded in the database 22 of the server device SV connected thereto, and acquired (downloaded) as necessary. It is efficient to use Further, when a plurality of users perform language learning according to the first embodiment, it is preferable that score management and the like of each user be performed centrally in the server apparatus SV. Therefore, when using the server apparatus SV according to the first embodiment in such a case, for example, data as shown in FIG. 2 needs to be recorded as the database 22.

  That is, as shown in FIG. 2, the scene data QD, the learning history data HD, and the image data GD are stored in the database 22 for recording image and audio data for each scene and data for performance management in the above case. And item data ITD are preferably recorded. At this time, as the scene data QD, for example, audio data for each of the above-mentioned scenes with difficulty according to school age, and English or Japanese text data are recorded so as to be mutually distinguishable by a school age ID indicating the school age or the like. Further, as the learning history data HD, data indicating the past learning history for each user and the degree of progress thereof are recorded so as to be mutually distinguishable by the user ID. Furthermore, as the image data GD, the image data for each of the above-mentioned scenes are recorded so as to be mutually distinguishable. Finally, as item data ITD, data of an item corresponding to a reward given to the user who finished the above-mentioned listening step through the above-mentioned translation step is recorded in an identifiable manner for each item. The item and its data will be described in detail later.

(Ii) Regarding the language learning process according to the first embodiment Next, the whole of the language learning process according to the first embodiment, which is executed in the language learning system S having the above-described configuration, is specifically shown in FIGS. This will be described using 3. FIG. 3 is a flowchart showing the entire language learning process according to the first embodiment. The language learning process according to the first embodiment shown in FIG. 3 is a process that is mainly executed around the processing unit 1 of each terminal device T.

  As the whole is shown in FIG. 3, in the language learning process according to the first embodiment using the language learning system S including the server device SV, initially, it is preset using the operation unit 3 of the terminal device T. Login processing is performed by executing the login operation (step S1). Specifically, as the login process, for example, login process similar to the conventional one is performed, such as exchange of login information with the server apparatus SV, personal identification process, and the like.

  Next, the processing unit 1 acquires various data necessary for the logged-in user to execute the language learning process according to the first embodiment from the server apparatus SV via the network NW (step S2). In step S2, the processing unit 1 searches the database 22 of the server apparatus SV for learning history data HD related to the user who has performed the login process in step S1. In addition to this, the processing unit 1 adds scene data QD corresponding to the degree of difficulty of the next stage corresponding to the learning history of the user indicated by the acquired learning history data HD and the image data GD corresponding thereto. Search and acquire at. The acquired image data GD and scene data QD are subjected to language learning processing according to the first embodiment, which is executed later.

  Next, the listening step (step S10) by the listening step processing unit 10 of the processing unit 1, the repeat step (step S20) by the repeat step processing unit 11, the record step (step S30) by the record step processing unit 12 and the transformer The translation step (step S40) by the translation step processor 13 is executed in this order. Each of these steps will be described in detail later.

  Next, the processing unit 1 obtains item data ITD corresponding to an item corresponding to a reward for the user due to the completion of the above-mentioned listening step to the above-mentioned translation step from the server apparatus SV ( Step S3). For example, a map showing a virtual area is set in advance as this item, and in the case of building a building or playground equipment preferred by the user one by one on the map and finally constructing a large amusement park etc. Each of the buildings or playground equipment corresponds to the above item. Then, every time the listening step to the translation step having a high level of difficulty or the translation step are finished (completed), item data ITD corresponding to a user's favorite item is acquired, and finally the user likes an amusement It is possible to construct the ground virtually. This allows the user to work on language learning according to the first embodiment while maintaining high motivation at all times. In addition, examples of concrete items and an amusement park virtually constructed using the items will be described in detail later.

  Thereafter, the processing unit 1 uploads data indicating a learning record including the above-mentioned listening step to the above-mentioned translation step at that time to the server apparatus SV, and records this in the learning history data HD of the user (step S4). .

  Then, the processing unit 1 determines whether or not a preset logout operation using the operation unit 3 of the terminal device T has been performed (step S5). If it is determined in step S5 that the logout operation is not performed (step S5: NO), the processing unit 1 assumes that the language learning process according to the first embodiment of the next stage is performed, Returning to step S2, the series of processes described above are repeated. On the other hand, when it is determined that the logout operation has been performed by the determination of step S5 (step S5: YES), the processing unit 1 executes a necessary logout process (step S6). Specifically, as the logout process, for example, a logout process similar to the conventional one, such as exchange of logout information with the server apparatus SV, is performed. After that, the processing unit 1 ends the language learning process according to the first embodiment.

(Iii) About the listening step according to the first embodiment Next, the listening step according to the first embodiment (refer to step S10 in FIG. 3), which is executed centering on the listening step processing unit 10, will be more concretely illustrated. This will be described using FIG. FIG. 4 is a flowchart showing the contents of the listening step, and FIGS. 5 and 6 are diagrams showing examples of images displayed in the listening step.

  As shown in FIG. 4, as the listening step (refer to step S10 in FIG. 3) according to the first embodiment, the listening step processor 10 first uses the image data GD and the scene data QD acquired in step S2. The output of the first listening scene and two sentences of the above grammatical relationship with each other, one sentence each, the speech of the English sentence read in English using the speaker 7 and the English sentence of the English sentence using the display 4 A karaoke like display is executed (step S100 and step S101).

  Here, an image displayed on the display 4 by the listening step processing unit 10 in the step S100 and the step S101 will be described with reference to FIG.

  In step S100 in which a voice obtained by reading the first English sentence aloud in English of the above two sentences is output, as shown in FIG. 5, together with an image GS11 including a character having an action expressed in the first English sentence As a first English sentence, English ST111 is displayed like English karaoke. The content of the English sentence ST111 illustrated in FIG. 5 is "I know that she is cooking dinner for us." Then, in the English karaoke-like display of the English sentence ST111, the words “know” and “is”, which are words that should be alerted to the user as the corresponding grammatical relation, are different from other words in italics to emphasize it. The characters are displayed in different display colors (for example, red). In addition to this, in step S100, with respect to the word in English ST 111 (for example, "cooking" shown in FIG. 5) outputted as voice as an English karaoke-like display, at the output timing, other words other than the word It is displayed with a character modification distinguishable from words (that is, other words from which speech is output before or after "cooking" in English ST111). Although this character modification is illustrated as a marker MK in FIG. 5, in addition to the marker MK itself being attached at the output timing of the voice, the word (character) itself is not output voice It may be displayed differently from the letter of the word. As described above, the listening step processing unit 10 performs the speech output of English ST111 and the display of English karaoke like step S100. At this time, the English sentence ST112 as the second English sentence of the above two sentences has a lower visibility than the English sentence ST111 (that is, for example, using a color that blends in with the background color). Along with the display 4. As this English ST 112 does not display the voice read aloud in English, the English karaoke like display of it is not performed, but it is a word that should be alerted to the user as the corresponding grammatical relation In ST112, "knew" and "was" are displayed in italics as in English ST111.

  Next, in step S101 where a voice obtained by reading the aloud of the English ST112 in English is outputted, as in the case of the above-mentioned English ST111, the English ST112 together with the image GS11 including the character represented by the English ST112. Is displayed like English karaoke. The content of the English sentence ST112 illustrated in FIG. 5 is "I knew that she was cooking dinner for us." In this case, the grammatical relationship corresponding to the English ST 111 and the English ST 112 is “temporal match”. Then, in the English karaoke-like display of the English ST112, the words “knew” and “was”, which should be alerted to the user as the corresponding grammatical relation, are displayed in the same italics and display color as the English ST111. Ru. In addition to this, in step S101, the listening step processing unit 10 performs English karaoke like display similar to English ST111 as English karaoke like display of English ST112. As described above, the listening step processing unit 10 performs the speech output of English ST112 and the display of English karaoke like step S101. At this time, for English sentence ST111 for which English karaoke like display and output of English reading voice are performed in step S100, like English sentence ST112 for step S100, English karaoke like display and output of English reading voice are currently performed. It may be displayed on the display 4 together with the English ST 112 so that the visibility is low with respect to the existing English ST 112, or displayed on the display 4 so as not to perform the English karaoke like display while having the same visibility as the English ST 112 You may

  After executing the above step S100 and the above step S101, the listening step processing unit 10 operates the retry button of the operation unit 3 by the user who desires that the above step S100 and the above step S101 be executed again It is monitored whether it has been done (step S102). If a retry operation is performed in the monitoring of step S102 (step S102: YES), the listening step processing unit 10 returns to step S100 and executes step S100 and step S101 again. On the other hand, when the retry operation is not performed within the waiting time set in advance, for example, in the monitoring of step S101 (step S102: NO), the listening step processing unit 10 next translates the Japanese translations of the English ST111 and the English ST112. It displays on the display 4 (step S103).

  On the other hand, in parallel with the execution of the above steps S100 to S103, the listening step processor 10 does not execute the above steps S100 and S101 until the end but the user wants to proceed to the next listening scene. It is monitored whether or not the skip operation has been performed by operating the above-described skip button 3 (step S104. NO at step S104). When the skip operation is performed in the monitoring of step S104 (step S104: YES), the listening step processing unit 10 proceeds to step S105 described later.

  Then, when step S103 is performed or when the skip operation is performed in the monitoring of step S104 (step S104: YES), the listening step processing unit 10 sets the number of scenes preset as the listening step ( In the case of the first embodiment, it is determined whether or not the listening scene (ie, the above steps S100 to S104) has been repeated by 4) (step S105). If it is determined in step S105 that the predetermined number of scenes has not been repeated (step S105: NO), the listening step processing unit 10 switches to the next listening scene preset as the listening step (step S106), The above steps S100 to S105 are repeated for the next switched listening scene. Here, as illustrated in FIG. 6, the listening step according to the first embodiment will be described with reference to FIG. 5 and a first listening scene (FIG. 6 (a)) including the image GS11 and two English ST111 and two English ST112. 6), the second listening scene consisting of two English sentences ST121 and ST122, the image GS13 exemplified in FIG. 6 (c) and the second listening scene consisting of two English sentences ST131 and ST132). 3 listening scene, and an image GS 14 illustrated in FIG. 6D, and a fourth listening scene consisting of two English ST 141 and English ST 142. And in the above-mentioned step S100 and the above-mentioned step S101 at the time of each listening scene being executed, as shown in FIG. 6 (a) to FIG. Is performed for each listening scene.

  Returning to FIG. 4, when the predetermined number of scenes is repeated in the determination of step S105 (step S105: YES), the listening step processing unit 10 explains the above-mentioned grammatical relationship which is the same among the four listening scenes. Are displayed on the display 4 as the comment Cm illustrated in FIG. The data (character data) for the explanation Cm is also used as previously acquired in step S2. Thereafter, the listening step processing unit 10 ends the listening step according to the first embodiment, whereby the processing unit 1 shifts to the repeat step according to the next first embodiment.

  In the listening step described with reference to FIGS. 4 to 6, although the presence or absence of the retry operation in step S102 is monitored after steps S100 and S101 end for one listening scene, step S100 ends other than this. After that, the presence or absence of the retry operation at the step S100 may be monitored, and furthermore, the presence or absence of the retry operation at the step S101 may be monitored after the step S101 is completed.

(Iv) Next, with regard to the repeat step according to the first embodiment, a diagram more specifically for the repeat step according to the first embodiment (see step S20 in FIG. 3) to be executed centering on the repeat step processing unit 11 It demonstrates using FIG. 7 and FIG. FIG. 7 is a flowchart showing the contents of the repeat step, and FIG. 8 is a view showing an example of an image displayed in the repeat step.

  As shown in FIG. 7, as the repeat step (see step S20 in FIG. 3) according to the first embodiment, the repeat step processing unit 11 first uses the image data GD and the scene data QD acquired in step S2. Then, the output of English reading speech of English ST 211 as the first English equivalent to the first repeat scene and English karaoke like display are performed (step S 200). Image data GD and scene data QD in this case are image data GD and scene data QD similar to those used in the above-mentioned listening step. Then, as exemplified in FIG. 8A, the repeat step processing unit 11 displays the image GS21 similar to the above first listening scene and the English karaoke like display of ST211 in English, as shown in FIG. 8A in step S200. Output the English reading speech of ST211. In this case, even in the English karaoke-like display of English ST211 in this case, as in the case of the above-mentioned listening step, the word to be alerted to the user as the corresponding grammatical relation is different from other words to emphasize it. It is displayed in italics and different display colors. Also, the English sentence ST212 as the second English sentence corresponding to the first repeat scene is displayed on the display 4 together with the English sentence ST211 so that the visibility with respect to the English sentence ST211 is low similarly to the listening step.

  When the above step S200 is completed, next, the repeat step processing unit 11 outputs from the speaker 7 a sound generation instruction sound indicating that the same English sound is to be produced in conformity with the English speech read sound output in the step S200 (step S201). Then, as shown in FIG. 8B, the repeat step processing unit 11 displays, for example, an image GS21 w in which the entire tone is thinned in addition to the display of the English karaoke like the above-mentioned English ST 211 (step S202). In this step S202, the output of the English reading speech in English ST211 is not performed. Thus, while the step S202 is being performed, the user of the terminal device T repeats (replays) the English reading speech output in the step S200.

  Thereafter, the repeat step processing unit 10 monitors whether the user desiring to execute the steps S200 to S202 again operates the retry button of the operation unit 3 to perform the retry operation (step S203). . If a retry operation is performed in the monitoring of step S203 (step S203: YES), the repeat step processing unit 11 returns to step S200 and executes step S200 to step S203 again.

  On the other hand, if a retry operation is not performed, for example, within a preset waiting time in the monitoring of step S203 (step S203: NO), repeat step processing unit 11 next uses image data GD and scene data QD. The output of the English reading speech of English ST 212 (see FIG. 8A) as the second English corresponding to the first repeat scene and English karaoke-like display are performed (step S 204). Image data GD and scene data QD in this case are also image data GD and scene data QD similar to those used in the above-mentioned listening step. Then, in step S204, the repeat step processing unit 11 outputs the English sound reading voice of English ST212 while displaying the image GS21 similar to the first listening scene and displaying the English karaoke like English ST212 as its contents.

  When the above step S204 is completed, next, the repeat step processing unit 11 outputs, from the speaker 7, a sound generation instruction sound similar to the above step S205 (step S205). Then, the repeat step processing unit 11 displays, for example, an image GS21 w in which the entire tone is thinned in addition to the display of English karaoke like the above-mentioned English ST 212 (step S206). In this step S206, the output of the English reading speech in English ST212 is not performed. Thereby, while the said step S206 is performed, the user of the terminal device T will repeat the English reading speech output by step S206.

  Thereafter, the repeat step processing unit 10 monitors whether or not the user desiring to execute steps S204 to S206 again performs a retry operation (step S207). When a retry operation is performed in the monitoring of step S207 (step S207: YES), the repeat step processing unit 11 returns to step S204 and executes step S204 to step S207 again.

  On the other hand, in parallel with the execution of the steps S200 to S207, the repeat step processing unit 11 performs a Japanese translation of the English sentence ST211 while executing the steps S200 to S203 and executes the steps S204 to S207. In step S208, the Japanese translation of English ST212 is displayed on the display 4 (step S208), and the process proceeds to step S210 described later.

  Furthermore, the repeat step processing unit 11 controls the operation unit 3 by the user who wants to proceed to the next repeat scene without executing the steps S200 to S207 in parallel with the execution of the steps S200 to S207. It is monitored whether or not the skip operation is performed by operating the above-described skip button (step S209. NO at step S209). When the skip operation is performed in the monitoring of step S209 (step S209: YES), the repeat step processing unit 11 proceeds to step S210 described later.

  Next, while the step S208 is being executed, whether the retry operation is not performed within the waiting time set in advance, for example, in the monitoring of the step S207 (step S207: NO), or the skip operation is performed during the monitoring of the step S209. (Step S209: YES), the repeat step processing unit 11 repeats repeat scenes (that is, the above steps S200 to S209) by the number of scenes (four in the first embodiment) set in advance as repeat steps. It is determined whether or not it has been received (step S210). If it is determined in step S210 that the predetermined number of scenes is not repeated (step S210: NO), the repeat step processing unit 11 switches to the next repeat scene preset as the repeat step (step S211), The above-described steps S200 to S210 are repeated for the next repeated scene that has been switched.

  Here, the repeat step according to the second embodiment is, as exemplified in FIG. 8, a first repeat scene (see FIG. 8A) including an image GS21 and two English ST211 and English ST212, and FIG. And a second listening scene consisting of two English ST221 and two English ST222, and the same image as the two English sentences illustrated in FIGS. 6 (c) and 6 (d) in the listening step. It consists of a third repeat scene and a fourth repeat scene consisting of an image and two English sentences, respectively. Then, in steps S200 to S207 when the respective repeat scenes are executed, as shown in FIG. 8A to FIG. 8D, the English karaoke like display such as each English ST211 etc. It is executed for each repeat scene. At this time, when causing the user of the terminal apparatus T to perform the repeat as the second repeat scene (see step S202 or step S206), the repeat step processing unit 11 exemplifies as shown in FIG. In addition to the English karaoke-like display of English ST221 (in the case of step S202) or the English karaoke-like display of English 222 (in the case of step S206), for example, the image GS22w in which the entire tone is lightened is displayed.

  Referring back to FIG. 7, when the predetermined number of scenes is repeated in the determination of step S210 (step S210: YES), the repeat step processing unit 11 interprets the grammatical relationship which is the same among the four repeat scenes. (Ie, an explanation of the same grammatical relationship as the grammatical relationship corresponding to the listening step) is displayed on the display 4 as a comment Cm illustrated in FIG. 8E (step S212). The data (character data) for the explanation Cm is also used in the same manner as in the case of the listening step, which is acquired in advance in the step S2. Thereafter, the repeat step processing unit 11 ends the repeat step according to the first embodiment, whereby the processing unit 1 shifts to the record step according to the next first embodiment.

(V) Next, with regard to the record step according to the first embodiment, the figure more specifically shows the record step according to the first embodiment (refer to step S30 in FIG. 3) to be executed centering on the record step processing unit 12 It demonstrates using 9 and FIG. FIG. 9 is a flowchart showing the contents of the record step, and FIG. 10 is a view showing an example of an image displayed in the record step.

  As shown in FIG. 9, as the record step (see step S30 in FIG. 3) according to the first embodiment, the record step processing unit 12 first uses the image data GD and the scene data QD acquired in the step S2. Then, the output of English reading speech of English ST 311 as the first English equivalent to the first record scene and English karaoke like display are performed (step S 300). Image data GD and scene data QD in this case are image data GD and scene data QD similar to those used in the listening step and the repeat step. Then, the record step processing unit 12 displays the image GS31 similar to the first listening scene or the first repeat scene as its contents as shown in FIG. While displaying, output the English reading speech of English ST311. In this case, as in the case of the English karaoke like display of English ST311 in this case, as in the case of the listening step and the repeating step, the word that should be alerted to the user as the corresponding grammatical relation emphasizes it. The words are displayed in different italics and different display colors. Also, the English sentence ST312 as the second English sentence corresponding to the first record scene is displayed on the display 4 together with the English sentence ST311 so that the visibility with respect to the English sentence ST311 becomes low similarly to the listening step and the repeat step. Is displayed.

  When the above step S300 is completed, the record step processing unit 12 next records the above-mentioned recording button of the operation unit 3 for the user who wants to record and reproduce his / her English voice following the English reading voice output in step S300. To monitor whether the recording operation has been performed (step S301). When the recording operation is not performed in the monitoring of step S301 (step S301: NO), next, the record step processing unit 12 determines whether or not the standby time set in advance has passed since the execution of the step S300. (Step S302). If it is determined in step S302 that the standby time has not elapsed (step S302: NO), the record step processing unit 12 returns to step S301 to determine execution of the recording operation. On the other hand, when the waiting time has elapsed in the determination of step S302 (step S302: YES), the record step processing unit 12 returns to the step S300 and repeats the step S300.

  On the other hand, when the recording operation is performed in the monitoring of step S301 (step S301: YES), the record step processing unit 12 uses the microphone 8 to speak the English voice of the user who will be pronounced immediately after the recording operation. The sound is collected, and an audio signal as the sound collection result is temporarily recorded, for example, in the recording unit 5 using a general-purpose audio file format set in advance (step S303). During execution of step S303, the record step processing unit 12 adds, for example, the image GS31d in which the entire tone is darkened, to the English karaoke-like display of the above-described English ST311, as illustrated in FIG. A graphic imitating a waveform along an axis is superimposed and displayed on the display 4. Thereby, the user executing the step S303 can intuitively recognize that his / her voice is recorded.

  Next, after a predetermined time has elapsed since the end of step S303, the record step processing unit 12 reproduces the English voice of the user recorded in step S303 (step S304). After that, the record step processing unit 12 again performs the output of the English reading speech of the English sentence ST 311 and the English karaoke like display again as in the above step S 300 (step S 305). By executing the above steps S304 and S305, the correct pronunciation can be acquired by comparing the own English voice (step S304) recorded in step S303 with the correct English voice (step S305).

  Thereafter, the record step processing unit 12 monitors whether or not the user desiring to execute the steps S300 to S306 again operates the retry button of the operation unit 3 to perform the retry operation (step S306). . When a retry operation is performed in the monitoring of step S306 (step S306: YES), the record step processing unit 12 returns to step S300 and executes step S300 to step S306 again.

  On the other hand, when the retry operation is not performed within the waiting time set in advance, for example, in the monitoring of step S306 (step S306: NO), the record step processing unit 12 next uses the image data GD and the scene data QD. Output of English reading speech and English karaoke like display of English ST 312 (see FIG. 10A) as a second English corresponding to the first record scene are performed (step S307). The image data GD and the scene data QD in this case are also image data GD and scene data QD similar to those used in the listening step and the repeating step. Then, in step S307, the record step processing unit 12 outputs the English sound reading voice of English ST312 while displaying the image GS31 similar to the first record scene and displaying the English karaoke like English ST312 as its contents.

  When step S307 is completed, the record step processing unit 12 next records the above-mentioned recording button of the operation unit 3 for the user who wants to record and reproduce his / her English voice following the English reading voice output in step S307. To monitor whether the recording operation has been performed (step S308). When the recording operation is not performed in the monitoring of step S308 (step S308: NO), next, the record step processing unit 12 determines whether or not the standby time set in advance has passed since the execution of step S307. (Step S309). If it is determined in step S309 that the standby time has not elapsed (step S309: NO), the record step processing unit 12 returns to step S307 to determine execution of the recording operation. On the other hand, when the waiting time has elapsed in the determination of step S309 (step S309: YES), the record step processing unit 12 returns to the step S307 and repeats the step S307.

  On the other hand, when the recording operation is performed in the monitoring of step S308 (step S308: YES), the record step processing unit 12 uses the microphone 8 to speak the English voice of the user who will be pronounced immediately after the recording operation. The sound is collected, and an audio signal as the sound collection result is temporarily recorded, for example, in the recording unit 5 in the same manner as the step S303 (step S310). During the execution of step S310, the record step processing unit 12 superimposes, for example, an image GS31d in which the entire tone is darkened in addition to the English karaoke-like display of the above-mentioned English ST312, a graphic imitating a waveform along the audio time axis The display 4 is displayed. Thus, the user who is executing step S310 can intuitively recognize that his / her voice is being recorded, as in step S303. Next, after a predetermined time has elapsed since the end of step S310, the record step processing unit 12 reproduces the English voice of the user recorded in step S310 (step S311). Thereafter, the record step processing unit 12 again performs the output of the English reading speech of the English sentence ST 312 and the English karaoke-like display as in the case of the step S 307 (step S 312). By executing the above steps S311 and S312, it is possible to acquire correct pronunciation by comparing the own English voice (step S311) recorded in step S310 with the correct English voice (step S312).

  Thereafter, the record step processing unit 12 monitors whether or not the user desiring that the steps S307 to S312 are to be executed again performs the retry operation by operating the retry button of the operation unit 3 (step S313). . When a retry operation is performed in the monitoring of step S313 (step S313: YES), the record step processing unit 12 returns to step S307 and executes step S307 to step S313 again.

  On the other hand, in parallel with the execution of the steps S300 to S313, the record step processor 12 does not execute the steps S300 to S313 until the end, and the user wishes to proceed to the next record scene. It is monitored whether or not the skip operation has been performed by operating the above-described skip button (step S317; step S317: NO). When the skip operation is performed in the monitoring of step S317 (step S317: YES), the record step processing unit 12 proceeds to step S314 described later.

  On the other hand, if the retry operation is not performed, for example, within a preset waiting time in the monitoring of step S313 (step S313: NO), or the skip operation is performed in the monitoring of step S317 (step S317: YES) The record step processing unit 12 determines whether or not the record scenes (that is, the steps S300 to S313 and step S317) are repeated by the number of scenes (four in the first embodiment) set in advance as the record steps. (Step S314). If it is determined in step S314 that the predetermined number of scenes is not repeated (step S314: NO), the record step processing unit 12 switches to the next record scene preset as the record step (step S315), The above steps S300 to S313 and step S317 are repeated for the next record scene switched.

  Here, the record step according to the first embodiment is, as illustrated in FIG. 10, a first record scene (see FIG. 10A) including an image GS 31 and two English ST 311 and two English ST 312; And the second record scene consisting of two English ST 321 and English ST 322, and, for example, the same image as the one illustrated in FIG. 6 (c) and FIG. 6 (d) in the listening step and the two English sentences. The third record scene and the fourth record scene, each consisting of an image of and two English sentences. Then, in steps S300 to S313 when each record scene is executed, as shown in FIGS. 10 (a) to 10 (d), English karaoke-like displays such as English sentences ST311 etc. Executed for each record scene. At this time, when making the user of the terminal device T record the voice of the user as the second record scene (see the above step S303 or the above step S310), the record step processing unit 12 exemplifies as shown in FIG. In addition to the English karaoke-like display of English ST321 (in the case of step S303) or the English karaoke-like display of English ST322 (in the case of step S310), for example, the image GS32d in which the entire tone is darkened is Are superimposed on each other and displayed on the display 4.

  Returning to FIG. 9, if it is determined in step S314 that the predetermined number of scenes has been repeated (step S314: YES), the record step processing unit 12 uses the end of the four record scenes as labor information. For example, the characters "Good Job!" Are displayed on the display 4, and the sound of "Good Job!" Is output from the speaker 7 (step S316). Thereafter, the record step processing unit 12 ends the record step according to the first embodiment, whereby the processing unit 1 shifts to the translation step according to the next first embodiment.

  In the above-described record step, step S304 and step S305 are separately executed separately, but in addition to this, in parallel with the reproduction of the English voice of the user as step S304, the step is performed. It is also possible to configure to output an English reading speech as S305 and to display English karaoke like. In this case, the output of the English reading voice and the English karaoke-like display as step S305 may be started simultaneously with the start timing of the reproduction of the English voice of the user as step S304, or as step S304. The output of the English reading voice and the display of the English karaoke like in step S305 may be started with a slight delay from the start timing of the reproduction of the English voice of the user. Furthermore, in the same manner as this, in parallel with the reproduction of the user's English speech as the step S311 of the record step, the output of the English speech reading speech as the step S312 and the English karaoke-like display can be performed. Also in this case, the output of the English reading voice and the English karaoke-like display as step S312 may be started simultaneously with the start timing of the reproduction of the English voice of the user as step S311, or step S311 It may be configured to start output of English reading speech and English karaoke-like display as step S312 slightly delayed from the start timing of reproduction of the user's English speech.

(Vi) Regarding the translation step according to the first embodiment Finally, the translation step (see step S40 in FIG. 3) according to the first embodiment, which is executed centering on the translation step processor 13, is more specific This will be described using FIG. 11 to FIG. FIG. 11 is a flowchart showing the contents of the translation step, and FIGS. 12 and 13 are diagrams showing examples of images displayed in the translation step.

  As shown in FIG. 11, as the translation step (see step S40 in FIG. 3) according to the first embodiment, the translation step processor 13 first receives the image data GD acquired in step S2 and the scene data Using the data of each Japanese translated sentence contained in the QD, the Japanese translated sentence ST41 J of the English sentence ST41 as the first English sentence corresponding to the first translation scene is displayed (step S400). The image data GD and the scene data QD in this case are image data GD and scene data QD similar to those used in the listening step to the recording step. Then, as illustrated in FIG. 12A, the translation step processing unit 13 displays the image GS41 similar to the above-mentioned first listening scene to the above-mentioned first record scene and the display of the Japanese translation sentence ST41J as step S400. , And displays the English translation problem sentence QU prompting the pronunciation of the English translation of the displayed Japanese translation sentence ST41J. In this case, in Japanese translation text ST41J at this time, a reading point is placed not at the position of a normal (natural) punctuation mark as a Japanese text but at a position corresponding to the word break in the corresponding English sentence. More specifically, for example, when the first English sentence corresponding to the first translation scene is "I know that she is cooking dinner for us.", The corresponding Japanese translation sentence ST41J is illustrated in FIG. 12 (a). As I do, "I know that she is making dinner for us,".

  When the above step S400 is completed, next, the translation step processing unit 13 performs output of English speech translated from the Japanese translated text ST41J displayed in step S400 into English and displaying of the English translation like English karaoke (that is, in step S400). It is monitored whether the user who desires output and display of an answer for the displayed English translation problem sentence QU operates the above-mentioned answer confirmation operation button of the operation unit 3 to perform an answer confirmation operation (step S401) ). If the answer confirmation operation is performed in the monitoring of step S401 (step S401: YES), the translation step processing unit 13 proceeds to step S406 described later. On the other hand, when the above-mentioned answer confirmation operation is not performed in the monitoring of step S401 (step S401: NO), next, the translation step processing unit 13 records its own English voice translated from the Japanese translation ST41J displayed in step S400. It is monitored whether the user who wants to reproduce the music operates the recording button of the operation unit 3 to perform the recording operation (step S402). When the recording operation is not performed in the monitoring of step S402 (step S402: NO), next, the translation step processing unit 13 determines whether or not the standby time set in advance has passed since the execution of step S400. It determines (step S403). If it is determined in step S403 that the standby time has not elapsed (step S403: NO), the translation step processing unit 13 returns to step S402 to determine execution of the recording operation. On the other hand, when the above standby time has elapsed in the determination of step S403 (step S403: YES), the translation step processing unit 13 returns to the above step S400 and repeats the step S400 and subsequent steps.

  On the other hand, when the recording operation is performed in the monitoring of step S402 (step S402: YES), the translation step processing unit 13 microphones the English voice of the user who will be pronounced immediately after the recording operation. The sound signal as the sound collection result is temporarily recorded in, for example, the recording unit 5 in the same manner as step S303 or step S310 in the recording step (step S404).

  Next, the translation step processing unit 13 reproduces the user's English voice recorded in step S404 after a preset time has elapsed since the end of step S404 (step S405). Thereafter, as an answer to the above-mentioned English translation problem sentence QU, the translation step processing unit 13 further performs output of English voice translated from the Japanese translation text ST41J displayed in step S400 and English karaoke display of the English translation (step S406). Whether the user performed the correct answer (translation) by comparing his own English voice (step S405) recorded in step S404 with the correct English voice (step S406) by executing steps S405 and S406 described above Can be confirmed.

  Thereafter, the translation step processing unit 13 monitors whether or not the user desiring to execute the steps S400 to S406 again operates the retry button of the operation unit 3 to perform the retry operation (step S407). ). If a retry operation is performed in the monitoring of step S407 (step S407: YES), the translation step processing unit 13 returns to step S400 and executes step S400 to step S407 again.

  On the other hand, in parallel with the execution of the above steps S400 to S407, the translation step processing unit 13 is operated by the user who wants to proceed to the next translation scene without performing the steps S400 to S407 to the end. It is monitored whether or not the skip operation has been performed by operating the skip button of part 3 (step S410. NO at step S410). When the skip operation is performed in the monitoring of step S410 (step S410: YES), the translation step processing unit 13 proceeds to step S408 described later.

  On the other hand, if the retry operation is not performed, for example, within a preset waiting time in the monitoring of step S407 (step S407: NO), or the skip operation is performed in the monitoring of step S410 (step S410: YES) The translation step processing unit 13 determines whether or not the translation scenes (that is, the steps S400 to S407 and step S410) are repeated by the number of scenes (four in the first embodiment) set in advance as the translation steps. It is determined (step S408). If it is determined in step S408 that the predetermined number of scenes has not been repeated (step S408: NO), the translation step processing unit 13 switches to the next translation scene preset as the translation step (step S409). The above-mentioned steps S400 to S407 and step S410 are repeated for the switched next translation scene.

  Here, as illustrated in FIG. 12, the translation step according to the first embodiment is a first translation scene (see FIG. 12A) including an image GS41, a Japanese translation sentence ST41J, and an English translation problem sentence QU. The second translation scene (see FIG. 12 (b)) including GS42 and Japanese translation sentence ST42J and English translation problem sentence QU, and the third translation scene including image GS43 and Japanese translation sentence ST43J and English translation problem sentence QU (FIG. 12 (c) And a fourth translation scene (see FIG. 12 (d)) including an image GS44, a Japanese translation sentence ST44J, and an English translation problem sentence QU. Then, in step S406 of each translation scene, the display of the image GS41 illustrated in FIG. 13A and the display of English karaoke like English ST41 are performed in the case of the first translation scene, and the case of the second translation scene. In the case of the third translation scene, the display of the image GS43 illustrated in FIG. 13B and the display of the English translation ST42 in English karaoke-like display are performed, and in the case of the third translation scene, the display of the image GS43 illustrated in FIG. A karaoke-like display is performed, and in the case of the fourth translation scene, the display of the image GS44 illustrated in FIG. 13D and the display of English karaoke-like display of English ST44 are performed. At this time, as is clear from comparison between FIG. 12 and FIG. 13, in Japanese translation sentence ST41J illustrated in FIG. 12A, the reading point is located at the position corresponding to the word break in English sentence ST41 illustrated in FIG. In Japanese translation sentence ST42J illustrated in FIG. 12B, a reading point is placed at a position corresponding to a word break in English sentence ST42 illustrated in FIG. 13B, and is illustrated in FIG. 12C. In Japanese translation sentence ST43J to be read, a reading point is placed at a position corresponding to the word break in English sentence ST43 illustrated in FIG. 13C, and in Japanese translation sentence ST44J illustrated in FIG. A reading point is placed at a position corresponding to the word break in English sentence ST44.

  Returning to FIG. 11, when the predetermined number of scenes is repeated in the determination of step S408 (step S408: YES), the translation step processing unit 13 proceeds to step S3 shown in FIG.

  Here, the item data ITD acquired from the server apparatus SV in the step S3 and the item etc. according to the first embodiment corresponding thereto will be described using FIG. FIG. 14 is a view showing an example of the item etc.

  In the step S3, based on the map data acquired in advance from the server apparatus SV and recorded in the recording unit 5 of the terminal apparatus T, an area including a virtual amusement park etc. illustrated in FIG. A map MP shown is displayed on the display 4. When the user of the terminal device T completes the listening step or the translation step described above (see steps S10, S20, S30 and S40 in FIG. 3) once, it is illustrated in FIG. For the map MP, one item IT (see FIG. 14B), which is one of a building or a playground equipment, is acquired. At this time, the processing unit 1 of the terminal device T of the user acquires item data ITD corresponding to the acquired item IT from the server device SV via the network NW. The user repeatedly acquires item data ITD corresponding to one item IT every time the listening step or the translation step is completed, and further maps the item IT corresponding to the acquired item data ITD. Reflect on MP. As a result, every time the user finishes the above-mentioned listening step or the above-mentioned translation step, which has a high degree of difficulty, item data ITD corresponding to the item IT of the preference is acquired, and finally the user likes the amusement of the preference A ground etc. can be built in map MP. As a result, as described above, the user can work on language learning according to the first embodiment while always maintaining high motivation. The acquisition of the item IT is performed not only every time the listening step or the translation step as described above is completed once but, for example, after each step or the listening step or the translation step is completed a plurality of times It is also possible to change in accordance with the user's school age, for example.

  As described above, according to the language learning process of the first embodiment, a plurality of images including English ST111 and the like spelled in English, which is a language to be learned, and voices uttering the English ST11 and the like Perform a listening step to output as a listening scene (see step S10 of FIG. 3 and FIGS. 4 to 6), and then execute a recording step consisting of a plurality of record scenes (FIG. 3 step S30 and FIGS. 9 and 10). reference). At this time, as the listening step, outputting the image GS11 etc. constituting one listening scene and the English speech reading speech is repeated as a plurality of listening scenes respectively composed of different English sentences of the same grammar. On the other hand, as a record step, the image GS31 etc. constituting one record scene and the English speech reading voice are outputted, and the voice of the user who read aloud (pronounced) the image GS31 etc. and the English ST 311 etc. Recording (see step S303 and step S310 in FIG. 9) and outputting the recorded voice (see step S304 and step S311 in FIG. 9) are performed for each of a plurality of record scenes respectively composed of different sentences of the same grammar. Repeated. Therefore, the learning effect (practice effect) of English can be improved by performing the listening step and the recording step as so-called pattern practice composed of different sentences of the same grammar.

  In the recording step, when the user performs a recording operation, the voice is collected and recorded (see step S301 and step S308 in FIG. 3). Therefore, since pronunciation and recording can be performed at an arbitrary timing of the user, English can be learned with a margin.

  Furthermore, in the listening step, the comment Cm is displayed at the timing when repetition for a plurality of (for example, four) listening scenes is completed (see step S107 in FIG. 4), the content of the grammar targeted for learning in the listening step It can be easily understood.

  Furthermore, in the recording step, since the work information is output at the timing when the repetition for a plurality of record scenes is completed (see step S316 in FIG. 9), the user's motivation to continue learning can be maintained or enhanced.

  Further, a repeat step consisting of a plurality of repeat scenes is executed between the listening step and the record step (see step S20 in FIG. 3 and FIGS. 7 and 8). At this time, as a repeat step, an image GS21 etc. constituting one repeat scene and an English speech reading speech are outputted, and an output of a pronunciation instructing speech prompting the pronunciation of a word constituting the English sentence corresponding to the image GS21 etc. Is repeated for each of a plurality of repeat scenes each composed of different sentences of the same grammar. Therefore, as the so-called pattern practice consisting of different sentences of the same grammar, the repetition step is executed in addition to the listening step and the recording step, thereby further improving the learning effect (practicing effect) of the English language to be learned Can.

  In addition, in the repeat step, the comment Cm is displayed at the timing when the repetition for a plurality of (for example, four) repeat scenes is completed (see FIG. 7, step S 212). Can be easily understood.

  Furthermore, since the number of repetitions of the listening scene in one listening step, the number of repetitions of the record scene in one record step, and the repetition number of the repeat scene in one repeat step are all the same, the learning effect as pattern practice is further added. It can be enhanced. Furthermore, since the number of repetitions is four, it is possible to prevent boredom and further enhance the learning effect as the pattern practice.

  Also, after the recording step, a translation step consisting of a plurality of translation scenes is executed (see step S40 of FIG. 3 and FIGS. 11 to 13). At this time, as a translation step, an image including a Japanese translation sentence ST41J and the like constituting one translation scene is displayed, and Japanese speech corresponding to the Japanese translation sentence ST41J and the like is recorded (see step S404 in FIG. 11). The reproduced voice is reproduced (see step S405 in FIG. 11), and after that, the output of the English voice translated from the Japanese translation text ST41J and the display of the English karaoke like English karaoke are performed (see step S406 in FIG. 11). The process is repeated for each of a plurality of translation scenes composed of mutually different Japanese translation sentences ST41J and the like. Therefore, by further executing the translation step as so-called pattern practice based on different Japanese translated sentences ST41J and the like having the same grammar, it is possible to further improve the learning effect (practicing effect) of English as a learning target.

  Also, in the translation step, when the user performs a recording operation, the translated speech is collected and recorded (see step S402 in FIG. 11). Because you can, you can learn English with a margin.

  Furthermore, in the translation step, since the position of the reading point in the Japanese translation sentence ST41 J etc. is the position corresponding to the word break in the English equivalent to the Japanese translation sentence ST 41 J etc. (see FIG. 12), more accurate speech It can be made to record etc.

  Furthermore, in the translation step, when the answer confirmation operation is performed, the English phonetic pronunciation corresponding to the displayed translated Japanese sentence ST41J and the like is output (see FIG. 11 step S401 and step S406). When it is difficult to translate, etc., English learning can be advanced without stagnation by outputting the corresponding English pronunciation by answer confirmation operation by the user.

  Also, in the translation step, the number (one) of the Japanese translated sentence ST41 J etc. corresponding to one translation scene is the number of English sentences ST 111 etc. constituting one listening scene, and the number of English sentences ST311 equaling one record scene Since the number of English sentences ST211 etc. constituting one repeat scene is smaller than any of them (each two), the number of Japanese translated sentences ST41 J etc. in the translation scene with high degree of difficulty can be suppressed to prevent a decline in learning motivation. it can.

  Furthermore, since English-language karaoke-like display is performed in the listening step, the repeat step, and the record step, it is possible to easily recognize which word is output as a voice.

  Furthermore, in an English karaoke-like display, the display color in the image of another secret word at the output timing as voice is output while being made different from the display colors of other words output as voice after the output timing. Therefore, it can be more easily recognized which word is output as speech.

  In addition, since at least the characters constituting the respective scenes of mutually corresponding listening scenes, repeat scenes, record scenes and translation scenes and their operations are the same, the same as listening step → repeat step → record step → translation step The learning effect can be further enhanced by advancing the learning by the character.

  Furthermore, when the listening step, the repeat step, the record step, and the translation step are completed, the user can acquire and record the favorite item IT (see FIG. 14). The motivation can be further increased.

(II) Second Embodiment Next, a second embodiment which is another embodiment according to the present invention will be described with reference to FIG. 15 and FIG. FIG. 15 is a flow chart showing the contents of the record step in the language learning process according to the second embodiment, and FIG. 16 is a flow chart showing the contents of the translation step in the language learning process.

  Further, the hardware configuration of the language learning system according to the second embodiment is basically the same as the hardware configuration of the language learning system S according to the first embodiment, so in the description of the second embodiment below About the component similar to the said language learning system S, the same member number is attached | subjected and description of detail is abbreviate | omitted. Further, in the language learning process according to the second embodiment, the same processes as those in the language learning process according to the first embodiment are denoted by the same step numbers, and the detailed description will be omitted.

  In the language learning process according to the second embodiment described below, the record step and the translation step are different from the record step and the translation step in the language learning process according to the above-described first embodiment. Since the listening step and the repeat step in the language learning process according to the second embodiment are the same as the listening step and the repeat step in the language learning process according to the first embodiment described above, the description of the details will be omitted.

(I) Record Steps According to the Second Embodiment First, the record steps according to the second embodiment will be specifically described with reference to FIG.

  As shown in FIG. 15, in the record step (step S30-1) according to the second embodiment, the record step processing unit 12 first performs the same process as step S300 to step S303 in the record step according to the first embodiment. Run.

  Next, in parallel with the process similar to step S304 in the record step according to the first embodiment, the record step processing unit 12 recognizes the user's English voice collected / recorded in step S303, and the recognition result is Then, the contents of the English reading aloud of English ST 311 output in step S 305 are automatically compared, and it is determined whether or not both are the same. Is the same) or "x (if both are different)" (step S320). As the method of automatic determination in step S320, for example, the same voice determination method as that in the related art can be used. Thereafter, the record step processing unit 12 executes the same processing as step S305 and step S305 in the record step according to the first embodiment.

  Thereafter, the record step processing unit 12 executes the same processing as steps S307 to S310 in the record step according to the first embodiment.

  Next, in parallel with the process similar to step S311 in the record step according to the first embodiment, the record step processing unit 12 recognizes the user's English voice collected / recorded in step S310, and the recognition result Then, the content of the English reading aloud of English ST 312 outputted in step S312 is automatically compared, and it is determined whether or not both are the same, and the result is, for example, the same as step S320 above. (Step S321). The method of the automatic determination in step S320 is the same as that of step S320. Thereafter, the record step processing unit 12 executes the same processing as steps S312 to S316 and step S317 in the record step according to the first embodiment.

(Ii) Translation step according to the second embodiment Next, the translation step according to the second embodiment will be specifically described with reference to FIG.

  As shown in FIG. 16, in the translation step (step S40-1) according to the second embodiment, the translation step processing unit 13 is the same as steps S400 to S404 in the translation step according to the first embodiment. Execute the process of

  Next, in parallel with the process similar to step S405 in the translation step according to the first embodiment, the translation step processing unit 13 recognizes and recognizes the user's English voice collected / recorded in step S404. The result is automatically compared with the contents of the English voice translated from the translated Japanese sentence ST41J output in the subsequent step S406, it is determined whether or not both are the same, and the result is the same as in step S320 above. For example, it is displayed on the display 4 by the method (step S411). The method of automatic determination in step S411 can also use, for example, the same method as step S320. Thereafter, the translation step processing unit 13 executes the same processes as steps S406 to S410 in the translation step S40 according to the first embodiment.

  As described above, according to the record step according to the second embodiment, in addition to the effects of the language learning process according to the first embodiment, the content of the voice of the recorded user is recognized, and the recognition result Since the contents of the English reading speech such as English ST311 etc. are compared and it is judged whether both are the same or not and displayed (refer to FIG. 15 step S320 or step S321), the contents of the said speech uttered by the user match Learn English quickly while recognizing whether or not

  Further, according to the translation step of the second embodiment, in addition to the effects of the language learning process of the first embodiment, the content of the recorded user's voice is recognized, and the recognition result and the Japanese translation sentence The contents of the English voice translated from ST41 J are compared, and it is judged whether or not both are the same and displayed (see step S411 in FIG. 16). Therefore, whether the contents of the voice uttered by the user are correct or not It is possible to advance the learning of the English while quickly recognizing it.

  In step S320, step S321 and step S411 of the language learning process according to the second embodiment described above, for example, the recognition result of the user's English voice collected / recorded in step S303 and the result of step S305 are output. Although the content of the English reading speech in English ST 311 is uniformly determined whether "the same or not", in addition to this, for example, the degree of coincidence between the two is calculated and displayed on the display 4 It may be configured as follows.

  Furthermore, although the same grammatical relationship is used in each step such as the listening step in each embodiment described above, part or all of the grammatical relationship may be different for each of one or more steps.

  Furthermore, although each embodiment mentioned above demonstrated the case where this invention was applied to English learning processing, this invention is not limited to this, When learning various languages other than Japanese efficiently It can apply.

  Also, programs corresponding to the flowcharts shown in FIGS. 3, 4, 7, 9, 11, 15, and 16 are recorded in a recording medium such as an optical disk or a hard disk, or through the network NW. It is also possible to cause the microcomputer or the like to function as the processing unit 1 according to each embodiment by acquiring and recording the data in a non-volatile manner and reading and executing it using a general-purpose microcomputer or the like.

  As described above, the present invention can be used in the field of language learning, and in particular, when applied to the field of language learning using a computer system, particularly remarkable effects can be obtained.

1, 21 processing unit 2, 20 communication unit 3 operation unit 4 display 5 recording unit 6 memory 7 speaker 8 microphone 10 listening step processing unit 11 repeat step processing unit 12 record step processing unit 13 translation step processing unit S language learning system T 1 , T2, T3, Tn Terminal device SV server device NW network 22 database QD scene data,
HD Learning History Data GD Image Data MK Marker Cm Explanation QU English Translation Problem Statement MP Map IT Item ITD Item Data GS11, GS12, GS13, GS14, GS21, GS21, GS21, GS22, GS22, GS31, GS31d, GS32, GS32, GS41, GS42, GS43, GS44 Image ST111, ST112, ST121, ST131, ST132, ST141, ST142, ST211, ST211, ST212, ST221, ST222, ST311, ST312, ST322, ST41, ST42, ST43, ST44 English ST41 J, ST42 J, ST43 J, ST43 J ST44J Japanese translation

Claims (21)

Viewing step execution means for executing a viewing step of outputting an image including a sentence spelled out in a language to be learned and a sound uttering the sentence as a plurality of viewing scenes;
Recording step execution means for executing a recording step consisting of a plurality of recording scenes after the viewing step;
A language learning device comprising
The viewing step executing means is
Viewing output means for outputting a viewing image and a viewing sound which are the image and the sound constituting the one viewing scene;
Viewing control means for controlling the viewing output means to repeat the output of the viewing image and the viewing audio for each of a plurality of viewing scenes respectively composed of the sentences different from each other in the same grammar;
Equipped with
The recording step execution means is
Recording output means for outputting a recorded image and a recorded voice which are the image and the voice constituting one of the recorded scenes;
Recording means for collecting and recording the voice of a learner who pronounces the recorded image and the sentence corresponding to the recorded voice;
Learner voice output means for outputting the voice of the recorded learner;
The recording image and the output of the recording voice, the collection and recording of the voice of the learner, and the output of the voice of the learner are each of a plurality of the recording scenes respectively configured by the sentences different from each other in the same grammar. Recording control means for controlling the recording output means, the recording means, and the learner voice output means, as described above;
A language learning apparatus comprising: In the language learning device according to claim 1,
It further comprises operating means operated when collecting sound and recording by the recording means,
The language learning apparatus according to claim 1, wherein the recording unit performs the sound collection and the recording when an operation using the operation unit is performed. In the language learning device according to claim 1 or 2,
A language further comprising comment information output means for outputting grammar commentary information which is an explanation of the grammar corresponding to the viewing step at a timing when repetition of the plurality of viewing scenes by the viewing control means is finished. Learning device. The language learning apparatus according to any one of claims 1 to 3.
A language learning apparatus, further comprising labor information output means for outputting labor information to labor the learner at the timing when repetition of a plurality of the recording scenes by the recording control means is finished. The language learning apparatus according to any one of claims 1 to 4.
Recognition means for recognizing the content of the recorded learner's voice;
Coincidence degree information output means for comparing the recognition result by the recognition means with the contents of each of the recorded image and the recorded speech and outputting the degree of coincidence information indicating the degree of coincidence between the two;
The language learning apparatus further comprising: The language learning apparatus according to any one of claims 1 to 5,
Between the viewing step and the recording step, an iterative step executing means for performing an iterative step consisting of a plurality of repetitive scenes,
Repetition output means for outputting a repetition image and a repetition sound which are the image and the sound which constitute one of the repetition scenes;
A pronunciation promoting means for outputting pronunciation promoting information for prompting the pronunciation of words constituting the sentence corresponding to each of the output repeated images and repeated voices;
The repetitive output means and the speech promotion so that the repetitive image, the output of the repetitive speech, and the output of the pronunciation promotion information are repeated for each of a plurality of the repetitive scenes respectively constituted by the different sentences of the same grammar. Iterative control means for controlling the means;
A language learning apparatus, further comprising iterative step executing means comprising: In the language learning device according to claim 6,
It further comprises iterative commentary information output means for outputting grammatical commentary information which is the commentary of the grammar corresponding to the repetitive step at the timing when repetition of a plurality of repetitive scenes by the repetitive control means is finished. Language learning device. In the language learning device according to claim 6 or 7,
A language learning apparatus, wherein the number of repetitions of the viewing scene, the number of repetitions of the recording scene, and the number of repetitions of the repetition scene are the same. In the language learning device according to claim 8,
A language learning apparatus characterized in that each of the number of repetitions is four. The language learning apparatus according to any one of claims 1 to 9.
A translation step executing means for executing a translation step consisting of a plurality of translation scenes after the recording step;
Japanese image display means for displaying a Japanese image including one of the translation scenes and including Japanese sentences spelled in Japanese corresponding to the image and the voice;
A translation and recording means for collecting and recording the voice of the learner in the language corresponding to the Japanese sentence contained in the displayed Japanese image as a translation voice;
First translation output means for outputting the recorded translated speech;
A second translation output unit that outputs an image including a sentence of the language corresponding to the Japanese sentence and a voice generated by uttering the sentence of the language;
The display of the Japanese image, the collection and recording of the translated voice, the output of the translated voice, and the output of the image and the voice by the second translation output unit are performed by the Japanese sentences different from each other in the same grammar. Translation control means for controlling the Japanese image display means, the translation recording means, the first translation output means, and the second translation output means so as to repeat for each of a plurality of translation scenes respectively configured;
A language learning apparatus, further comprising: a translation step execution unit comprising: In the language learning device according to claim 10,
The system further comprises translation operation means operated at the start of sound collection and recording by the translation recording means,
A language learning apparatus according to claim 1, wherein said translation recording means picks up and records said translated speech when an operation using said translation operation means is performed. In the language learning device according to claim 10 or 11,
A language learning apparatus, wherein a position of a reading point in the Japanese sentence is a position corresponding to a word break in a sentence of the language corresponding to the Japanese sentence. The language learning apparatus according to any one of claims 10 to 12,
Translation voice output operation means operated at the time of outputting a voice corresponding to the Japanese sentence contained in the displayed Japanese image and uttering a translated sentence which is a sentence spelled in the language;
Translation voice output means for outputting a voice uttering the translated sentence when an operation using the translation voice output operation means is performed;
The language learning apparatus further comprising: In the language learning device according to any one of claims 10 to 13,
Translation recognition means for recognizing the contents of the recorded translated speech;
Translation match degree information output means for comparing the recognition result by the translation recognition means with the content of the sentence of the language corresponding to the Japanese sentence, and outputting match degree information indicating the degree of match between the two;
The language learning apparatus further comprising: The language learning apparatus according to any one of claims 10 to 14, which cites claims 1 and 6.
The number of the Japanese sentences corresponding to the one translation scene is the number of the images corresponding to the one viewing scene and the number of the sentences corresponding to the voice, and the images and the voices constituting the one recording scene A language learning apparatus characterized by having a number smaller than any of the number of the sentences corresponding to and the number of the sentences corresponding to the image and the voice constituting the one repetitive scene. In the language learning device according to claim 10, wherein claim 1 and claim 6 are cited,
At least one of the viewing output means, the recording output means, the repetitive output means, and the second translation output means is a voice of a word constituting the sentence corresponding to the image and voice being output. A language characterized by outputting an image and a sound while making the display mode in the image at the output timing different from the display mode in the image of the other word output as the voice after the output timing. Learning device. In the language learning device according to claim 16,
A language learning apparatus characterized in that the display mode is a display color in the image of the word. In the language learning device according to claim 10, wherein claim 1 and claim 6 are cited,
A language learning apparatus characterized in that the viewing scene, the repetitive scene, the recording scene, and at least characters constituting the scene of the translation scene corresponding to each other are the same. In the language learning device according to claim 10, wherein claim 1 and claim 6 are cited,
When each of the viewing step, the repeating step, the recording step, and the translating step is completed, an end information recording unit is further provided which records end information indicating the end, in an outputable manner in association with the end learner. A language learning device characterized by A viewing step execution means for executing a viewing step of outputting an image including a sentence spelled in a language to be learned and a voice uttering the sentence as a plurality of viewing scenes, the view output means, A viewing step execution means comprising control means; and a recording step execution means for performing a recording step consisting of a plurality of recording scenes after the viewing step, the recording output means, the recording means, the learner voice output A language learning method executed in a language learning apparatus, comprising: recording means; and recording step execution means comprising recording control means,
The viewing step is
A viewing output step of outputting, by the viewing output means, the viewing image and the viewing audio that are the image and the audio that make up the one viewing scene;
A viewing control step of controlling the viewing output means by the viewing control means so that the output of the viewing image and the viewing sound is repeated for each of a plurality of viewing scenes respectively composed of different sentences of the same grammar When,
Including
The recording step is
A recording and outputting step of outputting, by the recording output means, the recording image and the recording sound which are the image and the sound constituting the one recording scene;
A recording step of collecting and recording by the recording means the voice of a learner who has pronounced the recorded image and the sentence corresponding to the recorded voice;
A learner voice output step of outputting the recorded voice of the learner by the learner voice output unit;
The recording image and the output of the recording voice, the collection and recording of the voice of the learner, and the output of the voice of the learner are each of a plurality of the recording scenes respectively configured by the sentences different from each other in the same grammar. A recording control step of controlling the recording output means, the recording means, and the learner voice output means by the recording control means, as described above;
A language learning method characterized by including: A computer included in the language learning device,
Viewing step executing means for executing a viewing step of outputting an image including a sentence spelled in a language to be learned and a sound generated by uttering the sentence as a plurality of viewing scenes;
Recording step execution means for executing a recording step consisting of a plurality of recording scenes after the viewing step;
A language learning program that works as a
The computer functioning as the viewing step executing means;
Viewing output means for outputting a viewing image and a viewing sound which are the image and the audio constituting one of the viewing scenes;
Viewing control means for controlling the viewing output means to repeat the output of the viewing image and the viewing audio for each of a plurality of the viewing scenes respectively composed of different sentences of the same grammar;
To act as
The computer functioning as the recording step execution means;
Recording output means for outputting a recorded image and a recorded voice which are the image and the sound constituting the one recorded scene;
Recording means for collecting and recording the voice of a learner who pronounces the recorded image and the sentence corresponding to the recorded voice;
Learner voice output means for outputting the voice of the recorded learner, and
The recording image and the output of the recording voice, the collection and recording of the voice of the learner, and the output of the voice of the learner are each of a plurality of the recording scenes respectively configured by the sentences different from each other in the same grammar. Recording control means for controlling the recording output means, the recording means, and the learner voice output means, as described above;
A language learning program characterized by functioning as