KR100703047B1 - Dynamic learning system and method of foreign language speaking led by the user acting oneself in the screen of true-life scenes - Google Patents

Abstract

The present invention relates to an active foreign language speaking learning system and method, in which a learner plays and leads in a scene of a real scene. The user can freely apply the language according to his or her level according to the disclosed invention to complete foreign language speaking. In the method of the present invention, after registering a user through the registration processing module, recording the directive through the instruction recording module, the user selects the set interactively directly through the set selection processing module, and the word family and level processing module. Determining the level through the dialogue processing module, the user directly enters and acts on the given scene with the camera through the dialogue processing module, and records and compares the dialogue between the native speaker's model dialogue and the user's dialogue through the recording processing module. Steps, analyzing, synthesizing, and synchronizing the voice / image through the voice processing / image processing module, testing one's own skills through the test processing module, and promptly answering the native language as a foreign language and vice versa. Processing steps, and converting digital content (movies) into textbooks for purposes of the present invention. And a step of statistical analysis through statistical processing module, and further comprising: separate exercise and test the pronunciation.

Foreign language learning, word family, level, dialogue, topic, knowledge base, voice processing, image processing

Description

DYNAMIC LEARNING SYSTEM AND METHOD OF FOREIGN LANGUAGE SPEAKING LED BY THE USER ACTING ONESELF IN THE SCREEN OF TRUE-LIFE SCENES}

1 is a hardware block diagram of a first embodiment according to the present invention;

2 is a hardware block diagram of a second embodiment according to the present invention;

3 is a software functional block diagram of a system in accordance with the present invention;

4 is a flowchart illustrating an entire procedure of a foreign language learning method according to the present invention;

5 is a detailed flowchart of the registration and directive recording step shown in FIG. 4;

FIG. 6 is a flowchart showing a detailed procedure of set 1 learning shown in FIG. 4;

FIG. 7 is a flowchart showing a detailed procedure of set 2 learning shown in FIG. 4;

8 is a flowchart showing a detailed procedure of learning set3 shown in FIG. 4;

FIG. 9 is a detailed flowchart showing the recording / comparison processing procedure shown in FIG. 4;

10 is a detailed flowchart illustrating the audio / video processing procedure shown in FIG. 4;

11 is a detailed flowchart showing the statistical processing procedure shown in FIG. 4;

12 is a flowchart showing an immediate response process shown in FIG. 4;

13 is a flowchart illustrating a general content conversion processing procedure shown in FIG. 4;

14 is a schematic diagram showing the contents of a set 1 learning example according to the present invention;

15 is a schematic diagram showing the contents of a set 2 learning example according to the present invention;

16 is a schematic diagram showing the contents of a set 3 learning example according to the present invention;

17 is an example of a user registration screen according to the present invention;

18 is an example of a word family map according to the present invention;

19 is an example of a word family dictionary according to the present invention;

20 is an example of a set 1 learning screen according to the present invention;

21 is an example of a set 2 learning screen according to the present invention;

22 is an example of a set 2 learning screen according to the present invention;

23 is an example of an audio / video selection screen according to the present invention;

24 is an example of a smoke learning screen according to the present invention;

25 is an example of a voice comparison graph screen according to the present invention;

26 is an example of a set 3 learning screen according to the present invention;

27 is an example of Set 3 learning screen according to the present invention.

* Explanation of symbols for main parts of the drawings

102: key input unit 104: memory

106: Database 108: Micom

110: video processor 112: LCD

114: digital camera 116: microphone

118: speech recognizer 120: speech synthesizer

122: speaker 210: PC

220: internet 230: web server

The present invention relates to a computer-aided language-learning system, and more particularly, the user actively selects the content to be learned according to his or her level and selection and acts in a real scene. The present invention relates to a method and a system for learning a foreign language.

Recently, with the advent of global society, there is a growing interest in foreign language learning. Currently, foreign language education methods are widely used.

First, through the electronic voice device or the voice of the teacher, the learner first listens carefully to the content arranged for a particular subject or schedule skill, and then writes orally repeats or hears the general idea of the subject matter. The learner's answer is checked to match the answer provided by the subject or the teacher's answer. These methods only follow a given process regardless of their choice or wishes.

Second, with the development of computer technology, some manufacturers have developed computer-aided learning materials with listening and speaking contents. This textbook is often computerized from simply printed textbooks. With the help of voice output devices, learners try to hear and understand the content arranged for a specific topic or schedule. After that, they answer the questions provided by the learning process. But in the case of speaking, it does not provide an immediate indication of whether the answer is negatively correct and what needs to be improved.

Third, classify students into groups of at least two students. Group members begin their conversation practice on a specific theme and schedule pattern. The problem with this method is that the members who participate in the conversation are learners. They may have accents that are difficult for other members to understand. It is also inappropriate for one member to determine the mastery of another.

Fourth, real-time conversation through video conferencing system is used for language education. This method is very limited in time and space.

Summarizing the problems of the conventional foreign language learning method described above, the learner is often limited to a predetermined subject, comprehension or practice method, or the learner is passively limited in listening because there is no interaction and active self-expression. Therefore, learners cannot express their thoughts and feelings and participate in learning activities, and thus lose good learning opportunities. Other methods also include two-way conversations, but these two-way conversations are problematic only for learners with limited language skills, since the model native teachers are not available to anyone when needed.

On the other hand, in everyday conversation, even young children choose the subject or object of the conversation and develop the conversation around it. As a result, there is an active foreign language learning system that provides an environment for exposing foreign languages and expects immediate response to any material at any time, as if native speakers, mothers, fathers, brothers and friends, teachers, and guides are always on hand. Required.

The present invention to solve the above problems, in order to focus on improving the foreign language of the user to learn the foreign language to develop the language ability while going directly into the scene to perform at the appropriate level at various levels. The aim is to provide a system and a method for learning a foreign language.

Another object of the present invention is suitable for both self-learning and classroom classes, and can be applied to a network, a mobile communication system, or a special device for this special purpose, so that a person who wants to learn a foreign language concentrates on speaking. Listening is to provide an active foreign language learning system and method that the user takes place in the actual scene acting automatically.

According to one aspect of the present invention for achieving the above objects, the active foreign language speaking learning system that the user acts and leads in the real scene,

Directive database, question type / answer type database, pronunciation database, word family map and word family dictionary database, level set database, dialogue kit database, topic kit database, sentence-paragraph-article-movie knowledge base, level test database, personal record Database,

Registration processing module, pronunciation practice / test module, instruction recording module, user confirmation module, set selection processing module, word family / level processing module, dialogue processing module, voice processing (analysis-synthesis-synchronization) module, image processing (analysis- Synthesis-synchronization) module, recording processing module, comparison processing module, test processing module, statistical processing module, general content conversion module, and immediate response module.

In accordance with another aspect of the present invention, a dynamic foreign language speaking learning method in which a user acts in a real scene and leads a registrant verification using a user identification module when performing foreign language speaking learning on a network using the present system is performed. step; And performing the learning using a web browser utilizing all provided databases and modules.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention may be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a hardware configuration of a first embodiment according to the present invention, wherein the first embodiment is an example in which a language learning system according to the present invention is configured as a dedicated device.

Referring to FIG. 1, the system of the first embodiment includes a key input unit 102, a memory 104, a database 106, a microcomputer 108, a video processor 110, an LCD 112, and a digital camera 114. ), A microphone 116, a voice recognizer 118, a voice synthesizer 120, and a speaker 122.

The key input unit 102 transmits a user's operation to the microcomputer 108, the memory 106 stores software or data processed by the microcomputer 108, and the database 106 stores various data necessary for learning. Doing. The microcomputer 108 provides a foreign language learning service to a learner by performing a program according to the present invention. The video processor 110 processes the image under the control of the microcomputer 108 and displays the image on the LCD 112, and the voice recognizer 118 recognizes the voice transmitted from the microphone 114 and transmits the image to the microcomputer 108. The voice synthesizer 120 reproduces the native speaker's voice of the database 106 under the control of the microcomputer 108 and outputs it through the speaker 122. The digital camera 114 captures the image of the learner and transmits it to the microcomputer 108, and the microcomputer 108 combines the learner image with the image of the corresponding learning scene and displays the image on the LCD 112.

2 is a block diagram showing the hardware configuration of the second embodiment according to the present invention. The second embodiment is an example implemented using a widely-used PC or a web server.

Referring to FIG. 2, the present invention is configured in the form of software distributed on a CD 202 or the like, and can be executed independently in a personal computer 210 equipped with a digital camera (web camera), a microphone, a speaker, and the like. 220 may be connected to the web server 230 to the web server 230 to be serviced on the web.

3 is a software functional block diagram of a system according to the present invention, which is a structure of an active foreign language speaking learning system in which a user acts and leads in a real scene.

Referring to FIG. 3, the system 300 of the present invention includes a word family map 312, a level set 314, a language database (DB: 330), a pronunciation database (DB: 316), and a directive database (DB: 318). ), Question type and answer type base (DB: 320), personal record database (DB: 322), voice processing module 342, image processing module 344, registration processing module 351, directive recording module 352 , Pronunciation practice / test module (353), user confirmation module (354), set selection processing module (355), word family / level processing module (356), speaking practice postponement module (357), recording processing module (358), Comprising a comparison processing module 359, test processing module 360, statistical processing module 361 to interface with the learner through the UOI (302) to train all foreign language speaking in an active manner in which the user acts and leads in the actual scene To deal with.

The user operates the user operating interface (UOI) 302 to enter the language learning system 300 according to the present invention. The UOI 302 performs input / output using a basic input / output device, and the basic input / output device includes a keyboard, a mouse, a digital touch-control panel, a speech playing system, and a digital camera. ), A remote controller, and a cellular phone.

In FIG. 3, the direction database 318 is composed of a native speaker model and a user's voice (image) to send and receive various directions, and includes a question pattern 320-1 and an answer pattern 320. -2) The database 320 stores question types and answer types that can be generated by one or more interrogators according to the parts of words, and the pronunciation database 316 stores the standard pronunciation of the model.

The word family dictionary database 331 combines the word family map 312 according to the classification of the object and the words that are inevitably related to the word, including the actual pronunciation, pictures, and meaning of each word. The level set database 314 stores the data for each set arranged according to the words and grammar levels designated based on the school level (12 grade) or language level. In the exemplary embodiment of the present invention, 'SET' is a classification concept according to a foreign language learning process, 'Set 1' is a beginner course, a word-centered learning course, and 'Set 2' is an intermediate course. 'Set 3' is an advanced course that is developed around multilateral dialogue. This learning process is just one example and can be configured in multiple stages as needed.

The metabolic kit database 332 stores learning data developed by a single dialogue guide, and the topic kit database 333 stores learning data of a structure in which several people develop a topic. The knowledge base 334 provides all types of sentences, paragraphs, articles, movies, and all kinds of information including explanation paragraphs and articles, and the level test database 335 speaks according to the level. Provides testing and comprehensive test questions, and a personal record database 322 keeps records of the user's own speaking performance.

The registration processing module 351 registers the user with voice and / or image to identify the registrant. The pronunciation practice / test module 353 confirms the accuracy of the pronunciation, and the instruction recording module 352 stores a communication for the user to instruct the system and receive guidance or instruction, praise, correction, and the like.

The user confirmation module 354 confirms the user by comparing the stored voice information or the image information when the user uses the system 300 according to the present invention with the voice information or the image information input by the user, and the set selection processing module 355 Allows you to select sets based on your level and interests.

The word family / level processing module 356 processes the words and grammar levels according to the word family map 312, and the dialogue processing module 357 processes various types of conversations according to the levels and displays the user's image. It is a module that displays in the actual acting.

The voice processing module 342 is an auxiliary module for dialogue processing, and processes an analysis, synthesis, synchronization, etc. of voice, and the image processing module 344 is an auxiliary module for dialogue processing, analysis, synthesis, synchronization, etc. of an image. To provide.

The recording processing module 358 records (stores) the contents actually learned by the user in correspondence with the stored native speaker expression model, and the comparison processing module 359 compares and evaluates the native speaker expression model and the corresponding record of the user.

The test processing module 360 may test the user's speaking and comprehensive language skills, and the statistics processing module 361 may provide various statistics to evaluate the user's achievement.

The general content conversion module 370 provides a process of converting and using digital content 371 such as all general movies into textbooks suitable for the purpose of the present invention.

The immediate answer processing module 380 is a module that instantly translates the native language into a foreign language or a foreign language in any expression.

On the other hand, when the present invention is implemented on a network, a voice processing module which processes a user's voice data through a microphone (not shown), converts and reproduces the transmitted voice data, and processes image data input from a web camera (not shown) And it is preferable to include each of the image transmission and reception module for restoring and display the transmitted image data.

Each of the above modules may be a program or a device storing a program that independently performs each function, and each module may be a single program or a device storing a program in which the functions are integrated and operated. It is not limited only to the Example.

In addition, each of the databases (DB) may be an information storage medium storing each independent information, each database may be an information storage medium for storing each information as a single information by dividing each information into a plurality of tables (table) have.

In addition, the active foreign language learning system 300 that the user acts and leads in the actual scene may be executed on a computer executable hardware platform as shown in FIGS. 1 and 2. In this case, the hardware platform executable by the computer may be a personal computer, a notebook or a personal digital assistant (PDA), or a cellular phone. Those skilled in the art can make various equivalent modifications for implementing the described invention in other electronic devices or devices that can be connected to a network.

Next, the process of learning a foreign language with the system of the present invention configured as described above will be described in detail with reference to the flowcharts of FIGS. 4 to 13.

4 is a flowchart illustrating the entire procedure of a foreign language learning method according to the present invention.

Referring to Figure 4, the foreign language learning method according to the present invention registration processing and instruction recording step (S1), user confirmation, set selection, level determination, and conversation practice step (S2), test step (S4), statistical analysis step (S5), pronunciation practice / testing step (S6), audio and video processing step (S7), general content conversion step (S8), and immediate response processing step (S9).

In the registration process and the instruction recording step (S1), as shown in FIG. 5, the user is registered through the registration processing module 351, and the instruction is recorded through the directive recording module 352 (S1).

After the basic input (setting) as described above, the actual learning can be started using the foreign language learning system according to the present invention.

In the user confirmation, set selection, level determination, and dialogue step (S2), the user is confirmed through the user confirmation module 354, the set is selected through the set selection processing module 355, and the word family / level processing module 356 is performed. Choose a place to go from the word family map 312 and determine the level (S2). In addition, the conversation processing module 357 executes the conversation practice in the selected set. For example, when the user selects 'set 1', as shown in FIG. 6, the user learns a word based on the word family dictionary 331, and when 'set 2' is selected, FIG. As shown in FIG. 8, a learning process of two-way conversation is performed, and when set 3 is selected, a multi-party conversation learning process is performed as shown in FIG. 8.

Subsequently, in the recording process and comparison step (S3), as shown in FIG. 9, the recording processing module 358 records the individual voice (image) corresponding to the voice (image) content in the DB during each conversation process, and compares them. The processing module 359 compares the two voices.

Subsequently, in the test step S4, the speaking and synthesis tests are executed through the test processing module 360 as shown in the rear end of FIG. 9, and in the statistical analysis step S5, the statistical processing module is shown in FIG. 11. Statistical analysis for each user through (S5).

On the other hand, in the pronunciation practice / test step (S6) to practice and test the pronunciation through the pronunciation practice / test module 353, in the voice and video processing step (S7), as shown in Figure 10 voice processing module 342 The voice is analyzed, synthesized, and synchronized to display a graph, and the image processing module 344 analyzes, synthesizes, and synchronizes an image (or a video) to insert a user's image on the screen.

FIG. 5 is a detailed flowchart of the registration and directive recording step S1 shown in FIG.

Referring to FIG. 5, the registration processing module 351 registers a user (501). An example of the registration screen is as shown in FIG. 17. Items required for registration include name (character), name (with voice or image), gender (character), age (character), start date (character), and the like. The name registered together with the audio or image becomes an identification name later.

Then, if necessary, the user can go to the pronunciation practice / test module 353 to practice and test the pronunciation (502).

Next, all instructions required for an instruction / question / induction / praise / correction, etc. are input and stored as voice and / or video through the instruction recording module 352 (503 to 506). In this case, the general directive and the directive / question of the set 1, the directive / question of the set 2, the directive / question of the set 3 can be recorded and / or recorded. In the embodiment, the first display of Korean (character) and foreign language (character), and then the native language is displayed by voice (image), the learner is placed on the microphone (camera), as it is stored as your own voice (image) . The necessary basic directives, set directives, questions, and the like are stored in the directive database 318. In this step, the types of questions and answers according to the questions such as "what is it", "what are you doing", "how are you doing", etc. are also inputted, and the necessary parts are also inputted and stored in the question type and answer type database 320.

If the present invention is to be used for other purposes, such as reading and writing training, other input methods may be used, and the present invention is not limited to these embodiments.

6 is a flowchart illustrating a detailed procedure of a set 1 learning process according to the present invention.

Referring to FIG. 6, in the exemplary embodiment of the present invention, 'set 1' is based on the level of speaking only the basic language of the mother tongue, and learns a language through extremely rudimentary conversations such as "what is this" and "what are you doing" in your language. This is a learning process for users who are expanding their vocabulary. The overall configuration of this set 1 is configured as shown in FIG.

First, when the foreign language speaking learning system according to the present invention is started, the user confirmation module 354 checks whether a registrant is present (601). At this point, the registrant simply speaks his registered name to the microphone (camera). If the name is not registered, it can be restarted in the previous step. If the registrant is confirmed, the non-registrant goes to the initial registration step, and if the registrant, the set is selected in the set selection step (602).

If 'set 1' is selected in the set selection step (602, 603), the word family map 312 is guided to the picture (604). Here, the word family map 312 corresponds to the classification of things as shown in FIG. 18. In the embodiment, the word family map 312 is largely classified into nature, moving creatures, and the human world, and nature is again the sky (SKY) and the earth. (LAND), sea (SEA), plant (PLANT), etc., MOVING CREATURE (man), animal (ANIMAL), bird (BIRD), insect (INSECT), fish (FISH), etc. Classify. And the HUMAN WORLD reclassifies and maps people / family, food, clothes / accessories, home / building, tools, telephone / communication, and so on. In this case, the level set 314 checks the level to which the word currently selected by the user belongs, and confirms whether the necessary vocabulary is completed before going to the next level (605).

After this step, a picture indicating the word in the word family dictionary 331 having the configuration as shown in FIG. 19 is displayed in the interface window (606). In an embodiment, the word family dictionary 331 includes a foreign language spelling (letter), a phonetic symbol (letter), a pronunciation (native voice / picture), a picture, a part-of-speech, one or more questions related to the word, a native language (letter), Native language (speech), word family 1, word family 2, word family 3, one or more relations (as many as necessary), one or more functions, one or more subjects corresponding to the function, idiom form (required), synonyms, antonyms, etc. It includes.

When a picture and a questionnaire are displayed from this word family dictionary 331 as shown in FIG. 20, the user asks a question that begins with a questionnaire corresponding to "what is this", "what are you doing", "what are you doing?" 607). The answer is the native speaker's voice (picture), and the user copies it (608-610). At this time, the voice graph and the lip-synch are displayed by synchronizing the expression (pronouncement) with the expression of the native speaker.

In an embodiment, it is induced to repeat at least three times, and when a foreign word is pronounced, the picture, foreign language, and the native language are linked and remembered by immediately speaking the same word in the mother tongue. The last representation of the iteration is then stored in the system (611).

Subsequently, the meaning may be further enlarged according to the level to practice using the related word (612). If you choose a relative word, you go to the word's heading, bring up a picture, practice it, and endlessly link it to the word chain. In this way, we can learn the word family of the word and learn the words that are used together in everyday conversations at once or by connecting them to learn to connect with natural reality. In an embodiment, this is equivalent to connecting to "apple-eat-sweet / sweet, apple juice, apple tree" and the like.

If the present invention is to be used for other purposes, such as reading and writing training, other input methods may be used, and the present invention is not limited to these embodiments.

7 is a flowchart illustrating a detailed procedure of a set2 learning process according to the present invention.

Referring to FIG. 7, when the user selects 'set 2' in the set selection step S2 of the foreign language speaking learning system according to the present invention, a screen for selecting a conversation guide appears as shown in FIG. 21. This screen is selected to select the conversation guide (701). The principle applied here is that once children are able to learn basic words and use simple sentences, they can have a simple conversation with the person they meet. You will usually have the ability to have a one-on-one conversation with one person. At this stage, you will need a conversation guide to lead the conversation, which will provide you with a variety of information in each area.

Referring to FIG. 21, in an embodiment of the present invention, a conversation guide is divided into eight persons, namely, a mother, a dad, an older brother, an older sister, a friend, a teacher, a guest (visitor), a guide (travel), and the like. Have them lead a dialogue in each of the areas they most encounter. In other words, the conversation guide (Mom), father (Dad), brother (Sister), friend (Friend), teacher (Teacher), guest (Guest), tour guide (etc.) Divided into two categories, Mother selected Man / Family, Food / Kitchen / Cooking, Furniture / Home appliances, Clothes / Accessories / Accessory, and Dad is the House / Building, Book / Library / Education, Weather, Work, Sky, and Brother is Healthy, sports / game, Animal, Tool, Telephone / communication, Television, and Sister is Music / painting / poem, Bird, and Friend are Movie, Leisure, Hobby, Insect, and Teacher ) Is Religion, Social, Car / Train / Airplane / Boat n / Plane / Ship, Machine / Science / Factory, Fish, Plant, Language / Grammar, History, and Guest ) Is for politics, business / economy, culture, and tour guide is for department store / market, station, airport, Hospital, Farm, Zoo / Park / Plantation, Museum, Land / mountain, Sea / lake, Country , International Institute, etc.

Subsequently, a desired conversational material is selected on the conversational material screen as shown in FIG. 22 (702). At this time, his level is checked (715).

Referring to FIG. 22, a mother tells about a person / family, food / cooking / kitchen, clothes / accessories, furniture / appliances, and the like. On the other hand, dad talks about home / building, weather, books / library / education, work / occupation, and sky. In other words, it is the same principle that we learn from different people from different ages. Here, in the case of a friend, a teacher, or a guide, a desired gender (male / female) can be selected. In the embodiment, when "mother"-"food" is selected, a screen for selecting audio / video appears as shown in FIG.

Subsequently, it is possible to select whether to use audio (Tell me) or video (Show me) in the selection screen shown in FIG. 23 (Show me). If audio is selected, audio is separated and processed (704). In addition, it is possible to select whether to set the normal speed or the slow speed (704-1).

Subsequently, if the selected dialogue is called up from the metabolic kit database 332 and displayed, the user listens to the screen and learns the expression (705, 706).

When the conversation practice begins (Let's talk), the learner talks to the conversation guide (mom) (707). In this process, voice processing (analysis-synthesis-synchronization) and image processing (analysis-synthesis-synchronization) (S7) are performed according to the procedure shown in FIG. 10 to compare the voice graph and the lip-synch on the screen. It may be displayed, and through the digital camera as in the screen shown in FIG. 24, the user directly enters and plays his role in the screen, and it appears synthesized and synchronized on the screen.

In operation S3, as shown in FIG. 9, the user's voice is recorded, and if the user selects the comparison with the user's (COMPARE) (708), the native speaker's voice is compared with the user's voice as shown in FIG. 25. Graph them together and score them according to similarity.

It may then retake to take turns to digest the necessary metabolism (709).

In addition, the meta kit 332 has a kit heading and an identification name, so that the dialogue material can be expanded in association with the word family dictionary 331 and the level set 314 (710).

It also allows the practice of similar (alternative) expressions (710-1).

In conversation, on the other hand, in order to save reality, use induction expression so that one can remember and speak the dialogue if possible (711 ~ 714). In other words, if the user can't respond to the dialogue immediately, wait about three seconds, use the expression "Come on." It starts as it is, and at any time during the process, the user stops the native speaker's dialogue to complete the dialogue. And you can add expressions that naturally compliment ("Good", "Well done.") To make your reality alive.

In this way, the user talks dynamically like a native speaker (mother, father, etc.) speaks directly in real life, and naturally completes speaking and listens automatically.

When the present invention is to be used for other purposes such as reading and writing training, other input methods and expression methods may be selected, and the present invention is not limited to the above embodiments.

8 is a flowchart illustrating a detailed procedure of a set3 learning process according to the present invention.

Referring to FIG. 8, when the user selects 'set 3' in the selection step S2 of the foreign language speaking learning system according to the present invention, the set 3 start screen is displayed, where a desired topic is selected (801). The user's level record is then referenced (812).

Next, audio (tell me) / video (show me) may be selected on the selection screen as illustrated in FIG. 21 (802). If audio is selected, audio is separated and processed (803). In addition, it is possible to select whether to set the normal speed or the slow speed (803-1).

In Set 3, a number of characters are composed of multiple dialogues. Accordingly, the topic kit 333 brings up a screen as shown in FIG. 26 with a number of equivalent figures (804).

Next, in the expression learning stage, first, a conversation between a plurality of characters is studied (805), and in the acting in the scene stage, a desired role is selected from the plurality of characters as illustrated in FIG. 26 (I will play xxx.) and talk freely in the actual scene (806). At this time, through the voice and image processing step (S7), the voice and the image are analyzed, synthesized and synchronized so that their role is displayed on the screen shown in FIG. Therefore, the user can participate in the real screen and learn the conversation as if it were real.

In this process, as shown in FIG. 9, the user's voice is recorded by recording the user's voice, and if the user selects the comparison with the user's (COMPARE) (807), the native speaker's voice and the user's voice can be compared and scored. To be.

This can then be retaken by taking turns to digest the necessary metabolism (808).

When all of this is done and finally has the ability to talk freely, move on to free discussion and discuss freely on all topics (809). For example, in an embodiment, when a user speaks a topic (I'd like to talk about xxxx. Or Let's talk about xxxx.), The word family dictionary 331, the dialogue kit 332, the topic kit 333, the knowledge base ( All databases, such as sentences, paragraphs, articles, and films) (334), are linked together, and the dialogue continues in chain-talking form with headings (810,811). That is, all sentences, kits, and scenes with the main word are searched for, enabling conversation at all levels.

And when you change a topic, you can change the subject at any time by using the expression "By the way". At this time, the user's level information is stored so that information corresponding to the level can be output.

When the present invention is to be used for other purposes, such as reading and writing training, other input methods may be selected, and the present invention is not limited to the above embodiments.

FIG. 9 is a detailed flowchart showing the recording / comparison processing procedure S3 shown in FIG.

Referring to FIG. 9, in the recording / comparison process of the foreign language speaking learning system according to the present invention (S3), first, a record of content is temporarily stored in the system, and a user record is also temporarily stored (901, 902).

Next, in the content-user comparison (903), the two items are synchronized and compared. When comparing only the voice, a native speaker model voice is heard first, and then the user's own voice is followed. As shown in Figure 6, the graph is synchronized by symmetry up and down to score its own achievements, and if desired, the entire personal record is indexed and stored (903 to 907). If it is synchronized to an image, the native speaker model image is displayed first, and then the user's own one can be displayed and compared, and the contents of the storage are the same as the voice. And separate records for each heading, kit-specific statistics, the embodiment stores the identification code, heading (or omitted), date, self-scoring score and the like.

Subsequently, in the test step S4, a level test is designated and then a speaking test is performed and a comprehensive test is performed (908). In this case, the test may be performed by specifying the level of the test itself, or the level may be designated by the level based on the existing record of the person (911).

In order to simply test the speaking level, in some embodiments, several arrays may be moved up, down, left, and right by a series of questions arranged like an eye check board (909). In addition, the database 330 includes data for comprehensive tests (listening, speaking, reading, writing, comprehension, etc.) for each level at the same time to provide data for the comprehensive test (910).

FIG. 10 is a detailed flowchart illustrating the audio / video processing procedure S7 illustrated in FIG. 4.

Referring to FIG. 10, in the process of processing audio / video of a learning system according to the present invention (S7), first, the corresponding content of the content is analyzed by voice while the entire system is operated. The analysis is performed in a graph as shown in FIG. 23 (S101 to S103).

Correspondingly, the user analyzes the expression input through the microphone / camera, analyzes the height, intensity, time, syllable, etc. of the sound and expresses it in a graph, and displays it on the screen so that it is symmetrical with the native speaker's model S104 to S106).

In the video, as shown in FIG. 27, the person's image on the content can be analyzed and cut out for each person, and if the user sets a role, the video inputted by the camera is overlaid on the person's image in real time. By synthesizing, the user can perceive that he or she is acting directly in the scene and can realistically speak (S107 to S110). However, if the whole body treatment is too difficult to produce the same effect only by the face part. When the user changes the role, the screen is synthesized and synchronized according to the same procedure (S111).

When the present invention is to be used for other purposes, such as reading and writing training, other input methods may be selected, and the present invention is not limited to the above embodiments.

FIG. 11 is a detailed flowchart showing the statistical processing procedure S5 shown in FIG.

Referring to FIG. 11, the statistical processing module 361 of the foreign language speaking learning system that the user acts and leads in a real scene calculates daily set statistics for each user and calculates monthly set statistics (S201, S202). ). In addition, the average score scored by the user may be calculated for each area, and the trend may be analyzed by the period and expressed as a graph (S203, S204). You can also check the words, grammar level, etc. for each set you have learned.

12 is a diagram illustrating an immediate response process according to the present invention.

Referring to FIG. 12, in the process of acquiring a foreign language, various questions arise every moment. However, following a predetermined process does not solve the question according to the situation faced in daily life. As you can see, there is a need for a function that instantly converts a native language expression into a foreign language and a foreign language expression into a native language. In the embodiment, if the native language is changed to English, say "What do you say?" (S301). If the native language is changed to English, and then answer the question into the microphone (camera) (S302) , And take the English expression corresponding to this expression and answer it by voice, with an image (with letters), so that it can be used immediately (S303).

On the contrary, if English is your native language, you can use "What does it mean?" To perform other functions, and input English expressions (by camera) or keyboard to get the answer. (S304).

13 is a flowchart illustrating a detailed procedure of converting general digital content into a textbook.

Referring to FIG. 13, first, a general digital content (eg, a movie) is uploaded (S401), and a character name is analyzed by analyzing a character (S402). Analyzing and separating the characters appearing on each screen is prepared to connect the characters and lines (S403). Next, combine the scene number and the person's metabolic number to make a unique metabolic code (S404). In this way, the person and the dialogue for each scene are separated, so that when the user postpones the person to select later, it is possible to connect the scene and the dialogue (S405).

In order for the user to use this converted content (movie) as a textbook of the present invention, the name of each character must be input by the user's own voice, and the character name is actually referred to the microphone (camera). Will be postponed (S406). The completed content can be used as a teaching material according to the present invention at any time (S407).

14 is a schematic diagram showing the contents of a set 1 learning example according to the present invention.

Referring to FIG. 14, through an interface window provided by the present invention, a user looks at a picture in an interface window and asks a question that starts with a question by an input means applied.

In the present exemplary embodiment, the target foreign language is 'English', and an 'apple' appears in a picture of the window, and a user's question is input through a PC camera (with a built-in microphone) and is “What is this?”. Again, the interface window sounds "Apple" through the speaker, or shows lip-synch and the word "Apple" under the picture. The user copies the word "Apple" and again the interface window hears "Yes, it is an apple." (With the shape of the mouth) through the speaker. Repeat for the PC camera. At this time, every time the user calls "Apple", the speaker then speaks "Apple" in Korean so that the picture-foreign language voice-Korean voice (and the apple spelling, if desired) can be linked and remembered at the same time.

Going one step further, when the picture of the apple is displayed in the interface window, the user can say "What do we do with it?" Ask. Then the interface window will say "eat" through the speaker (with the shape of the mouth) along with the picture of eating an apple, and if the user says "eat" to the microphone / camera, the picture will continue to be displayed in the interface window. Yes, we eat it. " The user then speaks to the microphone / camera three times, "eat," followed by the "eat" Korean.

When the "COMPARE" character is displayed in the interface window, the user says "Compare" to the microphone / camera. Then the native speaker's voice (with the shape of the mouth) comes out first with the picture of the heading just practiced, followed by the speaker's corresponding voice (the mouth is just below the native speaker's). When "SELF-GRADING" is displayed on the interface window, the user enters his / her own grading score (in letters). This part can be omitted if desired.

When "RECORD SAVE-SAVE, NO SAVE" is displayed on the next interface window, the user says "Save" or "No save" to indicate whether to save. When the next interface window shows "GO BACK", "THAT'S ENOUGH.", "STOP", and "OTHER WORD", you can make your choice and speak to the microphone / camera, repeat, stop, or skip to the next word. The user continues to practice the vocabulary according to the word family and at his own choice. The foreign language learning model of the elementary word acquisition level according to the present invention can be used in any language of the world.

15 is a schematic diagram showing the contents of a set 2 learning example according to the present invention.

Referring to FIG. 15, set 2 is a one-to-one conversation practice. First, a conversation guide (helper) is displayed with pictures and letters in an interface window.

In the embodiment of the present invention, the helpers are all eight, including mother, father, brother, sister, friend, teacher, visitor / guest, Guides, etc. If you say "MOM" to the microphone / camera, the interface window now displays the pictures and text of MAN / FAMILY, FOOD / KITCHEN, FURNISH / APPLIANCE, and CLOTHES / ACCESSORY. When you speak "MAN" into the camera, "MAN / FAMILY" is selected, and again "TELL ME" and "SHOW ME" are displayed in the interface window, allowing you to decide whether you want to listen to the voice or view it on the screen. You may also choose to use only English or two languages spoken in English followed by "ENGLISH" or "BILINGUAL". In addition, the speed is selected to be "Normal" or "Slow". After selecting "SHOW ME", "ENGLISH" and "NORMAL", the dialogue window will be displayed from the kit and all the dialogues will be played with subtitles.

The following is a full-fledged acting and dialogue. If you say "LET'S TALK", you will automatically take on the role of son or daughter in the interface window. A user touches the camera and says "Dad, why do you wear eyeglasses?" The user appears at the position of the son on the screen. You can also synthesize only the face part as needed. Then it follows "Because I have poor eyesight." If the user says "Where did you buy them, Dad?" To the camera, it will also appear in the window. "I bought them at an eyeglasses store (shop)." Is displayed.

When the interface window displays "GO BACK", "THAT'S ENOUGH.", "STOP", and "NEXT", make your choice and speak to the microphone / camera, repeat, stop, or go to the next kit.

At the end of all exercises / exercises, a "COMPARE" window will appear, and if the user "Compare", the user's acting will be compared with the voice, following the model acting in the content. Again, scoring is done at "SELF-GRADING" and "SAVE" is determined.

You should also practice the expressions of the other person, so if you choose to switch roles by saying "TAKE TURNS", you will play the role of mother, father, older brother and older sister. In this process, if the user cannot continue the conversation and the time elapses for a certain time (eg 3 seconds), the opponent may say "Oh, come on." Give a sense of reality to encourage expression. At this point, the user will say "I'm coming." If the user fails to speak after a certain time (for example, 5 seconds), the user starts the dialogue of the model (with the screen). At any time during the induced metabolism, if the user starts metabolism, the model stops metabolism and displays the user's metabolism immediately synthesized.

Next, continue with the conversation practice of choosing "ONE MORE TIME", "MORE TELL ME", "TELL ME WHY", "MORE TELL ME WHY", etc. Practice.

16 is a schematic diagram showing the contents of a set 3 learning example according to the present invention.

Referring to FIG. 16, in set 3, all the cases that can occur in reality with a plurality of characters are practiced. In this section, the program is divided into two stages. The first is to practice according to the topic kit topic. In the embodiment, select "PARTY", and at first, to learn the contents, say "LET ME TAKE A LOOK AT IT." Speak into the glimpse of the kit to familiarize yourself with the expression.

The following is called "LET ME PLAY TOM.", Which plays the role of TOM. The part of the TOM in the scene is then cut out and the user can be inset to facilitate the conversation. In this process, all of the existing derived expressions are used. Repeat "ONE MORE TIME" if you want to repeat it, or "LET ME PLAY JUDY." Speak into the microphone / camera to change roles.

If the film is used here, the content of the film is digitized and connected to the system so that the main character in the film can play the role of an actor / actress through a play-in that directly enters the screen. As you go along, you can complete the actual speech.

The next step is to have a totally free discussion, "LET ME SELECT TOPIC", and "WEATHER" to retrieve all weather data and present them in various forms, one by one. The user responds freely accordingly, and the data outputs the content corresponding thereto to allow the conversation to proceed freely. This level corresponds to artificial intelligence and selects the closest content from knowledge base 334 at that time to respond to the conversation.

In this process, all the techniques and procedures of the above-mentioned processes are applied to complete the ability to speak a foreign language under the same conditions as creating an environment surrounded by native speakers in real life.

The embodiments of the present invention are merely one embodiment, and many modifications and variations of the components are possible without departing from the gist of the present invention, and the present invention is not limited to the above embodiments.

As described above, the dynamic foreign language speaking learning system and method which the user of the present invention acts and leads in the actual scene enables the user to perform foreign language speaking learning while performing in the scene.

In particular, the present invention, according to the level of the user, complete beginners to learn the words as a young child learns words one by one to the mother, living words based on the voice and pictures in the word family unit, at the next level ( Depending on the grade, students will practice speaking directly to reality by practicing one-on-one conversations directly on the screen, based on basic and simple sentences and expressions.

After learning various expressions at all levels while playing various roles in the live performance of multiple performers (even in the movie), they can talk or discuss any topic or form through chain-talking. It also makes it possible.

In addition, any expressions faced in everyday life can be used as a textbook by using the dictionary function and general contents (movies), which immediately answer the native language as a foreign language and the foreign language as a native language, enabling various practical teaching materials. As a result, foreign language speaking is completed as if it is a native language, and other abilities, such as listening, are automatically solved, and reading and writing also have a synergistic effect. The invention is also applicable to anyone who wants to learn a language.

Claims (26)

A Menu / Direction Database for storing all commands that the user commands to the computer or communication device and all correspondences with the native speaker model as text-to-video; A pronunciation database for mapping and storing all native speakers' standard pronunciations in lip-synch; A Word-family Dictionary Database, which stores everything of each word according to the word family map; An Idiom Collection Database, which is a collection of idioms where words are used as idioms; A level set database that organizes words and grammatical forms by level or grade; Dialogue kit database where one-to-one dialogue is performed according to the dialogue assistant; Topic kit database that collects the actual conversation scene of several people according to the topic; Question / Answer Pattern database, which collects various question types and corresponding answer types; A level test database that collects questions and their answers so that you can test the level at each level; A personal record database that stores all of its records as the user exercises; A knowledge base that aggregates all possible kinds of sentences, paragraphs, articles, movies, and the like by topic; A pronunciation practice / test module for practicing and testing pronunciation using the pronunciation database; A registration module for registering a user; Menu / Direction Recording Module to record menus and all directives; Set 1 which is a learning process of a word family and related words according to a user's selection using the word family dictionary database, a conversation guide using the dialogue kit database, and a conversation related to the topic related to the conversation guide Set 2, which is a process of learning metabolic-oriented learning, which consists of a process of practicing various substitution expressions, and a multilateral conversation-based learning process using the topic kit database and various conversations using a knowledge base. A speaking practice acting module, consisting of a set 3 for processing conversations according to each selected set; A test module configured to process tests of each level using the level test database; A statistical processing module for processing various statistics based on the personal record database; Instant Answering Module that directly answers the foreign language expression corresponding to the native language and the native language meaning of the foreign language expression; A conversion module of general digital contents that converts general content (movies) existing in digital form to be used as a textbook for practicing conversation; In case of executing on a network, a voice processing module for processing voice data of a user through a microphone and converting and reproducing the transmitted voice data; A video processing module for processing image data input from a web camera / mobile communication device camera and restoring and displaying the transmitted image data; And It includes a recording and compare module that records and compares all the running records of the user, Speaking and Listening Assistive The user's video is synthesized and synchronized directly in the scene that appears in the interface window according to the set (combination) selected by the user operating interface (UOI). An active foreign language speaking learning system, which is characterized by being able to learn, acting and leading in a scene of a real scene. The system of claim 1, wherein the learning system is For all lessons other than basic system tasks, save menus and directives recorded with your own voice in the form of directives (characters) + directives (model voices) + directives (your voices), so that you Actively give instructions by voice and listen to the native speaker's voice to induce, correct, and praise according to the corresponding result, so that all commands can be carried out by voice other than mouse, keyboard, and button through two-way communication. Active foreign language learning system, which is characterized by the user acting in the scene of the actual scene. The method of claim 2, wherein the learning system When you make a menu as a voice menu, "What is this?", "What do we do with it?", "How do we do that?", "I am now Since I use real life expressions like "Where am I now?", "What do you say?", "What does it mean?" An active foreign language learning system in which a user acts and leads in a scene of a real scene, characterized by learning. delete delete delete The method of claim 1, wherein the record comparison module It includes a recording module that records all performance records of the user, and a compare module that compares and processes the native fisherman's model. Foreign language speaking learning system. The word family map of claim 1, wherein  In the word family map corresponding to the classification of things, they are divided into nature, moving creatures, and human world, and nature is divided into sky, land, sea, plants, etc., and moving creatures are divided into people, animals, birds, insects, fish, etc. , Human World, People / Family, Food, Clothing / Accessories, Home / Building, Tools, Telephone / Communication, Furniture / Households, Health / Hospital, Books / Schools / Training, Transportation (Cars / Ships / Trains / Airplanes), Machinery / Science / Factory, Music / Art / Film / Theatre / Poetry, Sports / Culture / Game, Business / Shopping, Politics / Social / Military, Farm, International Organization, History, Language / Grammar, Cyber World / Hot News Active foreign language learning system, in which the user acts and acts on the screen of a real scene, wherein words other than nouns are classified according to a part of speech. The method of claim 8, wherein the word family dictionary Foreign language spelling (letter), phonetic symbols (letters), pronunciation (native voice / picture) of the heading word, pictures describing the heading, parts of the heading, one or more questions related to the heading, the native language corresponding to the heading, the mother tongue (Speech / image), word family 1 (nature / or part-of-speech), word family 2 (moving creature), word family 3 (human world), relational word, one or more words that indicate the function of the heading, corresponding to the functional word An active foreign language learning system in which a user acts and leads in a scene of a real scene, characterized by including one or more target words, idiom-type words, synonyms, and antonyms. delete delete The method of claim 1, wherein the level set database is Words and phrases are set for each level level of the language, and phrases and patterns are set for each level level, and all words and sentences are set. Active Foreign Language Speaking Learning System where the user acts and leads in the scene of the real scene. The method of claim 1, wherein the speaking level set database is Group a series of questions by word / idiom level and grammar level by word family map and measure them up into one field, if blocked at a certain level, measure directly below the other field, and then move back to another field, Active Foreign Language Speaking Learning System where the user acts and acts in the scene of a real scene, which includes a database organized to avoid the imbalance of measuring up and down and checking one part or one part. The method of claim 1, wherein the speaking practice postponement module In order to ensure the authenticity of the conversation between the user and the native speaker model, first wait for a certain time and then "come on.", "Come on." Leading conversation starts through leading words such as ("Go ahead."), And if it does not continue even after a certain time (e.g. 5 seconds), it will begin to listen to the native speaker's conversation. When the user starts a conversation, stop the native speaker's dialogue and let the user complete the dialogue so that the user can An active foreign language learning system, which includes a user acting and leading in a scene of a real scene. The method of claim 1, wherein the learning system Active foreign language learning system that the user acts on the screen of the real scene, characterized by being implemented as a dedicated learning device. The method of claim 1, wherein the learning system Active foreign language learning system, which acts on the screen of a real scene and is led by a user, which can be executed in a network or a mobile communication device. delete Registering a user and registering menus / directives and questions; Selecting a learning process comprising a set 1 using a word family dictionary database, a set 2 using a metabolic kit database, and a set 3 using a topic kit database after checking a user; In the learning process selection step, when set 1 is selected, the word family and related words are learned according to a user's selection using the word family dictionary database, and when set 2 is selected, a conversation guide using the metabolic kit database is selected. Study the dialogue-centered process consisting of the process of selecting the process, the process of conducting the conversation around the topic related to the conversation guide, and the process of practicing various substitutions, and when the set 3 is selected, the topic kit database is used. Performing a learning process for freely learning various conversations using a multilateral conversation-oriented learning process and a knowledge base; Testing by presenting a speaking and comprehensive test problem according to a learner's choice; Statistically processing the learning results and the test results for each user; And And converting a specific native language phrase into a foreign language and immediately converting the foreign language phrase into a native language; Sentences that contain the most words and related words by searching knowledge bases such as related words, sentences, passages, articles, and movies for free discussion and chain-talking. An active foreign language speaking learning method in which a user acts and leads in a scene of a real scene, by displaying a light on an interface window. The method of claim 18, wherein the learning process It analyzes, synthesizes and synchronizes the video and audio so that the user's video appears in the screen that performs the dialogue of various real scenes. Analyze the person's image on the content and cut out for each person, and if the user sets the role, the image input by the camera of the PC / mobile communication device is overlapped with the person's image and synthesized in real time. To match the input voice An active foreign language learning method in which a user acts and leads in a scene of a real scene, in which the user perceives and acts as if he is acting directly in the scene, and leads to real life. 20. The method of claim 19, wherein when analyzing, synthesizing, synchronizing the voice The native speaker's speech is analyzed and displayed as a graph by analyzing the tone, intensity, time, and syllables, and the corresponding user's dialogue is analyzed and expressed as a graph by analyzing the tone, intensity, time, and syllable. To show the image symmetrically up and down so that it can be compared statically and dynamically, When analyzing, synthesizing and synchronizing the images The lip-synch of the ambassador is displayed at the bottom right of the screen or at the most suitable place.When the user speaks the dialogue, the lip-synch of the native speaker is synchronized with the user's mouth. An active foreign language learning method, in which a user acts and leads in a scene of a real scene, by displaying the left and right symmetrically so that the user can compare the differences. The method of claim 19, wherein the learning method is When digitizing any real movie and playing it on a computer system or a mobile communication device, it converts by adding necessary functions to use such contents as a teaching material, so that the images are analyzed, synthesized, synchronized and cut out by person. When the user plays in the camera, the user overlaps the person who acts and synthesizes in real time so that the user selects the cast in any movie and performs the acting while performing the speaking. A proactive way of learning to speak a foreign language, acting and acting on the screen of a person. delete The method of claim 18, wherein the learning method is If you choose Bilingual instead of English only, you will be able to speak a foreign language in the form of a native speaker's dialogue, a user's dialogue, or a foreign-mother language. Active foreign language learning method that the user acts and leads in the scene of the actual scene characterized in that the pairs to remember at the same time. The method of claim 18, wherein the learning method is When you choose the dialogue practice, you can adjust the speed according to your level, so that the user can practice slowly by selecting the normal speed (Slow) and slow (Slow), characterized by the user A proactive way of learning to speak a foreign language, acting and acting on the screen of a person. delete The method of claim 18, wherein the learning method is Compare the expression of the native speaker's model with the user's expression, and add comments that correct pronunciation, strength, time, stress, etc., so that the user can be consciously the same as learning from the native speaker. Active language speaking learning method led by acting inside.

Images