KR100447667B1 - Interactive Language Teaching System consisted of Dolls and Voice Recognizing Computer - Google Patents

Abstract

The present invention provides a voice input means for receiving a user's voice and converting it into an electrical signal; and modulating the output electric signal of the voice input means and transmitting it wirelessly to a computer, and receiving an image, sound, or audio signal sent from the computer. A first wireless transmission and reception means for demodulating; and a video or audio output means for signal processing a video signal or an audio signal and outputting the same to a monitor screen or an audio system; And second wireless transmission / reception means for receiving and demodulating the radio signal transmitted from the doll, and modulating the output image, sound, or audio signal of the computer and transmitting the radio signal to the doll. First voice recognition means for recognizing and recognizing the meaning of the voice signal; and question data of various situations that can be developed in a real-life conversation; a plurality of answer data according to the question; Data storage means for storing video data or sound data suitable for the contents of the answer data, and selecting data stored in the data storage means in response to questions or responses of users, and outputting the selected video or audio data. Control means for controlling the output to the computer comprising a.

Description

Interactive Language Teaching System consisted of Dolls and Voice Recognizing Computer}

The present invention uses a computer having a speech recognition function and a learning doll having a speech recognition function that enables one-to-one or one-to-many users to conduct language learning in an interactive manner through a learning doll wirelessly connected to the computer. The present invention relates to an interactive language learning system. In particular, a user-led interactive dialogue can be developed between a user and a computer through a learning doll by allowing a computer to recognize and judge a user's question or response through voice recognition. In addition, the contents of conversations in various situations that can be developed in real-life conversations and background image data or sound data suitable for the conversations are stored in advance in a database, and the next conversation is responded to in response to a user's question or answer. While choosing according to Interactive dialogue language learning system using computer and learning dolls with voice recognition function to actively respond to user's intention by sending background screen and sound to wireless learning doll and outputting sound and voice. It is about.

Conventional education system using a doll consists of a one-way computer-driven conversation process of setting a story to be developed in advance and requiring an answer for a given sentence. Therefore, the user is asked to memorize and answer the sentence, so that the story development of the learning conversation is monotonous, there is no variety of choices in the answer, and there is a lack of flexibility, so that the user is easily tired and the learning efficiency is reduced. There was this. That is, the conversation in daily life develops various situations according to the contents of the question, and since the conversation proceeds with various forms of answers, the conversation form fixed in a specific form is unrealistic. This results in loss of learning motivation and disgust of the user, resulting in poor learning effect.

In addition, since it does not include a speech recognition function, there is no ability to determine the validity of the user's answer contents. After giving the user time to answer for a certain time, the conversation proceeds to the next sentence regardless of the contents of the answer, or to the next conversation only by the manual sentence input by the keyboard or the selection of the answer sentence presented by the button. It was so constructed that no real interaction was possible.

As described above, the conventional language learning systems using puppets do not have a mutual cooperation system using multimedia and a speech recognition method is not used, and thus, it is impossible for a user and a computer to communicate with each other and learn languages autonomously. In addition, the development of real-time interactive three-dimensional language education software is not at home and abroad.

Therefore, the user can present a variety of answers appropriate to the situation according to the dialogue question, and the speech recognition allows the conversation to proceed to a new situation according to the answer sentence presented so that real-life language learning such as the actual daily conversation situation can proceed. Development of an interactive language learning system using a computer having a function and a learning doll is required.

Accordingly, the present invention is derived to solve the above problems, the present invention allows a computer having a voice recognition function to recognize and determine the questions or responses of a plurality of users using the learning doll, real life The conversation contents of various situations that can be developed in the conversation and the background image data or sound data suitable for the conversation contents are stored in the database in advance so that the next conversation in the tree structure can be selected while responding to the user's question or answer. A computer having a voice recognition function that actively outputs a background image and a sound, thereby coping with a user's intention, and enables a user-led interactive dialogue to be developed between a plurality of users and a computer through a doll. Interactive language learning system using English and learning dolls There is a purpose.

In order to achieve the above object, according to an embodiment of the present invention, voice recognition means for analyzing a voice signal received from a learning doll and recognizing the meaning of the voice signal; Video and audio output means for signal-processing video and audio signals, respectively, and outputting them to a monitor screen and a speaker; Storing question data based on various situations that can be developed in real life conversations, a plurality of answer data according to the question, and image data and sound data corresponding to the contents of the question and the plurality of answer data as learning materials. Data storage means; Read the data desired by the user from the data storage means to export the question data of the corresponding learning material to the video and audio output means, when the answer data of the user through the voice recognition means the response data read from the data storage means And control means for judging whether the answer is correct or incorrect, comparing the next question data corresponding to the user's answer data according to the determination result, and reading the learning data of the data storage means and outputting them to the video and audio output means. An interactive dialogue language learning system using a computer having a speech recognition function and learning dolls is provided.

In one embodiment of the present invention, the data storage means, a question material for language learning, a plurality of answer data for the question material, and the data divided into background image data and background sound data according to the conversation contents, respectively The base is composed, but the question material and a plurality of answer data for the question material, and a new question material for each answer material and a plurality of new answer data for the new question material in a tree structure It is characterized by storing the training data.

1 is a block diagram of an interactive conversational language learning system using a computer having a speech recognition function and learning dolls according to an embodiment of the present invention;

FIG. 2A is a detailed block diagram illustrating the subprocessing unit and the data storage unit of FIG. 1; FIG.

2b is a block diagram illustrating a tree data structure in detail;

3 is an operation flowchart of an interactive dialogue language learning system using a computer having a speech recognition function and learning dolls according to the present invention;

4A is an operational flow diagram for the data processing subroutine of FIG. 3;

4b is an operation flowchart of the speech recognition subroutine of FIG.

4C is a flowchart illustrating an operation of a background image output subroutine of FIG. 3;

4d is an operation flowchart of the background sound output subroutine of FIG. 3;

5 is a block diagram illustrating another embodiment of an interactive language learning system using a computer having a voice recognition function and a plurality of learning dolls according to the present invention.

<Explanation of symbols for the main parts of the drawings>

10: key input unit 20: voice recognition unit

21: microphone 30: audio output unit

31: speaker 40: video output unit

41: monitor 50: main processing unit

60: subprocessing unit 61: background image processing unit

62: background sound processor 63: voice output processor

64: speech recognition processing unit 70: data storage unit

71: background image database 72: background sound database

73: voice output database 74: voice recognition database 81,82: first and second wireless transceiver 100: voice recognition computer 110,120,130: learning doll

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration of an interactive language learning system according to the present invention. First, the learning doll 110 includes a microphone 21 for converting an externally input voice signal into an electrical signal and the Analyzing a voice signal from the outside through the microphone 21, the voice recognition unit 20 for recognizing the meaning of the voice signal and converting it into text data, and processing the video signal and the audio signal, respectively, the monitor ( 41 and an image output unit 40 and an audio output unit 30 output to the speaker 31, a key input unit 10 for inputting training data to be stored in a database and for selecting a learning type requested by a user, And a first wireless transmission / reception unit 81 for modulating or demodulating data transmitted and received wirelessly between the voice recognition computers 100. Next, the voice recognition computer 100 includes the learning. The second wireless transceiver 82 for modulating or demodulating data transmitted and received wirelessly between the doll 110 and the questionnaire configured based on various situations that can be developed in a real-life conversation, the answer data for the question, and the conversation. Data learning unit (70) for storing the background image data or sound data according to the content and the learning data stored in the data storage unit 70, the type of learning data desired by the user to read the learning doll (110) The voice of the user recognized by the voice recognition unit 20 provided in the learning doll 110 or the voice recognition processing unit 64 separately provided in the voice recognition computer 100. And the next question material corresponding to the contents of the user's response according to the result of the comparison, and read out from the data storage unit 70. It consists of a main processing unit 50 and a sub-processing unit 60 output to the learning doll 110.

The voice recognition unit 20 receives the signal output from the microphone 21 and converts the first and second radio transceivers 81 and 82 into a matching word based on the pronunciation and accent of the user. Output to the main processing unit 50, which can also be configured as a subroutine called and driven by the main processing unit 50.

The sub-processing unit 60 is provided with each processing routine individually, the main processing unit 50 outputs a plurality of process control signals for controlling the sub-processing unit 60, the sub-processing unit 60 is learning A data storage unit 70 for storing data may be included or may be configured separately. When there is a learning request by the user, the main processor 50 controls the sub processor 60 using a plurality of tasks to read training data of the data storage unit 70 or read the read data. Export to audio and video output. In addition, it compares whether the signal received from the outside through the voice recognition unit 20 and the answer data of the data storage unit 70, and controls the output of the background image and the background sound according to the comparison result, and at the same time You can read the training material for the next conversation or iterate the current training data along the previous movement flow for repetitive learning.

FIG. 2A is a detailed block diagram illustrating the sub processor 60 and the data storage 70 of FIG. 1. As shown in the figure, the data storage unit 70 is divided into a language learning question material for voice output, a plurality of answer data for the question material, and a background image data and background sound data according to the conversation contents. It consists of a database (71-74), a plurality of answers to the questionnaire and the questionnaire, and a new questionnaire for each of the answer, and a number of new answers to the new questionnaire Is formed into one file, and it is configured to have a tree structure as shown in FIG. 2B.

In this case, the background image database 71 stores image data for outputting a background screen, the background sound database 72 stores sound data for outputting a background sound, and the audio output database 73 outputs audio. For user data (such as question data and a number of answer data corresponding to them, and each memory address where background image data and sound data are stored to create an atmosphere according to the conversation contents), and a voice recognition database ( 74 may store word data corresponding to the recognized voice.

The sub processor 60 loads data corresponding to the process control signal output from the main processor 50 from the background image database 71 so that the second and first wireless transceivers 82, A background image processor 61 for outputting to the monitor 41 in turn and returning data on a current progress state to the main processor 50; The background sound data corresponding to the process control signal output from the main processor 50 is loaded from the background sound database 72 and sequentially passed through the second and first wireless transceivers 82 and 81 to the speaker 31. A background sound processor 62 for outputting the data to the main processor 50 and outputting the data on the current progress state to the main processor 50; The voice data corresponding to the process control signal output from the main processor 50 is loaded from the voice output database 73 and output to the speaker 31, and data on a current progress state is output to the main processor 50. A voice output processor 63 for returning the voice signal; Controlled according to the process control signal of the main processor, the signal recognized by the voice recognition unit 20 is compared with the data loaded from the voice recognition database 74, and the data of the comparison result is output to the main processor 50. And a component speech recognition unit 20 which performs a speech recognition function on the wireless recognition computer 100 and the learning doll 110 as in the above-described example. All of the speech recognition processing unit 64 may be provided and operated, and the component for performing the speech recognition function may be provided and operated only in one of the wireless recognition computer 100 and the learning doll 110.

3 is an operation flowchart of the interactive language learning system according to the present invention, which shows a signal processing flow executed by the main processing unit 50.

In the drawing, the present invention comprises a first step (S311-S313) of constructing and outputting an initial screen for learning guidance at the same time as the program start and waiting for a learning request through the initial screen; When there is a learning request by the user, the subroutine for voice output (FIG. 4A) is executed to read out the question data of the training data stored in the database and output it as a voice, and at the same time, the subroutine for voice recognition (Fig. 4A). A second step (S318, S319-1, S320) of determining whether the response contents of the user are correct from the result of speech recognition and stored answer data in the database by executing 4b); According to the determination result, the question material and the answer data answered by the user are combined to form a file, and the file is read from the database, and the data for the next question is selected and outputted by voice, thereby providing a response to the user's response. It may be configured as a third step (S320-1, S321) to continue the next learning of the content that matches.

Here, in the first step, for example, a learning type (a) that can be divided by 'sentence practice', 'following a sentence' or 'pronounced practice', and a learning level (b) that can classify difficulty according to each learning type. Outputting a guide screen to select a user's intention and the like (S314); Waiting to select a learning type (a) or a learning grade (b) through the guide screen; and setting the selected learning type and grade as a learning level when the selection is made in the waiting state (S316). )Wow; If the selection is not made in the standby state, the method may further include setting a predetermined basic learning type and grade as a learning level (S317).

In the second step, the background data of the learning data stored in the database is read and monitored by executing a subroutine (FIG. 4C) for outputting the background screen simultaneously with the voice data processing subroutine (FIG. 4A) of the training data. The process may further include outputting the screen to a screen, and executing a subroutine for outputting a background sound (FIG. 4D) to read the background sound data of the learning data stored in the database and output the same to the speaker (S319).

The third step further includes the step (S322) of outputting a warning message for notifying a correct answer when it is determined that the user's answer and the answer stored in the database are inconsistent from the subroutine execution result. In addition, the method may further include a step of executing a voice output subroutine to read out the stored answer contents and the new question data in the database and output the result in voice (S323). At this time, the warning message, the stored answer contents, and the new question data can be output by text, and voice or text can be used simultaneously.

In addition, the third step may further include a step (S324) of checking whether to repeat the learning data for the incorrect answer data, it can be configured to perform the repetitive learning according to the user's intention.

4A to 4D are flowcharts illustrating operations of the voice output subroutine, the voice recognition subroutine, the background image output subroutine, and the background sound output subroutine of FIG. 3, respectively. In the figure, each subroutine is called and driven by the main routine (FIG. 3), and the data for searching the database is provided from the main routine, and the result of searching the database and the speech recognition result are again. It is configured to return to the main routine.

FIG. 5 is a block diagram illustrating another embodiment of an interactive language learning system using a computer having a speech recognition function and learning dolls according to the present invention, wherein one voice having the components shown in FIG. The voice recognition computer 100 includes a plurality of learning dolls 110-130, which are passively operated by the voice recognition computer 100 by receiving data and addresses from the recognition computer 100 and receiving a control signal wirelessly. In parallel, it is structured so that language learning can be carried out between one person and many people (one professor and several students).

Referring to the operation of the present invention configured as described above and the effect thereof is as follows.

Referring to each of the drawings, at the same time as the start of the program operation (S311) outputs the initial screen (S312), and determines whether there is a learning request (S313). If there is a learning request as a result of the determination, a selection screen for selecting a learning type or a learning grade (difficulty) again is output (S314). At this time, the selection screen allows the user to select a desired type of learning and presents and selects menus of various levels so that the user can select the education level. The learning type and difficulty selected in the state are set to a level to be learned later (S316). If there is no selection, the learning level is set using a predetermined value (S317).

Then, the voice output subroutine is called (S318) to execute. In this case, as shown in FIG. 4A, the voice output subroutine checks an initial value of a database using data provided from a main routine, and loads training data corresponding to a set level from a database (S402). The question data of the data is output by voice (or text) (S403), and the value is returned to the main routine (S404).

Next, the background image output subroutine and the background sound output subroutine are simultaneously called (S319) and executed by using the data returned from the voice output subroutine, and the voice recognition subroutine is executed (S319-1). . Therefore, in the speech recognition subroutine, as shown in FIG. 4B, after checking the initial value of the database using the data provided in the main routine, the user waits for the speech recognition result (S412). In step S413, the answer data is compared with the voice recognition result and the answer data of the database (S414), and the comparison results are returned to the main routine, respectively (S415 and S416). In the background image output subroutine and the background acoustic subroutine, as shown in FIGS. 4C and 4D, the initial value of the database is checked using the data provided by the main routine, and the background image data of the training data corresponding to the set level is checked. And load background sound data from respective databases (S422 and S423), and output background image data and background sound data of the loaded learning data (S432 and S433).

Next, the result returned from the speech recognition subroutine is checked (S320), and if the answer is correct, the question data and the response data are combined into a file to read the next user fee from the database (S320-1). And, by searching the file in the database and executing the voice output subroutine (S321), the new question material corresponding to the user's response is output as voice, so that new learning can be continued.

The result returned by the speech recognition subroutine S319-1 is checked (S320), and when it is not a correct response, a message for notifying that the speech is not a correct response is output as voice or text (S322). The correct answer data stored in the database is read and output in voice or text (S323), so that the user can check the correct answer. If the question is asked again to review the wrong problem (S324), and if the answer is affirmatively, the screen returns to the step of sequentially performing the background screen and sound output and speech recognition so as to learn the wrong problem again (S319). If you do not want to review the wrong problem, in order to read the next user fee from the database, the question material and the response data are combined into a file (S320-1), and the file is found in the database. By executing the routine (S321), a new question material corresponding to the user's response is output as a voice so that new learning can be continued.

The above processes are described as specific examples.

First, assuming a conversational situation where a Korean who greets a foreign visitor at Gimpo Airport inquires about the visitor's experience in Korea, the mutual three-dimensional conversational English learning method proceeds as follows.

Welcome to Korea! Have you been in Korea before?

All possible answer sentences to answer the above questions are stored in the data storage unit 70. At this time, the wallpaper becomes Gimpo Airport, and the dynamic background and sound effects of the plane take off and land simultaneously. Assume that the answer stored in the data storage unit 70 is one of the following.

A) Yes, I have.

B) Yes, I have been there many times.

C) No, I have not.

When the user gives a matching answer among the above three answers, the contents of the next conversation according to the selected answer are stored in the data storage 70. For example, it is assumed that the new questions stored in the data storage unit 70 for the sentence A are as follows.

How long have you stayed?

As a possible answer here, it is assumed that the data storage unit 70 is stored as follows.

A) a week

2) two weeks

A) a month

D) More than two months

When the user answers a), it is assumed that the question of the next progress question and answer stored in the data storage unit 70 is as follows.

What was your impression of Korea?

The background screen for the question is a content for properly displaying the content of the question and is stored in the data storage unit 70. In the case of the above question, various three-dimensional background images such as a heart symbol meaning good impression, autumn Seoraksan mountain meaning beauty, and the inside of a complicated subway with a disgusting commute time are used as a background screen that emphasizes impression. It is stored at 70.

As described above, the image output unit 40 increases the learning efficiency by visually stimulating the user's understanding while outputting the image material on the screen. The present invention is not limited to the above-described embodiments and is provided to those skilled in the art. Various modifications and changes may be brought about by the present invention, and in particular, an embodiment in which data is transmitted and received by wire between at least one or more learning dolls 110 and the voice recognition computer 100 may be proposed, which is defined in the appended claims. It is included in the spirit and scope.

Therefore, according to the learning system of the present invention, by allowing the user to interact with the user through a doll connected to the computer by wireless, enabling the language development of various contents according to the learning contents, so that the user can proceed as a subject In addition, the video and background sound according to the contents of the dialogue are output to the monitor screen and the speaker, so that the education of the three-dimensional dialogue method with the doll like the real language can enhance the user's concentration and learning effect. There is an advantage to that.

Claims (13)

A voice input means for receiving a voice of a user and converting the voice into an electrical signal; First wireless transmission and reception means for modulating the output electrical signal of said audio input means and transmitting it wirelessly to a computer, and receiving and demodulating an image, sound or audio signal sent from said computer; And A doll including a video or audio output means for processing a video signal or an audio signal and outputting the signal to a monitor screen or an audio system; And Second wireless transmission / reception means for receiving and demodulating a radio signal transmitted from the doll, and modulating an output image, sound, or audio signal of a computer and transmitting the radio signal to the doll; First speech recognition means for analyzing the speech signal extracted from the received signal and recognizing the meaning of the speech signal; The question data for language learning, a plurality of answer data for the question, new question data corresponding to each of the plurality of answer data, and a plurality of new answer data for the new question in a tree structure. It forms a learning data structure in which a conversation proceeds in a tree structure in response to a question or an answer, and stores image data or sound data corresponding to the contents of the question and the plurality of answer data, respectively, wherein the question data and the question are stored. When storing a plurality of answer data for, including information for extracting the corresponding material of the next question data corresponding to each answer data and the output image data or audio data corresponding to each answer data together Data storage means for storing the questions and answers in a file form, and Read the question data of the learning material selected by the user from the data storage means to export to the video or audio output means, when the response or response data of the user through the first voice recognition means, the answer data pre-stored in the data storage means And the next question data corresponding to the user's answer data and the corresponding video or audio data are read from the learning data of the data storage means and output the video or audio according to the determination result. And a control means for outputting the means. The interactive conversational language learning system using a computer having a speech recognition function and a learning doll, comprising a computer. The method of claim 1, wherein the doll, And a button key input means for allowing a user to input by manual operation instead of voice input in response to a question. The computer having a voice recognition function according to claim 1, wherein the doll further comprises second voice recognition means for analyzing a voice signal input from a user and recognizing the meaning of the voice signal. Interactive language learning system using language and learning puppets. The method of claim 3, wherein the second voice recognition means, An interactive conversational language learning system using a computer having a speech recognition function and a learning doll, characterized by analyzing an input voice signal, converting the meaning of the input voice signal into text data, and outputting the same to text data of the doll. . delete delete delete 4. The system of claim 3, wherein the second wireless transmission / reception means is capable of wireless transmission and reception with a plurality of dolls. The method of claim 1, wherein the control means, Main processing means for simultaneously processing a plurality of process control signals to support a plurality of tasks at once; Operated by the processor control signal of the main processing means to search the learning data of the data storage means, and to provide the main processing means with the next question content of the searched learning data data and the position information of the corresponding background image data or background sound data Interactive language learning system using a computer having a speech recognition function and the learning doll, characterized in that it comprises a sub-processing means. The method of claim 9, wherein the control means, It operates by the process control signal of the main processing means to drive the first speech recognition means, and compares the answer data read from the data storage means with the answer data of the user coming in through the first speech recognition means, and And a speech recognition processing means for providing a comparison result to the main processing means. The method of claim 9, wherein the control means And a background image processing unit operating by the process control signal of the main processing unit to read out the image data of the learning data structure from the data storage unit and output the processed image through the image output unit. Interactive language learning system using a computer and learning doll having a. The method of claim 9, wherein the control means, Speech recognition means further comprises a background sound processing means for operating by the process control signal of the main processing means to read the sound data of the learning data structure from the data storage means, and output the processing through the sound output means. Interactive language learning system using computer with learning function and learning dolls. 10. The computer program according to claim 9, wherein the main processing means processes a plurality of doll processor control signals at the same time to support a plurality of tasks communicating with a plurality of dolls at once. Interactive language learning system.