KR20010065286A - apparatus and method for providing learning capacity program - Google Patents

Abstract

PURPOSE: A device and a method are provided to promote the studying efficiency by a studying program. CONSTITUTION: The device includes a monitor(11), a camera(12), a key board(13) and a service supplying system(20). The method comprises the steps of: constructing the database for position patterns, expression patterns and mixed patterns thereof; receiving a user's image from the camera installed on the monitor; making a screen pattern with the gained patterns from the image; searching a most similar pattern after comparing the saved patterns in the database; considering a key signal of the key board operated by the user as a studying status; determining the studying status of the user by summarizing the key signals, the position pattern and the expression pattern; and outputting a proper studying program to the monitor.

Description

Apparatus and method for providing learning capacity program}

The present invention is to efficiently provide a learning program suitable for a user, and more particularly, to a method and apparatus for providing a learning efficiency program suitable for a user according to a learning state of the user.

Commonly used software for learning programs includes language learning programs, math learning programs, and programs with content for acquiring specific leisure knowledge, such as Go. However, all such learning programs contain only the content of a certain knowledge, and the contents considering the learning state of the user have not been considered at all.

On the other hand, conventional apparatuses for operating the learning programs are, as are well known, compact disc players, computers, video cassette recorders, and laser disc players. These devices display the contents of the loaded disk or tape on the monitor as they are, without any other functionality.

An object of the present invention is to provide an apparatus and method for providing a learning program for efficiently providing a learning program to the user according to the learning state of the user.

In accordance with one aspect of the present invention for achieving the above object, the apparatus displays a learning program content and provides a monitor for providing information on what content is displayed, a camera taking an image of a learner, and the learner learning the learning. A tool used to operate a program includes a key board for providing corresponding key input signals generated by manipulation of the learner as one learning status signal, and a service providing device.

The service provider receives information about the learner from the keyboard and the camera. The information is finally determined using the information, and the optimal learning program is displayed on the monitor according to the finally determined learning state.

According to another aspect of the present invention for achieving the above object, the method comprises the steps of pre-establishing databases of all possible posture patterns, facial expression patterns, and composite patterns thereof, from the camera installed in the monitor. A step of receiving an image, extracting an image from the received image, creating an image pattern consisting of a posture pattern and a facial expression pattern from the extracted image, and creating the created posture pattern and facial expression pattern in the pre-built database Comparing the stored posture patterns and facial expression patterns with the stored posture patterns and facial expression patterns, and retrieving the most similar posture patterns and facial expression patterns among the stored posture patterns and facial expression patterns; Considering step, the considered key signals, the most similar A training program suitable for the position and pattern of expression and synthesis steps of the pattern ultimately determine the learning status of the user, and a pre-stored number of training learning condition of the determined users of the includes a step of outputting to the monitor.

1 is a block diagram showing the configuration of an apparatus for providing a learning program according to the present invention.

2 is a block diagram showing a detailed configuration of the central processing unit in FIG.

3 is a flow chart for explaining the operation of the apparatus according to the present invention.

4A is a flowchart showing a process of building a posture pattern database

4B is a flowchart showing a process of building a facial expression pattern database

4C is a flowchart showing a process of building a composite pattern database 234 of the posture and facial expression.

4D is a flowchart illustrating a process of extracting a posture pattern from input image data.

4E is a flowchart showing a process of extracting an expression pattern from input image data

5A is a flowchart illustrating a posture pattern search process by the posture pattern search module of FIG. 3.

5B is a flowchart illustrating a facial expression pattern retrieval process by the facial expression pattern retrieval module of FIG.

FIG. 5C is a flowchart illustrating an algorithm for determining a learner final state by the learner state determining module of FIG. 3. FIG.

FIG. 5D is a simplified flow chart showing the process of creating the educational program database of FIGS. 2 and 3.

* Explanation of symbols for the main parts of the drawings

11: camera 12: monitor

13: key board 20: service provider

21: central processing unit 22: video receiving device

23: image pattern database 24: user input processing device

25: video transmission device 26: training program database

Hereinafter, a preferred embodiment of an apparatus and method for providing a learning program according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the configuration of an apparatus for providing a learning program according to the present invention.

The apparatus of FIG. 1 includes a monitor (or television receiver) 11, a camera 12 mounted on the monitor 11, a key board 13, and a service providing apparatus 20. First, the camera 12 may be installed in the television receiver (or monitor) or may be installed in a separate fixed device, and may take an image of a user (or learner) of the device and provide it to the service providing device 20. It plays a role. The monitor 11 displays the contents of the learning program from the service providing apparatus 20 and transmits the contents to the service providing apparatus 20. The key board 13 is a tool used by the user to operate the corresponding learning program, and serves to provide the corresponding key input signals to the service providing apparatus 20 by the user's manipulation.

Meanwhile, the service providing apparatus 20 finally determines the learning state of the user (or the learner) by using the information from the monitor 11, the camera 12, and the key board 13, and determines the determined learning. It is to provide the user with the most optimal learning program according to the state. The service providing apparatus 20 may include a central processing unit 21, an image receiving apparatus 22, a pattern database 23, a user input signal processing apparatus 24, an image transmitting apparatus 25, and a learning program database. It consists of 26. First, the central processing unit 21 serves to control all the components in the service providing apparatus 20, and also input signals from the monitor 11, the camera 12, and the keyboard 13. And vice versa for processing signals to be provided to these devices. The image receiving device 22 serves to receive an image from the camera 12 or the monitor 11 and provide the image to the central processing unit 21. The pattern database 23 stores and stores various pattern information about the image of the user as needed. The user input signal processing device 24 processes the key signals from the key board 13 and provides them to the central processing unit 21. The image transmission device 25 serves to transmit an image corresponding to an appropriate learning program to the monitor 11 (or a television receiver) according to the learning state of the user under the control of the central processing unit 21. .

The educational program database 26 stores a large number of programs and provides necessary learning programs at the request of the central processing unit 21 according to the learning state of the user.

2 is a block diagram showing a detailed configuration of the central processing unit 21 in FIG.

The central processing unit 21 of FIG. 2 includes an image image extraction module 211 for extracting an image image from an image signal received from the camera 12, and an image pattern extraction module for extracting an image pattern from the input image image ( 212), a posture pattern search module 213 for searching for a posture pattern of a learner, that is, a user, from a posture pattern database 231 of the pattern database 23 using the input image pattern, and the input image pattern The facial expression pattern search module 214, the posture pattern search module 213, and the facial expression pattern search module for searching for the facial expression pattern of the user from the facial expression pattern database 232 of the pattern database 23 by using a. Subject state database 233 of the pattern database 23 using the outputs of 214 and the key input signal from the keyboard 13. Learner state determination module 215 for determining the current state of the learner, and retrieves and manages the required education program from the education program database 26 by a predetermined control signal according to the current state of the learner. Provide the predetermined control signal to the education program database management module 217 in response to an education program database management module 217 and a current state signal of the learner from the learner state determination module 215; And a central processing module 216 for providing the monitor 11 (or television receiver) with the necessary educational program retrieved by the control signal. In other words, the central processing module 216 provides the monitor 11 with an education program suitable for the current learning state instead of the program currently being displayed on the monitor 11.

3 is a flow chart for explaining the operation of the apparatus according to the present invention.

Operation of the apparatus will be described in detail with reference to FIGS. 1 to 3.

When the service flow of the training program starts, first, the service providing apparatus 20 of the apparatus according to the present invention receives the image of the user from the camera 12 installed in the monitor 11 and the image receiving apparatus 22. Receive through. Subsequently, the image image extraction module 211 of FIG. 3 extracts an image from the received image. On the other hand, the image pattern extraction module 212 generates a pattern from the extracted image. In this case, the image pattern extraction module 212 extracts the image pattern by dividing the image pattern into a posture pattern and a facial expression pattern. The posture pattern search module 213 and the facial expression pattern search module 214 use the posture pattern and the facial expression pattern, respectively, to store the stored posture patterns of the posture pattern database 231 and the facial expression pattern database 232. And compare the facial expression pattern with the extracted posture pattern and the facial expression pattern to search for a comparison posture pattern and a comparative facial expression pattern, respectively. Meanwhile, the central processing module 216 of FIG. 3 receives and processes a key input signal by a user's operation and sends the key signal to the learner state determination module 215. Subsequently, the learner state determining module 215 finally aggregates the signals from the posture pattern search module 213, the facial expression pattern search module 214, and the central processing module 216 to finally determine the learning state of the user. Decide Meanwhile, the central processing module 216 reads a program suitable for the determined learning state of the user through the education program database management module 217 from the education program database 26 and then monitors the program output device. Transfer the corresponding program to (12).

Types of suitable programs according to the user's state are as follows.

First, when the user's facial expressions are curious, a program of an easier version is provided than the currently provided program.

Second, if the user is in a boring state, the program transmits a program that may cause interest.

Third, when the learning attitude of the user is bad, a warning message is transmitted.

Fourth, when the user is dozing, it provides a signal to generate a light music or alarm sound.

Fifth, if the user is seriously working on the current learning state, the message of encouragement or other program of interesting content is transmitted according to the state. 4A is a flowchart illustrating a process of building the posture pattern database 231.

First, in order to construct the database of the posture pattern, the possible posture pattern data is divided into characteristic data about arms, legs, and hands. Subsequently, the following types of posture patterns are sequentially constructed by various combinations of the three types of feature data.

First, construct attitude patterns for active learning, second, construct attitude patterns for boredom, third, posture patterns for leaving positions, and fourth, posture patterns for feeling tired, and fifth, construct attitude patterns other than learning. do. In addition, other various posture patterns may be further constructed in the posture pattern database 231.

4B is a flowchart illustrating a process of building the facial expression pattern database 232.

First, in order to construct the database of the facial expression pattern, the facial expression pattern data is divided into feature data for eyes, mouth, and eyebrows. Subsequently, the following types of facial expression patterns are sequentially constructed by various combinations of the three types of feature data.

The first serious expression pattern, the second boring expression pattern, the third sad expression pattern, the fourth glad expression pattern, the fifth tired expression pattern, the six embarrassed facial expression pattern, the seventh curious facial expression pattern, the eighth smiley facial expression pattern, The ninth crying pattern is constructed in sequence. In addition, you can build more different kinds of facial expression patterns.

4C is a flowchart illustrating a process of building the composite pattern database 234 of the posture and facial expression.

First make all possible combinations of the posture patterns and facial expression patterns. Subsequently, the complex pattern database is constructed by arranging the above combined patterns in a predetermined order.

4D is a flowchart illustrating a process of extracting a posture pattern from input image data.

First, the image pattern extraction module 212 of FIG. 3 extracts sample data to be compared with posture patterns of a previously constructed database from input image data. Then, the sample data is divided into head, arms, legs, and hands, and then feature points are extracted from sample data of each head, arms, legs, and hands. Subsequently, the extracted feature points are organized into data of each element for comparison with the built-in database.

4E is a flowchart illustrating a process of extracting an expression pattern from input image data.

First, the image pattern extraction module 212 of FIG. 3 extracts sample data to be compared with expression patterns of a previously constructed database from input image data. Subsequently, the sample data is divided into eyes, mouths, and eyebrows, and then feature points are extracted from each eye, mouth, and eyebrow sample data. Subsequently, the extracted feature points are organized into data of each element for comparison with the built-in database.

FIG. 5A is a flowchart for describing a posture pattern search process by the posture pattern search module 213 of FIG. 3.

A process of searching for and comparing the posture feature data extracted and arranged by the image pattern extraction module 212 and the posture pattern of the pre-established posture pattern database 231 is performed. Then, the most similar posture pattern among the posture patterns of the posture pattern database 231 is searched.

FIG. 5B is a flowchart illustrating a facial expression pattern retrieval process by the facial expression pattern retrieval module 214 of FIG. 3.

A process of searching and comparing the facial expression feature data extracted and arranged by the image pattern extraction module 212 and the facial expression pattern of the already constructed facial expression pattern database 232 is performed. Then, the most similar facial expression pattern is searched among the facial expression patterns of the facial expression pattern database 232.

FIG. 5C is a flowchart illustrating an algorithm for determining a learner final state by the learner state determination module 215 of FIG. 3.

First, the searched comparison posture pattern and the searched comparison facial expression pattern are combined. Subsequently, a process of comparing and searching the combined pattern and various complex patterns stored in the complex pattern database 234 is performed. Subsequently, the learner state determination module 215 refers to the user key input signal from the key board 13 input through the central processing module 216. Then, the learning state of the user is finally determined.

FIG. 5D is a simplified flow chart showing the process of creating the educational program database 26 of FIGS. 2 and 3.

First the main educational program is produced, followed by separate learning programs for all possible user learning states.

As described above, according to the apparatus and method of the present invention, the educational program can be reasonably changed according to the current subject, condition, and the like, so that the learner or user can efficiently and suitably use the learning program.

Claims (15)

A monitor that displays the content of the learning program and provides information about what is displayed; A camera that takes an image of the subject; A key board used by the learner to operate the learning program, the key board providing corresponding key input signals as one learning state signal by manipulation of the learner; And. A service providing device which is provided from the keyboard and the camera using the information on the learner and finally determines the learning state of the learner using the information, and displays the optimal learning program on the monitor according to the finally determined learning state. Apparatus for providing a learning program, characterized in that consisting of. The apparatus of claim 1, wherein the camera may be installed on any one of a fixed device separate from the monitor. An apparatus according to claim 1, wherein said monitor is a general television receiver. According to claim 1, The service providing device, A central processing unit controlling all the components in the service providing apparatus; An image receiving apparatus for receiving an image of the subject of learning from the camera and providing the image to the central processing unit; An image pattern database storing various pattern information of the image of the subject of learning and providing an image pattern required by the control of the CPU; A user input signal processing apparatus for processing key input signals from the key board and providing the processed key input signals to the central processing unit; An image transmission device for transmitting an image corresponding to an appropriate learning program to the monitor according to a learning state of the learning subject under control of the central processing unit; And And a training program database for storing a plurality of training programs and for providing the appropriate learning program at the request of the central processing unit according to the learning state of the user. The apparatus of claim 4, wherein the image pattern comprises a posture pattern and a facial expression pattern. The method of claim 4, wherein the central processing unit, Image image extraction module for extracting a video image from the video signal received from the camera, An image pattern extraction module for extracting an image pattern from the input image image; A posture pattern search module for searching for a posture pattern of a learner from the pattern database using the input image pattern; An expression pattern search module for searching for an expression pattern of the user from the pattern database using the input image pattern; A learner state determination module configured to determine a current learning state of the learner using data of the pattern database by using outputs of the posture pattern search module and the facial expression pattern search module and a key input signal from the keyboard; An education program database management module for searching for and managing a necessary education program from the education program database by a predetermined control signal according to a current learning state of the learner; and Providing the predetermined control signal to the education program database management module in response to the current learning state of the learner input from the learner state determining module, and displaying the necessary education program retrieved by the control signal on the monitor Apparatus for providing a learning program, characterized in that consisting of a central processing module. The method of claim 1, wherein the image pattern database includes a posture pattern database, a facial expression pattern database, a complex pattern database in which postures and expression patterns are combined, and a learner state database. Device providing a program. Pre-building databases of all possible posture patterns, facial expression patterns, and composite patterns thereof; Receiving an image of a user from a camera installed in the monitor; Extracting an image from the received image; Generating an image pattern consisting of a posture pattern and a facial expression pattern from the extracted image; Comparing the created posture pattern and facial expression pattern with posture patterns and facial expression patterns stored in the constructed database and searching for the most similar posture pattern and facial expression pattern among the stored posture patterns and facial expression patterns; Considering key signals of the keyboard by the user's operation as a learning state; Finally determining the learning state of the user by combining the considered key signals, the most similar posture pattern and facial expression patterns; And And outputting an education program suitable for the determined learning state of the user among the many stored education programs to the monitor. The method of claim 8, wherein the pre-stored educational programs, Program of the easy explanation version (version) provided when the user's facial expressions, A program that may induce interest provided when the user is in a boring state, A warning message provided when the learning attitude of the user is bad, Cheerful music-like alarm sound provided when the user is asleep, and And a message of encouragement and other content of interesting content provided when the user is seriously engaged in the current lesson. The method of claim 8, wherein the building of the posture pattern database comprises: Dividing all possible posture pattern data into characteristic data for arms, legs, hands; Creating several combinations of the three kinds of feature data; And And a step of constructing an active learning participation posture pattern, a bored posture pattern, a seated posture pattern, a tired posture pattern, and a behavioral posture pattern other than learning from the various combinations in any order. How to give a learning program. The method of claim 8, wherein the building of the facial expression pattern database comprises: Dividing all possible facial expression pattern data into feature data for eyes, mouth, and eyebrows; Creating several combinations of the three kinds of feature data; And A step of constructing a serious facial expression pattern, a boring facial expression pattern, a sad facial expression pattern, a happy facial expression pattern, a tired facial expression pattern, an embarrassed facial expression pattern, a curious facial expression pattern, a smiley facial expression pattern, and a crying facial expression pattern in random order from the above combinations Method for providing a learning program characterized in that it comprises a. The method of claim 8, wherein the building of the database of the complex patterns comprises: Making all possible combinations of the posture patterns and facial expression patterns; And And arranging each of the coupling patterns in an arbitrary order. The method of claim 8, wherein the extracting the posture pattern from the input image data comprises: Extracting sample data to be compared with posture patterns of the pre-built database from the input image data; Dividing the sample data into head, arms, legs, and hand sample data; Extracting feature points from sample data of the head, arm, leg, and hand; And And arranging the extracted feature points into a data form for comparison with the pre-established database. The method of claim 8, wherein the extracting the facial expression pattern from the input data comprises: Extracting sample data to be compared with expression patterns of the pre-built database from the input image data; Dividing the sample data into eye, mouth, and eyebrow sample data; Extracting feature points from each eye, mouth, and eyebrow sample data; And arranging the extracted feature points into a data form for comparison with the pre-established database. The method of claim 8, wherein the determining of the final subject state of the learner comprises: Combining the retrieved posture pattern and the retrieved facial expression pattern; Comparing and searching the combined pattern with various complex patterns stored in the complex pattern database; Referencing user key input signals from the key board; And And finally determining a learning state of the user.