KR20160143422A - Smart education system based on learner emotion - Google Patents

Smart education system based on learner emotion Download PDF

Info

Publication number
KR20160143422A
KR20160143422A KR1020150080133A KR20150080133A KR20160143422A KR 20160143422 A KR20160143422 A KR 20160143422A KR 1020150080133 A KR1020150080133 A KR 1020150080133A KR 20150080133 A KR20150080133 A KR 20150080133A KR 20160143422 A KR20160143422 A KR 20160143422A
Authority
KR
South Korea
Prior art keywords
learner
information
data
emotion
emotional
Prior art date
Application number
KR1020150080133A
Other languages
Korean (ko)
Other versions
KR101715323B1 (en
Inventor
권장환
Original Assignee
(주)인클라우드
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by (주)인클라우드 filed Critical (주)인클라우드
Priority to KR1020150080133A priority Critical patent/KR101715323B1/en
Publication of KR20160143422A publication Critical patent/KR20160143422A/en
Application granted granted Critical
Publication of KR101715323B1 publication Critical patent/KR101715323B1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00—Systems or methods specially adapted for specific business sectors, e.g. utilities or tourism
    • G06Q50/10—Services
    • G06Q50/20—Education
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances

Landscapes

  • Business, Economics & Management (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Tourism & Hospitality (AREA)
  • Theoretical Computer Science (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • General Business, Economics & Management (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Electrically Operated Instructional Devices (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

One embodiment of the present invention relates to a smart education system based on learner emotion, and the technical problem to be solved is analyzing real-time emotion information of a learner to provide personalized education according to academic level and mutual consent.
To this end, an embodiment of the present invention provides learning contents for customized education, and the learning data and the pool data, the emotional data of the learner, and the sympathy activity according to the emotional state of the learner Data is input, emotional index information is calculated by the emotional index calculation algorithm based on the emotional data, and then the emotional index information is mapped to the emotional index information of the emotional index information, and the emotional index information and the mapped emotional word An education support server for generating and managing emotional analysis information of the learner based on a sensibility analysis algorithm as a basis; A personalized education application is installed, and the personalized learning information is input by accessing the education support server by execution of the customized education application, and then registered as a learning member, then requested learning contents, photographing the learner's image data, A learner terminal for calculating emotional data of the learner based on the image data and providing the learning data and emotional data of the learner in real time through the learning contents and receiving the pool data and the sympathetic activity data; And a personalized education application installed in the learner terminal, accessing the education support server by execution of the personalized education application, inputting pool data of the learner data inputted by the learner terminal, A smart training system based on a learner's emotion including a teacher terminal that provides a learner's emotion.

Description

{SMART EDUCATION SYSTEM BASED ON LEARNER EMOTION}

One embodiment of the present invention relates to a smart education system based on learner emotion for analyzing real-time emotion information of a learner and providing personalized education according to academic level and mutual consent.

Recently, the issue of smart education is defined as an intelligent customized learning system for enhancing the capacity of learners in the 21st century and is defined as a power to innovate the education system such as educational environment, contents of education, methods of education and evaluation.

This is a holistic approach to improve learning effectiveness by converting learning methods horizontally, participatory, and interactively for communication and cooperation between professors and learners. In 2011, We have pursued various support policies.

However, until now, smart education has been concentrated on the hardware center such as electronic blackboard and smart pad, and as the technology has been developed centering on tools for learning support, it has become increasingly difficult for teachers and learners The problem is that the quality of education is deteriorated due to the weakening bonds and interactions.

Korean Patent Registration No. 10-1289870 'Smart class progress system using smart terminal' Japanese Patent Laid-Open Publication No. 10-2014-0147243 'Learning system and method based on user participation'

One embodiment of the present invention provides a smart education system based on a learner's emotion by analyzing real-time emotion information of a learner and providing personalized education according to academic level and mutual consent.

The smart education system based on the learner emotion according to an embodiment of the present invention provides learning contents for customized education, and the learning data and the pool data, the emotion data of the learner, and the learner The emotion index information is calculated by the emotional index calculating algorithm based on the emotional data, and then the emotional index information is mapped to the emotional index information of the emotional index information, An education support server for generating and managing emotional analysis information of the learner based on a mapped emotional word for each section; A personalized education application is installed, and the personalized learning information is input by accessing the education support server by execution of the customized education application, and then registered as a learning member, then requested learning contents, photographing the learner's image data, A learner terminal for calculating emotional data of the learner based on the image data and providing the learning data and emotional data of the learner in real time through the learning contents and receiving the pool data and the sympathetic activity data; And a personalized education application installed in the learner terminal, accessing the education support server by execution of the personalized education application, inputting pool data of the learner data inputted by the learner terminal, And a faculty terminal for providing the faculty terminal.

The emotion data may be a plurality of facial expression data calculated by a facial expression recognition algorithm for recognizing facial expressions of the learner.

The sensitivity index calculating algorithm can quantify the change of the facial expression data at a preset numerical value.

Wherein the emotional analysis algorithm calculates a learner emotion index according to the change of the emotional data, sets the calculated learner emotion index according to the sex or age of the learner, and then maps a preset emotion word for each emotion interval To generate emotion analysis information.

The emotion analysis information includes target emotion information and ideal emotion information, and the education support server may provide target emotion information or ideal emotion information for the learner to the instructor terminal.

The instructor terminal may compare the target emotion information or the ideal emotion information for the received learner with the change emotion information of the learner in a state in which the sympathy activity data is provided and provide the comparison result to the education support server.

The emotional analysis information includes learning capability and inclination analysis information, learner interest information for each subject, level of understanding level for each subject and detail, learning concentration information for each time period, information on recommended learning methods for each learner and subject, A content level for each degree of difficulty, and adequacy indicator information.

Wherein the education support server comprises: a communication module for transmitting and receiving data with the learner terminal or the instructor terminal; A member management module for registering and authenticating as a member using the learner information or the instructor information input from the learner terminal or the instructor terminal; A learning contents opening module for opening the learning contents at the request of the learner terminal; An emotion index information calculating module for calculating emotion index information by an emotion index calculating algorithm based on emotion data provided from the learner terminal; A sensibility section setting module for setting the sensibility index information according to sex or age of the learner; An emotional index information mapping module for mapping emotional words according to intervals according to the classified emotional index information; An emotional analysis information generation module that generates emotional analysis information including target emotion information or ideal emotion information by an emotion analysis algorithm based on the mapped emotional word for each section; A change emotion information generation module that generates change emotion information based on the emotion analysis information of the regenerated learner while providing the volunteer activity data; An algorithm automatic correction module that corrects the emotion analysis algorithm when the target emotion information does not match the change emotion information; Providing the instructor terminal with the target emotion information, the ideal emotion information, or the change emotion information for the learner, and generating the pool data, the emotion index information, and the sympathetic activity corresponding to the mapped word of the mapped section A data providing module for providing data to the learner terminal; A storage module for storing at least one of the learning data, pool data, learner information, instructor information, emotional data, emotional index information, emotional word information for each section, emotional analysis information, change emotional information, facial expression recognition algorithm, and emotional analysis algorithm; A instructor management module for matching and managing the instructor terminal with the learner terminal on the learning contents; A lock reservation module for reserving a lock status of the learner terminal during a learning process with the learner terminal; And a control module for controlling operations of the components constituting the education support server.

Wherein the education support server further comprises a sensitivity pattern information generation module that generates sensitivity pattern information for the learner individual or the learner group using the target emotion information and the change sensitivity information, To the instructor terminal, and can receive the liaison activity data for each learner individual or the learner group from the instructor terminal and provide it to each learner terminal.

The education support server may further include an alarm transmission module for transmitting an alarm signal previously set to the learner terminal when the abnormal sensation information is generated.

Wherein the automatic algorithm correction module re-calculates the learner emotion index by receiving emotional data from the learner terminal when the abnormal emotion information is generated, and determines whether or not an error is generated based on the re-calculated learner emotion index And if the error is detected as a result of the determination, the emotion analysis algorithm can be corrected.

Wherein the learner terminal comprises: a first communication unit for transmitting and receiving data with the education support server; Inputting the learner information and the customized learning information for accessing the education support server to receive the customized education service, inputting the request information for requesting the opening of the learning channel, inputting the learning data and the image data on the learning contents A first information input unit for inputting information; A first photographing unit for photographing a video of a learner in the course of learning; A emotional data calculation unit for calculating the emotional data of the learner based on a facial expression recognition algorithm for recognizing the facial expression of the learner based on the image data of the taken learner after the learning contents are opened; A first storage unit for storing learning process information performed on the learning content; A first display unit for displaying data transmitted to and received from the education support server, wherein the first display unit displays only data of the learning process while the learning process is in progress, An alarm signal output unit receiving an alarm signal from the education support server and outputting an alarm signal preset to the outside; And a first control unit installed with a customized education application and controlling the operation of each component constituting the learner terminal by executing the customized education application.

Wherein the instructor terminal comprises: a second communication unit for transmitting / receiving data to / from the education support server; A second photographing unit for photographing a lecture image of learning data performed by the instructor; A second information input unit connected to the education support server, for inputting the instructor information, and for inputting pool data for the lecture image; A sympathetic activity data generation unit for generating the sympathetic activity data corresponding to the emotion index information and the mapped word for each of the mapped sections; A data upload unit for editing the lecture video, pool data, and sympathetic activity data and providing the same to the education support server; A second display unit for displaying data transmitted to and received from the education support server; And a second controller for controlling an operation of each component constituting the instructor terminal.

Wherein the sympathetic activity data generation unit compares the emotion analysis information with preset change sensitivity information and generates sympathetic activity data according to the comparison result, and the second control unit generates the sympathetic activity data by comparing the sensibility analysis information with the changed sensibility information , It is possible to notify the education support server to correct the emotion analysis algorithm, or to recommend replacement contents replacing the learning contents.

According to the embodiment of the present invention, the smart education system based on the learner's emotions can be classified into the learning difficulty, the selection of the learning materials, the education progress method, etc. based on the comprehensive level of learner analysis including the academic level, individual learning ability, It is possible to provide customized learner-oriented training.

In addition, an embodiment of the present invention can grasp the real-time emotional state of a learner with respect to a class in progress, take immediate action upon detection of an ideal emotional state, and allow the algorithm to evolve automatically according to the learner's attainment of the target emotional state.

In addition, an embodiment of the present invention enables rapid measures so that the teacher can check the emotional state of the ever-changing learner while the class is in progress, adjust the education level, or improve the concentration of the class.

In addition, one embodiment of the present invention can provide a data analysis technique capable of converting emotion data collected and analyzed by various biometric information for emotional analysis of a learner into learning emotion by subdividing the emotion data so as to be specialized for a learner .

In addition, one embodiment of the present invention enables mutual-conscious type customized education through monitoring of the learner's real-time emotional state, and analyzes cumulative learner emotional data to implement intelligent customized learning, establish an effective education policy, This will provide a basis for the innovative transformation of the national education paradigm for cultivating global creative talents.

1 is a diagram schematically showing a smart education system based on learner emotion according to an embodiment of the present invention.
2 is a diagram schematically illustrating the operation of the smart learning system based on the learner's emotion shown in FIG.
3 is a block diagram schematically showing the education support server of FIG.
FIG. 4 is a block diagram schematically showing the learner terminal of FIG. 1. FIG.
FIG. 5 is a block diagram schematically showing the instructor terminal of FIG. 1. FIG.
FIG. 6 is a diagram illustrating a process of analyzing real-time learner emotion information through the education support server of FIG. 1;
FIG. 7 is a diagram illustrating a process of mapping an emotional word and an algorithm for calculating a emotional index of a learner through the education support server of FIG. 1;
FIG. 8 is a diagram showing analysis results of learner emotion information for each learner through the education support server of FIG. 1. FIG.
9A and 9B are diagrams showing results of a learner and contents analysis through analysis of learner's emotional pattern information through the education support server of Fig.

The terms used in this specification will be briefly described and the present invention will be described in detail.

While the present invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not limited to the disclosed embodiments. Also, in certain cases, there may be a term selected arbitrarily by the applicant, in which case the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the term used in the present invention should be defined based on the meaning of the term, not on the name of a simple term, but on the entire contents of the present invention.

When an element is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements as well, without departing from the spirit or scope of the present invention. Also, the terms "part," " module, "and the like described in the specification mean units for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software .

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly illustrate the present invention, parts not related to the description are omitted, and similar parts are denoted by like reference characters throughout the specification.

FIG. 1 is a diagram schematically showing a smart education system based on learner emotion according to an embodiment of the present invention, FIG. 2 is a view schematically showing the operation of a smart education system based on learner emotion shown in FIG. 1, 1 is a block diagram schematically showing a learner terminal of Fig. 1, Fig. 5 is a block diagram schematically showing a teaching terminal of Fig. 1, and Fig. 6 is a block diagram FIG. 7 is a diagram illustrating a process of mapping an emotional word and an algorithm for calculating a emotional index of a learner through the education support server of FIG. 1; FIG. , FIG. 8 is a diagram showing analysis results of learner's emotional intelligence information through the education support server of FIG. 1, and FIGS. 9A and 9B 1 is a diagram showing learner and content analysis results through analysis of emotional pattern information of a learner through the education support server of Fig.

Referring to FIGS. 1 and 2, a smart education system based on learner emotion according to an embodiment of the present invention analyzes real-time emotion information of a learner and provides customized education according to academic level and mutual consent type, The server 10, the learner terminal 20, and the instructor terminal 30. [

This smart learning system based on learners 'emotions is based on real - time learner emotional information analysis technology that allows the learner' s emotional state to be monitored in real time in the course of teaching, and instructor prepares individual learner or learner group education plan Learner emotional pattern analysis technology that provides various analytical materials for customized education can be made to interact with learners.

The education support server 10 is connected to the learner terminal 20 or the instructor terminal 30 through a wired / wireless communication network to provide learning contents for customized education and stores learning data and pool data, The learner's emotional data, and the sympathy activity data according to the learner's emotional state are uploaded. Also, the education support server 10 calculates emotional index information by the emotional index calculating algorithm (refer to FIG. 7) based on the emotional data, maps the emotional word for each section of the emotional index information, The emotion analysis information of the learner is generated and managed by the emotion analysis algorithm (refer to FIG. 6) based on the mapped sensibility word.

2, the education support server 10 includes a communication module 110, a member management module 115, a learning contents opening module 120, an emotion index information calculating module 120, A sensibility section setting module 135, a sensibility index information mapping module 140, a sensibility analysis information generation module 145, a change sensitivity information generation module 150, an algorithm automatic correction module 155, A sensory pattern information generation module 180, an alarm transmission module 185, and a control module 190. The control module 190 controls the operation of the controller module 160, the storage module 165, the instructor management module 170, the lock reservation module 175,

The communication module 110 is a device for transmitting and receiving data to or from the learner terminal 20 or the instructor terminal 30. The communication module 110 is connected to the learner terminal 20 or the instructor terminal 30 on the basis of a protocol stack defined in the communication network, And transmits and receives learning contents, learning data, swimming data, emotional data, sympathetic activity data, and the like using a communication protocol defined in a communication program provided in the learner terminal 20 or the teaching terminal 30 . In the present invention, a wired communication (e.g., a serial communication) or a short-range wireless communication (e.g., Bluetooth, ZigBee, NFC (Near Field Communication), or the like) is performed through a network for data exchange with the learner terminal 20 or the instructor terminal 30. [ (WCDMA), 4G (LTE, LTE-A), 5G, or a wired or wireless IP (Internet Protocol) communication method (E.g., WiBro), a WAP (Wireless Application Protocol), or a WEB.

The member management module 115 is a device for registering and authenticating as a member by using the personalized learning information or the instructor information inputted from the learner terminal 20 or the instructor terminal 30. [ At this time, the customized learning information may include at least one of a gender, an age, a learning level (for example, elementary, middle, high, university, general), a learning subject (e.g., language, science, common sense) Information, notification, employment, certification), and learning time. In addition, the instructor information may include at least one of a gender, an age, a specialty field, a lecture course, and a pool time of the instructor.

The learning contents opening module 120 is a device for opening learning contents selected by the learner terminal 20. The learning contents are implemented as one window on a customized education application installed in the learner terminal 20, The learning data may be input by the learner individual or the learner group, and the pool data inputted from the instructor terminal 30 may be received and displayed.

The emotional index information calculation module 130 is a device for calculating emotional index information by an emotional index calculation algorithm based on emotional data provided from a learner terminal. As shown in FIG. 7, the emotion index calculating algorithm calculates a value corresponding to the primary emotion of the learner and a value corresponding to the changed secondary emotion, respectively, as a predetermined quantization value, Or an average value to an emotion index.

The emotional interval setting module 135 is a device for setting the emotional index information calculated by the emotional index information calculating module 130 according to the sex or age of the learner.

The emotional index information mapping module 140 is a device for mapping emotional words according to intervals according to emotional index information classified by the sensuous section setting module 135. [ The emotional index information mapping module 140 maps the emotional index information having the emotional period set therein to each learner emotional word. That is, as shown in FIG. 7, the emotional index information mapping module 140 maps the learner emotional words a, b, c,..., N into the A section, the B section, the C section, and the N section.

The emotional analysis information generation module 145 generates emotional analysis information including target emotional information or ideal emotional information by an emotional analysis algorithm based on the emotional word for each section mapped by the emotional index information mapping module 140 Device. That is, as shown in FIG. 6, the emotional analysis algorithm may include emotional words (for example, normal, interesting, enjoyment, intensive (class immersion), novelty, etc.) mapped by the emotional index information mapping module 140, Drowsiness, despair, etc.) are categorized into target emotion information with good concentration and emotion emotion information with interest.

The change sensitivity information generation module 150 is a device that generates change sensitivity information based on the sensitivity analysis information of a learner who has been regenerated while providing sympathetic activity data input from the instructor terminal 30. [

The algorithm automatic correction module 155 corrects the emotion analysis algorithm for the learning contents when it is determined by the instructor terminal 30 that the target emotion information and the changed emotion information do not match. The algorithm automatic correction module 155 receives the emotion data from the learner terminal 20 when the abnormal emotion information is generated, recalculates the learner emotion index, and determines whether an error is generated based on the re-calculated learner emotion index And if the error occurs as a result of the determination, the emotion analysis algorithm can be corrected. That is, in the present invention, the emotional analysis algorithm for the learning contents can be automatically corrected according to the emotional change of the learner after the sympathetic act of the instructor due to the occurrence of the learner's sensibility (Abnomal Senitivity, emotional needing the attention of the instructor).

The data providing module 160 provides the target emotion information, the ideal emotion information, or the change emotion information to the learner 30 and provides the pool data, the emotion index information, and the emotion index And provides the learner terminal 20 with the learner activity data corresponding to the learner-specific emotional word mapped by the information mapping module 140. That is, the data providing module 160 can transmit information generated or input to the learner terminal 20 or the instructor terminal 30 so that the individual learner or the learner group and the instructor can share information on the learning contents.

The storage module 165 stores at least one of learning data, pool data, learner information, instructor information, emotional data, emotional index information, emotional word information for each section, emotion analysis information, change emotion information, facial expression recognition algorithm, Such as a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory), and the like, A random access memory (SRAM), a read only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM) , An optical disc, and the like. That is, the storage module 165 can store the emotional change of the learner on the learning contents in progress, the subject, the education data, the vicarious activity data of the instructor, and the like in real time.

The instructor management module 170 is a device that matches and manages the instructor terminal 30 with the learner terminal 20 on the learning contents. The instructor management module 160 is a device that matches and manages instructors on the learning contents with the learner terminal 20. In addition, the instructor management module 160 may store and manage the instructor information supported by the instructor in order to provide the customized education service, and the learning progress status and results of the instructor.

The lock reservation module 175 is a device for reserving a locked state of the learner terminal 20 during a learning process with the learner terminal 20. [ The lock reservation module 165 is a device that reserves a locked state of the learner terminal 20 during a learning process with the learner terminal 20. [ That is, the lock reservation module 165 can maintain the learning contents regardless of whether the learner terminal 20 is in a locked state by fingerprint recognition, facial recognition, pattern recognition, or the like to provide a continuous learning process . In other words, the lock reservation module 165 is designed to provide this customized education service whether before or after the lockout state is released.

The emotional pattern information generation module 180 generates emotional pattern information for individual learners or learner groups using target emotion information and change emotion information. Accordingly, the emotional pattern information generation module 180 provides the emotional pattern information to the instructor terminal 30 through the data providing module 160, and transmits the emotional activity data for the learner individual or the learner group from the instructor terminal 30 And can provide it to each learner terminal 20. That is, as shown in FIGS. 9A and 9B, the emotional pattern information generation module 180 may include a learner to utilize the emotional pattern information generation module 180 as basic data for customized learning of the learner, Analysis of the average emotional patterns of learners about educational types and educational materials (contents) in order to present emotional analysis information and customized education indicators for various subjects such as subject, education type, education materials, And can provide analysis information to the learner terminal 20 or the instructor terminal 30.

Accordingly, the present invention can be utilized as an index for customized education of a teacher, intelligent customized learning of a learner, and establishment of an effective education policy by utilizing stored learner emotion data through real-time learner emotional analysis information.

Further, in the present invention, as shown in FIG. 8, based on the results of real-time analysis on the type of learners' emotion based on the type (by the teacher, the subject, and the student), the professor can establish a reasonable teaching- And the learner can monitor his / her learning attitude, so that he / she can improve his / her concentration on the learning by finding out the recognition pattern and the problem of the learning pattern which is not recognized by himself / herself.

The alarm transmission module 185 is a device for transmitting an alarm signal preset to the learner terminal 20 when abnormality sensation information is generated. In addition, the alarm transmission module 185 can guide the instructor terminal 30 when an abnormality that requires the attention of the instructor during monitoring of the sensibility of the learner occurs. Through this, the instructor can effectively support sympathy activities in response to the learner 's abnormal emotions.

The control module 190 is a device for controlling the operation of the components constituting the education support server. The control module 190 is a device for controlling the operation of the components constituting the education support server. The control module 190 includes learning contents for customized education, Data, learner's emotional data, and sympathetic activity data according to emotional state of the learner are uploaded. In addition, the education support server 10 calculates emotion index information by an emotion index calculation algorithm based on emotion data, maps the emotion word according to intervals of the emotion index information, and stores emotion index information and emotion- Generate and manage learner emotion analysis information by emotion analysis algorithm based on words.

The learner terminal 20 is provided with a customized education application, accesses the education support server 10 by executing a customized education application, inputs customized learning information, registers it as a learning member, requests the learning contents, The emotion data of the learner is calculated based on the photographed image data, the emotion data of the learner is provided in real time through the learning contents, and the pool data and the sympathy activity data are received. The learner terminal 20 is provided with a wired / wireless interface and has an internet browser capable of displaying the contents of the web so as to enable wireless Internet communication. The learner terminal 20 is connected to an external terminal device and is capable of transmitting and receiving images, (PDA), a hand-held personal computer (HPC), a web pad, a notebook, a wireless application protocol (WAP) phone, a palm PC, an e-book A terminal, and a hand held terminal (HHT).

3, the learner terminal 20 includes a first communication unit 210, a first information input unit 220, a first photographing unit 230, an emotional data calculation unit 240, a first storage unit 250, a first display unit 260, an alarm signal output unit 270, and a first control unit 280.

The first communication unit 210 is a device for transmitting and receiving data to and from the education support server 10. The first communication unit 210 connects a predetermined communication channel to the education support server 10 based on a protocol stack defined in the communication network, The learning data, the emotional data, the pool data, and the like using the communication protocol defined in the communication program provided in the personal computer 10. In the present invention, the first communication unit 210 may use a wireless technology such as a Wi-Fi scheme, a ZigBee scheme, a Bluetooth scheme, a 3G scheme, a 4G scheme, an LTE scheme and the equivalent scheme for data transmission / reception with the education support server 10, Technology can be applied.

The first information input unit 220 is a device for inputting customized learning information for accessing the education support server 10 to receive a customized education service, selecting desired learning contents, and inputting learning data on the learning contents. At this time, the customized learning information may include at least one of the sex, age, learning level, learning subject, learning purpose, and learning time of the learner.

The first photographing unit 230 is an apparatus for photographing a learner's image in progress of a learning process, and may be at least one web cam camera. The first photographing unit 230 may photograph an image of the learner in progress on the learning contents and transmit the image to the education support server 10. [ In the present invention, the learner can be made available for presentation through the first photographing unit 230, thereby allowing the learner to participate in the learning process by himself or herself. Also, the attitude of the learner in the course of learning is photographed in real time, Unit 240 to calculate the emotion data of the learner.

The emotional data calculating unit 240 calculates the emotional data of the learner based on the facial expression recognition algorithm that recognizes the facial expression of the learner based on the learner's image data captured by the first photographing unit 230 after the learning contents are opened . The emotional data calculation unit 240 calculates the emotional state of the learner by analyzing the learner's attitude image in real time. For this, the emotional data calculator 240 analyzes a learning attitude image of a learner according to a general facial expression recognition program or a motion recognition program, so that the learner can recognize the facial expressions in the course of the learning process as questions, distractions, anger, surprise, Disgust, despair, sleepiness, and the like are respectively digitized to calculate emotion data. In the present invention, a facial expression recognition program for recognizing facial expressions of a learner matching with a psychological state of a person is described as typical non-contact measurement data for the purpose of calculating emotional data. However, the present invention is not limited to this, Technology or speech recognition processing technology to recognize the video or audio data and use it as emotional data.

The first storage unit 250 stores learning process information to be performed on the learning contents. The first storage unit 250 may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, a RAM, an SRAM, a ROM, an EEPROM, , A magnetic memory, a magnetic disk, or an optical disk. The learning process information may include learning data, pool data, emotional data, sympathetic activity data, emotional pattern information, and the like.

The first display unit 260 is a device for displaying data transmitted to and received from the education support server 10. The first display unit 260 may be an LCD (Liquid Crystal Display), an LED (Liquid Emitting Diode), an OLED ) Method can be used. In addition, the first display unit 260 may be configured to hold the lock state when the learning process is performed by executing the customized education application, and may be set to display only the learning process data while the learning process is in progress. In addition, when the number of learners participating in the learning process is large, the first display unit 260 may be configured to divide the screen according to the number of learners.

The alarm signal output unit 270 receives the notification signal from the education support server 10 and outputs an alarm signal preset to the outside. The alarm signal output unit 270 outputs alarm information to the alarm support module 210 in response to the alarm signal transmitted from the alarm transmission module 185 when abnormality sensibility information is generated by the sensibility analysis information generation module 145 of the education support server 10 It is possible to induce the learner to concentrate by outputting a signal.

The first control unit 280 is a device in which a customized education application is installed and controls the operation of each component constituting the learner terminal 20 by executing a customized education application. That is, the first control unit 280 accesses the education support server 10 by executing the customized education application, inputs the customized learning information, registers it as a learning member, requests the learning contents, and photographs the learner's image data The emotion data of the learner is calculated based on the photographed image data, the emotion data of the learner is provided in real time through the learning contents, and the pool is controlled to receive the data and the sympathy activity data. Meanwhile, the first control unit 280 downloads and stores a customized education application capable of providing customized education services from the education support server 10.

The instructor terminal 30 is a device for providing the education support server 10 with the pool data for the learning data input from the learner terminal 20 and the sympathetic activity data corresponding to the emotional analysis information. That is, the instructor terminal 30 is installed with a customized education application, connects to the education support server 10 by executing a customized education application, inputs pool data about the learning data inputted by the learner terminal 20 , And provides sympathetic activity data corresponding to the sensitivity analysis information. The instructor terminal 30 is equipped with a wired / wireless interface and has an internet browser capable of displaying the contents of the web so as to enable wireless Internet communication. The instructor terminal 30 is connected to an external terminal device and is capable of transmitting and receiving images, A device for driving an application that implements such an operation, and may include personal information terminals such as PDA, HPC, Web pad, notebook, WAP phone, Palm PC, e-Book terminal, HHT as well as a smart phone.

4, the instructor terminal 30 includes a second communication unit 310, a second photographing unit 320, a second information input unit 330, a sympathetic activity data generation unit 330, A data upload unit 350, a second display unit 360, and a second control unit 370.

The second communication unit 310 is a device for transmitting and receiving data to and from the education support server 10. The second communication unit 310 connects a predetermined communication channel with the education support server 10 based on a protocol stack defined in the communication network, The emotion analysis information, the change emotion information, the sympathetic activity data, and the like using the communication protocol defined in the communication program provided in the personal computer 10. In the present invention, wired technologies such as Ethernet, RS-232C, RS-485, TCP / IP, RS-232 and IEEE 1394, IEEE 802.11 WLAN, UWB, Wireless technology such as Wi-Fi, ZigBee, Bluetooth, 3G, 4G, LTE, and the like may be applied.

The second photographing unit 320 is an apparatus for photographing a lecture image of learning data performed by a teacher, and may be at least one web cam camera.

The second information input unit 330 is a device connected to the education support server 10 for inputting instructor information and inputting pool data, emotional analysis information, sympathetic activity data, and the like for the learning data. Here, the instructor information may include at least one of a sex, an age, a specialty field, a lecture course, and a pool time of the instructor. In addition, the second information input unit 330 may receive the recommendation information regarding the matching of the target emotion information and the changed emotion information from the instructor, and the recommendation information about the content substitution.

The sympathetic activity data generation unit 340 generates emotional index information and sympathetic activity data corresponding to the sensible words mapped by the sensibility index information mapping module 140.

The data upload unit 350 edits the lecture image, the pool data, and the sympathetic activity data and provides the edited data to the education support server 10. That is, the data upload unit 350 may construct a digital editing system to edit the lecture image, the pool data, and the sympathetic activity data, and then provide the data to the learner terminal 20 through the education support server 10.

The second display unit 360 displays data transmitted to and received from the education support server 10, and may be an LCD, an LED, an OLED, or an AMOLED system.

The second controller 370 is an apparatus for controlling the operation of each component constituting the instructor terminal 30. The second controller 370 accesses the education support server 10 by executing a customized education application installed in advance, To input the data of the learning data inputted by the user, and to provide the sympathetic activity data corresponding to the sensitivity analysis information. Meanwhile, the second control unit 370 downloads and stores a customized training application capable of providing a customized training service from the training support server 10.

According to the smart learning system based on the learner's emotion based on the above-described embodiment of the present invention, based on a comprehensive learner analysis including the academic level, individual learning ability, interest courses, learning attitude and inclination, Selection, and educational process. In addition, an embodiment of the present invention can grasp the real-time emotional state of a learner with respect to a class in progress, take immediate action upon detection of an ideal emotional state, and allow the algorithm to evolve automatically according to the learner's attainment of the target emotional state. In addition, an embodiment of the present invention enables rapid measures so that the teacher can check the emotional state of the ever-changing learner while the class is in progress, adjust the education level, or improve the concentration of the class. In addition, one embodiment of the present invention can provide a data analysis technique capable of converting emotion data collected and analyzed by various biometric information for emotional analysis of a learner into learning emotion by subdividing the emotion data so as to be specialized for a learner . In addition, one embodiment of the present invention enables mutual-conscious type customized education through monitoring of the learner's real-time emotional state, and analyzes cumulative learner emotional data to implement intelligent customized learning, establish an effective education policy, This will provide a basis for the innovative transformation of the national education paradigm for cultivating global creative talents.

As described above, the smart education system based on the learner's emotion based on the present invention is only one embodiment, and the present invention is not limited to the above-described embodiment, It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention.

10: education support server 20: learner terminal
30: Instructor terminal 110: Communication module
115: Member management module 120: Learning contents opening module
130: emotion index information calculating module 135: emotion interval setting module
140: Emotion index information mapping module 145: Sensibility analysis information generation module
150: Change sensitivity information generation module 155: Algorithm automatic correction module
160: Data providing module 165: Storage module
170: Instructor management module 175: Lock reservation module
180: Sensitivity pattern information generation module 185: Alarm transmission module
190: control module 210: first communication section
220: first information inputting section 230: first photographing section
240: emotion data calculation unit 250: first storage unit
260: first display unit 270: alarm signal output unit
280: first control unit 310: second communication unit
320: second photographing unit 330: second information input unit
340: sympathetic activity data generation unit 350: data upload unit
360: second display unit 370: second control unit

Claims (14)

Wherein learning data for a personalized education is provided by a learner or a instructor on the learning contents, emotional data of the learner, and sympathetic activity data according to the emotional state of the learner are inputted, The emotion index information is calculated by the emotion index calculation algorithm based on the emotion index information, and then the emotion word of each emotion index information is mapped to the emotion index information. Based on the emotion index information and the mapped emotion word, An education support server for generating and managing emotional analysis information of the user;
A personalized education application is installed, and the personalized learning information is input by accessing the education support server by execution of the customized education application, and then registered as a learning member, then requested learning contents, photographing the learner's image data, A learner terminal for calculating emotional data of the learner based on the image data and providing the learning data and emotional data of the learner in real time through the learning contents and receiving the pool data and the sympathetic activity data; And
A personalized education application is installed, and by accessing the education support server by executing the customized education application, the pool data for the learning data input by the learner terminal is input, and the volunteer activity data corresponding to the emotional analysis information A smart training system based on the learner's emotions.
The method according to claim 1,
Wherein the emotional data is a plurality of facial expression data calculated by a facial expression recognition algorithm for recognizing facial expressions of the learner.
The method of claim 2,
Wherein the emotional index calculation algorithm quantifies the change of the facial expression data at a preset numerical value.
The method according to claim 1,
Wherein the emotional analysis algorithm calculates a learner emotion index according to the change of the emotional data, sets the calculated learner emotion index according to the sex or age of the learner, and then maps a preset emotion word for each emotion interval And the sensory analysis information is generated based on the sensory information.
The method of claim 4,
Wherein the emotion analysis information includes target emotion information and ideal emotion information,
Wherein the education support server provides target emotion information or ideal emotion information for the learner to the instructor terminal.
The method of claim 5,
Wherein the instructor terminal compares the target emotion information or the ideal emotion information of the received learner with the emotion sensitivity information of the learner provided with the sympathy activity data and provides the comparison result to the education support server Smart learning system based on learners' emotions.
The method according to claim 1,
The emotional analysis information includes learning capability and inclination analysis information, learner interest information for each subject, level of understanding level for each subject and detail, learning concentration information for each time period, information on recommended learning methods for each learner and subject, A content level according to the degree of difficulty, and an adequacy indicator index information.
The method according to claim 1,
The education support server
A communication module for transmitting / receiving data to / from the learner terminal or the instructor terminal;
A member management module for registering and authenticating as a member using the learner information or the instructor information input from the learner terminal or the instructor terminal;
A learning contents opening module for opening the learning contents at the request of the learner terminal;
An emotion index information calculating module for calculating emotion index information by an emotion index calculating algorithm based on emotion data provided from the learner terminal;
A sensibility section setting module for setting the sensibility index information according to sex or age of the learner;
An emotional index information mapping module for mapping emotional words according to intervals according to the classified emotional index information;
An emotional analysis information generation module that generates emotional analysis information including target emotion information or ideal emotion information by an emotion analysis algorithm based on the mapped emotional word for each section;
A change emotion information generation module that generates change emotion information based on the emotion analysis information of the regenerated learner while providing the volunteer activity data;
An algorithm automatic correction module that corrects the emotion analysis algorithm when the target emotion information does not match the change emotion information;
Providing the instructor terminal with the target emotion information, the ideal emotion information, or the change emotion information for the learner, and generating the pool data, the emotion index information, and the sympathetic activity corresponding to the mapped word of the mapped section A data providing module for providing data to the learner terminal;
A storage module for storing at least one of the learning data, pool data, learner information, instructor information, emotional data, emotional index information, emotional word information for each section, emotional analysis information, change emotional information, facial expression recognition algorithm, and emotional analysis algorithm;
A instructor management module for matching and managing the instructor terminal with the learner terminal on the learning contents;
A lock reservation module for reserving a lock status of the learner terminal during a learning process with the learner terminal; And
And a control module for controlling the operation of the components constituting the education support server.
The method of claim 8,
The education support server
And a sensitivity pattern information generation module that generates sensitivity pattern information for each learner individual or group of learners using the target emotion information and the change sensitivity information,
And provides the emotion pattern information to the instructor terminal through the data providing module and receives the sympathy activity data for each learner individual or the learner group from the instructor terminal and provides the data to each learner terminal. Smart education system.
The method of claim 8,
The education support server
Further comprising an alarm transmission module for transmitting an alarm signal previously set to the learner terminal when the abnormal sensation information is generated.
The method of claim 8,
Wherein the automatic algorithm correction module re-calculates the learner emotion index by receiving emotional data from the learner terminal when the abnormal emotion information is generated, and determines whether or not an error is generated based on the re-calculated learner emotion index Wherein the emotional analysis algorithm is calibrated when an error occurs as a result of the determination.
The method according to claim 1,
The learner terminal
A first communication unit for transmitting / receiving data to / from the education support server;
Inputting the learner information and the customized learning information for accessing the education support server to receive the customized education service, inputting the request information for requesting the opening of the learning channel, inputting the learning data and the image data on the learning contents A first information input unit for inputting information;
A first photographing unit for photographing a video of a learner in the course of learning;
A emotional data calculation unit for calculating the emotional data of the learner based on a facial expression recognition algorithm for recognizing the facial expression of the learner based on the image data of the taken learner after the learning contents are opened;
A first storage unit for storing learning process information performed on the learning content;
A first display unit for displaying data transmitted to and received from the education support server, wherein the first display unit displays only data of the learning process while the learning process is in progress,
An alarm signal output unit receiving an alarm signal from the education support server and outputting an alarm signal preset to the outside; And
And a first controller installed with a customized education application and controlling the operation of each component constituting the learner terminal by executing the customized education application.
The method according to claim 1,
The instructor terminal
A second communication unit for transmitting / receiving data to / from the education support server;
A second photographing unit for photographing a lecture image of learning data performed by the instructor;
A second information input unit connected to the education support server, for inputting the instructor information, and for inputting pool data for the lecture image;
A sympathetic activity data generation unit for generating the sympathetic activity data corresponding to the emotion index information and the mapped word for each of the mapped sections;
A data upload unit for editing the lecture video, pool data, and sympathetic activity data and providing the same to the education support server;
A second display unit for displaying data transmitted to and received from the education support server; And
And a second controller for controlling an operation of each component constituting the instructor terminal.
14. The method of claim 13,
The sympathetic activity data generation unit
Compares the emotion analysis information with preset change sensitivity information, generates sympathetic activity data according to the comparison result,
Wherein the second control unit notifies the education support server and corrects the emotion analysis algorithm when the emotional analysis information and the changed emotion information do not match, or recommends an alternative content to replace the learning content Smart learning system based on learners' emotions.
KR1020150080133A 2015-06-05 2015-06-05 Smart education system based on learner emotion KR101715323B1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
KR1020150080133A KR101715323B1 (en) 2015-06-05 2015-06-05 Smart education system based on learner emotion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
KR1020150080133A KR101715323B1 (en) 2015-06-05 2015-06-05 Smart education system based on learner emotion

Publications (2)

Publication Number Publication Date
KR20160143422A true KR20160143422A (en) 2016-12-14
KR101715323B1 KR101715323B1 (en) 2017-03-22

Family

ID=57575888

Family Applications (1)

Application Number Title Priority Date Filing Date
KR1020150080133A KR101715323B1 (en) 2015-06-05 2015-06-05 Smart education system based on learner emotion

Country Status (1)

Country Link
KR (1) KR101715323B1 (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180085449A (en) 2017-01-19 2018-07-27 김현옥 Learning management method and system based on question and answer
KR20190036769A (en) * 2017-09-28 2019-04-05 오종현 The analyzing system of lecture feedback
US20190311371A1 (en) * 2016-12-20 2019-10-10 Sony Corporation Information processing device and information processing method
CN112084814A (en) * 2019-06-12 2020-12-15 广东小天才科技有限公司 Learning auxiliary method and intelligent device
KR102232344B1 (en) * 2020-02-27 2021-03-26 주식회사 아이스크림에듀 personalized and adaptive learning method based on artificial intelligence (AI) using big data, and system
KR102393246B1 (en) * 2021-08-26 2022-05-02 김동혁 Methods and devices using communication figures
CN116645252A (en) * 2023-07-21 2023-08-25 广州市润心教育咨询有限公司 Artificial intelligence education system using big data

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102287404B1 (en) * 2019-08-23 2021-08-10 주식회사 로지브라더스 Block coding education intervention metho using camera and system therefore

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006023506A (en) * 2004-07-07 2006-01-26 Tokai Univ Electronic teaching material learning support device, electronic teaching material learning support system, electronic teaching material learning support method, and electronic learning support program
KR20100005863A (en) * 2008-07-08 2010-01-18 주식회사 글맥학원 Desk type apparatus for studying and method for studying using it
JP2011007963A (en) * 2009-06-24 2011-01-13 Tokyo Denki Univ Remote learning system and remote learning method
KR101289870B1 (en) 2012-08-27 2013-07-26 권장환 Smart class progress system using smart terminal
KR20140147243A (en) 2013-06-19 2014-12-30 김인영 System for Study based on User Participation and Method thereof
KR20150007936A (en) * 2013-07-11 2015-01-21 삼성전자주식회사 Systems and Method for Obtaining User Feedback to Media Content, and Computer-readable Recording Medium

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006023506A (en) * 2004-07-07 2006-01-26 Tokai Univ Electronic teaching material learning support device, electronic teaching material learning support system, electronic teaching material learning support method, and electronic learning support program
KR20100005863A (en) * 2008-07-08 2010-01-18 주식회사 글맥학원 Desk type apparatus for studying and method for studying using it
JP2011007963A (en) * 2009-06-24 2011-01-13 Tokyo Denki Univ Remote learning system and remote learning method
KR101289870B1 (en) 2012-08-27 2013-07-26 권장환 Smart class progress system using smart terminal
KR20140147243A (en) 2013-06-19 2014-12-30 김인영 System for Study based on User Participation and Method thereof
KR20150007936A (en) * 2013-07-11 2015-01-21 삼성전자주식회사 Systems and Method for Obtaining User Feedback to Media Content, and Computer-readable Recording Medium

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190311371A1 (en) * 2016-12-20 2019-10-10 Sony Corporation Information processing device and information processing method
KR20180085449A (en) 2017-01-19 2018-07-27 김현옥 Learning management method and system based on question and answer
KR20190036769A (en) * 2017-09-28 2019-04-05 오종현 The analyzing system of lecture feedback
CN112084814A (en) * 2019-06-12 2020-12-15 广东小天才科技有限公司 Learning auxiliary method and intelligent device
CN112084814B (en) * 2019-06-12 2024-02-23 广东小天才科技有限公司 Learning assisting method and intelligent device
KR102232344B1 (en) * 2020-02-27 2021-03-26 주식회사 아이스크림에듀 personalized and adaptive learning method based on artificial intelligence (AI) using big data, and system
KR102393246B1 (en) * 2021-08-26 2022-05-02 김동혁 Methods and devices using communication figures
CN116645252A (en) * 2023-07-21 2023-08-25 广州市润心教育咨询有限公司 Artificial intelligence education system using big data

Also Published As

Publication number Publication date
KR101715323B1 (en) 2017-03-22

Similar Documents

Publication Publication Date Title
KR101715323B1 (en) Smart education system based on learner emotion
Järvenoja et al. Understanding regulated learning in situative and contextual frameworks
US20190087831A1 (en) Generating digital credentials based on sensor feedback data
Fastré et al. Towards an integrated model for developing sustainable assessment skills
CN108711320B (en) Immersive online education system and method based on network
Khalfallah et al. Facial expression recognition for intelligent tutoring systems in remote laboratories platform
Radosavljevic et al. Ambient intelligence-based smart classroom model
US20190385471A1 (en) Assessment-based assignment of remediation and enhancement activities
US20220406207A1 (en) Systems and methods for objective-based skill training
US20180253989A1 (en) System and methods that facilitate competency assessment and affinity matching
US20220415199A1 (en) Smart-learning and knowledge concept graphs
US20220415200A1 (en) Smart-learning and learning path
Pramesworo et al. Identification of New Approaches to Information Technology-Based Teaching for Successful Teaching of Millennial Generation Entering 21st Century Education
Renawi et al. A simplified real-time camera-based attention assessment system for classrooms: pilot study
Vihervaara et al. Internet of things: Opportunities for vocational education and training-presentation of the pilot project
Webster et al. Leading school communities to implement a sustainable school-wide model leading to enhancing learning outcomes for students with ASD
Cabada et al. Intelligent tutoring system with affective learning for mathematics
WO2017021760A1 (en) Smart teaching system for a classroom
KR101650863B1 (en) Mobile education system using companion invitation
Mershad et al. Using Internet of Things for automatic student assessment during laboratory experiments
López et al. EMO-Learning: Towards an intelligent tutoring system to assess online students’ emotions
Fesol et al. Framework for enhancing learning experience with wearable technology in technical MOOC
Mutahi et al. Seamless blended learning using the cognitive learning companion: A systemic view
Shoukry et al. ClasScorer: Towards a Gamified Smart Classroom
US20220138881A1 (en) Systems and methods for skill development monitoring and feedback

Legal Events

Date Code Title Description
A201 Request for examination
E902 Notification of reason for refusal
E701 Decision to grant or registration of patent right
GRNT Written decision to grant