KR20230053553A - Cathexis learning system and method using AI in an untact learning based on Web browser - Google Patents

Abstract

Disclosed are a cathexis learning system and a method using artificial intelligence (AI) in non-contact learning based on a web browser. The cathexis learning system in non-contact learning based on a web browser according to the present invention comprises: a learning content server providing non-face-to-face lecture learning content; and a user terminal accessing the learning content server to run an educational content viewer program, having a facial recognition module that recognizes the facial outline of the front camera image data of the user terminal and facial feature points of two eyes, one nose, and two ears and a voice recognition module in the non-face-to-face online learning, and including an educational content viewer installed to use AI facial recognition/gesture recognition and sound reproduction technology to allow learning content to be learned in cathexis with artificial intelligence. In non-face-to-face online learning, the educational content viewer on the user terminal has a media playback unit and a facial outline recognition module, issues an alarm/warning message to the user terminal using sound and messaging technology when a gaze is found to deviate at a certain angle or a distance in the corresponding direction of the eye-to-ear distance or nose-to-ear distance is found to exceed a certain reference by facial recognition of the face outline and the facial features of two eyes, one nose, and two ears on a 5-point scale in order to enable a student to pay attention to non-face-to-face lectures focused on the camera of the user terminal linked to the learning content server (ubcloud server) and perform cathexis learning, and provides cathexis learning using web browser-based ubcloud artificial intelligence in the non-face-to-face online learning.

Description

{Cathexis learning system and method using AI in an untact learning based on Web browser}

The present invention relates to an attentional learning system and method using artificial intelligence in non-face-to-face learning (ubcloud), and more particularly, in online lecture non-face-to-face learning, an educational content viewer of a user terminal is a media playback unit and a facial contour recognition unit module. , and five facial feature points of eyes 2/nose/ear 2 so that attention is focused on the learning content focused on the front camera of the user terminal linked to the learning content server (ubcloud server). Through face recognition that recognizes the scale, when the gaze is deviating at a certain angle, or when the distance in the corresponding direction of the eye-to-ear distance or the nose-to-ear distance exceeds a certain standard value, the user uses sound and messaging technology to focus attention and learn. System and method for attention learning using artificial intelligence in web browser-based non-face-to-face learning (ubcloud) that generates alarm/warning messages to terminals and provides attention-focused learning using web browser-based ubcloud artificial intelligence in non-face-to-face learning It is about.

As face recognition technology, an appearance based matching method and feature based face recognition are mainly used. Face recognition is differently recognized according to a photographing angle of a camera, a direction of lighting, a pose, a change in expression, and a change in face over time.

For feature-based face recognition, a detection method using Haar-like features and a detection method using MCT (Modified Census Transform) images are used for image data captured by a digital camera, IoT device camera, or smartphone camera. . Using the face and eye detectors learned as Haar-like features from the input image of the smartphone camera, the contours of the face and the forehead/eyes/nose/mouth are detected, and the region of interest (ROI, Region of Interest) is detected to detect the circular pupil. The eye area set as Interest) is converted to grayscale, and the histogram of the eye image [pixel value of each pixel in the x-axis, The number of pixel values corresponding to the y-axis] is obtained, and after binarization of the eye image, pre-processing of the eye area photo is performed through histogram equalization, and contours, eyebrows, eyes, nose, The facial features of the mouth are detected by face data, texture features and shape features are extracted, and the face is recognized by comparing the similarity with the feature points of the face photos stored in the face recognition DB.

The feature values of the eyebrows, eyes, nose, mouth, and chin of the face area are expressed as the difference between the sum of pixels included in the white area of the Haar-like feature and the sum of pixels included in the black area.

For example, the distance from the eye area detected in a standard horizontal and vertical face area picture to the endpoints of the right and left eyes, the size of the iris extracted using a hough circle transform algorithm value is used as the feature value.

1A is a configuration diagram of an existing face recognition device.

The face recognition device 100 includes an image display device, an image recording device, a face recognition server, a tablet PC, a laptop (Laptop), a personal PC, a smartphone, a personal digital assistant (PDA), a mobile communication terminal, and It may be any one of an intelligent robot and the like.

The face recognition apparatus 100 includes an input image acquisition unit 112 acquiring an input image from a camera; In order to generate a front pose image by detecting a face region in the input image and normalizing a face pose, and removing perspective distortion according to the distance between the camera and the subject, the perspective of the front pose image is measured. a normalization unit 114 for normalizing ) to generate a normalized image; a feature vector extractor 116 extracting a feature vector representing the subject's face from the normalized image; and a face recognition unit 118 recognizing the face of the subject included in the input image by applying the feature vector to a pre-learned classification model.

The input image acquisition unit 112 acquires an input image from a camera. The camera may be a depth-sensing camera, a stereo camera, and a color camera (eg, a Kinect camera, etc.). In addition, the input image is an image including a face of a subject to be recognized, and includes a 2D still image and a moving image. The input image may include a color image, a depth image, and a color-depth (RGB-D) image.

The normalization unit 114 detects a face region from an input image and normalizes a face pose and perspective to generate a normalized image. When there is a change in the facial pose, the face recognition rate is degraded because the gray scale, shape, and positions of feature points of the face are changed. In addition, if the distance between the camera and the subject is different, perspective distortion (distortion) occurs differently for each photographed position even for the same subject, so it may seem as if a different subject has been photographed. Therefore, it is necessary to normalize the face pose and perspective of the input image in order to improve the face recognition rate.

The normalization unit 114 inputs face images for learning in various poses to the input layer of the first artificial neural network, and trains the first artificial neural network so that face images for learning in front poses are output from the output layer of the first artificial neural network. face pose normalization learning unit; And inputting the data output from the output layer of the first artificial neural network to the input layer of the second artificial neural network, and learning the second artificial neural network so that a face image for learning without perspective distortion is output from the output layer of the second artificial neural network. Includes a perspective normalization learning unit.

The normalization unit inputs face images for learning in various poses with various perspective distortions to the input layer of the integrated artificial neural network in which the first artificial neural network and the second artificial neural network are integrated, and the front pose is free from perspective distortion. The integrated artificial neural network is trained so that a facial image for learning is output from an output layer of the integrated artificial neural network.

The feature vector extractor 116 is determined by machine learning and extracts a feature vector representing a face of a subject from a normalized image.

A feature vector is a vector having feature values used for face recognition as elements. Filters used to extract feature vectors include Gabor filter, Haar filter, LBP (Local Binary Pattern) - including DLBP (Discriminative LBP), ULBP (Uniform LBP), NLBP (Number LBP), etc. - but must be limited to these and other filters may be used.

The face recognition unit 118 recognizes the face of a subject included in the input image by applying the feature vector extracted by the feature vector extraction unit 116 based on the pre-learned classification model. The pre-learned classification model may include, but is not necessarily limited to, a Support Vector Machine (SVM), a Linear Discriminant Analysis (LDA), and Softmax.

The virtual face image generation unit 124 may generate a plurality of virtual face images used for learning by the normalization unit 114 , the feature vector extraction unit 116 , and the face recognition unit 118 .

The plurality of virtual face images refer to face images generated by transforming a 3D face model synthesized by the virtual face image generator 124 using one or more 2D reference images obtained from a camera.

As prior art 1 related to this, Patent Registration No. 10-2103521 has registered “a video object recognition system and method based on artificial intelligence deep learning”.

1B is a block diagram of a conventional video object recognition system based on artificial intelligence deep learning. Various images are pre-learned through deep learning of artificial intelligence, and images are analyzed frame-by-frame for distributed video products by reflecting the above-mentioned pre-learning results, and in some cases, audio information of video materials is also analyzed in chronological order. After deriving keywords, we provide a video recognition system and method based on artificial intelligence deep learning that compares them with the keywords accumulated in advance with the time-series keywords of video scripts.

A system implemented by artificial intelligence includes a dataset learning unit for deep learning using a plurality of object images as keywords; a DB unit for extracting keywords from video scripts, chronologically arranging them, and storing them; a video object analysis unit that extracts objects on the image and the relationship between the objects as keywords through frame-by-frame image analysis of the video object, and sequentially arranges them; and a comparison determination unit that compares keywords of the DB unit and the video analysis unit to determine similarity.

However, existing e-learning or U-learning systems simply provided online learning content, and when learners carelessly lose their gaze by using face recognition technology and sound and messaging technology, an alarm is generated to the user terminal and attention learning is performed. It didn't provide a function to do so.

Patent Registration No. 10-2103521 (Registration Date: April 16, 2020), "Video object recognition system and method based on artificial intelligence deep learning", Sangmyung University Industry-University Cooperation Foundation

An object of the present invention to solve the above problem is that in online lecture non-face-to-face learning, the educational content viewer of the user terminal has a media playback unit and a facial contour recognizing unit module, and the user terminal linked to the learning content server (ubcloud server). Directly look at the learning content focused on the front camera and comb the gaze at a certain angle through face recognition on a 5-point scale of face outline and eye 2/nose/ear 2 facial feature points so that non-face-to-face online lecture learning content can be focused. When going out, or when the eye-to-ear distance or nose-to-ear distance exceeds a certain standard value, an alarm/warning message is generated using sound and messaging technology, and in non-face-to-face learning, using ubcloud artificial intelligence based on a web browser Provides an attention learning system using artificial intelligence in non-face-to-face learning based on a web browser that provides attention learning.

Another object of the present invention is to provide an attention learning method using artificial intelligence in web browser-based non-face-to-face learning.

In order to achieve the object of the present invention, an attention learning system using artificial intelligence in web browser-based non-face-to-face learning includes a learning content server providing non-face-to-face lecture learning content; And accessing the learning content server through a wired/wireless communication network to drive an educational content viewer program, recognizing the contours of the face of the front camera image data of the user terminal and the facial feature points of the eyes 2 / nose / ears 2 in non-face-to-face learning It has a face recognition module and a voice recognition module, and includes a user terminal with an educational content viewer installed so that artificial intelligence attention-focused learning of learning content using AI face recognition / motion recognition sound reproduction technology is installed,

The facial recognition module uses the posenet algorithm as an AI-based facial contour recognition technology,

The learning content server or the user terminal analyzes the eye2/nose/ear2 face feature points of the online learner's face image detected by the camera in the user terminal of each learner, and when the gaze is off at a certain angle or the eye-ear distance If the distance in the corresponding direction of the nose-to-ear distance exceeds a certain standard value, a warning message or an alarm is generated to the corresponding user terminal so that artificial intelligence attention-focused learning occurs.

In order to achieve another object of the present invention, the attention learning method in web browser-based non-face-to-face learning includes: (a) registering and storing learner information and a standard-sized frontal face picture in a learning content server; (b) issuing, by the learning content server, a QR code corresponding to user information and a front face picture; (c) In a user terminal equipped with an educational content viewer program and a face recognition module, the recognition result of the front face photo of the camera is received by the learning content server using the front face recognition algorithm of the front camera image, and the previously registered learner information and comparing the photograph and its facial features to confirm whether the learner is himself or not; (d) providing learning content from the learning content server to a user terminal of a pre-registered member; and (e) when the learning content server or the user terminal analyzes eye2/nose/ear2 face feature points of the online learner's face image detected by the camera in each learner's user terminal, and the line of sight deviates at a certain angle; or and generating a warning message or an alarm to a corresponding user terminal so that artificial intelligence attention-focused learning is performed when the eye-to-ear distance or nose-to-ear distance in the corresponding direction exceeds a predetermined reference value.

In the web browser-based non-face-to-face learning system and method of the present invention, in the non-face-to-face learning, the educational content viewer of the user terminal is provided with a media playback unit and a facial contour recognition module, and a learning content server ( ubcloud server) to direct the eyes to the non-face-to-face lecture learning contents focused on the front camera of the user terminal linked to the user terminal, and to provide attention-intensive education. If the line of sight deviates at an angle or if the eye-to-ear distance or the nose-to-ear distance exceeds a certain standard value, an alarm/warning message is generated using sound and messaging technology, and a web browser-based non-face-to-face learning There is an effect of providing attention-focused learning using ubcloud artificial intelligence.

1A is a configuration diagram of an existing face recognition device.
1B is a block diagram of a conventional video object recognition system based on artificial intelligence deep learning.
2a and 2b are implementation examples of an attention-focused learning system using web browser-based ubcloud artificial intelligence in non-face-to-face online learning.
3A is a diagram showing the concept of a facial contour recognition artificial intelligence platform having a tablet PC, a smart phone, and a PC-based learning content server during online learning.
3B is a diagram showing a face recognition function on the ubiquitous-based NSDAI platform.
3c and 3d show the goals and User Experience (1. User registration -> 2. Learning -> 3. QR Code generation -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor confirmation and test -> 6.7 UBT App/Web camera facial contour recognition 8. End of test) It is a picture.
4 shows the process from learner/test taker registration to test taker registration, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) supervisor confirmation (applicant face/test taker information), 6) Taking the test - This is a picture showing the process.
5 is a screen for registering/learning/QR code issuing a candidate-face matching code issuing unit-at the time of an online test/UBT test based on machine learning and learning results of a registered candidate's face.
6 is a test taker identification unit screen in the UBT system that measures the distance and similarity of facial feature points of a standard-sized frontal face picture based on the vertex of the nose through AI-based facial contour recognition of the test taker's face.
7 is a rule user definition unit and a program test screen when determining a learner.
8 is a learning content server access/login/left-QR code recognition/recognition of QR code during non-face-to-face online learning (TTS)/right-face recognition (smartphone/tablet PC front camera)/face in the background This is the demonstration screen including the process of starting area recognition/recognition rate display - the UBT authentication process user definition part using the generated user code.
9 is a block diagram of an attentional learning system using artificial intelligence in tablet PC, smartphone, and PC-based web browser-based non-face-to-face learning using facial contour recognition artificial intelligence in web browser-based non-face-to-face learning according to the present invention.
10 is a diagram explaining the functions of an AI-based facial contour recognition module during online learning and testing.
11 is a flowchart showing a tablet PC, smartphone, and PC-based attention learning method using facial contour recognition artificial intelligence in web browser-based non-face-to-face learning according to the present invention.

Hereinafter, the configuration and operation of a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the description of the present invention, if it is determined that a detailed description of a related known technology or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, the attached drawing numbers refer to the same drawing numbers in other drawings when indicating the same configuration.

In the web browser-based non-face-to-face learning system and method of the present invention, an attention-focused learning system and method using artificial intelligence in non-face-to-face online learning includes an educational content viewer of a user terminal equipped with a media playback unit and a facial contour recognition module, and a learning content server. Through facial recognition on a 5-point scale, facial contours and eye 2/nose/ear 2 facial feature points allow direct attention to the non-face-to-face lecture learning content focused on the front camera of the user terminal linked to the (ubcloud server). If the user's gaze is diverted from the user's terminal at a certain angle, an alarm or warning message is generated to the user's terminal using sound and messaging technology, and attentional learning using web browser-based ubcloud artificial intelligence is provided in non-face-to-face online learning.

* Same facial features and contour recognition technology (ubcloud)

* Added learning content server and exam server function

* Browser-based online (learning data is read online and stored online)/offline (library browser built-in and web page built-in, and recognition data is stored on the user's device) can be used.

* Attention learning using sound and messaging

The attention learning system and method using artificial intelligence in web browser-based non-face-to-face learning of the present invention is an educational content viewer in which a user terminal (tablet PC, smartphone, PC) is linked with a learning content server during non-face-to-face online learning. is installed, using AI face recognition / motion recognition / sound and messaging technology, using AI-based face recognition (posenet algorithm, machine learning model which allows for real-time face pose estimation) technology using the front camera of the user terminal A face recognition module is installed that recognizes the contours of the face and the feature points of the eyes 2/nose/ear 2 on a 5-point scale. When recognizing a face, it learns a face picture taken using the front camera (C) of the test taker terminal. Comparing the feature points of the learner's face photo in the face photo DB of the content server to detect whether or not the learner is present, and in non-face-to-face online learning, AI-based face recognition is performed in real time using the front camera image data of the user terminal to recognize the learner's By detecting the behavioral pattern of the face in the video, the user terminal's front camera linked to the learning content server (ubcloud server) focuses on the non-face-to-face lecture learning content to directly look at the eyes and focus on the outline of the face and eyes 2/nose/ Ear 2 facial feature point 5-point scale If the line of sight deviates at a certain angle through face recognition, or if the eye-to-ear distance or the nose/to-ear distance in the corresponding direction exceeds a certain standard value, an alarm/warning message is sent to the user terminal. and uses sound and messaging technology to enable learners to focus their attention.

User terminals include not only smart phones and tablet PCs that receive online learning contents from the learning contents server through wired/wireless communication networks, but also laptops capable of accessing the Internet.

The user terminal is installed with an educational content viewer linked to the learning content server, and using AI face recognition/motion recognition/sound and messaging technology, the outline of the learner's face and the facial feature points of eyes 2/nose/ear 2 5-point scale feature points It has a face recognition module that recognizes them.

2a and 2b are implementation examples of an attention-focused learning system using web browser-based ubcloud artificial intelligence in non-face-to-face online learning.

3A is a diagram showing the concept of a facial contour recognition artificial intelligence platform having a tablet PC, a smart phone, and a PC-based learning content server during online learning.

3B is a diagram showing a face recognition function on the ubiquitous-based NSDAI platform.

3c and 3d show the goals and User Experience (1. User registration -> 2. Learning -> 3. QR Code generation -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor confirmation and test -> 6.7. Facial contour recognition of face by UBT App/Web camera 8. End of test) this is the picture shown

4 shows the process from learner/test taker registration to test taker registration, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) supervisor confirmation (applicant face/test taker information), 6) Taking the test - This is a picture showing the process.

5 is a screen for registering/learning/QR code issuing a candidate-face matching code issuing unit-at the time of an online test/UBT test based on machine learning and learning results of a registered candidate's face.

6 is a test taker identification unit screen in the UBT system that measures the distance and similarity of facial feature points of a standard-sized frontal face picture based on the vertex of the nose through AI-based facial contour recognition of the test taker's face.

7 is a rule user definition unit and a program test screen when determining a learner.

8 is a learning content server access/login/left-QR code recognition/recognition of QR code during non-face-to-face online learning (TTS)/right-face recognition (smartphone/tablet PC front camera)/face in the background This is the demonstration screen including the process of starting area recognition/recognition rate display - the UBT authentication process user definition part using the generated user code.

(1) The educational content viewer is installed on the user terminal, and the educational content viewer uses artificial intelligence facial recognition to recognize the outline of the learner's face image and the facial feature points of eyes 2/nose/ear 2 on a 5-point scale Facial recognition module has been installed.

QR codes were used as recognition codes for online learning and testing.

In the embodiment, the user terminal used a tablet PC and a QR code.

The artificial intelligence learning the registration photo of the learner/test taker generates a user-specific recognition code (QR code), and the test taker holds the examination ticket with the corresponding recognition code (QR code) attached to the camera of the test site PC or tablet PC, The user-recognized AI learns the candidate information and front face photo (standard-size front face photo with the nose as the vertex) in advance learning the learner/applicant information registered in the face DB of the content server and the standard-size front face photo. As a result, the model is compared, and the distance and similarity of the feature points of the face are calculated, and when the distance exceeds a certain value, it is recognized as the person, moves to the learning screen, and the learner learns non-face-to-face online. In case the similarity of the facial feature points does not reach a certain level (completely insufficient, pending judgment), the information is transmitted to the supervisor terminal linked to the server, and the supervisor conducts online learning after confirming the actual face and personal information.

<System configuration>

1) Non-face-to-face online learning: test taker terminal (PC/smartphone/tablet) user program -> wired/wireless communication network (WAN, LTE 4G/5G) and internal network (LAN, Wi-Fi communication department) -> learning content server (learning curriculum information/learner information), the supervisor terminal determines whether or not to participate in learning by transmitting the face picture and learner information of each user terminal.

2) When using offline: The AI face recognition module is installed in the user program (PC/Smartphone/Tablet PC), and only the final facial recognition result is transmitted to the learning content server after recognizing the facial contour of the candidate's face.

<Partial APK of the app prototype of the learner's device - Attachment for Android>

https://we.tl/t-wFdXexsors File Download Password: nsdevil

Install the app on your Android smartphone or tablet PC

<web version function address>

1) Face Recognition Module: https://facere.nsdai.org (id: nsdevil, passwd: nsdevil) > Take a picture of the QR code with your smartphone and place it on the recognition screen on the PC camera to activate.

2) 5-point facial contour recognition module: https://headpos.ublrandd.com.np (id: nsdevil, passwd: nsdevil), 5-point scale of the outline of the learner's face and eyes 2/nose/ear 2 facial features facial recognition

3) Voice recognition module

Speech Recognition is a man-machine interface technology that removes noise from an acoustic speech signal input by a microphone, extracts features of the speech signal, and converts it into a set of words or text of a sentence (mapping). ) process, microphone -> AMP -> LPF -> ADC -> saved in the voice database.

The speech recognition module uses a method using vector quantization, a method using dynamic time warping (DTW), a method using a neural network, and a method using HMM (Hidden Markov Model). method is used The voice recognition module uses a voice recognition module of hardware and software and a voice recognition API of STT (Speech To Text) technology.

9 is a block diagram of an attentional learning system using artificial intelligence in tablet PC, smartphone, and PC-based web browser-based non-face-to-face learning using facial contour recognition artificial intelligence in web browser-based non-face-to-face learning according to the present invention.

10 is a diagram explaining the functions of an AI-based facial contour recognition module during online learning and testing.

The learning content server 100 having a test management unit includes a WWW server 101, a control unit 102, a member registration unit 103, a user authentication unit 104, a learning content providing unit 105, a QR code management unit 106 , supervisor identification unit, face recognition unit 107, examination management unit 109, examination information DB 120, candidate DB 121, and face photo DB 123.

In the web browser-based non-face-to-face learning of the present invention, the attention learning system

A learning content server 100 that provides non-face-to-face lecture learning content authored by an authoring tool; and

The learning content server 100 is accessed through a wired/wireless communication network to drive an educational content viewer program, and in non-face-to-face online learning, the facial contours and eye 2/nose/ear 2 face feature points of the front camera image data of the user terminal are displayed. It includes a user terminal (300, 310, 311) equipped with an educational content viewer so that artificial intelligence attention-focused learning of learning content using AI face recognition / motion recognition sound reproduction technology is provided,

The facial recognition module uses the posenet algorithm as an AI-based facial contour recognition technology,

The learning content server or the user terminal analyzes the eye2/nose/ear2 face feature points of the online learner's face image detected by the camera in the user terminal of each learner, and when the gaze is off at a certain angle or the eye-ear distance If the distance in the corresponding direction of the nose-to-ear distance exceeds a certain standard value, a warning message or an alarm is generated to the corresponding user terminal so that artificial intelligence attention-focused learning occurs.

In non-face-to-face online learning, the instructor's terminal (supervisor terminal) transmits and receives question-and-answer chatting data to a plurality of learner terminals in a 1:N manner through the learning content server 100.

In non-face-to-face online learning, regardless of time and place, through the learning content server 100 during online learning, the instructor terminal underlines or texts the online learning memo space (shared workspace) of the educational content viewer provided to the learner terminal Sending/receiving files with report/learning files attached to the instructor's terminal (supervisor's terminal) and multiple learner terminals according to the instructor's floor control through drawings, photos, and voice recordings, and sending/receiving notes. to provide.

In non-face-to-face online learning, during non-face-to-face lectures, the instructor's terminal (supervisor terminal) is a 1:N method through the learning content server 100 to a plurality of learner terminals at the same time. Text/graphic notes/notes are written and displayed in the shared workspace.

The learning content server 100 provides a function of recognizing a QR code as a recognition code when using a smartphone.

The user terminal uses any one of a tablet PC, a smartphone, and a PC having a front camera, and the educational content viewer downloaded from the learning content server 100 is installed, and the educational content viewer is VOD media. A chat screen with the player is provided.

The user terminal

A VOD media player that plays non-face-to-face online lecture learning content;

A chat module that transmits and receives chat data between the instructor and the learner; and

a face recognition module for recognizing a facial behavior pattern of the front camera image of the user terminal and recognizing face outlines and feature points of the eye2/nose/ear2 face; and

If the screen deviates from the screen by more than a certain angle (in case the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value depending on whether the 3 eye/nose points get closer to the 2 ears at both ends), an alarm is sent to the user terminal or includes a learner's alarm generating unit that generates a warning message or outputs a warning message.

The face recognition module of the user terminal uses the posenet algorithm as an AI-based face contour recognition technology,

The face recognition module uses a frontal face recognition algorithm to extract a face object and a face behavior pattern for a standard-sized frontal face photo of a learner converted through size correction/angle correction/rotation to a standard size during non-face-to-face online learning. to extract the outline of the learner's face and the feature points of the face of Eye 2/Nose/Ear 2, and the center point (pupil) of the left/right ear and the left/right eye of the feature points of the face of Eye 2/Nose/Ear 2, respectively. Calculate the Euclidean distance (d) and similarity with the face, compare with specific points of the user's face photo in the face photo DB to check whether there is a gaze deviation, 3 eyes/nose points, 2 ears at both ends Detects head movement to the right/left depending on how close it is to , detects face patterns related to gaze deviation, and detects facial contours when recognizing faces, or when it is out of the screen by more than a certain angle (3 points on eyes/nose are positive If the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value depending on how close to the two end ears are, the angle at which the front face is turned left and right by recognizing the facial behavior pattern captured by the camera of the user terminal Since the distance between the right eye and the right ear and the distance between the left eye and the left ear are changed according to , an alarm is generated or a warning message is output to the corresponding user terminal.

The learning content server 100

WWW server 101;

a control unit 102 controlling to provide online learning content to the user terminal;

A member registration unit 103 connected to the control unit 102 and receiving registered member information and storing and managing ID/Passwd;

A user authentication unit 104 that is connected to the control unit 102 and authenticates using a QR code/Passwd or ID/Passwd or a personal certificate;

A QR code management unit 106 connected to the control unit 102 and issuing and managing a QR code corresponding to user information;

a learning content provider 105 connected to the controller 102 and providing the online learning content to the user terminal;

A chatting server 111 connected to the control unit 102 and transmitting and receiving chatting data between the instructor and the learner;

It is connected to the control unit 102, receives a picture taken by a camera of a candidate from a user terminal, converts it into a standard size front face picture through size correction/rotation/angle correction, and learning content server 100 a supervisor confirmation unit that compares user information pre-stored in the database with a standard-size frontal face photo and confirms eligibility for online learning/online examination by a supervisor at the supervisor's terminal;

Connected to the control unit 102, the user terminal uses a face recognition module to recognize the outline of the learner's face and facial feature points of the eyes 2/nose/ear 2, and face recognition receiving the face recognition result from the user terminal. part 107; and

Test information DB for storing test papers, written answers, and scoring results of test taker terminals; It includes a candidate DB and a face DB that store candidate information and a standard size frontal face picture.

Additionally, the learning content server 100 is connected to the controller 102 and further includes a chatting server 111 that transmits and receives chatting data between the instructor and the learner.

Additionally, the learning content server 100 is connected to the control unit 102 and further includes a send/receive note providing unit for transmitting/receiving note data to which the instructor and the learner can attach a report file.

Additionally, the learning content server 100 is connected to the controller 102 and further includes an FTP server for transmitting and receiving learning data files.

Additionally, during an online test or UBT test, the learning content server 100 provides a test program (App) and test papers to the test taker terminal and the supervisor terminal, learner/taker information, a frontal face picture of the learner/taker, and supervisor Information is managed, and during the online test or UBT test, the test paper writing answers are stored in each test taker's terminal within a certain test time, and then transmitted to the learning content server equipped with the test management unit at the end of the test, and the test takers' test paper writing answers and scoring A test management unit for storing and managing results, supervisor information, and test taker status information is further included.

Additionally, the AI-based face recognition technology used in the face recognition module of the test taker terminal and the cheating prevention module using the 5-point scale used the posenet algorithm to recognize the facial contours and feature points of the camera image of the tablet PC. During an online test or UBT test, AI-based face recognition technology is used to identify the tablet PC test taker through a proctor terminal linked to a learning content server equipped with a test management unit, to prevent proxy tests, and to prevent proxy tests. Prevent proxy tests or visual/auditory cheating by recognizing candidates' faces and voices.

The test paper questions of the test program (App) include subjective and/or multiple-choice test questions, and for each question, not only text and images, but also multimedia test questions including text, images, VR/AR content, voice and video are presented and server is displayed on the corresponding user terminal.

1. Artificial Intelligence Hub Platform

ㆍ NSD-AI (NSDevil's Artificial intelligence) platform-based artificial intelligence hub platform

ㆍ Instructor scoring guide through syntax recognition for language and medical education (collaborative research with Inje University College of Medicine and others)

ㆍ Experiential learning support system through object recognition (Fun Edu Lab and Korean Folk Village, etc.)

ㆍ Disease analysis function in dental education sector through tooth image analysis (joint research with Yonsei University College of Dentistry and other institutions in 3 countries)

ㆍ Based on UBT connect platform

ㆍ Tablet PC-based evaluation platform

ㆍ Language evaluation section (Speaking/Listening/Reading/Writing)

ㆍ Health and Medical Evaluation Division (Listening/Reading/Writing)

ㆍ Other job evaluation areas (read/write)

In addition, the learning content server 100 having a test management unit receives and stores test taker information and frontal face photos from a user terminal through a wired/wireless communication network, issues a QR code attached to the test taker to the test taker, and After recognizing each QR code, a greeting is provided through TTS conversion technology, and online test papers and a test program to conduct an online test or UBT test are provided to test taker terminals.

Additionally, the online exam or UBT exam offers subjective exams as well as 2/3/4/5 choice multiple-choice exams. The subjective test questions provide a subjective test in which the handwritten text of the stylus pen is recognized using the handwriting recognition unit of the test taker's terminal having a touch sensor and a display and converted into text to recognize the handwritten text.

Online test papers can be presented in a combination of all multiple-choice and multiple-choice/subjective forms.

Candidate terminals (300, 310, 311) are connected to the learning content server 100 having a test management unit through wired and wireless communication networks (LAN, Wi-Fi, LTE 4G/5G), drive test programs, and AI face recognition/motion recognition/ Using sound recognition technology, the front camera (C) of the test taker's terminal is used to perform facial recognition/motion recognition of the learner's/applicant's face image, and auditory misconduct is monitored through sound recognition through the microphone and voice recognition module of the test taker's terminal. The artificial intelligence face recognition module of the candidate's face photographed by the front camera (C) of the candidate's terminal and the facial behavior pattern photographed by the front camera (C) of the candidate's terminal A 5-point scale cheating prevention module, a voice recognition module that detects auditory cheating through voice recognition of the test taker, and a recording/recording program is installed on the test taker terminal linked to the learning content server 100 having a test management unit. installed

During an online test or UBT test, the proctor terminal receives face photos and cheating information from a plurality of test taker terminals through the learning content server 100 having a test management unit, checks them, and learns content server having a test management unit. Through (100), an anti-cheating alarm or message is transmitted to the corresponding test taker terminal.

The AI-based face contour recognition technology used in the face recognition module of the user terminal uses a posenet algorithm (machine learning model which allows for real-time face pose estimation).

In the embodiment, during online learning/online exam or UBT exam, the user terminal uses a tablet PC.

The test taker terminals 300, 310, and 311 are provided with a face recognition module for facial feature points on a five-point scale of the outline of the learner's face image and the eyes 2/nose/ear 2.

In addition, the test taker terminal further includes a handwriting recognition unit of the test taker terminal having a display and a touch sensor for recognizing and converting handwritten text of a stylus pen into text for subjective test questions.

The candidate terminals 300, 310, and 311 use the posenet algorithm as the AI-based facial contour recognition technology used in the face recognition module during online learning/online exam.

During the online test or UBT test, the face recognition module recognizes the contours of the face and the feature points of the face on a 5-point scale with eyes 2/nose/ear 2. (ROI) is detected and a face object is extracted using a front face recognition algorithm for a standard size frontal face photo converted through size correction/rotation/angle correction to a standard size based on the vertex of the nose, and a face action pattern is extracted. and extracts the contours of the candidate's face and facial feature points of eye 2/nose/ear 2, and through feature extraction and classification, the left/right ear and left/right ear of eye 2/nose/ear 2 face feature points, respectively. Calculate the Euclidean distance (d) and similarity with the central point (pupil) of the eye, compare it with the photo in the face photo DB of the online test or UBT test server, check whether it is a surrogate test, and check the online test or UBT test Head movement to the right/left is detected according to whether the 3 eye/nose points approach the 2 ears at both ends at the time of writing, abnormal facial behavior patterns related to cheating are detected, and facial contours are not recognized during face recognition. , If the distance from the test screen is more than a certain angle (depending on whether the 3 eye/nose points get closer to the 2 ears at both ends, if the distance in the corresponding direction of the eye-ear distance or nose-ear distance exceeds a certain standard value) To prevent this, the facial behavior pattern captured by the camera of the test taker terminal is recognized, and the distance between the right eye and the right ear and the distance between the left eye and the left ear change according to the angle at which the front face is turned left and right, so visual cheating is judged and the cheating image is determined. Alternatively, the video data is transmitted to the learning content server 100 equipped with the test management unit, and after confirmation by the supervisor terminal 200 linked to the learning content server 100 equipped with the test management unit, learning with the test management unit is performed. A warning message or an alarm is generated to the corresponding test taker terminal through the content server 100 .

(Example)

TensorFlow was used for pose estimation of a human face on Android on a tablet PC using the Posenet algorithm model. For reference, the Posenet algorithm is a vision model that estimates a person's pose from an image or video by detecting the position of a person's face, body, arms and legs, and major body parts for action recognition. At the time of face recognition, TensorFlow Lite uses the front camera of the test taker terminal to identify the contour of the face and facial feature points of eyes 2/nose/ear 2 in real time during face recognition, and implements a facial contour pose estimation model of the face. .

The posenet algorithm used in the artificial intelligence face recognition module uses a five-point scale anti-fraud module of leftEye, rightEye, nose, leftEar, and rightEar. implemented,

After finding the x,y coordinates of leftEar and the x,y coordinates of rightEar, you can find the diameter of a circle based on these and create an angel ring with ellipse( ). The diameter can be easily obtained with dist( ) in p5js without having to use the Pythagorean theorem.

<Connecting ml5js and Webcam>

<script src="https://unpkg.com/ml5@0.3.1/dist/ml5.min.js"></script>

let video;

let poseNet;

function setup() {

createCanvas(400, 400);

video = createCapture(VIDEO);

// Used to prevent the video from appearing twice

video.hide( );

poseNet = ml5.poseNet(video, modelLoaded);

poseNet.on('pose', gotPoses);

console.log(ml5);

}

// Pose-related callbacks are entered

function gotPoses(poses) {

}

// Used to check if loading was successful

function modelLoaded() {

console.log('Model Loaded');

}

function draw() {

// Draw the webcam at position 0,0.

image(video,0,0);

}

The face region is extracted in real time from the face image of the front camera of the test taker terminal, the standard size of the face picture is adjusted to the standard size of the face photo through size correction / rotation / angle correction to the standard horizontal x vertical size based on the vertex of the nose, and the test management unit is provided. Comparison with the learning data of the standard size face photo DB pre-stored in the database of the learning content server -> Learning model of the face photo -> Extracting and classifying the contours of the face and facial feature points of the eyes 2 / nose / ears 2, Group clustering (centre of cluster k-means algorithm) density estimation of facial feature points of the contours of the recognition data and eye 2/nose/ear 2 outlines, Euclidean distance (d) and similarity (similarity) of facial feature points By calculating the distance and similarity of the facial feature points of the standard size of the candidate's photograph stored in the online test or UBT test server's face image DB, the test taker's photo is checked by the candidate himself to prevent proxy testing, and during the online test or UBT test, visual to judge human misconduct.

For example, if the direction of the face photographed by the front camera of the test taker terminal (tablet PC) is changed or the test taker turns the face at a certain angle, the outline of the face and the eyes 2 / nose / ears of the face photo are changed. 2 If the feature points of the face are not recognized or the angle toward the test program changes within the effective distance between the user's face and the smartphone (20 ~ 30 cm), it is recognized as cheating, and the cheating detection result is displayed as an online test or UBT. It is transmitted to the learning content server 100 having a test management unit, transmitted to the supervisor terminal 200, and after confirmation by the supervisor, an alarm is generated or warned to the test taker terminal through the learning content server 100 equipped with a test management unit. Messages are sent to prevent the candidate from cheating.

In addition, [Text A] As soon as the warning threshold set in the predefined test taker program with reference to data becomes 0, a warning message is displayed on the test taker's device without server connection, and it can be transmitted back to the supervisor/cheating detection server/other test management server, , or after the test is finished, the raw data is sent to the fraud detection server/other test management server for post-processing.

In addition, the test taker terminal includes a face recognition module, an anti-cheating module for detecting visual cheating by recognizing facial contours and eye 2/nose/ear 2 facial feature points on a 5-point scale, and the cheating prevention module. further includes a voice recognition module for recognizing the candidate's speech in order to prevent the candidate's auditory cheating;

During the online test or UBT test, when the voice recognition module hears the voice of the test taker related to the cheating input through the microphone of the test taker's terminal, it immediately transmits it to the learning content server 100 having a test management unit after recognizing the voice. After the supervisor checks, an alarm is generated or a warning message is sent to the test taker terminal through the learning content server 100 to prevent cheating. Alternatively, the test taker program itself directly displays a warning message to the test taker terminal without a server command by referring to the threshold of the predefined distance of the eye-to-ear distance and the nose-to-ear distance of the standard size frontal face picture, and detects cheating information. is transmitted to the test management unit (test server) of the learning content server and the supervisor's terminal.

11 is a flowchart showing a tablet PC, smartphone, and PC-based attention learning method using facial contour recognition artificial intelligence in web browser-based non-face-to-face learning according to the present invention.

The tablet PC, smartphone, and PC-based attentional learning method using facial contour recognition artificial intelligence in web browser-based non-face-to-face learning of the present invention (a) transmits learner information and a standard-sized frontal face picture from a user terminal to a learning content server Registering and storing in (100); (b) issuing, by the learning content server 100, a QR code corresponding to user information and a front face photo; (c) The front face of the camera using the front face recognition algorithm of the front camera image in the user terminal equipped with the educational content viewer program and the face recognition module that recognizes the contours of the face and the facial feature points of the eyes 2/nose/ear 2 Receiving a photo recognition result to the learning content server 100 and comparing learner information pre-registered in the database of the server with a standard size frontal face photo and its facial features to confirm whether the learner is himself or not; (d) providing online learning content from the learning content server 100 to a user terminal of a pre-registered member; and (e) the facial recognition module uses the posenet algorithm as an AI-based facial contour recognition technology, and the learning content server 100 or the user terminal uses the face image of the online learner detected by the camera in the user terminal of each learner. Eye 2/Nose/Ear 2 Analyzing the feature points of the user's terminal screen when the line of sight deviates from a certain angle, or when the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value Artificial Intelligence and generating a warning message or an alarm to a corresponding user terminal so that attention-focused learning is performed.

The learning content server 100 provides a function of recognizing a QR code as a recognition code when using a smartphone.

The user terminal uses any one of a tablet PC, a smartphone, and a PC having a front camera, and the educational content viewer downloaded from the learning content server 100 is installed, and the educational content viewer is VOD media. A chat screen with the player is provided.

The user terminal

A VOD media player that plays non-face-to-face lecture learning content;

A chat module that transmits and receives chat data between the instructor and the learner; and

a face recognition module for recognizing a behavioral pattern of a face in the front camera image of the user terminal and recognizing feature points of the eye2/nose/ear2 face;

If the face image deviates from the screen by more than a certain angle (when the distance between the eyes and ears and the distance between the nose and ears in the corresponding direction exceeds a certain standard depending on whether the 3 points of the eye/nose are closer to the 2 points of the ears at both ends), an alarm is set. Include a learner's alarm generating unit that generates or outputs a warning message.

The facial recognition module uses the posenet algorithm as an AI-based facial contour recognition technology,

The face recognition module extracts a face object using a frontal face recognition algorithm for a face photo of a learner of a standard size that has been size-corrected/angle-corrected during non-face-to-face online learning, recognizes a facial behavior pattern, and recognizes the outline of the learner's face The facial feature points of Eye2/Nose/Ear2 are extracted, and the Euclidean distance (d) of the facial feature points of Eye2/Nose/Ear2 is between the left/right ear and the central point (pupil) of the left/right eye, respectively. and similarity are calculated, compared with specific points of the user's face photo in the face photo DB, whether the line of sight is out of sight, and according to whether the 3 points of the eye/nose are closer to the 2 points of the ears at both ends, it is moved to the right/left. Detects head movement, detects face patterns related to gaze deviation, and detects facial contours when recognizing faces, or when it is out of a certain angle or more from the screen of the user terminal (3 points of eyes/nose are close to 2 points of ears at both ends) If the eye-to-ear distance or nose-to-ear distance in the corresponding direction exceeds a certain standard value) Recognizes the facial behavior pattern of the video captured by the camera of the user terminal, and detects the right eye according to the angle at which the front face is turned left and right Since the distance between the right ear and the distance between the left eye and the left ear are different, an alarm is generated or a warning message is output to the corresponding user terminal so that the user terminal can focus on learning.

The learning content server 100

WWW server;

a control unit controlling to provide online learning content to the user terminal;

a member registration unit that is connected to the control unit and stores and manages ID/Passwd by receiving registered member information;

A user authentication unit that is connected to the control unit and authenticates using a QR code/Passwd or ID/Passwd or personal certificate;

a QR code management unit connected to the control unit and issuing and managing a QR code corresponding to user information;

a learning content provider connected to the controller and providing the online learning content to the user terminal;

a chatting server connected to the control unit and transmitting and receiving chatting data between the instructor and the learner;

It is connected to the control unit, receives a picture taken by a camera of a test taker from a user terminal, generates a front face picture through size correction/rotation/angle correction of a standard size, and user information stored in a database of a server and a standard size front face picture. a supervisor confirmation unit that compares and confirms eligibility for online learning by a supervisor in a supervisor terminal;

a face recognition unit connected to the control unit, recognizing the outline of the learner's face and facial feature points of the eyes 2/nose/ear 2 using a face recognition module in the user terminal, and receiving a face recognition result from the user terminal; and

Test information DB for storing test papers, written answers, and scoring results of test taker terminals; It includes a candidate DB and a face DB that store candidate information and a standard size frontal face picture.

Additionally, the learning content server 100 is connected to the controller 102 and further includes a chatting server 111 that transmits and receives chatting data between the instructor and the learner.

Additionally, the learning content server 100 is connected to the control unit 102 and further includes a send/receive note providing unit for transmitting/receiving note data to which the instructor and the learner can attach a report file.

Additionally, the learning content server 100 is connected to the controller 102 and further includes an FTP server for transmitting and receiving learning data files.

In non-face-to-face online learning, the instructor's terminal (supervisor terminal) transmits and receives question-and-answer chatting data to a plurality of learner terminals in a 1:N manner through the learning content server 100.

In non-face-to-face online learning, regardless of time and place, through the learning content server 100 during online learning, the instructor terminal underlines or texts the online learning memo space (shared workspace) of the educational content viewer provided to the learner terminal Sending/receiving files with report/learning files attached to the instructor's terminal (supervisor's terminal) and multiple learner terminals according to the instructor's floor control through drawings, photos, and voice recordings, and sending/receiving notes. to provide.

In non-face-to-face online learning, during non-face-to-face lectures, the instructor's terminal (supervisor terminal) is a shared workspace of the educational content viewer with multiple learner terminals simultaneously in a 1:N manner through the learning content server 100 Text/graphic notes/memos are written and displayed on the screen.

Additionally, during the online test or UBT test, the learning content server 100 stores and manages the candidate information, the candidate's on-site front face photo, and supervisor information in the database of the learning content server having the test management unit, and manages the online test or At the time of the UBT test, after storing the test paper writing answers in each test taker terminal for a certain test time, at the end of the test, the test taker terminal receives and stores them from the test taker terminal to the learning content server 100 having a test management unit, and the scoring results of the test takers' test paper writing answers are stored. A step of providing the information to the test taker terminal is further included.

In addition, the online and UBT tests include a 2nd/3rd/4th/5th multiple-choice multiple-choice test and a subjective test in which the handwriting of a stylus pen is recognized and converted into characters using the handwriting recognition unit of the test taker terminal to recognize handwritten characters. include

In addition, the method displays a warning message to the test taker terminal at the time when the user's facial movement of the camera image of the user terminal is stored in units of frames, such as [Text A], after subtracting a defined frame or time or number of times, and then becomes 0. / After exposing or sending a supervisory device or cheating detection image to the server, resetting the number of warning triggers and starting the deduction again from the next movement.

[Text A] - Facial behavior pattern information stored on the tablet PC of the test taker terminal (Ex. Face pattern data of Seoul National University dental school learners): information such as date, time, movement angle/distance, and phase (left and right) It is saved every frame or hourly, and based on this, a warning message is shown to the user, and the supervisor checks and the candidate's face recognition cheating is detected. - The corresponding RAW data is transmitted to the learning content server system for post-processing.

200519094106,0.02,R

200519094111,0.03,R

200519094120,0.42,R

200519094148,0.01,R

200519094150,0.21,R

200519094155,0.22,R

200519094158,0.09,R

200519094209,0.11,L

200519094214,0.09,R

200519094216,0.10,R

200519094219,0.09,R

200519094224,0.01,L

In the web browser-based non-face-to-face learning system and method of the present invention, in the non-face-to-face learning, the educational content viewer of the user terminal is provided with a media playback unit and a facial contour recognition module, and a learning content server ( ubcloud server) to directly look at the non-face-to-face lecture learning contents focused on the front camera of the user terminal linked to the user terminal, and recognize the facial contour and eye 2 / nose / ear 2 facial feature points on a 5-point scale so that attention is focused on the learning content When the gaze is diverted at a certain angle through facial recognition, an alarm or warning message is generated to the user terminal using sound and messaging technology, and attentional learning using web browser-based ubcloud artificial intelligence is provided in non-face-to-face online learning. .

Embodiments according to the present invention are implemented in the form of program instructions that can be executed through various computer means, and can be recorded in a computer readable recording medium. The computer readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, hard disks, magnetic media such as floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - A hardware device configured to store and execute program instructions in storage media such as magneto-optical media, ROM, RAM, flash memory, etc. may be included. Examples of program instructions may include those produced by a compiler, machine language codes as well as high-level language codes that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention is implemented as a program and can be read using computer software on a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) ) can be stored in

Although described with reference to specific embodiments of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiments to illustrate the technical idea as described above, and within the limit that does not deviate from the technical spirit and scope of the present invention It can be implemented with various modifications, and the scope of the present invention should be determined by the claims described later.

100: learning content server 101: WWW server
102: control unit 103: member registration unit
104: user authentication unit 105: learning content providing unit
106: QR code providing unit 107: face recognition unit (facial contour recognition)
109: test management department 111: chat server
300, 310, 311: user terminal

Claims (14)

A learning content server that provides non-face-to-face online lecture learning content; and
A face recognition module that connects to the learning content server and runs an educational content viewer program, recognizing facial contours and eye 2 / nose / ear 2 face feature points of the front camera image data of the user terminal in non-face-to-face online learning It has a voice recognition module, and includes a user terminal with an educational content viewer installed so that artificial intelligence attention-focused learning of learning content using AI face recognition / motion recognition sound reproduction technology,
The facial recognition module uses the posenet algorithm as an AI-based facial contour recognition technology,
The learning content server or the user terminal analyzes the eye2/nose/ear2 face feature points of the online learner's face image detected by the camera in the user terminal of each learner, and when the gaze is off at a certain angle or the eye-ear distance , When the distance in the corresponding direction of the nose-to-ear distance exceeds a certain standard, an attentional learning system in web browser-based non-face-to-face learning generates a warning message or an alarm to the corresponding user terminal so that artificial intelligence attentional learning occurs. According to claim 1,
The learning content server provides a function of recognizing a QR code as a recognition code when using a smartphone, an attentional learning system in web browser-based non-face-to-face learning. According to claim 1,
The user terminal uses any one of a tablet PC, a smartphone, and a PC having a front camera, and the educational content viewer downloaded from the learning content server is installed, and the educational content viewer chats with a VOD media player. A web browser-based attention learning system in non-face-to-face learning with a screen. According to claim 1,
The user terminal
A VOD media player that plays non-face-to-face online lecture learning content;
A chat module that transmits and receives chat data between the instructor and the learner;
a face recognition module for recognizing a facial behavior pattern of the front camera image of the user terminal and recognizing face outlines and face feature points of eyes 2/nose/ear 2; and
If the screen deviates from the screen by more than a certain angle (in case the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value depending on whether the 3 eye/nose points get closer to the 2 ears at both ends), an alarm is sent to the user terminal A learner's alarm generating unit for generating or outputting a warning message;
Attentional learning system in non-face-to-face learning based on web browsers. According to claim 4,
The face recognition module extracts a face object using a frontal face recognition algorithm for a face photo of a learner of a standard size that has been size-corrected/angle-corrected during non-face-to-face online learning, recognizes a facial behavior pattern, and recognizes the outline of the learner's face The facial feature points of Eye2/Nose/Ear2 are extracted, and the Euclidean distance (d) of the facial feature points of Eye2/Nose/Ear2 is between the left/right ear and the central point (pupil) of the left/right eye, respectively. and similarity are calculated, compared with specific points of the user's face photo in the face photo DB, whether the line of sight is out of sight, and according to whether the 3 points of the eye/nose are closer to the 2 points of the ears at both ends, it is moved to the right/left. Detects head movement, detects face patterns related to line of sight deviation, and detects facial contours when facial contours are not recognized during face recognition, or when it is out of a certain angle or more from the screen If the eye-to-ear distance and nose-to-ear distance in the corresponding direction exceed a certain standard value) Recognize the facial behavior pattern captured by the camera of the user terminal, and measure the distance between the right eye and the right ear according to the angle at which the front face is turned left and right. An attentional learning system in web browser-based non-face-to-face learning that generates an alarm or outputs a warning message to a corresponding user terminal because the distance between the left eye and the left ear is different. According to claim 1,
The learning content server
WWW server;
a control unit controlling to provide online learning content to the user terminal;
a member registration unit that is connected to the control unit and stores and manages ID/Passwd by receiving registered member information;
A user authentication unit that is connected to the control unit and authenticates using a QR code/Passwd or ID/Passwd or personal certificate;
a learning content provider connected to the controller and providing the online learning content to the user terminal;
a QR code management unit connected to the control unit and issuing and managing a QR code corresponding to user information;
a chatting server connected to the control unit and transmitting and receiving chatting data between the instructor and the learner;
It is connected to the control unit, receives a photograph taken by a camera of a candidate from a user terminal, generates a standard size frontal face picture through standard size size correction / rotation / angle correction, and compares user information stored in a database of a server with standard size. a supervisor confirmation unit that compares frontal face photos and confirms eligibility for online learning by a supervisor in a supervisor terminal;
a face recognition unit that is connected to the control unit and recognizes the outline of the learner's face and facial feature points of eyes 2/nose/ear 2 using the face recognition module of the user terminal, and receives a face recognition result from the user terminal; and
Test information DB for storing test papers, written answers, and scoring results of test taker terminals; a candidate DB and a face DB that store candidate information and a standard size frontal face picture;
Attentional learning system in web browser-based non-face-to-face learning, including. According to claim 6,
The learning content server provides a test program (App) and test papers to the test taker terminal and supervisor terminal, manages test taker information, on-site face photos of the test taker, and supervisor information, and each within a certain test time during an online test or UBT test. A test management unit is provided at the end of the test after storing test paper writing answers in the test taker's terminal, which is transmitted to the learning content server, and a test management unit that stores and manages test takers' test paper writing answers, scoring results, supervisor information, and test taker status information. Including web browser-based non-face-to-face learning system for attention-focused learning. (a) receiving and storing learner information and a standard-sized frontal face photo from a user terminal in a learning content server;
(b) issuing, by the learning content server, a QR code corresponding to user information and a standard size frontal face picture;
(c) In a user terminal equipped with an educational content viewer program and a face recognition module that recognizes facial contours and eye 2/nose/ear 2 facial feature points, the front face recognition algorithm of the front camera image data is used to detect the front face of the camera. Receiving a face photo recognition result from the learning content server and comparing pre-registered learner information with a frontal face photo and its facial feature points to confirm whether the learner is himself or not;
(d) providing learning content from the learning content server to a user terminal of a pre-registered member; and
(e) The facial recognition module uses the posenet algorithm as an AI-based facial contour recognition technology, and the learning content server or the user terminal uses the eyes 2/nose of the online learner's face image detected by the camera in each learner's user terminal. /Ear 2 analyzes facial feature points and sends a warning message to the corresponding user terminal for artificial intelligence attention-focused learning when the line of sight deviates at a certain angle or when the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value or generating an alarm;
Attentional learning method in non-face-to-face learning based on web browsers. According to claim 8,
The learning content server provides a function of recognizing a QR code as a recognition code when using a smartphone, an attentional learning method in web browser-based non-face-to-face learning. According to claim 8,
The user terminal uses any one of a tablet PC, a smartphone, and a PC having a front camera, and the educational content viewer downloaded from the learning content server is installed, and the educational content viewer chats with a VOD media player. An attentional learning method in web browser-based non-face-to-face learning having a screen. According to claim 8,
The user terminal
A VOD media player that plays non-face-to-face lecture learning content;
A chat module that transmits and receives chat data between the instructor and the learner;
a face recognition module for recognizing a facial behavior pattern of the front camera image of the user terminal and recognizing face outlines and feature points of the eye2/nose/ear2 face; and
If the screen deviates from the screen by more than a certain angle (in case the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value depending on whether the 3 eye/nose points get closer to the 2 ears at both ends), an alarm is sent to the user terminal A learner's alarm generating unit for generating or outputting a warning message;
Attentional learning method in non-face-to-face learning based on web browsers. According to claim 11,
The face recognition module extracts a face object using a frontal face recognition algorithm for a face photo of a learner of a standard size that has been size-corrected/angle-corrected during non-face-to-face online learning, recognizes a facial behavior pattern, and recognizes the outline of the learner's face The feature points of Eye 2/Nose/Ear 2 are extracted, and the Euclidean distance (d) and similarity between the central point (pupil) of the left/right ear and the left/right eye of the feature points of the face of Eye 2/Nose/Ear 2, respectively. (similarity) is calculated, and compared with specific points of the user's face photo in the face photo DB, whether the line of sight is out of sight is checked, and the head is moved to the right/left depending on whether the 3 points of the eyes/nose are closer to the 2 points of the ears at both ends. and detects a face pattern related to gaze deviation, and if the face contour is not recognized during face recognition, if it is out of a certain angle or more from the screen (eye and nose 3 points are closer to the 2 ears at both ends If the distance in the corresponding direction of the distance between the ears and the nose and ears exceeds a certain standard value) Recognizes the facial behavior pattern captured by the camera of the user terminal, and the distance between the right eye and the right ear and the left eye according to the angle at which the front face is turned left and right An attentional learning method in web browser-based non-face-to-face learning that generates an alarm or outputs a warning message at a corresponding user terminal because the distance between the ear and the left ear is different. According to claim 8,
The learning content server
WWW server;
a control unit controlling to provide online learning content to the user terminal;
a member registration unit that is connected to the control unit and stores and manages ID/Passwd by receiving registered member information;
A user authentication unit that is connected to the control unit and authenticates using a QR code/Passwd or ID/Passwd or personal certificate;
a learning content provider connected to the controller and providing the online learning content to the user terminal;
a QR code management unit connected to the control unit and issuing and managing a QR code corresponding to user information;
It is connected to the control unit, receives a photograph taken by a camera of a candidate from a user terminal, generates a standard size frontal face picture through standard size size correction / rotation / angle correction, and compares user information stored in a database of a server with standard size. a supervisor confirmation unit that compares frontal face photos and confirms eligibility for online learning by a supervisor in a supervisor terminal;
a face recognition unit connected to the control unit, recognizing the outline of the learner's face and facial feature points of the eyes 2/nose/ear 2 using a face recognition module in the user terminal, and receiving a face recognition result from the user terminal; and
Test information DB for storing test papers, written answers, and scoring results of test taker terminals; a candidate DB and a face DB that store candidate information and a standard size frontal face picture;
Attentional learning method in non-face-to-face learning based on web browsers. According to claim 8,
The learning content server stores and manages the candidate information, the candidate's on-site face picture, and the supervisor information in the database of the test server, and stores the test paper preparation answer in each candidate terminal during a certain test time during the online test or UBT test. Attentional learning method in web browser-based non-face-to-face learning, further comprising receiving and storing the test taker terminal to a learning content server having a test management unit at the end of the test, and providing the scoring results of the test takers' answers to the test taker terminal to the test taker terminal. .

Images