KR20210155303A - Online Test System using face contour recognition AI to prevent the cheating behaviour and method thereof - Google Patents

Abstract

Disclosed are an online test system using face contour recognition artificial intelligence (AI) to prevent cheating, which prevents proxy testing or cheating, and a method thereof. According to the present invention, the system comprises: an online test or ubiquitous (UBT) test server registering test applicant information and a frontal face photo to issue a QR code and providing a test program for performing an online or UBT test; a test applicant terminal connected to the online or UBT test server through a wired or wireless communication network and including an AI facial recognition module of the face photographed through a front camera of the test applicant terminal and a five-point scale cheating prevention module recognizing facial behavior patterns photographed by the front camera to monitor movement of facial feature points including the eyes, the nose, and the ears; and a supervisor terminal receiving the face photos and cheating information of a plurality of test applicant terminals through the online or UBT test server and transmitting a cheating prevention alarm or warning message to the test applicant terminal. The cheating prevention module of the test applicant terminal includes an AI-based module for visually detecting cheating in video data of a camera of the test applicant terminal in a real-time detection module and a voice recognition module.

Description

Online Test System using face contour recognition AI to prevent the cheating behavior and method thereof

The present invention relates to an online test system and method using artificial intelligence to prevent cheating, and more particularly, to an AI face recognition module and a 5-point scale of facial feature points, a cheating prevention module, during online tests and UBT tests. In the tablet PC, smartphone, PC-based online test and UBT test, the object of the test taker's face is detected in the test taker terminal using AI-based facial contour recognition technology (posenet algorithm) to prevent proxy tests or cheating. It extracts the facial contour and the eye2/nose/ear2 feature points, and detects and tracks the head movement to the right/left according to whether the 3 eye/nose points get closer to the 2 ear points at both ends by the anti-cheating module. It is equipped with an artificial intelligence facial recognition module that detects facial behavior patterns and an anti-fraud module on a 5-point scale of facial feature points. (If the distance in the corresponding direction, such as the distance between the eyes/ears and the distance between the nose/ears, etc., exceeds a certain value depending on whether the 3 eye/nose points get closer to the 2 ear points on both ends), a warning message or alarm is output or the test taker terminal Tablet PC, smartphone, PC-based online test system and method using facial contour recognition artificial intelligence to prevent cheating, which is stored in the information device of it's about

Face recognition technology is mainly used for face recognition based on an appearance based matching method and a feature. Face recognition is recognized differently depending on the camera's shooting angle, lighting direction, posture, facial expression changes, and face changes over time.

For face recognition based on features, a detection method using haar-like features for image data captured by a digital camera, a camera of an IoT device, or a camera of a smartphone, and a detection method using a Modified Census Transform (MCT) image are used. . Using the face and eye detectors learned as Haar-like features from the input image of the camera of the smartphone, the contour of the face and the forehead/eyes/nose/mouth are detected, and the region of interest (ROI) is used to detect circular pupils. The eye area set as Interest) is converted to grayscale, and the histogram [x-axis pixel value of each pixel, The number of corresponding pixel values on the y-axis] is calculated, the eye image is binarized, and the photo of the eye area is preprocessed through histogram equalization. In the face area, the eyebrows, eyes, nose, mouth, A face is recognized by detecting the facial features of the outline, extracting the texture features and shape features, and comparing the similarity with the feature points of the face photos stored in the face recognition DB.

The eyebrow and eye, nose, mouth, and chin feature values of the face region are expressed as the difference between the sum of pixels included in the white region of the Haar-like feature and the sum of pixels included in the black region.

For example, the distance from the detected eye region to the endpoints of the right and left eyes in the face region photos of horizontal and vertical standard sizes, and the size of the iris extracted using the Hough circle transform algorithm. The value is used as the feature value.

As a related prior art 1, Patent Publication No. 10-2017-0050465 discloses "facial recognition apparatus and method".

According to this embodiment, in recognizing a face from an input image using machine learning, the face recognition rate is improved by normalizing the face pose and perspective, and the face learning data is obtained by automatically generating a virtual face image as the face learning data. A face recognition device and method are provided for saving time and cost.

1 is a block diagram of an existing face recognition device.

The face recognition device 100 includes an image display device, an image capturing device, a face recognition server, a tablet PC, a laptop, a personal PC, a smart phone, a personal digital assistant (PDA), a mobile communication terminal, and It may be any one of intelligent robots and the like.

The face recognition apparatus 100 includes an input image acquisition unit 112 for acquiring an input image input from a camera; A front pose image is generated by detecting a face region in the input image and normalizing a face pose, and the perspective of the front pose image is removed in order to remove perspective distortion according to the distance between the camera and the subject. ) by normalizing the normalization unit 114 to generate a normalized image; a feature vector extracting unit 116 for extracting a feature vector representing the face of the subject from the normalized image; and a face recognition unit 118 for recognizing the face of the subject included in the input image by applying the feature vector to the previously learned classification model.

The input image acquisition unit 112 acquires an input image input from the camera. The camera may be a depth recognition camera, a stereo camera, or a color camera (eg, a Kinect camera, etc.). In addition, the input image is an image including the face of a subject to be recognized, and includes a two-dimensional still image and Includes video. The input image may include a color image, a depth image, and a color-depth (RGB-D) image.

The normalizer 114 generates a normalized image by detecting a face region from the input image and normalizing a face pose and perspective. When there is a change in the face pose, the face recognition rate is lowered because the gray scale, shape, and position of feature points change. Also, if the distance between the camera and the subject is different, even for the same subject, perspective distortion (eg, distortion) occurs differently at each photographed location, so it may appear as if a different subject was photographed. Therefore, in order to improve the face recognition rate, it is necessary to normalize the face pose and perspective of the input image.

The normalization unit 114 inputs the face images for learning of various poses to the input layer of the first artificial neural network, and trains the first artificial neural network so that the face images for learning of the front pose are output from the output layer of the first artificial neural network. face pose normalization learning unit; and inputting the data output from the output layer of the first artificial neural network to the input layer of the second artificial neural network, and learning the second artificial neural network so that a face image for learning without perspective distortion is output from the output layer of the second artificial neural network. Includes a perspective normalization learning unit.

The normalization unit inputs the learning face images of various poses with various perspective distortions to the input layer of the integrated artificial neural network integrating the first artificial neural network and the second artificial neural network in which learning is completed, and there is no perspective distortion of the front pose. The integrated artificial neural network is trained so that the face image for learning is output from the output layer of the integrated artificial neural network.

The feature vector extraction unit 116 is determined through machine learning and extracts a feature vector representing the face of the subject from the normalized image.

A feature vector is a vector having feature values used for face recognition as elements. Filters used to extract feature vectors include Gabor filters, Haar filters, LBP (Local Binary Pattern) - including DLBP (Discriminative LBP), ULBP (Uniform LBP), NLBP (Number LBP) - etc., but must be limited to this and other filters may be used.

The face recognition unit 118 recognizes the face of the subject included in the input image by applying the feature vector extracted by the feature vector extraction unit 116 to the pre-learned classification model. The previously learned classification model may include a support vector machine (SVM), a linear discriminant analysis (LDA), and a softmax, but is not necessarily limited thereto.

The virtual face image generator 124 may generate a plurality of virtual face images used for learning by the normalizer 114 , the feature vector extractor 116 , and the face recognizer 118 .

The plurality of virtual face images means a face image generated by transforming a three-dimensional face model synthesized by the virtual face image generator 124 using one or more two-dimensional reference images obtained from a camera.

* Features of face recognition system

The recognition rate changes depending on the lighting or distance from the camera.

The shape of the face continues to change according to the angle, expression, and age of the face, and may be recognized differently according to changes in the appearance such as eyebrows, glasses, and makeup.

* Configuration steps of face recognition system

- Image acquisition: Image acquisition from CCD or CMOS Image Sensor camera

- Pre-processing: noise-removing image processing and segmentation

- Face detection: Detects the face area of the ROI area from the input image

- Face normalization: standard size, feature extraction, brightness and geometric normalization

- Face recognition: comparison and recognition of detected images and database images

* Geometric method

- Features: Recognize by extracting geometrical feature points of the face and determining whether they match

- Performance: Because the face is three-dimensional and can be rotated, there is a limit to its application.

* Eigenfaces: Developed by Pentland in 1991.

- Features: PCA (Principal Component Analysis) is applied by extracting key points, and Euclidean distance is applied by measuring similarity.

- Performance: A representative face recognition method that responds sensitively to lighting or environmental changes, but has been widely used and has proven its performance.

* Fisherfaces

- Features: Face recognition method using FLD (Fisher Linear Discriminant) as a classification algorithm.

- Performance: An algorithm that is more accurate and insensitive to environmental changes by learning the characteristics of each person, and can be applied in real time in an on-line state.

* Method based on SVM (Support Vector Machine)

- Features: Algorithm using PCA and SVM (Support Vector Machine) algorithm.

- Performance: Algorithm that has a relatively high recognition rate, but consumes a lot of time and memory to be applied to multi-class applications such as face recognition. A representative algorithm of the face recognition algorithm currently being studied.

* Neural network

- Features: Search for learned face shapes using sliding windows in black and white still images

- Performance: It is possible to extract two or more faces, but the speed is slow and learning is difficult.

* Fuzzy + Neural Network

- Features: As the input of the neural network circuit, the fuzzy membership function is used instead of the pixel brightness value.

- Performance: Performance is improved compared to the method using only neural networks, but processing speed is lower.

* Wavelet + Elastic Matching

- Features: It uses frequency conversion and is effective in handling changes in posture and expression.

- Performance: The amount of computation is high compared to the recognition rate.

As a related prior art, in Patent Registration No. 10-1765770, a system and method for detecting cheating of an online test using image processing and a recording medium storing a program for implementing the same are registered.

The online test cheating detection system and method can supervise online exam cheating and secure the reliability and authenticity of the online exam by using image processing to detect suspected cheating of the examinee during the online exam. Provided are a system for detecting cheating of an online test using image processing that can be performed and a method for detecting the same, and a recording medium storing a program for implementing the same.

1 is a block diagram of an online evaluation system including a conventional online test cheating detection system.

The online evaluation system includes an examinee terminal 110 for an examinee to take an online exam; a camera 120 attached to the examinee's terminal or provided in a smart device to photograph the front of the examinee; a cheating detection server 100 that receives image information photographed by the examinee from the camera in frame units, and detects cheating through image processing; When the cheating detection server determines that the test taker's cheating is suspected, a supervisor terminal 130 for notifying the supervisor; and an online evaluation server 140 for examining the questions of the online test taken by the examinee.

The cheating detection server 100 receives the test taker's image information through the communication network, and when it is determined that cheating is suspected, it sends a cheating detection message to the supervisor, and sends it to the online evaluation server 140 . Transceiver 101 for notifying to stop the online test; a storage unit 102 for storing image information and coordinate information of the examinee received through the transceiver unit 101; a face recognition unit 103 for defining a rectangular area including a face area from the received image information of the examinee; a pupil coordinate obtaining unit 104 for setting pupil coordinates of both eyes by image processing the shape and contrast of both eyes in the coordinate system based on the rectangular region; An intermediate coordinate obtaining unit 105 that determines the intermediate coordinates of the pupil coordinates of both eyes, a background recognition unit 106 that sets the background for all areas except for the body and face areas of the examinee, and the face recognition unit ( 103), the real-time pupil coordinates are compared with the reference pupil coordinates received from the pupil coordinate acquisition unit 104 and the average values of the real-time pupil coordinates are compared to determine whether or not there are two or more face regions. Whether the equilibrium value is outside the preset error range, whether the movement distance of the intermediate coordinates received from the intermediate coordinate obtaining unit 105 is out of the preset error range, and the background received from the background recognition unit 106 and a misconduct determination unit 107 for determining whether a change in the area is detected.

In the example, the rectangular region including the face region was defined based on the content disclosed in the paper “Robust Real-Time Face Detection Using Face Certainty Map” (Volume 4642 of the series Lecture Notes in Computer Science pp 29-38).

The fraud detection server 100,

In the frame-by-frame image information taken by the examinee, a rectangular area including the examinee's face area is defined, and the shape and contrast of both eyes are image-processed in a coordinate system based on the rectangular area, and both end points (310) of both eyes. , 320), set the center points 350 and 360 of the black regions 330 and 340 located between them as the pupil coordinates of both eyes, and calculate the average value of the pupil coordinates of the examinee for a certain period of time before the start of the test to obtain a reference pupil coordinate value, and when the test starts, the average value of the pupil coordinates of both eyes of the examinee is calculated at a predetermined time unit to obtain the average value of the real-time pupil coordinates, and the real-time average value of the pupil coordinates is calculated based on the reference pupil coordinates. If it is out of the set error range, it is notified to the supervisor terminal,

The center point 370 of the pupil coordinates of both eyes is determined as the intermediate coordinate,

The movement direction and movement distance of the intermediate coordinates are tracked on a frame-by-frame basis, and when the movement distance of the intermediate coordinates is out of a preset error range, the examinee's head rotation direction or movement direction is notified, and frame-by-frame image information of the examinee If there are two or more face regions obtained from

It is characterized in that the background is set for all areas except the body and face areas of the examinee, and when a change in the background is detected during the test time, it is notified to the supervisor terminal,

The fraud detection server 100,

When the examinee's image information is received through the communication network and it is determined that cheating is suspected, a fraudulent detection message is sent to the supervisor terminal 130, and the online test question for the examinee is taken. to the online evaluation server 140 to stop the online test.

When the online evaluation server 140 receives a message from the cheating detection server 100 indicating that a suspected cheating operation has been detected while taking the questions of the online test taken by the examinee, the online test It is characterized in that after stopping the question-taking, and after receiving a message from the supervisor terminal 130 or the cheating detection server 100 that the question can be restarted, the online test question-taking is restarted.

Recently, in face recognition technology, head/eye/nose/mouth features using HOG, LBP, and Gabor feature extraction algorithms, which are the mainstream hand-crafted features, are deep learning-based features such as face detection, CNN Using the facial landmark detection and facial feature extraction and classification technology using the (Convolutional Neural Network) algorithm, the human recognition level provides results similar to 97.53%.

In 2014, AI facial recognition technologies such as Facebook's DeepFace were launched. In the case of DeepID1 ["Deep Learning Face Representation from Predicting 10,000 Classes," CVPR 2014 ] announced at CVPR, it uses 39x31 pixels, which is smaller than the 152x152 pixels of DeepFace's input resolution.

In addition, the face detector based on AlexNet and AlexNet trained in advance with ImageNet training data are fine-tuned with face images, so the detection performance is not yet high. are doing

Patent Publication No. 10-2017-0050465 (published on May 11, 2017), "Face Recognition Device and Method", SK Telecom Co., Ltd. Patent Registration No. 10-1765770 (registration date August 01, 2017), "A system and method for detecting cheating in an online test using image processing and a recording medium storing a program for implementing the same", Bukyung Son

An object of the present invention to solve the above problems is a tablet PC, a smartphone equipped with an artificial intelligence facial recognition module and an eye/nose/ear facial feature point 5-point scale anti-fraud module in the test taker terminal during the online test and UBT test , in PC-based online tests and UBT tests, AI-based facial contour recognition technology (posenet algorithm) is used to prevent proxy tests or cheating to detect the object of the test taker's face in the test taker's terminal, and the facial contours and eyes2 /Nose/Ear 2 feature points are extracted, and the head movement to the right/left is detected and tracked according to whether the 3 eye/nose points are closer to the 2 ear points at both ends by the anti-cheating module to detect facial behavior patterns. An intelligent face recognition module and an anti-fraud module on a 5-point scale of facial feature points are installed. If the distance in the direction of the eye/ear distance and nose/ear distance exceeds a certain value depending on whether the end ear approaches 2 points), a warning message or alarm is output or stored in the information device of the test taker terminal and the test is completed when the test is finished Provides a tablet PC, smartphone, and PC-based UBT system using facial contour recognition artificial intelligence that prevents cheating, which transmits to a server and provides scoring results to test takers.

Another object of the present invention is to provide a tablet PC, smartphone, PC-based UBT providing method using facial contour recognition artificial intelligence.

In order to achieve the object of the present invention, the UBT system using facial contour recognition artificial intelligence to prevent cheating is to register test taker information and a frontal face picture, issue a QR code, and conduct an online test or UBT test. Online exams or UBT exam servers that provide exam programs; It is connected to the online test or UBT test server through a wired/wireless communication network, and an artificial intelligence facial recognition module of the face photographed through the front camera of the test taker terminal, and the facial behavior pattern photographed with the front camera to configure the feature points of the face a test taker terminal having a 5-point scale anti-fraud module for eyes2, nose, and ears2; and a proctor terminal that receives face photos and cheating information of a plurality of test taker terminals through the online test or UBT test server during an online test or UBT test, and transmits a fraud prevention alarm or warning message to the test taker terminal do.

In order to achieve another object of the present invention, the UBT providing method using facial contour recognition artificial intelligence to prevent cheating is to register member information, log in/user authentication, Registering as a server and storing; issuing, by the online test or UBT test server, a QR code corresponding to test taker information and a frontal face picture; notifying, by the online test or UBT test server, a test schedule and location according to the subjective test; Using the front face recognition algorithm in the test taker terminal having a face recognition module using AI-based facial contour recognition technology, the result of the recognition of the front face photo of the camera is received by the online test or UBT test server, and the proctor terminal receives the test taker information and comparing the photos to confirm whether the test taker is the test taker, preventing a proxy test, and determining whether to take the test; The facial recognition module of the test taker terminal and the 5-point scale cheating prevention module recognize the outline of the test taker's face and the eye2/nose/ear2 feature points, and each of the eye/nose/ear face features left/right ear and left/ Calculate the Euclidean distance (d) and similarity with the center point (pupil) of the right eye, and detect the head movement to the right/left according to whether the 3 eye/nose points get closer to the 2 points on both ends of the face Detects abnormal behavior patterns of , when facial contours are not recognized during face recognition, when the test screen is out of the test screen of the test taker terminal If the distance in the corresponding direction of the ear distance exceeds a certain standard value) to prevent cheating, the face photo or test taker's voice data related to cheating is transmitted to the online test or UBT test server and output to the proctor's terminal, Receiving a warning message or an alarm from the server to the candidate terminal; and a test program (App) and an online test paper to the test taker terminal and the proctor terminal, and stores and manages test taker information, a photo of the test taker's on-site face, and proctor information in a database of the test server, when performing an online test or UBT test After storing the answers written on the test paper in each test taker's terminal for a certain test time, receiving and storing the answers from the test taker's terminal to the test server at the end of the test, and providing the scoring results of the test takers' written answers to the test taker's terminal.

The online system and method using facial contour recognition artificial intelligence to prevent cheating of the present invention is a tablet PC, smartphone, In PC-based online tests and UBT tests, in non-face-to-face tests, facial recognition and facial behavior patterns were detected to prevent proxy tests or cheating.

In the test taker terminal, we implemented a proxy test prevention and cheating prevention technique that combines an artificial intelligence facial recognition module and a cheating prevention module using a 5-point scale of eye/nose/ear facial feature points.

The AI-based facial contour recognition technology used in the face recognition module of the test taker terminal and the anti-fraud module using a 5-point scale of eye/nose/ear facial feature points uses the posenet algorithm to recognize the facial contour of the camera image of a tablet PC. During the online test or UBT test, AI-based facial recognition technology is used to identify the tablet PC test taker through the proctor terminal linked to the test server, and to prevent proxy tests. It is effective in preventing proxy testing or cheating.

1 is a block diagram of an existing face recognition device.
2 is a block diagram of an online evaluation system including a conventional online test cheating detection system.
3A is a diagram showing the concept of an artificial intelligence platform for recognizing facial contours having a tablet PC, a smartphone, and a PC-based test online test/UBT test server during the language and health education online test/UBT test.
3B is a diagram showing a face recognition function on the ubiquitous-based learning (UBL) and ubiquitous-based testing (UBT) NSDAI platform.
3c and 3d show the goals and User Experience (1. User registration-> 2. Learning-> 3. QR code of UBI cloud App, NS facere platform used in tablet PCs, smartphones, and PCs during the online test/UBT test. Create -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor confirmation and test-> 6.7 UBT App/Web camera’s facial contour recognition 8. Test end) to be.
Figure 4 shows the process of test taker registration from test taker registration, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) supervisor confirmation (taker face / test taker information), 6) test Gaze - This is a diagram showing the process.
5 is a screen showing the candidate registration/learning/QR code issuance screen of the candidate-face matching code issuance unit - online exam/UBT exam based on the registered candidate face machine learning and learning results.
6 is a screen of the test taker confirmation unit in the UBT system in which similarity is measured through AI-based facial contour recognition of the test taker's face.
7 is a view showing a rule user defining unit and a program test screen when determining a candidate.
8 shows test server access/login/left- QR code recognition/QR code recognition during online test or UBT test, greeting sound (TTS)/right-face recognition (smartphone/tablet PC front camera)/face in the background This is a demonstration screen including the area recognition start/recognition rate display process - the UBT authentication process user definition unit using the generated user code.
9 is a block diagram of a tablet PC, smartphone, and PC-based online test system using facial contour recognition artificial intelligence to prevent cheating according to the present invention.
10 is a diagram illustrating the functions of an AI-based facial recognition module and an anti-fraud module during an online test or UBT test.
11 is a flowchart illustrating a tablet PC, smartphone, and PC-based online test providing method using facial contour recognition artificial intelligence to prevent cheating according to the present invention.

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings, the configuration and operation of the invention. In the description of the present invention, if it is determined that a detailed description of a related known function or known configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, when the accompanying drawing numbers indicate the same configuration, the same reference numbers are given in different drawings.

9 is a block diagram of a tablet PC, smartphone, and PC-based online test system using facial contour recognition artificial intelligence to prevent cheating according to the present invention.

10 is a diagram illustrating the functions of an AI-based facial recognition module and an anti-fraud module during an online test or UBT test.

The UBT system and method using the facial contour recognition artificial intelligence of the present invention is an online test based on a tablet PC, a smartphone, and a PC equipped with an artificial intelligence facial recognition module and an anti-fraud module on a 5-point scale of eye/nose/ear facial features and UBT exams, using AI-based facial contour recognition technology (posenet algorithm, machine learning model which allows for real-time face pose estimation) to prevent surrogate exams or cheating. It extracts facial objects, facial outlines, and eye2/nose/ear2 feature points, and extracts facial behavior patterns in real time to prevent cheating. An artificial intelligence facial recognition module that detects and tracks the movement of the head to the right/left and detects the behavior pattern of the face according to whether the 3 eye/nose points of the face get closer to the 2 points of the ears at both ends by the anti-cheating module A five-point scale anti-fraud module is installed, and when facial contours are not recognized during face recognition through FACE RECOGNITION/FACE MOTION RECOGNITION/RESULT ANLAYSIS If the distance between the eyes/ear distance and nose/ear distance in the corresponding direction exceeds a certain value depending on whether the 3 nose point approaches the 2 ear points on both ends, a warning message or alarm is output or stored in the information device of the test taker terminal After the test is finished, it is sent to the test server and the scoring results are provided to test takers.

In the test taker terminal, we implemented a proxy test prevention and cheating prevention technique that combines an artificial intelligence facial recognition module and a cheating prevention module using a 5-point scale of facial feature points.

The AI-based facial contour recognition technology used in the face recognition module of the test taker terminal and the anti-fraud module using a 5-point scale uses the posenet algorithm to recognize the facial contour of the camera image of the tablet PC. During the online test or UBT test, AI-based facial recognition technology is used to identify the tablet PC test taker through the proctor terminal linked to the test server, and to prevent proxy tests. Avoid proxy testing or cheating.

The test paper questions of the test program (App) are displayed with multimedia test questions including text, images, VR/AR contents, audio and video as well as text and images for each question.

1. Artificial intelligence hub platform

NSD-AI (NSDevil's Artificial intelligence) 플랫폼 기반 인공지능 허브플랫폼

NSD-AI (NSDevil's Artificial intelligence) platform-based artificial intelligence hub platform

의학교육 부문 구문 인식을 통한 교수자 채점 가이드(인제대학교 의과대학 외 공동연구)

Instructor scoring guide through phrase recognition in medical education sector (joint research with Inje University College of Medicine and others)

사물인식을 통한 체험학습 지원시스템(펀에듀랩 및 한국민속촌 외)

Experiential learning support system through object recognition (FunEdu Lab and Korean Folk Village, etc.)

치아 이미지 분석을 통한 치과교육 부문 질병분석 기능(연세대학교 치과대학 외 3개국 기관 공동연구)

Disease analysis function in the dental education sector through tooth image analysis (joint research with Yonsei University College of Dentistry and other institutions in 3 countries)

UBT connect platform 기반

Based on UBT connect platform

태블릿 PC 기반 평가 플랫폼

Tablet PC-based evaluation platform

어학평가 부문(말하기/듣기/읽기/쓰기)

Language Assessment (Speaking/Listening/Reading/Writing)

보건의료평가 부문(듣기/읽기/쓰기)

Health care evaluation sector (listening/reading/writing)

기타 직무평가 부문(읽기/쓰기)

Other job evaluation areas (read/write)

Tablet PC, smartphone, and PC-based UBT system using artificial intelligence facial contour recognition

Candidate terminals 300, 310, 311 using an online test or UBT test server 100, a proctor terminal 200, and a PC and a smartphone and a tablet PC, and having a face recognition module and a fraud prevention module using a 5-point scale of facial features ) is included.

UBT system using facial contour recognition artificial intelligence to prevent cheating

It registers and stores test taker information and frontal face photos, issues a QR code, provides a greeting through TTS conversion technology after recognizing each test taker QR code, and provides an online test paper and a test program to conduct an online test or UBT test an online test or UBT test server 100;

It is connected to the online test or UBT test server 100 through a wired/wireless communication network (LAN, Wi-Fi, LTE 4G/5G), and an artificial intelligence facial recognition module of the face photographed through the front camera of the test taker terminal, and the front car Candidate terminals 300 , 310 , 311 having a five-point scale anti-fraud module of eyes 2, nose, and ears 2 that recognizes facial behavior patterns photographed by Mare and constitutes facial feature points; and

It is connected to the online test or UBT test server 100 through a wired/wireless communication network, and receives face photos and cheating information of a plurality of test taker terminals through the online test or UBT test server 100 during the online test or UBT test. and a supervisor terminal 200 that receives and transmits an anti-fraud alarm or message to the corresponding test taker terminal.

The test taker terminals 300, 310, and 311 use any one of a tablet PC, a smartphone, and a PC on which the test program (App) is installed, the test program is installed, the artificial intelligence facial recognition module and the facial feature point 5-point scale to prevent cheating module is provided.

The test taker terminal includes a face recognition module and a fraud prevention module, and the cheating prevention module includes a visual cheat detection module and a voice recognition module of the camera vision image data of the AI-based visual test taker terminal in real time.

The cheating prevention module further includes a voice recognition module for recognizing the test taker's speech in the test taker terminal to prevent the test taker's cheating. The online test or UBT test server transmits and stores cheating face photos and sound waveform data, and transmits them to the proctor terminal 200 so that the proctor sends an alarm to the test taker terminal through the server to prevent the test taker terminal from cheating. to occur or to send a message to prevent cheating.

The AI-based facial contour recognition technology used in the facial recognition module uses a posenet algorithm (machine learning model which allows for real-time face pose estimation).

In the embodiment, in the online test or UBT test, the test taker terminal used a tablet PC.

The test taker terminals 300 , 310 , and 311 are provided with a face recognition module and an anti-cheating module on a 5-point scale of eye/nose/ear facial feature points.

The test taker terminals 300, 310, and 311 detect the ROI of a face image captured in real time with the front camera of the test taker terminal during an online test or UBT test, and perform a front face recognition algorithm for a standard-sized face picture with size correction/angle correction. The facial object is extracted using a facial object and facial behavior pattern recognition (pattern recognition) to extract the outline of the candidate's face and the eye/nose/ear feature points. / Calculate the Euclidean distance (d) and similarity between the center point (pupil) of the right ear and the left/right eye, and compare it with the face photo DB of the online test or UBT test server 100 to determine whether a proxy test During the online test or UBT test, by recognizing facial behavior patterns captured by the candidate's terminal camera, the distance between the right eye and the right ear and the distance between the left eye and the left ear change depending on the angle of the front face turned left and right. is determined and transmitted to the server 100, the supervisor terminal linked to the server 100 confirms, and an alarm or warning message is generated to the candidate terminal.

The test taker terminals 300, 310, and 311 use AI-based facial contour recognition technology (posenet algorithm) for the camera image by means of a facial recognition module and an eye/nose/ear facial feature point 5-point scale anti-fraud module. Detects head movement to the right/left depending on whether the end ear approaches 2 points, detects abnormal behavior patterns of the face related to cheating, when facial contours are not recognized during face recognition, or when the test screen deviates from the test screen of the test taker terminal (If the distance in the direction of the distance between the eyes/ears and the distance between the nose/ears exceeds a certain threshold depending on whether the 3 eye/nose points get closer to the 2 ear points on both ends), a warning message or alarm is output to prevent cheating.

(Example)

TensorFlow was used for estimating human face poses on Android on tablet PC using PoseNet model. For reference, the PoseNet algorithm is a vision model that detects the location of major body parts and estimates a human pose from an image or video. TensorFlow Lite provides an Android sample application that utilizes the candidate's device's camera to detect and display major human body parts in real time. This model can estimate the position of a person's elbow and/or knee from the image, and in the embodiment, the facial contour pose estimation model was implemented by identifying the eyes/nose/ears in real time during face recognition.

The posebet algorithm used in the artificial intelligence facial recognition module uses a 5-point scale anti-cheating module for left eye (leftEye), right eye (rightEye), nose (nose), left ear (leftEar), and right ear (rightEar). implemented,

After obtaining the x,y coordinates of leftEar and the x,y coordinates of rightEar, you can find the diameter of the circle based on these and make an angel ring with ellipse(). The diameter can be easily obtained with dist() in p5js without having to use the Pythagorean theorem.

<Connecting ml5js and webcam>

<script src="https://unpkg.com/ml5@0.3.1/dist/ml5.min.js"></script>

let video;

let poseNet;

function setup() {

createCanvas(400, 400);

video = createCapture(VIDEO);

//Use to prevent the video from appearing twice

video.hide();

poseNet = ml5.poseNet(video,modelLoaded);

poseNet.on('pose',gotPoses);

console.log(ml5);

}

// Pose related callbacks are entered

function gotPoses(poses) {

}

// Used to check if the loading was successful

function modelLoaded() {

console.log('Model Loaded');

}

function draw() {

// Draw the webcam at position 0,0.

image(video,0,0);

}

The face area is extracted in real time from the front camera face image of the test taker terminal, adjusted to the standard size of the face photo through size correction/rotation/angle correction, and compared with the learning data of the pre-stored standard size face photo DB of the test server - > Face photo learning model -> Extract and classify the feature points of the contours of eye2/nose/ear2, and group clustering of the contours of the face recognition data and contours of eye2/nose/ear2 (center k of the cluster) -means algorithm) density estimation, calculating the distance (Euclidean distance) and similarity between facial feature points A proxy test is prevented by comparing the degree of similarity and checking the candidate's photo himself, and cheating is judged during the online test or UBT test.

For example, if the direction of the face photographed by the front camera of the test taker terminal changes to the test screen of the test taker terminal (tablet PC), or if the test taker turns the face, the feature points of the eye/nose/mouth outlines in the face photo may not be recognized or If the angle toward the test program changes within the effective distance (20-30 cm) between the user's face and the smartphone, it is recognized as cheating, and the cheating detection result is transmitted to the online test or UBT test server 100, This is transmitted to the proctor terminal 200 so that the supervisor generates an alarm or transmits a message to the test taker terminal through the server to prevent the test taker's terminal from cheating, thereby preventing the test taker's terminal from cheating.

or. [Text A] Data reference The moment the warning threshold set in the predefined candidate program becomes zero, a warning message is displayed to the candidate without a server connection, and can be sent back to the proctor/cheating detection server/other exam administration server, or the exam After completion, the RAW data is sent to the cheating detection server/other test management server for post-processing.

In addition, the test taker terminal includes a face recognition module and a cheating prevention module, and the cheating prevention module further includes a voice recognition module for recognizing the voice of the test taker to prevent the test taker's cheating in the test taker terminal, Alternatively, if a test taker's voice related to cheating is heard during the UBT test, the voice is recognized and immediately transmitted to the online test or UBT test server for storage, and it is transmitted to the proctor's terminal 200 and sent to the proctor In order for the candidate terminal to prevent cheating, an alarm is generated or a message is sent to the candidate terminal through the server to prevent cheating. Alternatively, by referring to the thresholds of defined distances such as eye/ear distance, nose/ear distance, etc., a warning message is displayed from the test taker program directly to the test taker terminal without a server command, and the proctor terminal or cheating detection server, or other test management send to the server

Online test or UBT test server 100 is WWW server 101, control unit 102, member registration unit 103, user authentication unit 104, test notification unit 105, QR code management unit 106, greetings Study (TTS) 107 , supervisor confirmation unit 108 , face recognition unit 109 , cheating processing unit 110 , test management unit 111 , exam information DB 120 , test taker DB 121 , and face Includes a photo DB (123).

The online test or UBT test server 100 is

a WWW server 101 connected to a test taker terminal and a supervisor terminal using any one of a tablet PC, a smartphone, and a PC through wired/wireless communication;

a control unit 102 for controlling to provide application services of online or ubiquitous-based learning (UBL) and ubiquitous-based testing (UBT) to tablet PCs, smartphones, and proctor terminals of tablet PCs, smartphones, and PCs through wired and wireless communication networks;

a member registration unit 103 connected to the control unit 102, receiving member information of test takers, storing and managing ID/Paaswd;

a user authentication unit 104 connected to the control unit 102 and authenticating a user using a QR code/Passwd or ID/Passwd;

A test notification unit that is connected to the control unit 102 and provides a schedule and location of a tablet PC, a smartphone, a PC-based online learning (Learning) and test, or a ubiquitous-based online learning (UBL) and test (UBT) schedule ( 105);

a QR code management unit 106 connected to the control unit 102, issuing a QR code corresponding to test taker information to test takers, and managing the QR code;

After recognizing the QR code, a greeting providing unit (TTS) 107 that provides a greeting through TTS conversion technology;

It is connected to the control unit 102, receives a photograph of the candidate from the camera from the candidate terminal, compares the candidate information stored in the database with the candidate photograph, and the supervisor confirms whether the examination is a proxy test and confirms the qualification to take the examination a supervisor confirmation unit 108;

A face recognition unit ( 109);

It is connected to the control unit 102 and uses the facial contour recognition technology (posenet algorithm) for the front camera image by the facial recognition module and the cheating prevention module in the test taker terminal, so that 3 eyes/nose points are close to 2 points on both ends of the ears Detects head movement to the right/left depending on whether or not you lose, and detects abnormal behavior patterns of the face related to cheating by viewing cheating paper. If the distance in the corresponding direction of the distance between the eyes/ears and the distance between the nose/ear exceeds a certain threshold depending on whether the dot gets closer to the 2 points on both ends of the ear, a warning message or alarm is output to prevent cheating, a warning is issued from the test taker terminal Cheating unit 110 for receiving a message or an alarm;

It is connected to the control unit 102, provides a test program (App) and test papers to the test taker terminal and the proctor terminal, manages test taker information, the test taker's on-site face photo, and proctor information, and when taking an online test or UBT test A test management unit 111 that stores and manages test paper written answers in each test taker's terminal within a certain test time, and is transmitted to the test server at the end of the test, and stores and manages test takers' written answers, scoring results, supervisor information and test taker status information;

Test information DB 120 for storing test takers' test papers, written answers, and scoring results; It includes a candidate DB 121 and a face DB 123 for storing candidate information and a standard-sized frontal face photograph.

Candidate terminals include smart phones and tablet PCs for an online test (Internet Based Test) or ubiquitous-based test (UBT), as well as a mobile communication terminal capable of accessing the Internet, and a laptop computer.

The test taker terminal uses any one of a tablet PC, a smartphone, or a PC in which the test program (App) is installed, and the artificial intelligence facial recognition module of the camera image of the test taker terminal and the facial feature points of the eyes/nose/ears 5 point scale cheating The prevention module is provided, and the test program (App) downloaded from the online test or UBT test server 100 is installed,

The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm.

The test taker terminal includes a face recognition module and a fraud prevention module, and the cheating prevention module includes a real-time AI-based visual cheat detection module and a voice recognition module of the camera vision image of the test taker terminal.

Online test or UBT test server 100 provides medical school/dental college/pharmacy college/engineering college exams, TOEIC/TOEFL exams, language exams, civil service exams, certification exams, health and medical education learning and exam papers The bank announces the test schedule and location of various accredited certification tests or non-accredited tests of the bank, and uses the test program in the test server's database to give online test or offline test questions to test takers' terminals through wired and wireless communication networks to take the test. UBT test using all possible types of PC/smart devices is provided.

When the test schedule and location are confirmed, the controller 102 appoints a proctor, provides the proctor appointment information to the proctor, and notifies test takers of the test schedule and location through text message/webpage, and the proctor of the test site on the day of the test The terminal 200 transmits and receives test information and test paper information to conduct and manage the test, and provides a scoring result from the results of the automatic scoring result unit 124 and the inspection management unit 125 to the corresponding candidate terminals.

The test information stored in the test information database 120 includes test title, test schedule and location, test time, location information of test location, assigned proctor information and test takers list, number of seats per test room, proctor information, A/ Includes question information, answer information, scoring information, and supervisor information on each type of test paper.

In addition, the candidate database 121 includes candidate information including name, resident registration number, home address, mobile phone number, e-mail, etc., a photograph of the candidate (standard-sized face-front photograph), QR code/passwd, ID/passwd identification information, and test Relevant information and candidate application status are saved.

11 is a flowchart illustrating a tablet PC, smartphone, and PC-based online test providing method using facial contour recognition artificial intelligence to prevent cheating according to the present invention.

The tablet PC, smartphone, and PC-based online test method using facial contour recognition artificial intelligence to prevent cheating is

After receiving member information registration and login/user authentication, the online test or UBT test server registers and stores test taker information and a frontal face photograph (S10);

Issuing a QR code corresponding to the candidate information and the front face photograph (S20);

The online test or the UBT test server notifying the test schedule and location according to the subjective test (S30);

Taking a QR code with a smartphone and placing it on the recognition screen on the camera of the test taker terminal (PC, tablet PC, etc.) is operated, and after recognizing the QR code, providing a greeting through a TTS (Text To Speech) function (S40);

Using the frontal face recognition algorithm in the candidate terminal having a face recognition module using AI-based facial contour recognition technology (posenet algorithm), the proctor terminal receives the recognition result of the camera's frontal face photo from the online test or UBT test server. A step (S50) of determining whether or not to take a test while preventing a proxy test by verifying whether the test taker is the test taker by comparing the test taker information, the photograph, and its characteristic points (S50);

The face recognition module of the test taker terminal recognizes the facial behavior pattern photographed by the front camera of the test taker terminal, extracts the face object, extracts the facial feature points on a 5-point scale of eyes/nose/ear, and the outline of the test taker's face and eyes2 Recognizes /nose/ear2 feature points, calculates the Euclidean distance (d) and similarity between the left/right ears and the center point (pupil) of the left/right eyes of the eye/nose/ear face feature points, respectively, , detects head movement to the right/left according to whether the 3 eye/nose points get closer to the 2 ear points at both ends to detect a pattern of facial abnormal behavior related to cheating. In case of deviation from the test screen of the terminal (if the distance in the corresponding direction of the eye/ear distance and the nose/ear distance exceeds a certain standard depending on whether the 3 eye/nose points get closer to the 2 ear points on both ends), denial to prevent cheating Transmitting the face photo and/or the test taker's voice data related to the action to the online test or UBT test server, outputting it to the proctor terminal through the server, and receiving a warning message or an alarm from the server to the test taker terminal (S60) );

The test program (App) and online test papers are provided to the test taker terminal and the proctor terminal, and the test taker information, the test taker's on-site face photo, and the proctor information are stored and managed in the test server's database. After storing the test paper written answers in each test taker terminal for a period of time, when the test is finished, the test paper is transmitted from the test taker's terminal to the test server and stored (S70), and the scoring results of the test takers' written answers are provided to the test taker's terminal (S80). .

The candidate terminal uses the posenet algorithm to recognize the facial contours of the camera image of the tablet PC as an artificial intelligence (AI)-based facial contour recognition technology. During an online test or UBT test, AI-based facial recognition technology is used to identify the tablet PC test taker through the proctor's terminal linked to the test server, prevent proxy tests, and facial recognition of the test taker's face online or at the UBT test center It can be used as a control system and prevents proxy tests or cheating in online and UBT tests.

The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which the test program (App) is installed, the test program is installed, and the artificial intelligence facial recognition module and the eye/nose/ear 5-point scale cheating A prevention module is provided.

The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm.

The test taker terminals 300, 310, and 311 are provided with a face recognition module and an anti-cheating module on a 5-point scale of eye/nose/ear facial feature points,

During the online test or UBT test, the ROI of the face image captured in real time with the front camera of the test taker's terminal is detected, and the face behavior pattern is recognized by using the front face recognition algorithm for the size and angle corrected standard size face photos. The outline of the candidate's face and eye/nose/ear feature points are extracted, and the center point (pupil) of the left/right ear and left/right eye of the facial feature points on a 5-point scale of eyes/nose/ear through feature extraction and classification, respectively. Calculates the Euclidean distance (d) and similarity of Since the distance between the right eye and the right ear and the distance between the left eye and the left ear vary depending on the angle turned left and right, the misconduct is determined and transmitted to the server 100, and the supervisor terminal linked to the server 100 confirms and confirms the candidate Generates an alarm or warning message to the terminal.

Candidate terminals (300, 310, 311) are connected to the online test or UBT test server 100 through a wired/wireless communication network (LAN, Wi-Fi, LTE 4G/5G), the artificial intelligence facial recognition module and the eyes 2, Equipped with a five-point scale anti-corruption module for nose and ears, it detects the ROI (Region of Interest) of the face image captured in real time with the front camera of the test taker terminal during the online test or UBT test, based on the vertex of the nose Creates a standard size face photo with size correction/angle correction with Obtain the feature vector x of the feature points, calculate the Euclidean distance (d) and similarity of the feature points of the face through feature extraction and classification of eyes 2, nose, and ear 2, and face the online test or UBT test server It is compared with the photo DB to check whether there is a proxy test, and during an online test or UBT test, cheating is judged according to the angle of the front face taken with the camera of the test taker's terminal turned left and right, and the cheating information of the face photo is sent to the server. When the supervisor terminal confirms the fraudulent information, it generates an anti-fraudulent alarm received from the server or outputs a message.

The test taker terminal uses the facial contour recognition technology (posenet algorithm) for the camera image by the facial recognition module and the cheating prevention module to move the head to the right/left according to whether the 3 eyes/nose points get closer to the 2 points on both ends of the ears Detects and detects abnormal behavior patterns of faces related to cheating. If the distance in the corresponding direction of the ear distance and nose/ear distance exceeds a certain standard value (value), a warning message or alarm is output to prevent cheating.

It is operated by taking the QR code with a smartphone and putting it on the recognition screen of the candidate terminal's camera, and after recognizing the QR code, further comprising the step of providing a greeting through a TTS (Text To Speech) function.

The test paper questions of the test program are displayed with multimedia test questions including text, images, VR/AR contents, audio and video as well as text and images for each question.

How to provide an online test using facial contour recognition artificial intelligence,

The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which the test program (App) is installed, the test program is installed, and the artificial intelligence facial recognition module and eye/nose/ear facial features 5 points A scale anti-fraud module is provided.

The test taker terminal includes the face recognition module and the cheating prevention module, and the cheating prevention module includes a visual cheating detection module and a voice recognition module of the camera vision image data of the AI-based visual test taker terminal in real time and

The voice recognition module is provided with a voice recognition module for recognizing the voice of the test taker in order to prevent cheating of the test taker in the test taker terminal. The test taker's voice data input through the microphone is immediately transmitted and saved to the online test or UBT test server, and it is transmitted to the proctor's terminal so that the proctor sends an alarm to the test taker's terminal through the server to prevent the test taker's terminal from cheating. generating or sending a message.

The test taker terminal detects the ROI of a face image captured in real time with the front camera of the test taker terminal during a non-face-to-face online test or UBT test, and uses a front face recognition algorithm for a standard-sized face photo with size correction/angle correction. By extracting face objects and recognizing facial behavior patterns, the outline of the candidate's face and eye/nose/ear feature points are extracted, and through feature extraction and classification, the left/right ears and left/right Calculate the Euclidean distance (d) and similarity to the central point (pupil) of the eye, compare it with the face photo DB of the online test or UBT test server, and check whether it is a proxy test. Recognizes facial behavior patterns photographed with the camera of the test taker terminal, and the distance between the right eye and right ear and the distance between the left eye and the left ear vary depending on the angle of the front face turned left and right, so it determines cheating and transmits it to the server, The supervisor terminal linked to the server checks and generates an alarm or warning message to the candidate terminal.

In addition, the method stores the user's face movement of the camera image of the test taker terminal in frame units like [Text A], and displays a warning message to the test taker terminal when it becomes 0 after subtracting a defined frame or time or number of times. / After exposure or sending the supervisor device or cheating detection image to the server, reset the number of warning triggers to start subtracting from the next movement.

[Text A] - Facial behavior pattern information stored in the tablet PC of the test taker's terminal (e.g., the face pattern data of the Seoul National University Dental College): information such as date, time, movement angle/distance and phase (left and right) It is saved every frame or hourly, and based on this, a warning message is displayed to the user, and the proctor is checked and the test taker's face recognition fraud is detected. handle

200519094106,0.02,R

200519094111,0.03,R

200519094120,0.42,R

200519094148,0.01,R

200519094150,0.21,R

200519094155,0.22,R

200519094158,0.09,R

200519094209,0.11,L

200519094214,0.09,R

200519094216,0.10,R

200519094219,0.09,R

200519094224,0.01,L

In an embodiment, when the test taker accesses the test server and applies for the online or UBT test, the test application reception process for performing the test application of the test taker to the test server; a test information announcement process of notifying test takers of the test schedule, time and location information including test schedule, time and location information to test takers of the test when the test schedule, time, and location are determined; When the test location and time are set, set the date and time of the set test and the examiner for each laboratory in the location, provide the supervisor appointment information notifying the supervisor of the appointment, and receive the appointment confirmation information from the examiner to the test server to complete the supervisor setting process Supervisor setting process; On the day of the test, the test site proctor waits for input of test site setting information, and when the test site setting information from the proctor is input, the test program (application, App) is provided and installed on the test takers’ smartphones, and the test takers’ smart phone The test site setting process that requests test access confirmation from test takers connected through the test program installed in the phone. When the test site setting is completed, the registered and saved enforcement is performed by referring to the set question type and problem selection setting information (difficulty level). The test proceeding process of selecting any one of the list of questions to be asked, communicating with the installed test program of the test takers to provide test questions, and the test taker terminal receiving the answer information from the test paper through the test program to the test server and storing it for each test taker; A test finishing process in which all test questions are asked and the application program installed on the test takers' tablet PCs is automatically deleted when the proctor's termination confirmation information is inputted, and when the proctor's termination confirmation information is inputted when the test is finished; It consists of a test presentation process in which the answers written by each test taker received and stored by the test server are compared with the scoring answer information and scored, and the scoring result information is announced.

Appointment of proctors is selected from pre-registered proctors, and these proctors are pre-contracted proctors, and they are registered and managed in the test information database of the test server.

When a supervisor is appointed, the supervisor appointment information is provided through the test server and the appointment confirmation information is received from the supervisor to complete the supervisor setting process.

Proctor appointment information includes test location, test time, and number of test takers.

The number of test takers information includes each test taker's photo, name, gender, and personal information including address, and contact information.

The test setting process is a process to check the parts necessary for the test before the pre-set preparation time on the day of the test and to set the device for the test.

When the test center sends test takers, the supervisor checks the test taker information and face picture, and enters the test site setting information, which is the confirmation information, the test server provides the test program (App) necessary for the test, and the test taker can download the test program to the tablet PC It is the process of completing the test access confirmation of test takers by inputting the identification information that the test takers entered when they entered the application information by sending and receiving data through the test program.

The test setting information of the proctor includes information on the number of test takers who participated in the test, and the identification information of test takers consists of any one of personal information such as name and resident registration number or member ID entered by test takers when entering application information.

The test program (App) consists of a process of accessing the system and inputting identification information to download and install the test program, or when the supervisor inputs test setting information in the system, the supervisor It may include a process of providing a path and allowing test takers to install the test program according to the path.

3A is a diagram showing the concept of an artificial intelligence platform for recognizing facial contours having a tablet PC, a smartphone, and a PC-based test online test/UBT test server during the language and health education online test/UBT test.

3B is a diagram showing a face recognition function on the ubiquitous-based learning (UBL) and ubiquitous-based testing (UBT) NSDAI platform.

3c and 3d show the goals and User Experience (1. User Registration->2. Learning-> 3. QR Codes of UBI cloud App, NS facere platform used in tablet PCs, smartphones, and PCs during the online test/UBT test) Create -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor check and test-> 6.7 UBT App/Web camera’s facial contour recognition 8. Test end) to be.

Figure 4 shows the process of test taker registration from test taker registration, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) proctor confirmation (taker face / test taker information), 6) test Gaze - This is a diagram showing the process.

5 is a screen of the candidate registration/learning/QR code issuance screen of the candidate-face matching code issuance unit - online exam/UBT exam based on the registered candidate face machine learning and learning results.

This is the screen of candidate registration/learning/QR code issuance during the online test/UBT test.

6 is a screen of the candidate confirmation unit in the UBT system in which similarity is measured through AI-based facial contour recognition of the test taker's face.

7 is a view showing a rule user defining unit and a program test screen when determining a candidate.

8 shows test server access/login/left- QR code recognition/QR code recognition during online test or UBT test, greeting sound (TTS)/right-face recognition (smartphone/tablet PC front camera)/face in the background It is a demonstration screen including the area recognition start/recognition rate display process - the UBT authentication process user definition unit using the generated user code.

(1) The test taker terminal is equipped with a proxy test prevention method that combines an artificial intelligence facial recognition module and a cheating prevention module using a 5-point scale and preventing cheating during the exam

The artificial intelligence that has learned the candidate registration photo generates a recognition code (QR code) for each user, and the test taker shines the examination ticket with the identification code (QR code) attached to the camera of the exam room PC/tablet PC, and then the AI recognized by the user Comparing the test taker information and the result model learned from the picture, calculating the similarity of the feature points of face recognition, if a certain number exceeds a certain number, it is recognized as the person, moves to the test screen, and takes the test. In the case of not reaching a certain level (completely underachieving, withholding judgment), the test is conducted after passing the information to the supervisor's device and verifying the personal information of the supervisor. During the test, the artificial intelligence of the face recognition module or the facial contour / 5 point scale recognition anti-fraud module allows the program to exist in the background or on another layer under the test screen or at the same coordinate value location as the test screen, but with a camera attached or built-in to the device. Recognizes the user until the end of the test using Information is saved as an image or text/numerical and transmitted to the test server after the test is completed to support the decision of the test supervisor/in charge of the test as to whether or not to take a proxy test/cheating. In case of giving advance warning about a test taker's misconduct, it is provided in the form of a warning message or alarm to the test supervisor or person in charge, and notified in various ways such as SMS transmission.

Online exams or UBT exams can prevent cheating, proxy exams, and reduce the number of exam proctors, dramatically lowering exam costs while ensuring minimal test reliability even in online evaluations without a proctor.

<System Configuration>

1) When using online: User program (PC/smartphone/tablet) > Wired/wireless communication network (WAN, LTE 4G/5G) and internal network (LAN, Wi-Fi) > Server (exam information/candidate information) and recognition information Check the necessary information by exchanging, True if it is correct/False if it is incorrect, and the user program executes the specified action according to the returned value. The candidate's recognition information and results are also transmitted to the supervisor's program (PC/smartphone/tablet). (Send by program or by SNS/SMS/E-mail)

2) Offline use: The artificial intelligence learning module is installed in the user program (PC/cell phone/tablet), and after recognizing the facial contour of the test taker's face, only the final facial recognition result is transmitted to the test server. When the final recognition result is transmitted from the test taker terminal to the test server, the corresponding information is displayed on a server program (dashboard), and transmitted to the corresponding information program or SNS/SMS/e-mail to the proctor or test manager terminal.

<Part of the actual app prototype in development APK - Attached for Android>

https://we.tl/t-wFdXexsors file download password: nsdevil

You can directly check some of the above functions by installing the app on your Android smartphone or tablet PC.

<Web version feature address>

1) Face Recognition Module: https://facere.nsdai.org (id: nsdevil, passwd: nsdevil) > Take a QR code with your smartphone and place it on the recognition screen on your PC camera to operate.

2) 5-point scale of facial contour anti-fraud module- Facial contour recognition module: https://headpos.ublrandd.com.np (id: nsdevil, passwd: nsdevil), eye/nose/ear facial feature point 5-point scale fraud anti-defense module

AI-based facial recognition to prevent proxy tests or cheating in tablet PC, smartphone, and PC-based online tests and UBT tests equipped with an artificial intelligence facial recognition module and an eye/nose/ear facial feature point 5-point scale anti-cheating module Using the contour recognition technology (posenet algorithm), the test taker's terminal recognizes the contour of the test taker's face and the eye2/nose/ear2 feature points, and moves them to the right/left depending on whether the 3 eye/nose points get closer to the 2 ear points at both ends. Detects and tracks movement of the head to detect facial behavior patterns. When facial contours are not recognized during face recognition, when the tablet PC camera image deviates from the test screen (3 points for eyes/nose, 2 points for ears at both ends) If the distance in the direction of the eye/ear distance and nose/ear distance exceeds a certain value depending on whether the , provide grading results to test takers, and prevent proxy testing or cheating in online and UBT exams.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. A hardware device configured to store and execute program instructions in a magneto-optical media, and a storage medium such as ROM, RAM, flash memory, and the like may be included. Examples of program instructions may include those generated by a compiler and not only machine language code but also high-level language code that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention is implemented as a program and is readable using computer software in the form of a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) ) can be stored in

Although described with reference to a specific embodiment of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiment in order to illustrate the technical idea as described above, and within the limit that does not depart from the technical spirit and scope of the present invention It can be implemented with various modifications. Accordingly, such modifications should be considered to fall within the scope of the present invention, and the scope of the present invention should be determined by the following claims.

100: Online exam or UBT exam server
101: WWW server 102: control unit
103: member registration 104: user authentication unit
105: test notification unit 106: QR code management unit
107: Greeting Provider (TTS) 108: Supervisor Verification Department
109: face recognition unit 110: cheating processing unit
111: test management unit 120: test information DB
121: candidate DB 123: face photo DB
200: supervisor terminal 300,310,311: test taker terminal

Claims (16)

an online test or UBT test server that registers test taker information and a frontal face picture, issues a QR code, and provides a test program for conducting an online test or UBT test;
It is connected to the online test or UBT test server through a wired/wireless communication network, and an artificial intelligence facial recognition module of the face photographed through the front camera of the test taker terminal, and the facial behavior pattern photographed with the front camera to configure the feature points of the face a test taker terminal having a 5-point scale anti-fraud module for eyes2, nose, and ears2; and
In the case of an online test or UBT test, a proctor terminal for receiving face photos and cheating information of a plurality of test taker terminals through the online test or UBT test server and transmitting a fraud prevention alarm or warning message to the test taker terminal;
An online test system using facial contour recognition artificial intelligence to prevent cheating, including According to claim 1,
The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which the test program (App) is installed, the test program is installed, and the artificial intelligence facial recognition module and the eye/nose/ear 5-point scale prevent cheating Online test system using facial contour recognition artificial intelligence to prevent cheating, having a module. According to claim 1,
The test taker terminal
During the online test or UBT test, the ROI of the face image captured in real time with the front camera of the test taker's terminal is detected, and the face object is extracted and the face By recognizing behavioral patterns, the outline of the candidate's face and eye/nose/ear feature points are extracted. Calculates the Euclidean distance (d) and similarity to the student, compares it with the face photo DB of the online test or UBT test server, and confirms whether the test is a proxy test. The distance between the right eye and the right ear and the distance between the left eye and the left ear change depending on the angle at which the front face is rotated to the left and right by recognizing the photographed facial behavior pattern. An online test system using facial contour recognition artificial intelligence to prevent cheating, which confirms this and generates an alarm or warning message to the test taker's terminal. 3. The method of claim 2,
The test taker terminal includes the face recognition module and the cheating prevention module, and the cheating prevention module includes a visual cheating detection module and a voice recognition module of the camera vision image data of the AI-based visual test taker terminal in real time and
The voice recognition module is provided with a voice recognition module for recognizing a test taker's speech in order to prevent cheating of the test taker in the test taker terminal. The test taker's voice data input through the terminal's microphone is immediately transmitted and stored to the online test or UBT test server, and it is transmitted to the proctor's terminal to generate an alarm to the test taker's terminal through the server to prevent the test taker's terminal from cheating. Online test system using facial contour recognition artificial intelligence to prevent cheating or sending messages. 3. The method of claim 2,
The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm, and an online test system using facial contour recognition artificial intelligence to prevent cheating. According to claim 1,
The test taker terminal uses the facial contour recognition technology (posenet algorithm) for the camera image by the face recognition module and the cheating prevention module to the right/left depending on whether the 3 eyes/nose points get closer to the 2 ear points at both ends. Detects head movement and detects abnormal behavior patterns of faces related to cheating. If the distance in the corresponding direction of the distance between the eyes/ears and the distance between the nose/ear exceeds a certain threshold), a warning message or alarm is output to prevent cheating, and an online test system using facial contour recognition artificial intelligence to prevent cheating. According to claim 1,
The online test or UBT test server
a WWW server connected to the test taker terminal and the supervisor terminal through wired/wireless communication;
a control unit for controlling to provide online or ubiquitous-based learning (UBL) and test (UBT) to the test taker terminal and the proctor terminal through a wired/wireless communication network;
a member registration unit connected to the control unit, receiving member information of test takers, storing and managing ID/Paaswd;
a user authentication unit connected to the control unit and authenticating a user using a QR code/Passwd or ID/Passwd;
a test notification unit connected to the control unit and providing a schedule and location of a tablet PC, a smartphone, a PC-based online learning (Learning) and test, or ubiquitous-based online learning (UBL) and a test (UBT);
a QR code management unit connected to the control unit, issuing a QR code corresponding to test taker information to test takers, and managing the QR code;
It is connected to the control unit, receives a photograph of the candidate of the camera from the candidate terminal, compares the candidate information stored in the database with the photograph of the candidate, and the supervisor confirms the examination at the supervisor terminal wealth;
a face recognition unit connected to the control unit, using a face recognition module in the test taker terminal to recognize the contour of the test taker's face and eye2/nose/ear2 feature points, and receive a face recognition result from the test taker terminal;
The test taker terminal measures the distance between the eyes/nose and ears using facial contour recognition technology (posenet algorithm) for the camera image, and moves the head to the right/left according to whether the 3 eye/nose points get closer to the 2 ear points at both ends. Detects and detects abnormal behavior patterns of faces related to cheating. When a warning message or an alarm is output to prevent cheating (when the distance in the corresponding direction of the distance, nose/ear distance exceeds a certain reference value), the cheating processing unit receives a warning message or alarm from the test taker terminal;
Provides a test program (App) and test papers to the test taker terminal and the proctor terminal, manages test taker information, on-site face photos, and proctor information a test management unit that stores and manages test takers' written answers, grading results, supervisor information, and test taker status information, which stores and manages test takers' written test paper answers and test taker status information; and
Test information DB for storing test papers, written answers, and scoring results of test takers terminals; a test taker DB and a face DB for storing test taker information and a standard-sized frontal face picture;
Online test system using facial contour recognition artificial intelligence to prevent cheating, including. 8. The method of claim 7,
The online test or UBT test server
Online test system using facial contour recognition artificial intelligence to prevent cheating further comprising a greeting providing unit (TTS) that provides a greeting through TTS conversion technology after recognizing the QR code. According to claim 1,
The test paper questions of the test program (App) are for each question, text and images as well as multimedia test questions including text, images, VR/AR content, voice and video are presented and displayed, facial contour recognition to prevent cheating Online exam system using artificial intelligence. After receiving member information registration and login/user authentication, registering and storing test taker information and a frontal face photograph to an online test or UBT test server;
issuing, by the online test or UBT test server, a QR code corresponding to test taker information and a frontal face picture;
notifying, by the online test or UBT test server, a test schedule and location according to the subjective test;
Using the front face recognition algorithm in the test taker terminal having a face recognition module using AI-based facial contour recognition technology, the result of the recognition of the front face photo of the camera is received by the online test or UBT test server, and the proctor terminal receives the test taker information and determining whether or not to take the test by comparing the photograph and its feature points to check whether the test taker is the applicant himself or not;
The face recognition module of the test taker terminal and the eye/nose/ear facial feature point 5-point anti-fraud module for recognizing facial behavior patterns recognize facial behavior patterns photographed with the front camera of the test taker terminal and extract a face object, Euclidean distance between the eye/nose/ear facial feature points and the center point (pupil) of the left/right ear and left/right eye, respectively. Calculating similarity to (d), detecting abnormal behavior patterns of the face by detecting head movement to the right/left according to whether the 3 eyes/nose points are closer to the 2 points of the ears at both ends When the outline cannot be recognized, when the test screen deviates from the test screen of the test taker terminal (when the distance in the corresponding direction of the eye/nose distance and the nose/ear distance exceeds a certain standard depending on whether the 3 eye/nose points get closer to the 2 ear points at both ends) ) To prevent cheating, facial photos or test taker's voice data related to cheating are transmitted to the online test or UBT test server, output to the proctor's terminal through the server, and a warning message or alarm is received from the server to the test taker's terminal to do; and
The test program (App) and the online test paper are provided to the test taker terminal and the proctor terminal, and the test taker information, the on-site face photo of the test taker, and the proctor information are stored and managed in the database of the test server. storing the test paper written answers in each test taker terminal for a predetermined test time, receiving the test paper answers from the test taker's terminal at the end of the test, storing them, and providing the scoring results of the test takers' test paper written answers to the test taker's terminal;
A method of providing an online test using facial contour recognition artificial intelligence to prevent cheating, including. 11. The method of claim 10,
The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which the test program (App) is installed, the test program is installed, and the artificial intelligence facial recognition module and the eye/nose/ear 5-point scale cheating An online test providing method using facial contour recognition artificial intelligence to prevent cheating, comprising a prevention module. 12. The method of claim 11,
The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm, an online test providing method using facial contour recognition artificial intelligence to prevent cheating. 11. The method of claim 10,
The test taker terminal uses the facial contour recognition technology (posenet algorithm) for the camera image by the facial recognition module and the cheating prevention module to move the head to the right/left according to whether the 3 eyes/nose points get closer to the 2 points on both ends of the ears Detects and detects abnormal behavior patterns of faces related to cheating. If the distance in the corresponding direction of the distance between the ears and nose/ear exceeds a certain threshold), a warning message or alarm is output to prevent cheating, and an online test using facial contour recognition artificial intelligence to prevent cheating. 11. The method of claim 10,
It operates by taking the QR code with a smartphone and placing it on the recognition screen on the camera of the test taker terminal, and after recognizing the QR code, providing a greeting through a Text To Speech (TTS) function A method of online exam delivery using contour recognition artificial intelligence. 11. The method of claim 10,
The test paper questions of the test program (App) are displayed with multimedia test questions including text, images, VR/AR content, voice and video as well as text and images for each question. Facial contour recognition artificial to prevent cheating How to give online exams using intelligence. 12. The method of claim 11,
The test taker terminal includes the face recognition module and the cheating prevention module, and the cheating prevention module includes a visual cheating detection module and a voice recognition module of the camera vision image data of the AI-based visual test taker terminal in real time and
The voice recognition module is provided with a voice recognition module for recognizing a test taker's speech in order to prevent cheating of the test taker in the test taker terminal. The test taker's voice data input through the terminal's microphone is immediately transmitted and saved to the online test or UBT test server, and it is transmitted to the proctor's terminal, and an alarm is issued to the test taker's terminal through the server to prevent the test taker's terminal from cheating. Online test providing method using facial contour recognition artificial intelligence to prevent cheating further comprising the step of sending or sending a message.

Images