KR20230013236A - Online Test System using face contour recognition AI to prevent the cheating behaviour by using speech recognition and method thereof - Google Patents

Abstract

Disclosed are an online test system using face contour recognition artificial intelligence (AI) to prevent cheating by using voice recognition and a method thereof. The system comprises: an online test or ubiquitous (UBT) test server registering test applicant information and a frontal face photo to issue a QR code and providing a test program for performing an online or UBT test; a test applicant terminal connected to the online or UBT test server through a wired or wireless communication network and including an AI facial recognition module of the face photographed through a front camera of the test applicant terminal, a five-point scale cheating prevention module using facial feature points including two eyes, the nose, and two ears with respect to the standard-size face and recognizing facial behavior patterns photographed by the front camera to detect visual cheating, and a voice recognition module detecting auditory cheating; and a supervisor terminal receiving cheating information including photos or videos of a plurality of test applicants, from the plurality of test applicant terminals through the online or UBT test server and transmitting a cheating prevention alarm or warning message to the test applicant terminals. The test applicant terminal includes the face recognition module and the cheating prevention module. The cheating prevention module further includes a five-point scale cheating prevention module using facial feature points including two eyes, the nose, and two ears to detect visual cheating from image data of a camera of the test applicant terminal in real time and a voice recognition module which detects auditory cheating.

Description

Online Test System using face contour recognition AI to prevent the cheating behavior by using speech recognition and method thereof}

The present invention relates to an online test system and method using artificial intelligence, and more particularly, in a non-face-to-face online test and a UBT test, an artificial intelligence face recognition module and a 5-point scale for eye 2/nose/ear 2 face features In online tests and UBT tests based on tablet PCs, smartphones, and PCs equipped with an anti-cheating module and a voice recognition module, when facial contours are not recognized during face recognition, for example, a test in which a camera image of a tablet PC is taken In the case of deviation from the screen (when the distance in the corresponding direction, such as the eye-to-ear distance or the nose-to-ear distance, depending on whether the 3 eye/nose points are closer to the 2 ear points at both ends), the test taker detected by the microphone of the test taker terminal When the voice signal of Gaussian noise is higher than the standard decibel (dB) of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server, output to the supervisor's terminal through the test server, and alerted to the test taker's terminal A tablet using facial contour recognition artificial intelligence that prevents cheating using voice recognition, outputs a message or alarm, or stores it in the information device of the test taker's terminal, transmits it to the test server at the end of the test, and provides scoring results to test takers. It relates to a PC, smartphone, and PC-based online test system and its method.

As face recognition technology, an appearance based matching method and feature based face recognition are mainly used. Face recognition is recognized differently depending on the camera's shooting angle, lighting direction, posture, facial expression change, and face change over time.

For feature-based face recognition, a detection method using Haar-like features and a detection method using MCT (Modified Census Transform) images are used for image data captured by a digital camera, IoT device camera, or smartphone camera. . Using the face and eye detectors learned as Haar-like features from the input image of the smartphone camera, the contours of the face and the forehead/eyes/nose/mouth are detected, and the region of interest (ROI, Region of Interest) is detected to detect the circular pupil. The eye area set as Interest) is converted to grayscale, and the histogram of the eye image [pixel value of each pixel in the x-axis, The number of pixel values corresponding to the y-axis] is obtained, and after binarization of the eye image, pre-processing of the eye area photo is performed through histogram equalization, and the eyebrows, eyes, nose, and mouth of the face area are The face is recognized by detecting facial feature data of the outline, extracting texture features and shape features, and comparing the similarity with feature points of face photos stored in the face recognition DB.

The feature values of the eyebrows, eyes, nose, mouth, and chin of the face area are expressed as the difference between the sum of pixels included in the white area of the Haar-like feature and the sum of pixels included in the black area.

For example, the distance from the eye area detected in a standard horizontal and vertical face area picture to the endpoints of the right and left eyes, the size of the iris extracted using a hough circle transform algorithm value is used as the feature value.

Speech Recognition is a man-machine interface technology that removes noise from an acoustic speech signal input by a microphone, extracts features of the speech signal, and converts it into a set of words or text of a sentence (mapping). It is a process, microphone -> AMP -> LPF -> ADC -> it is stored in the voice database. Speech recognition largely consists of a pre-processing unit and a recognition unit. The pre-processing unit removes noise from the voice signal spoken by the user and extracts features for the recognition process. Recognize sentences by outputting words as recognition results or limiting comparison words.

The speech recognition system uses a method using vector quantization, a method using dynamic time warping (DTW), a method using a neural network, and a method using HMM (Hidden Markov Model). method is being used.

MSVQ (Multi-Section Vector Quantization) is a method of applying vector quantization by considering the temporal relationship of speech when generating a representative pattern of speech. Since speech signals have different time lengths for the same pronunciation, speech is divided into several sections, and the number of voice sections is normalized to a constant by obtaining a feature vector from the divided voice sections. The length of the section is increased to have the same number of sections even if the voice has a different time length.MSVQ divides the voice into several sections, performs independent VQ for each section, generates a representative vector for each section, and creates a voice section Normalize the number Normalize the number of speech sections using MSVQ for speech of different lengths, and use the LBG (Linde-Buzo-Gray) algorithm to generate a representative vector in each section.

A speech recognition algorithm using a hidden Markov model (HMM) can process speech and language processing in a single hierarchical structure when a statistical language model is used.

A process of pre-processing a voice signal and then converting it into data representing voice features is called feature extraction. Voice features widely used in speech recognition include Mel-Cepstrum, MFCC, PLP, and RASTA-PLP. In this study, speech features are extracted using Perceptual Linear Prediction (PLP). Cognitive linear prediction technology simulates the human auditory spectrum and provides an effect of compressing voice information, so it is known as a voice feature suitable for speaker independent voice recognition, and is reported to be particularly useful in voice recognition using neural networks.

Recently, a voice recognizer in which hardware and software are modularized and a voice recognition API of STT (Speech To Text) technology are being released as open sources.

As related prior art 1, Patent Publication No. 10-2017-0050465 discloses "face recognition device and method".

According to the embodiment, in recognizing a face from an input image using machine learning, the face pose and perspective are normalized to improve the face recognition rate, and a virtual face image is automatically generated as face learning data to obtain face learning data. A face recognition device and method that save cost and time are provided.

1 is a configuration diagram of an existing face recognition device.

The face recognition device 100 includes an image display device, an image recording device, a face recognition server, a tablet PC, a laptop (Laptop), a personal PC, a smartphone, a personal digital assistant (PDA), a mobile communication terminal, and It may be any one of intelligent robots.

The face recognition apparatus 100 includes an input image acquisition unit 112 acquiring an input image from a camera; A front pose image is generated by detecting a face region in the input image and normalizing a face pose, and the perspective of the front pose image is determined to remove perspective distortion according to the distance between the camera and the subject. a normalization unit 114 for normalizing ) to generate a normalized image; a feature vector extractor 116 extracting a feature vector representing the subject's face from the normalized image; and a face recognition unit 118 recognizing the face of the subject included in the input image by applying the feature vector to a pre-learned classification model.

The input image acquisition unit 112 acquires an input image input from a camera. The camera can be a depth-aware camera, a stereo camera, and a color camera (eg, a Kinect camera, etc.). In addition, the input image is an image including a face of a subject to be recognized, and includes a 2D still image and a moving image. The input image may include a color image, a depth image, and a color-depth (RGB-D) image.

The normalization unit 114 detects a face area from an input image and normalizes a face pose and perspective to generate a normalized image. When there is a change in the face pose, the face recognition rate is lowered because the gray scale, shape, position of feature points, etc. are changed. In addition, if the distance between the camera and the subject is different, perspective distortion (distortion) occurs differently for each photographed position even for the same subject, so it may seem as if a different subject has been photographed. Therefore, it is necessary to normalize the face pose and perspective of the input image in order to improve the face recognition rate.

The normalization unit 114 inputs face images for learning in various poses to the input layer of the first artificial neural network, and trains the first artificial neural network so that face images for learning in front poses are output from the output layer of the first artificial neural network. face pose normalization learning unit; And inputting the data output from the output layer of the first artificial neural network to the input layer of the second artificial neural network, and learning the second artificial neural network so that a face image for learning without perspective distortion is output from the output layer of the second artificial neural network. Includes a perspective normalization learning unit.

The normalization unit inputs face images for learning in various poses with various perspective distortions to the input layer of the integrated artificial neural network in which the first artificial neural network and the second artificial neural network are integrated, and the learning face image of the front pose is free from perspective distortion. The integrated artificial neural network is trained so that the learning face image is output from the output layer of the integrated artificial neural network.

The feature vector extractor 116 is determined through machine learning and extracts a feature vector representing a face of a subject from a normalized image.

The feature vector is a vector having feature values used for face recognition as elements. As a filter used to extract the feature vector, there are Gabor filter, Haar filter, LBP (Local Binary Pattern) - including DLBP (Discriminative LBP), ULBP (Uniform LBP), NLBP (Number LBP), etc. Without limitation, other filters may be used.

The face recognition unit 118 recognizes the face of a subject included in the input image by applying the feature vector extracted by the feature vector extraction unit 116 to the previously learned classification model. The pre-learned classification model may include, but is not necessarily limited to, Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), and Softmax.

The virtual face image generation unit 124 may generate a plurality of virtual face images used for learning by the normalization unit 114, the feature vector extraction unit 116, and the face recognition unit 118.

The plurality of virtual face images refer to face images generated by transforming a 3D face model synthesized by the virtual face image generator 124 using one or more 2D reference images obtained from a camera.

* Characteristics of face recognition system

The face recognition rate changes depending on the lighting or the distance from the camera.

The shape of the face continues to change depending on the angle, expression, and age of the face, and can be recognized differently depending on changes in appearance, such as eyebrows, glasses, and makeup.

* Configuration steps of face recognition system

- Image Acquisition: Image acquisition from CCD camera or CMOS Image Sensor camera

- Pre-processing: noise removal image processing and segmentation

- Face detection: Detect face area in ROI area from input image

- Face standardization: standard size, feature extraction, brightness and geometric standardization

- Face Recognition: Comparison and recognition of detected images and database images

* geometric method

- Features: Recognize by extracting the geometrical feature points of the face and judging whether they match.

- Performance: Since the face is 3-dimensional and can be rotated, there is a limit to its application.

* Eigenfaces: developed by Pentland in 1991.

- Features: PCA (Principal Component Analysis) is applied to extract facial feature points, and Euclidean distance is applied to measure similarity.

- Performance: A representative face recognition method that responds sensitively to changes in lighting or environment, but is widely used and its performance has been verified.

*Fisherfaces

- Features: Face recognition method applying FLD (Fisher Linear Discriminant) as a classification algorithm.

- Performance: An algorithm that is more accurate and insensitive to environmental changes by learning the characteristics of each person, and can be applied in real time in an on-line state.

* Method based on SVM (Support Vector Machine)

- Features: Algorithm using PCA and SVM (Support Vector Machine).

- Performance: Algorithm that has a relatively high recognition rate, but takes a lot of time and memory to apply to multi-class such as face recognition. A representative face recognition algorithm currently being researched.

* Neural network

- Features: Retrieve learned face shapes using a sliding window in black and white still images

- Performance: Extraction of two or more faces is possible, but it is slow and difficult to learn.

* Fuzzy + neural network

- Characteristic: Use fuzzy membership function instead of pixel brightness value as input of neural network circuit

- Performance: The performance is improved compared to the method using only neural networks, but the processing speed is lower.

* Wavelet + Elastic Matching

- Characteristic: Uses frequency conversion and is effective in handling changes in posture and expression

- Performance: The amount of calculation is higher than the recognition rate.

As a prior art related to this, patent registration number 10-1765770 registers a cheating detection system and method for an online test using image processing and a recording medium storing a program for implementing the same.

The system and method for detecting cheating in an online test can supervise cheating in an online test and secure reliability and sincerity of the online test by detecting suspected cheating of an examinee during an online test using image processing. Provided is a cheating detection system and method for an online test using image processing that can be performed, and a recording medium storing a program for implementing the same.

2 is a configuration diagram of an online evaluation system including a cheating detection system of a conventional online test.

The online evaluation system includes an examinee's terminal 110 for an examinee to take an online test; A camera 120 attached to the examinee terminal or provided in a smart device to photograph the front of the examinee; A cheating detection server 100 that receives image information of the test taker captured by the camera in frame units and detects cheating through image processing; When the cheating detection server determines that the cheating of the test taker is suspected, a supervisor terminal 130 for notifying a supervisor of the cheating; and an online evaluation server 140 for setting questions of an online test that the examinee takes.

The cheating detection server 100 receives the image information of the test taker through a communication network, and when it is determined that cheating is suspected, it transmits a cheating detection message to a supervisor, and transmits the cheating detection message to the online evaluation server 140. Transmitting/receiving unit 101 for informing to stop the online test; a storage unit 102 for storing the image information and coordinate information of the examinee received through the transmission/reception unit 101; a face recognition unit 103 for defining a rectangular area including a face area from the received image information of the examinee; a pupil coordinate acquisition unit 104 for setting pupil coordinates of both eyes by image processing the shape and brightness of both eyes in a coordinate system based on the rectangular area; an intermediate coordinate acquisition unit 105 for determining intermediate coordinates of the pupil coordinates of the both eyes; a background recognition unit 106 that sets all areas except for the body and face areas of the examinee as a background; and whether or not there are two or more face regions acquired by the face recognition unit 103, and compares the standard pupil coordinates received from the pupil coordinates acquisition unit 104 with the average value of real-time pupil coordinates to obtain the reference pupil coordinates. As a criterion, whether the real-time pupil coordinate shape value is out of a preset error range, whether the movement distance of the intermediate coordinate received from the intermediate coordinate acquisition unit 105 is out of a preset error range, and the background recognizer ( 106) includes a cheating determining unit 107 that determines whether a change in the background area is detected.

In the embodiment, the rectangular area including the face area was defined based on the content disclosed in the thesis “Robust Real-Time Face Detection Using Face Certainty Map” (Volume 4642 of the series Lecture Notes in Computer Science pp 29-38).

The cheating detection server 100,

A rectangular region including the examinee's face region is defined in frame-unit image information captured by the examinee, and both end points of both eyes (310 , 320), set the center points (350, 360) of the black areas (330, 340) located between them as the pupil coordinates of both eyes, and calculate the average value of the pupil coordinates of the examinee for a certain period of time before the start of the test. to obtain a reference pupil coordinate value, and when the test starts, the average value of pupil coordinates of both eyes of the examinee is calculated in a unit of a predetermined time to obtain a real-time average pupil coordinate value, and the average real-time pupil coordinate value is based on the reference pupil coordinate value. If it is out of the set error range, it is notified to the supervisor terminal,

The center point 370 of the pupil coordinates of both eyes is determined as the intermediate coordinate,

The movement direction and movement distance of the intermediate coordinates are tracked on a frame-by-frame basis, and when the movement distance of the intermediate coordinates is out of a preset error range, the direction of rotation or movement of the examinee's head is notified, and the image information of the examinee's frame unit is notified. If there are two or more face regions obtained from , it is notified to the supervisor terminal,

All areas except the body and face areas of the examinee are set as the background, and when a change in the background is detected during the test time, it is notified to the supervisor terminal,

The cheating detection server 100,

When the video information of the examinee is received through a communication network and it is determined that cheating is suspected, a cheating detection message is transmitted to the proctor terminal 130, and questions of the online test for which the examinee takes the test are set. and informs the online evaluation server 140 to stop the online test.

When the online evaluation server 140 receives a message from the cheating detection server 100 informing that a suspected cheating operation has been detected while setting questions for the online test that the examinee is taking, the online evaluation server 140 determines the It is characterized in that the question setting is stopped, and after receiving a message from the supervisor terminal 130 or the cheating detection server 100 that the question may be restarted, the online test question setting is restarted.

Recently, in face recognition technology, head/eye/nose/mouth features using HOG, LBP, and Gabor feature extraction algorithms, which are hand-crafted features that have previously been mainstream, are deep learning-based features, face detection, CNN (Convolutional Neural Network) algorithm, facial landmark detection and facial feature extraction and classification technology are used to provide results similar to 97.53% of human recognition level.

In 2014, artificial intelligence face recognition technology such as Facebook's DeepFace was released. In the case of DeepID1 ["Deep Learning Face Representation from Predicting 10,000 Classes,” CVPR 2014] announced at CVPR, 39x31 pixels are used, which is smaller than the input resolution of DeepFace's 152x152 pixels.

In addition, AlexNet-based face detector and AlexNet pre-learned with ImageNet learning data are fine-tuned to face images, so the detection performance is not yet high, and the final stage uses an SVM classifier to detect faces there is.

Patent Publication No. 10-2017-0050465 (published on May 11, 2017), "Face recognition device and method", SK Telecom Co., Ltd. Patent Registration No. 10-1765770 (registration date August 01, 2017), "System for detecting cheating in online exam using image processing and its method and recording medium storing program for implementing it", Bukyung Son

The purpose of the present invention to solve the above problems is to mount an artificial intelligence face recognition module, an eye 2 / nose / ear 2 face feature point cheating prevention module and a voice recognition module on a 5-point scale during the non-face-to-face online test and UBT test. In tablet PC, smartphone, and PC-based online tests and UBT tests, when facial contours are not recognized during face recognition, when the camera image of the tablet PC is out of the test screen (3 points for eyes/nose at both ends of ears) If the distance in the corresponding direction, such as eye-to-ear distance or nose-to-ear distance, exceeds a certain number depending on how close it gets to 2 points), the test taker's voice signal detected by the microphone of the test taker's terminal is more than the standard decibel (dB) of Gaussian noise. When a sound is generated, the test taker's voice data is transmitted to the online test or UBT test server, output to the supervisor's terminal through the test server, and a warning message or alarm is output to the test taker's terminal or the information device of the test taker's terminal. Provides a tablet PC, smartphone, and PC-based online test system that uses facial contour recognition artificial intelligence to prevent cheating using voice recognition, which is stored in and transmitted to the test server at the end of the test, and provides scoring results to test takers. .

Another object of the present invention is to provide a tablet PC, smart phone, and PC-based online test providing method using facial contour recognition artificial intelligence using voice recognition.

In order to achieve the object of the present invention, an online test system using facial contour recognition artificial intelligence, which prevents cheating using voice recognition, registers test taker information and frontal face photos and issues a QR code attached to the test taker's gazette. and an online test or UBT test server that provides a test program for conducting an online test or UBT test; Connected to the online test or UBT test server through a wired or wireless communication network, a face recognition module of a face photographed by a front camera of a test taker terminal, and a standard size front face photo that detects visual cheating by recognizing facial behavior patterns A test taker terminal equipped with an eye 2/nose/ear 2 facial feature point 5-point scale cheating prevention module and a voice recognition module for detecting auditory cheating; and during an online test or UBT test, cheating information including visual face photos or face images of a plurality of test taker terminals and/or auditory audio signals is received through the online test or UBT test server and sent to the corresponding test taker terminal. Including a supervisor terminal that transmits an anti-cheating alarm or warning message,

After being taken with a QR code recognition smartphone attached to the test taker's test taker, the camera of the test taker's terminal is operated by facing the recognition screen, and the online test or UBT test server recognizes the QR code and provides a greeting through TTS conversion technology Further comprising a greeting service unit (TTS),

The face contour recognition technology used in the face recognition module uses the posenet algorithm, and when face contour recognition is not performed during face recognition, when it is out of the test screen, that is, the eye-to-ear distance and the nose-to-ear distance in the corresponding direction. If it exceeds a certain threshold, it is recognized as cheating, or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound that exceeds the standard value decibel (dB) of Gaussian noise, online test detected by the camera and microphone in the test taker's terminal When an audio-visual cheating behavior is detected by a test taker watching the test taker, the test taker's face picture or face image and the test taker's voice data are transmitted to the test server, output to the supervisor's terminal through the test server, and a warning is issued from the test server to the test taker's terminal. Send messages or alarms.

In order to achieve another object of the present invention, a method for providing an online test using facial contour recognition artificial intelligence to prevent cheating using voice recognition is registered with member information, and after log-in/user authentication, test taker information and a front face photo are collected. Registering and storing an online test or UBT test server; issuing, by the online test or UBT test server, a QR code attached to the candidate's application form corresponding to the candidate's information and the front face photo; notifying, by the online test or UBT test server, the test schedule and location according to the host test; The test taker terminal includes a face recognition module, a 5-point facial feature point anti-cheating module, and a voice recognition module, and the test taker terminal uses a frontal face recognition algorithm to transmit the recognition result of the frontal face picture of the camera to the online test or UBT. receiving the information received from the test server and checking whether the test taker is the test taker by comparing the test taker information with a standard size face photo, thereby preventing proxy test and determining whether to take the test; The face contour recognition technology used in the face recognition module uses the posenet algorithm, and when face contour recognition is not performed during face recognition, when it is out of the test screen, that is, the eye-to-ear distance and the nose-to-ear distance in the corresponding direction. If it exceeds a certain threshold, it is recognized as cheating, or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound that exceeds the standard value decibel (dB) of Gaussian noise, online test detected by the camera and microphone in the test taker's terminal When an audio-visual cheating behavior of a test taker is detected, the test taker's face photo or face image and voice data are transmitted to the test server, and output to the supervisor's terminal through the test server, and a warning message is sent from the test server to the test taker's terminal. or receiving an alarm; and provides a test program (App) and online test papers to the test taker terminal and the supervisor terminal, and stores and manages the test taker information, the on-site face photo of the test taker, and supervisor information in the database of the test server, and during the online test or UBT test After storing test paper preparation answers in each test taker terminal during a predetermined test time, when the test is over, the test taker terminal receives and stores the test takers' answers to the test server, and provides scoring results of the test takers' test paper preparation answers to the test taker terminal.

An online system and method using facial contour recognition artificial intelligence to prevent cheating using voice recognition of the present invention includes an artificial intelligence facial recognition module, an eye 2/nose/ear 2 facial feature point 5-point scale cheating prevention module, and voice recognition In tablet PCs, smartphones, PC-based online tests and UBT tests in which the module is installed, and in non-face-to-face tests, face recognition, facial behavior patterns, and voice signals are detected to detect audio-visual negatives during proxy tests, online tests, and UBT tests. action was prevented.

A face recognition module that prevents visual cheating by using artificial intelligence face recognition technology on the test taker terminal, a cheating prevention module using a 5-point scale for eye2/nose/ear2 facial features, and a voice recognition module that prevents auditory cheating We implemented a visual/audible cheating prevention technique during the proxy test prevention and online test that combines .

The AI-based facial contour recognition technology used in the facial recognition module of the test taker terminal and the anti-cheating module using the eye 2/nose/ear 2 facial feature points on a 5-point scale uses the posenet algorithm to recognize the facial contours of the camera image of the tablet PC. . In addition, when the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard decibel dB of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server, and output to the supervisor's terminal through the test server and receives a warning message or alarm from the test server to the corresponding test taker terminal. During an online test or UBT test, AI-based face recognition technology and voice recognition technology are used to identify the tablet PC test taker through the proctor terminal linked to the test server, and to prevent substitute test takers. Face recognition and voice recognition have the effect of preventing proxy tests or audio-visual cheating.

Online exams or UBT exams can dramatically reduce test costs by preventing test takers' proxy tests by recognizing their faces, preventing test takers' audio-visual cheating, and reducing the number of test supervisors, while at the same time, even in non-face-to-face online test evaluations without supervisors, at least reliability of the test can be guaranteed.

1 is a configuration diagram of an existing face recognition device.
2 is a configuration diagram of an online evaluation system including a cheating detection system of a conventional online test.
Figure 3a is a diagram showing the concept of a facial contour recognition artificial intelligence platform having a tablet PC, a smartphone, and a PC-based online test/UBT test server at the time of the language, health and medical education online test/UBT test.
Figure 3b is a diagram showing the face recognition function on the Ubiquitous-Based Learning (UBL) and Ubiquitous-Based Testing (UBT) NSDAI platform.
3c and 3d show the goals and User Experience (1. User registration -> 2. Learning -> 3. QR code Creation -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor check and test -> 6.7 UBT App/Web camera facial contour recognition 8. End of test) to be.
4 shows the process from test taker registration to test taker, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) supervisor confirmation (applicant face/test taker information), 6) test taking - This is a picture showing the process.
5 is a screen for registering/learning/QR code issuing a candidate-face matching code issuing unit-at the time of an online test/UBT test based on machine learning and learning results of a registered candidate's face.
6 is a screen of the candidate confirmation unit in the UBT system in which similarity is measured through AI-based facial contour recognition of the test taker's face.
7 is a rule user definition unit and a program test screen when determining candidates.
8 is a test server access / log-in / left-QR code recognition / greetings heard when QR code is recognized (TTS) / right-face recognition (smartphone / tablet PC front camera) / face in the background during an online test or UBT test This is the demonstration screen including the process of starting area recognition/recognition rate display - the UBT authentication process user definition part using the generated user code.
9 is a block diagram of a tablet PC, smart phone, and PC-based online test system using artificial intelligence for facial contour recognition to prevent cheating according to the present invention.
10 is a diagram illustrating functions of an AI-based face recognition module and an anti-cheating module during an online test or a UBT test.
11 is a flowchart illustrating a tablet PC, smartphone, and PC-based online test providing method using artificial intelligence for facial contour recognition to prevent cheating according to the present invention.

Hereinafter, the configuration and operation of a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. In the description of the present invention, if it is determined that a detailed description of a related known technology or known configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, the attached drawing numbers refer to the same drawing numbers in other drawings when indicating the same configuration.

9 is a block diagram of a tablet PC, smart phone, and PC-based online test system using artificial intelligence for facial contour recognition to prevent cheating according to the present invention.

10 is a diagram illustrating functions of an AI-based face recognition module and an anti-cheating module during an online test or a UBT test.

The UBT system and method using the facial contour recognition artificial intelligence of the present invention is a tablet PC and a smartphone equipped with an artificial intelligence face recognition module, an eye 2 / nose / ear 2 face feature point cheating prevention module on a 5-point scale, and a voice recognition module. , In the PC-based online test and UBT test, AI-based face contour recognition technology (posenet algorithm, machine learning model which allows for real-time face pose estimation) is used to prevent proxy tests or cheating. It extracts the object of the test taker's face, extracts the outline of the face and the feature points of the eye2/nose/ear2, and extracts the behavioral pattern of the face in real time to prevent visual cheating. Artificial intelligence facial that detects and tracks the movement of the head to the right/left according to whether the 3 facial feature points of the face are closer to the 2 points of the ears at both ends by the 5-point facial feature point scale cheating prevention module and detects the behavior pattern of the face. Recognition module, eye 2/nose/ear 2 facial feature points 5-point scale Anti-cheat module and voice recognition module are installed, and face contour recognition is not possible when face recognition is performed through FACE RECOGNITION/FACE MOTION RECOGNITION/RESULT ANLAYSIS. When the camera image of the tablet PC is out of the test screen where it is recorded (when the distance between the eye and ear and the nose and ear in the corresponding direction exceeds a certain number depending on whether the 3 points of the eye/nose get closer to the 2 points of the ears at both ends) In addition, when the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard value decibel (dB) of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server, and through the test server It is output to the supervisor's terminal, outputs a warning message or alarm from the test server to the corresponding candidate's terminal, or stores it in the information device of the corresponding candidate's terminal, transmits it to the test server when the test is over, and provides scoring results to candidates.

A proxy test prevention and cheating prevention technique was implemented by combining an artificial intelligence facial recognition module linked to the camera in the test taker terminal, a cheating prevention module using a 5-point scale for facial feature points, and a voice recognition module linked to a microphone.

The AI-based facial contour recognition technology used in the face recognition module of the test taker terminal and the cheating prevention module using the 5-point scale used the posenet algorithm for facial contour recognition of the camera image of the tablet PC. During an online test or UBT test, AI-based face recognition technology is used to identify the tablet PC test taker through a proctor terminal linked to the test server, to prevent proxy test, and to prevent test taker's face during the test time in the online test or UBT test. Recognition technology and voice recognition technology prevent proxy tests, visual cheating, and auditory cheating.

For the test paper questions of the test program (App), not only text and images for each question, but also multimedia test questions that include text, images, VR/AR content, voice and video are presented and displayed.

1. Artificial Intelligence Hub Platform

ㆍNSD-AI (NSDevil's Artificial intelligence) platform-based artificial intelligence hub platform

ㆍ Instructor scoring guide through phrase recognition in the field of medical education (collaborative research with Inje University College of Medicine and others)

ㆍ Experiential learning support system through object recognition (Fun Edu Lab and Korean Folk Village, etc.)

ㆍ Disease analysis function in dental education sector through tooth image analysis (joint research with Yonsei University College of Dentistry and other institutions in 3 countries)

ㆍ Based on UBT connect platform

ㆍ Tablet PC based evaluation platform

ㆍ Language evaluation section (Speaking/Listening/Reading/Writing)

ㆍ Health and Medical Evaluation Division (Listening/Reading/Writing)

ㆍ Other job evaluation areas (read/write)

A tablet PC, smartphone, and PC-based online test system using artificial intelligence facial contour recognition uses an online test or UBT test server (100), a supervisor terminal (200), and a PC, smartphone, and tablet PC, and uses a face recognition module. and eye 2/nose/ear 2 facial feature points using a 5-point scale, and test taker terminals 300, 310, and 311 equipped with a cheating prevention module and a voice recognition module.

The online test system using facial contour recognition artificial intelligence to prevent cheating using voice recognition of the present invention

Register and store test taker information and frontal face photos, issue a QR code attached to the test taker’s test taker, provide greetings through TTS conversion technology after recognizing the QR code for each test taker, and conduct online test papers and online tests or UBT tests It provides a test program for performing the online test or UBT test, through the online test or UBT test server 100, a negative face picture or face image of a plurality of test taker terminals and / or an audible voice signal is included. an online test or UBT test server 100 that receives and stores behavior information and transmits an alarm or warning message from a supervisor's terminal to a corresponding candidate's terminal;

An artificial intelligence facial recognition module of a face that is connected to the online test or UBT test server 100 through a wired and wireless communication network (LAN, Wi-Fi, LTE 4G/5G) and is photographed by the front camera of the test taker terminal during the test time; Eye2/nose/ear2 facial feature points 5-point scale Anti-cheating module for a standard size frontal face photo that prevents visual cheating by recognizing facial behavior patterns captured by the front camera, detects auditory cheating Test taker terminals (300, 310, 311) having a voice recognition module for recognizing the test taker's voice signal to do so; and

It is connected to the online test or UBT test server 100 through a wired or wireless communication network, and at the time of the online test or UBT test, a visual face photograph or face image of a plurality of test taker terminals through the online test or UBT test server 100 and / or a supervisor terminal 200 that receives cheating information including an audible voice signal and transmits an anti-cheating alarm or warning message to the corresponding test taker terminal;

After being taken with a QR code recognition smartphone attached to the test taker's test taker, the camera of the test taker's terminal is operated by facing the recognition screen, and the online test or UBT test server recognizes the QR code and provides a greeting through TTS conversion technology Further comprising a greeting service unit (TTS),

The face contour recognition technology used in the face recognition module uses the posenet algorithm, and when face contour recognition is not performed during face recognition, when it is out of the test screen, that is, the eye-to-ear distance and the nose-to-ear distance in the corresponding direction. If it exceeds a certain threshold, it is recognized as cheating, or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound that exceeds the standard value decibel (dB) of Gaussian noise, online test detected by the camera and microphone in the test taker's terminal When an audio-visual misconduct is detected by a test taker who is watching , the test taker's face photo or face image and the test taker's voice data are transmitted to the test server 100 and output to the supervisor terminal 200 through the test server 100, A warning message or an alarm is transmitted from the test server 100 to the corresponding test taker terminal.

The test taker terminals 300, 310, and 311 use any one of a tablet PC, a smartphone, and a PC on which a test program (App) is installed, and the test program is installed, and the artificial intelligence face recognition module and eye 2 / nose / ear 2 face A feature point 5-point scale cheating prevention module and a voice recognition module are provided.

The test taker terminals (300, 310, 311) have a face recognition module and a cheating prevention module, and the cheating prevention module has standard sized eyes 2/nose/ears to prevent visual cheating of the camera vision image data of the test taker terminal in real time. A cheating detection module on a five-point scale of two facial feature points and a voice recognition module for preventing auditory cheating are provided.

The voice recognition module detects the test taker's voice signal with the microphone of the test taker's terminal during the online test or UBT test when the sound of the test taker's voice is higher than the standard value decibel (dB) of Gaussian noise (if the test taker's speech related to auditory cheating is heard) , In order to prevent audiovisual cheating, cheating information including a visual face picture or face image and/or audible voice signal of the candidate terminal is transmitted to the test server, and through the test server, according to the control of the supervisor terminal, the corresponding An anti-cheating alarm or warning message is sent to the test taker's terminal to prevent cheating.

The AI-based face contour recognition technology used in the face recognition module uses a posenet algorithm (machine learning model which allows for real-time face pose estimation).

The voice recognition module uses a method using vector quantization, a method using dynamic time warping (DTW), a method using a neural network, and a hidden markov model (HMM). Any one of the methods used is used, and a STT (Speech To Text) module equipped with a voice recognition API is used.

When the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard decibel (dB) of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server 100, and the test server 100 ) through which cheating information of the candidate is output to the supervisor terminal, and a warning message or alarm is received from the test server 100 to the corresponding candidate terminal.

In the embodiment, during the online test or UBT test, the test taker terminal used a tablet PC.

The test taker terminals 300, 310, and 311 recognize a face, a face recognition module, an eye 2/nose/ear 2 facial feature point 5-point anti-cheating module for detecting visual cheating, and a voice recognition module for detecting auditory cheating. is provided

The test taker terminals 300, 310, and 311 detect the ROI of the face image captured in real time by the front camera of the test taker terminal during the online test or UBT test, and adjust the size/rotation/angle to the standard size of the face front photo. For face photos, a face object is extracted using a frontal face recognition algorithm, face behavior pattern recognition (pattern recognition), and size correction/rotation/angle correction to a standard size contrasted with a standard size face frontal photo. The outline of the face and facial feature points of Eye 2/Nose/Ear 2 are extracted, and through facial feature extraction and classification, the center points of the left/right ear and the left/right eye of the facial feature points of Eye 2/Nose/Ear 2, respectively ( Euclidean distance (d) and similarity with the pupil) are calculated, and whether a proxy test is confirmed by comparing with a standard size front face photo stored in the face photo DB of the online test or UBT test server 100, , During the online test or UBT test, the face behavior pattern captured by the camera of the test taker terminal is recognized, and the distance between the right eye and the right ear and the distance between the left eye and the left ear change according to the angle at which the front face is turned left and right, so cheating is judged. Then, the face photo or face image is transmitted to the test server 100, and the supervisor terminal linked to the test server 100 checks it and generates an alarm or warning message to the corresponding candidate terminal.

The test taker terminals 300, 310, and 311 use AI-based facial contour recognition technology (posenet algorithm) for the camera image by the facial recognition module and the eye/nose/ear facial feature point 5-point anti-cheating module to obtain positive eye/nose 3 points. Detects head movement to the right/left according to the proximity to the two end ears, detects abnormal facial behavior patterns related to cheating, and detects facial contours not being recognized when recognizing faces, or moving away from the test screen of the test taker terminal (Depending on whether the 3 eye/nose points get closer to the 2 ear points at both ends, if the distance between the eyes and ears and the distance between the nose and ears exceeds a certain standard value) A warning message or alarm is output to prevent cheating.

(Example)

Using the Posenet model, TensorFlow was used for estimating the pose of a human face on Android on a tablet PC. For reference, the Posenet algorithm is a vision model that detects the positions of major body parts and estimates a person's pose by recognizing an action from an image or video. TensorFlow Lite provides an Android sample application that uses the camera of the test taker's device to detect and display key human body parts in real time. This model can estimate the position of a person's elbow and/or knee in an image, and in the embodiment, a facial contour pose estimation model was implemented by identifying eyes 2/nose/ear 2 in real time during face recognition.

The posenet algorithm used in the artificial intelligence face recognition module uses a five-point scale anti-fraud module of leftEye, rightEye, nose, leftEar, and rightEar. implemented,

After finding the x,y coordinates of leftEar and the x,y coordinates of rightEar, you can find the diameter of a circle based on these and create an angel ring with ellipse( ). The diameter can be easily obtained with dist( ) in p5js without having to use the Pythagorean theorem.

<Connecting ml5js and Webcam>

<script src="https://unpkg.com/ml5@0.3.1/dist/ml5.min.js"></script>

let video;

let poseNet;

function setup() {

createCanvas(400, 400);

video = createCapture(VIDEO);

// Used to prevent the video from appearing twice

video.hide( );

poseNet = ml5.poseNet(video, modelLoaded);

poseNet.on('pose', gotPoses);

console.log(ml5);

}

// Pose-related callbacks are entered

function gotPoses(poses) {

}

// Used to check if loading was successful

function modelLoaded() {

console.log('Model Loaded');

}

function draw() {

// Draw the webcam at position 0,0.

image(video,0,0);

}

The face region is extracted in real time from the face image of the front camera of the test taker terminal, adjusted to the standard size of the face photo through size correction/rotation/angle correction to the standard size, and learning data of the standard size face photo DB pre-stored in the test server. Comparison with -> Learning model of face photo -> Extracts and classifies facial contours and eye 2 / nose / ear 2 facial feature points, outlines of face recognition data, eye 2 / nose / ear 2 face feature points Group clustering (center of cluster k-means algorithm) density estimation, distance (Euclidean distance) and similarity of face feature points are calculated, and standard size candidates stored in face photo DB of online test or UBT test server By comparing the Euclidean distance and similarity of the facial feature points in the photograph, the candidate himself/herself checks the photograph to prevent proxy testing, and cheating is determined during the online test or UBT test.

For example, when the direction of the face photographed by the front camera of the test taker device (tablet PC) changes or the test taker turns the face, the outline of the face and the facial features of eyes 2/nose/ear 2 are displayed. If it is not recognized or the angle towards the test program changes within the effective distance (20-30cm) of the user's face and the smartphone, it is recognized as cheating, and the cheating detection result of the face photo or face image is displayed as an online test or It is transmitted to the UBT test server 100, and transmitted to the supervisor's terminal 200 through the test server 100, and the supervisor's terminal transmits the corresponding test taker's terminal through the test server 100 to prevent cheating. It generates an alarm or sends a warning message to the terminal and prevents cheating.

Alternatively, [Text A] As soon as the warning threshold set in the predefined candidate program with data reference becomes 0, an alarm or warning message is displayed on the candidate device without connecting to the test server and transmitted back to the supervisor/cheating detection server/other test servers. Alternatively, after the test is over, the corresponding RAW DATA is transmitted to the cheating detection server/other test management server for post-processing.

In addition, the test taker terminal includes a face recognition module and an anti-cheating module, and the anti-cheating module further includes a voice recognition module for recognizing a speech of the test taker in order to prevent cheating of the test taker in the test taker terminal. Alternatively, if the voice of the test taker related to cheating is heard through the microphone of the test taker's terminal during the UBT test, the voice is recognized, and the voice signal of the test taker is immediately transmitted to the online test or UBT test server 100 for storage. , This is transmitted to the supervisor terminal 200 through the test server 100 so that the supervisor generates an alarm or sends a warning message to the corresponding candidate terminal through the test server 100 so that the candidate terminal prevents cheating. prevent the action. Alternatively, by referring to thresholds of predefined distances such as eye-to-ear distance, nose-to-ear distance, etc. in a frontal face picture of a preset standard size, an alarm or alarm when cheating is detected from the test taker's terminal directly from the test taker's program without a command from the test server. A warning message is displayed and transmitted to the supervisor's terminal, fraud detection server, or other test management server.

The online test or UBT test server 100 includes a WWW server 101, a control unit 102, a member registration unit 103, a user authentication unit 104, a test notice unit 105, a QR code management unit 106, a greeting system Study (TTS) 107, supervisor confirmation unit 108, face recognition unit 109, cheating processing unit 110, test management unit 111, test information DB 120, candidate DB 121, and face It includes a photo DB (123).

The online test or UBT test server 100

A tablet PC, a smartphone, a WWW server 101 connected to a test taker terminal using any one of a PC and a supervisor terminal through wired or wireless communication;

A control unit 102 for controlling to provide application services of online or ubiquitous-based learning (UBL) and ubiquitous-based test (UBT) to tablet PCs, smartphones, PC test taker terminals and supervisor terminals through wired and wireless communication networks;

a member registration unit 103 connected to the control unit 102 and receiving registered member information of the candidate and storing and managing ID/Passwd;

A user authentication unit 104 connected to the control unit 102 and authenticating a user using QR code/Passwd or ID/Passwd;

A test notice unit connected to the control unit 102 and providing tablet PC, smartphone, PC-based online learning (Learning and test) or ubiquitous-based online learning (UBL) and test (UBT) schedule and location ( 105);

a QR code management unit 106 that is connected to the control unit 102, issues a QR code corresponding to test taker information to test takers, and manages the QR code;

A greeting providing unit (TTS) 107 that provides a greeting through TTS conversion technology after recognizing the QR code attached to the test taker's examination ticket;

It is connected to the control unit 102, receives a face picture of the candidate taken by the camera from the candidate terminal, compares the candidate information previously stored in the database with the candidate's frontal face picture, and the supervisor checks at the supervisor terminal to determine whether or not to take the proxy test. A supervisor confirmation unit 108 that checks and confirms qualifications for taking the test;

A face that is connected to the control unit 102 and recognizes the outline of the test taker's face and the feature points of the eye 2/nose/ear 2 using the face recognition module in the test taker terminal, and receives the face recognition result from the test taker terminal. recognition unit 109;

It is connected to the control unit 102, and 3 eye/nose points are close to 2 points at both ends by using facial contour recognition technology (posenet algorithm) for the front camera image by a face recognition module and an anti-cheating module in the test taker terminal. It detects head movement to the right/left depending on whether it loses, detects abnormal behavior patterns of the face related to the cheating of viewing the cheat sheet, and detects facial contours when face recognition is not recognized, or when it is out of the test screen (Eyes/Nose 3 If the eye-to-ear distance or the nose-to-ear distance in the corresponding direction exceeds a certain standard value depending on whether the point approaches the two ends of the ears), or the test taker's voice signal detected by the microphone of the test taker terminal is the standard decibel of Gaussian noise (dB) or more, receive cheating information including a visual face picture or face image of the test taker terminal and/or an audible voice signal to the test server 100 to prevent audiovisual cheating. a cheating processing unit 110 for receiving and outputting an anti-cheating alarm or warning message to a corresponding test taker terminal through the test server 100 under the control of a supervisor terminal;

It is connected to the control unit 102, provides a test program (App) and test papers to the test taker terminal and the supervisor terminal, manages test taker information, on-site face photos of the test taker, and supervisor information, and during an online test or UBT test. A test management unit that stores test paper writing answers in each test taker terminal within a certain test time, and then transmits them to the test server 100 when the test ends, and stores and manages test takers' test paper writing answers, scoring results, supervisor information, and test taker status information (111);

Examination information DB (120) for storing test takers' test papers, written answers, and scoring results; It includes a candidate DB 121 and a face DB 123 which store candidate information and a standard size front face picture.

The test taker terminal includes a mobile communication terminal capable of accessing the Internet, or a laptop computer as well as a smart phone or a tablet PC for an Internet Based Test (IBT) or a Ubiquitous-based Test (UBT).

The test taker terminal uses any one of tablet PC, smartphone, or PC where the test program (App) is installed, and the AI face recognition module of the camera image of the test taker terminal and 5 facial feature points of eyes 2 / nose / ears 2 A scale cheating prevention module is provided, an online test or a test program (App) downloaded from the UBT test server 100 is installed,

The AI-based face contour recognition technology used in the face recognition module uses the posenet algorithm.

The test taker terminal has a face recognition module and a cheating prevention module, and the cheating prevention module detects visual cheating in real time. An action detection module and a voice recognition module for detecting auditory misconduct are provided.

The online test or UBT test server 100 provides college tests such as medical school / dental school / pharmacy college / engineering college, TOEIC / TOEFL test, language test, civil service test, certification test, health and medical education learning and test papers The test schedule and location for various official certification tests or non-certified tests of banks are announced, and the tests are taken using test questions stored online or offline with test taker terminals through wired or wireless communication networks using test programs in the database of the test server. We provide UBT tests that utilize all types of PCs/smart devices (tablet PCs, smartphones) that can be used.

When the exam schedule and location are confirmed, the control unit 102 appoints a supervisor, provides supervisor appointment information to the supervisor, notifies test takers of the test schedule and location through a text message/webpage, The terminal 200 transmits and receives test information and test paper information to manage the progress of the test, and provides scoring results from the results of the automatic scoring result unit 124 and the inspection management unit 125 to corresponding test taker terminals.

The test information stored in the test information database 120 includes the test title, test schedule and location, test time, test location location information, assigned supervisor information for each test room and list of test takers, number of seats for each test room, supervisor information, A/ Includes question information, answer information, grading information, and supervisor information for each type of test.

In addition, the candidate database 121 includes candidate information including name, resident registration number, home address, mobile phone number, e-mail, etc., candidate's face picture (standard size front face picture) and QR code/passwd, ID/passwd identification information and test-related information, and applicant application status are stored.

11 is a flowchart illustrating a tablet PC, smartphone, and PC-based online test providing method using facial contour recognition artificial intelligence to prevent cheating using voice recognition according to the present invention.

How to provide a tablet PC, smartphone, or PC-based online test using facial contour recognition artificial intelligence to prevent cheating using voice recognition

Registering member information, and after log-in/user authentication, registering and storing test taker information and a frontal face picture in an online test or UBT test server (S10);

Issuing a QR code attached to the candidate's gaze tag corresponding to the candidate information and the front face photo (S20);

The online test or UBT test server notifying the test schedule and location according to the subject test (S30);

The QR code attached to the test taker's test taker's test taker is taken with a smartphone and placed on the recognition screen to the camera of the test taker's device (PC, tablet PC, etc.). Providing a greeting through (S40);

The candidate terminal includes a face recognition module using AI-based facial contour recognition technology (posenet algorithm), an anti-cheating module with a 5-point scale for eyes 2/nose/ear 2 facial feature points, and a voice recognition module. Using the face recognition algorithm, the recognition result of the camera's front face photo is received by the online test or UBT test server, and the supervisor's terminal compares the test taker's information with the standard size face photo and the facial feature points to confirm whether the test taker is the applicant and act as an agent. Preventing the test and determining whether to take the test (S50);

Each test taker terminal includes a face recognition module and a cheating prevention module, and the cheating prevention module includes a visual cheating detection module and a reference value decibel (dB) using facial contour recognition technology of camera vision image data of the test taker terminal during the test time. ) A voice recognition module equipped with a voice recognition API that detects the voice signal of the test taker inputted with a microphone to detect misconduct of the test taker's voice signal aurally when the above sound is generated,

The face contour recognition technology used in the face recognition module uses the posenet algorithm, and when face contour recognition is not performed during face recognition, when it is out of the test screen, that is, the eye-to-ear distance and the nose-to-ear distance in the corresponding direction. If the test taker exceeds a certain threshold, it is recognized as cheating, or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound that exceeds the standard value decibel (dB) of Gaussian noise, the test taker's terminal detects the camera and microphone on the online test. When an audio-visual misbehavior is detected by a viewing candidate, the candidate's face picture or face image and/or voice data are transmitted to the test server 100, and output to the supervisor's terminal through the test server 100, and the test server Receiving a warning message or an alarm from (100) to the corresponding test taker terminal (S60); and

The test program (App) and online test papers are provided to the test taker terminal and supervisor terminal, and the test taker information, the on-site face picture of the test taker, and the supervisor information are stored and managed in the database of the test server 100, and during the online test or UBT test After storing the test paper writing answers in each test taker terminal during a certain test time, at the end of the test, the test taker terminal receives and stores them in the test server 100 (S70), and the test server 100 scores the test takers' test paper writing answers. and providing it to the test taker terminal (S80).

In the test taker terminal, artificial intelligence (AI)-based facial contour recognition technology used the posenet algorithm for facial contour recognition of the camera image of the tablet PC. During the online test or UBT test, AI-based face recognition technology is used to identify the tablet PC test taker through the supervisor terminal linked to the test server 100, prevent proxy test, and check the face of the test taker at the online or UBT test site. It can be used as a recognition access control system and prevents proxy tests or cheating in online tests and UBT tests.

The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which a test program (App) is installed, and the test program is installed, and the artificial intelligence face recognition module and 5 points of eyes 2 / nose / ears 2 are installed. A scale anti-cheating module and a voice recognition module are provided.

The AI-based face contour recognition technology used in the face recognition module uses the posenet algorithm.

The candidate terminals 300, 310, and 311 are equipped with a face recognition module and an eye 2 / nose / ear 2 facial feature point 5-point scale anti-cheat module,

During the online test or UBT test, the ROI of the face image captured in real time by the front camera of the test taker's device is detected and the face behavior pattern is recognized using the face recognition algorithm for the standard size frontal face photo with size correction/rotation angle correction. The outline of the candidate's face and facial feature points of Eye 2/Nose/Ear 2 are extracted, and through feature extraction and classification, the facial feature points of Eye 2/Nose/Ear 2 on a 5-point scale are left/right ear and left ear respectively. / Calculate the Euclidean distance (d) and similarity with the center point (pupil) of the right eye, compare it with the data of the face photo DB of the online test or UBT test server 100 to check whether it is a proxy test , In the online test or UBT test, the distance between the right eye and the right ear and the distance between the left eye and the left ear change according to the angle at which the front face is turned left and right. It is transmitted to the server 100, the supervisor terminal linked to the test server 100 checks it, and generates an alarm or warning message to the corresponding candidate terminal.

Each test taker terminal includes a face recognition module, an eye 2/nose/ear 2 facial feature point 5-point scale anti-cheating module, and a voice recognition module. A visual cheating detection module using facial contour recognition technology of the camera vision image data of the terminal and a test taker's voice signal input into the microphone related to auditory cheating of the test taker's voice signal when a sound higher than the standard decibel (dB) is generated. Equipped with a voice recognition module for detecting,

The face recognition module of the test taker device recognizes the facial behavior pattern captured by the front camera of the test taker device, extracts a standard size face object, extracts facial feature points on a 5-point scale of eyes/nose/ears, and outlines the face of the test taker. and eye 2/nose/ear 2 feature points, and Euclidean distance (d) and similarity ( similarity), detects head movement to the right/left according to whether 3 points of eyes/nose get closer to 2 points of ears at both ends, detects abnormal facial behavior patterns related to cheating, and recognizes facial contours during face recognition. If this is not possible, if it is out of the test screen of the test taker terminal (when the distance between the eyes and ears and the distance between the nose and ears in the corresponding direction exceeds a certain standard depending on whether the 3 eye/nose points are closer to the 2 ears at both ends) It is recognized as an act and a face photo or face image related to the cheating is transmitted to the test server 100, and / or

When the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard decibel (dB) of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server 100, and the test server 100 ) through which the test taker's cheating information is output to the supervisor's terminal, and a warning message or alarm is received and displayed on the test taker's terminal from the test server 100.

Candidate terminals (300, 310, 311) are connected to the online test or UBT test server 100 through wired and wireless communication networks (LAN, Wi-Fi, LTE 4G/5G), and the artificial intelligence face recognition module and the face of eyes 2 / nose / ears 2 Equipped with a 5-point scale anti-cheating module, detects the ROI (Region of Interest) of the facial image captured in real time with the front camera of the test taker's device during the online test or UBT test, and has a standard size based on the vertex of the nose. It creates a standard size face photo that has been calibrated/rotated/angle corrected, extracts the face object, face outline, and eye 2/nose/ear 2 facial feature points. The feature vector x of the contour line and eye 2/nose/ear 2 face feature points is obtained, and the Euclidean distance (d) and similarity of the face feature points are calculated through feature extraction and classification of eye 2, nose, and ear 2. In the online test or UBT test server, it is compared with the face photo DB of the online test or UBT test server to check whether it is a proxy test. to transmit cheating information including a face photo or facial image to the test server 100, and when the cheating information is confirmed by the supervisor's terminal, an anti-cheating alarm received from the test server 100 to the test taker's terminal is generated, or print a warning message

The test taker terminal uses facial contour recognition technology (posenet algorithm) for the camera image by the face recognition module and the facial feature point 5-point scale cheating prevention module, and determines whether the 3 eye/nose points approach the 2 ears at both ends. / Detects head movement to the left, detects abnormal behavior patterns of the face related to cheating, and detects facial contours during face recognition, when it is out of the test screen (3 eyes/nose points are close to 2 ears at both ends) If the eye-to-ear distance or nose-to-ear distance in the corresponding direction exceeds a certain reference value (value), or the voice signal of the test taker detected by the voice recognition module of the test taker terminal with the microphone of the test taker terminal is Gaussian When a sound higher than the standard decibel (dB) of noise is generated, cheating information including a visual face picture or face image of the test taker terminal and/or an audible voice signal is transmitted to the test server (100) to prevent audiovisual cheating. ), the audio-visual cheating information of the candidate is displayed on the supervisor's terminal through the test server 100, and an anti-cheating alarm or warning message is sent to the corresponding candidate's terminal under the control of the supervisor's terminal through the test server 100. to be sent and output.

After taking a picture of the QR code attached to the candidate's gaze card with a smartphone, the camera of the candidate's terminal is operated by touching the recognition screen, and after recognizing the QR code attached to the candidate's gaze card, a greeting is provided through TTS (Text To Speech) Include more steps.

For the test paper questions of the test program, not only text and images, but also multimedia test questions including text, images, VR/AR content, voice and video are presented and displayed for each question.

The method of providing an online test using facial contour recognition artificial intelligence,

The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which a test program (App) is installed, and the test program is installed, and the face recognition module and five facial feature points of eyes 2 / nose / ears 2 are installed. A scale anti-cheating module and a voice recognition module are provided.

The voice recognition module is provided with a voice recognition module for recognizing the candidate's speech in order to prevent the candidate's audible misconduct in the candidate terminal, and the candidate's voice signal detected by the microphone of the candidate terminal during the online test or UBT test When a sound higher than the standard decibel (dB) of Gaussian noise is generated (if the test taker's speech related to auditory cheating is heard), the test taker's voice data input through the microphone of the test taker terminal is immediately sent to the online test or UBT test server ( 100), and transmits audio-visual cheating information of the candidate to the supervisor's terminal through the test server 100, and the test server 100 so that the candidate's terminal prevents cheating under the control of the supervisor's terminal The method further includes generating an alarm or transmitting a message to the corresponding candidate terminal through

The speech recognition module uses a method using vector quantization, a method using dynamic time warping (DTW), a method using a neural network, and a method using HMM (Hidden Markov Model). Any one of the methods is used, and a STT (Speech To Text) module equipped with a voice recognition API is used.

When the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard decibel (dB) of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server 100, and the test server 100 Information about the candidate's cheating behavior is output to the supervisor's terminal through , and a warning message or alarm is output from the test server 100 to the candidate's terminal under the control of the supervisor's terminal.

In a non-face-to-face online test or UBT test, the test taker terminal detects the ROI of the face image captured in real time by the front camera of the test taker device and uses a face recognition algorithm for a standard size frontal face photo that is resized/rotated/angle corrected. Use to extract face objects and recognize facial behavior patterns to extract the contours of the candidate's face and facial feature points of eyes 2/nose/ear 2, and through feature extraction and classification, the facial feature points of eyes/nose/ears Calculate the Euclidean distance (d) and similarity between the center points (pupils) of the left/right ears and the left/right eyes, respectively, and calculate the standard size front face of the face photo DB of the online test or UBT test server 100 It compares the face photo to check whether it is a substitute test, and recognizes the facial behavior pattern captured by the camera of the test taker terminal during the online test or UBT test, and the distance between the right eye and the right ear and the left eye according to the angle at which the front face is turned left and right. Since the distance between the ear and the left ear is different, misbehavior is determined and the face photo or face image is transmitted to the test server 100, and the supervisor terminal linked to the test server 100 confirms this and an alarm or warning message is sent to the test taker terminal occurs.

In addition, the method displays a warning message to the test taker terminal at the time when the defined frame or time or number of times is subtracted from the camera image of the test taker's terminal in units of frames, such as [Text A], and becomes 0. /Including the step of resetting the warning trigger count and starting the deduction again from the next movement after exposing or transmitting the supervisory device or cheating detection image to the test server.

[Text A] - Facial behavior pattern information stored on the tablet PC of the test taker terminal (Ex. Face pattern data of the test taker at Seoul National University College of Dentistry): information such as date, time, movement angle/distance, and phase (left and right) Saves every frame or hourly, displays a warning message to the user based on this, checks the supervisor and detects cheating by recognizing the candidate's face - Sends the corresponding RAW data to the online test or UBT test server and the main online test or UBT server system handle it

200519094106,0.02,R

200519094111,0.03,R

200519094120,0.42,R

200519094148,0.01,R

200519094150,0.21,R

200519094155,0.22,R

200519094158,0.09,R

200519094209,0.11,L

200519094214,0.09,R

200519094216,0.10,R

200519094219,0.09,R

200519094224,0.01,L

In the embodiment, when the candidate connects to the test server and applies for taking the online or UBT test, a test application reception process of performing the test taker's test application information to the test server; A test information notification process in which test information including test schedule, time, and location information is notified to candidates for the test when the test schedule, time, and location are determined; When the test location and time are set, supervisors are set for each test room at the set time and place of the test, supervisor appointment information is provided to notify the supervisor of the appointment, and appointment confirmation information is received from the supervisor to the test server to complete the supervisor setup process. a supervisory establishment process; On the day of the test, wait for the input of the test site setting information from the supervisor of the test site, and when the test site setting information is input from the supervisor, the test program (app) is provided to install it on the smartphone of the test taker. A test site setting process of requesting test access confirmation from test takers connected through a test program installed in the phone; When the test site setting is completed, select one of the list of registered and stored questions to be implemented by referring to the set question type and question selection setting information (difficulty), and provide test questions by communicating with the installed test programs of test takers. And, the test taker terminal receives the answer information of the test paper through the test program to the test server and stores the test progress process for each test taker; A test finishing process in which all test questions are presented, and when the test is over, the supervisor waits for the supervisor to input end confirmation information, and the application program installed on the tablet PC of the test taker is automatically deleted when the supervisor's end confirmation information is input; and a test announcement process in which the answers prepared by each candidate received and stored in the test server are compared with the scoring answer sheet information and scored, and the scoring result information is announced.

Appointment of supervisors is selected from supervisors registered in advance, and these supervisors are contracted supervisors in advance, and they are registered and managed in the test information database of the test server.

When a supervisor is appointed, supervisor appointment information is provided through the test server, and appointment confirmation information is received from the supervisor to complete the supervisor setting process.

Supervisor appointment information includes test location, test time, and number of test takers.

The candidate information includes personal information including a photograph, name, gender, and address of each candidate, and contact information.

The test setting process is a process for checking the parts necessary for the test before the preparation time set in advance on the day of the test and setting up the equipment to proceed with the test.

When the test center checks the candidate information and the face photo of the test taker and the test site setting information, which is the confirmation information, is provided, the test server provides the test program (App) necessary for the test, and the test taker installs the test program on a tablet PC. It is installed in the test program, and data is sent and received through the test program to enter the identification information entered by the test takers when entering the application information to complete the test access confirmation of the test takers.

The test setting information of the supervisor includes information on the number of test takers who participated in the test, and the identification information of the test takers consists of any one of personal information such as names and resident registration numbers entered by the test takers when inputting application information or member IDs.

The test program (App) consists of a process of accessing the system and inputting identification information to download and install the test program, or when the supervisor inputs test setting information in the system, the supervisor It may include a process of providing a path and allowing test takers to install the test program according to the path.

Figure 3a is a diagram showing the concept of a facial contour recognition artificial intelligence platform having a tablet PC, a smartphone, and a PC-based online test/UBT test server at the time of the language, health and medical education online test/UBT test.

Figure 3b is a diagram showing the face recognition function on the Ubiquitous-Based Learning (UBL) and Ubiquitous-Based Testing (UBT) NSDAI platform.

3c and 3d show the goals and User Experience (1. User registration -> 2. Learning -> 3. QR code Creation -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor confirmation and test -> 6.7 UBT App/Web camera facial contour recognition 8. End of test) to be.

4 shows the process from test taker registration to test taking department, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) supervisor confirmation (applicant face/test taker information), 6) test taking - This is a picture showing the process.

5 is a screen for registering/learning/QR code issuing a candidate-face matching code issuing unit-at the time of an online test/UBT test based on machine learning and learning results of a registered candidate's face.

This is the screen for registering candidates/learning/QR code issuance during the online test/UBT test.

6 is a screen of the candidate confirmation unit in the UBT system in which similarity is measured through AI-based facial contour recognition of the test taker's face.

7 is a rule user defining unit and a program test screen when determining candidates.

8 is a test server access / log-in / left-QR code recognition / greetings heard when QR code is recognized (TTS) / right-face recognition (smartphone / tablet PC front camera) / face in the background during an online test or UBT test This is the demonstration screen including the process of starting area recognition/recognition rate display - the UBT authentication process user definition part using the generated user code.

(1) The test taker terminal is equipped with an artificial intelligence face recognition module, eye 2/nose/ear 2 facial feature point cheating prevention module on a 5-point scale, and a voice recognition module to prevent proxy testing and prevent cheating during the test.

The artificial intelligence learning the test taker registration photo generates a recognition code (QR code) for each user, and the test taker holds the test taker with the relevant recognition code (QR code) attached to the camera of the PC/tablet PC at the test site, and AI recognizes the user. Comparing the test taker information with the result model learned from the face photo, calculating the similarity of feature points of face recognition, recognizing it as the person if it exceeds a certain number, moving to the test screen, and taking the test. In the case of less than a certain level (completely insufficient, pending judgment), the information is transmitted to the supervisor's device and the supervisor's personal information is checked before the test is conducted. During the test, the artificial intelligence of the facial recognition module or facial contour/facial feature point 5-point scale recognition cheating prevention module exists in the background or another layer under the test screen or in the same coordinate value position as the test screen, and is attached to the test taker terminal. or use the built-in camera to recognize the user until the end of the test, and when an abnormal pattern is recognized, the user is warned or the test is stopped by taking an action according to a specified pattern, or the abnormal phenomenon is applied to the device without giving any information to the user and normal test status information are saved as images or text/numerical data and sent to the test server after the test is finished to support the test supervisor/test manager's decision on whether or not to take a proxy test/cheating. When the supervisor's terminal gives a prior warning to the test taker's cheating, it is provided in the form of a warning message or alarm related to the test taker's device's cheating through the test server to the test supervisor's or supervisor's device, and various methods such as SMS transmission be notified by

Online test or UBT test can reduce the number of test supervisors by preventing substitute test by face recognition, preventing audio-visual cheating, and reducing the number of test supervisors. can guarantee

<System Configuration>

1) When using online: User program (PC/Smartphone/Tablet)> Wired/wireless communication network (WAN, LTE 4G/5G) and internal network (LAN, Wi-Fi)> Server (test information/applicant information) and identification information It checks the necessary information by exchanging it, returns True if it is correct, False if it is wrong, and the user program executes the specified action according to the returned value. Recognition information and results of the candidate are also transmitted to the supervisor program (PC/Smartphone/Tablet). (Sent by program or via SNS/SMS/email)

2) When using offline: The artificial intelligence learning module is installed in the user program (PC/cell phone/tablet), and after recognizing the facial contours and feature points of the candidate's face, only the final face facial recognition result is transmitted to the test server. When the final recognition result is transmitted from the test taker terminal to the test server, the corresponding information is displayed on the server program (dashboard), and is transmitted to the test supervisor or test manager terminal to the corresponding information program or any one of SNS/SMS/email.

<Some APKs of app prototypes under development - attached for Android>

https://we.tl/t-wFdXexsors File Download Password: nsdevil

You can check some of the above functions directly by installing the app on your Android smartphone or tablet PC.

<web version function address>

1) Face Recognition Module: https://facere.nsdai.org (id: nsdevil, passwd: nsdevil) > Take a picture of the QR code attached to the test taker's test score with a smartphone and place it on the recognition screen on the PC camera to activate.

2) 5-point facial contour anti-cheating module - Facial contour recognition module: https://headpos.ublrandd.com.np (id: nsdevil, passwd: nsdevil), 5 points for eyes 2/nose/ear 2 facial features Scale anti-fraud module

Artificial intelligence face recognition module, eyes 2/nose/ear 2 facial feature points 5-point scale anti-cheating module, tablet PC equipped with camera and microphone equipped with voice recognition module, smartphone, PC-based online test and UBT test , AI-based facial contour recognition technology (posenet algorithm) is used to prevent proxy exams or cheating by recognizing the contours of the test taker's face and facial feature points of eyes 2 / nose / ears 2 on the test taker terminal, and eye / nose 3 Depending on whether the points get closer to the 2 ears at both ends, head movements are detected and tracked to the right/left to detect facial behavior patterns, and if facial contours are not recognized during face recognition, a test in which a camera image of the tablet PC is taken If it is out of the screen (when the distance in the corresponding direction of the eye-to-ear distance or the nose-to-ear distance exceeds a certain number depending on whether the 3 eye/nose points are closer to the 2 ears at both ends), or the test taker detected by the microphone of the test taker terminal If the audio signal of the test taker generates a sound higher than the standard value decibel (dB) of Gaussian noise, a visual face picture or face image of the test taker terminal and/or an audible audio signal are included to prevent audio-visual cheating during the test time. Behavior information is transmitted to the test server 100, audio-visual cheating information of the candidate is displayed on the supervisor's terminal through the test server 100, and the candidate's terminal is controlled by the supervisor's terminal through the test server 100. To send and print an anti-cheating alarm or warning message, or store it in the test taker's terminal and transmit it to the test server 100 at the end of the test, provide scoring results to test takers, substitute test or Audio-visual cheating has been prevented.

Embodiments according to the present invention are implemented in the form of program instructions that can be executed through various computer means, and can be recorded in a computer readable recording medium. The computer readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, hard disks, magnetic media such as floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - A hardware device configured to store and execute program instructions in storage media such as magneto-optical media, ROM, RAM, flash memory, etc. may be included. Examples of program instructions may include those produced by a compiler, machine language codes as well as high-level language codes that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention is implemented as a program and can be read using computer software on a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.) ) can be stored in

Although described with reference to specific embodiments of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiments to illustrate the technical idea as described above, and within the limit that does not deviate from the technical spirit and scope of the present invention It can be implemented with various modifications, and the scope of the present invention should be determined by the claims described later.

100: Online exam or UBT exam server
101: WWW server 102: control unit
103: member registration unit 104: user authentication unit
105: test notification unit 106: QR code management unit
107: Greeting provision unit (TTS) 108: Supervisor confirmation unit
109: face recognition unit 110: cheating processing unit
111: test management unit 120: test information DB
121: candidate DB 123: face photo DB
200: Supervisor terminal 300, 310, 311: Candidate terminal

Claims (12)

An online test or UBT test server that registers test taker information and front face photos, issues a QR code attached to the test taker's test taker, and provides a test program for conducting an online test or UBT test;
Connected to the online test or UBT test server through a wired or wireless communication network, a face recognition module of a face photographed by a front camera of a test taker terminal, and a standard size front face photo that detects visual cheating by recognizing facial behavior patterns A test taker terminal equipped with an eye 2/nose/ear 2 facial feature point 5-point scale cheating prevention module and a voice recognition module for detecting auditory cheating; and
During the online test or UBT test, cheating information including visual face photos or face images of a plurality of test taker terminals and/or auditory audio signals is received through the online test or UBT test server, and sent to the corresponding test taker terminal. Including a supervisor terminal that transmits an anti-cheating alarm or warning message,
After being taken with a QR code recognition smartphone attached to the test taker's test taker, the camera of the test taker's terminal is operated by facing the recognition screen, and the online test or UBT test server recognizes the QR code and provides a greeting through TTS conversion technology Further comprising a greeting service unit (TTS),
The face contour recognition technology used in the face recognition module uses the posenet algorithm, and when face contour recognition is not performed during face recognition, when it is out of the test screen, that is, the eye-to-ear distance and the nose-to-ear distance in the corresponding direction. If it exceeds a certain threshold, it is recognized as cheating, or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound that exceeds the standard value decibel (dB) of Gaussian noise, online test detected by the camera and microphone in the test taker's terminal When an audio-visual cheating behavior is detected by a test taker watching the test taker, the test taker's face picture or face image and the test taker's voice data are transmitted to the test server, output to the supervisor's terminal through the test server, and a warning is issued from the test server to the test taker's terminal. An online test system using facial contour recognition artificial intelligence that prevents cheating using voice recognition, sending messages or alarms. According to claim 1,
The test taker terminal uses any one of a tablet PC, a smartphone, and a PC equipped with a camera and a microphone on which a test program (App) is installed, and facial contour recognition that prevents cheating using voice recognition in which a test program is installed. Online exam system using artificial intelligence. According to claim 1,
The face recognition module of the test taker terminal and the eye 2/nose/ear 2 face feature point 5-point scale anti-cheating module use the face image captured in real time by the front camera of the test taker terminal during the test time during the online test or UBT test. ROI is detected and a face object is extracted using a frontal face recognition algorithm for a face photo of a standard size that has been resized/rotated/angled corrected, and facial behavior patterns are recognized to determine the outline of the candidate's face and eyes2/nose/ear2. The facial feature points of are extracted, and through feature extraction and classification, the Euclidean distance (d) and It calculates the similarity, compares it with the face photo DB of the online test or UBT test server, checks whether it is a proxy test, recognizes the facial behavior pattern captured by the camera of the test taker device during the online test or UBT test, and recognizes the front face. Since the distance between the right eye and the right ear and the distance between the left eye and the left ear vary according to the left and right angles, misconduct is determined, and a face photo or face image related to the cheating is transmitted to the test server and interlocked with the test server. An online test system using facial contour recognition artificial intelligence that prevents cheating using voice recognition, where the inspector terminal confirms this and generates an alarm or warning message to the corresponding candidate terminal. According to claim 1,
The voice recognition module immediately transmits the test taker's voice data input through the microphone of the test taker's terminal to the test server when the sound of the test taker's voice signal detected by the microphone of the test taker's terminal is higher than the standard decibel (dB) of Gaussian noise. Facial contour recognition that prevents cheating using voice recognition by generating an alarm or sending a warning message to the test taker terminal through the test server to prevent cheating by the test taker terminal. Online exam system using artificial intelligence. According to claim 1,
The test taker terminal uses facial contour recognition technology (posenet algorithm) with respect to the camera image by the face recognition module and the anti-cheating module, and moves to the right/left according to whether 3 eye/nose points get closer to 2 ears at both ends. Detects head movement, detects abnormal behavior patterns of faces related to cheating, and detects facial contours when recognizing faces, or when it is out of the test screen (depending on whether 3 eye/nose points get closer to 2 ear points If the eye-to-ear distance or nose-to-ear distance in the corresponding direction exceeds a certain standard), a warning message or alarm is output to prevent cheating using facial contour recognition artificial intelligence to prevent cheating using voice recognition. online exam system. According to claim 1,
The online test or UBT test server
a WWW server connected to the test taker terminal and the supervisor terminal through wired or wireless communication;
A control unit for controlling to provide online or ubiquitous learning (UBL) and test (UBT) to the test taker terminal and the supervisor terminal through a wired or wireless communication network;
a member registration unit that is connected to the control unit and stores and manages ID/Passwd by receiving registered member information of the candidate;
a user authentication unit that is connected to the control unit and authenticates the user using a QR code/Passwd or ID/Passwd;
A test notice unit connected to the control unit and providing a tablet PC, smartphone, PC-based online learning and test or ubiquitous-based online learning (UBL) and test (UBT) schedule and place;
a QR code management unit connected to the control unit, issuing and managing a QR code attached to an examination ticket corresponding to test taker information to test takers;
It is connected to the control unit, receives the face picture of the candidate from the camera from the candidate terminal, compares the candidate information stored in the database of the test server with the face picture, and the supervisor checks whether or not to take the test as a substitute by checking the candidate's face picture at the supervisor terminal and determines the eligibility to take the test. Supervisor checking department to check;
a face recognition unit connected to the control unit, recognizing the outline of the test taker's face and facial feature points of the eye 2/nose/ear 2 using a face recognition module in the test taker terminal and receiving a face recognition result from the test taker terminal;
The distance between the eyes/nose and ears is measured using the facial contour recognition technology (posenet algorithm) for the camera image in the test taker terminal, and the head is moved to the right/left according to whether the 3 eye/nose points get closer to the 2 ears at both ends. It detects and detects abnormal behavior patterns of faces related to cheating, and when facial contours are not recognized during face recognition or when it is out of the test screen, the eye-to-ear distance and the nose-to-ear distance in the corresponding direction reach a certain standard value. or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard decibel (dB) of Gaussian noise, a visual face picture or face image of the test taker's terminal to prevent audio-visual cheating and/or or a cheating processing unit that receives cheating information including an audible voice signal and transmits and outputs a cheating prevention alarm or warning message to a corresponding test taker terminal under the control of a supervisor terminal;
It provides test programs (Apps) and test papers to the candidate terminal and the supervisor terminal, manages candidate information, on-site face photos of candidates, and supervisor information, and manages each candidate terminal within a certain test time during an online test or UBT test. A test management unit that stores test paper writing answers and transmits them to the test server at the end of the test, and stores and manages test takers' test paper writing answers, scoring results, supervisor information, and test taker status information; and
Test information DB for storing test papers, written answers, and scoring results of test taker terminals; a candidate DB and a face DB that store candidate information and a standard size frontal face picture;
An online test system using facial contour recognition artificial intelligence to prevent cheating using voice recognition. According to claim 1,
The test paper questions of the test program (App) prevent cheating using voice recognition, in which multimedia test questions including not only text and images but also text, images, VR/AR content, and audio and video are presented and displayed for each question. Online test system using artificial intelligence for facial contour recognition. Registering member information, and after log-in/user authentication, registering and storing test taker information and frontal face photos in an online test or UBT test server;
issuing, by the online test or UBT test server, a QR code attached to the candidate's application form corresponding to the candidate's information and the front face photo;
notifying, by the online test or UBT test server, the test schedule and location according to the host test;
Taking the QR code attached to the candidate's gaze card with a smartphone, operating it by touching the recognition screen to the camera of the candidate's terminal, and providing a greeting through a TTS function after recognizing the QR code attached to the candidate's gaze card;
The test taker terminal includes a face recognition module, a 5-point facial feature point anti-cheating module, and a voice recognition module, and uses a front face recognition algorithm in the test taker terminal to convert the recognition result of the front face photo of the camera into the online test or UBT. receiving the information received from the test server and checking whether the test taker is the test taker by comparing the test taker information with standard-sized photos and facial feature points, thereby preventing proxy test and determining whether to take the test;
The face contour recognition technology used in the face recognition module uses the posenet algorithm, and when face contour recognition is not performed or when it is out of the test screen, that is, the eye-to-ear distance or the nose-to-ear distance in the corresponding direction. If it exceeds a certain threshold, it is recognized as a visual cheating, or if the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound of more than the standard value decibel (dB) of Gaussian noise, the test taker's device detected by the camera and microphone When an audiovisual cheating behavior of a candidate taking an online test is detected, the candidate's face photo or face image and the candidate's voice data are transmitted to the test server, output to the supervisor's terminal through the test server, and from the test server to the candidate's terminal Receiving a warning message or an alarm with; and
A test program (App) and online test papers are provided to the candidate terminal and the supervisor terminal, and the candidate information, the candidate's on-site face picture, and supervisor information are stored and managed in the database of the test server, and during the online test or UBT test During a certain test period, each test taker terminal stores test paper preparation answers, and at the end of the test, the test taker terminal receives and stores the answers to the test server, and provides the scoring results of the test takers' test paper preparation answers to the test taker terminal. An online test delivery method using facial contour recognition artificial intelligence to prevent cheating. According to claim 8,
The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which a test program (App) is installed, and the test program is installed, and the face recognition module and eye 2 / nose / ear 2 feature points 5 A method for providing an online test using facial contour recognition artificial intelligence to prevent cheating using voice recognition, comprising a dot scale anti-cheating module and a voice recognition module. According to claim 8,
Each test taker terminal includes a face recognition module and an anti-cheating module, and the anti-cheating module detects visual cheating using facial contour recognition technology of the camera vision image data of the visually test taker terminal during the test. /Nose/Ear 2 facial feature point 5-point scale cheating detection module and voice recognition module that detects the test taker's voice signal input into the microphone related to auditory misconduct of the test taker's voice signal when a sound higher than the standard decibel dB occurs Equipped with
The face behavior pattern captured by the front camera of the test taker terminal by the face recognition module of the test taker terminal and the eye 2/nose/ear 2 facial feature points 5-point scale cheating prevention module and voice recognition module recognizing the face behavior pattern Recognizes and extracts a face object, extracts the outline of the viewer's face and facial feature points of eyes 2/nose/ear 2, and center points of left/right ears and left/right eyes of eye 2/nose/ear 2 facial feature points, respectively. Euclidean distance (d) and similarity with (pupil) are calculated, and head movement to the right/left is detected according to whether 3 eye/nose points get closer to 2 ears at both ends, and the abnormal behavior pattern of the face is detected, and if the face contour is not recognized during face recognition, or if it is out of the test screen of the test taker terminal (the distance between the eyes and ears and the distance between the nose and ears depends on whether the 3 eye/nose points are closer to the 2 ears at both ends) A method for providing an online test using facial contour recognition artificial intelligence that prevents cheating using voice recognition, including the step of transmitting a face photo or face image related to cheating to a test server if the distance in direction exceeds a certain standard value) . According to claim 10,
When the test taker's voice signal detected by the microphone of the test taker's terminal generates a sound higher than the standard decibel (dB) of Gaussian noise, the test taker's voice data is transmitted to the online test or UBT test server, and through the test server to the supervisor's terminal Cheating information of the test taker is output, and a warning message or alarm is output from the test server to the test taker terminal under the control of the supervisor terminal through the test server To prevent cheating using voice recognition Online test delivery method using facial contour recognition artificial intelligence. According to claim 8,
The test paper questions of the test program (App) are not only text and images for each question, but also multimedia test questions that include text, image, VR/AR content, voice and video are presented and displayed to prevent cheating using voice recognition. Online test delivery method using facial contour recognition artificial intelligence.

Images