KR20220017329A - Online Test System using face contour recognition AI to prevent the cheating behaviour by using a front camera of examinee terminal installed audible video recording program and a auxiliary camera and method thereof - Google Patents

Abstract

Disclosed are an online test system using face contour recognition artificial intelligence (AI) to prevent cheating by using a front camera of an examinee terminal having an audio/video recording program installed therein and an auxiliary camera and a method thereof. According to the present invention, the system comprises: an online test or ubiquitous-based test (UBT) server providing a test program; an examinee terminal having an audio/video recording program installed therein to interwork with the online test or UBT server, thereby being connected to the online test or UBT server to operate the test program and using AI face/motion/sound recognition technology to detect cheating through facial/motion/sound recognition of images of the examinee's face by using a front camera of the examinee terminal; and a supervisor terminal receiving face photos and cheating information of a plurality of examinee terminals through the online test or UBT server and transmitting a cheating prevention alarm or warning message to the corresponding examinee terminal. The examinee terminal includes a face recognition module, a cheating prevention module based on five face feature points of the eyes, the nose, and the ears, and a voice recognition module. Accordingly, a surrogate test is detected by using a face photo taken by a front camera (C) of an examinee terminal when face recognition is performed, visual cheating is detected by using camera image data captured by the front camera of the examinee terminal and an auxiliary camera of the examinee's rear cradle to detect facial behavior patterns through AI-based facial contour recognition of in real-time, and auditory cheating is prevented through sound recognition.

Description

An online test system using face contour recognition AI to prevent cheating by using a front camera and a secondary camera of a test taker terminal equipped with a recording and recording program using a front camera of examinee terminal installed audible video recording program and a auxiliary camera and method thereof}

The present invention relates to an online test system using artificial intelligence and a method therefor, and more particularly, during a non-face-to-face online test or UBT test, a test taker terminal (tablet PC, smart phone, PC-based) is a recording interworking with a test server /Recording program is installed, AI facial recognition/motion recognition/sound recognition technology is used, and artificial intelligence face recognition module using the front camera and auxiliary camera of the test taker terminal and eye2/nose/ear2 facial feature point 5-point scale negation The behavior prevention module and voice recognition module are mounted, and the face photo taken using the front camera of the test taker terminal during face recognition is compared with the test taker's face photo in the face photo DB of the test server to detect whether or not a proxy test is present, and the test taker terminal AI-based facial contours are recognized in real time using the camera image data taken with the front camera of the candidate’s front camera and the auxiliary camera of the candidate’s rear cradle to detect facial behavior patterns, prevent audiovisual cheating through sound recognition, and record / It relates to an online test system using facial contour recognition artificial intelligence that prevents cheating by using a front camera and an auxiliary camera of a test taker terminal equipped with a recording program.

Face recognition technology is mainly used for face recognition based on an appearance based matching method and a feature. Face recognition is recognized differently depending on the camera's shooting angle, lighting direction, pose, facial expression change, and face change over time.

For face recognition based on features, a detection method using haar-like features for image data captured by a digital camera, a camera of an IoT device, or a camera of a smartphone, and a detection method using a Modified Census Transform (MCT) image are used. . Using the face and eye detectors learned as Haar-like features from the input image of the camera of the smartphone, the contour of the face and the forehead/eyes/nose/mouth are detected, and the region of interest (ROI) is used to detect circular pupils. The eye area set as Interest) is converted to grayscale, and the histogram [x-axis pixel value of each pixel, The number of corresponding pixel values on the y-axis] is calculated, the eye image is binarized, and the photo of the eye area is preprocessed through histogram equalization. In the face area, the eyebrows, eyes, nose, mouth, A face is recognized by detecting the facial features of the outline, extracting texture features and shape features, and comparing the similarity with feature points of the face photos stored in the face recognition DB.

The eyebrow and eye, nose, mouth, and chin feature values of the face region are expressed as the difference between the sum of pixels included in the white region of the Haar-like feature and the sum of pixels included in the black region.

For example, the distance from the detected eye region to the endpoints of the right and left eyes in the face region photos of horizontal and vertical standard sizes, and the size of the iris extracted using the Hough circle transform algorithm. The value is used as the feature value.

As a related prior art 1, Patent Publication No. 10-2017-0050465 discloses "facial recognition apparatus and method".

According to the present embodiment, in recognizing a face from an input image using machine learning, the face recognition rate is improved by normalizing the face pose and perspective, and a virtual face image is automatically generated as face learning data. Provided are a face recognition apparatus and method that saves cost and time for acquiring training data.

1 is a block diagram of an existing face recognition device.

The face recognition device 100 includes an image display device, an image capturing device, a face recognition server, a tablet PC, a laptop, a personal PC, a smart phone, a personal digital assistant (PDA), a mobile communication terminal, and It may be any one of intelligent robots and the like.

The face recognition apparatus 100 includes an input image acquisition unit 112 for acquiring an input image input from a camera; A front pose image is generated by detecting a face region in the input image and normalizing a face pose, and the perspective of the front pose image is removed in order to remove perspective distortion according to the distance between the camera and the subject. ) by normalizing the normalization unit 114 to generate a normalized image; a feature vector extracting unit 116 for extracting a feature vector representing the face of the subject from the normalized image; and a face recognition unit 118 for recognizing the face of the subject included in the input image by applying the feature vector to the previously learned classification model.

The input image acquisition unit 112 acquires an input image input from the camera. The camera may be a depth recognition camera, a stereo camera, or a color camera (eg, a Kinect camera, etc.). In addition, the input image is an image including the face of a subject to be recognized, and includes a two-dimensional still image and Includes video. The input image may include a color image, a depth image, and a color-depth (RGB-D) image.

The normalizer 114 generates a normalized image by detecting a face region from the input image and normalizing a face pose and perspective. When there is a change in the face pose, the face recognition rate is lowered because the gray scale, shape, and position of feature points change. Also, if the distance between the camera and the subject is different, even for the same subject, perspective distortion (eg, distortion) occurs differently at each photographed location, so it may appear as if a different subject was photographed. Therefore, in order to improve the face recognition rate, it is necessary to normalize the face pose and perspective of the input image.

The normalization unit 114 inputs the face images for learning of various poses to the input layer of the first artificial neural network, and trains the first artificial neural network so that the face images for learning of the front pose are output from the output layer of the first artificial neural network. face pose normalization learning unit; and inputting the data output from the output layer of the first artificial neural network to the input layer of the second artificial neural network, and learning the second artificial neural network so that a face image for learning without perspective distortion is output from the output layer of the second artificial neural network. Includes a perspective normalization learning unit.

The normalization unit inputs the learning face images of various poses with various perspective distortions to the input layer of the integrated artificial neural network integrating the first artificial neural network and the second artificial neural network in which learning is completed, and there is no perspective distortion of the front pose. The integrated artificial neural network is trained so that the face image for learning is output from the output layer of the integrated artificial neural network.

The feature vector extraction unit 116 is determined through machine learning and extracts a feature vector representing the face of the subject from the normalized image.

A feature vector is a vector having feature values used for face recognition as elements. As a filter used to extract the feature vector, there are Gabor filter, Haar filter, LBP (Local Binary Pattern) - including DLBP (Discriminative LBP), ULBP (Uniform LBP), NLBP (Number LBP) - etc., but must be limited to this and other filters may be used.

The face recognition unit 118 recognizes the face of the subject included in the input image by applying the feature vector extracted by the feature vector extraction unit 116 to the pre-learned classification model. The previously learned classification model may include a support vector machine (SVM), a linear discriminant analysis (LDA), and a softmax, but is not necessarily limited thereto.

The virtual face image generator 124 may generate a plurality of virtual face images used for learning by the normalizer 114 , the feature vector extractor 116 , and the face recognizer 118 .

The plurality of virtual face images means a face image generated by transforming a three-dimensional face model synthesized by the virtual face image generator 124 using one or more two-dimensional reference images obtained from a camera.

* Features of face recognition system

The recognition rate changes depending on the lighting or distance from the camera.

The shape of the face continues to change according to the angle, expression, and age of the face, and may be recognized differently according to changes in the appearance such as eyebrows, glasses, and makeup.

* Configuration steps of face recognition system

- Image acquisition: Image acquisition from CCD or CMOS Image Sensor camera

- Pre-processing: noise-removing image processing and segmentation

- Face detection: Detects the face area of the ROI area from the input image

- Face normalization: standard size, feature extraction, brightness and geometric normalization

- Face recognition: comparison and recognition of detected images and database images

* Geometric method

- Features: Recognize by extracting geometrical feature points of the face and determining whether they match

- Performance: Because the face is three-dimensional and can be rotated, there is a limit to its application.

* Eigenfaces: Developed by Pentland in 1991.

- Features: PCA (Principal Component Analysis) is applied by extracting key points, and Euclidean distance is applied by measuring similarity.

- Performance: A representative face recognition method that responds sensitively to lighting or environmental changes, but has been widely used and has proven its performance.

* Fisherfaces

- Features: Face recognition method using FLD (Fisher Linear Discriminant) as a classification algorithm.

- Performance: An algorithm that is more accurate and insensitive to environmental changes by learning the characteristics of each person, and can be applied in real time in an on-line state.

* Method based on SVM (Support Vector Machine)

- Features: Algorithm using PCA and SVM (Support Vector Machine) algorithm.

- Performance: Algorithm that has a relatively high recognition rate, but consumes a lot of time and memory to be applied to multi-class applications such as face recognition. A representative algorithm of the face recognition algorithm currently being studied.

* Neural network

- Features: Search for learned face shapes using sliding windows in black and white still images

- Performance: It is possible to extract two or more faces, but the speed is slow and learning is difficult.

* Fuzzy + Neural Network

- Features: As the input of the neural network circuit, the fuzzy membership function is used instead of the pixel brightness value.

- Performance: Performance is improved compared to the method using only neural networks, but processing speed is lower.

* Wavelet + Elastic Matching

- Features: It uses frequency conversion and is effective in handling changes in posture and expression.

- Performance: The amount of computation is high compared to the recognition rate.

As a related prior art, in Patent Registration No. 10-1765770, a system and method for detecting cheating of an online test using image processing and a recording medium storing a program for implementing the same are registered.

The online test cheating detection system and method can supervise online exam cheating and secure the reliability and authenticity of the online exam by using image processing to detect suspected cheating of the examinee during the online exam. Provided are a system for detecting cheating of an online test using image processing that can be performed and a method for detecting the same, and a recording medium storing a program for implementing the same.

1 is a block diagram of an online evaluation system including a conventional online test cheating detection system.

The online evaluation system includes an examinee terminal 110 for an examinee to take an online exam; a camera 120 attached to the examinee's terminal or provided in a smart device to photograph the front of the examinee; a cheating detection server 100 that receives image information photographed by the examinee from the camera in frame units, and detects cheating through image processing; When the cheating detection server determines that the test taker's cheating is suspected, a supervisor terminal 130 for notifying the supervisor; and an online evaluation server 140 for examining the questions of the online test taken by the examinee.

The cheating detection server 100 receives the test taker's image information through the communication network, and when it is determined that cheating is suspected, it sends a cheating detection message to the supervisor, and sends it to the online evaluation server 140 . Transceiver 101 for notifying to stop the online test; a storage unit 102 for storing image information and coordinate information of the examinee received through the transceiver unit 101; a face recognition unit 103 for defining a rectangular area including a face area from the received image information of the examinee; a pupil coordinate obtaining unit 104 for setting pupil coordinates of both eyes by image processing the shape and contrast of both eyes in the coordinate system based on the rectangular region; An intermediate coordinate obtaining unit 105 that determines the intermediate coordinates of the pupil coordinates of both eyes, a background recognition unit 106 that sets the background for all areas except for the body and face areas of the examinee, and the face recognition unit ( 103), the real-time pupil coordinates are compared with the reference pupil coordinates received from the pupil coordinate acquisition unit 104 and the average values of the real-time pupil coordinates are compared to determine whether or not there are two or more face regions. Whether the equilibrium value is outside the preset error range, whether the movement distance of the intermediate coordinates received from the intermediate coordinate obtaining unit 105 is out of the preset error range, and the background received from the background recognition unit 106 and a misconduct determination unit 107 for determining whether a change in the area is detected.

In the example, the rectangular region including the face region was defined based on the content disclosed in the paper “Robust Real-Time Face Detection Using Face Certainty Map” (Volume 4642 of the series Lecture Notes in Computer Science pp 29-38).

The cheating detection server 100,

In the frame-by-frame image information taken by the examinee, a rectangular area including the examinee's face area is defined, and the shape and contrast of both eyes are image-processed in a coordinate system based on the rectangular area, and both end points (310) of both eyes. , 320), set the center points 350 and 360 of the black regions 330 and 340 located between them as the pupil coordinates of both eyes, and calculate the average value of the pupil coordinates of the examinee for a certain period of time before the start of the test to obtain a reference pupil coordinate value, and when the test starts, the average value of the pupil coordinates of both eyes of the examinee is calculated at a predetermined time unit to obtain the average value of the real-time pupil coordinates, and the real-time average value of the pupil coordinates is calculated based on the reference pupil coordinates. If it is out of the set error range, it is notified to the supervisor terminal,

The center point 370 of the pupil coordinates of both eyes is determined as the intermediate coordinate,

The movement direction and movement distance of the intermediate coordinates are tracked on a frame-by-frame basis, and when the movement distance of the intermediate coordinates is out of a preset error range, the examinee's head rotation direction or movement direction is notified, and frame-by-frame image information of the examinee If there are two or more face regions obtained from

It is characterized in that the background is set for all areas except the body and face areas of the examinee, and when a change in the background is detected during the test time, it is notified to the supervisor terminal,

The cheating detection server 100,

When the examinee's image information is received through the communication network and it is determined that cheating is suspected, a fraudulent detection message is sent to the supervisor terminal 130, and the online test question for the examinee is taken. to the online evaluation server 140 to stop the online test.

When the online evaluation server 140 receives a message from the cheating detection server 100 indicating that a suspected cheating operation has been detected while taking the questions of the online test taken by the examinee, the online test It is characterized in that after stopping the question-taking, and after receiving a message from the supervisor terminal 130 or the cheating detection server 100 that the question can be restarted, the online test question-taking is restarted.

Recently, in face recognition technology, head/eye/nose/mouth features using HOG, LBP, and Gabor feature extraction algorithms, which are the mainstream hand-crafted features, are deep learning-based features such as face detection, CNN Using the facial landmark detection and facial feature extraction and classification technology using the (Convolutional Neural Network) algorithm, the human recognition level provides results similar to 97.53%.

In 2014, AI facial recognition technologies such as Facebook's DeepFace were launched. In the case of DeepID1 ["Deep Learning Face Representation from Predicting 10,000 Classes," CVPR 2014 ] announced at CVPR, it uses 39x31 pixels, which is smaller than the 152x152 pixels of DeepFace's input resolution.

In addition, the face detector based on AlexNet and AlexNet trained in advance with ImageNet training data are fine-tuned with face images, so the detection performance is not yet high. are doing

However, due to the proliferation of online non-face-to-face evaluation, the existing online test has a positive effect in terms of convenient non-face-to-face online test taking, and the occurrence of cheating by test takers due to autonomous test execution without supervisor's control is emerging as a social problem. In order to prevent audiovisual misconduct of online exams, there is a testing method that applies technologies such as real-time camera shooting and monitoring of the candidate's face. Due to problems such as the need for input, it is difficult to find a balance in terms of fair and safe testing and high-efficiency non-face-to-face testing.

Patent Publication No. 10-2017-0050465 (published on May 11, 2017), "Face Recognition Device and Method", SK Telecom Co., Ltd. Patent Registration No. 10-1765770 (registration date August 01, 2017), "A system and method for detecting cheating in an online test using image processing and a recording medium storing a program for implementing the same", Bukyung Son

An object of the present invention to solve the above problem is to record a test taker terminal (tablet PC, smart phone, PC-based) interworking with the test server during non-face-to-face online test or UBT test, during non-face-to-face online test or UBT test /Recording program is installed, AI facial recognition/motion recognition/sound recognition technology is used, and artificial intelligence face recognition module using the front camera and auxiliary camera of the test taker terminal and eye2/nose/ear2 facial feature point 5-point scale negation A behavior prevention module and a voice recognition module are mounted, and when face recognition is performed, a face photo taken using the front camera and auxiliary camera of the test taker terminal is compared with the face photo of the test taker in the face photo DB of the test server to detect whether the test is a proxy test, It detects facial behavior patterns by recognizing AI-based facial contours in real time to detect visual misconduct, and prevents auditory misconduct through sound recognition. Provides an online test system using facial contour recognition artificial intelligence that prevents cheating using

In order to achieve the object of the present invention, an online test system using facial contour recognition artificial intelligence that prevents cheating using a front camera and a secondary camera of a test taker terminal, a barcode or an online test or UBT test server that issues QR codes and provides test programs; Connect to the online test or UBT test server to run the test program, and use the front camera of the test taker terminal using AI facial recognition/motion recognition/sound recognition technology for facial recognition/motion recognition and sound recognition of the candidate's face a test taker terminal in which a recording/recording program interlocked with the online test or UBR test server is installed to monitor cheating through ; and a proctor terminal for receiving face photos and cheating information of a plurality of test taker terminals through the online test or UBT test server, and transmitting a fraud prevention alarm or warning message to the test taker terminals.

In order to achieve another object of the present invention, there is provided an online test providing method using facial contour recognition artificial intelligence that prevents cheating by using the front camera and auxiliary camera of the test taker terminal, online test taker information and standard-sized frontal face photos Registering and storing the test or UBT test server; issuing, by the online test or UBT test server, a QR code corresponding to the test taker information and a frontal face photograph, and attaching the QR code to the test ticket; notifying, by the online test or UBT test server, a test schedule and location according to the subjective test; The test taker terminal having a face recognition module receives the recognition result of the front face photo of the camera from the online test or UBT test server using the front face recognition algorithm, and compares the test taker information with the photo and the facial feature points to determine whether the test taker is himself or not. being confirmed; The online test or UBT test server provides a test program (App) to the test taker's terminal and the proctor's terminal. transmitting the face photo, face image, and/or voice data of the test server to the test server, outputting the result to the proctor terminal through the test server, and receiving a warning message or an alarm from the test server to the test taker terminal; and save and manage candidate information, on-site face photos, and proctor information in the test server database and storing the received data from the test server to the test server, and providing the scoring results of the test takers' written answers to the test taker's terminal.

The online system using facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal of the present invention is a non-face-to-face online test or UBT test, a non-face-to-face online test or UBT test, a test taker terminal (Tablet PC, smartphone, PC-based) is installed with a recording/recording program that is linked with the test server, and uses AI facial recognition/motion recognition/sound recognition technology to use the frontal camera and auxiliary camera of the test taker terminal. Recognition module and eye2/nose/ear2 facial feature point 5-point scale anti-fraud module and voice recognition module are installed. It detects whether or not a surrogate test is performed by comparing the test taker's face photo in the DB, and recognizes AI-based facial contours in real time using the camera image data taken with the front camera of the test taker terminal and the auxiliary camera of the test taker's rear cradle to recognize facial behavior patterns It has the effect of detecting visual misconduct by detecting and preventing auditory misconduct through sound recognition.

The test taker terminal is installed with a recording/recording program linked to the online or UBT test server, and the artificial intelligence facial recognition module and eye2/nose/ear2 facial feature 5-point scale used in the test taker terminal are used to prevent cheating, and voice recognition. A method for preventing proxy test combined with modules and preventing audio-visual cheating during online tests was implemented.

The AI-based facial contour recognition technology used in the face recognition module and the anti-fraud module using the eye/nose/ear facial feature point 5-point scale used the posenet algorithm to recognize the facial contour of the tablet PC's camera image. During the online test or UBT test, AI-based facial recognition technology is used to identify the tablet PC test taker through the proctor terminal linked to the test server, and to prevent proxy tests. It is effective in preventing proxy tests and audiovisual cheating.

1 is a block diagram of an existing face recognition device.
2 is a block diagram of an online evaluation system including a conventional online test cheating detection system.
3A is a diagram showing the concept of an artificial intelligence platform for recognizing facial contours having a tablet PC, a smartphone, and a PC-based test online test/UBT test server during the language and health education online test/UBT test.
3B is a diagram showing a face recognition function on the ubiquitous-based learning (UBL) and ubiquitous-based testing (UBT) NSDAI platform.
3c and 3d show the goals and User Experience (1. User Registration->2. Learning-> 3. QR Codes of UBI cloud App, NS facere platform used in tablet PCs, smartphones, and PCs during the online test/UBT test) Create -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor check and test-> 6.7 UBT App/Web camera’s facial contour recognition 8. Test end) to be.
Figure 4 shows the process of test taker registration from test taker registration, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) proctor confirmation (taker face / test taker information), 6) test Gaze - This is a diagram showing the process.
5 is a screen of the candidate registration/learning/QR code issuance screen of the candidate-face matching code issuance unit - online exam/UBT exam based on the registered candidate face machine learning and learning results.
6 is a screen of the candidate identification unit in the UBT system in which the similarity of facial feature points of standard sizes is measured based on the vertex of the nose through AI-based facial contour recognition of the test taker's face.
7 is a view showing a rule user defining unit and a program test screen when determining a candidate.
8 is an online test or UBT test, when connecting to the test server/login/left- QR code recognition/QR code recognition, greeting sound (TTS)/right-face recognition (smartphone/tablet PC front camera)/in the background This is a demonstration screen including the start/recognition rate display process for face region recognition - a UBT authentication process user defining unit using the generated user code.
9 is a block diagram of a tablet PC, smartphone, and PC-based online test system using facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal according to the present invention.
10 is a diagram illustrating the functions of an AI-based facial recognition module and an anti-fraud module during an online test or UBT test.
11 is a flowchart illustrating a tablet PC, smartphone, and PC-based online test providing method using facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal according to the present invention.
12 is a facial contour recognition for preventing cheating using a front camera and an auxiliary camera of a test taker terminal using facial recognition/motion recognition/sound recognition technology during an online test or UBT test according to the second embodiment of the present invention; It is a conceptual diagram of a tablet PC and PC-based online test using artificial intelligence.
13 to 17 are examples of using one or two auxiliary cameras provided in the front camera of the candidate terminal equipped with a GPS receiver and a Wi-Fi communication unit and the rear cradle for monitoring the candidate monitoring blind spot during an online test or UBT test is a drawing showing
18 is a diagram illustrating an interworking unit between the auxiliary camera and the ubtcloud test server.
19 is a diagram showing the operation of a recording/recording program of the developed test taker terminal.
20 is an implementation screen of the test taker recording/recording program (the test taker recording/recording program).
21 is a screen for implementing a test taker's recording/recording program (test program and test server of the test taker's terminal).
22 is a screen showing the characteristics and advantages of a test taker recording/recording program.
23 is an example of automated face tracking and video capturing using a tablet PC camera installed with a recording/recording program interlocked with the test server during an online test or UBT test (Journal of dental education attached document).

Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings, the configuration and operation of the invention. In the description of the present invention, if it is determined that a detailed description of a related known function or known configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, when the accompanying drawing numbers indicate the same configuration, the same reference numbers are given in different drawings.

The online test system using facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal equipped with the recording program of the present invention is a non-face-to-face online test or UBT test. During the test or UBT test, the test taker terminal (tablet PC, smartphone, PC) has a recording/recording program linked to the test server installed, and the front camera of the test taker terminal using AI facial recognition/motion recognition/sound recognition technology Face recognition module using AI-based face recognition (posenet algorithm, machine learning model which allows for real-time face pose estimation) technology using a secondary camera and eye2/nose/ear2 facial feature point 5-point scale to prevent cheating A module and a voice recognition module are mounted, and the face photo taken using the front camera (C) of the test taker terminal during face recognition is compared with the feature points of the test taker's face photo in the face photo DB of the test server to detect the presence of a proxy test The AI-based facial contour line is recognized in real time using the camera image data taken with the front camera of the candidate terminal and the auxiliary camera of the candidate's rear cradle to detect facial behavior patterns to detect visual misconduct and perform sound recognition. to prevent auditory cheating.

The test taker terminal includes a smartphone and tablet PC for an online test (Internet Based Test) or a ubiquitous-based test (UBT), as well as a laptop computer capable of accessing the Internet.

Non-face-to-face online test or UBT test includes elementary, middle and high school exams, university exams, language exams, accredited exams and non-certified exams.

The test taker terminal is equipped with a recording/recording program that is linked with the online or UBT test server, and uses AI facial recognition/motion recognition/sound recognition technology to create an artificial intelligence face recognition module and 5 points of eye 2 / nose / ear 2 facial features It is equipped with a measure-using fraud prevention module and a voice recognition module.

3A is a diagram showing the concept of an artificial intelligence platform for recognizing facial contours having a tablet PC, a smartphone, and a PC-based test online test/UBT test server during the language and health education online test/UBT test.

3B is a diagram showing a face recognition function on the ubiquitous-based learning (UBL) and ubiquitous-based testing (UBT) NSDAI platform.

3c and 3d show the goals and User Experience (1. User Registration->2. Learning-> 3. QR Codes of UBI cloud App, NS facere platform used in tablet PCs, smartphones, and PCs during the online test/UBT test) Create -> 4. UBT App/Web login (QR code/passwd, ID/passwd) -> 5. Supervisor check and test-> 6.7 UBT App/Web camera’s facial contour recognition 8. Test end) to be.

Figure 4 shows the process of test taker registration from test taker registration, 1) test taker registration, 2) learning, 3) QR code issuance, 4) QR code and face recognition, 5) proctor confirmation (taker face / test taker information), 6) test Gaze - This is a diagram showing the process.

5 is a screen of the candidate registration/learning/QR code issuance screen of the candidate-face matching code issuance unit - online exam/UBT exam based on the registered candidate face machine learning and learning results.

This is the screen of candidate registration/learning/QR code issuance during the online test/UBT test.

6 is a screen of the candidate identification unit in the UBT system in which the similarity of facial feature points of standard sizes is measured based on the vertex of the nose through AI-based facial contour recognition of the test taker's face.

7 is a view showing a rule user defining unit and a program test screen when determining a candidate.

8 shows test server access/login/left- QR code recognition/QR code recognition during online test or UBT test, greeting sound (TTS)/right-face recognition (smartphone/tablet PC front camera)/face in the background It is a demonstration screen including the area recognition start/recognition rate display process - the UBT authentication process user definition unit using the generated user code.

(1) The test taker terminal has a recording/recording program installed, and an artificial intelligence facial recognition module, an anti-cheating module using a 5-point scale, and a proxy test prevention technology that combines a voice recognition module and visual/auditory cheating prevention technology during the test has been mounted

The test ticket for the online test or UBT test uses a barcode or QR code as a recognition code, and test takers are provided with a test ticket with the barcode or QR code attached.

In the embodiment, the test taker terminal used a tablet PC and a QR code.

The artificial intelligence that has learned the candidate registration photo generates a user-specific identification code (QR code), and the candidate shines the examination ticket with the identification code (QR code) attached to the camera of the exam room PC or tablet PC, and then the AI recognized by the user Compare the candidate information and the face picture (standard size picture with the nose as the vertex) with the previously registered candidate information in the face DB of the test server and the pre-learned face model, and calculate the similarity of the facial feature points Therefore, if the number exceeds a certain number, it is recognized as a person and moves to the test screen, and the test taker takes the test. In case the similarity of facial feature points is below a certain level (completely below a certain level, withholding judgment), information is transmitted to the supervisor's terminal, and the supervisor conducts an online or UBT test after checking the actual face and personal information.

During the online or UBT test, each test taker's terminal runs the artificial intelligence program of the facial recognition module, the facial contour/five-point scale recognition cheating prevention module, and the voice recognition module in the background.

The test screen is displayed in the front center of the test taker terminal, and the test taker terminal runs a recording program as a background process. 2 5-point scale recognition Misconduct detection and voice recognition are recognized until the end of the test.

When a face pattern or a sound related to auditory misconduct is recognized, an abnormality in which the test taker's face related to visual misconduct is turned left/right at a certain angle or more, cheating notification information is transmitted to the proctor terminal through the test server, and the proctor terminal from the test server to give attention to the test taker or cause an action according to a specified face pattern to stop the test, or to provide information about the anomaly and normal test status to the test taker terminal without giving any information to the user. It is saved as a numerical value and transmitted to the test server after the test is completed, and one or more test supervisor/test manager judges whether the test is a proxy test/whether or not there is cheating. Regarding the test taker's visual/audible misconduct, the test server sends a notification message or alarm to the test taker's terminal to the proctor's terminal through the test server, or an SMS message, and the proctor's terminal A fraud warning message is sent to the test taker's terminal through

Online exams or UBT exams can prevent visual/auditory cheating, proxy exams, and reduce the number of exam proctors, dramatically lowering the cost of online or UBT exams, while also providing minimal test reliability in unsupervised online evaluations. can guarantee

<System Configuration>

1) Online test: Candidate terminal (PC/smartphone/tablet) user program -> Wired/wireless communication network (WAN, LTE 4G/5G) and internal network (LAN, Wi-Fi) -> Server (exam information/candidate information) to transmit the recognition information of each test taker terminal to the test taker terminal and the supervisor terminal checks it. If the misconduct is correct, True/False is returned to the test server do. The candidate's recognition information and results are also transmitted to the supervisor's program on the supervisor's terminal (PC/smartphone/tablet) or via SNS/SMS/e-mail.

2) Offline use: The artificial intelligence learning module is installed in the user program (PC/cell phone/tablet), and after recognizing the facial contour of the test taker's face, only the final facial recognition result is transmitted to the test server. When the final recognition result is transmitted from the test taker terminal to the test server, the corresponding information is displayed on a server program (dashboard) and transmitted to the test manager or the proctor program of the proctor terminal, or SNS/SMS/e-mail is transmitted.

<Part of the app prototype of the test taker's terminal APK - Attached for Android>

https://we.tl/t-wFdXexsors file download password: nsdevil

Install the app on your Android smartphone or tablet PC

<Web version feature address>

1) Face Recognition Module: https://facere.nsdai.org (id: nsdevil, passwd: nsdevil) > Take a QR code with your smartphone and place it on the recognition screen on your PC camera to operate.

2) 5-point scale of facial contour anti-fraud module- Facial contour recognition module: https://headpos.ublrandd.com.np (id: nsdevil, passwd: nsdevil), 2 eyes/nose/ears 2 facial feature points 5 points Scale Anti-Cheating Module

3) Voice recognition module

Speech Recognition is a man-machine interface technology that removes noise from an acoustic speech signal input into a microphone, extracts the characteristics of the speech signal, and converts it into a set of words or text of a sentence (mapping) It is a process, and it is stored in the microphone -> AMP -> LPF -> ADC -> voice database.

The speech recognition module uses a method using vector quantization, a method using dynamic time warping (DTW), a method using a neural network, and a method using HMM (Hidden Markov Model, Hidden Markov Model). method is used The speech recognition module used a speech recognizer with modular hardware and software and a speech recognition API of STT (Speech To Text) technology.

9 is a block diagram of an online test system using facial contour recognition artificial intelligence that prevents cheating using a front camera and an auxiliary camera of a test taker terminal according to the present invention.

10 is a diagram illustrating the functions of an AI-based facial recognition module and an anti-fraud module during an online test or UBT test.

The online test system using facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker's terminal can be used in non-face-to-face online tests or UBT tests.

The online test system using facial contour recognition artificial intelligence that prevents cheating by using the front camera and auxiliary camera of the test taker terminal of the present invention is an online test or UBT test server 100, a proctor terminal 200, and a test taker terminal ( 300,310,311).

The online test system using facial contour recognition artificial intelligence to prevent cheating by using the front camera and auxiliary camera of the test taker terminal.

Issuing barcodes and QR codes attached to test takers' test scores, providing test programs and online test papers to test takers' terminals during online or UBT tests, and recording/recording voice/sound to prevent cheating. / On-line test or UBT test server 100 for controlling the recording start/recording end message of the recorded program, and storing, scoring, and managing test data of a plurality of test taker terminals;

Connect to the online test or UBT test server to run the test program, and use the front camera (C) of the test taker's terminal using AI facial recognition/motion recognition/sound recognition technology to recognize/recognize the image of the test taker's face , test taker terminals 300, 310, 311 in which a recording/recording program interlocked with the online test or UBR test server 100 is installed to detect visual/auditory misconduct through sound recognition; and

The online test or UBT test server 100 receives the face photo and cheating information of the test taker's terminal, checks it, and sends an anti-fraud alarm or and a supervisor terminal 200 for transmitting a warning message.

The test ticket for the online test or UBT test uses a QR code as a recognition code, and test takers are provided with a test ticket with the QR code attached.

The test taker terminals 300, 310, and 311 use any one of a tablet PC, a smartphone, and a PC having a front camera C, and the test program (App) downloaded from the online test or UBT test server 100 is installed. and a recording/recording program linked to the online test or UBT test server is installed.

The test taker terminals 300 , 310 , and 311 are

an artificial intelligence facial recognition module for recognizing facial feature points of a front camera image of a test taker terminal;

During an online test or UBT test, a five-point scale anti-cheating module of eyes2, nose, and ears2, which recognizes a facial behavior pattern of a front camera image of a test taker terminal to prevent visual cheating and constitutes facial feature points; and

It includes a voice recognition module that recognizes the voice of a test taker to prevent auditory cheating during an online test or UBT test.

Additionally, the test taker terminal further includes a cursive recognition unit of the test taker terminal having a touch sensor for recognizing the cursive text by recognizing the cursive of the stylus pen and converting it into text for the subjective test questions, and a display.

Additionally, the online exam or UBT exam offers subjective exams as well as 2/3/4//5 multiple multiple choice exams. The subjective test questions include a subjective test for recognizing cursive characters by recognizing the cursive handwriting of the stylus pen by using the cursive recognition unit of the test taker terminal having a touch sensor and a display, and converting them into characters.

The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm.

The face recognition module and the 5-point scale anti-fraud module detects facial behavior patterns by detecting and tracking head movement to the right/left according to whether the three facial feature points, the eyes/nose, get closer to the two ends of the ears, and FACE RECOGNITION/FACE MOTION If facial contour is not recognized during face recognition through RECOGNITION/RESULT ANLAYSIS, or if the tablet PC camera image is deviated from the test screen by more than a certain angle (3 eyes/nose points to 2 points on both ends of the ears) If the distance in the direction of the eye/ear distance and nose/ear distance exceeds a certain number depending on how close it gets, a warning message or alarm is output or stored in the test taker terminal and sent to the test server when the test is finished. Provides scoring results to test takers.

The test taker terminal is equipped with a recording/recording program that is linked with the online or UBT test server, and uses AI facial recognition/motion recognition/sound recognition technology to create an artificial intelligence facial recognition module and 5 points of eye2/nose/ear2 facial features It is equipped with a scale-using cheating prevention module and a voice recognition module, and in conjunction with an online or UBT test server, it implements an online or UBT proxy test prevention and audiovisual cheating prevention technique.

The AI-based facial recognition technology used in the face recognition module of the test taker terminal and the anti-fraud module using a 5-point scale used the posenet algorithm to recognize the facial contours and facial features of the camera image of the tablet PC. During the online test or UBT test, AI-based facial recognition technology is used to identify the tablet PC test taker through the proctor's terminal linked to the test server, and to prevent proxy test, and to recognize the test taker's face and voice in the online test or UBT test to prevent proxy testing or visual/auditory cheating by

The test paper questions of the test program (App) include subjective and/or multiple-choice test questions, and for each question, multimedia test questions including text, images, VR/AR contents, audio and video as well as text and images are presented and displayed. do.

1. AI Hub Platform

ㆍ NSD-AI (NSDevil's Artificial intelligence) platform-based artificial intelligence hub platform

ㆍ Instructor scoring guide through syntax recognition in language and medical education fields (joint research with Inje University College of Medicine and others)

ㆍ Experiential learning support system through object recognition (FunEdu Lab and Korean Folk Village, etc.)

ㆍ Disease analysis function in the dental education sector through tooth image analysis (joint research with Yonsei University College of Dentistry and other institutions in 3 countries)

ㆍ Based on UBT connect platform

ㆍ Tablet PC-based evaluation platform

ㆍ Language evaluation section (speaking/listening/reading/writing)

ㆍ Health care evaluation sector (listening/reading/writing)

ㆍ Other job evaluation areas (read/write)

The online test or UBT test server 100 registers and stores test taker information and a frontal face photo from a user terminal through a wired/wireless communication network, and issues a barcode or QR code to be attached to the test taker to test takers, and a barcode or QR code for each test taker. After code recognition, a greeting is provided through TTS conversion technology, and an online test paper and a test program for conducting an online test or UBT test are provided to test takers terminals.

Additionally, the online exam or UBT exam offers subjective exams as well as 2/3/4//5 multiple multiple choice exams. The subjective test questions provide a subjective test for recognizing cursive characters by recognizing cursive characters of a stylus pen using a cursive recognition unit of a test taker terminal having a touch sensor and a display, and converting them into characters.

The online exam papers can be given in a mixture of multiple multiple-choice and multiple-choice/subjective.

Candidate terminals (300, 310, 311) are connected to the online test or UBT test server 100 through a wired/wireless communication network (LAN, Wi-Fi, LTE 4G/5G), run the test program, AI face recognition / motion recognition / sound recognition technology to monitor auditory misconduct through facial recognition/motion recognition of images of the candidate's face using the front camera (C) of the candidate terminal using technology, and sound recognition through the microphone and voice recognition module of the candidate terminal. The artificial intelligence facial recognition module of the candidate's face photographed by the front camera (C) and the eye2/nose/ear2 constituting the feature points of the face by recognizing the facial behavior pattern photographed by the front camera (C) of the candidate terminal A five-point scale cheating prevention module and a voice recognition module for detecting auditory cheating through a test taker's voice recognition are provided, and a recording/recording program is installed in the test taker's terminal interlocked with the online test or UBR test server 100 do.

The proctor terminal 200 receives, through the online test or UBT test server 100, face photos and cheating information of a plurality of test taker terminals during an online test or UBT test, checks it, and sends the test server 100 A fraud prevention alarm or message is transmitted to the test taker's terminal through

The system monitors the front camera (C) and the candidate monitoring blind spot of the candidate terminal having a GPS receiver and a Wi-Fi communication unit during a non-face-to-face online examination or UBT examination, and the rear end of the candidate's shoulder line of each candidate terminal It further includes one or two auxiliary cameras (C1, C2, C3, C4, .. Ck-1, Ck) provided on the cradle of the column support.

The auxiliary camera uses a smartphone built-in camera or a separate Wi-Fi communication unit or LTE 4G/5G communication unit built-in IP camera (IoT device having a communication unit, a control unit, and a storage unit) that communicates with each candidate terminal via Wi-Fi, One or two auxiliary cameras are used on the back shoulder line (left/right) of the candidate for each candidate terminal,

In the embodiment, the secondary camera is a smartphone camera that transmits camera image data related to cheating in which the face is rotated at a certain angle during an online or UBT test using UDP/IP directly as a test server, but is not limited thereto. A separate communication unit (Wi-Fi communication unit or LTE 4G/5G communication unit) that transmits camera image data related to cheating using UDP/IP directly as a test server, and an IP camera having a control unit and a storage unit can be used.

The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm (machine learning model which allows for real-time face pose estimation).

In the embodiment, in the online test or UBT test, the test taker terminal used a tablet PC.

The test taker terminals 300 , 310 , and 311 are provided with a face recognition module, an eye 2/nose/ear 2 face feature point 5-point scale anti-fraud module, and a voice recognition module.

Additionally, the test taker terminal further includes a cursive recognition unit of the test taker terminal having a touch sensor for recognizing the cursive text by recognizing the cursive of the stylus pen and converting it into text for the subjective test questions, and a display.

Candidate terminals 300 , 310 , and 311 use the posenet algorithm as the AI-based facial contour recognition technology used in the facial recognition module during an online test or UBT test.

During the online test or UBT test, the 5-point scale anti-fraud module detects the ROI of the face image captured in real time with the front camera of the test taker terminal, and the standard size correction/angle correction is made based on the vertex of the nose. Face object is extracted using frontal face recognition algorithm for size face photo, facial behavior pattern is recognized to extract the outline of the candidate's face and feature points of eyes2/nose/ear2, and eyes through feature extraction and classification Calculate the Euclidean distance (d) and similarity of the facial feature points of 2/nose/ear2 with the center point (pupil) of the left/right ear and the left/right eye, respectively, of the online test or UBT test server It checks whether the proxy test is by comparing it with the photo in the face photo DB, and detects head movement to the right/left according to whether the 3 eye/nose points get closer to 2 points on both ends during the online test or UBT test and detects faces related to cheating Detects abnormal behavior patterns of , and when facial contours cannot be recognized during face recognition, when they deviate from the test screen by more than a certain angle (Eye/nose distance, nose /When the distance in the corresponding direction of the distance of the ear exceeds a certain standard) To prevent cheating, the facial behavior pattern captured by the camera of the test taker's terminal is recognized, and the distance between the right eye and the right ear and the left Since the distance between the eye and the left ear is different, the visual cheating is determined, the cheating image or video data is transmitted to the test server 100, and the proctor terminal 200 linked to the test server 100 checks the A warning message or an alarm is generated to the test taker terminal through the test server 100 .

(Example)

TensorFlow was used for estimating poses of human faces on Android on tablet PC using Posenet algorithm model. For reference, the Posenet algorithm is a vision model that estimates a person's pose from an image or video by detecting the location of the body, arms, and legs, and the location of major body parts. During face recognition, TensorFlow Lite identifies eyes2/nose/ears2 in real time during face recognition using the front camera of the test taker's terminal and implements a facial contour pose estimation model.

The posebet algorithm used in the artificial intelligence facial recognition module uses a 5-point scale anti-cheating module for left eye (leftEye), right eye (rightEye), nose (nose), left ear (leftEar), and right ear (rightEar). implemented,

After obtaining the x,y coordinates of leftEar and the x,y coordinates of rightEar, you can find the diameter of the circle based on these and make an angel ring with ellipse(). The diameter can be easily obtained with dist() in p5js without having to use the Pythagorean theorem.

<Connecting ml5js and webcam>

<script src="https://unpkg.com/ml5@0.3.1/dist/ml5.min.js"></script>

let video;

let poseNet;

function setup() {

createCanvas(400, 400);

video = createCapture(VIDEO);

//Use to prevent the video from appearing twice

video.hide();

poseNet = ml5.poseNet(video,modelLoaded);

poseNet.on('pose',gotPoses);

console.log(ml5);

}

// Pose related callbacks are entered

function gotPoses(poses) {

}

// Used to check if the loading was successful

function modelLoaded() {

console.log('Model Loaded');

}

function draw() {

// Draw the webcam at position 0,0.

image(video,0,0);

}

The face area is extracted in real time from the face image of the front camera of the test taker terminal, and it is adjusted to the standard size of the face photo through size correction/rotation/angle correction based on the vertex of the nose, and the standard size face photo DB stored in the test server Compare with the training data of -> Learning model of face photo -> Extract and classify the feature points of the contours of eyes2/nose/ear2 Clustering (central k-means algorithm) density estimation, and calculating the distance (Euclidean distance) and similarity of facial feature points, standard-sized test taker photos stored in the face photo DB of the online test or UBT test server By comparing the distance and similarity of the facial feature points of the test taker, the proxy test is prevented by verifying the test taker's photo, and visual misconduct is judged during the online test or UBT test.

For example, when the direction of the face photographed by the front camera of the test taker terminal is changed to the test screen of the test taker terminal (tablet PC), or when the test taker turns the face at a certain angle, the feature points of the eye/nose/mouth outlines in the face photo are recognized If this does not work or the angle toward the test program changes within the effective distance (20-30 cm) between the user's face and the smartphone, it is recognized as cheating, and the cheating detection result is sent to the online test or UBT test server 100 transmits it to the proctor terminal 200, and after confirmation by the proctor, an alarm is generated or a warning message is transmitted to the test taker terminal through the test server 100 to prevent the test taker from cheating.

In addition, [Text A] data reference When the warning threshold set in the predefined candidate program becomes 0, a warning message is displayed to the candidate without a server connection, and can be sent back to the proctor/cheating detection server/other test management server, Or, after the test is finished, the raw data is sent to the cheating detection server/other test management server for post-processing.

In addition, the test taker terminal further includes a facial recognition module, a five-point misbehavior prevention module, and a voice recognition module for recognizing the test taker's speech in order to prevent the test taker's visual misconduct,

During the online test or UBT test, the voice recognition module recognizes the voice of the test taker when the test taker's voice related to cheating is heard through the microphone of the test taker's terminal and immediately transmits it to the online test or UBT test server for storage, and the supervisor By sending a notification of cheating to the terminal 200, after the supervisor confirms it, an alarm is generated or a warning message is transmitted to the candidate terminal through the test server to prevent cheating. Alternatively, a warning message is displayed directly from the test taker program itself to the test taker terminal without a server command by referring to predefined distance thresholds such as eye/ear distance and nose/ear distance, and information about cheating is transmitted to the test server and the proctor terminal send to

Online test or UBT test server 100 is WWW server 101, control unit 102, member registration unit 103, user authentication unit 104, test notification unit 105, QR code management unit 106, greetings Study (TTS) 107 , supervisor confirmation unit 108 , face recognition unit 109 , cheating processing unit 110 , test management unit 111 , exam information DB 120 , test taker DB 121 , and face Includes a photo DB (123).

The online test or UBT test server 100 is

a WWW server 101 connected to a test taker terminal and a supervisor terminal using any one of a tablet PC, a smartphone, and a PC through wired/wireless communication;

a control unit 102 for controlling to provide application services of online or ubiquitous-based learning (UBL) and ubiquitous-based testing (UBT) to tablet PCs, smartphones, and proctor terminals of tablet PCs, smartphones, and PCs through wired and wireless communication networks;

a member registration unit 103 connected to the control unit 102, receiving member information of test takers, storing and managing ID/Paaswd;

a user authentication unit 104 connected to the control unit 102 and authenticating a user using a QR code/Passwd or ID/Passwd or an online certificate;

A test notification unit that is connected to the control unit 102 and provides a schedule and location of a tablet PC, a smartphone, a PC-based online learning (Learning) and test, or a ubiquitous-based online learning (UBL) and test (UBT) schedule ( 105);

a code management unit 106 connected to the control unit 102 and configured to issue a barcode or QR code corresponding to candidate information to an examination ticket issued to test takers, and manage it;

After recognizing a barcode or QR code, a greeting providing unit (TTS) 107 that provides a greeting through TTS conversion technology;

It is connected to the control unit 102, receives a photograph of the candidate from the camera from the candidate terminal, compares the candidate information stored in the database with the candidate photograph, and the supervisor confirms whether the examination is a proxy test and confirms the qualification to take the examination a supervisor confirmation unit 108;

A face recognition unit ( 109);

It is connected to the control unit 102 and uses the facial contour recognition technology (posenet algorithm) for the image of the front camera by the facial recognition module and the 5-point scale anti-fraud module in the test taker terminal, so that the three eyes/nose points are both ends of the ears. Detects head movement to the right/left depending on whether it approaches 2 points and detects abnormal behavior patterns of faces related to cheating by viewing cheating papers. In case of deviation by more than an angle (when the distance in the corresponding direction of the eye/ear distance and the nose/ear distance according to whether the 3 eye/nose points are closer to the 2 ear points at both ends exceeds a certain threshold), a warning message to prevent cheating or When the alarm is output, the cheating processing unit 110 for receiving a warning message or an alarm from the test taker terminal;

It is connected to the control unit 102, provides a test program (App) and test papers to the test taker terminal and the proctor terminal, manages test taker information, the test taker's on-site face photo, and proctor information, and when taking an online test or UBT test The test management unit 111 stores and manages test paper written answers in each test taker's terminal within a certain test time, and is transmitted to the test server at the end of the test, and stores and manages test takers' written answers, scoring results, supervisor information and test taker status information;

Test information DB 120 for storing test takers' test papers, written answers, and scoring results; It includes a candidate DB 121 and a face DB 123 for storing candidate information and a standard-sized frontal face photograph.

Online test or UBT test server 100 is a college test, TOEIC/TOEFL test, language test, civil service test, certification test, language education, health and medical education, such as medical school / dental school / pharmacy / engineering college Announcing the test schedule and location of various accredited certification tests or unofficial tests of the question bank that provides online test papers, and using the test program in the test server's database to provide online or offline stored test questions to test takers terminals through wired and wireless communication networks UBT test using all types of PC/smart devices that can take the test using

When the test schedule and location are confirmed, the controller 102 appoints a proctor, provides the proctor appointment information to the proctor, and notifies test takers of the test schedule and location through text message/webpage, and the proctor of the test site on the day of the test The terminal 200 transmits and receives test information and test paper information to conduct and manage the test, and provides a scoring result from the results of the automatic scoring result unit 124 and the inspection management unit 125 to the corresponding candidate terminals.

The test information stored in the test information database 120 includes test title, test schedule and location, test time, location information of test location, assigned proctor information and test takers list, number of seats per test room, proctor information, A/ Includes question information, answer information, scoring information, and supervisor information on each type of test paper.

In addition, the candidate database 121 includes candidate information including name, resident registration number, home address, mobile phone number, e-mail, etc., a photograph of the candidate (standard-sized face-front photograph), QR code/passwd, ID/passwd identification information, and test Relevant information and candidate application status are saved.

11, the tablet PC, smartphone, and PC-based online test providing method using facial contour recognition artificial intelligence to prevent cheating using the front camera and auxiliary camera of the test taker terminal according to the present invention

After receiving member information registration and login/user authentication, the online test or UBT test server registers and stores test taker information and a frontal face photograph (S10);

Step (S20) of the online test or UBT test server issuing a QR code corresponding to the test taker information and the frontal face photo, and attaching the QR code to the test taker's test ticket;

The online test or the UBT test server notifying the test schedule and location according to the subjective test (S30);

Using the frontal face recognition algorithm in the test taker terminal equipped with a face recognition module using AI-based facial contour recognition technology (posenet algorithm), the recognition result of the frontal face photo of the camera is received by the online test or UBT test server, Step (S50) of determining whether or not to take the test by checking whether the candidate is the candidate himself or not by comparing the candidate information and the photograph of the face photo DB with the facial feature points based on the vertex of the nose with the feature points (S50);

The online test or UBT test server provides the test program (App) and the online test paper to the test taker's terminal and the proctor's terminal. Extracts and extracts facial feature points on a 5-point scale of Eye2/Nose/Ear2, recognizes the outline of the candidate's face and Eye2/Nose/Ear2 feature points, and each of the feature points of the Eye2/Nose/Ear2 face Calculate the Euclidean distance (d) and similarity between the left/right ear and the center point (pupil) of the left/right eye, and move to the right/left depending on whether the 3 eye/nose points get closer to the 2 ends of the ears. Detects head movement and detects abnormal behavior patterns of the face related to visual misconduct. When the distance in the corresponding direction of the distance between the eyes/ears and the distance between the nose/ear exceeds a certain threshold according to whether the end ear approaches 2 points) transmitting a test or UBT test server, outputting a cheating notification to the proctor terminal through the test server, and receiving a warning message or an alarm from the test server to the test taker terminal (S60);

The online test or UBT test server stores and manages the test taker information for each classroom, the test taker's on-site face photo, and the supervisor's information in the test server's database. After storing the written answers, the test server receives and stores the written answers from the test taker's terminal to the test server at the end of the test (S70), and the test server provides the scoring results of the test takers' written answers to the test taker's terminal (S80).

The test ticket for the online test or UBT test uses a barcode or QR code as a recognition code, and test takers are provided with a test ticket with the barcode or QR code attached.

In the embodiment, the test taker terminal used a tablet PC and a QR code.

The method is operated by taking a QR code with a smartphone and touching the recognition screen to the camera of the test taker terminal (PC, tablet PC, etc.), and after recognizing the QR code, providing a greeting through TTS (Text To Speech) function (S40 ) is further included.

In addition, the online and UBT exams are a 2, 3, 4, 5, multiple choice multiple-choice test and a subjective test that recognizes cursive characters by recognizing the cursive characters of the stylus pen using the cursive recognition unit of the test taker terminal and converting them into characters. include

The test taker terminal further includes a cursive recognition unit of the test taker terminal including a touch sensor for recognizing the cursive text by recognizing the cursive of the stylus pen and converting it into text for the subjective test questions, and a display.

The candidate terminal uses the posenet algorithm to recognize the facial contours of the camera image of the tablet PC as an artificial intelligence (AI)-based facial contour recognition technology. During an online test or UBT test, AI-based facial recognition technology is used to identify the tablet PC test taker through the proctor's terminal linked to the test server, prevent proxy tests, and facial recognition of the test taker's face online or at the UBT test center It can be used as a control system and prevents proxy tests or cheating in online and UBT tests.

The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which a recording/recording program is installed and a test program (App) is installed, the test program is installed, and the artificial intelligence facial recognition module and eye2 It is equipped with a five-point scale anti-fraud module and voice recognition module of /nose/ear2.

The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm.

The test taker terminals 300, 310, and 311 are provided with a face recognition module and an anti-cheating module on a 5-point scale of facial feature points of eyes 2 / nose / ears 2,

During the online test or UBT test, the ROI of the face image captured in real time with the front camera of the test taker's terminal is detected, and the size correction/angle correction for the standard size face photo based on the vertex of the nose is corrected. Recognizes facial behavior patterns through feature extraction and classification using Calculate the Euclidean distance (d) and similarity between the left/right ear and the center point (pupil) of the left/right eye, respectively, and compare it with the online test or the face photo DB of the UBT test server 100 surrogate test In the online test or UBT test, the distance between the right eye and the right ear and the distance between the left eye and the left ear change depending on the angle of the front face turned left and right, so the misconduct is determined and transmitted to the test server 100 Then, the proctor terminal 200 linked to the test server 100 checks and generates an alarm or warning message to the test taker terminal.

Candidate terminals (300, 310, 311) are connected to the online test or UBT test server 100 through a wired/wireless communication network (LAN, Wi-Fi, LTE 4G/5G), and an artificial intelligence facial recognition module and eyes 2, Equipped with a five-point scale anti-corruption module for nose and ears, it detects the ROI (Region of Interest) of the face image captured in real time with the front camera of the test taker terminal during an online test or UBT test to detect the vertex (vertex) of the nose. ), create a standard size face photo with size correction/angle correction based on the standard size, extract the facial object and eye/nose/ear feature points, and the outline of the face and eyes2 /Nose/Ear2 Obtain the feature vector x of facial feature points, calculate the Euclidean distance (d) and similarity of facial feature points through feature extraction and classification of Eye2, Nose, and Ear2, and conduct an online test Alternatively, it is compared with the feature points of the test taker's face in the face photo DB of the UBT test server to check whether the test is a proxy test, and the front face photographed with the camera of the test taker's terminal during the online test or UBT test is negative depending on the angle turned left and right It determines the behavior and transmits the cheating information of the face photo to the server, and when the supervisor terminal confirms the cheating information, it generates a fraud prevention alarm or outputs a message received from the server.

The test taker terminal uses the facial contour recognition technology (posenet algorithm) for the camera image by the facial recognition module and the cheating prevention module to move the head to the right/left according to whether the 3 eyes/nose points get closer to the 2 points on both ends of the ears Detects and detects abnormal behavior patterns of faces related to cheating. If the distance in the corresponding direction of the ear distance and nose/ear distance exceeds a certain standard value (value), a warning message or alarm is output to prevent cheating.

The test paper questions of the test program include subjective and/or multiple-choice test questions, and for each question, not only text and images, but also multimedia test questions including text, images, VR/AR contents, and audio and video are presented and displayed.

The test taker terminal uses any one of a tablet PC, a smartphone, and a PC on which the test program (App) is installed, the test program is installed, and the artificial intelligence facial recognition module and eye/nose/ear facial features 5 points A scale anti-fraud module is provided.

The test taker terminal further includes a cursive recognition unit of the test taker terminal including a touch sensor for recognizing the cursive text by recognizing the cursive of the stylus pen for the subjective test questions and converting it into text, and a display.

In the above method, when the test taker's voice related to cheating is heard during the online test or UBT test, the test taker's voice signal detected by the microphone and the voice recognition module of the test taker's terminal is more than the threshold decibel dB of Gaussian noise. transmitting the voice data of the test taker to the online test or UBT test server, outputting information about the candidate's misconduct to the proctor terminal through the test server, and outputting a warning message or alarm from the test server to the test taker terminal include

The test taker terminal detects the ROI of a face image captured in real time with the front camera of the test taker terminal during a non-face-to-face online test or UBT test, and uses a front face recognition algorithm for a standard-sized face photo with size correction/angle correction. Extracts facial objects through feature extraction and classification and recognizes facial behavior patterns to extract the outline of the candidate's face and eye/nose/ear feature points A surrogate test by calculating the Euclidean distance (d) and similarity to the central point (pupil) of the eye and comparing it with the feature points of the previously registered candidate's face photo in the face photo DB of the online test or UBT test server The distance between the right eye and the right ear and the distance between the left eye and the left ear are different depending on the angle of the front face turned left and right by recognizing the facial behavior pattern recorded with the camera of the test taker terminal during the online test or UBT test. Based on this, a fraudulent act is determined and transmitted to the test server, checked by the proctor terminal linked to the test server, and an alarm or warning message is generated to the test taker terminal through the test server.

In addition, the method stores the user's face movement of the camera image of the test taker terminal in frame units like [Text A], and displays a warning message to the test taker terminal when it becomes 0 after subtracting a defined frame or time or number of times. / After exposure or sending the supervisor device or cheating detection image to the server, reset the number of warning triggers to start subtracting from the next movement.

[Text A] - Facial behavior pattern information stored in the tablet PC of the test taker's terminal (e.g., the face pattern data of the Seoul National University Dental College): information such as date, time, movement angle/distance and phase (left and right) It is saved every frame or hourly, and based on this, a warning message is displayed to the user, and the proctor is checked and the test taker's face recognition fraud is detected. handle

200519094106,0.02,R

200519094111,0.03,R

200519094120,0.42,R

200519094148,0.01,R

200519094150,0.21,R

200519094155,0.22,R

200519094158,0.09,R

200519094209,0.11,L

200519094214,0.09,R

200519094216,0.10,R

200519094219,0.09,R

200519094224,0.01,L

In an embodiment, when the test taker terminal accesses the test server through a wired/wireless communication network and applies for an online or UBT test test, the test application reception process of performing the test application of the test taker to the test server; a test information announcement process of notifying test takers of the test schedule, time and location information including test schedule, time and location information to test takers of the test when the test schedule, time, and location are determined; When the test location and time are set, set the date and time of the set test and the examiner for each laboratory in the location, provide the supervisor appointment information notifying the supervisor of the appointment, and receive the appointment confirmation information from the examiner to the test server to complete the supervisor setting process Supervisor setting process; On the day of the test, the test site proctor waits for input of test site setting information, and when the test site setting information from the proctor is input, the test program (application, App) is provided and installed on the test takers’ smartphones, and the test takers’ smart phone The test site setting process that requests test access confirmation from test takers connected through the test program installed in the phone. When the test site setting is completed, the registered and saved enforcement is performed by referring to the set question type and problem selection setting information (difficulty level). The test proceeding process of selecting any one of the list of questions to be asked, communicating with the installed test program of the test takers to provide test questions, and the test taker terminal receiving the answer information from the test paper through the test program to the test server and storing it for each test taker; A test finishing process in which all test questions are asked and the application program installed on the test takers' tablet PCs is automatically deleted when the proctor's termination confirmation information is inputted, and when the proctor's termination confirmation information is inputted when the test is finished; It consists of a test presentation process in which the answers written by each test taker received and stored by the test server are compared with the scoring answer information and scored, and the scoring result information is announced.

Appointment of proctors is selected from pre-registered proctors, and these proctors are pre-contracted proctors, and they are registered and managed in the test information database of the test server.

When a supervisor is appointed, the supervisor appointment information is provided through the test server and the appointment confirmation information is received from the supervisor to complete the supervisor setting process.

Proctor appointment information includes test location, test time, and number of test takers.

The number of test takers information includes each test taker's photo, name, gender, and personal information including address, and contact information.

The test setting process is a process to check the parts necessary for the test before the pre-set preparation time on the day of the test and to set the device for the test.

When the test center sends test takers, the supervisor checks the test taker information and face picture, and enters the test site setting information, which is the confirmation information, the test server provides the test program (App) necessary for the test, and the test taker can download the test program to the tablet PC It is the process of completing the test access confirmation of test takers by inputting the identification information that the test takers entered when they entered the application information by sending and receiving data through the test program.

The test setting information of the proctor includes information on the number of test takers who participated in the test, and the identification information of test takers consists of any one of personal information such as name and resident registration number or member ID entered by test takers when entering application information.

The test program (App) accesses the system and enters identification information to download and install the test program. Depending on the path, it may include a process for test takers to install the test program.

12 is a facial contour recognition for preventing cheating using a front camera and an auxiliary camera of a test taker terminal using facial recognition/motion recognition/sound recognition technology during an online test or UBT test according to the second embodiment of the present invention; It is a conceptual diagram of a tablet PC and PC-based online test using artificial intelligence.

13 to 17 show one or two auxiliary cameras (WiFi) provided in the front camera of the candidate terminal equipped with a GPS receiver and Wi-Fi communication unit and the rear cradle for monitoring the candidate monitoring blind spot during an online test or UBT test. A drawing showing an example of using a smartphone camera).

In the non-face-to-face online test or UBT test, using the front camera, microphone, and voice recognition module of the test taker terminal, and using the rear auxiliary camera, AI facial recognition/motion recognition/sound recognition technology is used to detect cheating based on test taker behavior. prevent.

The auxiliary camera is a smartphone built-in camera that communicates with each candidate terminal via Wi-Fi or a separate Wi-Fi communication unit or LTE 4G/5G communication unit built-in IP camera (IoT device having a communication unit, a control unit, and a storage unit) or an image storage server A wired CCTV camera equipped with a cable connection connected to (NVR) is used, and one or two auxiliary cameras are used on the back of the candidate for each candidate terminal.

[Secondary Camera] (off-line or on-line connection)

ㆍ Auxiliary camera technology: Camera is connected online from the moment the test taker connects by using the built-in cell phone camera, auxiliary camera, shooting/sound recording (offline), or using a dedicated program to start monitoring.

ㆍ Resolution: Use one or two auxiliary cameras on the back of the test taker to eliminate blind spots for monitoring test takers (attaching a note monitor, attaching it to the front, attaching it to the knee, and taking almost the entire camera of cheating in the blind spot)

ㆍAdditional effect: As evidence to prevent cheating and increase reliability of online test/UBT test, it is transmitted to the test server along with test results at the end of the final test, post-processing, and scoring.

The position of the auxiliary camera provides a guide according to the user's site, and the user adjusts the camera position accordingly. Verification of camera position can be done manually/automatically using remote visual inspection by a supervisor, AI or the system's screen recognition capabilities.

It can be seen that the most optimal position for the current non-face-to-face online test or UBT test is the position where the test taker's legs/whole upper body/front wall can be seen diagonally from the left/right side of the test taker's shoulder line and one span behind.

In the case of a fisheye camera equipped with two lenses that cover 360 degrees, it is also possible to install it on the ceiling or on the seated side of the examinee, so it is necessary to define the optimal position by the user according to the characteristics of the camera, the field situation, the posture of the examinee and the arrangement of the desk. .

- Place the camera including the communication unit in the correct position according to the guide before the test.

- The auxiliary camera that communicates with the test device or directly communicates with the test server/image storage server (NVR) is initialized/set before and after running the test program of the test taker terminal, or when the user logs in. sent to the test server).

- In accordance with the command of the online (Internet or LAN or communication function) test management server/exam supervisor, the test taker shooting begins. The test server continuously checks whether there is communication between the test taker terminal and the auxiliary camera.

- Real-time video recording from the left and right sides of the back of the candidate's shoulder line from the secondary camera is simultaneously transmitted to the exam server through the NVR server, or only a snapshot is transmitted for a certain period of time or according to the administrator's command to reduce the enormous communication usage, The recorded sound is transmitted to the test server.

- Encryption can be performed to protect personal information before transmission to the test server.

- After successful transmission to the test server, the video file is automatically deleted.

[Location Information]

ㆍLocation positioning technology: location information technology such as tablet PC built-in GPS and Wi-Fi location information

ㆍResolution: It is easy to understand the test status of test takers, it is possible to check the initial location of test takers who are taking the test together, and it provides information on the location of the test start/end time, and secures the basis for judging the moving test taker

1) When a test taker logs in, location/time information is transmitted from the test taker terminal to the test server

2) Display the location of the test taker terminal on the map and start monitoring

3) Transmitting location information at regular intervals during the test taker test

4) Candidate answer -> Send final location/time information when submitted to the test server

5) Final analysis in the test server to determine whether or not there is an abnormality in the test taker during the excessive movement distance or in the case of cluster analysis

18 is a diagram illustrating an interworking unit between the auxiliary camera and the ubtcloud test server.

Candidate recording/recording program

(a) Transmission of test information (exam code, test time, test name, examinee information, etc.) encrypted with AES-256 from the ubtcloud test server to multiple test taker terminals;

(b) The test server, which has received the test start message from the test taker's test device, starts video/sound recording to the test taker's terminal's recording/recording program;

(c) After the test server, which has received the test end message from the test taker test device, sends the recording end message to the recording/recording program of the test taker terminal, the video/sound recording ends;

(d) During the test time, the recording/recording data (video file by integrating the sound file and image file) of the recording/recording program of the test taker's terminal is transmitted and saved to the test server.

(1) QR code or barcode recognition or facial recognition with the camera of the device where the APP is installed

(2) Manually input the test code from the test server to the test taker terminal (App, auxiliary camera)

(3) Transmission of test information (exam code, test time, test name, examinee information, etc.) encrypted with AES-256 from the test server to multiple test taker terminals

(4) Sending the test start message from the test device (smart device and PC) to the test server

(5) Start video/sound recording of multiple test taker terminals from the test server -> 1) Automatically during test time, or 2) Video/sound recording according to the test start message from the test taker’s device (Recording rule: a large number of entire video files) It suppresses generation and records only video files judged to be cheating during facial recognition. In order to effectively prevent cheating, images are saved in a few frames/second according to the communication environment and device specifications instead of full frame/second, and the sound is recorded in full)

(6) Sending the test end message from the test device (smart device and PC) to the test server

(7) Transmission of a recording end message from the test server to a plurality of test taker terminals;

- 1) Automatically ends during test time, or 2) Ends recording/recording as soon as the test end message is received (Rules after sending the end message: Pre-processing of saved image files and recorded files, compression, encryption, image visualization, if necessary, image creation, hash value generation, etc.) , and

For encrypted files, automatic deletion process is applied after a certain period of time after the specified date for exception handling such as transmission errors and subsequent verification, temporary storage in the test taker terminal and data transmission process to the test server start)

(8) Transmission of online test data and recorded/recorded data (creating video files by integrating sound files and image files, compressing, and transmitting encryption) from the test taker terminal to the test server

- When transmission to the test server fails, manual/automatic retransmission, supervisor manual upload

To prevent audiovisual cheating of non-face-to-face online or UBT exams;

The test taker terminal is equipped with a recording/recording program that is linked with the online or UBT test server, and uses AI facial recognition/motion recognition/sound recognition technology to create an artificial intelligence face recognition module and 5 points of eye 2 / nose / ear 2 facial features Equipped with a scale-using cheating prevention module and voice recognition module, it is linked with an online or UBT test server to prevent proxy testing of online or UBT tests and to implement audiovisual cheating prevention techniques.

The test method was applied using AI facial recognition/motion recognition/sound recognition technology to capture and monitor the candidate's face camera in real time, and voice recognition technology.

Develop a recording/recording method that limits the data to tens of megabytes or less, even when shooting a camera video for more than an hour on the test taker's terminal, and encrypts and safely transmits the recording/recording data belonging to sensitive data including personal information It provides a function to temporarily store test data and facial recognition and misconduct related data that can be used for exception handling for technical failures such as network disconnection and verification of test organizations. In addition, the secondary camera shooting function that allows test takers to take pictures in a posture that meets the criteria for resolving the monitoring blind spots suggested by the test administration organization without great difficulty is an area that was focused on developing this technology.

For this purpose, the following functions are loaded into the recording/recording program.

ㆍReceives the test program and user's test data from the test server when candidates are identified through QR/Barcode/AI facial recognition so that the cumbersome log-in process can be omitted.

ㆍFor users of older devices where the AI function does not operate smoothly to set the user's correct posture, it is determined whether the user enters the AR guide marker displayed on the augmented reality (AR) and augmented reality screens to support the optimal shooting posture, and In the case of a device with AI function, it uses artificial intelligence CNN and Posenet algorithm to recognize the user's posture feature point, recognizes whether the point enters the AR guide marker, and captures the optimal shooting location.

ㆍThe function of the recording/recording program must be linked with the test server, so if it is not a user-started environment, the test server and the user app of the tablet PC automatically communicate with the test server and refer to the test data in real time or when the network is disconnected run by.

ㆍAutomatic post-processing function at the end of the test (creation of video files by integrating sound files and image files, compression, encryption, and automatic deletion after timer is attached and stored in the test taker’s terminal (user device), and data without automatic deletion timer attached) sent to the test server.

ㆍ Exception handling function (automatic retry when transmission from the test taker terminal to the test server fails, and manual upload to the test server under the supervision of a supervisor in case of continuous failure)

In the embodiment, the test taker terminal used a tablet PC.

19 is a diagram illustrating the operation of a recording/recording program of the developed test taker terminal.

App execution and test data request, QR code (barcode) recognition, face recognition -> data confirmation -> waiting for recording -> start recording (automatic offline recording based on test data) -> handling exceptions such as forcibly closing the app during recording (restart, continue Recording function) -> Waiting for the end of recording -> End of recording -> Processing after the end of recording (collection of images and video files, recorded files) -> Transmission from the test taker terminal to the test server (images, video files, recorded files related to cheating) - > Exception handling of transmission failure (auto retry, manual upload) -> Terminate the app after receiving the processing end message from the test server

20 is an implementation screen of the test taker recording/recording program (the test taker recording/recording program).

Recording/recording program function

ㆍ Test information confirmation function

- QR code or barcode recognition

- Facial Recognition

ㆍ Re-upload the saved file saved in the test taker terminal to the test server

Location recognition guidance after recognizing test information

ㆍ Recognize the posture of the test taker’s face or determine the shooting location

- AR (Augmented Reality) user location designation

- AR based manual positioning

* The user self-checks that the correct posture guide marker shows the position.

- Automatic location setting based on AR and AI

*Using POSENET and CNN algorithm.

*Artificial intelligence determines whether the AR guide marker includes the recognized user's characteristic points, guides the correct posture and then confirms the posture

ㆍWaiting for system recording/recording start message

ㆍ Recording starts automatically after receiving the message

ㆍIn case of network disconnection, automatic recording starts using test time among previously acquired test information

ㆍWaiting for system shutdown message

ㆍ Compression, encryption, and post-processing are executed automatically after receiving the end message from the test server to the test taker's terminal.

ㆍAfter sending data (images, video files, recorded files) related to cheating to the test server

End of program.

ㆍ In case of network error transmission impossible exception, go to the first step of the app on the tablet PC and execute manual upload.

ㆍLeave the encrypted file with the automatic deletion function on the user's device for later verification

21 is a screen for implementing a test taker's recording/recording program (test program and test server of the test taker's terminal).

* Candidate recording/recording program

- Exam information recognition QR code/bar code

- After recognizing QR code/barcode, connect to recording/recording app and send information

- Test server -> Send test start and start message to test taker terminal

- Check recording/recording status and communication status

- Receive the test end, end message from the test server to the test taker's terminal;

- Check the results of data misconduct-related images, video files, and recorded files) to the test server

22 is a screen showing the characteristics and advantages of a test taker recording/recording program.

ㆍ A method of recognizing the correct position of the user defined to photograph the test taker during an online test or UBT test

ㆍ Recognition of candidate position (position/posture) using augmented reality (AR), guide function

ㆍ Candidate posture recognition and guidance function using artificial intelligence (CNN and POSENET algorithm)

ㆍ Automated test site recording/recording function in conjunction with the evaluation system: Control the recording/recording program of the test taker terminal using real-time and asynchronous communication such as the test server and websocket, and automatically start/stop camera shooting

ㆍ Additional functions to strengthen verification: After the online test/UBT test is finished, the recorded/recorded image, sound, and video files are encrypted, and automatically deleted after a certain period of time, including a timer function, and stored in the test taker terminal for a certain period of time and added to be used for verification purposes

ㆍ Compatible with all types of online test systems based on browser-based web/native apps

23 is an example of automated face tracking and video capturing using a tablet PC camera installed with a recording/recording program interlocked with the test server during an online test or UBT test (Journal of dental education attached document).

At the time of face recognition, by using the face photo taken using the front camera (C) of the test taker terminal, the face recognition of the ROI area of the face is compared with the pre-registered face photo of the test taker in the test server and facial feature points. It detects whether the test is a proxy test and detects facial behavior patterns by recognizing AI-based facial contours in real time using the camera image data taken with the front camera of the test taker terminal and the auxiliary camera of the test taker's rear cradle to detect visual misconduct. It is possible to detect and prevent auditory cheating through voice recognition of the test taker's speech.

Embodiments according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, and data structures alone or in combination. Computer-readable recording media include storage, hard disks, magnetic media such as floppy disks and magnetic tapes, optical recording media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - A hardware device configured to store and execute program instructions in a magneto-optical media, and a storage medium such as ROM, RAM, flash memory, or the like may be included. Examples of program instructions may include those generated by a compiler and not only machine language code but also high-level language code that can be executed by a computer using an interpreter. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention.

As described above, the method of the present invention is implemented as a program and can be read using computer software in a form that can be read on a recording medium (CD-ROM, RAM, ROM, memory card, hard disk, magneto-optical disk, storage device, etc.). ) can be stored in

Although described with reference to a specific embodiment of the present invention, the present invention is not limited to the same configuration and operation as the specific embodiment in order to illustrate the technical idea as described above, and within the limit that does not depart from the technical spirit and scope of the present invention It can be implemented with various modifications. Accordingly, such modifications should be considered to fall within the scope of the present invention, and the scope of the present invention should be determined by the following claims.

100: Online exam or UBT exam server
101: WWW server 102: control unit
103: member registration 104: user authentication unit
105: test notification unit 106: QR code management unit
107: Greeting Provider (TTS) 108: Supervisor Verification Department
109: face recognition unit 110: cheating processing unit
111: test management unit 120: test information DB
121: candidate DB 123: face photo DB
200: supervisor terminal 300,310,311: test taker terminal

Claims (23)

an online test or UBT test server that issues bar codes or QR codes on test tickets and provides test programs;
Connect to the online test or UBT test server to run the test program, and use the front camera of the test taker terminal using AI facial recognition/motion recognition/sound recognition technology for facial recognition/motion recognition and sound recognition of the candidate's face a test taker terminal in which a recording/recording program interlocked with the online test or UBR test server is installed to detect cheating through ; and
a proctor terminal for receiving face photos and cheating information of a plurality of test taker terminals through the online test or UBT test server and transmitting a fraud prevention alarm or warning message to the test taker terminal;
An online test system using facial contour recognition artificial intelligence to prevent cheating by using the front camera and auxiliary camera of the test taker terminal, including: According to claim 1,
The test ticket for the online test or UBT test uses a barcode or QR code as a recognition code. Online test system using facial contour recognition artificial intelligence. According to claim 1,
The test taker terminal uses any one of a tablet PC, a smartphone, and a PC equipped with a front camera, and the test program downloaded from an online test or UBT test server is installed, and a recording/recording program is installed,
The recording/recording program of the test taker terminal is
(a) Transmitting AES-256 encrypted test information (exam code, test time, test name, examinee information, etc.) from the ubtcloud test server to multiple test taker terminals;
(b) The test server, which has received the test start message from the test taker test device, starts video/sound recording of the test taker terminal's recording/recording program;
(c) After the test server, which has received the test end message from the test taker test device, sends the recording end message to the recording/recording program of the test taker terminal, the video/sound recording ends;
(d) During the test time, using the front camera and auxiliary camera of the test taker terminal, the recording data (video file by integrating the sound file and image file) of the recording/recording program of the test taker's terminal is transmitted and saved to the test server. Online test system using facial contour recognition artificial intelligence to prevent cheating. According to claim 1,
The test taker terminal
an artificial intelligence facial recognition module for recognizing facial feature points of a front camera image of a test taker terminal;
During the online test or UBT test, a 5-point scale anti-cheating module for eyes2, nose, and ears2 that constitutes facial feature points by recognizing facial behavior patterns from the front camera image of the test taker terminal to prevent visual cheating ; and
In the online test or UBT test, facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal, including a voice recognition module that recognizes the test taker's speech to prevent auditory cheating The online test system used. 5. The method of claim 4,
The AI-based facial contour recognition technology used in the facial recognition module uses the posenet algorithm,
The 5-point scale anti-fraud module extracts a face object using a frontal face recognition algorithm for a standard size face photo with size correction/angle correction during an online test or UBT test, and recognizes a facial behavior pattern to recognize a test taker's face Extract the outline of and the feature points of eye2/nose/ear2, and the Euclidean distance (d) between the left/right ear and the center point (pupil) of the left/right eye of the facial feature points of eye2/nose/ear2, respectively. ) and similarity, compare with specific points in the face photo of the test taker in the online test or UBT test server's face photo DB to check whether the test is a proxy test, and 3 eyes/nose points to 2 points on both ends of the ear Detects head movement to the right/left depending on whether it is getting closer, detects abnormal behavior patterns of the face related to cheating. If the distance in the corresponding direction of the distance between the eyes/ears and the distance between the nose/ear exceeds a certain threshold depending on whether the 3 points get closer to the 2 points on both ends of the ears) Recognizes facial behavior patterns captured by the candidate's camera to prevent cheating Thus, the distance between the right eye and the right ear and the distance between the left eye and the left ear vary according to the angle at which the front face is turned left and right. Facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal, which generates an alarm or a warning message or an alarm to the test taker terminal through the test server after confirmation by the supervisor terminal linked to Online exam system. 5. The method of claim 4,
The voice recognition module immediately transmits and stores the test taker's voice data input through the microphone of the test taker terminal related to auditory misconduct during the online test or UBT test to the online test or UBT test server, and transmits it to the proctor terminal A facial contour recognition artificial intelligence that generates an alarm or sends a warning message to the test taker terminal through the test server to prevent cheating by the test taker terminal, and uses the front camera and auxiliary camera of the test taker terminal to prevent cheating The online test system used. According to claim 1,
During the online test or UBT test, the front camera and auxiliary camera of the test taker terminal further comprising an auxiliary camera provided on a cradle in the rear row behind each test taker's shoulder line to monitor the front camera of the test taker terminal and the candidate monitoring blind spot Online test system using facial contour recognition artificial intelligence to prevent cheating using 8. The method of claim 7,
The secondary camera is a camera of a smartphone that transmits camera image data related to cheating using UDP/IP directly to the online or UBT test server, or a camera image related to cheating using UDP/IP directly to the test server. Facial contour recognition to prevent cheating using the front camera and auxiliary camera of the test taker terminal using an IP camera having a separate communication unit (Wi-Fi communication unit or LTE 4G/5G communication unit) for transmitting data, a control unit, and a storage unit Online exam system using artificial intelligence. According to claim 1,
The online test or UBT test server
WWW Server;
a control unit controlling to provide online or ubiquitous-based learning (UBL) and test (UBT) to the test taker terminal and the proctor terminal;
a member registration unit connected to the control unit, receiving member information of test takers, storing and managing ID/Passwd;
a user authentication unit connected to the control unit and authenticating using a QR code/Passwd or ID/Passwd or a personal certificate;
a test notification unit connected to the control unit and providing a schedule and location of a tablet PC, a smartphone, a PC-based online learning (Learning) and test, or ubiquitous-based online learning (UBL) and a test (UBT);
a code management unit connected to the control unit and configured to issue and manage barcodes or QR codes corresponding to information about test takers in an examination ticket issued to test takers;
Connected to the control unit, the examiner receives the photograph of the candidate from the camera from the candidate terminal, compares the candidate information stored in the database with the photograph of the candidate, the supervisor confirms at the supervisor terminal, checks whether a proxy test is present, and confirms the qualification to take the examination wealth;
a face recognition unit connected to the control unit, using a face recognition module in the test taker terminal to recognize the contour of the test taker's face and eye2/nose/ear2 feature points, and receive a face recognition result from the test taker terminal;
The test taker terminal measures the distance between eye2/nose and ear2 using facial contour recognition technology (posenet algorithm) for the camera image, and moves to the right/left depending on whether the 3 eye/nose points are close to the 2 ear points at both ends. Detects head movement and detects abnormal behavior patterns of the face related to cheating. When a warning message or alarm is output to prevent cheating (if the distance in the corresponding direction of the distance between the eyes/ears and the distance between the nose/ear exceeds a certain threshold depending on whether it approaches the point), a warning message or alarm is received from the test taker's terminal cheating processing unit;
Provides a test program (App) and test papers to the test taker terminal and the proctor terminal, manages test taker information, on-site face photos, and proctor information a test management unit that stores and manages test takers' written answers, grading results, supervisor information, and test taker status information, which stores and manages test takers' written answers and test taker status information; and
Test information DB for storing test papers, written answers, and scoring results of test takers terminals; a test taker DB and a face DB for storing test taker information and a standard-sized frontal face picture;
An online test system using facial contour recognition artificial intelligence that prevents cheating by using the front camera and auxiliary camera of the test taker terminal, including a. 10. The method of claim 9,
The online test or UBT test server
Online using facial contour recognition artificial intelligence to prevent cheating using the front camera and auxiliary camera of the test taker terminal further comprising a greeting providing unit (TTS) that provides a greeting through TTS conversion technology after recognizing the barcode or QR code test system. According to claim 1,
The test paper questions of the test program (App) include subjective and/or multiple-choice test questions, and for each question, not only text and images, but also text, images, VR/AR contents, multimedia tests that include at least one of audio and video Online test system using facial contour recognition artificial intelligence to prevent cheating using the front camera and auxiliary camera of the test taker terminal, in which the questions are asked and displayed. According to claim 1,
The test taker terminal further comprises a handwriting recognition unit of the test taker terminal having a touch sensor for recognizing the cursive text of the stylus pen and converting it into text to recognize the cursive text and a display for the subjective test questions. A front camera and an auxiliary camera Online test system using facial contour recognition artificial intelligence to prevent cheating using According to claim 1,
The test taker terminal
1) When a test taker logs in, the location/time information is transmitted from the test taker terminal to the test server, 2) The location of the test taker terminal is displayed on the map and monitored, 3) The location information of the test taker terminal at regular intervals during the test 4) An online test system using facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal, which transmits final location/time information when submitting candidate answers to the test server. Registering and storing test taker information and a standard-sized frontal face photo as an online test or UBT test server;
issuing, by the online test or UBT test server, a QR code corresponding to the test taker information and a frontal face photograph, and attaching the QR code to the test ticket;
notifying, by the online test or UBT test server, a test schedule and location according to the subjective test;
The test taker terminal having a face recognition module receives the recognition result of the front face photo of the camera from the online test or UBT test server using the front face recognition algorithm, and compares the test taker information with the photo and the facial feature points to determine whether the test taker is himself or not. being confirmed;
The online test or UBT test server provides a test program (App) and an online test paper to the test taker terminal and the proctor's terminal. transmits the test taker's face photo, face image and/or voice data related to the test server, is output to the proctor terminal through the test server, and receives a warning message or alarm from the test server to the test taker terminal from the proctor terminal to do; and
Candidate information, on-site face photos, and proctor information are stored and managed in the test server's database, and during online or UBT tests, the written test paper answers are stored in each test taker's terminal for a certain period of time and then retrieved from the test taker's terminal when the test is finished receiving, storing, and transmitting to the test server, and providing the scoring results of the test takers' answers written on the test paper to the test taker's terminal;
An online test providing method using facial contour recognition artificial intelligence to prevent cheating by using the front camera and auxiliary camera of the test taker terminal, including a. 15. The method of claim 14,
The test ticket for the online test or UBT test uses a QR code as a recognition code, and a face contour recognition artificial How to deliver online exams using intelligence. 15. The method of claim 14,
It operates by taking the QR code with a smartphone and placing it on the recognition screen on the camera of the test taker terminal, and after recognizing the QR code, providing a greeting through a Text To Speech (TTS) function. A method of online exam delivery using facial contour recognition artificial intelligence that uses a secondary camera to prevent cheating. 15. The method of claim 14,
The test taker terminal uses any one of a tablet PC, a smartphone, and a PC equipped with a front camera, and the test program downloaded from an online test or UBT test server is installed, and a recording/recording program is installed,
The recording/recording program of the test taker terminal is
(a) Transmitting AES-256 encrypted test information (exam code, test time, test name, examinee information, etc.) from the ubtcloud test server to multiple test taker terminals;
(b) The test server, which has received the test start message from the test taker test device, starts video/sound recording of the test taker terminal's recording/recording program;
(c) After the test server, which has received the test end message from the test taker test device, sends the recording end message to the recording/recording program of the test taker terminal, the video/sound recording ends;
(d) During the test time, using the front camera and auxiliary camera of the test taker terminal, the recording data (video file by integrating the sound file and image file) of the recording/recording program of the test taker's terminal is transmitted and saved to the test server. An online test delivery method using facial contour recognition artificial intelligence to prevent cheating. 15. The method of claim 14,
The test paper questions of the test program (App) include subjective and/or multiple-choice test questions, and for each question, not only text and images, but also text, images, VR/AR contents, multimedia tests that include at least one of audio and video Online test system using facial contour recognition artificial intelligence to prevent cheating using the front camera and auxiliary camera of the test taker terminal, in which the questions are asked and displayed. 15. The method of claim 14,
The test taker terminal
an artificial intelligence facial recognition module for recognizing facial feature points of a front camera image of a test taker terminal;
During the online test or UBT test, a 5-point scale anti-cheating module for eyes2, nose, and ears2 that constitutes facial feature points by recognizing facial behavior patterns from the front camera image of the test taker terminal to prevent visual cheating ; and
In the online test or UBT test, facial contour recognition artificial intelligence that prevents cheating using the front camera and auxiliary camera of the test taker terminal, including a voice recognition module that recognizes the test taker's speech to prevent auditory cheating The online exam delivery method used. 20. The method of claim 19,
The test taker terminal further comprises a handwriting recognition unit of the test taker terminal having a touch sensor for recognizing the cursive text of the stylus pen and converting it into text to recognize the cursive text and a display for the subjective test questions. A front camera and an auxiliary camera Online test system using facial contour recognition artificial intelligence to prevent cheating using 20. The method of claim 19,
The AI-based facial contour recognition technology used in the facial recognition module uses a posenet algorithm, and uses a front camera and an auxiliary camera of the test taker terminal to prevent cheating. 20. The method of claim 19,
The facial recognition module and the face recognition module and the facial feature point 5-point scale for recognizing facial behavior patterns recognize facial behavior patterns photographed with the front camera of the test taker terminal and extract the face objects by the anti-fraud module and the voice recognition module Then, the outline of the candidate's face and the facial feature points on a 5-point scale of eyes/nose/ears are extracted, and the Euclidean point (pupil) of the left/right ear and the left/right eye of the eye/nose/ear facial feature points, respectively. It calculates the distance (d) and similarity, detects the abnormal behavior pattern of the face by detecting head movement to the right/left depending on whether the 3 eyes/nose points get closer to the 2 points on the ears at both ends. When facial contours cannot be recognized, when the test screen is deviated from the test screen of the test taker's terminal at a certain angle (the distance in the corresponding direction of the eye/ear distance and the nose/ear distance are constant depending on whether the 3 eye/nose points get closer to the 2 ear points at both ends) In case of exceeding the standard value) using facial contour recognition artificial intelligence to prevent cheating using the front camera and auxiliary camera of the test taker terminal, including the step of transmitting the face photo and image related to the cheating at that point in time to the test server How to give online exams. 20. The method of claim 19,
When the test taker's voice signal detected by the microphone and the voice recognition module of the test taker's terminal generates a sound higher than the Gaussian noise threshold decibel dB, the test taker's voice data is transmitted to the online test or UBT test server, and through the test server A facial contour line for preventing cheating by using a front camera and an auxiliary camera of the test taker terminal, including outputting information on cheating of the test taker to the proctor terminal, and outputting a warning message or alarm from the test server to the test taker terminal How to deliver online exams using cognitive artificial intelligence.

Images