KR20220009287A - Online test fraud prevention system and method thereof - Google Patents

Abstract

The present invention relates to an online test cheating prevention system using deep learning-based image processing, and a method thereof. The online test cheating prevention system of the present invention comprises: an examinee terminal; an online test server for receiving image information generated by a camera of the examinee terminal through a network; and an image processing server group for receiving the image information from the online test server through the network to perform user authentication, and initiating an online test only for an examinee who is user-authenticated and is not detected with an object used for cheating. Accordingly, the authenticity of an examinee can be secured and cheating can be detected even during a test so as to provide the fairness, such that a field to which an online education and an online evaluation can applied can be expanded.

Description

Online test fraud prevention system and method using deep learning-based image processing

The present invention relates to a system and method for preventing cheating in an online exam, and more specifically, it is possible to prevent cheating by detecting an object corresponding to a test taker's cheating or an article when conducting an online exam using deep learning-based image processing. It relates to a system and method for preventing online exam cheating that can be done.

Our society is changing to the untact era for convenience and environmental changes such as the spread of virus infections, and accordingly, online tests are being introduced instead of test takers gathering in a certain place to take the test. However, the online test provides considerable advantages such as convenience and time security by taking the test in each space provided by the test taker, but on the other hand, there is a big problem in preventing cheating.

Accordingly, various technical factors have been proposed to secure the fairness and reliability of the online test. For example, there were measures to disable the manipulation of keyboard or mouse other than certain characters, or to prevent the use of other programs. In addition, there was a video-based fraud prevention method in which the supervisor monitors the video input through the webcam to prevent cheating. Have.

Looking at the prior art, Korean Patent Application Laid-Open No. 10-2016-0053155 discloses a system for preventing cheating that can block proxy attendance, negligence in class attitude, and proxy exams that occur during online learning using a USB camera connected to a computer. is presented. In Korean Patent Application Laid-Open No. 10-2018-0050968, a method of detecting cheating in real time using video, sound, etc. of test takers is also proposed.

However, in these prior art techniques, when information suspected of cheating is confirmed during the test, the test administrator must always monitor the test management status in order to detect the reason for ineligibility in real time. there was. Moreover, in the prior art, surrogate attendance is checked based on the examiner's face, and if the examinee keeps items to be used for cheating around during the online exam and attempts to cheat, there is no way to detect or limit it.

(Prior Art 001) Laid-Open Patent Publication No. 10-2016-0053155

(Prior art 002) Patent Publication No. 10-2018-0050968

The present invention is to solve this problem, an online test cheating prevention system that can secure reliability and fairness by detecting an act or object suspected of cheating using image processing during an online test and notifying the supervisor about this and methods.

In addition, the present invention provides a system and method capable of not only securing the authenticity of a test taker, but also detecting cheating while taking an exam, and preventing online exam cheating by using deep learning-based image processing. However, the object of the present invention is not limited to those explicitly stated, and includes those that are obvious to those skilled in the art.

In order to achieve the above object, the online test fraud prevention system of the present invention provides image information from a test taker terminal, an online test server that receives image information generated by a camera of the test taker's terminal, and an online test server through a network. The image processing server group may include an image processing server group that allows only test takers to receive through a network, perform identity authentication, authenticate themselves, and for which an object used for cheating is not detected, to take an online test.

The test taker terminal of the present invention may acquire images of the test taker's surrounding environment, face, and motion from a camera attached to any one of a PC, a laptop computer, and a mobile phone.

In addition, the image processing server group of the present invention may include a self-certification server that receives image information from an online test server, extracts the candidate's facial information, and performs self-authentication using the extracted facial information. An object detection server that receives image information from the server, detects an object that can be used for cheating in the surrounding environment using information stored in the module information DB from the image information, and provides the detected object information to an online test server. can

In addition, the image processing server group may include a motion recognition server that receives image information from the online test server and transmits illegal motion information detected from the image information to the online test server, and receives the image information from the online test server to It may include a pupil recognition server that transmits the negative gaze information detected from the image information to the online test server.

On the other hand, the online test fraud prevention method of the present invention comprises a method in which the online test server receives image information generated by the camera of the test taker's terminal through a network, and then transmits the image information to the image processing server group through the network. Step 1, the image processing server group performs self-authentication with the image information received from the online test server, and as a self-certified test taker, the second step of starting the online test only for test takers whose objects used for cheating are not detected. may include

In addition, after the second step, the image processing server group receives image information from the online test server, detects an object to be used for cheating from the image information using the module information DB, and provides the detected object information to the online test server A third step may be included. In addition, a fourth step in which the online test server provides the detected object information to the proctor terminal, and a fifth step of transmitting a test stop command to the test control means of the online test server when the proctor inputs it to the proctor terminal by fraud can do.

The online test cheating prevention system and method of the present invention can provide fairness by not only securing the authenticity of the test taker, but also detecting cheating during the test taking, which can be directly applied to online education and evaluation.

In addition, although multiple supervisors are required to supervise test takers taking an online-based test at a time, the present invention provides a method for inspectors when suspicious actions occur through fraud detection or items that can be used for cheating are detected. There is no need for multiple supervisors because an alarm is generated to notify

In addition, the present invention has the effect of providing flexibility of the online test since the supervisor can restart the test if it is determined that there is no problem after the supervisor checks the video information related to the examinee who has acted suspiciously of cheating. However, the effect of the present invention is not limited to such verbal descriptions, and includes all things that those skilled in the art can infer through the present invention.

1 is a diagram illustrating an online test cheating prevention system of the present invention.
2 is a block diagram illustrating components of an online test server in the online test fraud prevention system according to the present invention.
3 is a flowchart illustrating an object recognition process in an online test fraud prevention method according to the present invention.
4 is a flowchart illustrating a motion recognition process in an online test cheating prevention method according to the present invention.
5 is a flowchart illustrating a pupil recognition process in an online test cheating prevention method according to the present invention.
6 is a flowchart illustrating a learning process for deep learning among the online test fraud prevention methods of the present invention.

The present invention will be described with reference to the accompanying drawings. In the present specification, when any one component transmits data or signal to another component, it means that data or signal is transmitted directly or data or signal is transmitted via at least one other component.

1 is a diagram showing an online test cheating prevention system 1 of the present invention, and FIG. 2 is a block diagram showing components of an online exam server 300 of the online exam cheating prevention system 1 .

Referring to FIG. 1 , an online test fraud prevention system 1 using deep learning-based image processing includes a test taker terminal 100 , a network 200 , an online test server 300 , an image processing server group 400 , It may include a DB server 500 and a deep learning learning server 600 .

The test taker terminal 100 is a test terminal including a PC, a laptop computer, and a mobile phone, and may acquire images of the test taker's surroundings and the test taker's face and motion from an attached camera. It is possible to remove the noise of the image information input from the camera, convert it into a binary image, and provide it to the online test server 300 through the network 200 .

The network 200 is a high-speed backbone network of a large-scale communication network capable of large-capacity, long-distance voice and data services, and may be a next-generation wired or wireless network for providing the Internet or high-speed multimedia services. The network 200 mutually transmits signals and data between the test taker terminal 100 , the online test server 300 , the image processing server group 400 , the DB server 500 and the deep learning learning server 600 , and other systems. plays a role in conveying

The image processing server group 400 may include an identity authentication server 410 , an object detection server 420 , a motion recognition server 430 , and an eye recognition server 440 .

The DB server 500 may include a facial information DB 510 , an exam information DB 520 , and a module information DB 530 . In this specification, the DB may mean a functional and structural combination of software and hardware for storing information corresponding to each database. It may be implemented as at least one table, and may further include a separate DBMS for searching, storing, and managing information stored in the database.

Referring to FIG. 2 , the online test server 300 of the present invention may include a transceiver 310 , a controller 320 , and a database 330 . The control unit 320 may include a test control unit 321 and an alarm unit 322 .

The test control means 321 transmits image information and test taker ID information captured by photographing the user's face and the surrounding environment with a camera installed on the test taker terminal 100 corresponding to the computer terminal operated by the test taker during the online test to the network 200 ), the transceiver 310 may be controlled to receive the image information, and the received image information and candidate ID information may be stored in the database 330 .

Thereafter, the test control means 321 may control the transceiver 310 to transmit the image information and the candidate ID information to the identity authentication server 410 of the image processing server group 400 through the network 200 .

The self-authentication server 410 extracts facial feature information such as the candidate's facial contour, the depth and position of the eyes, nose, and mouth, the shape of the chin, and the height of the cheekbones from the image information and the surrounding environment information of the place where the candidate looks and exam information {taker name, gender, test taker number, server access time, test take The start time, the terminal number (IMEI) and IP address of the test taker's terminal 100, the location to take the test, etc.} and test information can be extracted.

Thereafter, the identity authentication server 410 extracts the stored facial recognition information through access to the facial information DB 510 of the DB server 500, and then extracts the extracted pre-stored facial recognition information (initial facial recognition information) and the received image. In the case of matching through comparison between facial recognition extracted from the information, it may be provided to the test control means 321 of the online test server 300 so that the user can take the test on the extracted test information by completing the authentication.

Specifically, the identity authentication server 410 uses the feature elements of the candidate's face in the initial facial recognition information, and the feature elements are the facial contour, the depth and position of the eyes, nose, and mouth, the shape of the chin, the height of the cheekbones, and the eyebrow bone. From the received image information based on the components such as height and face width, matching can be analyzed based on whether the facial feature components match a preset threshold or percentage (%) or not, as well as from the received image information around the candidate It is possible to identify whether there are objects that are unnecessary for the test or that can be used for cheating in the environment of

The test control means 321 extracts test content corresponding to the test information of the test taker terminal 100 from the database 330 according to the test taker permission of the identity authentication server 410, and then extracts the extracted test information from the test taker terminal 100 ) to start the control in a streaming manner in real time, thereby performing test initiation.

Thereafter, the test control means 321 transmits the initial image information provided and used in the self-authentication process before the start of the test to the object detection server 420 of the image processing server group 400 through the network 200 and the test taker terminal 100 . The transceiver 310 may be controlled to transmit real-time image information provided from the camera together with the candidate ID information.

The object detection server 420 compares real-time image information based on the initial image information, and an object that can be used for cheating in environment information including the environment of the desk on which the test taker's terminal 100 is placed or the test taker's surrounding environment. to detect If detected, it is transmitted to the alarm means 322 of the online test server 300 together with the candidate ID information. Specifically, after extracting the proctor terminal 300a corresponding to the terminal identification number (IMEI) matching the candidate ID information, , the extracted object information detected by the supervisor terminal 300a, and a captured image of the image information from which the object information is detected are transmitted together with the candidate ID information. In addition, candidate ID information, detected object information, captured image information, and object detection time information may be stored in the database 330 .

Here, the object detection server 420 detects an object to be used for cheating in comparison with an object that can be used for cheating stored in the module information DB 530 of the DB server 500 . For example, as an object that can be used for cheating, 2D or 3D product pattern information corresponding to the shape of an electronic calculator, mobile phone, laptop, etc., information about tilting at a preset angle of pattern information, information about inverted information, etc. is stored, and it can be made by comparing it with an object on the image information.

Meanwhile, the test control means 321 extracts test content corresponding to the test information of the test taker terminal 100 from the database 330 according to the test taker permission of the identity authentication server 410, and then extracts the extracted test information from the test taker terminal ( 100) to start the control in a streaming manner in real time, and to perform the test initiation, a request to the motion recognition server 430 of the image processing server group 400 to recognize the motion of the test taker can also be performed. have.

Specifically, the test control means 321 is the motion recognition server 430 of the image processing server group 400 through the network 200, and the initial image information provided and used in the self-authentication process before the start of the test and the test taker terminal 100 It is possible to control the transceiver 310 to transmit real-time image information provided from the camera of the candidate together with the candidate ID information.

Accordingly, the motion recognition server 430 compares the real-time image information based on the initial image information to allow the test taker to leave the seat for a certain period of time or longer while taking the test, or to perform an abnormal behavior (for example, touching the hand for more than a certain period of time). In the event of an action that can be understood as cheating, it is transmitted to the alarm means 322 of the online test server 300 along with the test taker ID information. After extracting the proctor terminal 300a corresponding to the terminal identification number (IMEI) matching the candidate ID information with the extracted proctor terminal 300a, unnecessary or prohibited The object information and the captured image of the image information in which the object information is detected are transmitted together with the candidate ID information. In addition, candidate ID information, detected object information, captured image information, and object detection time information may be stored in the database 330 .

In another embodiment of the present invention, the test control means 321 extracts test content corresponding to the test information of the test taker terminal 100 from the database 330 according to the test taker permission of the self-authentication server 410, and then By starting the control in a streaming manner to the terminal 100 in real time, a request to the eye recognition server 440 of the image processing server group 400 may be performed to recognize the motion of the test taker together with the start of the test.

The test control means 321 is the pupil recognition server 440 of the image processing server group 400, and the initial image information provided and used in the self-authentication process before the start of the test and real-time image information provided from the camera of the test taker terminal 100 The transceiver 310 may be controlled to transmit , along with the candidate ID information.

The pupil recognition server 440 detects the movement of the pupil's pupils through comparison of real-time image information based on the initial image information, and in order for the test taker to deviate from the test taker terminal 100 or a designated place in the test to cheat. If the behavior continues, such as staring at the bottom of the table sitting for a long time, the terminal identification number (IMEI) matching the candidate ID information is transmitted to the alarm means 322 of the online test server 300 together with the candidate ID information. After extracting the supervisor terminal 300a corresponding to ), the captured image of the candidate's negative gaze information detected by the extracted supervisor terminal 300a is transmitted together with the candidate ID information.

When the supervisor terminal 300a receives the captured image and candidate ID information from the object detection server 420, the motion recognition server 430, the eye recognition server 440, etc. together with the alarm information, the touch screen corresponding to the display unit output to the top When it is finally recognized as cheating according to the supervisor's confirmation and confirmation is input on the touch screen, a test stop command is transmitted to the test control means 321 of the online test server 300 . The test control means 321 may transmit a test stop command to the test taker terminal 100 in which the cheating has occurred among the test taker terminals 100 taking the test. If it is determined that there is no cheating, the examination may be continued.

Among the DB servers 500, the facial information DB 510 uses the facial feature elements obtained through the test taker's image information, and the feature elements include facial contours, depth values and positions of eyes, nose, and mouth, chin shape, and cheekbones. Face recognition information such as the height of the eyebrow bone, the height of the eyebrow bone, and face width is stored, and facial recognition information can be stored based on the candidate's ID, date of birth, exam number, and test subject information. To this end, the test control server 321 of the online test server 300 uses the test taker ID, test taker's test information and test information stored in the test information DB 520 before the test as metadata, and converts the facial recognition information into the face information DB ( 510) can be stored.

In the DB server 500, the exam information DB 520 stores the candidate ID after the candidate ID is generated through the network 200 between the candidate terminal 100 and the exam control means 321. Test control means of the online test server 300 receiving the test taker number, the terminal number (IMEI) of the test taker terminal 100 corresponding to the personal information, etc.) and the test information from the test taker terminal 100 through the network 200 321 may store what is stored after access via network 200 .

Accordingly, the test control means 321 for the image information received from the camera attached to the test taker's terminal 100 of the test taker, when the facial recognition information is extracted from the self-certification server 410, the extracted facial recognition information is facial information After controlling to store in the DB 510, and after receiving the test taker information and test information received from the test taker terminal 100 of the test taker, control to store the test information in the test information DB 520, and in the stored state, The authentication server 410 may allow the user authentication to be completed. After that, the test control means 321 may provide the online test questions to the test taker terminal 100 through the network 200 based on the test information and the test information.

3 to 5 are flowcharts illustrating an object recognition process, a motion recognition process, and a pupil recognition process among the online test cheating prevention methods according to the present invention. Referring to FIG. 3 , the online test server 300 uses a front camera or webcam attached to the test taker terminal 100 operated by the test taker before the start of the test, or if the test taker terminal 100 is a smartphone, the test taker using the camera and the candidate's surrounding environment, and transmits the captured image information to the object detection server 420 through the network 200 (S11).

After the step (S11), the object detection server 420 detects an object that can be used for cheating when taking an image in the surrounding environment from the image information in the module information DB 530 through the network 200, and then the detected object The information may be provided to the online test server 300 (S12).

After step S12 , the online test server 300 provides the detected object information to the supervisor terminal 300a ( S13 ), so that the inspector who operates the supervisor terminal 300a may be informed of the detected object information.

After the step (S13), the proctor terminal 300a temporarily stops the provision of the online test question to be performed between the test taker terminals 100 in the online test server 300 until the object information is not recognized in the photographed image information. It may request the online test server 300 (S14).

As described above, in the present invention, the online test server 300 receives the image information generated by the camera of the test taker terminal 100 and then transmits the image information to the image processing server group 400, the image processing server group (400) performing self-authentication with the image information received from the online test server 300 to initiate an online test only for test takers whose identity is authenticated and an object used for misconduct is not detected, an image processing server group ( 400) receives the image information from the online test server 300, detects an object that can be used for cheating from the image information using the module information DB 530, and then transfers the detected object information to the online test server 300 Step of providing, the online test server 300 provides the detected object information to the supervisor terminal 300a, when the supervisor inputs that it is cheating to the supervisor terminal 300a Test of the online test server 300 It includes the step of transmitting a test stop command to the control means 321, and the like.

4 is a flowchart illustrating a motion recognition process in an online test cheating prevention method according to the present invention. The online test server 300 provides a smart front camera or webcam of the test taker terminal 100 and the test taker terminal 100 even during the test taker's test through the online test question provision process for the test taker terminal 100 through the network 200 . In the case of a phone, after receiving the photographed image information of the candidate's entire motion provided through the camera, it may be transmitted to the motion recognition server 430 through the network 200 (S21).

After step (S21), the motion recognition server 430 recognizes the candidate's movement through deep learning-based pose estimation, identifies the candidate's actions, and uses the stored module information DB 530 for the candidate. Through the recognized action of and a pre-specified scenario, an action in which the test taker tries to cheat or an action that is not allowed is extracted (S22).

After step S22, the motion recognition server 430 transmits the detected illegal motion information to the online test server 300 through the network 200, and is detected by the online test server 300 as the supervisor terminal 300a. It is possible to transmit a captured image of the image information in which the illegal operation information including the information about the left-leaning behavior and the abnormal behavior information is detected (S23).

After the step (S23), the proctor terminal 300a temporarily performs the online test question provision process being performed between the test taker terminals 100 in the online test server 300 until the illegal operation information is not recognized in the photographed image information. You can request the online test server 300 to stop at (S24).

5 is a flowchart illustrating a pupil recognition process in an online test cheating prevention method using deep learning-based image processing according to an embodiment of the present invention. The online test server 300 provides through the front camera, webcam, or smartphone camera of the test taker terminal 100 even during the test of the test taker through the online test question provision process for the test taker terminal 100 through the network 200. After receiving the image information of the entire motion of the test taker, it can be transmitted to the pupil recognition server 440 through the network 200 (S31).

After step S31 , the pupil recognition server 440 recognizes the pupil movement in which the candidate attempts cheating through a predetermined scenario, and the recognized operation and the unauthorized operation of the candidate in the stored module information DB 530 . can be extracted (S32).

After step S32 , the motion recognition server 430 transmits the detected negative gaze information to the online test server 300 through the network 200 , and is detected by the online test server 300 as the proctor terminal 300a When the test taker continues to stare at the test taker terminal 100 or a place other than a designated place within the test, the detected negative gazing information may transmit a captured image of the detected image information (S33).

After the step (S33), the proctor terminal 300a temporarily performs the online test question provision process being performed between the test taker terminals 100 in the online test server 300 until the negative test information is not recognized in the captured image information. You can request the online test server 300 to stop at (S34).

6 is a flowchart illustrating a learning process for deep learning among the online test fraud prevention methods using deep learning-based image processing of the present invention. In the online test, the test taker's facial information, surrounding environment, detection object information, eye movement (including negative gaze information), and the overall motion of the test taker photographed using a camera attached to the test taker's terminal 100 or a webcam or smartphone camera Movement (including illegal operation information) may be stored in the module information DB server 530 (S41).

After the step (S41), the deep learning learning server 600 accesses the module information DB server 530 of the DB server 500 through the network 200 and additionally learns the negative stored in the existing DB server 530. By recognizing the cheating situation in addition to the behavioral scenario and additionally storing the cheating situation in the module information DB 530 of the DB server 500 (S42), the online test server 300 is networked to another online exam for the same candidate ID. It is possible to directly access the module information DB 530 of the DB server 500 through 200 .

Through this, even if only partial processes among a plurality of processes corresponding to the flow of cheating situations for each component including the test taker's facial information, surrounding environment, eye movement, and overall movement movement are performed, the network 200 Through this, it is possible to improve the accuracy of the detection of cheating by providing a warning message to the supervisor terminal 300a, thereby improving the reliability of the online test fraud prevention system using deep learning-based image processing.

On the other hand, the deep learning learning server 600 accesses the module information DB server 530 of the DB server 500 through the network 200 and additionally learns cheating in addition to the cheating scenarios stored in the existing DB server 530. By recognizing the situation, it is possible to additionally store the misconduct situation in the module information DB 530 of the DB server 500 . In addition, it is possible to analyze the collected data distributed and stored in the module information DB 530 which is a DCS DB by the distributed file program through a machine learning algorithm and recognize the misconduct situation.

As described above, preferred embodiments of the present invention have been disclosed in the present specification and drawings, and although specific terms are used, these are only used in a general sense to easily explain the technical content of the present invention and help the understanding of the present invention. , it is not intended to limit the scope of the present invention. It will be apparent to those of ordinary skill in the art to which the present invention pertains that other modifications based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein.

1: Online exam fraud prevention system using deep learning-based image processing
100: candidate terminal 200: network
300: online test server 310: transceiver
320: control unit 321: test control means
322: alarm means 330: database
400: image processing server group 410: identity authentication server
420: object detection server 430: motion recognition server
440: eye recognition server 500: DB server
510: face information DB 520: test information DB
530: module information DB 600: deep learning learning server

Claims (6)

Candidate terminal 100;
an online test server 300 for receiving the image information generated by the camera of the test taker terminal 100 of the test taker through the network 200; and
An image processing server that receives image information from the online test server 300 through the network 200, performs self-authentication, is authenticated, and allows only test takers who have not detected an object used for cheating to take the online test Group 400; Online exam cheating prevention system comprising a. The method according to claim 1,
The test taker terminal 100 is an online test cheating prevention system, characterized in that it acquires images of the test taker's surroundings and the test taker's face and motion from a camera attached to any one of a PC, a laptop computer, and a mobile phone. 3. The method according to claim 2,
The image processing server group 400 receives image information from the online test server 300 through the network 200, extracts facial information of the test taker, and performs identity authentication using the extracted facial information. (410); Online exam cheating prevention system comprising a. 4. The method according to claim 3,
The image processing server group 400 receives image information from the online test server 300 through the network 200, and uses information stored in the module information DB 530 from the image information to be used for cheating in the surrounding environment. and an object detection server (420) that detects possible objects and provides the detected object information to the online test server (300). 5. The method of claim 4,
The image processing server group 400 receives image information from the online test server 300 through the network 200 , and transmits the false motion information detected from the image information to the online test server 300 , a motion recognition server 430 . ); Online exam cheating prevention system, characterized in that it includes. A first step of transmitting the image information to the image processing server group 400 after the online test server 300 receives the image information generated by the camera of the test taker's terminal 100 through the network 200;
The image processing server group 400 performs self-authentication with the image information received from the online test server 300 to initiate an online test for a test taker whose identity is authenticated and an object used for misconduct is not detected. step;
After the online test is started, the image processing server group 400 receives the image information from the online test server 300 and uses the module information DB 530 to detect an object that can be used for cheating from the image information, a third step of providing the detected object information to the online test server 300;
a fourth step of providing, by the online test server 300, the detected object information to the supervisor terminal 300a; and
A method for preventing cheating on an online test, comprising a fifth step of transmitting a test stop command to the test control means 321 of the online test server 300 when the proctor inputs cheating into the proctor terminal 300a .

Images