KR20190118993A - Surrogate Interview Prevention Method and Processing Technology Using Deep Learning - Google Patents

Abstract

A solution for solving side effects as a blind interview gradually expands in a college entrance examination using deep learning may be provided. During a blind interview process, when applicants must be individually identified in an interview waiting room and assigned a temporary number to enter an interview room, an interview assessor conducts interviews based on the blind screening data, which may impair fairness in evaluating the applicants. Therefore, there is problem that the interview assessor may not confirm whether the applicants are the same as the applicants in the reviewed screening data. An objective of the present invention is to provide a solution capable of assisting a blind interview by solving the problem.

Description

Surrogate Interview Prevention Method and Processing Technology Using Deep Learning}

The present invention relates to prevention of proxy interview using deep learning during blind interview, and an embodiment relates to a blind interview evaluation management technology.

Recently, deep learning algorithms based on artificial intelligence technology have been applied to various industries. Deep learning is defined as a set of machine learning algorithms that attempt to achieve a level of abstraction through a combination of several nonlinear transformations, and can be said to be a field of machine learning that teaches computers to think about human thinking in a large framework. .

This technology solves the problems of blind interviews in universities.

During the blind interview, applicants will be admitted to the interview room with their individual identification and provisional number.

Interviewers will conduct interviews based on blinded data that may impair fairness in evaluating applicants.

Therefore, the interviewer has a problem that it is difficult to confirm whether the applicants and the applicants are reviewing the same person, and try to solve these problems.

In order to secure fairness, the Blind Interview Evaluation is conducted to cover the areas where bias against applicants may be involved in the interview interview.

Since there are many school uniforms on the application form, there is a possibility of inferring the school of origin from the school uniform.

For this reason, various problems are likely to occur.

First, in the interview room, it is possible to confuse the applicant with other applicants because only the number is used to confirm the correspondence between the applicant and the application.

Second, if the applicant deliberately attempts to cheat, such as a proxy interview, the evaluator cannot immediately recognize it.

Third, there is no evidence of applicants who actually took the interview after the interview.

The present invention seeks to solve the above problems.

The present invention studies the deep learning model in the face recognition system so that real face recognition can be utilized to solve the problem of blind interview.

The present invention focused on face detection and feature extraction, which are the core stages of face recognition. By applying the cascade model design and the inception model, we confirmed that the model parameters can be lowered and the computation amount can be reduced, and the objective function setting for applying these models in the image processing field was derived.

In the evaluation system according to the embodiment of the present invention, the QR code is given to the candidate's test ticket, the applicant on the day of the interview with the test ticket and ID card, the supervisor terminal in the waiting room to scan the QR code from the test ticket and perform attendance processing Transmitting identification and photographing information to the administrator terminal through the identification terminal, and transmitting the identification result from the administrator terminal to the terminal of the interviewer.

The interviewer's terminal detects a face in the application reception photograph and blinds the uniform information by performing image processing on an area under the face.

The interviewer's terminal may calculate the similarity between the student's application reception picture and the picture transmitted from the supervisor's terminal. If the similarity is lower than the reference value, the interviewer may inform the interviewer of the similarity result.

This process will be a solution to the problems identified above.

Face recognition system using deep learning model plays an important role in solving various problems of blind interview.

First, in the interview room, it is possible to confuse the applicants with the application data only by the temporary number. However, if a photograph of applicants with uniforms is provided, the applicants and the application data are matched based on the faces of the applicants who participated in the evaluation. You can intuitively notice whether or not.

Second, if a candidate deliberately attempts to cheat, such as a proxy interview, he / she may be notified in advance of suspicious behavior based on the similarity to the applicant's face photographed in the interview room through the photo of the application and the attendance management system. You can notice it right away.

Third, after the interview was completed, there was no data to prove that the person who actually took the interview had been, but the uniforms submitted to the application documents were blinded and the applicants' photos taken by the attendance management system. Similarity analysis results are recorded in the system, so the photos and information of the interviewees are retained even after the screening is completed.

Figure 1: Structure of Face Recognition System
Figure 2: Structure of Facial Detection System
3: Layer structure diagram
4: Inception model structure diagram
5: Feature extraction network structure diagram
6: Triplet Loss conceptual diagram
7: Application example of the school uniform blind
8: Example of Cosine Distance Measurement Between Facial Feature Vectors
9: System Configuration

Advantages and features of the present invention, and methods of achieving them are fully explained by way of examples which will be described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various forms. Unless otherwise defined, all terms used in the present specification may be used in a sense that can be commonly understood by those skilled in the art. In addition, terms that are defined in a commonly used dictionary are not ideally or excessively interpreted unless they are defined.

Hereinafter, technical features of the present invention will be described in detail with reference to the accompanying drawings.

The following is a description of the face recognition system.

Referring to FIG. 1, a general face recognition system is composed of four steps as follows.

1.Face detection in a given image

2. Face Alignment of the detected face size, direction, etc.

3. Feature Extraction from Aligned Faces

4. Search for who the extracted feature is in the database (Feature Matching)

Face detection and feature extraction, which are the main stages of face recognition, have been studied in the existing image processing fields such as Histogram of Oriented Gradient (HOG), Local Binary Pattern (LBP), and Scale Invariant Feature Transform (SIFT). The present invention analyzes papers on face detection and feature extraction using deep learning rather than conventional image processing methods, and detects faces using deep learning learning models used in each paper. Applied to feature extraction.

Referring to FIG. 2, the face detection system proposed in the paper connects three deep convolutional networks in stages, converts an input image into various sizes, and uses the first stage as an input. In each step, the position of the face is estimated in the input image, and non-maximum suppression is integrated for candidate regions that overlap a lot. The processed area is extracted and used as input for the next step, and the area is predicted more precisely as the step proceeds. The layer structure of each step is shown in FIG.

Referring to FIG. 3, all three stages are composed of a convolutional layer, a max-pooling layer, and a fully connected layer. The final output is divided into three types: face classification, region prediction, and landmark coordinate prediction. The later stages deepen the structure and increase the number of filters, enabling more precise and accurate predictions. In addition, it is possible to reduce the amount of computation than passing a complex layer from the beginning for the entire image by passing a simple structure layer and passing a complex structure layer while narrowing an area.

The following is a description of how to learn three objective functions (face classification, region prediction, landmark coordinate prediction).

. y는 정답 라벨에 해당되며, p는 네트워크를 통해 예측된 확률이다.Face classification can be considered as a classification problem of two classes whether the area is a face or not. Therefore, cross-entropy is used as the objective function and the formula is as follows.

. y corresponds to the correct answer label, and p is the probability predicted over the network.

Region prediction predicts a rectangular area with a face, and the rectangular area may be expressed by x, y coordinates and height and width of the left vertex. Therefore, in the case of region prediction, it can be regarded as a regression problem that predicts four values. For the objective function, Euclidean distance is used.

Landmark coordinate prediction is a regression problem that predicts 10 values by predicting x and y of five feature points of the face (left eye, right eye, nose, left mouth vertex, and right mouth vertex). Therefore, as with domain prediction, the objective function is Euclidean distance.

The following is a description of how to extract facial features.

In the case of a typical convolutional neural network, the deeper the network, the better the performance. However, continuous convolution has the disadvantage of lowering the resolution of the image. As the number of channels is multiplied, the parameters of the model are also greatly increased.

The model structure used in the present invention is based on the inception model published by GoogLeNet, and the structure of the inception model is shown in FIG. Referring to FIG. 4, the Inception model is a proposed model to solve this problem and adopts a method of increasing the network width.

The overall network structure using the inception model is shown in FIG.

Referring to FIG. 5, the entire network structure is stacked several times with an inception layer, and the structure that increases the depth and the width is selected. By adopting this structure, it is possible to extract features of various sizes effectively and reduce the computation amount rather than simply stacking the same number of convolution filters. At the end, embedding by connecting Fully Connected, and matching the length of embedding vector through L2 normalization.

The following is a description of how to train the model to extract facial features.

The final output through the network is a 128-dimensional feature vector, and Triplet Loss is introduced for face recognition. As shown in Figure 6, the feature vector of the same person is similar, the feature vector of the other person should be learned away.

로

는 anchor의 특징 벡터,

는 anchor와 동일인의 특징 벡터,

는 타인의 특징 벡터이다.Referring to Figure 6 and expressed as a formula,

in

Is the feature vector for anchor,

Is the feature vector of the same person as anchor,

Is the feature vector of another person.

The present invention analyzes the deep learning model and applies these models to face recognition to solve problems that may occur in current blind interviews.

Referring to FIGS. 7 and 8, these models play an important role in solving blind problems when applied to blind interviews, such as removing uniforms of applicants' photographs and checking similarity between applicants and interviewees in the blind interview system.

9 is a block diagram illustrating in detail an embodiment of proxy interview prevention using the deep learning of the present invention.

9, the present invention includes the examinee terminal 100, the supervisor terminal 200, the management server 300, the electronic authentication server 400, and the interviewer terminal 500. Each terminal can be used a plurality of terminals for each test room and is a terminal capable of wired and wireless communication with each other to transmit and receive real-time information.

Examinee terminal 100 is issued through the management server 300, the electronic test ticket. The examinee terminal 100 requests the electronic authentication process to the electronic authentication server 400 by using the electronic examination table issued from the management server 300.

The electronic authentication server 400 informs the user's electronic authentication processing result requested from the examinee terminal 100.

The examinee terminal 100 temporarily issues the examinee identification code and the electronic authentication encryption token using the result of the electronic authentication process processed from the electronic authentication server 400 and stores the result in the management server 300. The examinee terminal 100 Create a QR code on the screen and print it out.

The supervisor terminal 200 scans the QR code output to the examinee terminal 100 with the camera recognition module to identify the examinee identification code and the electronic authentication encryption token in the QR code to check the authenticity of the management server 300. When the authenticity is confirmed, the supervisor terminal photographs the attendance photograph of the applicant and transmits it to the management server 300.

The management server 300 temporarily stores the electronic authentication encryption token generated from the examinee terminal 100 and receives a confirmation request from the supervisor terminal 200 using the examinee identification code and the electronic authentication encryption token. After checking whether this is valid and notifies the supervisor terminal 200 of the result. At this time, the management server 300 discards the electronic authentication encryption token can no longer be used. In addition, the management server 300 is updated to the attendance processing state if the encryption token received from the supervisor terminal 200 is valid, and if not valid to guide the identification card offline.

The management server 300 calculates the cosine similarity between the attendance picture received from the supervisor terminal 200 and the feature vector extracted from the reception photo submitted by the applicant when the applicant is updated to the attendance processing state. To send).

In this case, the management server 300 transmits the received photographs submitted at the time of application reception as photographs of the school uniform.

100: examinee terminal
200: supervisor terminal
300: management server
400: Electronic authentication server
500: Interviewee Terminal

Claims (3)

The supervisor terminal 200 scans the QR code outputted to the examinee terminal 100 with a camera recognition module to identify the examinee identification code and the electronic authentication encryption token from the QR code to check the authenticity of the management server 300;
If the authenticity is confirmed, the supervisor terminal photographing the attendance photograph of the applicant and transmitting it to the management server 300;
The management server 300 calculates the cosine similarity between the attendance picture received from the supervisor terminal 200 and the feature vector extracted from the applicant's application filed when the applicant receives the application, when the applicant is updated to the attendance processing state. Method for preventing surrogate interview using a deep learning comprising the step of transmitting to.
The method according to claim 1,
Detecting a face in a received photograph submitted at the time of application;
A blind interview prevention method using deep learning comprising the step of blind processing the lower face detected in the application photograph.
The method according to claim 1,
Detecting a face in a photograph taken by the attendance management system;
Face aligning the size, direction, etc. of the detected face;
Feature Extraction from the aligned face;
A method for preventing surrogate interviews using deep learning, comprising measuring cosine similarity between an application photograph and a feature vector extracted from the photograph.

Images