KR102310135B1 - Method, System and Computer-readable Medium for Conducting Recruitment Processes for Applicants Using Machine Learning Model - Google Patents

Abstract

The present invention relates to a method, system and computer-readable medium for conducting a recruitment process for an applicant using a machine learning model and, more specifically, to a method, system and computer-readable medium for deriving authenticity information referring to a degree of reliability for answer content of a user for a personality and aptitude test through the machine learning model and comparing the answer content for the personality and aptitude test and an interview answer video to determine whether the user is the same person.

Description

Method, System and Computer-readable Medium for Conducting Recruitment Processes for Applicants Using Machine Learning Model

The present invention relates to a method, system, and computer-readable medium for performing a recruitment process for applicants using a machine learning model, and more particularly, to the authenticity information corresponding to the degree of trust in the user's personality answer content by machine It relates to a method, system, and computer-readable medium derived through a learning model, and a method, system, and computer-readable medium for determining whether the user is the same person by comparing the personality response content and the interview response image.

In the general type of recruitment process performed by companies or institutions, applicants submit applications (documents) for the company or institution, and based on the application, applicants who have passed the first stage are tested and interviewed for personality or job ability. Applicants who have passed the 1st round will be finally recruited from among the applicants who passed the 1st round.

In particular, in the case of a personality test, a company or institution that conducts the test must prepare a separate physical space such as a school to conduct the personality test, and provide not only the physical space but also human resources for conducting and supervising the personality test. Therefore, there is a problem that a lot of money, time, and human resources are required from the perspective of a company or institution that proceeds with the conventional hiring process.

In addition, in order to prevent fraudulent acts such as taking a proxy test, additional procedures such as receiving the electronic device possessed by the applicant or comparing the applicant's face with the ID card before performing the personality test are required. Procedural time is increased.

On the other hand, in the composition of the questions for the conventional personality test, the same questions are duplicated or similar types of questions are included in order to determine the reliability (authenticity) of the applicant's answers to the personality test questions. . For this reason, the time required for the personality test is usually about 90 minutes, so there is a problem that aggravates the fatigue of both the companies that are hiring and the applicants who are taking the personality test. In addition, since the above-mentioned composition of the questions of the personality test can intentionally prepare the answers to have relatively high authenticity when the applicant prepares for it in advance, it may be difficult to determine the applicants actually needed by the company.

In addition, as the business environment changes rapidly both internally and externally in recent years, companies prefer the frequent hiring method that can flexibly hire talents according to the situation where they need talent rather than regular hiring. However, in the occasional hiring method that is performed whenever necessary, it may act as a burdensome factor for the company that is hiring to perform the above-described offline personality test every time.

Therefore, there is a need to develop a hiring method that can reduce the cost and time required in the hiring process, such as a personality test, and at the same time prevent fraudulent acts such as taking a proxy test.

The present invention relates to a method, system, and computer-readable medium for performing a recruitment process for applicants using a machine learning model, and more particularly, to the authenticity information corresponding to the degree of trust in the user's personality answer content by machine It aims to provide a method, system, and computer-readable medium that is derived through a learning model and determines whether the user is the same person by comparing the personality response content and the interview response image.

In order to solve the above problems, an embodiment of the present invention is a method of performing an online applicant recruitment process performed in a computing system having one or more processors and one or more memories, wherein one or more personality question contents are provided to a user terminal. A personality providing step of providing, and receiving, from the user terminal, answer contents including a personal response image of the user for each of the personality question contents; Authenticity derivation step of deriving authenticity information of the personality answer image from the personality answer image by the machine-learning model; a personality result deriving step of deriving a personality test result for the user based on the authenticity information of one or more of the personality answer images and the user's answer information for the personality question content; an online interview providing step of providing one or more interview question contents to the user terminal, and receiving an interview answer image of the user for each of the interview question contents from the user terminal; and one or more first face images including the user's face extracted from the personality answer image and one or more second face images including the user's face extracted from the interview answer image, based on the user in the personality answer image It provides a method of performing an online applicant recruitment process, including; identity determination step of determining the identity of the user in the interview answer image.

In one embodiment of the present invention, the user's answer information to the personality question content may be derived based on the voice information of the user's personality answer image to the personality question content.

In an embodiment of the present invention, the authenticity information is derived individually based on each personality answer image for one or more of the personality question content, and the personality test result is item evaluation information for each of a plurality of evaluation items and Item authenticity information is included, wherein the item evaluation information is derived based on answer information for one or more personality question contents corresponding to the evaluation item and a preset rule, and the item authenticity or satellite information is at least one corresponding to the evaluation item. It can be derived based on authenticity information about the personality question content.

In an embodiment of the present invention, the authenticity information is derived individually based on each personality answer image for one or more of the personality question content, and the personality test result is the comprehensive authenticity information and each of a plurality of evaluation items. Item evaluation information is included, and the comprehensive authenticity information is derived based on a plurality of the authenticity information, and the item evaluation information includes answer information and preset rules for one or more personality question contents corresponding to each evaluation item. can be derived based on

In an embodiment of the present invention, the online interview providing step includes an automatic evaluation result deriving step, and the automatic evaluation result deriving step is based on a machine-learned model for the interview answer image for one or more comprehensive evaluation items. Automatic evaluation results including information can be derived.

In an embodiment of the present invention, the step of determining the identity comprises: extracting each first facial feature information from a plurality of first face images included in a plurality of frames of the personality response image; extracting each second facial feature information from a plurality of second face images included in a plurality of frames of the interview answer image; and determining the identity of the user in each image by comparing the first facial feature information and the second facial feature information.

In an embodiment of the present invention, the method of performing the online applicant recruitment process further comprises the step of providing identity determination information, wherein the step of providing the identity determination information includes the personality response image received in the personality provision step or the online interview. extracting one or more judgment images including the user's face from the interview answer image received in the providing step; and transmitting the extracted one or more judgment images to an external terminal to be displayed on the terminal of the interviewer.

In an embodiment of the present invention, the providing of personality comprises providing one or more reference question contents to a user terminal, and a reference question providing step of receiving a user's reference answer image for each of the reference question contents from the user terminal ; further performing, the authenticity deriving step, the personality answer image by a machine-learned model; and a reference index derived from the reference answer image or the reference answer image; it is possible to derive authenticity information about the personality answer image. In order to solve the above problems, an embodiment of the present invention provides an online As a computing system for performing a candidate recruitment process, a personality providing unit that provides one or more personality question contents to a user terminal, and receives answer contents including an image of a user's personality answer for each of the personality question contents from the user terminal ; Authenticity derivation unit for deriving authenticity information of the personality answer image from the personality answer image by the machine-learned model; a personality result deriving unit for deriving a personality test result for the user based on the authenticity information of the one or more personality answer images and the user's answer information for the personality question content; an online interview providing unit for providing one or more interview question contents to the user terminal and receiving an interview answer image of the user for each of the interview question contents from the user terminal; and one or more first face images including the user's face extracted from the personality answer image and one or more second face images including the user's face extracted from the interview answer image, based on the user in the personality answer image It provides a computing system comprising a; identity determination unit for determining the identity of the user in the interview answer image.

In order to solve the above problems, an embodiment of the present invention provides a method of performing an online applicant recruitment process performed in a computing system having one or more processors and one or more memories, wherein one or more interview question contents are provided to a user terminal. an online interview providing step of providing, and receiving, from the user terminal, the user's interview answer image for each of the interview question content; A personality providing step of providing one or more personality question contents to the user terminal, and receiving answer contents including an image of the user's personality answer for each of the personality question contents from the user terminal; Authenticity derivation step of deriving authenticity information of the personality answer image from the personality answer image by the machine-learning model; a personality result deriving step of deriving a personality test result for the user based on the authenticity information of one or more of the personality answer images and the user's answer information for the personality question content; and one or more first face images including the user's face extracted from the personality answer image and one or more second face images including the user's face extracted from the interview answer image, based on the user in the personality answer image It provides a method of performing an online applicant recruitment process, including; identity determination step of determining the identity of the user in the interview answer image.

In order to solve the above problems, an embodiment of the present invention provides a computer-readable recording medium in which a program for performing the method for performing the above-described online applicant recruitment process is recorded.

According to an embodiment of the present invention, since the personality test is performed in an online manner, it is possible to exhibit the effect of saving the cost required for the recruitment on the side of the company that is hiring.

According to an embodiment of the present invention, since authenticity information is derived from a personality answer image for a personality test performed in an online manner using a machine-learning model, the test time compared to the existing personality test consisting of overlapping similar questions can have the effect of significantly reducing

According to an embodiment of the present invention, since the personality test result is derived based on the authenticity information and the answer information for each question, it is possible to exert the effect of deriving more accurate results.

According to an embodiment of the present invention, since the interview is performed by receiving the interview answer image of the user (applicant) with respect to the interview question content provided in an online manner, it is possible to exhibit the effect of saving the cost of hiring have.

According to an embodiment of the present invention, since the online interview providing step derives the evaluation result for the interview by using a machine-learning model for the interview answer image, it is possible to provide more objective results compared to the evaluation result for the conventional interview. possible effect can be exerted.

According to an embodiment of the present invention, since the identity determination step determines the identity of the user in the personality answer image and the user in the interview answer image, it is possible to easily find fraudulent acts such as a proxy test in each hiring step. can have an effect.

According to an embodiment of the present invention, since the identity determination information providing unit provides the interviewer with a judgment image of the user who actually participated in the offline recruitment screening, the user who performed the online recruitment process by the interviewer and the user who actually participated offline It can exhibit the effect of easily grasping whether it is the same.

1 schematically shows the form of an overall system for performing a recruitment process for an online applicant according to an embodiment of the present invention.
2 schematically illustrates an internal configuration of a computing system according to an embodiment of the present invention.
3 schematically shows steps for performing a hiring process according to an embodiment of the present invention.
FIG. 4 schematically shows personality question content and answer content according to personality question content according to an embodiment of the present invention.
5 schematically shows the operation of the authenticity or authenticity derivation unit according to an embodiment of the present invention.
6 schematically shows the configuration of personality question content corresponding to evaluation items according to an embodiment of the present invention.
7 schematically shows steps for deriving a personality test result according to an embodiment of the present invention.
8 schematically shows steps of another configuration for deriving a personality test result according to an embodiment of the present invention.
9 schematically shows an operation of an automatic evaluation result deriving unit according to an embodiment of the present invention.
10 schematically illustrates an example of a screen for personality question content displayed on a user terminal according to an operation of an online interview providing unit according to an embodiment of the present invention.
11 schematically illustrates an example of a screen for soft skill information displayed on an interviewer terminal according to an operation of an online interview providing unit according to an embodiment of the present invention.
12 schematically illustrates steps for determining the identity of a user according to an embodiment of the present invention.
13 schematically shows the operation of the identity determining unit according to an embodiment of the present invention.
14 schematically shows the internal configuration of each evaluation model according to an embodiment of the present invention.
15 schematically shows an internal configuration of a feature extraction model according to an embodiment of the present invention.
16 schematically shows the internal configuration of a feature inference model according to an embodiment of the present invention.
17 schematically shows detailed steps for performing the step of providing identity determination information according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that other components may be further included rather than excluding other components unless otherwise stated.

Also, terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or may be configured to reproduce one or more processors. Thus, as an example, '~' denotes components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

The "user terminal" and "interviewer terminal" mentioned below may be implemented as a computer or portable terminal that can access a server or other terminal through a network. Here, the computer includes, for example, a laptop, a desktop, and a laptop equipped with a web browser (WEB Browser), and the portable terminal is, for example, a wireless communication device that ensures portability and mobility. , PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code) Division Multiple Access)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro) terminals, and the like may include all types of handheld-based wireless communication devices. In addition, "network" refers to a wired network such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), or a mobile radio communication network or satellite. It may be implemented as any kind of wireless network, such as a communication network.

On the other hand, the "personality test" described in the present invention is a personality test that can evaluate whether it conforms to the company's talent image, such as a job aptitude test, and an aptitude test that can evaluate the competency for basic tasks in the general hiring process. It can include all, mean only personality tests, or only aptitude tests.

In addition, "authenticity" described in the present invention may be understood as a concept including the consistency or reliability of the user's response to the personality test.

1 schematically shows the form of an overall system for performing a recruitment process for an online applicant according to an embodiment of the present invention.

The computing system 1000 shown in FIG. 1 receives the answer content and the interview answer image for the personality test from the user terminal 2000 in an automated manner, and includes authenticity information about the answer content based on the received answer content derives a personality test result, and determines the identity of the user included in the corresponding image based on the answer content and the interview answer image. In addition, the derived personality test result and the identity determination result for the user may be provided to the user terminal 2000 or the interviewer terminal 3000 . In a preferred embodiment of the present invention, the computing system 1000 may perform an online-based recruitment process of one or more companies, and more specifically, the computing system 1000 performs an online-based interview and personality test for each company. can be done Therefore, the computing system 1000 is based on the interview answer image for the online interview performed by each company and the answer content for the personality test, the analysis result of the interview answer image, the personality test result, and the identity of the user for each image The determination result may be derived, and the derived results may be provided to the interviewer terminal 3000 of the interviewer including the hiring manager of the corresponding company.

Also, the computing system 1000 may include one or more computing devices and/or one or more servers, and may communicate with the user terminal 2000 and the interviewer terminal 3000 . Accordingly, the computing system 1000 may be implemented by one or more physically separated computing devices and/or one or more servers, and the one or more computing devices and/or one or more servers included in the computing system 1000 are mutually exclusive. communication can be performed. In addition, the computing system 1000 may receive from the interviewer terminal 3000 the response content of the applicant who received an excellent test result or a low test result with respect to the personality test result of the company, and the computing system 1000 can learn and improve the evaluation model corresponding to the corresponding company based on the corresponding answer content.

On the other hand, the user corresponding to the subject of the user terminal 2000 corresponds to a job seeker who applied for employment of a specific company by performing communication with the computing system 1000, and after filling out an application form for the specific company, the computing system ( 1000) through the online interview and personality test for the company can be included.

The subject using the interviewer terminal 3000 may include an interviewer who performs an offline interview in the hiring process of the corresponding company and a person in charge of hiring of the corresponding company. As described above, the interviewer terminal 3000 may receive the personality test result for the user and the automatic evaluation result for the online interview from the computing system 1000 and display it. In addition, the interviewer terminal 3000 is provided with a personality answer image or interview answer image itself for the user from the computing system 1000, or one or more judgment images extracted from the personality answer image or interview answer image for the user. It may be provided and displayed, and the interviewer may determine whether the user who participated in the offline interview and the user included in the displayed image or image are the same, and may input the information into the interviewer terminal 3000 .

The user terminal 2000 or the interviewer terminal 3000 accesses a web page provided by the computing system 1000 through a web browser on the computing system 1000 or through a separate dedicated application (program). It can communicate with (1000). The user terminal 2000 or the interviewer terminal 3000 may use the service provided by the computing system 1000 by performing communication with the computing system 1000 with each account of the subject using the terminal. On the other hand, the user terminal 2000 or the interviewer terminal 3000 communicates with the computing system 1000 through different types of accounts, for example, a user (applicant) type account, an interviewer (recruiter) type account. can be done

In addition, the computing system 1000 performs the actual hiring process performed by each company to derive a personality test result, and not only determines the identity of a user, but also provides a service for job seekers to practice an interview or personality test Alternatively, information on suitable companies to apply for based on the interview or personality test practiced by job seekers may be provided, and the company may additionally provide information on job seekers suitable for the company.

2 schematically shows an internal configuration of a computing system 1000 according to an embodiment of the present invention.

The computing system 1000 in the present invention may be used as a system for performing online-based recruitment processes. Specifically, a user provides answer content and an interview answer image to the computing system 1000 through the user terminal 2000, and the computing system 1000 provides a personality test result and an interview answer image based on the answer content and interview answer image A judgment result for the user's identity is derived. The computing system 1000 may provide the derived personality test result and the user's identity determination result to the interviewer terminal 3000 .

In addition, the computing system 1000 passes through learning data about the company from the interviewer terminal 3000, specifically, the interview response image of the applicant who passed or failed through the online interview for the company, and an online-based personality test. Alternatively, the response content including the personality response image of the unsuccessful applicant may be provided, and based on this, each evaluation model may be learned and improved.

In the computing system 1000 shown in FIG. 2 , the personality providing unit 1100 transmits the personality question content displayed or reproduced in the user terminal 2000 to the user terminal 2000 in order to receive the answer content including the personality answer image. and receives, from the user terminal 2000, answer content including a personality answer image performed on the personality question content.

On the other hand, the computing system 1000 stores one or more personality question contents configured for each company, and when a user performs a personality test on a specific company, one or more personality question contents corresponding to the company is stored in the user terminal. (2000) can be displayed. Meanwhile, the personality question content includes all types of content that can be reproduced or displayed in the user terminal 2000, and may preferably be implemented as one or more of text, video, and audio.

The authenticity derivation unit 1200 derives authenticity information about the answer content of the personality test through the authenticity evaluation model using machine learning. Preferably, the authenticity evaluation model may include one or more detailed authenticity evaluation models based on machine learning, and specifically, one or more detailed authenticity evaluation models based on deep learning.

Preferably, the authenticity evaluation model includes one or more deep learning-based trained artificial neural network modules. In addition, the authenticity evaluation model may further include a detailed authenticity evaluation model based on a preset rule as well as the learned artificial neural network module. That is, in an embodiment of the present invention, the authenticity evaluation model may correspond to a configuration using a learned artificial neural network module and a detailed authenticity evaluation model based on rules in a complex manner.

That is, the computing system 1000 automatically derives the authenticity information by using the authenticity evaluation model after receiving the answer content. The authenticity information is derived individually for each answer image included in the answer content, and specifically, based on the user's voice information such as the user's voice pitch, tremor, and intonation in the answer image, and frame information such as the user's expression, movement, etc. Thus, authenticity information can be derived for each answer image. In addition, when a question is delivered by voice, the time taken from the time the question is presented to the answer can be measured, and authenticity information can be derived by further considering the measured time information.

Therefore, in the conventional paper-and-pencil personality test, the user has to repeatedly answer the same or similar questions in order to derive authenticity information, so the test time is long, but in the present invention, based on the user's answer image Therefore, since authenticity information is derived, it is possible to exert the effect of significantly reducing the execution time of the personality test.

Personality result deriving unit 1300 is based on the user's answer content to the personality test, specifically, the answer information for one or more answer images included in the answer content, and the authenticity information for each answer image, the personality test of the user To derive the results. As for the personality test result, the test result is derived individually for each answer image, and the personality test result is derived by synthesizing each of the derived test results, or the personality test result is obtained by synthesizing each answer image can be derived.

Preferably, the personality result derivation unit 1300 derives test results for each evaluation item with respect to the answer image corresponding to one or more evaluation items for the personality test, and based on the derived test results and authenticity information for each of the at least one evaluation item Thus, the personality test result can be derived.

The online interview providing unit 1400 provides interview question content displayed or reproduced in the user terminal 2000 to the user terminal 2000 in order to receive the interview answer image from the user terminal 2000, and from the user terminal 2000 Receives an interview response image performed on the interview question content.

On the other hand, the computing system 1000 stores one or more interview question contents configured for each company, and when a user conducts an online interview for a specific company, one or more interview question contents corresponding to the corresponding company are stored in the user terminal. (2000) can be displayed. On the other hand, the interview question content includes all kinds of content that can be reproduced or displayed on the user terminal 2000, and may preferably be implemented as one or more of text, video, and audio.

In addition, the online interview providing unit 1400 may additionally derive an automatic evaluation result for the interview answer image received from the user terminal 2000 . Specifically, the online interview providing unit 1400 derives an automatic evaluation result for the interview answer image through the interview image evaluation model using machine learning. Preferably, the interview image evaluation model may include one or more detailed interview image evaluation models based on machine learning, and more preferably, one or more detailed interview image evaluation models based on deep learning.

Preferably, the interview image evaluation model includes one or more deep learning-based trained artificial neural network modules. The interview image evaluation model may further include a detailed interview image evaluation model based on a preset rule as well as the learned artificial neural network module. That is, in an embodiment of the present invention, the interview image evaluation model may correspond to a configuration using a detailed interview image evaluation model based on the learned artificial neural network module and rules in a complex manner.

That is, the computing system 1000 automatically derives an automatic evaluation result by using the interview image evaluation model after receiving the interview answer image from the user terminal 2000 . The automatic evaluation result is information on communication and attitude, including at least one of the employment recommendation index, favorable sensitivity, concentration, trust, liveliness, composure, communication ability, eye contact, kindness, pleasantness, and use of habitual words, etc. including soft skill information; personality trait information including at least one of openness, conscientiousness, extraversion, affinity, and neuroticism; And it may include one or more of the gaze change information of the user.

The identity determination unit 1500 determines the identity of the user based on the answer content including the personality answer image received from the personality providing unit 1100 and the interview answer image received from the online interview providing unit 1400 . Specifically, it is possible to determine whether the user of each image is the same person based on the similarity between each image by comparing one or more images included in the answer content and one or more images included in the interview answer image.

Preferably, the identity determination unit 1500 may determine the identity of the user through the identity evaluation model using machine learning. Specifically, the identity evaluation model may include one or more detailed identity judgment models based on machine learning, and more specifically, includes one or more deep learning-based trained artificial neural network modules. The identity evaluation model may further include a detailed identity evaluation model based on a preset rule as well as the learned artificial neural network module. That is, in an embodiment of the present invention, the identity evaluation model may correspond to a configuration using a learned artificial neural network module and a detailed identity evaluation model based on rules in a complex manner.

Meanwhile, in another embodiment of the present invention, in order to derive the similarity between the respective images, the identity evaluation model may be implemented in a metric learning method.

The model learning unit 1600 performs steps for learning each of the evaluation models used in the computing system 1000 such as the authenticity evaluation model, the interview image evaluation model, the identity evaluation model, and the like. The model learning unit 1600 is the answer content including the personality response image of the successful and unsuccessful obtained through the online-based personality test for the recruitment of the company, the interview answer image of the successful and unsuccessful obtained through the online interview Thus, each of the evaluation models can be trained.

Preferably, the model learning unit 1600 is a authenticity evaluation model learning unit (not shown) that learns the authenticity evaluation model by answering content including the personality response images of successful and unsuccessful candidates obtained through the online-based personality test. ); an interview image evaluation model learning unit (not shown) for learning the interview image evaluation model based on the interview response images of successful and unsuccessful candidates obtained through the online interview; and an identity evaluation model learning unit (not shown) for learning the identity evaluation model based on the answer content and the interview answer image including the personality answer image of the successful or unsuccessful person.

On the other hand, the identity evaluation model learning unit (not shown) in order to learn the similarity between two or more different images or two or more images, as well as an image of a successful or unsuccessful person of the company, as well as a plurality of images or images of various types. Thus, the identity evaluation model can be trained.

The identity determination information providing unit 1700 transmits the answer content of the user who has performed the online-based personality test of the company or the interview response image for the online interview to an external terminal so that it can be displayed on the interviewer terminal 3000 of the company do. Specifically, the identity determination information providing unit 1700 may provide the answer content or the interview answer image to the interviewer terminal 3000 directly or indirectly, and directly or directly on the interviewer terminal 3000. The external terminal may correspond to the interviewer terminal 3000, and when it is indirectly displayed on the interviewer terminal 3000, the external terminal corresponds to a separate server, another terminal, or other computing device and is primarily the same A corresponding image may be received from the determination information providing unit 1700 , and the corresponding image may be secondarily transmitted to the interviewer terminal 3000 .

Preferably, the identity determination information providing unit 1700 may provide the answer content or the interview answer image itself to the interviewer terminal 3000, but the identity determination information providing unit 1700 is the answer content or interview answer image. may extract one or more judgment images including the user's face, and provide the one or more judgment images to the interviewer terminal 3000 .

On the other hand, the interviewer terminal 3000 displays an interface for displaying one or more of the determination images or corresponding images, and the interface includes an element for receiving the result of determining the identity of the interviewer. Therefore, when a user participates in a recruitment screening performed in an offline manner in the hiring process, such as a face-to-face interview, the interviewer identifies whether the user who actually attended is the same as the user included in the image displayed on the interviewer terminal 3000, and the element The determination result may be input through , and the interviewer terminal 3000 may directly or indirectly provide input information including the determination result to the computing system 1000 .

In the DB 1800 included in the computing system 1000, the personality question content provided for each company, the answer content for the personality test received from the user terminal 2000, and the authenticity derived by performing the authenticity derivation unit 1200 Information, personality test results derived by performing the personality result deriving unit 1300, interview question content provided for each company, the interview answer image received from the user terminal 2000 through the online interview providing unit 1400, identity determination Learning such as the result of the determination of identity derived by performing in the unit 1500, the determination image extracted from the identity determination information providing unit 1700, the authenticity evaluation model used in the computing system 1000, the interview image evaluation model, the identity evaluation model, etc. Model, user information, and company information may be stored.

On the other hand, user information and company information may be input from the user terminal 2000 and the interviewer terminal 3000, respectively, and transmitted to the computing system 1000 and stored. It may be stored in the DB 1800 through the example of inputting user information such as .

The computing system 1000 illustrated in FIG. 2 may further include other components in addition to the illustrated components, but for ease of explanation, the method of performing the recruitment process for applicants according to an embodiment of the present invention is performed. Only the components to be used are shown, and therefore, the computing system 1000 is not limited to only the components shown in FIG. 2 .

Meanwhile, the computing system 1000 illustrated in FIG. 2 may include a plurality of computing devices. For example, when the computing system 1000 includes a first computing device and a second computing device, the first computing device and the second computing device communicate with each other, and the first computing device has a personality. It may include a configuration for providing a test and deriving a personality test result, and the second computing device may include a configuration for providing an online interview and deriving an online interview result. Meanwhile, the configuration for determining the identity or providing the judgment image to the interviewer terminal 3000 and the configuration for learning the model may be included in the first computing device or the second computing device, or may be included in a separate computing device.

3 schematically shows steps for performing a hiring process according to an embodiment of the present invention.

As shown in FIG. 3 , as a method of performing an online applicant recruitment process performed in a computing system having one or more processors and one or more memories, one or more personality question contents are provided to a user terminal, and each of the personality question contents Personality providing step (S10) of receiving the answer content including the user's personality response image for the user terminal from the user terminal; Authenticity derivation step (S11) of deriving authenticity information of the personality answer image from the personality answer image by the machine-learned model; a personality result deriving step (S12) of deriving a personality test result for a corresponding user based on the authenticity information of one or more of the personality answer images and the user's answer information for the personality question content; an online interview providing step (S13) of providing one or more interview question contents to the user terminal, and receiving an interview answer image of the user for each of the interview question contents from the user terminal; and one or more first face images including the user's face extracted from the personality answer image and one or more second face images including the user's face extracted from the interview answer image, based on the user in the personality answer image and an identity determination step (S14) of determining the identity of the user in the interview answer image.

Specifically, according to the execution input of the user or the interviewer (recruiter of the company) on each user terminal 2000 or the interviewer terminal 3000, the personality providing unit 1100 provides one or more personality question contents for the company. It is provided to the user terminal 2000, and the user performs a personality test according to one or more personality question contents displayed on the user terminal 2000 to receive answer contents generated by performing a personality test (S10). The reply content received in the corresponding step (S10) may be stored together with separate identification information in the DB 1800 in the computing system 1000 .

The authenticity or falsehood derivation unit 1200 performs a step (S11) of deriving authenticity information based on each answer image and the authenticity evaluation model included in the answer content received in the step (S10). Specifically, the authenticity or falsehood derivation unit 1200 may input each answer image to the authenticity evaluation model to derive authenticity information for each answer image.

Next, the personality result deriving unit 1300 performs a step (S13) of deriving a personality test result based on the derived authenticity information for each answer image and each answer information derived from each answer image. More specifically, one or more evaluation items exist for the personality test for each company, and one or more personality question contents correspond to one or more evaluation items. Accordingly, the personality result derivation unit 1300 may derive detailed personality test results for each evaluation item based on the answer information for the personality question content corresponding to each evaluation item.

On the other hand, the online interview providing unit 1400 is the online interview providing unit 1400 according to the execution input of the user or the interviewer (recruiter of the company) on each user terminal 2000 or the interviewer terminal 3000. Step of providing one or more interview question contents corresponding to the company to the user terminal 2000, and receiving the interview answer image generated by the user performing an online interview according to the one or more interview question contents displayed on the user terminal 2000 (S13) is performed. The interview answer image received in the corresponding step (S13) may be stored in the DB 1800 of the computing system 1000 together with separate identification information.

In another embodiment of the present invention, the step (S13) may be preceded before the steps (S10, S11, and S13) for providing the online-based personality test performed by the computing system 1000 are performed. That is, the step (S13) performed by the online interview providing unit 1400 may be performed before the step (S10) performed by the personality providing unit 1100.

Finally, the identity determination unit 1500 performs a step (S14) of determining the identity of each user included in the received personal response image and interview answer image. Specifically, in step S14, one or more images are extracted from the personality response image, one or more images are extracted from the interview response image, and the identity can be determined through comparison between images extracted from each image.

Preferably, in the step S14, feature information of each image extracted from each image is derived using the identity evaluation model, and the identity between the images can be determined based on the derived feature information. On the other hand, the identity evaluation model may be implemented in a metric learning method, and in this case, similarity information corresponding to information on the degree of similarity between the user's personality answer image and the interview answer image may be included.

In another embodiment of the present invention, the personality providing step (S10) is to provide one or more reference question contents to the user terminal, and to receive the reference answer image of the user for each of the reference question contents from the user terminal It may include a step of providing a standard question.

Specifically, the standard question content provided in the standard questioning stage includes 'What is your gender?', 'What is your name?', and 'Where is your hometown? ', etc., it is composed of questions that do not require a long time for the user performing the personality test to answer, that is, cannot give a wrong answer.

Therefore, the user's reference answer image to the reference question content can be utilized in deriving the authenticity information in the above-described authenticity deriving step (S11). More specifically, the user's facial expression, voice, or time until answering in the reference answer image may be a reference index, and authenticity information for the personality answer image may be derived based on the reference index.

As such, according to an embodiment of the present invention, it is possible to exert the effect of deriving more accurate authenticity information for each user who performs a personality test through the reference question content provided through the reference question providing step.

FIG. 4 schematically shows personality question content and answer content according to personality question content according to an embodiment of the present invention.

Figure 4 (A) corresponds to a view showing an embodiment of the answer form to the personality question content and personality question content, Figure 4 (B) is displayed on the user terminal 2000 to generate the answer content It corresponds to a drawing showing an embodiment of a personality interface for

Specifically, as shown in FIG. 4(A), one or more personality question content may be configured according to the hiring criteria such as the talent image of the corresponding company, and the personality question content is not only text, but also voice or video and It may also be configured in various forms.

On the other hand, the form of the answer to the personality question content is a method of selecting a specific number from among a plurality of numbers corresponding to 'not at all', 'normal', and 'very much' as shown in FIG. 4A In the personality test, such as answering in the form of alternatives such as 'yes' and 'no' to the personality question content, or no options are provided, and freely answering within a preset time. All commonly used answer forms can be included.

On the other hand, one or more personality question content as described above may be displayed on the personality interface in the user terminal 2000 of the corresponding user, as shown in FIG. 4B .

The personality interface includes an interview image layer L1 in which the user is photographed in real time through a photographing module provided in the user terminal 2000 . In the interview image layer (L1), a preset number (for example, 5) is displayed so that the user can recognize that the personality test is started or that each personality question content is taken when executed, and then sequentially numbers is decremented and can be counted down.

On the other hand, the personality interface further includes a question content layer (L2), so that when shooting starts, personality question content can be sequentially displayed on the question content layer (L2). On the other hand, in the question content layer (L2), as shown in FIG. 4 (A), the form of an answer to the personality question content may be displayed, and thus the user can answer the question displayed on the content layer (L2). Depending on the form of , shooting can be done by saying the answer information in the form of a voice. For example, when the user reads question 2 displayed in the question content layer L2 of FIG. 4B and determines that it corresponds to number 2 with reference to the answer form, question 2 is said to be voiced 2 Response information may be included in the response image.

In another embodiment of the present invention, the question content layer (L2) may be omitted, and if omitted, the personality question content may be provided to the user as voice information. Alternatively, even when the question content layer L2 is included, the personality question content may be simultaneously provided to the user as text information and voice information.

Accordingly, the user's answer information to the personality question content may be derived based on the voice information of the user's personality answer image to the personality question content.

More specifically, the personality result deriving unit 1300 derives the answer information based on the voice information in the user's personality answer image in order to derive the personality test result, and based on the answer information and the authenticity information about the answer image In this way, the results of the personality test can be derived. On the other hand, in order to derive the answer information based on the voice information in the personality response image, the personality result deriving unit 1300 may use a commonly used voice recognition related technology.

In another embodiment of the present invention, the answer information is based on the input answer information additionally input through the input device (keyboard, mouse or touch screen) provided in the user terminal 2000 as well as the user's voice information form. may be derived. Accordingly, the computing system 1000 can easily derive answer information from the answer image.

In this way, the answer content may include one or more personal answer image for each personality question content, input answer information, corresponding user information, and the like.

On the other hand, the personality answer image is generated in the form of a plurality of images divided according to each personality question content, or each personality answer image is generated in the form of one image, and is divided by each personality answer image may be created as

In addition, the personality interface further includes an additional information layer (L3), and the additional information layer (L3) displays an answer time limit and a required time for the corresponding answer, and is converted to the next question according to the user's selection input Elements may be displayed. On the other hand, when the answer time limit is exceeded, the personality interface may automatically switch to the next question.

5 schematically illustrates the operation of the authenticity or falsehood derivation unit 1200 according to an embodiment of the present invention.

As shown in FIG. 5 , it corresponds to a diagram illustrating the deriving of authenticity information for a personality answer image by using the machine-learned authenticity evaluation model in the authenticity derivation unit 1200 .

The authenticity evaluation model may include various detailed authenticity evaluation models for performing evaluation on the personality response image. The detailed authenticity evaluation model corresponds to a detailed authenticity evaluation model capable of deriving authenticity information based on a deep learning-based learned evaluation model, or according to a preset routine or algorithm rather than learning, characteristic information about the personality response image. It may correspond to a detailed authenticity evaluation model that derives (eg, gaze position, facial expression) and derives authenticity information with respect to the derived feature information.

In an embodiment of the present invention, the authenticity derivation unit 1200 basically receives an image including a plurality of continuous image (video) information and voice information, and evaluates the authenticity learned through machine learning technology such as deep learning. Authenticity information is derived through the model. In addition, the authenticity deriving unit 1200 may further analyze the personality response image based on a preset rule rather than machine learning, and may derive authenticity information. In addition, the detailed authenticity evaluation model extracts image and audio information from a personality response image including a plurality of consecutive images (videos) and voices, and inputs them into each separately provided detailed authenticity evaluation model to derive authenticity information You may.

On the other hand, the authenticity deriving unit 1200 derives authenticity information based on the feature information derived from the authenticity evaluation model, including the authenticity evaluation model, or the authenticity deriving unit 1200 is provided separately for authenticity evaluation. By calling the model, authenticity information can be derived based on the feature information derived from the authenticity evaluation model. In addition, the derived authenticity information may be provided to the interviewer terminal 3000 of the corresponding company.

6 schematically shows the configuration of personality question content corresponding to evaluation items according to an embodiment of the present invention.

As shown in FIG. 6 , one or more evaluation items exist with respect to the personality test performed by each company, and the content and number of evaluation items may be different depending on the hiring criteria or talent image of each company. . The items of 'passion', 'innovation', 'competition', and 'sincerity' shown in FIG. 6 correspond to examples of evaluation items, and item evaluation information is calculated for each evaluation item, and the item evaluation information is personality. may be included in the test results.

Meanwhile, the personality test questions of FIG. 6 may correspond to the aforementioned personality question contents. In addition, each personality test item may correspond to one or more evaluation items. 6 as an example, 'question 1' may correspond to evaluation items of 'passion' and 'competition', respectively, and 'question 2' may correspond to evaluation items of 'innovation'.

Accordingly, item evaluation information for the evaluation item corresponding to the personality question content may be calculated according to the user's answer to the personality question content, that is, answer information. Taking Fig. 6 as an example, the personality test questions corresponding to the evaluation items for 'sincerity' are 'question 4' and 'question 5', and based on the answer information that the user answers to 'question 4' and 'question 5' Thus, item evaluation information of the evaluation item for 'sincerity' can be calculated.

In addition, when specific personality question content corresponds to two or more evaluation items, the user's answer information for the corresponding personality question content may be reflected differently for each corresponding evaluation item. For example, for evaluation item A, the corresponding answer information may be reflected with a weight of 2 points, and for evaluation item B, the corresponding answer information may be reflected with a weight of 5 points. In this way, a score or weight according to the content of the answer information for the personality question content corresponding to each evaluation item may be included in a preset rule, and the preset rule may be stored in the computing system 1000 for each company. .

On the other hand, item authenticity information can be derived for each evaluation item, and the item authenticity information can be derived based on one or more items of authenticity information derived through the user's answer image to one or more personality question contents corresponding to each evaluation item. have.

7 schematically shows steps for deriving a personality test result according to an embodiment of the present invention.

As shown in FIG. 7 , the authenticity information is individually derived based on each personality answer image for one or more of the personality question contents, and the personality test result includes item evaluation information for each of a plurality of evaluation items and Item authenticity information is included, wherein the item evaluation information is derived based on answer information for one or more personality question contents corresponding to the evaluation item and a preset rule, and the item authenticity or satellite information is at least one corresponding to the evaluation item. It is derived based on the authenticity information about the personality question content.

Specifically, the personality result deriving unit 1300 performs a step (S20) of extracting answer information for each of the user's one or more personality response images. In the step (S20), each answer information is derived based on the voice information included in each personality answer image.

In step S21, the personality result deriving unit 1300 derives item evaluation information based on the answer information corresponding to each evaluation item and preset rules for each of one or more evaluation items. The preset rule may include rules for a reflection ratio such as a score or weight in the corresponding evaluation item according to the content of the answer information.

In step S22, the personality result deriving unit 1300 derives item authenticity information based on the authenticity information for each personality answer image derived from the authenticity deriving unit 1200 corresponding to each evaluation item for each of one or more evaluation items. More specifically, the item authenticity information may be derived by a preset derivation method, such as a sum or average of authenticity information corresponding to each evaluation item. As such, since the item authenticity information includes the level of authenticity for each evaluation item of the user, the personality test result including the item authenticity information to be described later provides detailed information on whether the user makes a reliable response for each evaluation item. possible effect can be exerted.

In step S23, the personality result deriving unit 1300 derives a personality test result including at least one item evaluation information and at least one item authenticity information for each evaluation item derived in steps S21 and S22. The personality test result may include a comprehensive personality test result of the user according to one or more items of evaluation information and one or more items of authenticity information derived for each evaluation item. In addition, the personality test result may include detailed personality test results for each evaluation item.

Therefore, the personality test result not only includes the results according to the company's hiring standards based on the answer information, but also includes the reliability of the answer information of the user who performed the test, so that more suitable applicants can be recruited in the hiring process. can have an effect.

8 schematically shows steps of another configuration for deriving a personality test result according to an embodiment of the present invention.

As shown in FIG. 8 , the authenticity information is individually derived based on each personality answer image for one or more of the personality question contents, and the personality test result is the comprehensive authenticity information and each of a plurality of evaluation items. Item evaluation information is included, and the comprehensive authenticity information is derived based on a plurality of the authenticity information, and the item evaluation information includes answer information and preset rules for one or more personality question contents corresponding to each evaluation item. is derived based on

Specifically, since steps S30 and S31 shown in FIG. 8 are substantially the same as steps S20 and S21 of FIG. 7 described above, respectively, a detailed description thereof will be omitted.

On the other hand, the personality result deriving unit 1300 performs a step (S32) of deriving synthetic authenticity information based on each authenticity information for each personality answer image. Unlike the item authenticity information derived for each evaluation item, the comprehensive authenticity information corresponds to information derived by considering each authenticity information as a whole without considering the evaluation items. In addition, the comprehensive authenticity/falseness information may be derived by adding up each authenticity or falsehood information, or may be derived by various methods such as adding weights according to the contents of each authenticity/falseness information and adding them together.

Next, the personality result deriving unit 1300 performs a step (S33) of deriving a personality test result including each item evaluation information and comprehensive authenticity information. The personality test result derived by performing the step S33 may consider the degree of confidence in the overall user's response to the personality test.

Meanwhile, in another embodiment of the present invention, authenticity information may be derived by additionally considering the reference answer image received through the above-described reference question providing step.

Specifically, the personality result deriving unit 1300 derives a reference index including at least one of the user's facial expression, voice tone, and time until answering in the above-described reference answer image for each user, and the reference index and personality answer Authenticity information can be derived based on the image. In an embodiment of the present invention, the reference index extracted from the reference answer image for the reference question content; and input the feature information extracted from the personality answer image or the personality answer image for the personality question content into a machine-learning model to minimize the influence of individual characteristics such as facial expression, voice, and response time, while more accurately providing authenticity information can be derived

Alternatively, in another embodiment of the present invention, the authenticity information derived through a machine-learning model based on the personality response image may be adjusted based on the reference index to derive the final authenticity information.

9 schematically shows the operation of the automatic evaluation result deriving unit 1410 according to an embodiment of the present invention.

9, the online interview providing unit 1400 includes an automatic evaluation result deriving unit 1410, and the automatic evaluation result deriving step performed by the automatic evaluation result deriving unit 1410 includes: An automatic evaluation result including information on one or more comprehensive evaluation items is derived by a machine-learning model for an interview response image.

Specifically, the interview image evaluation model may include various detailed interview image evaluation models for evaluating the interview response image. The interview image evaluation model corresponds to a detailed interview image evaluation model that can perform evaluation based on a deep learning-based learned evaluation model, or characteristic information about the interview response image according to a preset routine or algorithm rather than learning. It may correspond to a detailed interview image evaluation model that derives (eg, the position of the gaze) and evaluates the derived feature information.

In an embodiment of the present invention, the automatic evaluation result deriving unit 1410 basically receives an interview answer image including a plurality of continuous image (video) information and voice information, and through machine learning technology such as deep learning An automatic evaluation result is derived through the learned interview image evaluation model. In addition, the automatic evaluation result deriving unit 1410 may analyze the interview answer image based on a preset rule rather than machine learning, and may derive specific evaluation values. The detailed interview image evaluation model extracts video and audio information from an interview answer image including a plurality of consecutive images (videos) and audio and inputs it to each detailed interview image evaluation model to derive a result value, or It is also possible to derive the result value by synthesizing the voice information and inputting it into the detailed interview image evaluation model.

On the other hand, the automatic evaluation result deriving unit 1410 derives an automatic evaluation result based on the feature information derived from the interview image evaluation model, including the interview image evaluation model, or the automatic evaluation result deriving unit 1410 ) may call a separately provided interview image evaluation model to derive an automatic evaluation result based on the feature information derived from the interview image evaluation model.

Preferably, the detailed interview image evaluation model may include a module for extracting a plurality of positional information of the user's gaze from a plurality of frames of the interview answer image, and through this, the automatic evaluation result deriving unit 1410 In this way, the interviewer terminal 3000 may provide gaze change information including information on the position status of the user's gaze and information on the motion status of the gaze.

Preferably, the detailed interview image evaluation model may include a module for extracting the movement of the user's head from a plurality of frames of the interview answer image, and through this, the interviewer terminal At 3000 , information on the movement of the user's head may be provided.

Preferably, the detailed interview image evaluation model may include a module for extracting sound characteristic values of speech, such as pitch and volume of speech in a plurality of frames, and through this, the automatic evaluation result deriving unit 1410 ), the interviewer terminal 3000 may provide information about the user's voice.

Preferably, the detailed interview image evaluation model may include a module for extracting the intensity of the user's emotional elements for each section of the image, and through this, the user in the interviewer terminal 3000 by the automatic evaluation result deriving unit 1410 can provide information about their emotions. In this case, the detailed interview image evaluation model by the artificial neural network learned by the deep learning may be used.

In this way, the automatic evaluation result deriving unit 1410 uses the interview image evaluation model to determine the user's gaze change information, information about head movement, information about voice, and information about emotions, as described above. Information about personal characteristics can be derived.

On the other hand, in another embodiment of the present invention, the specific detailed interview image evaluation model may include a module for extracting the user's characteristic information from a plurality of frames of the user's interview answer image, through which the automatic evaluation result deriving unit In the interviewer terminal 3000 by (1410), the user's recommendation information including information on the prediction of acceptance of the company, information on the prediction of the degree of performance in the company, and information on the prediction of service in the company, etc. can provide

10 schematically illustrates an example of a screen for personality question content displayed on the user terminal 2000 according to the operation of the online interview providing unit 1400 according to an embodiment of the present invention.

The online interview providing unit 1400 may generate or manage interview question content by a terminal used by a hiring manager of a corresponding company, such as the interviewer terminal 3000 . FIG. 10 shows an initial screen displayed on the user terminal 2000 by the online interview providing unit 1400 .

In FIG. 10, information on interview question content that has already been created or initially given is displayed. The information of the interview question content may include a question of the corresponding interview question content, identification information of the interview question content, and a creation date of the interview question content. Also, as in the first box, an interface element capable of arbitrarily generating interview question content before the user performs an actual online interview may be displayed. Accordingly, when a box for specific interview question content on the screen shown in FIG. 10 is selected, the screen may be switched to a screen for performing an online interview.

11 schematically illustrates an example of a screen for soft skill information displayed on the interviewer terminal 3000 according to the operation of the online interview providing unit 1400 according to an embodiment of the present invention.

The automatic evaluation result deriving unit 1410 may enable various evaluation screens based on the automatic evaluation result of the user's interview answer image to be displayed on the interviewer terminal 3000 .

On the other hand, the automatic evaluation result is soft skills information including information about the user's communication and attitude, the user's personality characteristic information, the user's gaze change information in the image, and the employment recommendation index. It may include one or more.

Meanwhile, in another embodiment of the present invention, each of the soft skill information, the personality characteristic information, the gaze change information, and the employment recommendation index may be provided to the interviewer through an interface displayed on the interviewer terminal 3000 .

11 shows a screen that provides a comprehensive evaluation for the corresponding interview response image including soft skill information (when an element for 'AI synthesis' is selected in the L6 area of FIG. 11).

The display element in the interviewer terminal 3000 provided by the automatic evaluation result deriving unit 1410 includes: an image layer (L4) capable of reproducing the user's interview answer image; an image information layer (L5) on which identification information, date, and explanatory information for the interview answer image can be displayed; a category layer (L6) capable of selecting categories of information on two or more automatic evaluation results; and an automatic evaluation result layer (L7, L8, L9) in which an automatic evaluation result for a selected evaluation result category is displayed among the categories of information on the at least two automatic evaluation results.

In the image layer L4, the corresponding interview answer image may be reproduced according to the user's input for reproduction.

In the category layer L6, the category of information on the automatic evaluation result is changed according to the input of the interviewer, and the providing screen is switched accordingly. 'AI synthesis' of the category layer (L6) means providing soft skill information included in the automatic evaluation result, 'personality characteristic' means providing the personality characteristic information included in the automatic evaluation result, and 'frame Star analysis' means to provide gaze change information included in the automatic evaluation result.

That is, the screen in the interviewer terminal 3000 provided by the automatic evaluation result deriving unit 1410 is a first screen ( FIG. 11 ) that displays soft skill information for the user's interview answer image, the user's It includes a second screen (not shown) for displaying personality characteristic information with respect to the interview answer image and a third screen (not shown) for displaying an element for evaluation of a partial unit of the image, preferably gaze change information.

In the automatic evaluation result layer (L7, L8, and L9 may correspond to this) of FIG. 11, a plurality of evaluation information regarding the automatic evaluation result is displayed, and soft skill information of the corresponding user is displayed (L9).

The displayed soft skill information includes quantitative values for one or more of favorable sensitivity, concentration, trust, liveliness, composure, communication ability, eye contact, kindness, use of habitual words, and pleasantness. On the other hand, in the L7 layer, the comprehensive evaluation score, grade information, and recommendation index based on the quantitative values included in the soft skill information are displayed, and in the L8 layer, the soft skill information and the resulting comprehensive evaluation score, grade information and recommendation index are displayed. Thus, the evaluation information in the form of text automatically created is displayed.

On the other hand, although not shown in FIG. 11, in another embodiment of the present invention, the category layer (L6) includes a separate element for displaying the employment recommendation index, and when the element is selected, hiring recommendation for the user A separate screen on which the index is displayed may be output to the interviewer terminal 3000 .

12 schematically illustrates steps for determining the identity of a user according to an embodiment of the present invention.

12, the step of determining the identity may include: extracting each first facial feature information from a plurality of first face images included in a plurality of frames of the personality response image (S40); extracting each second facial feature information from a plurality of second face images included in a plurality of frames of the interview answer image (S41); and determining the identity of the user in each image by comparing the first facial feature information and the second facial feature information (S42).

Specifically, the identity determination unit 1500 is based on the answer content for the personality test performed by the user, preferably the personality answer image included in the answer content, and the interview answer image for the online interview additionally performed by the user. It is determined whether the person in the answer image is the same person or not.

To this end, the identity determining unit 1500 performs a step (S40) of extracting the first facial feature information from each of the plurality of frames of the personality response image. In order to perform the step S40, the identity determination unit 1500 extracts the first facial feature information using the identity evaluation model, or extracts the first facial feature information using a preset rule-based algorithm. have.

In addition, the identity determination unit 1500 performs a step (S41) of extracting the second facial feature information from each of the plurality of frames of the interview answer image. Similarly, in order to perform the step S41, the identity determining unit 1500 extracts the second facial feature information using the identity evaluation model, or extracts the second facial feature information using a preset rule-based algorithm. can do.

On the other hand, the plurality of frames for each of the personality answer image and the interview answer image may correspond to a plurality of consecutive frames adjacent to each other, but preferably, the plurality of frames are extracted according to a predetermined interval or randomly from the entire image. may be Accordingly, the identity determining unit 1500 can determine the identity of the entire section of each answer image.

Finally, the identity determination unit 1500 compares each extracted one or more first facial feature information and one or more second facial feature information to finally determine whether the user of the personality answer image and the interview answer image is the same person ( S42) is performed. More specifically, in order to mutually compare one or more first facial feature information and one or more second facial feature information in the corresponding step (S42), only specific second facial feature information that is matched one-to-one with respect to a single first facial feature information is mutually Alternatively, all or one or more second facial feature information may be compared with respect to a single first facial feature information.

In this way, in order to mutually compare the one or more first facial feature information and the one or more second facial feature information, the identity determination unit 1500 determines the identity using a machine-learned identity evaluation model or a preset rule-based algorithm. can judge

13 schematically illustrates the operation of the identity determining unit 1500 according to an embodiment of the present invention.

FIG. 13 corresponds to a diagram illustrating analysis of the user's personality response image and interview response image by using the identity evaluation model learned by the identity determination unit 1500 .

The identity evaluation model may include a variety of detailed identity evaluation models that perform evaluation on the personality response image and the interview response image. The detailed identity evaluation model corresponds to a detailed identity evaluation model that can perform evaluation based on a deep learning-based evaluation model, or according to a preset routine or algorithm that is not learning. It may correspond to a detailed identity evaluation model in which the first facial feature information and the second facial feature information are derived, and the identity is evaluated based on the derived first facial feature information and the second facial feature information.

In an embodiment of the present invention, the identity determination unit 1500 basically receives a personality answer image and an interview answer image including a plurality of consecutive image (image) information, and learns through machine learning technology such as deep learning. The result of determining the identity of the user is derived through the identified identity evaluation model. Additionally, the identity determination unit 1500 may analyze the personality response image and the interview response image based on a preset rule rather than machine learning, and may derive the first facial feature information and the second facial feature information, respectively. The detailed identity evaluation model may extract image information from a personality answer image and an interview answer image including a plurality of consecutive images (images) and input it into each detailed identity evaluation model to derive a result value.

On the other hand, the identity determination unit 1500 includes the identity evaluation model, and derives an identity determination result based on the first facial feature information and the second facial feature information derived from the identity evaluation model, or the identity determination unit 1500 may call a separately provided identity evaluation model to derive an identity determination result based on the first facial feature information and the second facial feature information derived from the identity evaluation model.

On the other hand, the identity determination result may be expressed as information on whether or not each user is identical in the personality answer image and the interview answer image, or may be expressed as information about a possibility such as a probability.

In another embodiment of the present invention, the identity evaluation model is metric-learned by the answer image obtained in the hiring step preceding the personality test or online interview, and the user's answer image and the metric-learned answer obtained in the following hiring step It may include a metric learning model for deriving information including similarity information between images.

14 schematically shows the internal configuration of each evaluation model according to an embodiment of the present invention.

As described above, the computing system 1000 includes the authenticity evaluation model as shown in FIG. 14(A), the interview image evaluation model as shown in FIG. 14(B), and the evaluation model shown in FIG. 14(C). As such, it includes an identity evaluation model.

14, in an embodiment of the present invention, the authenticity evaluation model, the interview image evaluation model, and the identity evaluation model are characteristic information for a user's personality response image or interview response image corresponding to input information, respectively. It may include a feature extraction model for deriving , and a feature inference model for deriving inference information for each evaluation model based on the feature information.

More specifically, the feature inference model included in the authenticity evaluation model may include a authenticity inference model for deriving authenticity information about the user's personality response image, and the feature inference model included in the interview image evaluation model includes a user may include an interview result inference model for deriving an automatic evaluation result for the interview response image of It may include an identity inference model for

Each of the authenticity evaluation model, the interview image evaluation model, and the identity evaluation model may include a feature extraction model and a feature inference model. The feature extraction model may correspond to a machine learning-based model, for example, may consist of one or more CNN modules and one or more LSTM modules, or may be configured in the form of a three-dimensional convolutional neural network (Conv3D).

The feature extraction model is a plurality of continuous image (video) information or a plurality of continuous image (video) information and voice information of the user's personality answer image or interview answer image including the input as an input for the user's answer image Derive feature information.

On the other hand, the feature inference model derives inference information related to authenticity information, an automatic evaluation result, or an identity determination result based on the feature information about the personality answer image or interview answer image derived from the feature extraction model.

The authenticity inference model included in the authenticity evaluation model can be learned by the personality response image of the successful and unqualified company through the model learning unit 1600, and the interview result inference model included in the interview image evaluation model is a model Through the learning unit 1600, it can be learned by the interview answer image of the successful and unsuccessful candidates of the company. On the other hand, the identity inference model included in the identity evaluation model may be learned through separate learning data for learning similarities between different images or images.

On the other hand, in another embodiment of the present invention, the feature inference model included in the authenticity evaluation model can derive authenticity information by processing the problem of classifying according to a preset discrimination criterion using a Softmax activation function, and the identity evaluation The feature inference model included in the model can be implemented in a way that derives the sameness determination result by using a sigmoid activation function for regression analysis, etc.

15 schematically shows the internal configuration of a feature extraction model according to an embodiment of the present invention.

The feature extraction model according to the embodiment shown in FIG. 15 is a first deep extracting spatial feature information for deriving a plurality of image feature information from image information of a plurality of frames of the user's personality answer image or interview answer image neural network; a second deep neural network for extracting spatial feature information for deriving a plurality of voice feature information from a plurality of voice information of the user's personality response image or interview response image; a first cyclic neural network module for receiving the plurality of image feature information and deriving first feature information; and a second recurrent neural network module that receives the plurality of voice feature information and derives second feature information.

The first deep neural network and the second deep neural network may correspond to a CNN module, etc., and in the embodiment shown in FIG. 15, the first deep neural network corresponds to the first CNN module, and the second deep neural network The network may correspond to the second CNN module.

The first recurrent neural network module and the second recurrent neural network module may correspond to an LSTM module and the like, and in the embodiment shown in FIG. may correspond to the second LSTM module.

Hereinafter, an operation of a neural network according to an embodiment of the present invention will be described based on the embodiment shown in FIG. 15 .

The plurality of frames may be generated by dividing an image of an image at preset time intervals. In addition, the plurality of image feature information derived by the first CNN module is preferably input to the first LSTM module in chronological order.

On the other hand, the characteristic information (pitch, intensity, etc.) of the voice for a preset time period, or the data of the voice itself, is input to the second CNN module, and the voice feature information derived from the second CNN module is sent to the second LSTM module in time series order. input is preferred.

In addition, the feature information for the voice may correspond to pitch or intensity, but more preferably, the voice is divided into certain sections, and the spectrum for each section is applied to Mel Filter Bank to extract features through Cepstral analysis. A Mel-Frequency Cepstral Coefficient (MFCC) may be applicable.

Meanwhile, feature information (a vector sequence) corresponding to the output of the feature extraction model is derived based on the first feature information and the second feature information. In the simplest method, the characteristic information can be derived by simply combining the first characteristic information and the second characteristic information, or the characteristic information can be obtained by applying a weight to the first characteristic information and the second characteristic information. can also be derived.

16 schematically shows the internal configuration of a feature inference model according to an embodiment of the present invention.

As described above, the feature inference model included in the authenticity evaluation model includes the authenticity inference model, the feature inference model included in the interview image evaluation model includes the interview result inference model, and the feature inference included in the identity evaluation model The model may include an identity inference model, and each of the authenticity inference model, the interview result inference model, and the identity inference model may be configured as shown in FIG. 16 .

The feature inference model derives inference information by performing a process of deriving feature information by assigning weights learned by a plurality of fully-connected layers to the feature information derived from the feature extraction model.

For example, the authenticity inference model analyzes the user's personality response image to derive information on the authenticity of the user's personality test, and the interview result inference model analyzes the user's interview response image to conduct an online interview of the user. can derive information on the automatic evaluation result for , and the identity inference model can derive information about the user's identity determination result in each image by analyzing the user's personality answer image and interview answer image.

In addition, the number of the Fully-Connected Layers is not limited to the number shown in FIG. 16 , and the feature inference model may include one or more Fully-Connected Layers. When the feature inference model consists of a single fully-connected layer, the intermediate result may be omitted.

On the other hand, in another embodiment of the present invention, the feature inference model can be implemented in such a way that it uses a Softmax activation function to process the problem of classifying according to a preset criterion, or derives a score using a sigmoid activation function, etc. .

17 schematically illustrates detailed steps for performing the step of providing identity determination information according to an embodiment of the present invention.

As shown in FIG. 17 , the method of performing the online applicant recruitment process further includes a step of providing identity determination information, wherein the step of providing the identity determination information includes the personality response image received in the personality provision step or the online interview. extracting one or more judgment images including the user's face from the interview answer image received in the providing step (S50); and transmitting the extracted one or more judgment images to an external terminal to be displayed on the interviewer's terminal (S51).

Specifically, the identity determination information providing unit 1700 performs the identity determination information providing step, and the above-described identity determination unit 1500 determines the identity of the user between the user's personality response image and the interview answer image. In contrast, in the step of providing the identity determination information, the interviewer can confirm whether the actual user (applicant) is the same person through a face-to-face interview in the hiring process of the user and the company in the user's personality response image or interview answer image. do.

The identity determination information providing unit 1700 performs a step (S50) of extracting one or more determination images from the user's personality answer image or interview answer image. The judgment image extracted in step S50 may correspond to an image in which the user's face is clearly displayed. In addition, when a plurality of judgment images are extracted, judgment images for consecutive frames may be extracted, but preferably, a plurality of judgment images are extracted at a preset time interval or randomly, and a plurality of judgment images are taken in consideration of the entire section of the answer image. A judgment image of may be extracted.

In another embodiment of the present invention, the identity determination information providing unit 1700 does not extract a separate judgment image, but only the whole or part of one answer image among the personality answer image or the interview answer image. The interviewer terminal (3000) can also be provided to

Then, the identity determination providing unit performs the step (S51) of transmitting the extracted one or more determination images to the external terminal in order to provide the interviewer terminal (3000). In the step S51 , the identity determination providing unit may provide one or more determination images to the interviewer terminal 3000 directly or indirectly. Therefore, when directly providing the one or more judgment images, the external terminal may correspond to the interviewer terminal 3000, and when providing one or more judgment images indirectly, the external terminal is a separate server or computing device, etc. may be applicable.

Meanwhile, one or more judgment images provided to the interviewer terminal 3000 may be displayed through a separate interface in the interviewer terminal 3000 . Accordingly, the interviewer may determine whether the person is the same person by comparing one or more judgment images displayed on the corresponding interface with the face of the user who has actually met.

The interviewer may input the determination result information for determining the identity of the user to the interface displayed on the interviewer terminal 3000, and the identity determination information providing unit 1700 may transmit the determination result information input by the interviewer to the interviewer terminal. Receiving from (3000) (S52) may be performed. Meanwhile, in the step (S52), the identity determination information providing unit 1700 may receive the determination result information from the interviewer terminal 3000 directly or indirectly, and the determination result information is stored in the DB 1800. can be saved.

In this way, in addition to the identity determination unit 1500, through the identity determination information providing step performed by the identity determination information providing unit 1700, double prevention of fraudulent acts such as proxy examinations that may occur in the online-based recruitment process. possible effect can be exerted.

According to an embodiment of the present invention, since the personality test is performed in an online manner, it is possible to exhibit the effect of saving the cost required for the recruitment on the side of the company that is hiring.

According to an embodiment of the present invention, since authenticity information is derived from a personality answer image for a personality test performed in an online manner using a machine-learning model, the test time compared to the existing personality test consisting of overlapping similar questions can have the effect of significantly reducing

According to an embodiment of the present invention, since the personality test result is derived based on the authenticity information and the answer information for each question, it is possible to exert the effect of deriving more accurate results.

According to an embodiment of the present invention, since the interview is performed by receiving the interview answer image of the user (applicant) with respect to the interview question content provided in an online manner, it is possible to exhibit the effect of saving the cost of hiring have.

According to an embodiment of the present invention, since the online interview providing step derives the evaluation result for the interview by using a machine-learning model for the interview answer image, it is possible to provide more objective results compared to the evaluation result for the conventional interview. possible effect can be exerted.

According to an embodiment of the present invention, since the identity determination step determines the identity of the user in the personality answer image and the user in the interview answer image, it is possible to easily find fraudulent acts such as a proxy test in each hiring step. can have an effect.

According to an embodiment of the present invention, since the identity determination information providing unit provides the interviewer with a judgment image of the user who actually participated in the offline recruitment screening, the user who performed the online recruitment process by the interviewer and the user who actually participated offline It can exhibit the effect of easily grasping whether it is the same.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer-readable media may include computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The above description of the present invention is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be interpreted as being included in the scope of the present invention. do.

Claims (11)

A method of performing an online applicant recruitment process performed in a computing system having one or more processors and one or more memories, the method comprising:
A personality providing step of providing one or more personality question contents to a user terminal, and receiving answer contents including an image of the user's personality answer for each of the personality question contents from the user terminal;
Authenticity derivation step of deriving authenticity information of the personality answer image from the personality answer image by the machine-learning model;
a personality result deriving step of deriving a personality test result for the user based on the authenticity information of one or more of the personality answer images and the user's answer information for the personality question content;
an online interview providing step of providing one or more interview question contents to the user terminal, and receiving an interview answer image of the user for each of the interview question contents from the user terminal; and
Based on at least one first face image including the user's face extracted from the personality answer image and at least one second face image including the user's face extracted from the interview answer image, the user in the personality answer image and the A method of performing an online applicant recruitment process, including; an identity determination step of determining the identity of the user in the interview answer image.
The method according to claim 1,
The user's answer information to the personality question content is,
A method of performing an online applicant recruitment process, which is derived based on the voice information of the user's personality answer image for the personality question content.
The method according to claim 1,
The authenticity information is
Individually derived based on each personality answer image for one or more of the personality question content,
The personality test result includes item evaluation information and item authenticity information for each of a plurality of evaluation items,
The item evaluation information is derived based on the answer information and preset rules for one or more personality question content corresponding to the evaluation item,
Wherein the item authenticity information is derived based on the authenticity information on one or more personality question content corresponding to the evaluation item, a method of performing an online applicant recruitment process.
The method according to claim 1,
The authenticity information is
Individually derived based on each personality answer image for one or more of the personality question content,
The personality test result includes comprehensive authenticity information and item evaluation information for each of a plurality of evaluation items,
The comprehensive authenticity/satellite information is derived based on a plurality of the authenticity/satellite information;
Wherein the item evaluation information is derived based on preset rules and answer information for one or more personality question content corresponding to each evaluation item, a method of performing an online applicant recruitment process.
The method according to claim 1,
The online interview providing step includes an automatic evaluation result deriving step,
The automatic evaluation result deriving step is,
A method of performing an online applicant recruitment process for deriving an automatic evaluation result including information on one or more comprehensive evaluation items by a machine-learned model with respect to the interview answer image.
The method according to claim 1,
The identity determination step is,
extracting each first facial feature information from a plurality of first face images included in a plurality of frames of the personality response image;
extracting each second facial feature information from a plurality of second face images included in a plurality of frames of the interview answer image; and
and determining the identity of the user in each image by comparing the first facial feature information and the second facial feature information with each other.
The method according to claim 1,
The method of performing the online applicant recruitment process further comprises the step of providing identity determination information,
The step of providing the identity determination information comprises:
extracting one or more judgment images including the user's face from the personality answer image received in the personality providing step or the interview answer image received in the online interview providing step; and
Transmitting the extracted one or more judgment images to an external terminal so that they can be displayed on the terminal of the interviewer.
The method according to claim 1,
In the step of providing the personality,
A reference question providing step of providing one or more reference question contents to a user terminal, and receiving a reference answer image of the user for each of the reference question contents from the user terminal;
In the step of deriving the authenticity,
The personality response image by the machine-learning model; and a reference index derived from the reference answer image or the reference answer image; a method of performing an online applicant recruitment process for deriving authenticity information about a personality answer image based on the.
A computing system for performing an online applicant recruitment process, comprising:
Personality providing unit for providing one or more personality question content to the user terminal, and receiving answer contents including the user's personality answer image for each of the personality question content from the user terminal;
Authenticity derivation unit for deriving authenticity information of the personality answer image from the personality answer image by the machine-learned model;
a personality result deriving unit for deriving a personality test result for the user based on the authenticity information of the one or more personality answer images and the user's answer information for the personality question content;
an online interview providing unit for providing one or more interview question contents to the user terminal and receiving an interview answer image of the user for each of the interview question contents from the user terminal; and
Based on at least one first face image including the user's face extracted from the personality answer image and at least one second face image including the user's face extracted from the interview answer image, the user in the personality answer image and the Computing system, including; identity determination unit for determining the identity of the user in the interview answer image.
A method of performing an online applicant recruitment process performed in a computing system having one or more processors and one or more memories, the method comprising:
An online interview providing step of providing one or more interview question content to a user terminal, and receiving an interview answer image of the user for each of the interview question content from the user terminal;
A personality providing step of providing one or more personality question contents to the user terminal, and receiving answer contents including an image of the user's personality answer for each of the personality question contents from the user terminal;
Authenticity derivation step of deriving authenticity information of the personality answer image from the personality answer image by the machine-learning model;
a personality result deriving step of deriving a personality test result for the user based on the authenticity information of one or more of the personality answer images and the user's answer information for the personality question content; and
Based on at least one first face image including the user's face extracted from the personality answer image and at least one second face image including the user's face extracted from the interview answer image, the user in the personality answer image and the A method of performing an online applicant recruitment process, including; an identity determination step of determining the identity of the user in the interview answer image.
A computer-readable recording medium recording a program for performing the method of performing the online applicant recruitment process according to claim 1 or 8.

Images