KR102397112B1 - Managing Method, System and Computer-readable Medium for Interview Automatic Evaluation Model - Google Patents

Abstract

The present invention relates to a method, system, and computer-readable medium for managing an automatic interview image evaluation model, and more particularly, in a server system that performs an online interview and provides an automatic evaluation result for an interview image, the company's It relates to a method, system, and computer-readable medium for managing an automatic evaluation model of an interview video to provide an automatic evaluation result that actually matches the actual evaluation, recruitment, evaluation, and the like.

Description

Managing Method, System and Computer-readable Medium for Interview Automatic Evaluation Model

The present invention relates to a method, system, and computer-readable medium for managing an automatic interview image evaluation model, and more particularly, in a server system that performs an online interview and provides an automatic evaluation result for an interview image, the company's It relates to a method, system, and computer-readable medium for managing an automatic evaluation model of an interview video to provide an automatic evaluation result that actually matches the actual evaluation, recruitment, evaluation, and the like.

The interview generally consists of an interviewer who is evaluated through an interview and an interviewer who evaluates the interviewer.

The purpose of the interview is to understand the interviewer's hard skills such as expertise, professional skills, or information literacy ability, as well as soft skills such as the interviewer's attitude and communication ability through question and answer. .

On the other hand, in the case of soft skills, it is difficult to provide an objective evaluation result. In particular, in the case of soft skills, only an evaluator with experience as various interviewers can make a correct evaluation. In particular, there is a problem in that it is difficult to accurately evaluate soft skills in the case of conducting an online interview according to the interview video.

In addition, the soft skill is an area that is judged differently according to the interviewer's subjective experience and intuition, and the evaluation criteria and evaluation method may vary depending on the interviewer. As such, there is a problem in that it is difficult to objectively evaluate the interviewer's soft skills by the evaluation method that varies depending on the subjectivity of the interviewer.

On the other hand, when a company collects interview videos of numerous applicants through online interviews, there is a problem in that it is difficult to check all of them individually. In addition, in the case of providing automatic evaluation results for applicants' interview images through the evaluation model, the result of the evaluation model does not match the actual applicant's evaluation. Alternatively, there is a problem that it may not meet the talent image of the applicant company, or it may not meet the actual performance and performance after hiring.

The present invention is a method of managing an automatic evaluation model of an interview image to provide an automatic evaluation result that actually matches the actual evaluation, recruitment, and evaluation of a company in a server system that performs an online interview and provides an automatic evaluation result for an interview image , a system and a computer-readable medium for its purpose.

In order to solve the above problems, an embodiment of the present invention manages an evaluation model that is performed in a server system having one or more processors and one or more memories, and provides automatic evaluation results for an interview image of a person to be evaluated A method comprising: an image evaluation result learning step of learning an evaluation model that provides an automatic evaluation result for an interview image of the server system according to the image evaluation results of the evaluators for the interview image performed by the evaluator during the online interview; and a recruitment evaluation result learning step of learning the evaluation model that provides an automatic evaluation result for the interview image of the server system according to the result of the employment evaluation in the actual employment interview in the company applying for the subject to be evaluated. The evaluation model provides a method of managing an evaluation model, including one or more detailed evaluation models that have been machine-learned.

In one embodiment of the present invention, the method for managing the evaluation model provides an automatic evaluation result for the interview image of the server system according to the high evaluation result of the job ability after the actual hiring of the subject to be evaluated in the supporting company It may further include a high-performance evaluation result learning step of learning the evaluation model.

In an embodiment of the present invention, the method for managing the evaluation model provides an evaluation result that provides an automatic evaluation result derived based on the evaluation model for a company that wants to support the received interview image from the interview image The method further includes the step of, wherein the evaluation model may be provided in the server system for each company for which the evaluated person wishes to support the online interview.

In one embodiment of the present invention, the evaluation model, a common evaluation model for deriving common characteristic information for the interview image; an image evaluation result prediction model for deriving an evaluation result related to the image evaluation result based on the common characteristic information; and a recruitment evaluation result prediction model for deriving an evaluation result related to the employment evaluation result based on the common characteristic information, wherein the common evaluation model and the image evaluation result prediction model according to the comprehensive evaluation results of the evaluators This is learned, and the common evaluation model and the employment evaluation result prediction model can be learned according to the employment evaluation results of the evaluators.

In one embodiment of the present invention, the evaluation model, a common evaluation model for deriving common characteristic information for the interview image; an image evaluation result prediction model for deriving an evaluation result related to the image evaluation result based on the common characteristic information; an employment evaluation result prediction model for deriving an evaluation result related to the employment evaluation result based on the common characteristic information; and a high evaluation result prediction model for deriving evaluation results related to the high evaluation result based on the common characteristic information, wherein the common evaluation model and the image evaluation result prediction model are determined by the comprehensive evaluation results of the evaluators. is learned, the common evaluation model and the employment evaluation result prediction model are learned according to the employment evaluation results of the evaluators, and the common evaluation model and the high evaluation result prediction model are learned according to the high evaluation results of the evaluators. .

In an embodiment of the present invention, the evaluation model includes a common evaluation model for deriving common feature information for the interview image, and the common evaluation model includes a plurality of images from a plurality of frames of the interview image. a first deep neural network for extracting spatial feature information for deriving image feature information of ; a first deep neural network for extracting spatial feature information for deriving a plurality of voice feature information from a plurality of voice information of the interview image; a first cyclic neural network module for receiving the plurality of image feature information and deriving first feature information; and a second recurrent neural network module that receives the plurality of voice feature information and derives second feature information.

In an embodiment of the present invention, the method for managing the evaluation model provides an evaluation result that provides an automatic evaluation result derived based on the evaluation model for a company that wants to support the received interview image from the interview image The method may further include a step, wherein the automatic evaluation result may include an evaluation prediction result related to the image evaluation result, and an evaluation prediction result related to the employment evaluation result.

In an embodiment of the present invention, the method for managing the evaluation model provides an evaluation result that provides an automatic evaluation result derived based on the evaluation model for a company that wants to support the received interview image from the interview image further comprising the steps of

The automatic evaluation result may include a comprehensive evaluation result for the interview image based on the evaluation values of one or more detailed evaluation models learned by the image evaluation result and one or more detailed evaluation models learned by the employment evaluation result. there is.

In order to solve the above problems, an embodiment of the present invention manages an evaluation model that is performed in a server system having one or more processors and one or more memories, and provides automatic evaluation results for an interview image of a person to be evaluated A method comprising: an image evaluation result learning step of learning an evaluation model that provides an automatic evaluation result for an interview image of the server system according to the image evaluation results of the evaluators for the interview image performed by the evaluator during the online interview; and a high evaluation result learning step of learning an evaluation model that provides an automatic evaluation result for the interview image of the server system according to the high evaluation result of the job ability after the actual hiring of the subject to be evaluated in the supporting company; The evaluation model provides a method for managing an evaluation model, including one or more machine-learned detailed evaluation models.

In order to solve the above problems, an embodiment of the present invention is a server system for managing an evaluation model that provides an automatic evaluation result for an interview image of a person to be evaluated. an image evaluation result learning unit for learning an evaluation model that provides an automatic evaluation result for the interview image of the server system according to the image evaluation result of the evaluators; and a recruitment evaluation result learning unit that learns the evaluation model that provides an automatic evaluation result for the interview image of the server system according to the result of the employment evaluation in the actual employment interview in the company applying for the subject to be evaluated; The evaluation model provides a server system for managing the evaluation model, including one or more machine-learned detailed evaluation models.

In order to solve the above problems, an embodiment of the present invention provides a computer-readable recording medium in which a program for performing the method for managing the above-described evaluation model is recorded.

1 schematically shows the overall system form of a method for managing an automatic interview image evaluation model according to an embodiment of the present invention.
2 schematically shows an internal configuration of a server system according to an embodiment of the present invention.
3 schematically shows steps for learning an evaluation model according to an embodiment of the present invention.
4 schematically shows a screen in the terminal to be evaluated according to the operation of the interview image storage unit according to an embodiment of the present invention.
5 schematically shows the operation of the interview image evaluation unit according to an embodiment of the present invention.
6 schematically shows internal models constituting an evaluation model according to an embodiment of the present invention.
7 schematically shows an internal structure of an evaluation model according to an embodiment of the present invention.
8 schematically shows learning steps of an evaluation model according to an embodiment of the present invention.
9 schematically shows the internal configuration of a common evaluation model according to an embodiment of the present invention.
10 schematically shows the internal configuration of a partial evaluation model according to an embodiment of the present invention.
11 illustrates an example of a display screen in a user terminal by an operation of an evaluation result providing unit according to an embodiment of the present invention.
12 illustrates an example of a display screen in a user terminal by an operation of an evaluation result providing unit according to an embodiment of the present invention.
13 illustrates an example of a display screen in a user terminal by an operation of an evaluation result providing unit according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement them. However, the present invention may be embodied in several different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . Also, when a part "includes" a certain component, it means that other components may be further included, rather than excluding other components, unless otherwise stated.

Also, terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In this specification, a "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. In addition, one unit may be implemented using two or more hardware, and two or more units may be implemented by one hardware. Meanwhile, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or to reproduce one or more processors. Accordingly, as an example, '~' indicates components such as software components, object-oriented software components, class components, and task components, and processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. The functions provided in the components and '~ units' may be combined into a smaller number of components and '~ units' or further separated into additional components and '~ units'. In addition, components and '~ units' may be implemented to play one or more CPUs in a device or secure multimedia card.

The "user terminal" referred to below may be implemented as a computer or portable terminal that can access a server or other terminal through a network. Here, the computer includes, for example, a laptop, a desktop, a laptop, etc. equipped with a web browser (WEB Browser), and the portable terminal is, for example, a wireless communication device that guarantees portability and mobility. , PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code) Division Multiple Access)-2000, W-Code Division Multiple Access (W-CDMA), Wireless Broadband Internet (Wibro) terminals, etc. may include all types of handheld-based wireless communication devices. In addition, "network" refers to a wired network such as a local area network (LAN), a wide area network (WAN), or a value added network (VAN), or a mobile radio communication network or satellite. It may be implemented in any kind of wireless network such as a communication network.

1 schematically shows the overall form of a server system 1000 according to a method for performing an online interview or managing an automatic interview image evaluation model according to an embodiment of the present invention.

In the server system 1000 shown in FIG. 1, an interview image is received from the terminal to be evaluated in an automated manner, and the automatic evaluation result for soft skills, etc. is transmitted through the machine-learned evaluation model to the connected member, support organization (company) , it can be provided to the evaluator terminal, etc. In a preferred embodiment of the present invention, the server system 1000 is used to conduct an actual online interview, the terminal to be evaluated corresponds to a terminal of a user who wants to apply for the company, and the terminal to be evaluated is provided by the terminal to be evaluated. It may correspond to a terminal that provides an evaluation result for an interview image being used, or an evaluation result for an applicant of the interview image.

That is, the server system 1000 shown in FIG. 1 may be used as a system for performing an actual online interview with a specific institution of the applicant. In this case, the interview results such as the interview image received from the terminal of the person to be evaluated may be transmitted to the terminal of the institution.

In addition, the server system 1000 shown in FIG. 1 may be used in the form of accessing the evaluator terminal to input evaluation information. In this case, the evaluation model learning unit 1600 of the server system 1000 may learn and improve the automatic evaluation model based on the evaluation information input from the evaluator terminal or the like.

In this way, the server system 1000 shown in FIG. 1 may receive the interview image from the terminal to be evaluated, and transmit automatic evaluation information about it to one or more of the institution terminal, the terminal of the assessee, and the terminal of the assessee. At the same time, the server system 1000 may perform a complex function of learning or improving the evaluation model of the server system 1000 through the evaluation result received from the evaluator terminal.

Hereinafter, the server system 1000 of the present invention will be described with a focus on the method in which the server system 1000 is used for an actual online interview, and then the automatic evaluation model is improved and managed by various learning data. .

The terminal to be evaluated and the terminal to be evaluated in FIG. 1 may correspond to the user terminals described above, and the server system 1000 corresponds to a computing device including one or more processors and one or more memories.

The evaluator terminal may correspond to a terminal used by a hiring side in an online interview, that is, a company-side person in charge, or a terminal that transmits an interview image or an evaluation result of an applicant to the server system 1000 .

The terminal to be evaluated or the terminal to be evaluated may access the server system 1000 through a web browser program or may access the server system 1000 through a dedicated application.

The terminal to be evaluated or the terminal to be evaluated may log in to the server system 1000 with each user's account and use the service of the server system 1000 .

The evaluator terminal or the evaluator terminal logs in to the server system 1000 with different types of accounts, for example, an applicant (evaluator) type account, an evaluator type account, and an institution type (company to be hired, etc.) can do.

2 schematically shows an internal configuration of a server system 1000 according to an embodiment of the present invention.

The server system 1000 of the present invention may be used as a system for conducting an online interview with an actual company. Specifically, the applicant transmits the interview image to the server system 1000 through the terminal to be evaluated, and the server system 1000 provides the received interview image and/or automated evaluation information to the terminal (evaluator terminal) of the institution. may be

At the same time, the server system 1000 of the present invention may perform an operation of further improving the evaluation model for automatically generating evaluation information.

Meanwhile, the server system 1000 of the present invention may be implemented in a form including a plurality of servers rather than a single server physically. For example, the evaluation model learning unit 1600 may be configured as a separate physical server. In addition, the evaluation model included in the DB may be implemented as a separate server.

The question set setting unit 1100 configures a question set for question content reproduced in the terminal to be evaluated in order to receive the interview image.

Preferably, the examinee inputs information on an institution to which the examinee wants to apply, and the question set setting unit 1100 may determine a question set according to the question information set by the relevant institution.

In the server system 1000, question content that a company wants to conduct in an online interview is stored for each company in which each online interview is performed, and such question content may be implemented as one or more of text, video, and audio.

That is, when the server system 1000 performs an actual online interview, the question set setting unit 1100 generates a question set based on a list of questions about a company to which the applicant wants to apply.

Thereafter, the interview image storage unit 1200 extracts the questions of the question set sequentially or randomly and transmits them to the terminal to be evaluated, and after the user identifies the question in the terminal to be evaluated, the response image is sent to the server system (1000) send to

The interview image storage unit 1200 transmits one or more pieces of question information or question content to a terminal to be evaluated, and receives and stores an interview image in which a response to the question information is performed from the terminal to be evaluated.

The question information may be implemented in the form of one or more of a question text, a question voice, and a question image.

Specifically, in the terminal to be evaluated, an interface in which a question according to the question information is displayed or reproduced is provided, and the interview image captured by the camera and microphone provided in the user terminal is transmitted from the terminal to the server to the server system 1000, and the corresponding It is stored in DB together with identification information about the interview image.

The evaluation information receiving unit 1300 receives an evaluation result related to the online interview, a video interview for learning an evaluation model, or an evaluation result of the applicant. The evaluation information receiving unit 1300 may receive the evaluation result from the evaluator terminal shown in FIG. 1 or may receive the evaluation result from the manager of the server system 1000 .

The interview image evaluation unit 1400 generates automatic evaluation information for the interview image through an evaluation model using machine learning. Preferably, the evaluation model may include one or more detailed evaluation models based on machine learning, and more preferably, one or more detailed evaluation models based on deep learning.

That is, after receiving the interview image, the server system 1000 automatically evaluates using the evaluation model, and generates automatic evaluation information through this. The automatic evaluation information includes an interview-related video evaluation result including at least one of favorable sensitivity, concentration, trust, liveliness, composure, communication ability, eye contact, kindness, use of habits, and pleasantness; Recruitment evaluation results including at least one of application motive, challenging spirit, professionalism, cooperation, and company talent matching information, which are judged when the applicant undergoes an actual company hiring review; and one or more of the high evaluation results including one or more of the work integrity, management competency, work competency, and comprehensive evaluation grade that are judged when the applicant is actually hired by the supporting company and then undergoes a high evaluation. More preferably, the automatic evaluation result may include a comprehensive evaluation result calculated in consideration of two or more of the image evaluation result, the employment evaluation result, and the high evaluation result.

The image evaluation result includes personality characteristics including at least one of openness, conscientiousness, extraversion, affinity, and neuroticism; It may include one or more of quantitative information on at least one attitude characteristic of gaze processing, head movement, voice pitch, voice pitch, voice volume, and emotion.

Preferably, the evaluation model includes one or more deep learning-based trained artificial neural network modules. The evaluation model may further include a detailed evaluation model based on a preset rule as well as the learned artificial neural network module. That is, in an embodiment of the present invention, the evaluation model may correspond to a form in which a detailed evaluation model based on a learned artificial neural network module and a rule is used in combination.

The evaluation result providing unit 1500 provides an automatic evaluation result to the user terminal based on the automatic evaluation information. Specifically, the evaluation result providing unit 1500 transmits the automatic evaluation information or the processed data of the automatic evaluation information (data related to the implementation of the interface in which the automatic evaluation information is reflected) to the user terminal (for example, the terminal of the personnel team of the company) ), and various evaluation information is displayed according to the user's selection in the web browser and dedicated application of the user terminal.

The basic operation of the server system 1000 is to derive automatic evaluation information for the interview image received from the terminal of the evaluator, thereby reducing the burden on the evaluator's image confirmation in performing the online interview.

Meanwhile, in an embodiment of the present invention, the server system 1000 may be used to train an evaluation model. Such a function is performed by the evaluation model learning unit 1600 . That is, the evaluation model learning unit 1600 performs a method of learning an evaluation model that provides an automatic evaluation result with respect to the interview image of the subject.

The evaluation model learning unit 1600 learns an evaluation model that provides an automatic evaluation result for the interview image of the server system 1000 according to the image evaluation results of the evaluators for the interview image performed by the evaluator during the online interview. an image evaluation result learning unit 1610; a recruitment evaluation result learning unit 1620 for learning the evaluation model that provides an automatic evaluation result for the interview image of the server system 1000 according to the result of the employment evaluation in the actual employment interview at the company applying for the subject to be evaluated; and a high evaluation result learning unit 1630 that learns an evaluation model that provides an automatic evaluation result for the interview image of the server system 1000 according to the high evaluation result of the job ability after the actual hiring in the supporting company of the subject to be evaluated ), including one or more of

Preferably, the evaluation model learning unit 1600 basically includes an image evaluation result learning unit 1610, and includes at least one of the employment evaluation result learning unit 1620 and the high evaluation result learning unit 1630. do.

The evaluation model learning unit 1600 learns the evaluation model using the machine learning based on the evaluation information of the evaluator. In an embodiment of the present invention, the server system 1000 may transmit the interview image to the evaluator terminal, and receive the evaluation of the image from the evaluator terminal. The evaluation result received from the evaluator terminal is used in the online interview and used as an evaluation factor in recruiting applicants for the interview video, while at the same time learning the evaluation model that calculates the automatic evaluation result of the server system 1000. can be used

With the operation of the evaluation model learning unit 1600 as described above, while the online interview is in progress, the server system 1000 can exhibit the effect of automatically and continuously improving the performance of the evaluation model.

On the other hand, in the DB 1600 of the server system 1000, the interview image received from the terminal to be evaluated, the automatic evaluation information performed by the interview image evaluation unit 1400 for the interview image, and the evaluation information receiving unit 1300 are received one evaluation information (evaluation information about the video, information about the hiring result of the applicant, information about the actual performance after joining the applicant), user information of the person being evaluated or the evaluator, question information about the question to be answered in the interview video, And the interview image evaluation unit 1400 is used to evaluate the interview image, the evaluation model learning unit 1600 additional learning is performed by the evaluation model may be stored.

The server system 1000 shown in FIG. 2 may further include elements other than the elements shown, but for convenience, only the elements related to online interview performance and learning about the evaluation model according to embodiments of the present invention are included. indicated.

3 schematically shows steps for learning an evaluation model according to an embodiment of the present invention.

The method for managing an evaluation model that provides an automatic evaluation result for an interview image of a person to be evaluated according to an embodiment of the present invention is performed in the server system 1000 having one or more processors and one or more memories.

In step S10, the question content is provided to the terminal to be evaluated, and an interview image is received from the terminal to be evaluated.

In step S11, an image evaluation result for the interview image is received from an evaluator terminal or the like. Preferably, an evaluation performed on the interview image by a person in charge of a human resources team of a company to which the applicant wishes to apply or by hiring managers related to the company may correspond to the image evaluation result. Alternatively, such an image evaluation result may be provided by the side that provides the service of the server system 1000, but in order to hire a person who meets the desired talent image in the company and learn the evaluation model accordingly, the company's recruitment It is desirable for the person concerned with the relevant company who knows the standards, etc., to proceed.

In step S11, an image evaluation result learning step of learning an evaluation model that provides an automatic evaluation result for the interview image of the server system 1000 based on the image evaluation results of the evaluators for the interview image performed by the evaluator during the online interview is performed Through this, the evaluator who conducts the online interview of the company or the criteria for judging the company's employment can be reflected in the evaluation model.

Meanwhile, in step S20, an actual online interview may be conducted in consideration of the video evaluation result, etc., and an actual offline interview may be performed after the online interview. Through the actual interview, the corresponding company evaluates the applicant of the interview image, and in step S13, the recruitment evaluation result of the applicant of the interview image may be received from the system of the enterprise and the terminal of the person in charge of recruitment of the enterprise. Alternatively, step S13 may be performed in such a way that the service manager of the server system 1000 inputs the employment evaluation result received from the person in charge of the company.

Preferably, the employment evaluation result corresponds to a result evaluated by the company in a state in which evaluation of the applicant is substantially completed. The hiring evaluation result may include part of the evaluation of the interview video, but in reality, it may correspond to an index for one or more items determined by the company in determining whether to hire the applicant in the end.

That is, the recruitment evaluation result corresponds to the evaluation result that determines whether or not the applicant is hired by the actual company, which may or may not include the evaluation of the interview video partially.

In step S14, there is a recruitment evaluation result learning step of learning the evaluation model that provides the automatic evaluation result for the interview image of the server system 1000 according to the result of the employment evaluation in the actual employment interview in the applicant company of the subject to be evaluated. is carried out

Through this learning step, not only the judgment of the evaluator who conducts the online interview, but also the judgment of the person in charge of the HR team who conducts the actual hiring of the company and the judgment of the applicant's competency in the actual offline are reflected in the evaluation model. . Through this process, the evaluation model learns the correlation between the interview image and the actual interview result, and accordingly, it is possible to predict the actual interview result from the interview image, and at the same time, the prediction of the image evaluation result is more sophisticated. can be done

Thereafter, in step S20, after the applicant is hired, for example, after one or two years have elapsed, the hiring company may calculate the high performance of the applicant. Thereafter, in step S15, a high evaluation result for the corresponding evaluator is received. Similarly, in step S15, the recruitment evaluation result of the applicant of the interview image may be received from the system of the company and the terminal of the person in charge of the high performance evaluation of the company. Alternatively, step S15 may be performed in such a way that the service manager of the server system 1000 inputs the employment evaluation result received from the person in charge of the company.

In step S16, high evaluation result learning for learning the evaluation model that provides the automatic evaluation result for the interview image of the server system 1000 according to the high evaluation result of the job ability after the actual employment in the supporting company of the subject to be evaluated Steps are performed.

According to this learning step, not only the judgment of the evaluator who conducts the online interview, the judgment of the person in charge of the HR team who conducts the actual hiring of the company, and the judgment of the applicant's competency offline, but also the judgment of the applicant after the actual hiring of the interview video. The actual judgment on competency is reflected in the evaluation model. Through this process, the evaluation model learns the correlation between the interview image and the work competency during actual work, and accordingly, it is possible to predict the work capacity during the actual work from the interview image, and at the same time, the image evaluation Prediction of outcomes can also be more sophisticated.

Thereafter, in step S17, an automatic evaluation result for the subsequent interview image is calculated by the evaluation model learned through the above process, and the result is provided to the company. Through the above steps S11 to S16, the evaluation model can make a more accurate determination of deriving the talent that the actual company wants.

In particular, through step S13, the evaluation model can derive with high accuracy the final recruitment evaluation result through an offline interview of an actual applicant only with the interview video, and also through step S15, the evaluation model is an interview video It is possible to derive work competency and output with high accuracy when working in In addition, through steps S13 and S15, it is possible to exhibit the effect of increasing the accuracy of the evaluation model for deriving the image evaluation result according to the step S12 according to the actual operation.

4 schematically shows a display screen in the user terminal according to the execution of the interview image storage unit 1200 according to an embodiment of the present invention.

In one embodiment of the present invention, according to the user's input, the interview image storage unit 1200 transmits a question included in a question set for a company to which the user wants to apply to the terminal to be evaluated, and the terminal to be evaluated responds to the question. After the content is played and the process of (A) and (B) of FIG. 4 is performed, the captured interview image is transmitted to the server system 1000, and then the server system 1000 stores the received interview image. do.

In FIG. 4A , a display element for counting down is displayed while a question is displayed in the currently captured image. Specifically, '7' in FIG. 4(A) is a display indicating to the user that image shooting is in progress after 7 seconds.

4(B) shows a screen in which image capturing is performed after the screen (A) of FIG. 4 is finished. In FIG. 4B , the user answers the question on the previous screen, and the terminal to be evaluated takes an image and transmits it to the server system 1000 .

5 schematically shows the operation of the interview image evaluation unit 1400 according to an embodiment of the present invention.

The interview image evaluation unit 1400 analyzes the interview image using the learned evaluation model.

The evaluation model may include various detailed evaluation models for evaluating the interview image. The detailed evaluation model corresponds to a detailed evaluation model that can perform evaluation based on the deep learning-based learned evaluation model, or features information (eg, the position of the gaze) according to a preset routine or algorithm rather than learning. It may correspond to a detailed evaluation model that derives and evaluates the derived characteristic information.

In an embodiment of the present invention, the interview image evaluation unit 1400 basically receives an interview image including image information and audio information and automatically evaluates information through an evaluation model learned through machine learning technology such as deep learning. derive

Also, in addition to this, the interview image evaluation unit 1400 may analyze the image according to a preset rule rather than machine learning and derive specific evaluation values.

The detailed evaluation model extracts audio and video information from the interview video, which is a moving picture, and inputs it to the detailed evaluation model to derive the result value, or synthesizes the audio and image information and inputs it into the detailed evaluation model to derive the result value. there is.

Preferably, the detailed evaluation model according to an embodiment of the present invention may include a module for inferring whether to speak through the shape of the mouth and evaluating the subject of the interview image in consideration of this. Specifically, the specific detailed evaluation model separately detects the mouth-shaped part of the face included in the interview image and uses artificial neural network technology that includes temporal concepts such as RNN, LSTM, and GRU to distinguish whether or not a person is speaking. and, based on this information, can be configured to automatically adjust how much weight will affect the evaluation of image information and audio information, and the model can be configured to learn about such weights. .

That is, in this configuration, if the subject or the speaker is not speaking through the mouth shape in the interview image in this configuration, the image information is automatically treated as more important (a relatively high weight is given than in the normal state) and the voice information is given less importance. It can be processed (weighed relatively lower than the normal state).

Preferably, the evaluation model may include a module for extracting a plurality of positional information of the user's gaze from a plurality of frames of the interview image, and through this, the evaluation result providing units 1500 and 1400 provide The terminal may provide information on the positional status of the gaze and the movement status of the gaze.

Preferably, the detailed evaluation model may include a module for extracting the user's head movement from a plurality of frames of the interview image, and through this, the evaluation result providing unit 1500 provides information about the head movement in the terminal to be evaluated. information can be provided.

Preferably, the detailed evaluation model may include a module for extracting sound characteristic values of voices, such as voice pitch and voice volume, in a plurality of frames, and through this, the evaluation result providing unit 1500 provides the evaluation result terminal can provide information about the voice.

Preferably, the detailed evaluation model may include a module for extracting the intensity of emotional elements for each section of the interview image, and through this, information on emotions is provided in the terminal of the person to be evaluated by the evaluation result providing units 1500 and 1400. can provide In this case, the detailed evaluation model by the artificial neural network learned by the deep learning may be used.

As such, the evaluation model may include one or more detailed evaluation models, and a part of the detailed evaluation model may be a detailed evaluation model based on rules, but preferably, the evaluation model is one or more machine-learned detailed evaluation models. or , and in an embodiment of the present invention, learning the evaluation model means learning the detailed evaluation model.

6 schematically shows evaluation models built in the server system 1000 according to an embodiment of the present invention.

As described above, the server system 1000 for performing the method of managing the evaluation model provides an automatic evaluation result derived based on the evaluation model to the company that wants to support the received interview image in the interview image. and an evaluation result providing unit 1500 .

Preferably, the evaluation model is provided in the server system 1000 for each company for which the evaluated person wishes to support the online interview.

In this way, the standards of talent desired by each company are built in the server system 1000 as an evaluation model, respectively, and the evaluation model is learned according to the evaluation results of each company. For example, assuming that companies A and B use the server system 1000, each evaluation model of company A and company B is built in the server system 1000, and when the evaluation model of company A is learned, By learning the evaluation model of Company A by reflecting the video evaluation results, recruitment evaluation results, and high performance evaluation results drawn by the officials of Company A on the interview video of applicants who applied to Company A, the evaluation results from the personnel team of Company A and Company A's evaluation model is trained so that almost similar evaluation results can be obtained.

In this way, the evaluation model can perform the prediction of the job competency in the actual company that the HR team of the corresponding company cannot guess only with the interview image.

7 schematically shows an internal structure of an evaluation model according to an embodiment of the present invention.

* The evaluation model may include a plurality of detailed evaluation models or evaluation modules. The detailed evaluation model (common model, first evaluation model, etc.) shown in FIG. 8 corresponds to an evaluation module based on machine learning.

The evaluation model according to the embodiments of the present invention may further include detailed evaluation models for performing rule-based evaluation as well as the detailed evaluation model shown in FIG. 7 , but hereinafter will be described based on the detailed evaluation model related to learning to do Preferably, the evaluation model shown in FIG. 7 corresponds to the evaluation model for a specific company.

The evaluation model includes: a common evaluation model for deriving common characteristic information for the interview image; A first evaluation model, a second evaluation model, and the like for deriving detailed evaluation results based on the common characteristic information are included.

Preferably, the first evaluation model, the second evaluation model, etc. are an image evaluation result prediction model that derives an evaluation result related to the image evaluation result based on the common characteristic information, and the common characteristic information. , an employment evaluation result prediction model for deriving an evaluation result related to the employment evaluation result, and a high performance evaluation result prediction model for deriving an evaluation result related to the high performance evaluation result based on the common characteristic information.

That is, the image evaluation result prediction model, the employment evaluation result prediction model, and the high performance evaluation result prediction model can derive the evaluation result according to the characteristic information derived from the interview image in the common evaluation model.

In this way, it is possible to learn the common evaluation model from the image evaluation result, the employment evaluation result, and the high performance evaluation result in common. That is, when the model for deriving the image evaluation result, the recruitment evaluation result, and the high evaluation result is constructed in a form that is independent, the correlation between the respective evaluation results or strong learning of the common evaluation model cannot be achieved, whereas in FIG. As shown, in the common evaluation model, the correlation between each evaluation result or strong learning of the common evaluation model can be made.

For example, in the evaluation model shown in FIG. 7 , it is possible to accurately predict the evaluation of the applicant's actual employment and the evaluation of the competency during actual work from the interview image. In addition, in learning, not only learning related to the evaluation of the superficial image, but also the result of actual employment hidden inside the image and the result of actual work can be learned.

8 schematically shows the learning steps of an evaluation model according to an embodiment of the present invention.

(A) of FIG. 8 shows that the common model and the image evaluation result prediction model among the detailed evaluation models are learned based on the interview image and the image evaluation result by the image evaluation result learning unit 1610, (B) shows that the common model and the employment evaluation result prediction model among the detailed evaluation models are learned based on the interview image and the employment evaluation result by the employment evaluation result learning unit 1620, (C) of FIG. shows that the common model and the high evaluation result prediction model among the detailed evaluation models are learned based on the interview image and the high evaluation result by the high evaluation result learning unit 1630

According to the composition and learning method of the detailed evaluation model, the common model can be learned through any evaluation result, and through this, the evaluation model that derives the evaluation result of another category through the evaluation result of a specific category can be learned. . For example, in the case where the applicant of the interview video receives a high evaluation after actually joining the company, and learns the evaluation model as shown in FIG. , the common model is learned, and accordingly, the accuracy of the model can be improved in the prediction of the image evaluation result and the prediction of the recruitment evaluation result.

The common model, image evaluation result prediction model, recruitment evaluation result prediction model, and high performance evaluation result prediction model shown in FIG. 8 each correspond to a separate artificial neural network model or a sub-network of one artificial neural network model. can That is, in the present invention, terms such as evaluation model, detailed evaluation model, common model, image evaluation result prediction model, employment evaluation result prediction model, and high performance evaluation result prediction model not only mean separate artificial neural network models, but also , should be interpreted in the broadest sense, meaning a subnetwork inside an artificial neural network model.

If the common model, image evaluation result prediction model, recruitment evaluation result prediction model, and high performance evaluation result prediction model shown in FIG. 8 correspond to each of the sub-networks of one artificial neural network model, the common feature shown in FIG. 8 is the It may correspond to an intermediate result value inside the artificial neural network model.

In addition, when the common model, the image evaluation result prediction model, the employment evaluation result prediction model, and the high performance evaluation result prediction model shown in FIG. 8 correspond to each of the sub-networks of one artificial neural network model, substantially a common model, The image evaluation result prediction model, the employment evaluation result prediction model, and the high performance evaluation result prediction model correspond to one artificial neural network model, and the one artificial neural network model can derive a result value composed of one or more parameters, and the result value may be in a form including all of the predicted image evaluation result, the predicted employment evaluation result, and the predicted high evaluation result. In this case, learning as in (A) of FIG. 8 is inputting the image evaluation result as learning data, learning as in (B) is inputting the hiring evaluation result as learning data, and learning as in (C) is Enter the high evaluation result as the learning data. In this case, learning may be performed focusing on related subnetworks in the artificial neural network model.

9 schematically shows the internal configuration of a common evaluation model according to an embodiment of the present invention.

The common evaluation model according to the embodiment shown in FIG. 9, the common evaluation model is a first deep neural for extracting spatial feature information for deriving a plurality of image feature information from image information of a plurality of frames of the interview image network; a first deep neural network for extracting spatial feature information for deriving a plurality of voice feature information from a plurality of voice information of the interview image; a first cyclic neural network module for receiving the plurality of image feature information and deriving first feature information; and a second recurrent neural network module that receives the plurality of voice feature information and derives second feature information.

The first deep neural network and the second deep neural network may correspond to a CNN module and the like. In the embodiment shown in FIG. 9, the first deep neural network corresponds to the first CNN module, and the second deep neural network may correspond to the second CNN module.

The first recurrent neural network module and the second recurrent neural network module may correspond to an LSTM module and the like, and in the embodiment shown in FIG. may correspond to the second LSTM module.

Hereinafter, an operation of a neural network according to an embodiment of the present invention will be described based on the embodiment shown in FIG. 9 .

The plurality of frames may be generated by dividing an image of an image at preset time intervals. In addition, the plurality of image feature information derived by the first CNN module is preferably input to the first LSTM module in chronological order.

On the other hand, the characteristic information (pitch, intensity, etc.) of the voice for a preset time section, or the data of the voice itself, is input to the second CNN module, and the voice feature information derived from the second CNN module is sent to the second LSTM module in chronological order. input is preferred.

On the other hand, feature information (a vector sequence) that is an output of the common model is derived based on the first feature information and the second feature information. The simplest method may be derived by simply combining the first characteristic information and the second characteristic information, or may be derived by applying a weight or the like.

10 schematically shows the internal configuration of a partial evaluation model according to an embodiment of the present invention.

The partial evaluation model shown in FIG. 10 may correspond to the image evaluation result prediction model, the employment evaluation result prediction model, or the high evaluation result prediction model shown in FIG. 8 .

Each partial evaluation model performs a process of deriving a representative vector by assigning weights learned by a plurality of fully-connected layers to the characteristic information derived from the common model, and performing the process of deriving the result values for each evaluation category to derive

For example, the image evaluation result prediction model is based on the image evaluation results related to the interview, including one or more of favorable sensitivity, concentration, trust, liveliness, composure, communication ability, eye contact, kindness, use of habits, and pleasantness. Data related to the prediction can be derived, and the recruitment evaluation result prediction model is at least one of application motivation, challenging spirit, professionalism, cooperation, and company talent matching information that is judged when the applicant undergoes an actual hiring review at a company. Data related to the prediction of recruitment evaluation results including It is possible to derive data related to the high performance evaluation results including at least one of competency and comprehensive high school grades.

Preferably, in one embodiment of the present invention, for a given interview image, a comprehensive evaluation result is derived from the feature information derived from the common model, or the image evaluation result prediction model, the employment evaluation result prediction model, and A detailed evaluation model capable of deriving a comprehensive evaluation result from at least one of the high evaluation result prediction models may be further included.

11 shows an example of a display screen in the user terminal by the operation of the evaluation result providing unit 1500 according to an embodiment of the present invention.

The evaluation result providing unit 1500 may provide various evaluation screens according to input from the terminal to the supporting company of the applicant. 11 shows a screen (when the user inputs the image evaluation result of L3 of FIG. 11) for providing the evaluation result for the interview image.

The display element in the user terminal provided by the evaluation result providing step includes: an image layer (L1) capable of reproducing the interview image; an image information layer (L2) on which identification information, date, and explanatory information for the interview image can be displayed; a category layer (L3) capable of selecting two or more evaluation result categories; and an automatic evaluation result layer L4 in which an automatic evaluation result for a selected evaluation result category among the two or more evaluation result categories is displayed.

That is, the screen on the user terminal provided by the evaluation result providing step is the first screen (FIG. 11) that displays the evaluation elements of the interview image itself as the center, and the result prediction when the applicant of the interview image is actually hired It includes a second screen (FIG. 12) that is displayed as , and a third screen (FIG. 13) that displays mainly the result (or high performance) prediction when the applicant of the interview image performs an actual job.

In the image layer L1, an interview image may be played according to a user's input.

In the category layer (L3), as the evaluation result category is changed according to the user's input, the provided screen is switched. 'Video evaluation' of the category layer (L3) corresponds to a screen that provides an evaluation of the interview video itself. Preferably, a screen showing evaluation results for social skills such as recruitment recommendation, favorable sensitivity, concentration, and trust according to the operation of the common model and the image evaluation result prediction model corresponds to the 'image evaluation'.

In FIG. 11 in which 'image evaluation' is selected, evaluation prediction results related to the image evaluation results are displayed in the automatic evaluation result layer L4 (L4.2 and L4.3). Preferably, the automatic evaluation result layer is a comprehensive evaluation result for the interview image based on the evaluation values of one or more detailed evaluation models learned by the image evaluation result and one or more detailed evaluation models learned by the employment evaluation result It can include the comprehensive evaluation layer (L4.1) that neglects the

By providing such information, when conducting an online interview, the company that wants to hire can intuitively receive an evaluation of the social skills in the interview video, even if they do not check the interview video individually.

Preferably, L4.3 shown in FIG. 11 corresponds to an evaluation factor for the interview image itself.

12 shows an example of a display screen in the user terminal by the operation of the evaluation result providing unit 1500 according to an embodiment of the present invention.

12 is an automatic evaluation result for at least one of application motivation, challenging spirit, professionalism, cooperation, and company talent match. This is the screen shown in L4.5. As described above, the evaluation model of the present invention provides a prediction result when an actual applicant undergoes an offline interview and finally receives a recruitment evaluation result from the interview video, so that it is difficult to confirm the actual job evaluation at the company in the online interview. provides evaluation predictions, and accordingly, companies that want to hire can conduct online interviews efficiently.

Preferably, L4.4 shown in FIG. 12 corresponds to a predictive factor for the actual hiring result inferred from the corresponding interview image.

13 illustrates an example of a display screen in the user terminal by the operation of the evaluation result providing unit 1500 according to an embodiment of the present invention.

13 is an automatic evaluation of at least one of the job integrity, management competency, work competency, and comprehensive evaluation, for example, the result of job evaluation after actual hiring of the applicant predicted by the evaluation model from the interview image. This is a screen showing the result in L4.6. As described above, the evaluation model of the present invention provides predictive results when receiving an evaluation of work competency, such as a high-level evaluation, assuming that an actual applicant has been hired from the interview video, so that the actual It provides evaluation predictions at work, and accordingly, companies that want to hire can conduct online interviews efficiently.

Preferably, L4.5 shown in FIG. 13 corresponds to a predictive factor for the actual job evaluation inferred from the interview image.

An embodiment of the present invention may also be implemented in the form of a recording medium including instructions executable by a computer, such as a program module to be executed by a computer. Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

Although the methods and systems of the present invention have been described with reference to specific embodiments, some or all of their components or operations may be implemented using a computer system having a general purpose hardware architecture.

The description of the present invention described above is for illustration, and those of ordinary skill in the art to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. For example, each component described as a single type may be implemented in a dispersed form, and likewise components described as distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be interpreted as being included in the scope of the present invention. do.

Claims (5)

A method of managing an evaluation model that is performed in a server system having one or more processors and one or more memories, and provides automatic evaluation results for an interview image of an appraiser,
an image evaluation result learning step of learning an evaluation model that provides an automatic evaluation result for the interview image of the server system according to the image evaluation results of the evaluators for the interview image performed by the evaluator during the online interview; and
High evaluation result of learning an evaluation model that provides automatic evaluation results for the interview image of the server system based on the interview image of the evaluated person and the high evaluation result of the job ability after the actual hiring of the evaluated person at the supporting company learning stage; including,
The evaluation model includes one or more artificial neural network models,
The evaluation model may provide a prediction result when the evaluated person receives the high evaluation result of the supporting company with respect to the input interview image of the arbitrary evaluated person,
The evaluation model is provided in the server system for each supporting company, and in the high evaluation result learning step, the evaluation model of the supporting company is learned based on the high evaluation result of the supporting company and the interview image of the subject to be evaluated. How to.
The method according to claim 1,
The evaluation model is
a common evaluation model deriving common characteristic information for interview images;
an image evaluation result prediction model for deriving an evaluation result related to the image evaluation result based on the common characteristic information;
and a high evaluation result prediction model for deriving evaluation results related to high evaluation results based on the common characteristic information,
The common evaluation model and the image evaluation result prediction model are learned by the comprehensive evaluation results of the evaluators,
A method for managing an evaluation model, wherein the common evaluation model and the high evaluation result predictive model are learned by the high evaluation results of the evaluators.
The method according to claim 1,
The evaluation model is
Including; a common evaluation model for deriving common characteristic information for the interview image;
The common evaluation model is
a first deep neural network for extracting spatial feature information for deriving a plurality of image feature information from image information of a plurality of frames of the interview image;
a first deep neural network for extracting spatial feature information for deriving a plurality of voice feature information from a plurality of voice information of the interview image;
a first cyclic neural network module for receiving the plurality of image feature information and deriving first feature information; and
A method of managing an evaluation model, including a; a second recurrent neural network module for receiving the plurality of speech characteristic information and deriving second characteristic information.
A server system that manages an evaluation model that provides automatic evaluation results for an interview image of a person being evaluated,
an image evaluation result learning unit for learning an evaluation model that provides an automatic evaluation result for the interview image of the server system according to the image evaluation results of the evaluators for the interview image performed by the person to be evaluated during the online interview; and
High evaluation result of learning an evaluation model that provides automatic evaluation results for the interview image of the server system based on the interview image of the evaluated person and the high evaluation result of the job ability after the actual hiring of the evaluated person at the supporting company learning unit; including;
The evaluation model includes one or more artificial neural network models,
The evaluation model may provide a prediction result when the evaluated person receives the high evaluation result of the supporting company with respect to the input interview image of the arbitrary evaluated person,
The evaluation model is provided in the server system for each supporting company, and in the high evaluation result learning step, the evaluation model of the corresponding supporting company is learned based on the high evaluation result of the supporting company and the interview image of the subject to be evaluated.
A computer-readable recording medium in which a program for performing the method for managing the evaluation model according to claim 1 is recorded.

Images