KR20210075946A - Non-verbal Evaluation Method, System and Computer-readable Medium Based on Machine Learning - Google Patents

Abstract

The present invention relates to a non-verbal evaluation method, system, and computer-readable medium based on machine learning, and more particularly, to a non-verbal evaluation method, system, and computer-readable medium based on machine learning. The method includes generating question data including non-verbal information from information about an appraisee and an appraiser, and analyzing and evaluating non-verbal information of answers to the question data through artificial intelligence. The method includes an appraise information inputting step, a question driving step, a question implementation step, and an answer analysis step.

Description

Non-verbal Evaluation Method, System and Computer-readable Medium Based on Machine Learning

The present invention relates to a non-verbal evaluation method, system, and computer-readable medium based on machine learning, and more particularly, to generate question data including non-verbal information from information about an evaluator and an evaluator, and to answer the question data It relates to a non-verbal evaluation method, system, and computer-readable medium based on machine learning that analyzes and evaluates non-verbal information through artificial intelligence.

The interview generally consists of an interviewer who is evaluated through an interview and an interviewer who evaluates the interviewer. It consists of the interviewer's questions and the interviewer's answers, and a comprehensive evaluation of all questions and answers.

The purpose of the interview is to identify the interviewer's hard skills such as professional knowledge, professional skills, or information literacy ability, as well as soft skills such as the interviewer's attitude and communication ability through question and answer. .

Traditionally, hard skills have been identified through past data such as self-introductions, resumes, and career certificates, as well as questions about majors, and this can be easily determined by asking and answering questions from experts in the field.

On the other hand, soft skills are an area that is judged differently depending on the subjective experience and intuition of the interviewer, and the evaluation criteria and evaluation method may vary depending on the interviewer. As such, there is a problem in that it is difficult to objectively evaluate an interviewer's soft skills by an evaluation method that varies depending on the subjectivity of the interviewer.

Therefore, in order to objectively evaluate the soft skills of the interviewer, it is necessary to conduct an objective and unbiased evaluation through an intelligent behavior analyzer such as artificial intelligence rather than the subjective evaluation of the interviewer.

Also, in the case of questions given in the process of evaluating the soft skills of the interviewer, the content of the question, that is, what is being asked, is important, but also how the question is asked. In addition to the content of the question actually delivered when the interviewer asks a question to the interviewer, the meta-message delivered including the method of delivering the question, attitude, and atmosphere has a great influence on the communication process with the interviewer. Therefore, for objective evaluation, it is necessary to ask a question that properly combines verbal and non-verbal elements, and to evaluate the non-verbal elements in the interviewer's answer.

The present invention is a non-verbal evaluation method, system and computer based on machine learning that generates question data including non-verbal information from information about an evaluator and an evaluator, and analyzes and evaluates non-verbal information of an answer to the question data through artificial intelligence. - Its purpose is to provide a readable medium.

In order to solve the above problems, the present invention provides a non-verbal evaluation method based on machine learning, comprising: a question realization step of delivering a question to an examinee by at least one of auditory and visual methods based on question data including non-verbal information; and an answer analysis step of analyzing and evaluating non-verbal information included in the answer of the subject to the question data through an artificial neural network; Including, wherein the question data, including evaluation criteria information for evaluation of non-verbal information in the answer analysis step, provides a non-verbal evaluation method based on machine learning.

In the present invention, the non-verbal evaluation method includes: a subject information input step of receiving at least one of the subject information and the evaluator information for generating the initial question data; and a question derivation step of generating the question data; may further include.

In the present invention, the step of inputting the subject information includes: inputting the subject's information and information on the evaluation status for generating the question data; and an initial information generation step of generating initial information for generating the question data based on the received information on the subject to be evaluated and the evaluation status; may include.

In the present invention, the evaluator information input step includes: evaluator information input step of inputting evaluator information for generating the question data; The method may further include, in the initial information generating step, initial information for generating the question data based on the received information of the subject to be evaluated, the evaluation status, and the information of the evaluator.

In the present invention, the question deriving step may generate question data based on the initial information, and the question realization step may transmit a question based on the question data generated in the question deriving step.

In the present invention, in the non-verbal evaluation method, the question derivation step, the question realization step, and the answer analysis step can be repeatedly performed, and the answer analysis step generates analysis information based on the evaluation result, and the question derivation step includes: Question data may be generated based on at least one of the initial information and the analysis information.

In the present invention, the evaluation reference information may be updated based on the initial information or the analysis information.

In the present invention, the answer analysis step quantifies the non-verbal information of the answer to the question data by classifying it into two or more categories, and the evaluation criterion information may include evaluation criteria for the numerical analysis score for each of the two or more categories. have.

In the present invention, the non-verbal information of the question data includes information on one or more of the sentence of the question, the voice conveying the question, and the expression of the evaluator conveying the question, and the question realization step is based on the non-verbal information of the question data. Based on this, one or more of auditory data and visual data to be transmitted to the subject may be generated.

In the present invention, the question data may include data for one question or data for a question set including two or more questions.

In order to solve the above problems, the present invention is a non-verbal evaluation system based on machine learning, and an evaluator information input unit that receives one or more of the evaluator's information and the evaluator's information for generating initial question data including non-verbal information ; a question derivation unit for generating the question data; a question realization unit which transmits a question to the subject to be evaluated in at least one of auditory and visual methods based on the question data; and an answer analysis unit that analyzes and evaluates non-verbal information included in an answer of the subject to the question data through an artificial neural network; Including, wherein the question data provides a non-verbal evaluation system based on machine learning, including evaluation reference information for evaluation of non-verbal information in the answer analysis unit.

In order to solve the above problems, the present invention provides a computer-readable medium for non-verbal evaluation based on machine learning, wherein the computer-readable medium stores instructions for causing a computing device to perform the following steps, , The steps may include: an evaluator information input step of receiving at least one of the evaluator's information and the evaluator's information for generating initial question data including non-verbal information; a question derivation step of generating the question data; a question realization step of delivering a question to the subject to be evaluated by at least one of auditory and visual methods based on the question data; and an answer analysis step of analyzing and evaluating non-verbal information included in the answer of the subject to the question data through an artificial neural network; and, wherein the question data includes evaluation criteria information for evaluation of non-verbal information in the answer analysis step, a computer-readable medium is provided.

According to an embodiment of the present invention, when generating a question for a non-verbal evaluation method, objective evaluation is performed by generating question data including non-verbal information as well as verbal information of the question and delivering a question to the subject to be evaluated based on the non-verbal information. can have the effect of performing

According to an embodiment of the present invention, when generating a question for a non-verbal evaluation method, a suitable evaluation criterion for each question is presented by generating question data including evaluation criterion information for a transmitted question, and the evaluation criteria according to the evaluation criterion are generated. By performing the evaluation, it is possible to exert the effect of performing an accurate evaluation.

According to an embodiment of the present invention, by analyzing non-verbal information in addition to the verbal information of the answeree's answer through artificial intelligence, it is possible to exert the effect of performing an objective and unbiased evaluation.

1 is a diagram schematically illustrating the configuration of a system for performing a non-verbal evaluation method according to an embodiment of the present invention.
2 is a block diagram schematically illustrating internal components of a user terminal according to an embodiment of the present invention.
3 is a flowchart schematically illustrating detailed steps of the operation of the subject information input unit according to an embodiment of the present invention.
4 is a diagram schematically illustrating an operation of a question elicitation unit according to an embodiment of the present invention.
5 is a diagram schematically illustrating an operation of a question realization unit according to an embodiment of the present invention.
6 is a diagram schematically illustrating a method of outputting a question implemented by a question realization unit according to an embodiment of the present invention.
7 is a diagram schematically illustrating an operation of an answer analysis unit according to an embodiment of the present invention.
8 is a diagram schematically illustrating a process in which question data is utilized according to an embodiment of the present invention.
9 is a diagram schematically illustrating an operation of a question extracting unit according to an embodiment of the present invention.
10 schematically illustrates a feature map generation process of an answer analysis unit according to an embodiment of the present invention.
11 is a diagram schematically illustrating an operation of a question realization unit according to an embodiment of the present invention.
12 is a graph schematically illustrating an example of non-verbal information of question data according to an embodiment of the present invention.
13 is a diagram schematically illustrating a set of question data according to an embodiment of the present invention.

Hereinafter, various embodiments and/or aspects are disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth to provide a thorough understanding of one or more aspects. However, it will also be recognized by one of ordinary skill in the art that such aspect(s) may be practiced without these specific details. The following description and accompanying drawings set forth in detail certain illustrative aspects of one or more aspects. These aspects are illustrative, however, and some of the various methods in principles of various aspects may be employed, and the descriptions set forth are intended to include all such aspects and their equivalents.

Further, various aspects and features will be presented by a system that may include a number of devices, components and/or modules, and the like. It is also noted that various systems may include additional devices, components, and/or modules, etc. and/or may not include all of the devices, components, modules, etc. discussed with respect to the drawings. must be understood and recognized.

As used herein, “embodiment”, “example”, “aspect”, “exemplary”, etc. may not be construed as an advantage or advantage in any aspect or design described above over other aspects or designs. . The terms '~part', 'component', 'module', 'system', 'interface', etc. used below generally mean a computer-related entity, for example, hardware, hardware A combination of and software may mean software.

Also, the terms "comprises" and/or "comprising" mean that the feature and/or element is present, but excludes the presence or addition of one or more other features, elements and/or groups thereof. should be understood as not

Also, terms including an ordinal number, such as first, second, etc., may be used to describe various elements, but the elements are not limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, are generally understood by those of ordinary skill in the art to which the present invention belongs. have the same meaning as Terms such as those defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in an embodiment of the present invention, an ideal or excessively formal meaning is not interpreted as

1 is a diagram schematically illustrating the configuration of a system for performing a non-verbal evaluation method according to an embodiment of the present invention.

Referring to FIG. 1 , in the non-verbal evaluation method according to an embodiment of the present invention, when an evaluated subject 1000 receives a question for evaluation through a user terminal 2000, etc., and answers the question, the user The terminal 2000 receives the answer of the person to be evaluated 1000 and analyzes and evaluates the answer of the person 1000 to be evaluated.

At this time, the user terminal 2000 includes a smart home control device, a wall pad, a smart phone, a tablet personal computer (PC, hereinafter referred to as 'PC'), a mobile phone, Video phone, e-book reader, desktop PC, laptop PC, netbook PC, personal digital assistant (PDA, hereinafter referred to as 'PDA') ), a portable multimedia player (PMP, hereinafter referred to as 'PMP'), an mp3 player, a mobile medical device, a camera, a wearable device (for example, a head-mounted device) (head-mounted device: HMD, for example, will be referred to as 'HMD'), electronic clothing, electronic bracelet, electronic necklace, electronic accessory, electronic tattoo, smart watch, etc. .

The user terminal 2000 receives a question about the subject 1000 through a server 3000 connected through a network, etc., and transmits an answer of the subject 1000 to the server 3000 to the subject 1000 ) can be evaluated for the subject 1000 through analysis of the answer.

Alternatively, the user terminal 2000 may generate a question about the subject 1000 inside the user terminal 2000 and analyze the answer of the subject 1000 so that the evaluation of the subject 1000 is made. have.

2 is a block diagram schematically illustrating internal components of a user terminal according to an embodiment of the present invention.

The user terminal 2000 according to an embodiment of the present invention may include a processor 100 , a bus 200 , a network interface 300 , and a memory 400 . The memory may include an operating system 410 , a service provision routine 420 , question content 430 , and evaluation information 440 . The processor 100 may include a subject information input unit 110 , a question derivation unit 120 , a question realization unit 130 , and an answer analysis unit 140 . In other embodiments, the user terminal 2000 may include more components than those of FIG. 2 .

The memory 400 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. In addition, program codes for the operating system 410 , the service providing routine 420 , the question content 430 , and the evaluation information 440 may be stored in the memory. These software components may be loaded from a computer-readable recording medium separate from the memory using a drive mechanism (not shown). The separate computer-readable recording medium may include a computer-readable recording medium (not shown) such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory through the network interface unit 300 instead of a computer-readable recording medium.

The bus 200 may enable communication and data transmission between components of a computing device that controls the user terminal 2000 . The bus may be configured using a high-speed serial bus, parallel bus, storage area network (SAN), and/or other suitable communication technology.

The network interface unit 300 may be a computer hardware component for connecting a computing device controlling the user terminal 2000 to a computer network. The network interface 300 may connect a computing device controlling the user terminal 2000 to a computer network through a wireless or wired connection. A computing device that controls the user terminal 2000 through the network interface unit 300 may be wirelessly or wiredly connected to the user terminal 2000 .

The processor may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations of a computing device controlling the user terminal 2000 . The command may be provided to the processor 100 by the memory or network interface unit 300 and via the bus 200 . The processor 100 may be configured to execute program codes for the subject information input unit 110 , the question derivation unit 120 , the question implementation unit 130 , and the answer analysis unit 140 . Such program code may be stored in a recording device such as a memory.

The subject information input unit 110 , the question derivation unit 120 , the question realization unit 130 , and the answer analysis unit 140 may be configured to perform the operation of the user terminal 2000 , which will be described below. In the processor, some components may be omitted, additional components not shown may be further included, or two or more components may be combined according to a method of controlling the user terminal 2000 .

The subject information input unit 110 receives at least one of the information of the subject 1000 and the information of the evaluator for generating question data including non-verbal information. The subject information input unit 110 may generate question data for evaluation by receiving information on the subject 1000 or information on the evaluation status, and the like. For example, when the person to be evaluated 1000 is interviewed for a job, information about the personal information such as the major, interests, and abilities of the person to be evaluated, the interview location, information on the evaluation status, such as the company to which the applicant has applied, etc. are input and appropriate Allows you to create questions.

The question extracting unit 120 generates the question data. The question derivation unit 120 generates question data suitable for the information based on the information received from the subject information input unit 110 . In this case, the question data may include the content of the question to be asked to the examinee 1000 , that is, verbal information of the question, and the method of delivering the question, that is, non-verbal information of the question. In addition, the question data may include evaluation reference information for evaluating an answer to the question.

The question realization unit 130 transmits a question to the subject 1000 through at least one of auditory and visual methods based on the question data. In an embodiment of the present invention, the question realization unit 130 transmits a question to the subject 1000 based on verbal and non-verbal information of the question data. To this end, the question realization unit 130 may transmit a question to the user through a visual display or voice through a display module or an audio output module.

The answer analysis unit 140 analyzes and evaluates non-verbal information included in the answer of the subject 1000 to the question data through artificial intelligence. The assessee 1000 answers the question received from the question realization unit 130 . Such an answer is input to the answer analysis unit 140 through a camera module and a voice input module.

The answer analysis unit 140 analyzes and quantifies the non-verbal information included in the answer through artificial intelligence for the image and audio information of the subject 1000 received through the camera module and the voice input module, etc., By evaluating this according to the evaluation criteria information included in the question data, it is possible to generate analysis information on the subject 1000 .

*

Such an operation of each component of the processor 1000 may constitute a non-verbal evaluation method composed of a series of steps. According to an embodiment of the present invention, the non-verbal evaluation method includes: an evaluator information input step of receiving at least one of evaluator information and evaluator information for generating question data including non-verbal information; a question derivation step of generating the question data; a question realization step of delivering a question to the subject to be evaluated by at least one of auditory and visual methods based on the question data; and an answer analysis step of analyzing and evaluating non-verbal information included in the answeree's answer to the question data through artificial intelligence; may include. In this case, the subject information input step may be performed by the operation of the subject information input unit 110 , the question deriving step may be performed by the question deriving unit 120 , and the question realization step may be performed by the question It may be performed by the implementation unit 130 , and the answer analysis step may be performed by the answer analysis unit 140 .

3 is a flowchart schematically illustrating detailed steps of the operation of the subject information input unit according to an embodiment of the present invention.

Referring to FIG. 3 , the subject information input step performed by the subject information input unit 110 includes the subject state input step (S100); evaluator information input step (S200); and an initial information generation step (S300).

In the subject state input step ( S100 ), one or more information of the subject 1000 information and the evaluation status for generating the question data is input. In an embodiment of the present invention, the information of the person to be evaluated 1000 may be information on personal information such as major fields, interests, abilities, and the like, and the evaluation status is the interview location where the evaluation of the evaluated person is made, the company applied for, and the job It may be information about the situation of By receiving information on the subject and situation of the evaluation as described above, the content of the question for accurate evaluation and non-verbal information about the question (for example, the tone, tone, atmosphere, etc. of the question) can be derived to increase the evaluation efficiency. have.

In the evaluator information input step S200, evaluator information for generating the question data is input. In an embodiment of the present invention, the evaluator may be an interviewer when the evaluation situation is an interview for employment. The information of the evaluator may include information such as the tendency of the interviewer and the like. For example, the interviewer may be an authoritative person, and when such information is input in the evaluator information input step S200 to generate the question data, it affects the non-verbal information of the question data, or the By influencing the evaluation criteria information included in the question data, it is possible to deliver a question in an authoritative atmosphere to the evaluator 1000 , and to receive a high evaluation when the subject 1000 's answer is modest. Alternatively, the interviewer may be an open person, and when such information of the evaluator is input, free and creative questions can be delivered to the evaluator 1000 , and when the evaluator 1000 confidently expresses his/her opinion, a high evaluation can allow you to receive

In an embodiment of the present invention, the evaluator information input step S200 may be omitted.

In the initial information generation step ( S300 ), initial information for generating the question data is generated based on the received information of the subject to be evaluated, the evaluation status, and the information of the evaluator. The initial information is generated based on the information input in the steps S100 and S200, and thus serves as basic data for generating question data in the question derivation step performed by the question derivation unit 120 . In an embodiment of the present invention, since step S200 may be omitted, when step S200 is omitted, the initial information may be generated based on the received information of the subject 1000 and the evaluation status.

4 is a diagram schematically illustrating an operation of a question elicitation unit according to an embodiment of the present invention.

The question deriving step performed by the question deriving unit 120 according to an embodiment of the present invention generates question data based on the initial information. In the question derivation step, question data including information on how and what kind of question will be transmitted is generated. In this case, the question data may be generated based on the initial information, and the question data may be generated based on the information of the subject 1000 and the evaluation status.

5 is a diagram schematically illustrating an operation of a question realization unit according to an embodiment of the present invention.

The question realization step performed by the question realization unit 130 according to an embodiment of the present invention transmits a question to the assessee 1000 based on the question data generated in the question derivation step. In this case, the question realization step performed by the question realization unit 130 may generate one or more of auditory data and visual data to be delivered to the subject based on the non-verbal information of the question data. Such visual data is output through a device such as a display module so that the subject 1000 can recognize it with the eyes, and the auditory data is output through a device such as a voice output module, and the subject 1000 is recognized by the ear. make it possible

At this time. The question realization unit 130 may visually transmit a question to the examinee 1000 through only the display module, or transmit a question audibly to the examinee 1000 through only the voice output module, or the display module and A question may be transmitted to the subject 1000 in a complex manner by using the voice output module at the same time.

6 is a diagram schematically illustrating a method of outputting a question implemented by a question realization unit according to an embodiment of the present invention.

FIG. 6A illustrates a state in which a question is transmitted to the subject 1000 in the form of text by the question realization unit 130 according to an embodiment of the present invention. In an embodiment of the present invention, as shown in FIG. 6A , a question asking for a motivation for company support and an emotion to be expressed through the question may be simultaneously displayed through the display module. In this regard, the subject 1000 may receive non-verbal information through the tone of the question and the displayed emotion, and may receive an evaluation by giving an answer.

FIG. 6B illustrates a state in which a question is delivered to the subject 1000 in the form of a voice by the question realization unit 130 according to another embodiment of the present invention. In the embodiment shown in FIG. 6B, the sentence as shown in FIG. 6A is transmitted to the subject 1000 through the voice output module. The voice output module includes a speaker device and the like so that the subject 1000 can receive a question. At this time, the question transmitted through the voice output module transmits non-verbal information to the subject 1000 through the intonation, tone, tone, etc. of the question, and the subject 1000 can be evaluated by giving an answer. .

FIG. 6C illustrates a state in which a question is transmitted to the subject 1000 in the form of video and audio by the question realization unit 130 according to another embodiment of the present invention. The image may be an image obtained by photographing an actual human movement, or data obtained by imaging the movement of a computer-generated human model. The question implementation unit 130 transmits non-verbal information through the movement of the image displayed through the display module as described above, and at the same time delivers non-verbal information through the intonation, tone, and tone of the question transmitted through the voice output module. , the subject 1000 may receive an evaluation by giving an answer.

7 is a diagram schematically illustrating an operation of an answer analysis unit according to an embodiment of the present invention.

The answer analysis unit 140 may receive the answer of the subject 1000 through at least one of a camera module and a voice input module. Through the camera module, information such as movement, facial expression, and gaze processing in the process of answering the subject 1000 can be obtained, and through the voice input module, the content and intonation of the respondent 1000 . , and information such as volume can be obtained.

The answer analysis unit 140 analyzes the obtained information through artificial intelligence. In an embodiment of the present invention, the answer analysis unit 140 may perform analysis through an artificial neural network model. Preferably, the artificial neural network may be a CNN artificial neural network showing high performance in image and audio processing. In the answer analysis step performed by the answer analysis unit 140 through the artificial neural network, the non-verbal information of the answer to the question data may be classified into two or more categories and digitized to derive an analysis score. For example, the category may be a category of confidence, professionalism, tension, and execution power, and an analysis score for each category may be derived from the answer of the assessee 1000 .

The answer analysis unit 140 may detect a face from the input image of the subject 1000, extract facial feature information from the face region, and extract motion feature information from a body part region other than the face. have. Also, voice characteristic information may be extracted from the inputted voice of the subject 1000 .

The answer analysis unit 140 generates a complex feature map from each of the facial feature map, the motion feature map, and the voice feature map from the extracted facial feature information, motion feature information, and voice feature information, and analyzes the result of the analysis on the complex feature map. By deriving it, the answer of the subject 1000 may be analyzed and evaluation may be performed.

In addition, the answer analysis step may generate analysis information based on the evaluation result. The analysis information may include score information obtained by quantifying the evaluation result, and the subject 1000 may be objectively evaluated through such score information.

8 is a diagram schematically illustrating a process in which question data is utilized according to an embodiment of the present invention.

The question data includes information necessary to implement a question to be transmitted to the evaluated person 1000 , and also includes evaluation criteria information for evaluating an answer of the evaluated person 1000 to the question.

Therefore, the question data generated by the question derivation unit 120 is input to the question implementation unit 130 and used to implement the question transmitted to the subject 1000, and is inputted to the answer analysis unit 140 at the same time. It is used to evaluate the answer of the subject 1000 .

That is, the answer analysis step performed by the answer analysis unit 140 evaluates the answer of the subject 1000 based on the evaluation reference information included in the question data. As described above, as the evaluation criterion information is included in the question data, it is possible to more accurately evaluate the answer to the question.

For example, even if it is a question asking the same question content, the answer that the evaluator expects from the subject 1000 may vary depending on the sentence, tone, expression, etc. delivered, and as such a difference is reflected in the evaluation criteria information, a more detailed evaluation becomes possible In addition, by analyzing the answers to the continuous flow of questions, it is possible to exert the effect of performing evaluation according to the flow of answers in addition to the individual evaluation of each question.

In an embodiment of the present invention, the answer analysis unit 140 may derive analysis scores for two or more categories as described with reference to FIG. 7 . In this case, the evaluation criteria information may include evaluation criteria for the numerical analysis score for each of the two or more categories. For example, when the category is a category of confidence, professionalism, tension, and execution power, the answer analysis unit 140 derives an analysis score for each of the categories. For example, the answer of the evaluator 1000 may be 80 points in confidence, 35 points in professionalism, 50 points in tension, and 60 points in execution power. At this time, the evaluation criteria information is evaluated for the analysis score for each of the categories. Such evaluation may be expressed as a numerical score, or it may be expressed as a pass/fail score for a preset criterion. For example, in the evaluation criteria information, when the analysis score of confidence is 80 or more, the evaluation score is 10 points, when the analysis score is 60 or more and less than 80 points, the evaluation score is 7 points, and when the analysis score is 40 or more and less than 60 points evaluation score of 5 points, and when the analysis score is less than 40 points, the evaluation score can be a numerical score such as 2 points, or, as another example, a pass when the analysis score of confidence is 50 or more, and a fail when the analysis score is less than 50 can be evaluated

As for such evaluation criteria, the lower the analysis score, the higher the evaluation score may be set according to the category. For example, when the analysis score of the tension is 80 or more, the evaluation score is 2 points, when the analysis score is 60 or more and less than 80 points, the evaluation score is 5, and when the analysis score is 40 or more but less than 60 points, the evaluation score is 8 points, and if the analysis score is less than 40 points, the evaluation score may be 10 points. Alternatively, it may be possible to receive a higher evaluation in a specific section, such as a pass when the analysis score of the tension is 20 or more and less than 50, and a fail when it is less than 20 or 50 or more.

As described above, the analysis information generated based on the evaluation result performed according to the evaluation criteria information may include the analysis score for each category as described above, the evaluation score accordingly, and information about the answer of the subject 1000. have. Such analysis information may be used as information for evaluating the subject 1000 by itself, or may be used to derive questions to be followed later.

9 is a diagram schematically illustrating an operation of a question extracting unit according to an embodiment of the present invention.

In the non-verbal evaluation method according to an embodiment of the present invention, the question derivation step, the question realization step, and the answer analysis step may be repeatedly performed. In the process of evaluating the evaluator 1000, such as in a general interview, not only one question is given, but the subject 1000 may be evaluated in multiple ways through a combination of a series of questions. To this end, the non-verbal evaluation method according to an embodiment of the present invention may repeatedly perform the question derivation step, the question realization step, and the answer analysis step twice or more in order to ask two or more questions and evaluate the answers.

At this time, if the analysis information derived through the answer analysis step is used to derive the next question in the question derivation step, it is helpful to perform a complex evaluation in connection with the previous question.

In this way, when the question derivation step, the question realization step, and the answer analysis step are repeatedly performed, in order to derive the first question, the subject information input step is first performed. As described with reference to FIG. 3 , in the step of inputting the subject information, initial information is generated based on the input of the subject information, the evaluation status, and the evaluator information.

Thereafter, the question derivation step generates question data based on the initial information. Then, the question realization step transmits a question to the subject 1000 based on the question data, and the answer analysis step analyzes and evaluates the answer to the question of the evaluator 1000 and then generates analysis information.

Thereafter, the process returns to the question derivation step and repeats. In this case, the question derivation step generates question data based on the analysis information. Accordingly, in the question deriving step that may be performed by the question deriving unit 120 , a subsequent question may be generated based on the analysis and evaluation results of the answer of the evaluator 1000 to the previous question. In this way, when generating question data based on the analysis information, the content of the next question may be generated based on the content of the answer to the previous question, or the content of the next question may be generated based on the evaluation result of the answer to the previous question. You can also define an area.

For example, when a fail is received because the analysis score of the tension of the examinee 1000 in the previous question is high, the question derivation unit 120 may generate question data for re-evaluating the tension. For example, since the analysis score of the tension was high, the non-verbal information included in the newly created question data may set the content, sentence, intonation, and expression of the question to relieve the tension, and such questions The evaluation criteria information that can evaluate the answer to the question can also be updated.

The evaluation reference information may be updated based on at least one of the initial information and the analysis information in order to evaluate an answer to the newly generated question data. The initial information and the analysis information include information about the person to be evaluated and information on the answer of the person to be evaluated, so information for evaluating an answer to a question of the question data newly created by the initial information or the analysis information can be used as

For example, the initial information includes information on the subject's major, asks about the subject's student experience in a previous question, and as a result of analyzing the evaluator's answer, evaluation of confidence in the student experience If the result is excellent, the question derivation unit 120 generates question data to ask another question about the experience of the student, and evaluates the confidence of the examinee in answering the other question in more detail. By performing the evaluation by updating the evaluation criteria information, it is possible to perform an in-depth and multifaceted evaluation of the subject.

When repeating questions and answers in this way, after a single question is given to the evaluator, the answer to the single question may be analyzed and evaluated, and then a question may be generated again, or a series of two or more questions may be sequentially given and , after analyzing and evaluating each of the answers to the two or more questions, the question may be generated again. Accordingly, the question data may include data for one question or data for a question set including two or more questions. When the question data includes data for one question, the answer can be evaluated immediately according to the answer to the question, and the next question can be derived based on the evaluation result, but the overhead required for analysis and question derivation is high. If the data for the question set including two or more questions is included, the questions for the two or more questions are derived at once, and the evaluation result for the two or more questions is again returned to a new question set including two or more questions. Since it is used for derivation, the resource required for derivation can be saved. However, in this case, since the evaluation result of the answer to each question is not reflected in the next question, it may affect the accuracy of the evaluation.

Accordingly, in the case of the question set including the two or more questions, it is preferable to configure the questions closely related to each other to increase the accuracy of the evaluation of the answers to the question set.

10 schematically illustrates a feature map generation process of an answer analysis unit according to an embodiment of the present invention.

10 exemplarily shows a process of generating a facial feature map for facial feature information in order to evaluate non-verbal information of an examinee's answer using a CNN artificial neural network model. The method shown in FIG. 10 may be performed by the answer analysis unit 140 .

Specifically, for example, the face feature information of size 64 X 64 is convolved with 32 filters of size 5 X 5, and 32 results of size 60 X 60 are output, and this is used by 3 X 3 max pooling. to reduce the size to 30 X 30. Convolution of 32 results of size 30 X 30 with 32 filters of size 5 X 5, and 3 X 3 max pooling again results in 32 results of size 13 X 13. Here, if 64 filters of size 5 X 5 are convolved, 64 results of size 9 X 9 are obtained. After filling the boundary with 0 to make it 10 X 10, 3 X 3 max pooling is applied to 5 X 5 Get 64 results of size.

After that, a face feature map consisting of 128 vectors is obtained through FC with 512 weights and FC with 128 weights. For example, the number of final output values may be the same as the number of characteristic information about a face to be obtained.

The face feature map may be the final 128 output values, or may correspond to a feature map convolved in the sphere intermediate stage. The configuration of such a CNN-based artificial neural network module can be formed by combining one or more convolutional layers, pooling layers, activation functions such as ReLu, FC layers or Global Average Pooling layers.

The answer analysis unit 140 according to an embodiment of the present invention generates a face feature map through the process as described above, and derives an analysis score of a preset category based on the feature map to determine the answer of the assessee 1000 . evaluation can be performed.

11 is a diagram schematically illustrating an operation of a question realization unit according to an embodiment of the present invention.

Referring to FIG. 11 , in an embodiment of the present invention, the question data includes verbal information containing the content of the question and non-verbal information containing the method of delivering the question. Such verbal and non-verbal information is configured in a form that can be recognized by the subject 1000 through the question realization unit 130 and delivered to the subject 1000 .

12 is a graph schematically illustrating an example of non-verbal information of question data according to an embodiment of the present invention.

Referring to FIG. 12 , the non-verbal information of the question data may include positive and negative level information on at least one of a sentence of a question, a voice conveying the question, and an expression of an evaluator conveying the question. Such step information may be classified according to the degree of positive and negative, respectively.

For example, when asking a question about the company's application motive in an interview, if the affirmative level of the above sentence is high, the question implementation unit 130 says, "Thank you for supporting our company. Can you tell me about your motivation for application?” and it can be implemented in the form of a sentence such as “Why did you apply for our company?” If the affirmative level of the sentence is low, the question implementation unit 130 asks, “Why did you apply for our company? It can be implemented in a form having a clerical sentence structure such as ” and delivered to the evaluated 1000 .

In addition, when the affirmation level of the voice is high, the question implementation unit 130 can be implemented to deliver the question in a calm voice with a soft tone, and when the affirmation stage of the voice is low, an aggressive question at a high speed in a high-pressure voice can also be implemented.

In addition, when the affirmative stage of the expression is high, the interviewer of the image implemented through the question realization unit 130 may ask a question while looking at the subject 1000 with a smile, and the affirmation stage of the expression is If it is low, the interviewer of the implemented video may ask questions while staring at a place other than the subject 1000 as if bored.

In this way, the non-verbal information of the question data is transmitted together in the process of transferring the question from the question realizing unit 130 to the subject 1000 . As shown in FIG. 12 , the non-verbal information of the question data may be composed of a combination of various parameter values such as sentences, voices, and facial expressions.

13 is a diagram schematically illustrating a set of question data according to an embodiment of the present invention.

As in the example of FIG. 12 , when the non-verbal information of the question data consists of a combination of three parameter values of sentence, voice, and expression, various question delivery methods can be implemented for the same question content according to the parameter values. . Such various question delivery methods can be stored by composing one question set as shown in FIG. 13, and the question realization unit 130 calls a corresponding question from among the question sets according to the parameter value and asks the examinee ( 1000) can be forwarded.

Methods according to an embodiment of the present invention may be implemented in the form of program instructions that can be executed through various computing devices and recorded in a computer-readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. The application to which the present invention is applied may be installed in the user terminal through a file provided by the file distribution system. For example, the file distribution system may include a file transmission unit (not shown) that transmits the file in response to a request from the user terminal.

The device described above may be implemented as a hardware component, a software component, and/or a combination of the hardware component and the software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that can include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

Software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. , or may be permanently or temporarily embody in a transmitted signal wave. The software may be distributed over networked computing devices and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible for those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result. Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims (5)

As a non-verbal evaluation method based on machine learning performed by a user terminal comprising one or more processors and one or more memories,
an evaluator information input step of receiving at least one of the evaluator's information and the evaluator's information for generating the initial question data;
a question derivation step of generating the question data;
a question realization step of delivering a question to the assessee in one or more of auditory and visual methods based on question data including non-verbal information; and
an answer analysis step of analyzing and evaluating non-verbal information included in the subject's answer to the question data through an artificial neural network; including,
In the step of inputting the subject information,
an assessee status input step of inputting information on the assessee and information on the assessment status for generating the question data;
an evaluator information input step of inputting evaluator information for generating the question data; and
an initial information generation step of generating initial information for generating the question data based on the received information of the evaluator and the information of the evaluator and the evaluation status; Including, a non-verbal evaluation method based on machine learning.
The method according to claim 1,
The evaluation criteria information is
A non-verbal evaluation method based on machine learning, which may be updated based on at least one of the initial information and the analysis information.
The method according to claim 1,
The answer analysis step quantifies the non-verbal information of the answer to the question data by classifying it into two or more categories,
The evaluation criterion information includes evaluation criteria for numerical analysis scores for each of the two or more categories, a non-verbal evaluation method based on machine learning.
The method according to claim 1,
The question data includes data on one question, or includes data on a question set including two or more questions, a non-verbal evaluation method based on machine learning.
A computer-readable medium for non-verbal evaluation based on machine learning, comprising:
The computer-readable medium stores instructions for causing a computing device to perform the following steps:
an evaluator information input step of receiving at least one of the evaluator's information and the evaluator's information for generating the initial question data;
a question derivation step of generating the question data;
a question realization step of delivering a question to the assessee in one or more of auditory and visual methods based on question data including non-verbal information; and
an answer analysis step of analyzing and evaluating non-verbal information included in the subject's answer to the question data through an artificial neural network; including,
In the step of inputting the subject information,
an assessee status input step of inputting information on the assessee and information on the assessment status for generating the question data;
an evaluator information input step of inputting evaluator information for generating the question data; and
an initial information generation step of generating initial information for generating the question data based on the received information of the evaluator and the information of the evaluator and the evaluation status; A computer-readable medium comprising:

Images