KR20190140805A - Non-verbal Evaluation Method, System and Computer-readable Medium Based on Machine Learning - Google Patents

Abstract

The present invention relates to a non-verbal evaluation method, system, and computer-readable medium based on machine learning and, more specifically, to a non-verbal evaluation method, system, and computer-readable medium based on machine learning for generating question data including non-verbal information from information about an appraisee and an appraiser, and analyzing and evaluating the non-verbal information of answers to the question data through artificial intelligence.

Description

Non-verbal Evaluation Method, System and Computer-readable Medium Based on Machine Learning

The present invention relates to a non-verbal evaluation method, system, and computer-readable medium based on machine learning, and more particularly, to generate question data including non-verbal information from information on an evaluator and an evaluator, and to answer the question data. A non-verbal evaluation method, system, and computer-readable medium based on machine learning for analyzing and evaluating non-verbal information through artificial intelligence.

Interviews generally consist of interviewers who are evaluated through interviews and interviewers who evaluate the interviewees. The interviews consist of questions from interviewers and interviewers.

The interview aims to identify the soft skills of the interviewer's attitude and communication skills as well as the hard skills of the interviewer's expertise, expertise or information literacy through questions and answers. .

Traditionally, hard data has been identified through historical data such as self-introductions, resumes, career certificates, and questions about majors, which can be judged easily by experts in the relevant fields.

On the other hand, soft skills are areas that are judged differently according to the interviewer's subjective experience and intuition, and evaluation criteria and evaluation methods may vary according to the interviewer. As described above, there is a problem in that the soft skills of the interviewer cannot be objectively evaluated by the evaluation method that depends on the subject of the interviewer.

Therefore, in order to objectively evaluate the interviewee's soft skills, it is necessary to perform an objective and unbiased evaluation through an intelligent behavior analyzer such as artificial intelligence rather than the subjective evaluation of the interviewer.

In addition, in the case of a question given in evaluating an interviewer's soft skills, the content of the question, namely, what is asked is important, but how is it also important. In addition to the content of the questions actually delivered during the interviewer's questions to the interviewer, the meta-message, which includes how the questions are delivered, attitudes, and mood, will have a significant impact on the interviewer. Therefore, in order to evaluate objectively, it is necessary to ask a question combining appropriate linguistic and nonverbal elements, and to evaluate nonverbal elements in the interviewer's answer.

The present invention is a non-verbal evaluation method, system, and computer based on machine learning that generates question data including non-verbal information from information on an evaluator and an evaluator, and analyzes and evaluates non-verbal information of an answer to the question data through artificial intelligence. To provide a readable medium.

In order to solve the above problems, the present invention provides a non-verbal evaluation method based on machine learning, the question implementation step of delivering a question to the subject by at least one method of hearing and vision based on the question data including non-verbal information; And an answer analysis step of analyzing and evaluating non-verbal information included in the answer of the evaluator with respect to the question data through an artificial neural network; And wherein the question data includes evaluation criteria information for evaluating non-verbal information in the answer analysis step.

In the present invention, the non-verbal evaluation method, the subject information input step of receiving at least one of the information of the evaluator and the information of the evaluator for generating the initial question data; And a question derivation step of generating the question data. It may further include.

In the present invention, the evaluator information input step, the subject status input step of inputting the information of the evaluator and the information of the evaluation situation for generating the question data; And an initial information generating step of generating initial information for generating the question data based on the received information of the evaluator and the evaluation situation. It may include.

In the present invention, the evaluator information input step, the evaluator information input step of inputting information of the evaluator for generating the question data; The initial information generating step may further include generating initial information for generating the question data based on the information of the evaluator, the evaluation situation, and the information of the evaluator.

In the present invention, the question derivation step may generate question data based on the initial information, and the question implementation step may transmit a question based on the question data generated in the question derivation step.

In the present invention, the non-verbal evaluation method, the question derivation step, question implementation step and answer analysis step may be repeatedly performed, the answer analysis step generates analysis information based on the evaluation result, the question derivation step Question data may be generated based on at least one of the initial information and the analysis information.

In the present invention, the evaluation criteria information may be updated based on the initial information or the analysis information.

In the present invention, the answer analysis step is to quantify the non-verbal information of the answer to the question data into two or more categories, and the evaluation criteria information may include an evaluation criteria for the analysis score digitized by the two or more categories. have.

In the present invention, the non-verbal information of the question data includes information on at least one of a sentence of a question, a voice conveying the question, and an expression of the evaluator conveying the question, wherein the question implementing step is based on the non-verbal information of the question data. On the basis of this, one or more of the auditory data and the visual data to be delivered to the subject may be generated.

In the present invention, the question data may include data for a question set or data for a question set including two or more questions.

In order to solve the above problems, the present invention is a non-verbal evaluation system based on machine learning, the subject information input unit for receiving at least one of the information of the evaluator and the information of the evaluator for generating the initial question data including non-verbal information ; A question extracting unit generating the question data; A question implementer configured to transmit a question to the subject through at least one of hearing and vision based on the question data; And an answer analyzing unit that analyzes and evaluates non-verbal information included in an answer of the evaluator with respect to the question data through an artificial neural network. And wherein the question data includes evaluation criteria information for evaluation of non-verbal information in the answer analysis unit.

In order to solve the above problems, the present invention provides a computer-readable medium for non-verbal evaluation based on machine learning, the computer-readable medium stores instructions for causing a computing device to perform the following steps: The steps may include: an evaluator information input step of receiving at least one of an evaluator's information and an evaluator's information for generating initial question data including non-verbal information; A question derivation step of generating the question data; A question implementation step of delivering a question to the assessee in at least one of hearing and sight based on the question data; And an answer analysis step of analyzing and evaluating non-verbal information included in the answer of the evaluator with respect to the question data through an artificial neural network; Wherein the question data includes evaluation criteria information for evaluating non-verbal information in the answer analysis step.

According to an embodiment of the present invention, in generating a question for a non-verbal evaluation method, objective data is generated by generating question data including not only linguistic information but also non-verbal information of the question and delivering the question to the evaluator based on the non-verbal information. It can exert the effect to perform.

According to an embodiment of the present invention, by generating question data including evaluation criteria information about a question transmitted in generating a question for a non-verbal evaluation method, an appropriate evaluation criterion is presented for each question, and By performing the evaluation, the effect of performing the accurate evaluation can be exerted.

According to an embodiment of the present invention, by analyzing the non-verbal information through artificial intelligence in addition to the linguistic information of the respondent's answer, the objective and unbiased evaluation can be performed.

1 is a diagram schematically showing the configuration of a system for performing a non-verbal evaluation method according to an embodiment of the present invention.
2 is a block diagram schematically illustrating internal components of a user terminal according to an exemplary embodiment of the present invention.
3 is a flowchart schematically showing the detailed steps of an operation of an evaluator information input unit according to an embodiment of the present invention.
FIG. 4 is a diagram schematically illustrating an operation of a question extractor according to an exemplary embodiment of the present invention.
5 is a diagram schematically illustrating an operation of a question implementing unit according to an exemplary embodiment of the present invention.
FIG. 6 is a diagram schematically illustrating a method of outputting a question implemented by a question implementing unit according to an embodiment of the present invention.
7 is a diagram schematically illustrating an operation of an answer analyzer according to an exemplary embodiment of the present invention.
8 is a diagram schematically illustrating a process of using question data according to an embodiment of the present invention.
FIG. 9 is a diagram schematically illustrating an operation of a question extractor according to an exemplary embodiment of the present invention.
10 schematically illustrates a feature map generation process of an answer analysis unit according to an embodiment of the present invention.
11 is a diagram schematically illustrating an operation of a question implementer according to an exemplary embodiment of the present invention.
12 is a graph schematically illustrating an example of non-verbal information of question data according to an embodiment of the present invention.
FIG. 13 is a diagram schematically illustrating a set of question data according to an embodiment of the present invention. FIG.

In the following, various embodiments and / or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. However, it will also be appreciated by one of ordinary skill in the art that this aspect (s) may be practiced without these specific details. The following description and the annexed drawings set forth in detail certain illustrative aspects of the one or more aspects. However, these aspects are exemplary and some of the various methods in the principles of the various aspects may be used and the descriptions described are intended to include all such aspects and their equivalents.

Moreover, various aspects and features will be presented by a system that may include a number of devices, components, and / or modules, and the like. The various systems may include additional devices, components, and / or modules, etc., and / or may not include all of the devices, components, modules, etc. discussed in connection with the drawings. It must be understood and recognized.

As used herein, “an embodiment”, “an example”, “aspect”, “an example”, etc., may not be construed as having any aspect or design described being better or advantageous than other aspects or designs. . The terms '~ part', 'component', 'module', 'system', 'interface', etc. used generally mean a computer-related entity, for example, hardware, hardware And a combination of software and software.

In addition, the terms "comprises" and / or "comprising" mean that such features and / or components are present, but exclude the presence or addition of one or more other features, components, and / or groups thereof. It should be understood that it does not.

In addition, terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes a combination of a plurality of related items or any item of a plurality of related items.

In addition, in the embodiments of the present invention, unless otherwise defined, all terms used herein including technical or scientific terms are generally understood by those skilled in the art to which the present invention belongs. Has the same meaning as Terms such as those defined in the commonly used dictionaries should be interpreted as having meanings consistent with the meanings in the context of the related art, and ideally or excessively formal meanings, unless explicitly defined in the embodiments of the present invention. Not interpreted as

1 is a diagram schematically showing the configuration of a system for performing a non-verbal evaluation method according to an embodiment of the present invention.

Referring to FIG. 1, in the non-verbal evaluation method according to an exemplary embodiment of the present invention, the evaluator 1000 who receives the evaluation receives a question for evaluation through the user terminal 2000 and the like and answers the question. The terminal 2000 receives the answer of the assessor 1000 and analyzes and evaluates the answer of the assessor 1000.

In this case, the user terminal 2000 may include a smart home control device, a wall pad, a smart phone, a tablet personal computer (PC, hereinafter referred to as a PC), a mobile phone, Video phones, e-book readers, desktop PCs, laptop PCs, netbook PCs, personal digital assistants (PDAs). ), Portable multimedia player (PMP, hereinafter referred to as 'PMP'), MP3 player, mobile medical device, camera, wearable device (eg, head-mounted device) (head-mounted device: HMD, for example, 'HMD'), electronic clothing, electronic bracelets, electronic necklaces, electronic accessories, electronic tattoos, or smart watches. .

The user terminal 2000 receives a question about the evaluator 1000 through a server 3000 connected through a network, etc., and transmits an answer of the evaluator 1000 to the subject 1000 in the server 3000. By evaluating the answer of), the evaluator 1000 may be evaluated.

Alternatively, the user terminal 2000 may generate a question for the evaluator 1000 in the user terminal 2000 and analyze the answer of the evaluator 1000 so that the evaluator 1000 may be evaluated. have.

2 is a block diagram schematically illustrating internal components of a user terminal according to an exemplary embodiment of the present invention.

The user terminal 2000 according to an embodiment of the present invention may include a processor 100, a bus 200, a network interface 300, and a memory 400. The memory may include an operating system 410, a service providing routine 420, a question content 430, and evaluation information 440. The processor 100 may include a subject information input unit 110, a question extractor 120, a question implementer 130, and an answer analyzer 140. In other embodiments, the user terminal 2000 may include more components than the components of FIG. 2.

The memory 400 is a computer-readable recording medium and may include a permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. In addition, the program code for the operating system 410, the service providing routine 420, the question content 430, and the evaluation information 440 may be stored in the memory. Such software components may be loaded from a computer readable recording medium separate from the memory using a drive mechanism (not shown). Such a separate computer-readable recording medium may include a computer-readable recording medium (not shown) such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, and the like. In other embodiments, the software components may be loaded into the memory via the network interface unit 300 rather than the computer readable recording medium.

The bus 200 may enable communication and data transmission between components of the computing device controlling the user terminal 2000. The bus may be configured using a high-speed serial bus, a parallel bus, a storage area network and / or other suitable communication technology.

The network interface unit 300 may be a computer hardware component for connecting the computing device controlling the user terminal 2000 to a computer network. The network interface 300 may connect the computing device controlling the user terminal 2000 to a computer network through a wireless or wired connection. The computing device controlling the user terminal 2000 may be wirelessly or wiredly connected to the user terminal 2000 through the network interface 300.

The processor may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations of the computing device controlling the user terminal 2000. The instructions may be provided to the processor 100 by the memory or network interface 300 and via the bus 200. The processor 100 may be configured to execute program codes for the evaluator information input unit 110, the question extractor 120, the question implementer 130, and the answer analyzer 140. Such program code may be stored in a recording device such as a memory.

The evaluator information input unit 110, the question extractor 120, the question implementer 130, and the answer analyzer 140 may be configured to perform an operation of the user terminal 2000 to be described below. According to the method of controlling the user terminal 2000, the processor may omit some components, further include additional components not shown, or two or more components may be combined.

The evaluator information input unit 110 receives one or more of information of the evaluator 1000 and evaluator's information for generating question data including non-verbal information. The evaluator information input unit 110 may generate question data for evaluation by receiving information about the evaluator 1000 or information on an evaluation situation. For example, when the evaluator 1000 is interviewed for a job search, the evaluator 1000 receives information on personal status such as the major, interests, and abilities of the evaluator 1000, and information on an evaluation situation such as an interview place and a company that applied. You can create a question.

The question extractor 120 generates the question data. The question extractor 120 generates question data suitable for the information based on the information received from the subject information input unit 110. In this case, the question data may include content of a question to be applied to the assessor 1000, that is, linguistic information of the question and a method of delivering the question, that is, non-verbal information of the question. In addition, the question data may include evaluation criteria information for evaluating an answer to the question.

The question implementing unit 130 transmits the question to the assessed person 1000 by at least one of hearing and vision based on the question data. In one embodiment of the present invention, the question implementing unit 130 transmits a question to the assessor 1000 based on linguistic information and non-verbal information of the question data. To this end, the question implementing unit 130 may transmit a question to the user through a display module or a voice output module as a visual display or a voice.

The answer analyzing unit 140 analyzes and evaluates non-verbal information included in the answer of the examinee 1000 with respect to the question data through artificial intelligence. The evaluator 1000 will answer the question received from the question implementation unit 130. Such an answer is input to the answer analyzing unit 140 through a camera module and a voice input module.

The answer analyzing unit 140 analyzes and quantifies the non-verbal information included in the answer through artificial intelligence on the video and audio information of the subject 1000 received through the camera module and the voice input module, and the like. The evaluation information may be generated based on the evaluation criteria information included in the question data to generate analysis information about the evaluator 1000.

Operation of each component of the processor 1000 may constitute a non-verbal evaluation method composed of a series of steps. According to an embodiment of the present invention, the non-verbal evaluation method includes: a subject information input step of receiving at least one of information of an evaluator and information of an evaluator for generating question data including non-verbal information; A question derivation step of generating the question data; A question implementation step of delivering a question to the assessee in at least one of hearing and sight based on the question data; And an answer analysis step of analyzing and evaluating non-verbal information included in the answer of the assessor with respect to the question data through artificial intelligence; It may include. In this case, the evaluator information input step may be performed by an operation of the evaluator information input unit 110, the question derivation step may be performed by the question derivation unit 120, and the question implementation step is the question. It may be performed by the implementation unit 130, the answer analysis step may be performed by the answer analysis unit 140.

3 is a flowchart schematically showing the detailed steps of an operation of an evaluator information input unit according to an embodiment of the present invention.

Referring to FIG. 3, the evaluator information input step performed by the evaluator information input unit 110 may include an evaluator state input step (S100); Evaluator information input step (S200); And it may include the initial information generation step (S300).

In the evaluator state input step (S100), one or more pieces of information of the evaluator 1000 and the evaluation situation for generating the question data are input. In an embodiment of the present invention, the information of the assessor 1000 may be information about a subject such as a major field, interests, and abilities, and the evaluation situation may include an interview place where an evaluation of the evaluator is performed, a company, a job, etc. It may be information about the situation. By receiving information on the subject and the situation in which the evaluation is made, the content of the question for accurate evaluation and non-verbal information (for example, the tone of the question, tone, mood, etc.) of the question can be derived to improve the evaluation efficiency. have.

In the evaluator information input step (S200), the evaluator information for generating the question data is input. In one embodiment of the present invention, the evaluator may be an interviewer when the evaluation situation is an interview for employment. The evaluator's information may include information such as the inclination of the interviewer. For example, the interviewer may be an authoritative person, and when such information is input in the evaluator information input step S200 to generate the question data, the interviewer may affect the non-verbal information of the question data, or The evaluation criteria information included in the question data may be influenced to transmit the authoritative question to the evaluator 1000, and the evaluator 1000 may receive a high evaluation when the answer of the evaluator 1000 is humble. Alternatively, the interviewer may be an open person, and when such evaluator's information is input, the interviewer may transmit free and creative questions to the evaluator 1000, and the evaluator 1000 expresses his or her opinions confidently and has a high evaluation. Can be received.

In an embodiment of the present invention, the evaluator information input step S200 may be omitted.

In the initial information generation step (S300), the initial information for generating the question data is generated based on the information of the evaluator, the evaluation situation, and the information of the evaluator. The initial information is generated based on the information input in the steps S100 and S200, and becomes the basic data for generating the question data in the question derivation step performed by the question derivation unit 120. Since the step S200 may be omitted in an embodiment of the present invention, when the step S200 is omitted, the initial information may be generated based on the information of the evaluator 1000 and the evaluation situation.

FIG. 4 is a diagram schematically illustrating an operation of a question extractor according to an exemplary embodiment of the present invention.

The question derivation step performed by the question derivation unit 120 according to an embodiment of the present invention generates question data based on the initial information. In the question derivation step, question data including information on how and what questions to deliver are generated. In this case, the question data may be generated based on the initial information, and the question data may be generated based on the information and the evaluation status of the assessed person 1000.

5 is a diagram schematically illustrating an operation of a question implementing unit according to an exemplary embodiment of the present invention.

The question implementing step performed by the question implementing unit 130 according to an embodiment of the present invention transmits a question to the evaluator 1000 based on the question data generated in the question deriving step. In this case, the question implementing step performed by the question implementing unit 130 may generate one or more of auditory data and visual data to be delivered to the assessed subject based on the non-verbal information of the question data. Such visual data is output through a device such as a display module so that the subject 1000 can recognize the eye, and the auditory data is output through a device such as a voice output module so that the subject 1000 is recognized as an ear. Do it.

At this time. The question implementation unit 130 may visually transmit a question to the examinee 1000 through the display module only, or audibly transmit the question to the testee 1000 through the voice output module, or the display module and The voice output module may be simultaneously used to transmit a question to the evaluator 1000 in a complex manner.

FIG. 6 is a diagram schematically illustrating a method of outputting a question implemented by a question implementing unit according to an embodiment of the present invention.

FIG. 6A illustrates a question in which the question is delivered to the evaluator 1000 in the form of text by the question implementer 130 according to an embodiment of the present invention. In an embodiment of the present invention, as shown in (A) of FIG. 6, a question asking the company support motivation and an emotion to be expressed through the question may be simultaneously displayed through the display module. In this regard, the assessor 1000 may receive the non-verbal information through the speech and the displayed emotion of the question, and may be evaluated by answering the nonverbal information.

FIG. 6B illustrates the question being transmitted to the subject 1000 in the form of a voice by the question implementing unit 130 according to another embodiment of the present invention. In the embodiment of FIG. 6B, the sentence as in FIG. 6A is transmitted to the assessor 1000 through the voice output module. The voice output module may include a speaker device or the like so that the assessor 1000 may receive a question. In this case, the question transmitted through the voice output module transmits non-verbal information to the evaluator 1000 through the intonation, tone, speech, etc. of the question, and the evaluator 1000 may be evaluated by answering the question. .

FIG. 6C illustrates how the question implementation unit 130 transmits the question to the examinee 1000 in the form of an image and an audio according to another embodiment of the present invention. The image may be an image of real human motion, or may be data obtained by imaging a motion of a computer-generated human model. The question implementation unit 130 transmits the non-verbal information through the movement of the image displayed through the display module as described above, and simultaneously transmits the non-verbal information through the intonation, tone, speech, etc. of the question transmitted through the audio output module. The evaluator 1000 may receive an evaluation by answering the question.

7 is a diagram schematically illustrating an operation of an answer analyzer according to an exemplary embodiment of the present invention.

The answer analyzing unit 140 may receive the answer of the assessor 1000 through at least one of a camera module and a voice input module. Through the camera module, information such as movement, facial expression, and gaze processing in the process of answering by the assessor 1000 may be obtained, and content and intonation of the answer of the assessor 1000 through the voice input module. Information can be obtained.

The answer analyzing unit 140 analyzes the obtained information through artificial intelligence. In one embodiment of the present invention, the answer analysis unit 140 may perform analysis through an artificial neural network model. Preferably, the artificial neural network may be a CNN artificial neural network showing high performance in image and audio processing. In the answer analyzing step performed by the answer analyzing unit 140 through the artificial neural network, the non-verbal information of the answer to the question data may be classified into two or more categories and quantified to derive an analysis score. For example, the category may be a category of confidence, professionalism, tension, and execution ability, and an analysis score for each category may be derived from the answer of the assessor 1000.

The answer analyzer 140 may detect a face from the image of the subject 1000 received, extract facial feature information from the face region, and extract motion feature information from a body region rather than a face. have. In addition, voice feature information may be extracted from the voice of the subject 1000 received.

The answer analyzing unit 140 generates a composite feature map from the facial feature map, the motion feature map, and the voice feature map, respectively, from the extracted facial feature information, motion feature information, and voice feature information, and generates an analysis result of the composite feature map. Derived, the answer of the assessor 1000 may be analyzed and the evaluation may be performed.

In addition, the answer analysis step may generate analysis information based on the evaluation result. The analysis information may include score information quantifying the evaluation result, and objectively evaluate the evaluator 1000 through the score information.

8 is a diagram schematically illustrating a process of using question data according to an embodiment of the present invention.

The question data includes information necessary to implement a question for conveying to the assessor 1000, and also includes evaluation criteria information for evaluating the assessor's answer to the question.

Therefore, the question data generated by the question extractor 120 is input to the question implementer 130 and used to implement a question transmitted to the assessor 1000, and is input to the answer analyzer 140. It is used to evaluate the answer of the assessor 1000.

That is, the answer analyzing step performed by the answer analyzing unit 140 evaluates the answer of the evaluator 1000 based on the evaluation criteria information included in the question data. As such, as the evaluation data is included in the question data, the answer to the question can be more accurately evaluated.

For example, even though the question asks the same question content, the answer that the evaluator expects from the evaluator 1000 may vary depending on the sentence, tone, facial expression, etc. to be delivered, and the difference is reflected in the evaluation information. Becomes possible. In addition, by analyzing the answers to the flows of consecutive questions, it is possible to perform the evaluation according to the flow of answers in addition to the individual evaluation of each question.

In an embodiment of the present invention, the answer analyzer 140 may derive an analysis score for two or more categories as described in FIG. 7. In this case, the evaluation criteria information may include an evaluation criterion for the analysis score digitized by the two or more categories. For example, when the category is a category of confidence, professionalism, tension, and execution power, the answer analyzing unit 140 derives an analysis score for each of the categories. For example, the answer of the examinee 1000 may be 80 points of confidence, 35 points of expertise, 50 points of tension and 60 points of execution ability. At this time, the evaluation criteria information is evaluated for the analysis score for each of the categories. Such an evaluation may be expressed as a digitized score or may be expressed as a pass / fail score for a predetermined criterion. For example, in the above evaluation criteria information, when the analysis score of confidence is 80 or more, the evaluation score is 10 points, when the analysis score is 60 or more and less than 80, the evaluation score is 7 points and the analysis score is 40 or more and less than 60 points. If the analysis score of 5 is less than 40 and the analysis score is less than 40, the evaluation score can be a digitized score such as 2, or in another example, if the analysis score of confidence is 50 or more, pass, fail if less than 50 Can be evaluated

Such evaluation criteria may be set higher as the analysis score is lower according to the category. For example, when the analysis score of the tension is 80 or more, the evaluation score is 2 points, when the analysis score is 60 or more and less than 80, the evaluation score is 5, and the analysis score is 40 or more and less than 60. Is 8 points, and if the analysis score is less than 40 points, the evaluation score may be 10 points. Alternatively, if the analysis score of the tension is 20 points or more and less than 50 points, a pass, or less than 20 points or 50 points or more, may fail, such that a higher evaluation may be obtained in a specific section.

As such, the analysis information generated based on the evaluation result of performing the evaluation according to the evaluation criteria information may include information on the analysis score for each category as described above, the corresponding evaluation score, and the answer of the evaluator 1000. have. Such analysis information may be used as information for evaluating the subject 1000 by itself, or may be used to derive a subsequent question.

FIG. 9 is a diagram schematically illustrating an operation of a question extractor according to an exemplary embodiment of the present invention.

In the non-verbal evaluation method according to an embodiment of the present invention, the question derivation step, question implementation step, and answer analysis step may be repeatedly performed. In the process of evaluating the evaluator 1000 such as a general interview, not only one question is given, but the evaluator 1000 may be evaluated in multiple ways through a combination of questions. To this end, the non-verbal evaluation method according to an embodiment of the present invention may repeat the question derivation step, question implementation step and answer analysis step two or more times in order to evaluate two or more questions and evaluate the answers.

At this time, if the analysis information derived through the answer analysis step is used to derive the next question in the question derivation step, it is helpful to perform a complex evaluation in connection with the previous question.

As described above, when the question derivation step, question implementation step, and answer analysis step are repeatedly performed, first, the evaluator information input step is performed to derive the first question. As described with reference to FIG. 3, in the input of the subject information, the subject information, the evaluation situation, and the information of the evaluator are input and initial information is generated based on the subject information.

Thereafter, the question derivation step generates question data based on the initial information. Subsequently, the question implementation step transmits a question to the evaluator 1000 based on the question data, and the answer analysis step analyzes and evaluates the answer to the question of the evaluator 1000 and generates analysis information.

After that, the process returns to the question derivation step and repeats. At this time, the question derivation step generates question data based on the analysis information. Therefore, in the question derivation step that may be performed by the question derivation unit 120, a question to be generated may be generated based on an analysis and evaluation result of the answer of the evaluator 1000 with respect to a previous question. As such, when the question data is generated based on the analysis information, the content of the question to be continued may be generated based on the content of the answer to the previous question, or may be evaluated based on the evaluation result of the answer to the previous question. You can also specify an area.

For example, in the previous question, when the score of the test subject 1000 is analyzed to be high and the fail is received, the question extractor 120 may generate question data for evaluating the tension again. For example, the non-verbal information included in the newly generated question data because the analysis score of the tension is high, the content of the question, sentence, intonation and delivery expression can be set to alleviate the tension, such a question The criteria information for evaluating the answers to can also be updated.

The evaluation criteria information may be updated based on at least one of the initial information and the analysis information to evaluate an answer to newly generated question data. The initial information and the analysis information includes information about the assessor and information about the answer of the assessor, and thus information for evaluating an answer to a question of question data newly generated by the initial information or the analysis information. Can be used as

For example, the initial information includes information about the subject's major, and the previous question asks about the student's experience in the student's life and analyzes the respondent's response to evaluate the confidence in the student's experience. If the result is excellent, the question derivation unit 120 generates question data to pass another question about the student's experience, and evaluates the confidence of the evaluator's answer to the other question in more detail. By performing evaluation by updating evaluation criteria information, it is possible to perform in-depth and multifaceted evaluation of the evaluator.

When the question and answer are repeated in this way, the subject may be asked one question and then analyzed and evaluated for the answer to the question, and then the question may be generated again, or a series of two or more questions may be given sequentially. In addition, each of the answers to the two or more questions may be analyzed and evaluated, and then a question may be generated again. Therefore, the question data may include data for one question or data for a question set including two or more questions. When the question data includes data for one question, the answer may be immediately evaluated according to the answer to the question, and the next question may be derived based on the evaluation result, but the overhead required for analysis and derivation of the question is large. If you include data for a question set that includes two or more questions, you derive the questions for two or more questions at once, and return the evaluation result of the two or more questions to a new question set that includes two or more questions. It can be used to derive, thus saving the resources required for derivation. In this case, however, the evaluation results of the answers to each question may not be reflected in the following questions, which may affect the accuracy of the evaluation.

Therefore, in the case of a question set including two or more questions, it is preferable to construct questions that are closely related to each other so as to increase the accuracy of evaluating an answer to the question set.

10 schematically illustrates a feature map generation process of an answer analysis unit according to an embodiment of the present invention.

10 exemplarily illustrates a process of generating a facial feature map for facial feature information in order to evaluate non-verbal information of an answer of a subject using a CNN artificial neural network model. The method illustrated in FIG. 10 may be performed by the answer analyzer 140.

Specifically, for example, 64 X 64 facial feature information is convolved with 32 filters of 5 X 5 size to output 32 results of 60 X 60 size, and 3 X 3 max pooling is used. To reduce the size to 30 X 30. 32 results of 30 X 30 size are convolved again with 32 filters of 5 X 5 size, and 3 X 3 max pooling results in 32 results of 13 X 13 size. Here, convoluting 64 filters of size 5 X 5 yields 64 results of size 9 X 9, fills the boundary with 0 to make 10 X 10, and then applies 3 X 3 max pooling to 5 X 5 You get 64 results in size.

Then, the 512 weighted FCs and 128 weighted FCs are used to obtain the face feature map of the final 128 vectors. For example, the number of final output values may be equal to the number of characteristic information about a face to be obtained.

The facial feature map may be the final 128 output values themselves, or may correspond to the feature map convolved in the middle stage. The CNN-based artificial neural network module may be formed by combining one or more convolutional layers, a pooling layer, an activation function such as ReLu, an FC layer, or a Global Average Pooling layer.

The answer analyzing unit 140 according to an embodiment of the present invention generates the facial feature map through the above process, and derives an analysis score of a preset category based on the feature map, thereby answering the evaluator 1000. Can be evaluated.

11 is a diagram schematically illustrating an operation of a question implementer according to an exemplary embodiment of the present invention.

Referring to FIG. 11, in one embodiment of the present invention, the question data includes linguistic information containing the content of a question and non-verbal information including a method of delivering the question. The linguistic information and the non-verbal information are configured in a form that the assessor 1000 can recognize through the question implementing unit 130 and are transmitted to the assessor 1000.

12 is a graph schematically illustrating an example of non-verbal information of question data according to an embodiment of the present invention.

Referring to FIG. 12, the non-verbal information of the question data may include positive and negative step information about one or more of a sentence of a question, a voice for transmitting the question, and an expression of the evaluator for transmitting the question. Such step information may be classified according to the degree of positive and negative, respectively.

For example, in asking questions about the company's support motivation in an interview, if the affirmative level of the sentence is high, the question implementation section 130 said, “Thank you for supporting our company. Can you tell me about the motivation to support? ”And if the affirmative level of the sentence is low, the question implementation unit 130 asks,“ Why did you support our company? ” It may be implemented in a form having a clerical sentence structure, such as ”and delivered to the evaluator (1000).

In addition, when the affirmation level of the voice is high, the question implementation unit 130 may be implemented to deliver a question in a calm voice in a soft tone, and when the affirmation level of the voice is low, an aggressive question at a high speed voice at a high speed. You can also implement

In addition, when the affirmative step of the facial expression is high, the interviewer of the image implemented through the question implementation unit 130 may ask a question by looking at the subject 1000 with a smile, and the affirmative phase of the facial expression is In a low case, the interviewer of the image to be implemented may ask questions by staring at a place other than the subject 1000, as if boring.

In this way, the non-verbal information of the question data is transmitted together in the process of passing the question from the question implementer 130 to the evaluator 1000. Such non-verbal information of the question data may be composed of various parameter values such as sentences, voices, and facial expressions, as shown in FIG. 12.

FIG. 13 is a diagram schematically illustrating a set of question data according to an embodiment of the present invention. FIG.

As in the example of FIG. 12, when the non-verbal information of the question data is composed of a combination of three parameter values of sentence, voice, and facial expression, various question delivery methods may be implemented according to the parameter value for the same question content. . Such various question delivery methods may be stored by constructing a single question set as shown in FIG. 13, and the question implementer 130 calls up a corresponding question among the question sets according to the parameter value to determine the subject ( 1000).

Methods according to an embodiment of the present invention may be implemented in the form of program instructions that may be executed by various computing devices and may be recorded in a computer readable medium. In particular, the program according to the present embodiment may be configured as a PC-based program or an application dedicated to a mobile terminal. An application to which the present invention is applied may be installed in a user terminal through a file provided by a file distribution system. For example, the file distribution system may include a file transmitter (not shown) for transmitting the file at the request of the user terminal.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments are, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable gate arrays (FPGAs). It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of these, and configure the processing device to operate as desired, or process it independently or in combination. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computing devices so that they are stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

Method according to the embodiment is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine code, such as produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described by the limited embodiments and the drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different manner than the described method, or other components. Or even if replaced or replaced by equivalents, an appropriate result can be achieved. Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims which follow.

Claims (12)

As a non-verbal evaluation method based on machine learning,
A question implementation step of delivering a question to the evaluator in at least one of hearing and sight based on question data including non-verbal information; And
An answer analysis step of analyzing and evaluating non-verbal information included in an answer of the evaluator with respect to the question data through an artificial neural network; Including,
The question data,
Non-verbal evaluation method based on machine learning, including evaluation criteria information for evaluating non-verbal information in the answer analysis step.
The method according to claim 1,
The non-verbal evaluation method,
An evaluator information input step of receiving one or more of the evaluator's information and the evaluator's information for generating the initial question data; And
A question derivation step of generating the question data; Further comprising, non-verbal evaluation method based on machine learning.
The method according to claim 2,
The evaluator information input step,
An evaluator state input step of inputting an evaluator's information and an evaluation situation's information for generating the question data; And
An initial information generation step of generating initial information for generating the question data based on the input information of the evaluator and the evaluation situation; Including, non-verbal evaluation method based on machine learning.
The method according to claim 3,
The evaluator information input step,
An evaluator information input step of inputting evaluator information for generating the question data; More,
The initial information generation step,
And generating initial information for generating the question data based on the information of the evaluator, the evaluation situation, and the information of the evaluator.
The method according to claim 3,
The question derivation step generates question data based on the initial information,
The question implementation step of delivering a question based on the question data generated in the question derivation step, non-verbal evaluation method based on machine learning.
The method according to claim 5,
In the non-verbal evaluation method, the question derivation step, question implementation step, and answer analysis step may be repeatedly performed.
The answer analysis step generates analysis information based on the evaluation result,
The question deriving step generates question data based on at least one of the initial information and the analysis information.
The method according to claim 6,
The evaluation criteria information,
Non-verbal evaluation method based on machine learning, which can be updated based on at least one of the initial information and the analysis information.
The method according to claim 5,
The answer analysis step is to quantify the non-verbal information of the answer to the question data into two or more categories,
The evaluation criterion information includes an evaluation criterion for an analysis score digitized for each of the two or more categories.
The method according to claim 1,
The non-verbal information of the question data includes information on at least one of sentences of questions, voices conveying questions and expressions of evaluators conveying questions,
The question implementation step is a non-verbal evaluation method based on machine learning, generating at least one of auditory data and visual data to be delivered to the subject based on the non-verbal information of the question data.
The method according to claim 1,
And the question data includes data for a question set or data for a set of questions that include two or more questions.
As a non-verbal evaluation system based on machine learning,
An evaluator information input unit configured to receive one or more of an evaluator's information and an evaluator's information for generating initial question data including non-verbal information;
A question extracting unit generating the question data;
A question implementer configured to transmit a question to the subject through at least one of hearing and vision based on the question data; And
An answer analyzer which analyzes and evaluates non-verbal information included in the respondent's answer to the question data through an artificial neural network; Including,
The question data,
A non-verbal evaluation system based on machine learning, comprising evaluation criteria information for evaluating non-verbal information in the answer analysis unit.
A computer-readable medium for non-verbal evaluation based on machine learning,
The computer-readable medium stores instructions for causing a computing device to perform the following steps, the steps:
An evaluator information input step of receiving at least one of an evaluator's information and an evaluator's information for generating initial question data including non-verbal information;
A question derivation step of generating the question data;
A question implementation step of delivering a question to the assessee in at least one of hearing and sight based on the question data; And
An answer analysis step of analyzing and evaluating non-verbal information included in an answer of the evaluator with respect to the question data through an artificial neural network; Including,
The question data,
And evaluation criteria information for evaluating non-verbal information in the answer analysis step.

Images