JP2022135881A - Skill evaluation program and skill evaluation device - Google Patents

Abstract

To provide a skill evaluation program and a skill evaluation device that can suppress reduction of the accuracy of evaluating a communication skill of an evaluation target person, caused by variations of guidance of actions that a participant who is not the evaluation target person makes.SOLUTION: A skill evaluation device 14 includes: acquisition means 30 for acquiring difficulty of a goal regarding formation of an agreement between an evaluation target person and another participant or regarding acquisition of a cooperative relationship; estimation means (44) for estimating actual difficulty from conversation data DD showing contents of utterance during conversation; calculation means (42, 48) for performing, on the conversation data DD, evaluation processing for evaluating the effectiveness of the contents of utterance during conversation or for evaluating the relevancy between contents of utterance and calculating an evaluation value showing the level of the communication skill of the evaluation target person; and correction means 36 for correcting an evaluation basis to apply to the evaluation target person or the evaluation value of the evaluation target person so that a difference between the difficulty of the goal and the actual difficulty becomes smaller.SELECTED DRAWING: Figure 3

Description

The present invention relates to a skill evaluation program and a skill evaluation device.

“Human resource development” has been one of the major issues for organizations such as companies, organizations, and schools. For example, various techniques have been proposed for analyzing utterances during dialogue by a plurality of participants and evaluating communication skills.

Patent document 1 captures video data and voice data of human communication, converts it into numerical data using artificial intelligence technology, analyzes based on the numerical data, and evaluates the contents and skills of communication. A system is disclosed. In paragraph [0090] of this patent document, there is a description to the effect that when there is a business communication purpose, the result of the communication as a whole is evaluated according to the degree to which the corresponding purpose has been achieved.

JP 2019-133451 A

In order to perform the above-described evaluation, a plurality of participants may be given a role and made to perform a pseudo task (here, an interactive role-play or an interactive simulation). In this case, in order to ensure the fairness of the evaluation, it is required that another participant, who is not the subject of the evaluation, faithfully fulfills the given role and behaves in accordance with a certain code of conduct.

However, the degree of difficulty of the task depends on the action guidelines taken by different participants, and the degree of accomplishment of the task may fluctuate. In other words, the method disclosed in Patent Document 1 has the problem that the evaluation results of communication skills are affected by fluctuations in task achievement levels, and the evaluation accuracy is reduced accordingly.

The present invention has been made in view of such problems, and its object is to improve the evaluation accuracy of the communication skills possessed by the person to be evaluated due to the variation in the action guidelines taken by other participants who are not the person to be evaluated. To provide a skill evaluation program and a skill evaluation device capable of suppressing a decrease in

A skill evaluation program according to a first aspect of the present invention is a program for evaluating the skills of an evaluation subject through interactive role-playing performed by a plurality of participants including the evaluation subject, and comprising: Acquisition means for acquiring the degree of difficulty of a goal relating to building consensus or obtaining a cooperative relationship between an evaluation subject and other participants, and dialogue obtained through the role-play assuming the degree of difficulty of the goal to be acquired by the acquisition means Estimating means for estimating the actual degree of difficulty in forming said consensus or obtaining said cooperative relationship from dialogue data indicating said contents of said dialogue, and evaluating the effectiveness of said contents of said dialogue or the relevance between said contents of said dialogue calculation means for performing an evaluation process for the dialogue data to calculate an evaluation value indicating the level of the communication skill of the person to be evaluated; It functions as correction means for correcting the evaluation criteria applied to the person to be evaluated or the evaluation value of the person to be evaluated so that the difference from the actual difficulty level obtained is reduced.

The skill evaluation program according to the second aspect of the present invention causes the computer to further function as determination means for determining whether correction by the correction means is necessary according to the difference between the target difficulty level and the actual difficulty level.

In the skill evaluation program according to the third aspect of the present invention, the determining means determines that correction by the correcting means is necessary when the absolute value of the difference exceeds the threshold, while the absolute value of the difference exceeds the threshold. is not exceeded, it is determined that correction by the correction means is unnecessary.

In the skill evaluation program according to the fourth aspect of the present invention, the calculating means calculates a plurality of types of individual evaluation values indicating the effectiveness of the statement content during the dialogue or the relevance between the statement content, and calculates the plurality of types of individual evaluation values. The persons to be evaluated are classified according to whether they belong to an area on the evaluation value space formed by the individual evaluation values.

A skill evaluation device according to a fifth aspect of the present invention is a device for evaluating the skills of an evaluation subject through interactive role-playing performed by a plurality of participants including the evaluation subject, and Acquisition means for acquiring the difficulty level of the target regarding the formation of consensus or acquisition of cooperative relationships with the participants of the above, and the contents of remarks during the dialogue obtained through the role play assuming the difficulty level of the target acquired by the acquisition means Estimation means for estimating the actual degree of difficulty in forming said consensus or obtaining said cooperative relationship from said dialogue data indicating the effectiveness of said contents of said dialogue or evaluation for evaluating the relevance between said contents of said dialogue Calculation means for performing processing on the dialogue data to calculate an evaluation value indicating the level of the communication skill of the person to be evaluated; and correction means for correcting the evaluation criteria applied to the evaluation subject or the evaluation value of the evaluation subject so that the difference between the actual difficulty level and the actual difficulty level is reduced.

According to the present invention, it is possible to suppress a decrease in the evaluation accuracy of the communication skills possessed by the person to be evaluated due to variations in action guidelines taken by participants other than the person to be evaluated.

1 is an overall configuration diagram of a simulation training system incorporating a skill evaluation device according to an embodiment of the present invention; FIG. 2 is a diagram showing an example of a hardware configuration of a skill evaluation device in FIG. 1; FIG. 3 is a diagram showing an example of functional blocks of the skill evaluation device in FIG. 2; FIG. 4 is a flowchart relating to the operation of the skill evaluation device shown in FIG. 3; 2 is a diagram showing an example of a setting screen displayed on the user terminal of FIG. 1; FIG. FIG. 2 is a diagram that visualizes the contents of a dialogue indicated by the dialogue data of FIG. 1; It is a figure which shows an example of the determination method regarding the necessity of correction|amendment. It is a figure which shows an example of the evaluation criteria assuming the customer of standard difficulty. FIG. 4 is a schematic diagram showing a first correction method regarding skill evaluation; FIG. 10 is a schematic diagram showing a second correction method regarding skill evaluation;

Embodiments of the present invention will be described below with reference to the accompanying drawings. In order to facilitate understanding of the description, the same constituent elements or the same steps are denoted by the same reference numerals as much as possible in each drawing, and overlapping descriptions are omitted.

[Configuration of simulation training system 10]
<Overall composition>
FIG. 1 is an overall configuration diagram of a simulation training system 10 incorporating a skill evaluation device 14 according to one embodiment of the present invention. This simulation training system 10 is configured to be able to provide a "skill improvement support service" that supports improvement of communication skills through interactive role play (or dialogue simulation).

This simulation training system 10 specifically includes one or more user terminals 12 , a skill evaluation device 14 , and one or more expert terminals 16 . The skill evaluation device 14 can mutually communicate with each user terminal 12 or with each expert terminal 16 through the network NT.

The user terminal 12 is a work terminal used by a user who belongs to an organization such as a company, an organization, or a school to do business. The user terminal 12 is composed of, for example, a personal computer, tablet, smart phone, wearable device, or the like. In the example of this figure, the user terminal 12 is communicably connected to a microphone 18 as an input device for audio data and a camera 20 as an input device for video data.

A microphone 18 and a camera 20 are installed in a place (specifically, a conference room, etc.) for interactive role-playing. This role play assumes a scene that requires high communication skills. Specific examples include [1] business negotiations between sales representatives and customers, [2] meetings between multiple members, [3] evaluation interviews between superiors and subordinates, and [4] ] classes between teachers and students, and [5] explanations of diagnosis results between doctors and patients.

The skill evaluation device 14 is a server computer that exchanges various information (for example, dialogue data DD, report data RD, etc.) necessary for improving communication skills with the user terminal 12 . The skill evaluation device 14 is configured to be able to evaluate the skill of the person to be evaluated through interactive role-playing performed by a plurality of participants including the person to be evaluated. A specific configuration of the skill evaluation device 14 will be described in detail later.

The expert terminal 16 is a work terminal owned by a communication skills expert (that is, a consultant). The expert uses the expert terminal 16 to evaluate communication skills while reproducing the dialogue data DD, and inputs the obtained evaluation results into the expert terminal 16 . Thereafter, the expert terminal 16 transmits data including human evaluation results to the skill evaluation device 14 in response to the transmission operation by the expert.

<Hardware configuration>
FIG. 2 is a diagram showing an example of the hardware configuration of the skill evaluation device 14 in FIG. 1. As shown in FIG. The skill evaluation device 14 includes a communication I/F 22 , a processor 24 , a memory 26 and a storage device 28 .

The communication I/F 22 is an interface that transmits and receives electrical signals to and from an external device. Thereby, the skill evaluation device 14 can exchange necessary information with the user terminal 12 or the expert terminal 16 .

The processor 24 may be a general-purpose processor including a CPU (Central Processing Unit), or may be a dedicated processor including an FPGA (Field Programmable Gate Array) or a GPU (Graphics Processing Unit). The memory 26 is a non-transitory storage medium and stores programs and data necessary for the processor 24 to control each component. The storage device 28 is a non-transitory storage medium including, for example, a hard disk drive (HDD: Hard Disk Drive) and a solid state drive (SSD: Solid State Drive).

<Functional block>
FIG. 3 is a diagram showing an example of functional blocks of the skill evaluation device 14. As shown in FIG. The processor 24 (FIG. 2) of the skill evaluation device 14 executes the skill evaluation program read from the memory 26 (FIG. 2) to obtain the acquisition means 30, the classification means 32, the evaluation means 34, the correction means 36, and the output instruction means. 38 and storage means 40 .

Here, the acquisition means 30 is means realized by the communication I/F 22 and the processor 24 in FIG. Also, the classifying means 32, the evaluating means 34 and the correcting means 36 are means implemented by the processor 24 of FIG. Also, the output instruction means 38 is a means realized by the communication I/F 22 and the processor 24 in FIG. Also, the storage means 40 is a means realized by the storage device 28 in FIG.

Acquisition means 30 acquires from the user terminal 12 data indicating the content of statements made during a dialogue by a plurality of participants (hereinafter referred to as “dialogue data DD”). The dialogue data DD is not limited to audio data or video data, and may be, for example, text data or sign language data. In addition, the acquisition means 30 acquires data including evaluation results (for example, a rubric table) by an expert using the dialogue data DD from the expert terminal 16 .

Acquisition means 30 also acquires role information 52 corresponding to the dialogue data DD together with the dialogue data DD. The role information 52 includes the degree of difficulty of goals regarding building consensus or obtaining cooperative relationships between the person to be evaluated and other participants. The difficulty level may be a continuous value including scores or a discrete value including levels. An example of "building an agreement" is the conclusion of a business negotiation, the conclusion of negotiations, and the like. One example of "acquisition of cooperative relationships" is support for accomplishing tasks.

The classifying means 32 divides the dialogue data DD acquired by the acquiring means 30 into chunks, and then classifies the content of statements during the dialogue for each participant. For example, the classification means 32 may perform frequency analysis processing on voice data and classify utterance contents estimated to have the same voice quality for each participant. Alternatively, the classification means 32 may perform image processing for recognizing mouth movements on the video data, and associate each utterance content with the participant. Note that, when the dialogue data DD includes text data, the classifying means 32 may perform preprocessing such as summarizing the contents of statements using artificial intelligence technology.

The evaluation means 34 evaluates the communication skills of the persons to be evaluated included in the plurality of participants based on the effectiveness of the utterance contents during the dialogue or the relevance between the utterance contents specified by the dialogue data DD acquired by the acquisition means 30. Evaluate the height of Specifically, the evaluation means 34 includes a first calculation part 42 (corresponding to "calculation means"), an estimation part 44 (corresponding to "estimation means"), and a determination part 46 (corresponding to "determination means"). , a second calculation unit 48 (corresponding to “calculation means”), and an integration unit 50 .

The first calculator 42 calculates a plurality of types of evaluation values (hereinafter referred to as “individual evaluation values”) that indicate the effectiveness of utterances during a dialogue or the relevance between utterances. As the evaluation method, various evaluation methods (rule-based type or machine learning type) known in the technical field of natural language processing are used. Individual evaluation values that indicate effectiveness include, for example, the presence or absence and frequency of use of keywords by the person being evaluated, the presence or absence and frequency of appearance of keywords by the other party, chunks ( cluster of words), and the like. As an individual evaluation value indicating relevance, for example, the degree of correlation between the chunks of the person to be evaluated, the degree of correlation between the chunks of the subject himself and the opponent, a specific reasoning pattern (a specific graph showing the relationship between three or more chunks structure) and the frequency of appearance.

The estimating unit 44 estimates the difficulty level (hereinafter also referred to as "actual difficulty level") related to building consensus or obtaining a cooperative relationship from the dialogue data DD obtained through a role-play assuming the target difficulty level acquired by the acquisition means 30. do. This estimation is based on information indicating the degree of positive consideration by other participants, specifically, the presence or absence and number of keywords (for example, product names for sale in the case of business negotiations), positive or This is done by analyzing the ratio of negative remarks, the frequency of remarks, the volume of the voice at the time of remarks, emotional changes, etc. Specifically, the estimation unit 44 determines that [1] the number of remarks about the product is small, [2] the proportion of negative remarks is high, [3] the frequency of remarks is low, or [4] the proportion of angry feelings It is assumed that the higher the number, the higher the difficulty. On the other hand, the estimating unit 44 determines that [1] the number of remarks regarding the product is large, [2] the proportion of positive remarks is high, [3] the frequency of remarks is high, or [4] the proportion of joyful emotions is high. Assume that the difficulty is high.

The determination unit 46 determines whether or not correction by the correction unit 36 is necessary according to the difference between the acquired target difficulty level and the estimated actual difficulty level. 36 is requested for correction. The judging unit 46 judges that correction is necessary when the difference in the difficulty level described above can affect the evaluation result. For example, when the absolute value of the difference exceeds the threshold, the determination unit 46 determines that the correction by the correction unit 36 is necessary, and when the absolute value of the difference does not exceed the threshold, the correction by the correction unit 36 is not necessary. I judge.

The second calculator 48 uses the N individual evaluation values calculated by the first calculator 42 and an evaluation criteria table 56 to be described later to calculate or determine the evaluation rank of the person to be evaluated. For example, the second calculator 48 may calculate the evaluation rank by determining whether the evaluation point 86 (FIGS. 8 to 10) belongs to the area in the N-dimensional evaluation value space. The evaluation rank includes, for example, three levels of high (H), medium (M), and low (L). Alternatively, the number of ranks may be two, or four or more. Note that this calculation may also take into consideration the degree of accomplishment of the task, the degree of achievement of the purpose, and the like.

The integration unit 50 integrates two types of evaluation results respectively obtained by the skill evaluation device 14 and the expert. Specifically, the integration unit 50 [1] connects the two types of evaluation results, [2] averages the two types of evaluation results, or [3] corrects one of the two types of evaluation results. and perform data processing such as In addition, the integration unit 50 generates report information 58 including evaluation results integrated by itself.

The correction means 36 corrects the evaluation criteria applied to the person to be evaluated or the evaluation value of the person to be evaluated according to the degree of difficulty in forming a consensus or obtaining a cooperative relationship between the person to be evaluated and the other participants. For example, the correction means 36 corrects the evaluation criteria so that the higher the difficulty of obtaining consent or the higher the difficulty of obtaining assistance of the follower, the lower the evaluation criteria. On the other hand, the correction means 36 corrects the evaluation value so that the higher the difficulty of obtaining consent or the higher the difficulty of obtaining the support of the follower, the higher the evaluation value. It should be noted that corrections opposite to those described above are performed for cases where the difficulty level is low.

Further, the correction means 36, in response to a request from the determination unit 46, adjusts the evaluation criteria or Correct the evaluation value of This difficulty level difference corresponds to the score difference when the difficulty level is defined by the score, and corresponds to the level difference when the difficulty level is defined by the level. For example, the correcting means 36 corrects such that the higher the actual difficulty, the lower the evaluation criteria or the higher the evaluation value. Conversely, the correcting means 36 corrects the lower the actual difficulty, the higher the evaluation criteria or the lower the evaluation value.

The output instruction unit 38 instructs the user terminal 12 to output the report information 58 generated by the evaluation unit 34 (more specifically, the integration unit 50). For example, the output instruction means 38 transmits report data RD including the report information 58 to the user terminal 12 in response to a request from the service user. Thereby, the user terminal 12 can output the report information 58 included in the report data RD received from the skill evaluation device 14 .

The storage means 40 sequentially stores the dialogue data DD and the evaluation results to construct a database relating to communication skill evaluation (hereinafter referred to as a skill evaluation DB 52). The storage means 40 also stores role information 54 , evaluation criteria table 56 and report information 58 .

The skill evaluation DB 50 stores, for example, [1] user information including user name, affiliation (organization, department, etc.), job title, and responsibility; )/implementation information including evaluation results, [3] dialog data DD, summary of statement content, dialog log information including model correspondence, and the like are accumulated.

The role information 54 includes data relating to role-play roles, specifically, situation settings, roles given to each participant, or difficulty levels. In the case of role-plays such as "business negotiations," the role of the "proposer" who makes a proposal (e.g., a salesperson) or the role of an "acceptor" who decides whether to accept the proposal (e.g., a customer) is set. can be In the case of a role play such as a "meeting", a role of a "leader" who leads the execution of tasks or a role of a "follower" who assists the leader can be set. Also, a combination of difficulty levels (proficiency level, negotiation toughness, etc.) for each role may be provided. For example, “high-difficulty approvers” who are relatively difficult to obtain consent, “veterans” who are highly skilled in their work and have a wealth of specialized knowledge, and “low-difficulty” who are relatively difficult to obtain assistance. 'followers' may be provided.

The evaluation criteria table 56 is tabular data describing evaluation criteria applied to the person to be evaluated. For example, when the evaluation rank is determined according to whether the evaluation points 86 (FIGS. 8 to 10) belong to the area in the N-dimensional evaluation value space, the evaluation criteria table 56 contains various Information (eg, type of function shape, function coefficients, set of plots, etc.) is included.

The report information 58 is information indicating the contents of the report to the service user. The report information 58 includes current evaluation results, history of past evaluation results, advice to evaluation subjects, customized educational contents, URLs (Uniform Resource Locators) indicating the location of educational contents, and the like. . Alternatively, the report information 58 may include a target difficulty level set in advance, an estimated actual difficulty level, a difference between the two difficulty levels described above, and the like.

[Operation of skill evaluation device 14]
The simulation training system 10 in this embodiment is configured as described above. Next, the operation of this simulation training system 10 (more specifically, the skill evaluation device 14) will be described in detail with reference to the flow chart of FIG. 4 and FIGS. 5 to 10. FIG.

At step SP10 in FIG. 4, the acquisition means 30 acquires the dialogue data DD and the role information 54 used for communication skill evaluation. Prior to this acquisition, [1] role-play setting for evaluating the person to be evaluated, and [2] role-play by a plurality of participants are sequentially performed.

FIG. 5 is a diagram showing an example of a setting screen 60 displayed on the user terminal 12 of FIG. 1. As shown in FIG. On the setting screen 60, there are an input field 61 for the name of the roleplay, an input field 62 for setting the situation, setting fields 63 and 64 for each participant, a button 65 labeled [Cancel], and [Save]. A button 66 labeled . Each setting column 63, 64 is provided with an icon 71 indicating whether or not the service is to be evaluated, and user controls 72, 73, 74 for selecting the service user, the role, and the difficulty level of the target, respectively. . The user clicks the [Save] button 66 on the setting screen 60 after completing the setting operation related to the role play. Thereby, the role information 54 is generated.

After the role-play setting is completed, multiple participants perform interactive role-plays according to their assigned roles. Then, an audio signal during the dialogue is supplied to the user terminal 12 through the microphone 18 (FIG. 1) and a video signal during the dialogue is supplied through the camera 20 (FIG. 1). After that, the user terminal 12 transmits the dialogue data DD including the audio data and the video data to the skill evaluation device 14 in a state of being associated with the role information 54 .

FIG. 6 is a diagram visualizing the content of the dialogue indicated by the dialogue data DD of FIG. This dialogue takes place between Participant A, who is a "salesperson", and Participant B, who is a "customer". More specifically, this dialogue shows a scene in which a salesperson sells a product under the setting that ``a customer finds it difficult to purchase the product at this time due to the business situation''.

In step SP12 of FIG. 4, the classifying means 32 divides the dialogue data DD acquired in step SP10 into chunks, specifies the number of participants (here, two), and then the content of the statement of participant A, Participant B's utterance contents are classified respectively.

In step SP14, the evaluation means 34 (more specifically, the first calculator 42) analyzes the statement content classified in step SP12, and evaluates the validity of the statement content during the dialogue or the relevance between the statement content. Calculate individual evaluation values (for example, N≧2).

In step SP16, the evaluation means 34 evaluates the level of the communication skill of the participant A, who is the person to be evaluated, in cooperation with the correction means 36 . The estimation unit 44 estimates the actual difficulty level from the dialogue data DD (step SP16A). The determination unit 46 determines whether correction is necessary according to the target difficulty level, the actual difficulty level, or both of the difficulty levels (step SP16B). The second calculator 48 determines the evaluation rank of the person to be evaluated using the calculated or corrected N individual evaluation values and the evaluation criteria table 56 (step SP16C).

FIG. 7 is a diagram illustrating an example of a determination method regarding whether or not correction is necessary. The axis of the graph indicates difficulty. Here, it is assumed that the standard level of difficulty is 70 (point P0). [Condition 1] When the absolute value of the difference is less than 10 (for example, point P1), the determination unit 46 determines that the actual difficulty level is appropriate and correction is unnecessary. [Condition 2] If the actual difficulty level is 10 or more higher than the target difficulty level (for example, at point P2), the determination unit 46 determines that the actual difficulty level is not appropriate and that correction to lower the difficulty level is necessary. conduct. [Condition 3] If the actual difficulty level is 10 or more lower than the target difficulty level (for example, point P3), the determination unit 46 determines that the actual difficulty level is not appropriate and that a correction to increase the difficulty level is necessary. conduct.

The correction means 36 adjusts the evaluation criteria applied to the evaluation subject or the evaluation value of the evaluation subject so that the difference between the target difficulty level acquired in step SP10 and the actual difficulty level estimated in step SP16A becomes small. is corrected (step SP16D). When correcting the evaluation criteria, steps SP16A→16B→16C→16D are executed in order, while when correcting the evaluation values, steps SP16A→16B→16D→16C are executed in order. Two correction examples will be described below with reference to FIGS. 8 to 10. FIG.

FIG. 8 is a diagram showing an example of evaluation criteria assuming customers with standard difficulty. As shown in this figure, an area forming an N-dimensional evaluation value space 80 is partitioned into three rank areas (L/M/H) by boundary lines 82 and 84 having substantially circular arc shapes. For convenience of illustration, only two components (i, j-th) are shown among the N evaluation value components. Also, each evaluation value component is normalized to the range of [0, 1].

Specifically, the low-rank area (L) and the medium-rank area (M) are separated by a boundary line 82, and the medium-rank area (M) and the high-rank area (H) are separated by a boundary line 84. be. If the evaluation criteria or evaluation values are not corrected, the evaluation rank of the person to be evaluated is determined according to which rank region the score of the person to be evaluated (evaluation points 86 shown in FIG. 8) belongs to. In the example of this figure, the evaluation rank of the person to be evaluated corresponds to "middle rank".

FIG. 9 is a schematic diagram showing a first correction method regarding skill evaluation. The dashed lines in the figure correspond to the evaluation criteria assuming customers of the standard difficulty level, that is, the positions of the boundary lines 82 and 84 of "before correction". On the other hand, the solid lines in the figure correspond to the evaluation criteria assuming customers with a high degree of difficulty, that is, the positions of the boundary lines 82 and 84 "after correction". In this way, by expanding the high-rank area (H) and reducing the low-rank area (L), when dealing with customers with a high level of difficulty, sales negotiations will be difficult and the evaluation value of communication skills will be relatively low. It is easy to reflect the trend of decreasing

FIG. 10 is a schematic diagram showing a second correction method for skill evaluation. The unfilled circle (o) in the figure corresponds to the evaluation value assumed for the standard difficulty customer, that is, the position of the evaluation point 86 "before correction". On the other hand, the filled circles (●) in the figure correspond to evaluation values assumed for customers with a high level of difficulty, that is, the positions of evaluation points 86 after correction. By increasing the evaluation value in this way, when dealing with a customer with a high degree of difficulty, sales negotiations tend to be difficult and the communication skill evaluation value tends to be relatively low.

At step SP18 in FIG. 4, the evaluation means 34 supplies the evaluation result obtained at step SP16 to the storage means 40. FIG. As a result, the communication skill evaluation result is accumulated in the skill evaluation DB 52 in a state of being associated with the identification information (evaluator ID) of the participant A who is the person to be evaluated.

In step SP20, the evaluation means 34 confirms whether or not the acquisition means 30 has acquired the expert's evaluation result. If the evaluation result has not yet been obtained (step SP20: NO), the process remains at step SP20 until the evaluation result is obtained. On the other hand, when the evaluation result is acquired (step SP20: YES), the process proceeds to the next step SP22.

In step SP22, the evaluation means 34 (more specifically, the integration unit 50) integrates the evaluation results obtained in steps SP16 and SP20, and then generates report information 58 indicating the content of the report to service users including participant A. to generate

At step SP24, the output instruction means 38 instructs the user terminal 12 to output the report information 58 including the evaluation results integrated at step SP22. Then, the user terminal 12 outputs report information 58 included in the report data RD received from the skill evaluation device 14 . Participant A can grasp the level of his own communication skill through the output report information 58 . Also, by accessing the URL displayed on the report screen, the participant A can view the educational content customized according to the evaluation result. Alternatively, if the report information 58 includes information about the difficulty level, the participant B can grasp whether or not the role play has been performed according to the given difficulty level through the output report information 58 .

Thus, a series of operations (flowchart in FIG. 4) by the skill evaluation device 14 are completed. Service users, including Participant A, can improve their communication skills by repeating and continuing role-plays and evaluation feedback using simulation training system 10 (FIG. 1).

[Effect of skill evaluation device 14]
As described above, the skill evaluation device 14 in this embodiment is a computer that evaluates the skills of the person to be evaluated through interactive role-plays performed by a plurality of participants including the person to be evaluated. The skill evaluation device 14 assumes an acquisition means 30 for acquiring a target difficulty level for building consensus or obtaining a cooperative relationship between an evaluation subject and other participants, and the target difficulty level acquired by the acquisition means 30. Estimating means (here, an estimating unit 44) for estimating the actual degree of difficulty in forming a consensus or obtaining a cooperative relationship from dialogue data DD indicating the content of statements during the dialogue obtained through role play; Calculation means (here, first calculation 42 or the second calculation unit 48), and the evaluation applied to the person to be evaluated so that the difference between the target difficulty level acquired by the acquisition means 30 and the actual difficulty level estimated by the estimation unit 44 becomes small. A correction means 36 is provided for correcting the reference or the evaluation value of the person to be evaluated.

In addition, in the skill evaluation program and method in this embodiment, the computer (here, the skill evaluation device 14) obtains a target difficulty level for building consensus or obtaining a cooperative relationship (SP10 in FIG. 4); An estimation step (SP16A) of estimating the actual difficulty of building consensus or obtaining a cooperative relationship, a calculation step (SP14, SP16C) of calculating an evaluation value indicating the level of the communication skill of the person to be evaluated, and a target difficulty level. A correction step (SP16D) of correcting the evaluation criteria applied to the person to be evaluated or the evaluation value of the person to be evaluated is executed so that the difference between the actual difficulty level and the actual difficulty level becomes smaller.

In this way, by correcting the evaluation criteria applied to the evaluation subject or the evaluation value of the evaluation subject so that the difference between the target difficulty level and the actual difficulty level is small, Even if the course of action of another participant fluctuates due to a difference in interpretation of the role assigned to the participant, this fluctuation can be properly reflected in the evaluation results. This suppresses deterioration in the evaluation accuracy of the communication skills possessed by the person to be evaluated due to variations in action guidelines taken by participants other than the person to be evaluated.

Further, the skill evaluation device 14 may further include a determination unit 46 that determines whether correction by the correction unit 36 is necessary according to the difference between the target difficulty level and the actual difficulty level. In particular, the determination unit 46 determines that correction by the correcting means 36 is necessary when the absolute value of the difference exceeds the threshold, while correction by the correcting means 36 is not necessary when the absolute value of the difference does not exceed the threshold. can be determined.

In addition, the first calculation unit 42 and the second calculation unit 48 calculate a plurality of types of individual evaluation values that indicate the effectiveness of utterances during a dialogue or the relevance between utterances. Evaluation subjects may be classified according to whether they belong to a region on the evaluation value space 80 .

Also, the role of the person to be evaluated may be the proposer who makes a proposal, and the role of another participant may be the approver who determines whether or not to approve the proposal. In this case, the correction means 36 corrects the evaluation criteria so that the higher the difficulty of obtaining consent, the lower the evaluation criteria, or the higher the evaluation value, the higher the difficulty of obtaining consent. Correct it so that This reduces the variability of evaluation results in role-playing with suggestions.

Also, the role of the person to be evaluated may be a leader who leads the performance of the task, and the role of another participant may be a follower who assists the leader. In this case, the correction means 36 corrects the evaluation criteria so that the more difficult the follower is to obtain assistance, the lower the evaluation criteria, or the more difficult the follower is to obtain assistance, the higher the evaluation value. corrected to This reduces variations in evaluation results in role-plays involving task performance.

[Modification]
It goes without saying that the present invention is not limited to the above-described embodiments, and can be freely modified without departing from the gist of the present invention. Alternatively, each configuration may be arbitrarily combined as long as there is no technical contradiction. Alternatively, the execution order of each step constituting the flowchart of FIG. 4 may be changed within a technically consistent range.

In the above-described embodiment, the skill evaluation device 14 of FIG. 1 is illustrated as a single computer, but instead of this, the skill evaluation device 14 may be a group of computers constructing a distributed system. Also, the skill evaluation device 14 may be either a cloud server or an on-premise server.

In the above-described embodiment, the case where the acquisition means 30 in FIG. 3 acquires the dialogue data DD generated through role play has been described, but the opportunity to acquire the dialogue data DD is not limited to this. For example, the obtaining means 30 may obtain the dialogue data DD generated through the actual dialogue on the premise that permission has been obtained from another participant.

In the above embodiment, the case where the skill evaluation device 14 includes the classification means 32 (FIG. 3) has been described, but instead of this, the skill evaluation device 14 may not be provided with the classification means 32 . In this case, the user terminal 12 or the expert terminal 16 may supply the manually classified dialogue data DD to the skill evaluation device 14 .

DESCRIPTION OF SYMBOLS 10... Simulation training system, 14... Skill evaluation apparatus, 16... Microphone, 18... Camera, 30... Acquisition means, 32... Classification means, 36... Correction means, 38... Output instruction means, 40... Storage means, 42... First Calculation unit (calculation means) 44 Estimation unit (estimation means) 46 Judgment unit (judgment means) 48 Second calculation unit (calculation means) 54 Role information DD Dialogue data

Claims (5)

A skill evaluation program for evaluating the skills of an evaluation subject through interactive role-playing performed by a plurality of participants including the evaluation subject,
the computer,
Acquisition means for acquiring the degree of difficulty of the target regarding the formation of consensus or the acquisition of cooperative relationships between the subject of evaluation and other participants;
Estimating means for estimating the actual degree of difficulty in forming the consensus or obtaining the cooperative relationship from the dialogue data indicating the content of statements during the dialogue obtained through the role play assuming the degree of difficulty of the target obtained by the obtaining means;
Evaluation processing for evaluating the effectiveness of utterances during the dialogue or the relevance between the utterances is performed on the dialogue data, and an evaluation value indicating the level of communication skill of the person to be evaluated is calculated. calculating means;
Evaluation criteria applied to the evaluation subject or evaluation of the evaluation subject so that the difference between the difficulty level of the target acquired by the acquisition means and the actual difficulty level estimated by the estimation means is small. correction means for correcting values;
A skill evaluation program that functions as a the computer,
Further functioning as determination means for determining whether or not correction by the correction means is necessary according to the difference between the target difficulty level and the actual difficulty level;
A skill evaluation program according to claim 1. The determination means determines that the correction by the correction means is necessary when the absolute value of the difference exceeds the threshold, and the correction by the correction means is necessary when the absolute value of the difference does not exceed the threshold. determine not
3. A skill evaluation program according to claim 2. The calculating means calculates a plurality of types of individual evaluation values that indicate the effectiveness of the statement content during the dialogue or the relevance between the statement content, and calculates a region on the evaluation value space formed by the plurality of types of individual evaluation values. Classifying the evaluation subject according to belonging or not;
A skill evaluation program according to any one of claims 1 to 3. A skill evaluation device that evaluates the skills of an evaluation subject through interactive role-playing performed by a plurality of participants including the evaluation subject,
Acquisition means for acquiring the degree of difficulty of a target regarding the formation of consensus or the acquisition of a cooperative relationship between the person to be evaluated and other participants;
an estimating means for estimating an actual degree of difficulty in forming the consensus or obtaining the cooperative relationship from dialogue data indicating content of statements during the dialogue obtained through the role play assuming the degree of difficulty of the target acquired by the acquiring means; ,
Evaluation processing for evaluating the effectiveness of utterances during the dialogue or the relevance between the utterances is performed on the dialogue data, and an evaluation value indicating the level of communication skill of the person to be evaluated is calculated. calculating means;
Evaluation criteria applied to the evaluation subject or evaluation of the evaluation subject so that the difference between the difficulty level of the target acquired by the acquisition means and the actual difficulty level estimated by the estimation means is small. a correction means for correcting the value;
A skill evaluation device.

Images