JP6285377B2 - Communication skill evaluation feedback device, communication skill evaluation feedback method, and communication skill evaluation feedback program - Google Patents

Description

  The present invention relates to a technique for feeding back an evaluation result evaluated during an exercise when training communication skills.

  Skills for smooth communication between people are called communication skills. Communication skills can be improved by training and can be acquired by teaching skills, practicing, feedback, and generalizing. Conventionally, skill measurement and feedback have been performed by experts or students of skill training, but in recent years, it has been required to perform the measurement with a computer.

  Therefore, Non-Patent Document 1 describes a technique for improving communication skills during a job interview. In this technology, an image including sound is acquired, the utterance speed of the interviewee, the presence / absence of a smile, etc. are measured, and the measurement result is fed back. As a result, the interviewee can grasp and review his / her own behavior during the interview.

  Non-Patent Document 2 describes a technique for improving presentation skills, although it is slightly different from communication skills. In this technique, a presentation practice exercise is recorded and displayed on a WEB screen. Thereby, the skills can be mutually evaluated.

M. Hoque, 4 others, “MACH: My Automated Conversation coacH”, Proc. UbiComp’13, September 2013, p.697-p.706 Harada and 2 others, “Web-based presentation and skill-up support system -Learning support by mutual evaluation between users and automatic evaluation of spoken speech-", IEICE Technical Report, Vol.111, No.473, March 2012, p.41-p.46

  However, since the conventional method only presents the remarks of the skill-measured person and the measurement results and evaluation results, specific improvement methods such as what should be done to improve one's own behavior. I can't figure out. Further, when presenting a negative evaluation result, only presenting the evaluation result may reduce self-esteem and self-confidence, which may adversely affect the skill-measured person.

  The present invention has been made in view of the above circumstances, and an object thereof is to effectively improve human communication skills.

  In order to solve the above problems, the communication skill evaluation feedback apparatus according to claim 1 is configured to analyze a plurality of features for each utterance from imaging data obtained by photographing a subject in communication, and to determine the degree of the features. Clustering means for clustering feature vectors based on each utterance, and selecting a center vector that is most similar to a feature vector related to negatively evaluated utterances by calculating the center vector of each cluster for a plurality of positively evaluated utterances. And a means for calculating the difference between the degree of each feature of the utterance corresponding to the selected center vector and the degree of each feature of the negatively evaluated utterance, and extracting the feature having the largest difference, Output means for outputting a feature as an improvement factor for the negatively evaluated utterance; And it is required to.

  The communication skill evaluation feedback device according to claim 2, wherein the analysis means includes a language emitted by the target person, a feature of a voice emitted by the target person, The gist is to analyze the characteristics of the actions performed by the subject.

  The communication skill evaluation feedback method according to claim 3, wherein the communication skill evaluation feedback method is a communication skill evaluation feedback method performed by a communication skill evaluation feedback device, wherein an analysis step is performed for analyzing a plurality of features for each utterance from imaging data obtained by imaging a subject in communication. A clustering step for clustering feature vectors based on the degree of feature for each utterance, and calculating a center vector of each cluster for a plurality of positively evaluated utterances, and most similar to a feature vector for a negatively evaluated utterance A selection step of selecting a center vector, and calculating a difference between the degree of each feature of the utterance corresponding to the selected center vector and the degree of each feature of the negatively evaluated utterance, and extracting the feature having the largest difference An extraction step, an output step of extracted features is output as an improvement element relative to the negative evaluated remarks, and summarized in that with.

  The gist of a communication skill evaluation feedback program according to claim 4 is to cause a computer to function as the communication skill evaluation feedback device according to claim 1 or 2.

  According to the present invention, human communication skills can be effectively improved.

It is a figure which shows the whole structure of a communication skill evaluation feedback system. It is a figure which shows the functional block structure of an analyzer. It is a figure which shows the analysis processing flow of language information. It is a figure which shows the evaluation analysis processing flow of a communication skill. It is a figure which shows the example of the word for documents.

  The present invention provides a feedback that is equivalent to a positive evaluation even for a user who is difficult to accept a negative evaluation by presenting a specific improvement method in an easy-to-understand manner when a statement or action is a negative evaluation. The effect is to give. Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram showing an overall configuration of a communication skill evaluation feedback system according to the present embodiment. The system measures the remarks and behaviors of students who are training communication skills, and presents the difference between remarks and behaviors with positive evaluations for remarks and behaviors with negative evaluations. . Hereinafter, a configuration example of this system will be described in detail.

  The system includes a measurement terminal 2 that measures a student's remarks and actions, an evaluation terminal 3 that inputs evaluations of the remarks and actions of the students, and an improvement on the remarks and actions of the students based on the input evaluations. An analysis apparatus 1 that analyzes the method and a browsing terminal 4 that feeds back the analyzed improvement method to the student are configured. All these terminals and devices are connected via a communication network N such as the Internet or an intranet. The terminal and the device can communicate with each other, regardless of whether they are wireless or wired.

  The measurement terminal 2 is a terminal for measuring the speech and behavior of the student. The student's remarks and actions expressed at the time of attendance are photographed, and the imaging data (video / audio data) is transmitted to the analysis device 1. For example, a video camera, an eye tracker, or an acceleration sensor is used.

  The evaluation terminal 3 is a terminal for a third party to evaluate a student's remarks and actions. Based on the audio data measured by the measuring terminal 2, the student's remarks are converted into text and displayed together with the video data on the screen. Then, for the remarks and actions of the displayed student, the input of evaluation values for each index such as explanatory power and understanding power is requested, and the input evaluation data is transmitted to the analysis device 1. For example, a personal computer or a tablet terminal is used.

  The browsing terminal 4 is a terminal for outputting the improvement method analyzed by the analysis apparatus 1 with respect to a student's remarks and actions by a screen or voice. Used by trained students for the purpose of providing feedback. For example, a tablet terminal is used.

  Next, the analysis apparatus 1 will be described. The analysis device 1 is a device that analyzes the improvement method of the communication skills of the student using the video / audio data from the measurement terminal 2 and the evaluation data from the evaluation terminal 3.

  A functional block configuration of the analysis apparatus 1 is shown in FIG. The analysis device 1 analyzes a student's speech and behavior from video / audio data, classifies each speech into a positive evaluation and a negative evaluation based on the evaluation data, and an improvement method for negative evaluation speech The analysis unit 11 that analyzes the evaluation item and the evaluation items for evaluating the remarks and actions of the student are transmitted to the evaluation terminal 3, and the evaluation data input by the evaluation terminal 3 and the improvement method analyzed by the analysis unit 11 And an input / output unit 12 for inputting and outputting. Hereinafter, the function of each part will be described in detail.

  The analysis unit 11 uses the video / audio data from the measurement terminal 2 to analyze the language information analysis unit 111 that analyzes language information, the meta language information analysis unit 112 that analyzes meta language information, and the non-language information. A non-linguistic information analysis unit 113, an evaluation analysis unit 114 that analyzes an improvement method for a student's remarks and behavior using the analysis result and the evaluation data from the evaluation terminal 3, and a sentence that describes the improvement method is generated And a sentence word DB 115 for storing words to be used.

  The input / output unit 12 outputs the evaluation information including the remarks and actions of the students and the evaluation items for inputting the evaluations to the evaluation terminal 3 and the evaluation data from the evaluation terminal 3. The evaluation result receiving unit 122 to receive and the analysis result output unit 123 to output the improvement method analyzed by the analysis unit 11 to the browsing terminal 4 are configured.

  Subsequently, an analysis process performed by the analysis apparatus 1 will be described.

  First, language information analysis processing performed by the language information analysis unit 111 will be described with reference to FIG. The language information analysis unit 111 executes a process of extracting a speech section from speech data measured by the measurement terminal 2 and calculating a speech start time and a speech end time, and a process of estimating a speech word by speech recognition. . An utterance refers to a voice uttered by a language, and an utterance refers to an utterance continuously uttered by the same student.

  First, in step S101, a time zone of a sound uttered by a person is extracted from speech data as an utterance section. For example, a threshold is set in advance for the sound level, and if the sound exceeds the threshold, the speech section is set. Other extraction methods may be used.

  Next, in step S102, the utterance start time and utterance end time of each extracted utterance section are calculated. For example, the start time of the input voice data is set to 0, the start time of the extracted speech section is calculated as “0: 00: 03.23”, and the end time is calculated as “0: 00: 13.55”. “” Indicates “hour: minute: second.millisecond”.

  Next, in step S103, speech recognition is performed on the speech data of the utterance section, and the utterance word uttered in each utterance section is acquired. For speech recognition, existing speech recognition tools and speech recognition methods are used. Note that the acquired utterance word is stored in a storage area that is not temporarily specified in order to be sent to a later processing unit.

  Next, in step S104, the time between two utterance intervals that are temporally adjacent to each other is obtained and compared with a threshold value th that is set in advance. Specifically, the difference between the utterance start time of the currently processed utterance section and the utterance end time of the previous utterance section is obtained as the inter-utterance time, except for the first utterance section in time series, and the threshold th Compare

  Next, when the inter-speech time is smaller than the threshold th, in step S105, it is determined whether or not the speaker in the utterance section currently being processed is the same as the speaker in the previous utterance section. For example, video data at the utterance time of each utterance section is analyzed, and a student whose mouth is moving is identified as a speaker of the utterance. Then, the determination is made based on whether both speakers identified in the two utterance intervals are the same. As a speaker identification method, other identification methods such as a method of collating voiceprints may be used.

  Next, when the speaker is the same as the speaker in the previous utterance section, in step S106, the utterance in the currently processed utterance section and the utterance in the previous utterance section are set as one utterance.

  On the other hand, if the inter-speech time is greater than the threshold th or if it is different from the speaker in the previous utterance section, the utterance in the utterance section currently being processed and the utterance in the previous utterance section are determined in step S107. Is a different statement.

  Finally, in step S108, a speech start time and a speech end time are calculated for the speech set in step S106 or step S107.

  Thereafter, steps S102 to S108 are repeated for all utterance sections. Thereafter, the utterance word of each utterance and the utterance start time and utterance end time are transmitted as language information to the metalanguage information analysis unit 112 and the evaluation analysis unit 114.

  Next, a meta language analysis process performed by the meta language information analysis unit 112 will be described. The meta-language information analysis unit 112 analyzes voice features other than the utterance contents from the voice data measured by the measurement terminal 2 and the voice recognition results (uttered words of each utterance) analyzed by the language information analysis unit 111. Extract and send the feature amount as meta language information to the evaluation analysis unit 114 including time information. For example, utterance speed, tone, and volume are analyzed and extracted. Other features may be used. Note that existing extraction tools and extraction methods are used for the voice feature extraction means / extraction method other than the speech content.

  Next, non-language analysis processing performed by the non-language information analysis unit 113 will be described. The non-linguistic information analysis unit 113 analyzes and extracts features other than the language at the time of utterance from the video data measured by the measurement terminal 2, and includes the time information as non-linguistic information in the evaluation analysis unit 114. send. For example, the line of sight, head movement, gesture, and facial expression are analyzed and extracted. Other features may be used. Note that existing extraction tools and extraction methods are used for extraction means / extraction methods of features other than language.

  Next, the evaluation analysis process performed by the evaluation analysis unit 114 will be described with reference to FIG. The evaluation analysis unit 114 vectorizes the analyzed linguistic information, meta linguistic information, and non-linguistic information for each utterance, and positively evaluates each utterance based on the evaluator's evaluation result for the utterance (hereinafter, a positive evaluation utterance vector). And negative evaluation (hereinafter referred to as negative evaluation utterance vector). Then, the negative evaluation message vector is compared with the positive evaluation message vector, and a feature amount for presenting a correction plan of the negative evaluation message is extracted.

  First, in step S201, language information, metalanguage information, and non-language information included in the speech time are acquired for each speech. For example, if the first speech time is from “0: 00: 03.23” to “0: 00: 21.52”, the speech recognition result, speech rate, tone, volume, target to watch, whisper, neck swing, Get hand movements and facial expressions.

  Next, in step S202, each acquired feature is digitized. Regarding the numerical method, if the feature amount can be added, the total number is used. If the value fluctuates with time, the average value within the speaking time is used. If the measurement data can be classified into any one within the speaking time, the ratio is within the speaking time.

  For example, for the speech recognition result, the number of independent words in the utterance is counted, and the ratio of the difficulty level {represented in n (n: natural number) steps) of each independent word is calculated. The difficulty level of the spoken word is calculated using existing technology. For example, the difficulty level is determined according to the Japanese Language Proficiency Test questions.

  For the speaking rate, tone, and volume, the average value in the speech is calculated. For the gaze target, the ratio of each gaze target in the statement is calculated. At that time, adding the respective ratios, the total is set to 1. For whispering, neck swinging, and hand movements, count the number of times in each statement. For facial expressions, the ratio of each facial expression in the statement is calculated. At that time, adding the respective ratios, the total is set to 1.

  Next, in step S203, each digitized feature quantity is expressed as a multidimensional vector. For example, [number of words, word difficulty level (1), word difficulty level (2),..., Word difficulty level (n), speech rate, tone, volume, gaze target (a), gaze target (b),. As a gaze target (others), whispering, swinging neck, hand movement, facial expression (laugh), facial expression (anger), facial expression (sad) ..., facial expression (nothing)] Express.

  Steps S201 to S203 are performed for all statements. Then, after vectorization is performed on all utterances, each feature is normalized in step S204. This normalization is normalization for performing multivariate analysis and refers to conversion such that the average of each feature amount is 0 and the variance is 1.

  Next, in step S205, the evaluation data transmitted from the evaluation result receiving unit 122 is referred to, and the multidimensional vectors of each utterance evaluated as positive are clustered for each utterance.

  A positive evaluation refers to an evaluation that is higher than the median value in the evaluation data. For example, if a two-level evaluation (good, bad), good is a positive evaluation. A five-point rating (very good, good, neither, bad, very bad) gives a very good or good but positive rating. For clustering, a hard clustering method is used to classify vectors without cluster overlap. For example, k-means is used. Other clustering methods may be used.

  Next, in step S206, the center vector of each cluster is calculated, and this is used as a positive speech pattern. Thus, a central vector is individually defined for all utterances evaluated as positive.

  The subsequent processing is executed for all negative evaluation message vectors in order to analyze an improvement method for negative evaluation messages.

  Next, in step S207, the negative evaluation message vector is compared with a plurality of center vectors, and the center vector (= positive statement pattern) closest to the negative evaluation message vector is selected.

  For example, the similarity between the negative evaluation message vector and each center vector is calculated, and the center vector having the highest similarity is extracted. As a method of calculating the similarity, for example, a method of obtaining an angle formed by two vectors and increasing the similarity as the angle formed is smaller can be considered. In addition, there is a method of obtaining the distance between two vectors and increasing the similarity as the distance is smaller. A calculation method other than these may be used. Note that a negative evaluation message vector may be obtained by clustering multidimensional vectors in the same manner as in step S205.

  Next, in step S208, a difference between each feature quantity of the negative evaluation speech vector and each feature quantity of the center vector having the highest similarity is obtained, and the feature quantity having the largest absolute value of the difference is corrected. Extract as a power element.

  Next, in step S209, based on the extracted correction element, a text object that is a specific action correction plan to be presented to the student is generated. The text that becomes the amendment proposal is called an advice text.

  Specifically, an advice sentence format such as “It is better to YY XX” is prepared in advance, and the word XX is selected from the sentence word DB 115 of FIG. 5 based on the correction element. For example, if the correction factor is the speaking speed, the word “speaking speed” is selected as XX.

  Finally, in step S210, the verb of the advice sentence is generated based on the sign of the difference obtained in step S208 (plus or minus). Specifically, the YY word in the advice sentence format is selected from the sentence word DB 115 by the ± sign. For example, if the correction factor is the speech rate and the plus or minus sign is plus, the word “slow” is selected as YY.

  Steps S207 to S210 are performed on all negative evaluation speech vectors. Thereafter, each analysis result of language, meta, and non-language, an evaluation result of the evaluator (= evaluation data), and an advice sentence for a negative evaluation statement are transmitted to the analysis result output unit 123.

  Next, evaluation information output processing performed by the evaluation information output unit 121 will be described. In the evaluation information output unit 121, when the evaluator inputs an evaluation value, in order to make it easy to understand which utterance is the evaluation, the evaluation data output unit 121 is used to evaluate the utterance data and the utterance data in which the student's utterance is converted to text Send evaluation items. These two data are transmitted to the evaluation terminal 3 every time one statement ends (= every time step S107 is executed), and the evaluator is prompted to input an evaluation value for each statement.

  Note that the utterance data is acquired by performing the same processing as the processing performed from step S101 to step S103. Evaluation items are input to the analysis apparatus 1 in advance. An evaluation item is an item for an evaluator to evaluate a conversation. For example, subjective or abstract evaluation items such as “Is the content easy to understand” or “Is my intention communicated to the other party”? Use.

  Next, an evaluation result reception process performed by the evaluation result receiving unit 122 will be described. The evaluation result receiving unit 122 receives the evaluation data input from the evaluation terminal 3 by the evaluator and transmits it to the evaluation analysis unit 114. When transmitting to the evaluation analysis unit 114, the time information of the speech start time and the speech end time is added and transmitted so that it can be understood which speech is the evaluation.

  Next, analysis result output processing performed by the analysis result output unit 123 will be described. In the analysis result output unit 123, all the measurement data (language information, meta language information, non-linguistic information) transmitted from the evaluation analysis unit 114, the evaluation result of the evaluator for the measurement data, and advice for negative evaluation remarks Display data for presenting a sentence is generated.

  The display data may be in any format as long as it can be viewed on the browsing terminal 4. For example, there is an HTML file format that can be viewed with a Web browser. The measurement data is displayed as a graph such as a bar graph or a line graph so that it can be understood that it is data for each statement. The evaluation result may be displayed superimposed on the graph, or may be displayed when data is selected (for example, an interaction operation that focuses on one data such as mouse over). The display is such that it can be seen whether the evaluation of the data indicated by each measurement value is positive or negative.

  The advice sentence may be displayed when the student selects negative evaluation data, or may be displayed in advance near the negative evaluation data. At that time, the display is made so that it can be understood which data is the advice sentence.

  According to the present embodiment, since the improvement method is specifically presented for each utterance of the student who has received a negative evaluation, a positive evaluation can be performed even for a user who is difficult to accept a negative evaluation. Equivalent feedback effect can be given, and the communication skills of students can be effectively improved.

  Finally, the effect of the present invention will be described. In the prior art, the remarks and actions during communication are measured by a computer and the measured values are presented or the quality of communication is simply presented. On the other hand, the present invention relates the evaluation performed by a person with the utterances and behaviors analyzed by a computer, and the behavior obtained with a positive evaluation for the behavior obtained with a negative evaluation. Present how different the features are. In other words, since a specific improvement method is presented by the advice sentence automatically generated from the evaluation result, it is possible to obtain the same communication skill improvement effect as a positive evaluation even for students who are difficult to accept a negative evaluation. it can.

  Finally, the analysis apparatus 1 described in the present embodiment can be realized by a computer having a calculation function such as a CPU and a storage function such as a memory. It is also possible to create a program for causing a computer to function as the analysis apparatus 1 and a storage medium for the program.

1 ... Analysis device (communication skill evaluation feedback device)
DESCRIPTION OF SYMBOLS 11 ... Analysis part 111 ... Language information analysis part 112 ... Meta language information analysis part 113 ... Non-language information analysis part 114 ... Evaluation analysis part 115 ... Word DB for sentences
DESCRIPTION OF SYMBOLS 12 ... Input / output part 121 ... Evaluation information output part 122 ... Evaluation result receiving part 123 ... Analysis result output part 2 ... Measurement terminal 3 ... Evaluation terminal 4 ... Viewing terminal S101-S108, S201-S210 ... Step

Claims (4)

An analysis means for analyzing a plurality of features for each utterance from imaging data obtained by photographing a subject in communication;
Clustering means for clustering feature vectors based on the degree of features for each utterance;
Selecting means for calculating a center vector of each cluster related to a plurality of positively evaluated utterances, and selecting a center vector most similar to a feature vector related to a negatively evaluated utterance;
Extraction means for calculating the difference between the degree of each feature of the speech corresponding to the selected center vector and the degree of each feature of the negatively evaluated speech, and extracting the feature having the largest difference;
Output means for outputting the extracted feature as an improvement factor for the negatively evaluated utterance;
A communication skill evaluation feedback device characterized by comprising: The analysis means includes
The communication skill evaluation feedback apparatus according to claim 1, wherein a language uttered by the target person, a feature of a voice uttered by the target person, and a feature of an action performed by the target person are analyzed. In the communication skill evaluation feedback method performed by the communication skill evaluation feedback device,
An analysis step for analyzing a plurality of features for each utterance from imaging data obtained by photographing a subject in communication;
A clustering step of clustering feature vectors based on the degree of features for each utterance;
A selection step of calculating a center vector of each cluster related to a plurality of positively evaluated utterances, and selecting a center vector most similar to a vector of features related to negatively evaluated utterances;
An extraction step of calculating the difference between the degree of each feature of the utterance corresponding to the selected center vector and the degree of each feature of the negatively evaluated utterance, and extracting the feature having the largest difference;
An output step of outputting the extracted feature as an improvement factor for the negatively evaluated utterance;
A communication skill evaluation feedback method characterized by comprising:   A communication skill evaluation feedback program for causing a computer to function as the communication skill evaluation feedback device according to claim 1.

Images