JP6110283B2 - Empathy-reactive spot detection device, empathy-reactive spot detection method, and program - Google Patents

Description

  The present invention relates to a technique for detecting a change in an emotional state such as empathy or affirmation possessed by a conversation person in a conversation.

  2. Description of the Related Art Conventionally, a technology for detecting an emotional state such as empathy or anti-feeling of a talker in a voice conversation using a telephone or the like has been used. For example, Patent Document 1 describes an invention that automatically scores an operator's response to a customer in a call center and reduces the burden of operator education.

  In the invention described in Patent Document 1, the emotional state is specifically detected as follows. First, a voice feature amount is detected from the inputted customer voice signal. Next, an emotion sequence for each call is generated by taking a time-series matching between an emotion model set obtained by modeling each of a plurality of predefined emotions and a voice feature amount. Next, each emotion in the emotion series is converted into an emotion score by using an emotion score list in which the emotion score is associated with each of a plurality of emotions. Then, based on the emotion score series, a value obtained by subtracting the emotion score at the start of the response from the emotion score at the end of the response, the average value of the emotion scores in one call, or the like is calculated as the response score.

JP 2007-286377 A

  The conventional technique described in Patent Document 1 models customer emotions in advance and then digitizes customer emotions generated by dialogue with the operator at the call center, and evaluates the response of the operator based on the numerical values. This is an automatic technology. Although this conventional technique can be expected to have advantages such as cost reduction such as operator training and improvement of work efficiency, it cannot be used for the purpose of directly improving sales. For example, if a location where the customer's emotional state has changed in the dialogue between the customer and the operator can be detected, the improvement point of the service or product can be estimated by grasping the content of the dialogue at that location.

  An object of the present invention is to detect a location where the emotional state of a conversation person has changed.

  In order to solve the above-described problem, the sympathetic reaction site detection apparatus of the present invention includes an emotion model storage unit, a dialog voice analysis unit, an emotion score calculation unit, a sympathy reaction site estimation score calculation unit, and a sympathy reaction site estimation unit. The emotion model storage unit stores an emotion model set obtained by modeling a plurality of predefined emotion scores. The dialog voice analysis unit extracts an acoustic feature amount for each frame from the input voice signal. The emotion score calculation unit calculates an emotion score for each frame using the emotion model set from the acoustic feature amount. The sympathetic reaction site estimation score calculation unit calculates a sympathy reaction site estimation score for each frame based on the emotion score. The sympathetic reaction part estimation unit estimates a sympathy reaction part that is a frame in which the emotional state has changed based on the sympathy reaction part estimation score.

  According to the sympathetic reaction spot detection technique of the present invention, it is possible to detect a place where the emotional state of the talker has changed. Moreover, if this technology is applied, it is possible to grasp the content of the dialogue that has caused the emotional state change to the interlocutor.

FIG. 1 is a diagram for explaining the relationship between an emotion score, a sympathetic reaction location estimation score, and a sympathy response location. FIG. 2 is a diagram illustrating a functional configuration of the sympathetic reaction site detection apparatus. FIG. 3 is a diagram illustrating a processing flow of the sympathetic reaction site detection method.

  Hereinafter, embodiments of the present invention will be described in detail. In addition, the same number is attached | subjected to the component which has the same function in drawing, and duplication description is abbreviate | omitted.

[Points of Invention]
In this invention, after estimating the time change of the customer's emotion from information (for example, voice information, dialogue information, etc.) that can be acquired in a usage scene such as a call center, the operator utters which operator utterance by paying attention to the time-series change pattern. Estimate whether was sympathetic or dissatisfied.

  In the present invention, for example, in the state of a call center or the like, the emotional state of the customer is estimated using conventional technology from voice information spoken by the customer or dialogue information between the customer and the operator. The emotion types range from “Positive (for example, pleasure, joy, enjoyment, etc.)” to “Negative (for example, discomfort, anger, sadness, etc.)”, and are calculated as a predetermined emotion score. For example, it is assumed that an integer value from 5 to -5 is provided as a score. In this case, if the output score is 5, the emotional state is surely “Positive”. If the output score is 0, it is an ambiguous emotional state in which it is impossible to determine whether it is “Positive” or “Negative”. If the output score is −5, the emotional state is definitely “Negative”.

  A time series change pattern can be obtained by shifting the calculation window in the feature quantity calculation. Furthermore, a time-series change pattern of emotional states can be obtained by calculating an emotional score from each feature quantity. Using the time-series change pattern of the customer's emotional state, it is possible to estimate from the location where the emotional state has changed which part of the operator utterance is sympathetic or disgusting. For example, operator utterances before and after the estimated emotion score changes from 5 to -5 are presumed to be utterances that make the customer uncomfortable (that is, a feeling of discomfort). Conversely, for example, operator utterances before and after the emotion score changes from -5 to 5 are presumed to be utterances that make the customer pleasant (ie, sympathetic state). In this way, by collecting operator utterances when score fluctuations of a certain amount or more occur within a certain time interval, it is possible to estimate which keyword the customer has responded to or sympathizes with. In particular, by counting this keyword, it is possible to select words that appear frequently when they sympathize or dislike. For example, if the customer often feels disliked with the keyword “price”, it can be assumed that the high amount of money has made the customer uncomfortable. On the other hand, if there is a lot of sympathy for the keyword "price", it is assumed that the customer felt that the amount was unexpectedly cheap.

  FIG. 1 shows the relationship between the emotion score and the sympathetic feeling location estimation score used in this invention and the sympathetic feeling location. The horizontal axis represents the elapsed time from the start of the dialogue, and the vertical axis represents the emotion score. The emotion score is estimated for each frame of the dialogue voice and varies with time. If there is a location where the empathy / relief location estimation score obtained based on the emotion score varies greatly, it can be estimated that the emotional state of the talker has changed. A place where the sympathetic reaction site estimation score greatly varies is estimated as a sympathy reaction site.

[Embodiment]
With reference to FIG. 2, an example of a functional configuration of the sympathetic reaction site detection apparatus 1 according to the embodiment will be described. The sympathetic reaction site detection apparatus 1 includes an input terminal 10, a threshold input unit 11, a voice signal acquisition unit 12, a dialogue voice analysis unit 13, a voice recognition unit 14, a morpheme analysis unit 15, an emotion model learning unit 16, and an emotion score calculation unit 17. , A sympathetic feeling part estimation score calculation part 18, a sympathy feeling part estimation part 19, a sympathy feeling feeling word totaling part 20, an emotion model storage part 91 and a sympathy feeling feeling word storage part 92. The sympathetic reaction location detection apparatus 1 is configured by, for example, reading a special program into a known or dedicated computer having a central processing unit (CPU), a main memory (Random Access Memory, RAM), and the like. Special equipment. For example, the sympathetic reaction site detection apparatus 1 executes each process under the control of the central processing unit. The data input to the sympathetic reaction spot detection device 1 and the data obtained by each processing are stored in, for example, the main storage device, and the data stored in the main storage device is read out as necessary to perform other processing. Used for Each storage unit included in the sympathetic reaction location detection device 1 includes, for example, a main storage device such as a RAM (Random Access Memory), an auxiliary storage device configured by a semiconductor memory element such as a hard disk, an optical disk, or a flash memory. Or middleware such as a relational database or key-value store. Each storage unit included in the sympathetic reaction site detection device 1 may be logically divided, and may be stored in one physical storage device.

  With reference to FIG. 3, an example of the processing flow of the sympathetic reaction site detection method executed by the sympathy response site detection apparatus 1 according to the embodiment will be described in the order of the procedures actually performed.

  In step S <b> 11, the threshold value input unit 11 receives an emotion score variation threshold value used by the sympathetic reaction site estimation unit 19. Details of the emotion score variation threshold will be described later. The input emotion score variation threshold is set in the sympathetic reaction site estimation unit 19. The emotion score variation threshold may be set in advance in the sympathetic reaction site estimation unit 19 and may not necessarily be input from the outside. In that case, the threshold value input unit 11 may be omitted.

  In step S12, the audio signal acquisition unit 12 converts an analog audio signal input from the input terminal 10 into an audio digital signal. The audio signal may be obtained by any existing means, for example, obtained by a microphone connected to the input terminal 10 or inputted by a recording device such as an IC recorder. It may be connected to the terminal 10 and input. The voice digital signal converted by the voice signal acquisition unit 12 is input to the dialogue voice analysis unit 13 and the voice recognition unit 14.

  In step S13, the dialog voice analysis unit 13 extracts an acoustic feature amount from the input voice digital signal. The extracted acoustic feature amount is input to the emotion model learning unit 16 and the emotion score calculation unit 17.

  As acoustic features to be extracted, for example, 1 to 12 dimensions of the Mel-Frequency Cepstrum Coefficient (MFCC), dynamic parameters such as ΔMFCC and ΔΔMFCC, and power and changes thereof Δ power and ΔΔ power, which are quantities, are used. Here, a cepstral mean normalization (CMN) process may be performed. The acoustic feature amount is not limited to MFCC and power, but parameters and speech section information used for speech recognition can be used. Further, the calculation window used at this time is about 30 milliseconds, and the calculation window shift may be executed in about 10 milliseconds.

  In step S14, the speech recognition unit 14 performs speech recognition processing on the input speech digital signal and generates a recognition result. The speech recognition method can be arbitrarily applied as long as it is an existing speech recognition technology. The generated recognition result is input to the morphological analysis unit 15.

  In step S15, the morphological analysis unit 15 performs morphological analysis on the input recognition result. The morphological analysis method can be arbitrarily applied as long as it is an existing morphological analysis technique. The generated morpheme analysis result is input to the sympathetic reaction word totaling unit 20.

  In step S <b> 16, the emotion model learning unit 16 learns an emotion model using the input acoustic feature amount. The emotion model can be expressed by, for example, a multi-dimensional mixed normal distribution (GaMMian Mixture Model, GMM) modeled based on probability statistical theory widely used in the field of speech recognition. For details of GMM, see, for example, “DA Reynolds, RC Rose,“ Robust Text-Independent speaker Identification using Gaussian mixture speaker models ”, IEEE Trans. Speech Audio Process., Vol. 3, no. 1, pp. 72-83. , Jan. 1995. ”. For example, when an integer emotion score is defined from “Positive (3)” to “Negative (−3)”, seven emotion models are generated. For details of the emotion model learning method, refer to Patent Document 1. A set of emotion models corresponding to each emotion score is stored in the emotion model storage unit 91.

  In step S <b> 17, the emotion score calculation unit 17 performs matching between the input acoustic feature quantity and the emotion model set stored in the emotion model storage unit 91, and calculates the most appropriate emotion score. Specifically, likelihood calculation is performed on the input acoustic feature quantity and each emotion model included in the emotion model set. For details of the emotion score calculation method, refer to Patent Document 1. The calculated emotion score is input to the sympathetic reaction site estimation score calculation unit 18.

  In step S18, the sympathetic reaction site estimation score calculation unit 18 calculates a sympathy response site estimation score from the input emotion score. The calculated sympathetic reaction site estimation score is input to the sympathy response site estimation unit 19. There are, for example, the following two methods for calculating the sympathetic reaction site estimation score.

  The first method is a method in which the difference between the emotion score of the frame and the emotion score of the immediately preceding frame is used as the sympathetic reaction site estimation score of the frame. Specifically, a value obtained by subtracting the emotion score of the Nth frame from the emotion score of the (N + 1) th frame is set as the sympathetic reaction site estimation score of the (N + 1) th frame.

  The second method is a method of calculating a sympathetic feeling location estimation score of the frame using the likelihood calculated from the emotion scores of a plurality of frames before the frame and the posterior distribution of the frame. Specifically, after predicting the emotion score of the (N + 1) th frame using machine learning, the value obtained by subtracting the actual emotion score of the (N + 1) th frame from the predicted emotion score is represented by the (N + 1) th frame. The score is the estimated score for the sympathy for the eyes. The machine learning described here includes, for example, Bayesian linear regression. In other words, pre-distribution parameters for Bayesian linear regression are estimated from some learning data in advance, the likelihood is calculated from the emotion score up to the Nth frame, and the posterior distribution of the (N + 1) th frame is obtained. The predicted value of the emotion score of the (N + 1) th frame is calculated using a probability maximization criterion. For more detailed calculation method, please refer to “C.M. Bishop,“ Pattern recognition and machine learning-Statistical prediction by Bayesian theory ”, Maruzen Publishing, pp. 151-159.

  In step S19, the sympathetic reaction part estimation part 19 estimates the sympathy reaction part which is a flame | frame in which the emotional state changed based on the sympathy reaction part estimation score. Whether the emotional state has changed is determined by comparing with the emotional score variation threshold set by the threshold input unit 11. That is, when the sympathetic reaction location estimation score exceeds the emotion score variation threshold, the frame is estimated as a sympathy response location. The estimated empathy feeling part is input to the empathy feeling word totaling unit 20.

  In step S <b> 20, the sympathetic response word totaling unit 20 extracts words uttered at the sympathetic response point from the recognition result after morphological analysis input from the morpheme analysis unit 15, and totals the number of appearances of each word. Hereinafter, the extracted word is referred to as a sympathetic reaction word. The count result of the sympathetic feeling words is stored in the sympathetic feeling word storage unit 92.

  Specifically, using the morpheme analysis result of the operator utterance sentence corresponding to the customer's sympathy negative part, the word (morpheme) issued by the operator when the customer sympathizes or disagrees is counted, and the sympathy negative word and The accumulated value is output. At this time, the empathy negative words are counted for each part of speech.

  In addition to simply outputting the total value, for example, in the case of a call center, “for each center (for example, fault reception, contract reception, etc.)” “for each time (for example, quarterly, specific period, etc.)” “for each operator” By adding various conditions such as “by call center installation area” and accumulating, more detailed analysis is possible. That is, it is possible to grasp the improvement points of the new product or new service by browsing the empathy negative feeling words for several days when the new product or new service appears.

  Since empathy words are tabulated by part of speech, it is possible to extract not only keywords based on nouns but also phrases. For example, if the customer frequently feels reluctant by making an expression, it can be used for business improvement such as instructing the operator to substitute the expression with a different expression.

  The empathy negative word totaling unit 20 may be configured to output not only the totaled words and wording itself but also utterances including the word and wording. Moreover, you may comprise so that the several utterances followed before and after the utterance containing the word or phrase may be output collectively.

  Assuming use at a call center or the like, a large amount of calls occur every day. Therefore, the above-described processing content is periodically executed once a day and is already stored in the empathy-relief word storage unit 92. It may be configured to add to the total value of the sympathetic negative feeling words.

  As described above, the empathy / reverse spot detection apparatus according to this embodiment can detect a place where the emotional state of the conversation person has changed, and can grasp the content of the utterance that caused the change of the emotional state. Therefore, it can be used for collecting information that directly leads to an increase in sales.

  The present invention is not limited to the above-described embodiment, and it goes without saying that modifications can be made as appropriate without departing from the spirit of the present invention. The various processes described in the above embodiment may be executed not only in time series according to the order of description, but also in parallel or individually as required by the processing capability of the apparatus that executes the processes or as necessary.

[Program, recording medium]
When various processing functions in each device described in the above embodiment are realized by a computer, the processing contents of the functions that each device should have are described by a program. Then, by executing this program on a computer, various processing functions in each of the above devices are realized on the computer.

  The program describing the processing contents can be recorded on a computer-readable recording medium. As the computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  The program is distributed by selling, transferring, or lending a portable recording medium such as a DVD or CD-ROM in which the program is recorded. Furthermore, the program may be distributed by storing the program in a storage device of the server computer and transferring the program from the server computer to another computer via a network.

  A computer that executes such a program first stores, for example, a program recorded on a portable recording medium or a program transferred from a server computer in its own storage device. When executing the process, the computer reads a program stored in its own recording medium and executes a process according to the read program. As another execution form of the program, the computer may directly read the program from a portable recording medium and execute processing according to the program, and the program is transferred from the server computer to the computer. Each time, the processing according to the received program may be executed sequentially. Also, the program is not transferred from the server computer to the computer, and the above-described processing is executed by a so-called ASP (Application Service Provider) type service that realizes the processing function only by the execution instruction and result acquisition. It is good. Note that the program in this embodiment includes information that is used for processing by an electronic computer and that conforms to the program (data that is not a direct command to the computer but has a property that defines the processing of the computer).

  In this embodiment, the present apparatus is configured by executing a predetermined program on a computer. However, at least a part of these processing contents may be realized by hardware.

DESCRIPTION OF SYMBOLS 1 Empathy feeling part detection apparatus 10 Input terminal 11 Threshold value input part 12 Voice signal acquisition part 13 Dialogue speech analysis part 14 Speech recognition part 15 Morphological analysis part 16 Emotion model learning part 17 Emotion score calculation part 18 Empathy feeling part estimation score calculation part 19 Empathy-sensitive part estimation unit 20 Empathy-sensitive word totaling unit 91 Emotion model storage unit 92 Empathy-sensitive word storage unit

Claims (8)

An emotion model storage unit for storing a set of emotion models each modeling a plurality of predefined emotion scores;
An interactive speech analysis unit that extracts acoustic features for each frame from the input speech signal;
An emotion score calculation unit for calculating an emotion score for each frame using the emotion model set from the acoustic feature amount;
A sympathetic reaction site estimation score calculation unit that calculates a sympathy response site estimation score for each frame based on the emotion score;
A sympathetic reaction part estimation unit that estimates a sympathy reaction part that is a frame in which the emotional state has changed based on the sympathy reaction part estimation score;
A sympathetic reaction site detection device including A sympathetic reaction spot detection device according to claim 1,
A voice recognition unit that generates a recognition result obtained by voice recognition of the voice signal;
A morphological analysis unit that generates an analysis result obtained by performing morphological analysis on the recognition result;
Extracting words spoken at the sympathetic reaction points from the analysis results, and summing up the number of times each word appears,
A sympathetic reaction site detection apparatus further comprising: A sympathetic reaction spot detection device according to claim 1 or 2,
A sympathetic feeling part detection device further comprising an emotion model learning unit that learns the emotion model using the acoustic feature amount. A sympathetic reaction spot detection device according to any one of claims 1 to 3,
The sympathetic reaction location estimation unit estimates a frame in which the sympathy response location estimation score exceeds a predetermined emotion score variation threshold as the sympathy response location detection apparatus. A sympathetic reaction site detection device according to any one of claims 1 to 4,
The sympathetic reaction site estimation score calculation unit is configured to use the difference between the emotion score of the frame and the emotion score of the immediately preceding frame as the sympathy response site estimation score of the frame. A sympathetic reaction site detection device according to any one of claims 1 to 4,
The sympathetic reaction location estimation score calculation unit calculates the sympathy response location estimation score of the frame using the likelihood calculated from the emotion scores of a plurality of frames before the frame and the posterior distribution of the frame. There is a sympathetic counterfeit spot detection device. A dialog voice analysis step in which a dialog voice analysis unit extracts an acoustic feature amount for each frame from the input voice signal;
An emotion score calculation step in which an emotion score calculation unit calculates an emotion score for each frame using an emotion model set obtained by modeling a plurality of predefined emotion scores from the acoustic feature amount;
A sympathetic reaction location estimation score calculation unit calculates a sympathy response location estimation score for each frame based on the emotion score,
A sympathetic reaction location estimation unit estimates a sympathy response location that is a frame in which the emotional state has changed based on the sympathy response location estimation score, and
A method for detecting empathy and reaction sites.   A program for causing a computer to function as the sympathetic reaction site detection device according to any one of claims 1 to 6.

Images