JP6712539B2 - Satisfaction determination device, method and program - Google Patents

Description

This invention relates to audio technology. In particular, the present invention relates to a technique for determining utterance satisfaction.

In the operation of a call center, there is a demand for a technique for determining whether a customer is satisfied from an utterance (hereinafter referred to as customer satisfaction determination). This technology can be applied to, for example, automate the operator evaluation by totaling the frequency of satisfaction of each customer for each operator, and investigate the customer's request by performing voice recognition and text analysis of satisfied utterances.

As a similar technique to the above, Non-Patent Documents 1 and 2 propose a technique for estimating customer satisfaction, dissatisfaction, or anger from a call. In Non-Patent Document 1, customer satisfaction/dissatisfaction is estimated by using speaking style features such as the customer's speech speed and linguistic features such as the presence or absence of product names of competitors. In Non-Patent Document 2, the prosody/non-anger of the customer is estimated by using prosodic features such as the pitch and loudness of the customer's voice and dialogue features such as the frequency of the mischief. In any of the technologies, machine learning technology is used to learn the relationship between each feature amount and customer satisfaction/dissatisfaction/anger from a large number of calls and use it for estimation.

Youngja Park, Stephen C. Gates, “Towards Real-Time Measurement of Customer Satisfaction Using Automatically Generated Call Transcripts”, in Proceedings of the 18th ACM conference on Information and knowledge management, p.1387-1396, 2009. Noo, Satoshi Kobashigawa, Satoshi Tamoto, Hirokazu Masataki, Osamu Yoshioka, Satoshi Takahashi, “Estimation of Anger Feeling from Dialogue Voice Using Temporal Relationship of Speech,” IEICE Transactions, Vol.J96- D, No. 1, p. 15-24, 2013.

Customer satisfaction is closely linked to local changes in voice. For example, a local change in voice pitch tends to be peculiar, such as a sudden rise in voice pitch as an expression of joy in an utterance that the customer is satisfied with. However, in the related art, the statistic of the voice pitch in a unit of utterance is used as the prosody feature amount, and a local change in the voice pitch is hardly reflected in the prosody feature amount. As a result, the accuracy of satisfaction determination may decrease.

An object of the present invention is to provide a satisfaction determination apparatus, method and program capable of determining the satisfaction of speech with higher accuracy than ever before.

A satisfaction determination device according to an aspect of the present invention includes a fundamental frequency extraction unit that extracts a fundamental frequency of each frame that forms each utterance included in a call, and a local variation amount that is a variation amount of the extracted fundamental frequency for the utterance. a local change amount calculation unit for calculating a per time, calculated call to a local variation statistics calculating local variation statistic is the average and variance of the local variation of the fundamental frequency of each time for each utterance contained Based on the calculated amount of local change statistic, a local change amount singular determination unit that determines whether the calculated local change amount for each time of each utterance is singular, and for each utterance , a singular When the number of times that the utterance is determined to be equal to or greater than a predetermined threshold is included, each utterance includes a satisfaction determination unit that determines that the utterance is satisfied by the utterer.

By determining the satisfaction of the utterance based on whether or not the local change in the voice pitch is specific, it is possible to determine the satisfaction of the utterance with higher accuracy than before.

The block diagram for explaining the example of a satisfaction judging device. 9 is a flowchart for explaining an example of a satisfaction determination method. The figure for demonstrating the example of calculation of a local change amount.

[Embodiment]
Hereinafter, embodiments of the present invention will be described.

As shown in FIG. 1, the satisfaction determination device includes a voice section detection unit 101, a fundamental frequency extraction unit 102, a local change amount calculation unit 103, a local change amount statistic calculation unit 104, a local change amount specific determination unit 105, and a satisfaction determination. The unit 106 is provided, for example.

The satisfaction determination method is realized, for example, by each unit of the satisfaction determination device by the processing of FIG. 2 and step S101 to step S106 described below.

<Encryption unit 11>
<Voice section detection unit 101>
A voice signal of a call is input to the voice section detection unit 101. An example of a call is, for example, a call between a customer who calls the call center and an operator of the call center.

The voice section detection unit 101 obtains at least one utterance of the customer by applying the voice section detection technique to the input voice signal of the call (step S101). Each utterance of the customer thus obtained is output to the fundamental frequency extraction unit 102.

As the voice section detection technology, an existing voice section detection technology such as a method based on power threshold processing or a method based on a likelihood ratio of a voice/non-voice model can be used. In the method based on the power threshold processing, the section in which the power of the audio signal is equal to or higher than a predetermined threshold is used as the utterance of the customer.

<Basic frequency extraction unit 102>
Each utterance of the customer is input to the basic frequency extraction unit 102.

The fundamental frequency extraction unit 102 extracts, for each utterance of the input customer, the sequence of the fundamental frequency system corresponding to the utterance. In other words, the basic frequency extraction unit 102 extracts the basic frequency of each frame forming the utterance included in the call (step S102). The extracted fundamental frequency is output to the local change amount calculation unit 103.

For example, a method based on autocorrelation (eg, Reference 1) can be used to extract the fundamental frequency. Of course, other fundamental frequency extraction methods may be used.

[Reference 1]
Shuichi Itabashi, "Voice Engineering", Morikita Publishing Co., Ltd., 2006, pp.104-105
Smoothing may be performed on the extracted sequence of the fundamental frequency system. As a smoothing method, for example, a method using a median filter can be used.

<Local change amount calculation unit 103>
The fundamental frequency extracted by the fundamental frequency extraction unit 102 is input to the local change amount calculation unit 103.

The local change amount calculation unit 103 obtains the local change amount of the fundamental frequency from the sequence of the fundamental frequency system. In other words, the local change amount calculation unit 103 calculates the local change amount that is the extracted change amount of the fundamental frequency (step S103). The calculated local change amount is output to the local change amount statistic calculation unit 104 and the local change amount peculiar determination unit 105.

The local change amount is, for example, a gradient obtained by linear regression of the fundamental frequency. That is, for example, as shown in FIG. 2, primary regression analysis is performed using fundamental frequencies included before and after (for example, 50 milliseconds before and after) from a certain time, and the slope of the obtained regression line is set as the local change amount. By applying this processing to the fundamental frequency of the entire utterance, the local variation amount of the fundamental frequency of the entire utterance can be obtained. The local change amount calculation unit 103 performs this process for each utterance that constitutes a call.

<Local change amount statistic calculation unit 104>
A local change amount of the fundamental frequency calculated by the local change amount calculation unit 103 is input to the local change amount statistic calculation unit 104.

The local change amount statistic calculation unit 104 obtains the average and the variance from the local change amount of the fundamental frequency of the entire call, and sets it as the local change amount statistic of the fundamental frequency of the entire call. In other words, the local change amount statistic calculation unit 104 calculates the local change amount statistic that is the average and variance of the local change amounts of the fundamental frequencies of the utterances included in the call (step S104). The calculated local change amount statistic is output to the local change amount peculiar determination unit 105.

The local variation statistic of the fundamental frequency of the entire call expresses "how the person who uttered the call in the call puts intonation". For example, when the variance is large, it is considered that the person speaks with inflection.

<Local change amount specific determination unit 105>
The local change amount peculiar determination unit 105 receives the local change amount of the fundamental frequency calculated by the local change amount calculation unit 103 and the local change amount statistic calculated by the local change amount statistic calculation unit 104.

The local change amount unique determination unit 105 determines from the local change amount of the fundamental frequency for each time point whether the local change amount at that time point is unique. In other words, the local change amount singular determination unit 105 has a unique local change amount calculated by the local change amount statistic calculation unit 104 based on the local change amount statistic calculated by the local change amount calculation unit 103. It is determined (step S105). The determination result is output to the satisfaction determination unit 106. The determination result when the local change amount is singular is referred to as a “singular determination”, and the determination result when the local change amount is singular is referred to as a “non-singular determination”.

If it is determined to be peculiar, it can be said that a specific voice pitch rise or fall has occurred in the person who is talking at that time, and these are related to whether or not the person is satisfied.

For example, a normal distribution and a predetermined confidence level obtained by a local variation statistic (for example, mean and variance) of the fundamental frequency of the entire call are used to determine whether or not the call is unique. The confidence interval range is determined from the local variation statistics of the fundamental frequency of the entire call and the confidence level. Regarding the local change amount of the fundamental frequency for each time, if the confidence interval is entered, it is not singular. The confidence level is 95%, for example.

<Satisfaction determination unit 106>
The satisfaction determination unit 106 receives the determination result obtained by the local change amount specific determination unit 105.

The satisfaction determination unit 106 returns a satisfaction determination result for each utterance based on the determination as to whether it is peculiar at each time (step S106). The satisfaction determination result is, for example, a binary value indicating whether or not the person who uttered the utterance is satisfied.

For example, in the utterance of a customer, when the singular determination for each time is included in the utterance at a predetermined threshold value or more, the utterance is regarded as an utterance satisfied by the person who uttered the utterance. Otherwise, the speech, regarded as a speech a person who has the speech is not satisfied. The predetermined threshold is, for example, 2. This corresponds to the case where at least one set of rapid rise and fall of voice pitch occurs. The predetermined threshold may be a positive integer other than 2.

[Second embodiment]
The satisfaction determination apparatus and method according to the second embodiment further performs the satisfaction determination by further utilizing the prosodic feature and the language feature for each utterance used in the conventional technique. By further utilizing the prosodic features and linguistic features of each utterance, the accuracy of the determination can be expected to be further improved.

Hereinafter, the description will be focused on the parts different from the first embodiment. Overlapping description of the same parts as those in the first embodiment will be omitted.

The satisfaction determination device of the second embodiment further includes a prosody feature extraction unit 201 and a language feature extraction unit 202, which are indicated by dotted lines in FIG.

<Prosodic feature extraction unit 201>
Each utterance of the customer obtained by the voice section detection unit 101 is input to the prosody feature extraction unit 201.

The prosody feature extraction unit 201 obtains a prosody feature from each utterance of the customer. The obtained prosody features are output to the satisfaction determination unit 106.

The prosodic feature is information including at least one or more of a voice pitch and a loudness, and a statistic (for example, average, variance, etc.) of speech speed.

<Language feature extraction unit 202>
Each utterance of the customer obtained by the voice section detection unit 101 is input to the language feature extraction unit 202.

The language feature extraction unit 202 obtains language features from each utterance of the customer. The obtained language feature is output to the satisfaction determination unit 106.

The language feature is information including at least one or more appearance numbers such as the number of fillers, the names of competing products, and the names of competitors.

<Satisfaction determination unit 106>
The prosody feature extraction unit 201 and the language feature determined by the language feature extraction unit 202 are input to the satisfaction determination unit 106 of the second embodiment.

The satisfaction determination unit 106 according to the second embodiment obtains a customer satisfaction determination result for each utterance by further using at least one of the prosody feature and the language feature for each utterance, in addition to the peculiar determination for each time. In other words, the satisfaction determination unit 106 of the second embodiment further determines at least one of the prosodic feature and the language feature amount to determine whether or not the utterance is an utterance satisfied by the person who uttered the utterance.
The customer satisfaction determination result is a binary value indicating whether or not the customer is satisfied with the utterance.

The customer satisfaction determination result is determined, for example, by a combination of the results of the threshold determinations of one or more feature amounts. For example, it is determined that the utterance is satisfied when the peculiar determination for each time is included in the utterance at a predetermined threshold value or more and the average value of the fundamental frequencies among the prosody features is the threshold value or more. This corresponds to deciding that the customer is satisfied when the voice is high on average and the phenomenon of a rapid increase in voice pitch occurs.

The predetermined threshold value is appropriately determined so as to obtain a desired result. The predetermined threshold may be determined by machine learning.

[Program and recording medium]
When each process in the satisfaction determination device is realized by a computer, the processing content of the function that the satisfaction determination device should have is described by a program. Then, by executing this program on a computer, the processing of the satisfaction determination device is realized on the computer.

The program describing the processing contents can be recorded in a computer-readable recording medium. The computer-readable recording medium may be, for example, a magnetic recording device, an optical disc, a magneto-optical recording medium, a semiconductor memory, or the like.

Further, each processing means may be configured by executing a predetermined program on a computer, or at least a part of the processing contents thereof may be realized by hardware.

[Modification]
Needless to say, other changes can be made without departing from the spirit of the present invention.

101 voice section detection unit 102 fundamental frequency extraction unit 103 local change amount calculation unit 104 local change amount statistic calculation unit 105 local change amount peculiar determination unit 106 satisfaction determination unit 201 prosody feature extraction unit 202 language feature extraction unit

Claims (4)

A fundamental frequency extraction unit that extracts the fundamental frequency of each frame that constitutes each utterance included in the call,
For the utterance, a local change amount calculation unit that calculates a local change amount that is a change amount of the extracted fundamental frequency for each time ,
A local change statistic calculation unit that calculates a local change statistic that is the average and variance of the local change amount of the fundamental frequency for each time of each utterance included in the calculated call,
Based on the calculated local change amount statistic, a local change amount peculiar determination unit that determines whether the calculated local change amount for each time of each utterance is peculiar,
For each of the utterances , when the number of times determined to be the peculiarity is greater than or equal to a predetermined threshold, each utterance is a satisfaction determination unit that determines that the utterer is a satisfied utterance. ,
Satisfaction determination device including. In the satisfaction determination device according to claim 1,
A prosody feature extraction unit that extracts prosody features from the utterance,
A language feature amount extraction unit that extracts a language feature amount from the utterance,
The satisfaction determination unit further uses at least one of the extracted prosodic feature and language feature amount to determine whether or not the utterance is an utterance satisfied by the utterer.
Satisfaction determination device. A fundamental frequency extraction unit, a fundamental frequency extraction step of extracting the fundamental frequency of each frame constituting each utterance included in the call,
A local change amount calculation section, for the utterance, a local change amount calculation step of calculating a local change amount that is a change amount of the extracted fundamental frequency for each time ,
The local change statistic calculation unit calculates a local change statistic that is an average and a variance of the local change amount of the fundamental frequency for each time of each utterance included in the call, which is calculated by the local change statistic calculation unit. Steps,
The local change amount peculiar determination unit, based on the local change amount statistics for each time of each of the calculated utterances, a local change amount peculiar determination step of determining whether the calculated local change amount is peculiar,
When the satisfaction determination unit includes, for each utterance, the number of times the singularity is determined to be equal to or greater than a predetermined threshold value, the utterance is determined to be a utterance satisfied by the utterer. A satisfaction determination step,
Satisfaction determination method including. A program for causing a computer to function as each unit of the satisfaction determination device according to claim 1.

Images