JP7164372B2 - Speech recognition system and speech recognition method - Google Patents

Description

The present invention relates to a speech recognition system and a speech recognition method.

In call centers and offices, it is common practice to record the contents of conversations between a customer and an operator to prepare for future troubles and to review the contents. Recorded data can be retrieved, displayed or printed by a computer system by recognizing voice and converting it into text data, and can be used more effectively as business data.

General use of speech recognition in call centers is mainly to improve response quality by recognition results obtained from operator's utterances and to improve customer service by recognition results obtained from customer's utterances.

On the other hand, the turnover rate of operators engaged in call center operations tends to be high, and suppressing the turnover rate of operators leads to lower operating costs. One of the reasons why operators leave their jobs is the burden that operators receive from calls with customers when answering telephone calls.

For example, Japanese Patent Laid-Open No. 2002-100002 discloses a technique for grasping the load state of an operator. In Patent Literature 1, the operator's load state is grasped based on the response time of the operator to the inquiry from the customer, the interview with the operator, and the like.

JP 2010-239353 A

In Japanese Unexamined Patent Application Publication No. 2002-100000, the load state of the operator is grasped by the manager of the operator based on interviews with the operator based on the response time of the operator. With this method, it is difficult to grasp the potential load state of the operator. The reason for this is that, in Patent Document 1, the operator's load state is grasped by focusing on the operator's utterance, not the customer's utterance.

SUMMARY OF THE INVENTION It is an object of the present invention to grasp the latent load state of an operator by paying attention to the content of a customer's utterance in a voice recognition system.

A voice recognition system according to one aspect of the present invention includes a call recording device for separately recording the customer's utterance content and the operator's utterance content among the call content between a customer and an operator, and a voice recognition device for performing voice recognition on the content of the customer's utterance and the content of the utterance of the operator; and a manager terminal for displaying the evaluation result of the load state obtained by the recognition result management device on a load state screen. do.

A speech recognition method according to one aspect of the present invention includes recording separately the customer's utterance content and the operator's utterance content among the call content between a customer and a plurality of operators, and recording the customer's utterance content and the operator's utterance content separately. Speech recognition is performed on the utterance content of the operator, and based on the speech recognition result of the speech-recognized utterance content of the customer, the operator evaluates the load state received from the customer, and the evaluation result is provided for each operator. and displaying the evaluation result of the load state on the load state screen for each operator.

According to one aspect of the present invention, in the speech recognition system, it is possible to grasp the potential load state of the operator by paying attention to the content of the customer's utterance.

1 is an overall configuration diagram of a call center system; FIG. FIG. 10 is a diagram showing the structure of a load evaluation value table (T-1); FIG. 10 is a diagram showing the configuration of a load word table (T-2); FIG. 10 is a diagram showing the configuration of a load state table (T-3); FIG. 10 is a diagram showing specifications of reference values (evaluation values) for “volume”, “speed of speech”, and “words” defined in a load evaluation value table (T−1). FIG. 10 is a diagram showing an example in which evaluation values of "word", "volume" and "speed of speech" are respectively set for an operator; It is a figure which shows the example of a setting of a load word. FIG. 10 is a diagram showing an example of calculation of a load state for two calls by operator A; It is a figure which shows an example of a load state screen. It is a figure which shows an example of a load state screen. It is a figure which shows an example of a load state screen. 1 is a configuration diagram of a speech recognition system; FIG. FIG. 4 is a diagram for explaining the operation of the speech recognition system; FIG. FIG. 10 is a flowchart for explaining a method (5-1) for selecting an evaluation value; FIG. It is an analysis flow of the load regarding the item "volume". It is an analysis flow of the load regarding the item "speech speed." It is an analysis flow of the load regarding the item "word".

An embodiment will be described below with reference to the drawings.

First, a call center system will be described with reference to FIG.
A speech recognition system in a call center generally associates CTI (Computer Telephony Integration) information such as a called party number with a speech recognition engine (dictionary) to perform speech recognition. CTI information is information that can identify a language. Here, CTI is a general term for technologies that utilize telephones and computers in cooperation. This is a technology that enables call centers, etc., to look up customer information in a database from the customer's telephone number, and to perform automatic dialing and automatic transfer.

As shown in FIG. 1, the call center system includes IP-PBX (Internet Protocol-Private Branch eXchange: IP line compatible private branch exchange) device 101, CTI device 102, voice call processing system 103, operator A terminal 104 and an administrator terminal 105 are connected to each other.

The IP-PBX device 101 receives a call from a call terminal 107 of a customer 106, performs protocol conversion between the IP network and the public network 108, and controls incoming and outgoing calls. The CTI device 102 acquires call information (called number, etc.) from the IP-PBX device 101 and transmits it to the voice call processing system 103 .

The operator terminal 104 is an operator PC terminal used by the operator 109 for operator work. The operator 109 makes a call with the call terminal 107 of the customer 106 via the public network 108 via the call terminal (IP telephone) 110 . The administrator terminal 105 displays the evaluation result of the load state obtained by the recognition result management device 115 on the load state screen.

An IP-PBX device 101 connected from a call terminal 107 of a customer 106 via a public network 108 connects to a call terminal (IP telephone) 110 via a network 100 to make a call. The operator 109 can operate the telephone from the call terminal (IP telephone) 110, and when there is an incoming call from the customer 106, the customer 106 and the operator 109 are in a talking state.

The voice call processing system 103 has a call recording information management device 111 , a call recording device 112 , a voice recognition control device 113 , a voice recognition device 114 and a recognition result management device 115 .

The call recording information management device 111 has a function of accumulating CTI information and recording information.
The call recording device 112 has a function of acquiring and recording the mirrored call voice and a function of transmitting it to the call recording information management device 111 .

The call recording device 112 is a device that records a data stream of a call exchanged by the call terminal 107 as recorded data via the IP-PBX device 101 . A call at the call terminal 107 is sent to the call recording device 112 and saved as a recording file.

Specifically, operator 109 uses call terminal (IP telephone) 110 to talk with customer 106 of public network 108 via IP-PBX device 101 . Voice (RTP packets) of operator 109 is transmitted from call terminal (IP telephone) 110 to public network 108 . Voice (RTP packets) of customer 106 is transmitted from public network 108 to call terminal (IP telephone) 110 . If the source of the RTP packet is the IP telephone 110, it is distinguished from the speech of the operator 109, and if the source of the RTP packet is the public network 108, it is distinguished from the speech of the customer 106. As a result, the call recording device 112 separately records the voice of the customer 106 and the voice of the operator 109, and creates a voice file of each voice only. Specifically, the voice of customer 106 is recorded in a customer voice file, and the voice of operator 109 is recorded in an operator voice file.

The call recording device 112 acquires the mirrored call voice, records it, and transmits it to the voice recognition device 114 . The call recording information management device 111 is a server for managing call information and recording information in association with each other. The speech recognition device 114 has a speech recognition engine that executes speech recognition, and transmits recognition results to the recognition result management device 115 . The voice recognition device 114 converts the recorded data into text data by means of a voice recognition engine.

The voice recognition control device 113 has a function of instructing the voice recognition device 114 to perform recognition. The recognition result management device 115 stores text data output by the speech recognition device 114 in a database, accumulates speech recognition results, and outputs the recognition results.

The recognition result management device 115 includes a load evaluation value table (T-1) (see FIG. 2) that defines evaluation values for quantifying the load state, and a load word table ( T-2) (see FIG. 3), and a load state table (T-3) (see FIG. 4) for storing evaluation results obtained by evaluating the load state of the operator from verbal information and non-verbal information.

An administrator sets evaluation values in the load evaluation value table (T-1) and the load word table (T-2). Specifically, the recognition result management device 115 provides and displays an evaluation value setting screen on the administrator terminal 105 for the administrator. The administrator sets the evaluation value via the evaluation value setting screen.

The configuration of the load evaluation value table (T-1) will be described with reference to FIG.
As shown in FIG. 2, in the "load evaluation value table" (T-1), reference values (evaluation values) for load calculation of "volume", "speed of speech", and "words" extracted from the customer's utterance set each.

For example, as shown in FIG. 2, for operator A, "volume": 40, "speech rate": "d", and "word": 10 are set. For operator B, "volume": 50, "speech rate": "50", and "word": "d" are set. For operator C, "volume": "d", "speech rate": "20", and "word": "d" are set. Also, as "default", "volume": 30, "speech rate": "40", and "word": "5" are set. Here, "d" means to use the "default" setting. For example, focusing on operator C, since "d" is set as "volume", "default" "volume": 30 is provided and "volume": 30 is set.

Here, with reference to FIG. 5, the specifications of the reference values (evaluation values) of "volume", "speech speed", and "word" defined in the "load evaluation value table" (T-1) will be described.
As shown in FIG. 5, the volume means the loudness of the customer's voice, and the unit is "dB". The speech rate means the number of syllables included in one second of the customer, and the unit is "word/sec". A word means the number of times a load word is included in the customer's utterance content, and the unit is "case". In terms of "load evaluation", the greater the sound volume, the greater the load. As the speech speed increases, the load increases. As for words, the larger the word, the greater the load.

An example in which the operator 109 is set with evaluation values for words, volume, and speed of speech will be described with reference to FIG.
As shown in FIG. 6, for operator A, "volume": 40, "speech rate": 2, and "word": 10 are set. For operator B, "volume": "d", "speech rate": "d", and "word": "5" are set. For operator C, "volume": "d", "speech rate": "20", and "word": "d" are set. For the operator "default", "volume": 30, "speech rate": "10", and "word": "2" are set.

For each parameter in FIG. 6, the value set in the operator "default" is used for the item for which "d" is set. For example, since "d" is set for the volume of operator B, the value is acquired from "default" and 30 is set as the evaluation value. Also, since "d" is set for the speech rate of operator B, the value is obtained from "default" and 10 is set as the evaluation value.

The configuration of the "load word table" (T-2) will be described with reference to FIG.
As shown in FIG. 3, in the "load word table" (T-2), load words that are burdensome to the operator are set. For example, "AAAA", "BBBB", and "ZZZZ" are set as load words.

A setting example of load words will be described with reference to FIG.
As shown in FIG. 7, setting examples of load words are, for example, "Please call my boss" (No. 1), "On behalf of another person" (No. 2), and the like.

The configuration of the load state table (T-3) will be described with reference to FIG.
As shown in FIG. 4, the load status table (T-3) defines call IDs, outside line calling numbers, call end times, and evaluation results for each of operators A, B, and C. FIG. Here, volume, speaking speed, and words are defined as evaluation results. The volume, speaking speed, and word evaluation results are represented by "load" and "normal."

For example, for the call ID "0x3001" of operator A, the evaluation result of "volume" is "load", the evaluation result of "speech speed" is "load", and the evaluation result of "word" is " load”. For the call ID "0x3007" of operator B, the evaluation result of "volume" is "normal", the evaluation result of "speech speed" is "normal", and the evaluation result of "word" is "normal". is. For the call ID "0x3010" of operator C, the evaluation result of "volume" is "load", the evaluation result of "speech speed" is "load", and the evaluation result of "word" is "normal". is.

Next, a method of obtaining the operator's load state and providing it to the administrator's terminal 105 will be described.

First, the administrator sets values in each item of the load evaluation value table (T-1) and the load word table (T-2). A speech recognition device 114 performs speech recognition when the call between the customer 107 and the operator 109 is completed.

The recognition result management device 115 evaluates the operator's load state from the recognition result of the customer 107 obtained from the speech recognition device 114, the load evaluation value table (T-1), and the load word table (T-2). , the evaluation result is stored in the load state table (T-3). Here, the voice recognition device 114 recognizes the voice of the customer 107 based on the customer voice file stored in the call recording device 112 .

Here, FIG. 8 shows an example of calculation of the load state of operator A for two calls.
As shown in FIG. 8, for the call ID "0x3001" of operator A, the "call time" is 300 seconds, the evaluation result of "volume" is "load", and the evaluation result of "speed" is is "load" and the evaluation result of "word" is "normal". For operator A's call ID "0x3002", the "call time" is 120 seconds, the evaluation result of "volume" is "load", and the evaluation result of "speech speed" is "load". and the evaluation result of "word" is "load".

The recognition result management device 115 obtains a total result along with the call start time for each operator and call customer, provides it to the manager terminal 105, and displays it on the load state reference screen (manager PC screen). The administrator judges the load state of the operator 109 from the content evaluated by the recognition result management device 115 via the load state reference screen.

As described above, in the embodiment, attention is focused on the content of the customer's utterance in a call, and the load on the operator 109 is calculated from the speech recognition result of the customer's utterance content. I will provide a.

In the conventional speech recognition system, attention was focused on the content of the operator's utterances to improve the response of the operator. On the other hand, in the embodiment, the point of view is changed to the content of the customer's utterance, and the judgment condition of the speech recognition result is not the improvement of the operator's response, but the evaluation of the factors that give the operator a load through speech recognition and analysis. do. In the voice recognition system of the embodiment, the operator's potential load state is grasped by paying attention to the content of the customer's utterance.

The elements of the recognition results that give the load are the words included in the "linguistic information" and the volume and speed of speech included in the "non-linguistic information". As the word evaluation, the number of occurrences of words that burden the operator in the contents of the customer's utterance is evaluated. As the volume evaluation, the volume when the customer speaks is evaluated. As speech speed evaluation, the speech speed at the time of customer's speech is evaluated. After setting the evaluation value for the evaluation item, the load state is determined from the magnitude relationship between the information obtained from the utterance content and the evaluation value. Then, the evaluated load state of the operator is provided to the administrator as a total result.

Next, a load condition reference screen (administrator PC screen) displayed on the administrator terminal 105 will be described. The administrator judges the load condition of the operator 109 from the administrator PC screen.
As shown in FIG. 9, the load state screen displays the load analysis result for the operator's call. The load status screen displays the load status for each call to the administrator from the contents of the "load status table" (T-3).

Further, as shown in FIG. 10, the load status screen displays the result of load analysis for the operator's call. The load status screen indicates to the administrator the week specified from the "load status table" (T-3), and the daily number of calls (loaded calls/total number of calls of the operator) for the specified week in percent notation. do.

Further, as shown in FIG. 11, the load status screen displays the result of load analysis for the operator's call. On the load status screen, the administrator can specify the month from the "load status table" (T-3), and for the specified month, the weekly number of calls (the number of calls that are the load / the total number of calls of the operator) will be displayed in percentage notation. indicate.

Next, with reference to the system configuration diagram of FIG. 12 and the flow chart of FIG. 13, the operation of the speech recognition system in the call center (the operation from receiving a call to calculating the load) will be described.

As an example, a case will be described in which the operator 109 calculates the load received from the customer 106's utterance (5) after executing speech recognition ((1) to (4)) after the end of the call between the operator 109 and the customer 106. .

(1) The call recording information management device 111 receives CTI information.
(2) The call recording device 112 transmits recording information to the call recording information management device 111 .
(3) Sending the CTI information and the recording information from the call recording information management device 111 to the voice recognition control device 113 .
(4) The speech recognition control device 113 transmits the CTI information and the recording information to the speech recognition device 112 and requests the speech recognition device 112 to perform speech recognition. The speech recognition device 112 performs speech recognition and transmits the recognition result to the speech recognition control device 113 . The speech recognition control device 113 transmits the recognition result to the recognition result management device 115 .
(5) The recognition result management device 115 uses the recognition result received from the speech recognition control device 113, the "load value table" (T-1) and the "load word table" (T-2) to provide the operator with information received from the customer. The load is analyzed and the results of the analysis are accumulated in the "load status table" (T-3) as evaluation results. As will be described later, the load analysis is performed in the order of evaluation value selection (5-1), load analysis (5-2), and analysis result storage (5-3). The recognition result management device 115 displays the recognition results on the manager's terminal 105, and the manager browses the recognition results.

Next, the evaluation value selection method (5-1) will be described with reference to FIG.
First, the evaluation value of the "load value table" (T-1) is obtained (S141).
Next, it is determined whether the operator name of the call information exists on the "load value table" (T-1) (S142).

If it exists, the values of "volume", "speech speed" and "word" of the record corresponding to the operator in the "load value table" (T-1) are obtained (S143).

If it does not exist, the value on the record of operator name: default in the "load value table" (T-1) is used as the evaluation value (S144).

Next, it is determined whether or not there is an operator record item of the "load value table" (T-1) with a value of "d" (S145).
If it exists, the value on the record of operator name: default in the "load value table" (T-1) is used as the evaluation value for items with a value of 0 (S146).
If it does not exist, the value of the "load value table" (T-1) is used as the evaluation value (S147).

Next, the load analysis method (5-2) will be described with reference to FIGS. 15 to 17. FIG. The operator 109 analyzes the load received from the contents of the speech of the customer 106 with respect to "volume", "speech speed", and "words".

A load analysis method for the item "volume" will be described with reference to FIG.
First, non-verbal information about the content of the speech of the customer 106 is acquired from the recognition result (S151).
Next, volume is extracted from the non-verbal information (S152).
Next, the speech volume of the customer 106 is compared with the volume evaluation value (S153).
As a result of the comparison, it is determined whether the speech volume of the customer 106 is greater than the volume evaluation value (S154).
As a result of the determination, if the speech volume of the customer 106 is greater than the volume evaluation value, the evaluation result for the item "volume" from the call is set to "load" (S155).
As a result of the determination, if the speech volume of the customer 106 is not greater than the volume evaluation value, the evaluation result for the item "volume" from the call is set to "normal" (S156).

Next, referring to FIG. 16, a load analysis method for the item "speech speed" will be described.
First, non-verbal information about the content of the speech of the customer 106 is acquired from the recognition result (S161).
Next, speech speed is extracted from the non-verbal information (S162).
Next, the speech speed of the customer 106 is compared with the speech speed evaluation value (S163).
As a result of the comparison, it is determined whether the speaking speed volume of the customer 106 is greater than the speaking speed evaluation value (S164).
As a result of the determination, if the speech rate volume of the customer 106 is greater than the volume evaluation value, the evaluation result for the item "speech rate" from the call is set to "load" (S165).
As a result of the determination, if the speech rate volume of the customer 106 is not greater than the speech rate evaluation value, the evaluation result for the item "speech rate" from the call is set to "normal" (S166).

Next, referring to FIG. 17, a load analysis method for the item "word" will be described.
First, linguistic information about the content of the customer 106's utterance is acquired from the recognition result (S171).
Next, word information is extracted from the non-verbal information (S172).
Next, it is determined whether or not the words registered in the "load word table" (T-2) exist in the word information (S173).

If it exists, the number of words registered in the "load word table" (T-2) that exists in the word information is counted. The value calculated by counting is set as the number of utterance load words (S174).
If it does not exist, the number of utterance load words is set to 0 (S175).

Next, the number of utterance load words of the customer 106 and the word number evaluation value are compared (S176).
Next, as a result of the comparison, it is determined whether the number of utterance load words of the customer 106 is larger than the word number evaluation value (S177).
As a result of the determination, if the number of uttered load words of the customer 106 is larger than the word count evaluation value, the evaluation result for the item "word" from the call is set to "load" (S178).
As a result of the determination, if the number of utterance load words of the customer 106 is not larger than the word count evaluation value, the evaluation result for the item "word" from the call is set to "normal" (S179).

Finally, a method (5-3) for saving analysis results will be described.
The load results of the analyzed "words", "volume" and "speech rate" are stored in the "load state table" (T-3). At the time of saving, in addition to the load result, the “operator name”, “call ID”, “outline number”, “call start time” and “call duration” associated with the call are acquired from the voice recognition control device 113 and recorded. Record as

According to the embodiment, the manager can grasp the load that the operator receives from the call with the customer and the state of the operator. Furthermore, the administrator can grasp the operator's load tendency at an early stage, so that an increase in the load due to telephone correspondence can be alleviated in advance.

In the embodiment, the operator's load is determined based on the customer voice file, but the present invention is not limited to this, and the operator's voice file may also be used to determine the operator's load.

100 network 101 IP-PBX device 102 CTI device 103 voice call processing system 104 operator terminal 105 manager terminal 106 customer 107 call terminal 108 public network 109 operator 110 call terminal (IP telephone)
111 call recording information management device 112 call recording device 113 speech recognition control device 114 speech recognition device 115 recognition result management device

Claims (8)

a call recording device for recording the customer's utterances ;
a speech recognition device that performs speech recognition on the content of the customer's speech recorded in the call recording device;
a recognition result management device for obtaining an evaluation result by evaluating the load state received by an operator from the customer based on the speech recognition result of the customer's utterance content recognized by the speech recognition device;
an administrator terminal that displays the evaluation result of the load state obtained by the recognition result management device on a load state screen;
has
The recognition result management device
a load evaluation value table defining evaluation values for evaluating the load state;
a load word table that defines load words that act as a load on the operator;
a load status table for storing the evaluation results of the operator evaluating the load status received from the customer;
The recognition result management device
By referring to the speech recognition result of the customer's utterance content obtained from the speech recognition device, the load evaluation value table and the load word table, the load state received by the operator from the customer is determined for each operator. and accumulate the evaluation result in the load state table for each operator,
The administrator terminal
displaying the evaluation results accumulated in the load state table on the load state screen for each operator;
The recognition result management device
In the load evaluation value table, as the evaluation values, the volume and speech rate of the content of the customer's utterance included in the non-verbal information of the content of the customer's utterance, and the load included in the linguistic information of the content of the customer's utterance. A speech recognition system characterized by defining the number of words . The recognition result management device
By comparing the recognition result of the customer's utterance content acquired from the speech recognition device with the evaluation value defined in the load evaluation value table , whether the customer's utterance content causes a load on the operator. 2. The speech recognition system according to claim 1, wherein the evaluation result is accumulated in the load state table as a load or a normal state. The recognition result management device
acquiring the non-verbal information of the customer's utterance content from the speech recognition result of the customer's utterance content;
extracting speech volume from the non-verbal information;
comparing the speech volume and the volume evaluation value to determine whether the speech volume is greater than the volume evaluation value;
If the speech volume is greater than the volume evaluation value, defining the volume evaluation result as the load,
3. The speech recognition system according to claim 2 , wherein when said speech volume is not greater than said volume evaluation value, said volume evaluation result is defined as said normal. The recognition result management device
acquiring the non-verbal information of the customer's utterance content from the speech recognition result of the customer's utterance content;
extracting speech speed from the non-verbal information;
comparing the speech speed and the evaluation value of the speech speed to determine whether the speech speed is greater than the evaluation value of the speech speed;
if the speech speed is greater than the speech speed evaluation value, defining the speech speed evaluation result as the load;
3. The speech recognition system according to claim 2 , wherein when said speech speed is not greater than said speech speed evaluation value, said speech speed evaluation result is defined as said normal. The recognition result management device
acquiring the language information of the customer's utterance content from the speech recognition result of the customer's utterance content;
extracting word information from the language information;
determining whether the load word defined in the load word table exists in the word information;
if the load word exists, calculating the number of uttered words of the load word in the word information;
comparing the number of spoken words with the evaluation value of the load word to determine whether the number of spoken words is greater than the evaluation value of the load word;
if the number of uttered words is greater than the evaluation value of the load word, defining the evaluation result of the load word as the load;
3. The speech recognition system according to claim 2 , wherein when the number of uttered words is not greater than the evaluation value of the load word, the evaluation result of the load word is defined as normal. The administrator terminal displays a setting screen,
The recognition result management device
2. The speech recognition system according to claim 1 , wherein the evaluation values are set in the load evaluation value table and the load words are set in the load word table via the setting screen . a call recording device for recording the customer's utterances ;
a speech recognition device that performs speech recognition on the content of the customer's speech recorded in the call recording device;
a recognition result management device for obtaining an evaluation result by evaluating the load state received by an operator from the customer based on the speech recognition result of the customer's utterance content recognized by the speech recognition device;
an administrator terminal that displays the evaluation result of the load state obtained by the recognition result management device on a load state screen;
has
The administrator terminal
A speech recognition system , wherein the aggregated results of the evaluation results are displayed on the load state screen for each of the operator and the customer in accordance with call start time . a call recording device for recording the customer's utterances ;
a speech recognition device that performs speech recognition on the content of the customer's speech recorded in the call recording device;
a recognition result management device for obtaining an evaluation result by evaluating the load state received by an operator from the customer based on the speech recognition result of the customer's utterance content recognized by the speech recognition device;
an administrator terminal that displays the evaluation result of the load state obtained by the recognition result management device on a load state screen;
has
The administrator terminal
A speech recognition system characterized in that the totalized result obtained by totaling the evaluation results is displayed in percentage notation obtained by dividing the number of calls that impose a load on the operator by the total number of calls of the operator on a daily or weekly basis .

Images